Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge dev into main #15

Merged
merged 101 commits into from
Jul 5, 2024
Merged
Changes from 1 commit
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
f21584a
Setup local dev env with dynamic module
TheTeaCat Jul 4, 2023
603c3c8
Doc local dev setup & compilation + configuration steps
TheTeaCat Jul 4, 2023
bcbd20d
Merge pull request #1 from FireTail-io/local-dev
TheTeaCat Jul 4, 2023
82c117f
Add .clang-format
TheTeaCat Jul 4, 2023
04ed442
Add .pre-commit-config.yaml
TheTeaCat Jul 4, 2023
7bad906
Initial clang-format
TheTeaCat Jul 4, 2023
73033f8
Merge branch 'dev' into clang-format
TheTeaCat Jul 4, 2023
150ac0a
Doc formatter
TheTeaCat Jul 4, 2023
9718af4
Add clang-format style check CI action
TheTeaCat Jul 4, 2023
bf8a84d
Use clang-format 16
TheTeaCat Jul 4, 2023
b51c5dc
Remove DecimalMinDigits
TheTeaCat Jul 4, 2023
bd25c53
Remove BinaryMinDigits
TheTeaCat Jul 4, 2023
b154df4
Remove HexMinDigits
TheTeaCat Jul 4, 2023
d5c1d9b
Merge pull request #2 from FireTail-io/clang-format
TheTeaCat Jul 4, 2023
e176209
Doc local deps setup
TheTeaCat Jul 5, 2023
023d64a
Add .gitignore
TheTeaCat Jul 5, 2023
2b0121d
Reorder & doc existing POC
TheTeaCat Jul 6, 2023
1579456
Just update content-length in header filter for now
TheTeaCat Jul 6, 2023
8575612
Update body filter to traverse chain to find its end & append message…
TheTeaCat Jul 6, 2023
72dad32
Nicer index.html for dev
TheTeaCat Jul 6, 2023
89039b2
Always include nginx headers first and disable SortIncludes
TheTeaCat Jul 6, 2023
4b0db37
Use dev nginx in dev image
TheTeaCat Jul 6, 2023
43cac3a
tweak custom index
TheTeaCat Jul 6, 2023
f4221e9
Log cumulative size of all buffers in the chain
TheTeaCat Jul 6, 2023
787efc4
Move the ngx_firetail_module out into a .h file
TheTeaCat Jul 6, 2023
176f248
Implement collecting up the complete request and response bodies
TheTeaCat Jul 6, 2023
a87aa34
Tweak logs
TheTeaCat Jul 6, 2023
c640e82
Add /health and /health-proxy to dev nginx.conf
TheTeaCat Jul 6, 2023
31b667e
Namespace overhaul
TheTeaCat Jul 7, 2023
9a680e7
missing k prefixes
TheTeaCat Jul 7, 2023
4bcc563
Logging request & response headers
TheTeaCat Jul 7, 2023
6479abc
Tidy header loops
TheTeaCat Jul 10, 2023
70191c5
Put headers into ctx
TheTeaCat Jul 10, 2023
5e060fb
Remove unecessary loops
TheTeaCat Jul 10, 2023
4b64e47
Move logging
TheTeaCat Jul 10, 2023
604f45d
JSON request body WIP
TheTeaCat Jul 10, 2023
0b4d205
Restructure
TheTeaCat Jul 10, 2023
ca4d01c
Tidy
TheTeaCat Jul 10, 2023
26b9b3a
Refactor for(;;)s
TheTeaCat Jul 10, 2023
c4352ec
Setup initial POST to /logs/bulk
TheTeaCat Jul 10, 2023
d1c4ea4
Merge pull request #3 from FireTail-io/doc-local-deps
rileyfiretail Jul 10, 2023
945530c
Add various TODOs
TheTeaCat Jul 13, 2023
69a909b
Add resource to request log
TheTeaCat Jul 13, 2023
32e9a6d
Add http:// prefix and TODO
TheTeaCat Jul 13, 2023
3a76b28
Add noop func for libcurl write callback
TheTeaCat Jul 14, 2023
9f5ec30
Initial firetail_api_token directive implementation
TheTeaCat Jul 14, 2023
fac2604
Setup POST to Firetail
TheTeaCat Jul 14, 2023
588d09e
Doc API token directive
TheTeaCat Jul 14, 2023
283651d
Update configuration doc
TheTeaCat Jul 14, 2023
c32cbd2
Fix missing null terminator on uri
TheTeaCat Jul 14, 2023
ed1c75c
Add more locations to dev nginx conf
TheTeaCat Jul 14, 2023
e542149
emoji breaks things
TheTeaCat Jul 14, 2023
2cce241
Add comments to nginx.conf
TheTeaCat Jul 14, 2023
b932348
Merge pull request #4 from FireTail-io/log-request-response
rileyfiretail Jul 14, 2023
85b318c
Merge pull request #5 from FireTail-io/post-log-to-firetail
rileyfiretail Jul 14, 2023
0904626
use multicurl for async requests
muhammadn Aug 10, 2023
5152a28
fix lint
muhammadn Aug 10, 2023
52a86d6
attempt to do async with response
muhammadn Aug 13, 2023
1ae1e3f
enhancements to multicurl
muhammadn Aug 13, 2023
40e4c99
cleanups
muhammadn Aug 13, 2023
bdcf4e2
run clang-format
muhammadn Aug 14, 2023
e2bdcd9
fix by adding line termination "\0" to prevent memory corruption bug
muhammadn Aug 20, 2023
327b4f5
fix lint errors
muhammadn Aug 20, 2023
653c7bf
Merge pull request #9 from FireTail-io/feat/convert-curl-to-multicurl
TheTeaCat Aug 21, 2023
1aa0439
remove callback and remove loop to ensure async
muhammadn Aug 24, 2023
94cd835
remove unnecessary code
muhammadn Aug 24, 2023
419a675
fix lint issues
muhammadn Aug 24, 2023
92f14b1
add missing cleanup code for curl
muhammadn Aug 24, 2023
d3d25cf
linting (again!)
muhammadn Aug 24, 2023
811b904
Merge pull request #10 from FireTail-io/feat/convert-curl-to-multicurl
TheTeaCat Sep 13, 2023
099414d
Move nginx module src into nginx_module subdir
TheTeaCat Aug 9, 2023
652f049
Add validator stubs to validator new subdir in src
TheTeaCat Aug 9, 2023
f431212
Create go mod
TheTeaCat Aug 9, 2023
12d273e
Update dir for clang check on C formatting in CI
TheTeaCat Aug 9, 2023
ee268a0
Update docs for adding module
TheTeaCat Aug 9, 2023
ea3a503
Add todo docs for Go module
TheTeaCat Aug 9, 2023
8e4be9b
Add go module build steps to dockerfile
TheTeaCat Aug 9, 2023
d62d783
Add firetail lib to go module deps
TheTeaCat Aug 10, 2023
8836b98
Implement method for initialising instance of Firetail middleware
TheTeaCat Aug 10, 2023
3acb41f
Create appspec for dev nginx
TheTeaCat Aug 10, 2023
8996421
Add logs around call to ResponseBodyValidator
TheTeaCat Aug 10, 2023
0c9c07e
Remove go version -m
TheTeaCat Aug 10, 2023
e0db590
Load the go module every call for now (POC)
TheTeaCat Aug 11, 2023
949472d
Create dev appspec
TheTeaCat Aug 21, 2023
5b8b80b
Add big bird profile endpoint
TheTeaCat Aug 21, 2023
3c41a4b
Copy appspec into dockerfile
TheTeaCat Aug 21, 2023
2f17685
Middlware doesn't need API token
TheTeaCat Aug 21, 2023
d663d2c
Implement call to go lib
TheTeaCat Aug 21, 2023
d88d1fe
Create middleware and use it in C
TheTeaCat Aug 21, 2023
416f842
Create big bird proxy
TheTeaCat Aug 23, 2023
c086e38
Fix: Load openapi spec in C only once (#11)
muhammadn Jan 31, 2024
dffb869
use the official library now
muhammadn Feb 13, 2024
e2aea43
Merge pull request #13 from FireTail-io/chore/use-official-go-lib
TheTeaCat Feb 13, 2024
7dd8992
Rewrite nginx code to use go-lib as the client (#14)
muhammadn Feb 21, 2024
0297739
flip the argument between token and response body which is incorrect
muhammadn Feb 23, 2024
f268641
remove nginx2.conf
muhammadn Feb 23, 2024
a0ad09d
more cleanups
muhammadn Mar 6, 2024
3b2e13a
fix formattingg
muhammadn Mar 7, 2024
f55530d
remove dead code
muhammadn Mar 7, 2024
b3b3875
switch firetail go-lib to use v0.2.1
muhammadn Apr 9, 2024
42ea705
Merge pull request #7 from FireTail-io/go-validation
rileyfiretail Jul 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
enhancements to multicurl
  • Loading branch information
muhammadn committed Aug 13, 2023
commit 1ae1e3f49c62be8dbdd4b5ee426e14a2b2122827
292 changes: 0 additions & 292 deletions filter_response_body.c

This file was deleted.

26 changes: 12 additions & 14 deletions src/filter_response_body.c
Original file line number Diff line number Diff line change
@@ -197,14 +197,14 @@ ngx_int_t FiretailResponseBodyFilter(ngx_http_request_t *request,
// Curl the Firetail logging API
// TODO: replace this with multi curl for non-blocking requests
CURLM *multiHandler = curl_multi_init();
int still_running;
//CURLMsg *msg = NULL;
//CURLcode return_code = 0;
//int msgs_left = 0;
//int http_status_code;
//const char *szUrl;
CURL *curlHandler = curl_easy_init();

int still_running;
CURLMsg *msg = NULL;
CURLcode return_code = 0;
int msgs_left = 0;
int http_status_code;

if (curlHandler == NULL) {
return kNextResponseBodyFilter(request, chain_head);
}
@@ -275,7 +275,7 @@ ngx_int_t FiretailResponseBodyFilter(ngx_http_request_t *request,
/* if there are still transfers, loop! */
} while(still_running);

/* while ((msg = curl_multi_info_read(multiHandler, &msgs_left))) {
while ((msg = curl_multi_info_read(multiHandler, &msgs_left))) {
if (msg->msg == CURLMSG_DONE) {
curlHandler = msg->easy_handle;
return_code = msg->data.result;
@@ -290,15 +290,13 @@ ngx_int_t FiretailResponseBodyFilter(ngx_http_request_t *request,
}

http_status_code = 0;
szUrl = NULL;

curl_easy_getinfo(curlHandler, CURLINFO_RESPONSE_CODE, &http_status_code);
curl_easy_getinfo(curlHandler, CURLINFO_PRIVATE, &szUrl);

if (http_status_code == 200) {
ngx_log_debug(NGX_LOG_DEBUG, request->connection->log, 0,
"200 OK for %s\n",
szUrl);
"200 OK for Firetail Backend\n",
NULL);
} else {
ngx_log_error(NGX_LOG_DEBUG, request->connection->log, 0,
"Request to Firetail logging API failed with status code: %d\n",
@@ -313,7 +311,7 @@ ngx_int_t FiretailResponseBodyFilter(ngx_http_request_t *request,
msg->msg);
}

} */
}

// If it err'd, log; otherwise we got a response (which might still be a
// non-2xx status code - so check it)
@@ -328,11 +326,11 @@ ngx_int_t FiretailResponseBodyFilter(ngx_http_request_t *request,
NGX_LOG_DEBUG, request->connection->log, 0,
"Status code from POST request to Firetail /logs/bulk endpoint: %d",
response_code);
} */
}

// Remember to clean up your mess
curl_easy_cleanup(curlHandler);
curl_multi_cleanup(multiHandler);
curl_multi_cleanup(multiHandler); */

// Pass the chain onto the next response body filter
return kNextResponseBodyFilter(request, chain_head);