-
Notifications
You must be signed in to change notification settings - Fork 1.7k
enhancement(fluent source): Add support for forwarding over unix socket #22212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ultimately this change is largely mechanical and is just gluing together the existing decoder with the existing unix stream machinery, but there is definitely the possibility I have missed something subtle.
listen_path: PathBuf, | ||
socket_file_mode: Option<u32>, | ||
decoder: Decoder, | ||
decoder: D, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is necessary to allow using the FluentDecoder. As the fluent protocol doesn't have an explicit framing mechanism that I could ascertain, there wasn't a good way to make this work with the codecs::Decoder setup which separates framing and decoding.
This, and better discoverability, is also the reason I opted not to implement this as a codec on the socket source, as suggested on the ticket
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this PR @tustvold! Left a couple of review comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @tustvold, custom deserialization is always tricky.
Is there some way I can help move this one along, I can't really effectively debug the CI (which appears to be failing for unrelated reasons) as it requires approval to run... |
Yes, please add a changelog fragment. I re-enabled the checks. |
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
Head branch was pushed to by a user without write access
I've merged main and regenerated the docs, let me know if anything further is required |
Some windows related failures: https://github.com/vectordotdev/vector/actions/runs/14501794689/job/40683006134. We need one or more feature gates. |
Hopefully fixed it, although I don't have a windows machine on which to test this... |
Summary
Change Type
Is this a breaking change?
How did you test this PR?
I couldn't work out how to get the integration tests working, but I tested by running fluent-bit with the following config
And then running vector with
And seeing the metrics being transferred.
I think it should be a relatively straightforward lift for someone with familiarity with the existing integration testing harness to hook this up, but I got all sorts of errors.
Does this PR include user facing changes?
Checklist
make check-all
is a good command to run locally. This check isdefined here. Some of these
checks might not be relevant to your PR. For Rust changes, at the very least you should run:
cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
cargo nextest run --workspace
(alternatively, you can runcargo test --all
)Cargo.lock
), pleaserun
dd-rust-license-tool write
to regenerate the license inventory and commit the changes (if any). More details here.References