-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
inet_dns support for UPDATE, NOTIFY and TSIG #6985
Conversation
8789ce0
to
729b7b5
Compare
Okay, the name compression tripped me up[1] so the API now requires the user to pass in the original binary packet. TSIG over TCP now works. So I have tested this against Knot DNS, but still lacking testing with OTP as the 'server'...I'll let you know once I have tested it, may be some time though. Would be great to have feedback on the API/interface as if I need to rework it to make it less offensive, it would really help to know sooner rather than later. An example of a TSIG signed DNS update is as shown:
An example of a TSIG signed DNS AXFR query is as shown:
[1] we need to do an HMAC on the original DNS packet and every DNS name compression algorithm is different so by decoding and re-encoding the packet we end up with a different output |
0eaaf75
to
4acbec4
Compare
Would be helpful to receive a summary of either "this is a pile of garbage, please haul it away" or "we like it in principle, we are currently too busy to chew through the sharp edges". Just depressing for a contributor to watch other PRs being worked through whilst receiving radio silence and being left in limbo. Really it is absolutely zero problem if this is considered a hot steaming pile of garbage, more than happy to just move it into my own private library. For my purposes it does not need to be in OTP, I only submitted as it was mentioned there was some interest in getting something like this into OTP. |
I will try to find the time to look at this now (tomorrow (...)). |
I think the biggest bit here is that the decision needs to be made to either:
Most of my PR is just a reimplementation of the DNS decoder; I did it this way so you could see what is needed rather than just jumping in an butchering the existing decoder. Of course happy to do this work, but need the old thumbs up or thumbs down on if this would ever be accepted. |
I am happy you have made the effort to understand the RFC:s, and to write code that works. From here we can look at details and structure. I have read RFC 8945 (could have been more thorough) and now try to understand how your code implements it... |
Let me know if I can help in any way, even if it is "hey, we think we want the DNS decoder to work like this, can you refactor things for us and we'll review" Thanks! |
I'll write down things in the order I stumble on them... I belong to the camp that limits lines to 80 characters, but we have people from the other camp in my team... If you have a strong opinion about not limiting code to 80 characters I will accept that, but I would really like the line limit to be 80 characters... I do not see the point of converting to Gregorian time for the user interface format. Why not simply have Posix time there. Furthermore I think using I have a hard time understanding the code in There was something in the RFC about at least 10 bytes for a truncated MAC, but I cannot find code for that. Should I think the abstraction border between
Is this a feasible API, or is there something in the RFC:s that I have missed? Is this any better? I think the duplicated functions in |
Sure, I don't mind either way, but if you asked me to
Was only personal taste, mostly as my brain does not come with the epoch converter upgrade. I am not overly bothered either way.
Right, so part of the problem (I like to see it as a feature) is that I have overloaded
To preserve the sanity of others, should I remove the overloading of
Agreed.
It is somewhat subtly implemented, the lower limit is referred to by
It is guess work at the moment on what I think is appropriate, but completely untested. I have be poking my DNS product codebase on and off around my other work...I only have had a chance to work on the client side as I am still working through implementing DNS-SD registration/lookups hence why the client side TSIG work is tested so far. Personally I would have the server side be non-configurable on I do not see any difference in effect punting this check post-verification compared to being forced to verify MACs from hostile actors but where they are using 'correct' values of
I quite like this. My reason for doing things the way they are is that I assumed there may be some push back about exporting anything more from
Decoding is a complete pain as you need to remove the TSIG RR (decrementing Annoyingly we know it is the last RR in the entire packet (anything else is invalid), but I cannot see a way to 100% reliably scan for the start of the RR from the end of the packet as name compression is allowed here (it is only forbidden on the algorithm name) and it is pretty much impossible to determine a length header in there. Only way I see it is we have to decode the whole thing. One idea I have is we could update Personally I prefer the idea of being able to call something like |
Now that you mention it ;-) ... no, but we prefer to use space only indent for new modules. However, if this is already written with tabs I will not insist on changing that. But I would be happy if you can rewrite to keep the line length under 80 (76) characters. And use Posix seconds in the user data, also use
I think better comments and clearer code will be good enough. I still think that Would using I missed
We can somewhat treat it as an internal module and modify it as we like, but we know we have external users, so we want to only do backwards compatible changes, if possible. Adding specialized functions for TSIG would be fine.
Then it would be preferable to not duplicate code getting a parallel decoder implementation.
I think it would be enough to add a byte offset field to the TSIG RR
|
Only a suggestion, it is useful at times when committing code to purposely merge incorrectly whitespaced segments. The theory is that at some later point that someone with OCD will come along later, fix it up...and then
Did not see any need to discuss it, took this as a signal on what the OTP team want to see in a v1 rework.
I suspect it will never occur, mostly due to the contract around TSIG, but treating things 'special' feels like something that will burn someone later on; also to date 100% of OTP only Erlang processed DNS packets are TSIG less :)
I think I would prefer to put it in the It is easier to reason about (no RR is special, I hate them all equally!) plus it will help others working on decoding problems where they can now tie back each RRs to where they appeared in the originating binary; my thinking here is when you are inspecting your PCAPs in Wireshark. I think I have everything I need from a first review to rework the TSIG component. Did you want to cherry pick out the EDNS(0) guard commit from this PR? Also what do you want me to do (if anything) with the NOTIFY/UPDATE commit...cherry pick or does it need some massaging in places? |
Well, since
I am afraid we much prefer to not change the I understand your reasoning about decoding, and debugging the wire format, but unfortunately ... the above.
If I should take the first two commits, then you would have to wait until I have merged to master for you to rebase on that, to avoid merge conflicts, furthermore, it seems UPDATE/NOTIFY are the major use cases you have for TSIG. So I do not mind keeping these commits in this PR. But, there seem to be two diff hunks in the 3:rd commit (inet_dns: support for TSIG requests), moving A, AAAA and WSIG records from class IN RRs to any class RRs, that maybe should be in the 2:nd commit (inet_dns: support for UPDATE and NOTIFY DNS packets). If so those diff hunks should be moved. |
I am 100% okay with waiting till OTP 28 for this; not the first time I have waited on a PR to be merged. For now I am happy to slum the solution at my end for my use case with double parsing and passing a binary blob into the DNS encoder as I used to for CAA RRs. If though the preference is to get this done sooner rather than later, then I'll start the work to drag that byte offset kicking'n'screaming through the decoder. I assume this is what you want unless I hear otherwise.
I suspect if the TSIG stuff was ready today for merging, it would not make it into OTP 26.x. Rebasing and tracking master is really little to no effort, and the TSIG work is not tangibly impacted by it. Only asking as whatever is easier for you? If you prefer to have the PR retain all three features, that works for me...the cost at my side is no different carrying one or three features out of tree.
Good catch, looks like I crossed the streams, I'll get that fixed. |
It would be nice to get this finished while I have not gotten too deep into to much other stuff and forgotten about the details. Other than that there is no hurry.
To just get the byte offset out there would not be much difference to have it as a new field in the Adding an API function that also removes the TSIG record and returns it separated from the DNS message would be an optimization to avoid completing a DNS message you do not want and then having to parse the DNS whole message again to do exactly that.
Maybe in theory, but, ... no.
I think, then, I would prefer to keep all 3 in the same branch, with the 2 first as separate commits, as it is now. |
...I'll get to work then. Might take a week or two for me to find the time, so "watch this space". Thanks for the review and the pointers! |
I removed the 'testing' label, but the branch is still part of our daily builds. I pull it in and add it internally... |
I managed to clear the backlog of my other work today, hence the rebasing that slipped out last night, and hope to have the squashing sorted by today. sorry it took a while to get back to this. |
No worries :-) |
Okay, here is the first cut, I'm now running through the unit tests at each commit to make sure it all is okay. Original unsquashed branch at https://github.com/jimdigriz/otp/tree/dns.unsquashed and there is no diff between the two. |
knotd supports UPDATE which is functionality incoming in a following commit we also amend the testing here to remove the five second warmup delay
Most calls to inet_res:resolve are missing 'nameservers' which means the request goes to your local resolver which may route single labels to LLMNR (eg. systemd-resolved). Also as Ericsson did not buy the TLD 'otptest' these are not publically usable. Also, inet_res:getbyname() does not accept options and we are not going to hijack the inet_res main resolver, so remove it.
Punted the nsd->knotd commit up the chain so we get better testing...typing Ran the following at each commit and they pass:
Let me know if you need anything else, but I think at the moment there is nothing else I need to do? |
For the sake of the procedures I will let it run a few nights in its squashed variant before merging. I also think it is ready to merge. |
Thank you for your pull request and hard work! |
Thanks for the guidance on beating this PR into shape. See you when I need to implement |
I will read up on that when it falls on my table... ;-) |
Some proposed enhancements to
inet_dns
:EDNS(0)
option present as discussedNOTIFY
andUPDATE
UPDATE
is a little more complicated as zero length data is now allowed and a newNONE
classUPDATE{A,D,DA,M,MA}
to kick up a conversation if these should still existIXFR
and a few other rcodesTSIG
suggested some time back that there would be interest to have this baked intoinet_dns.erl
Not yet added support for- this has now been added though mostly untestedTSIG
over TCP as described by RFC8945, section 5.3.1; this is on my list of things to resolve thoughI suspect none of this is particularly controversial but the
TSIG
API does need to be thrashed out.I am submitting what I see as version zero (0) and finally worth sharing around. I really would prefer the focus be on the API as the implementation will need burning and redoing a few more times from scratch as we carve out the interface.
In the spirit of
inet_dns
, this is an undocumented API in that it expects the user to handle policy whilst it only sets out to handles the encoding/decoding/verification.I tried a few interface/API iterations and found having settled on passing in
#dns_rec{}
ended up being the least offensive and made the most sense from a possible user perspective but this is my view only. Similarly I tried putting this all ininet_dns.erl
but after a while the sign/verify functionality just felt more correct in its own module, again this is my view, but it did mean I had to export the name encode/decode routines frominet_dns
so maybe this should be folded back intoinet_dns
.Broken out of the commit so to hack out the conversation here, this is what it smells like (for a simple client expecting a single UDP response packet to its request):
Usage for a client (shown above and mostly tested):
tsig:init/1
algs
really is only useful as a single item as we use the first algorithm (default:sha256
) providedkeys
really is only useful as a single item as we use the first name/secret entrytsig:sign/2
the request and send ittsig:verify/2
inet_dns_tsig.erl
Usage for a server (not tested as I am still in the middle of building my DNS server...will get back to you...):
tsig:init/1
algs
lists all the algorithms we are willing to support (default: onlysha256
, maybe the default for 'server' should be changed to all the algorithms)keys
lists all the name/secret combinations that are acceptedtsig:verify/2
on the request packettsig:sign/2
, sending it to the client; repeat this until the response is completely sent