Discussion:
sender source IP address on UDP socket bound to INADDR_ANY in golang
Patrick M. Hausen
2021-05-08 17:05:56 UTC
Permalink
Hi all,

I am facing a problem that is perfectly explained by the semantics
of the socket interface for UDP, if one assumes that the application
in question binds to INADDR_ANY and does not specifically set the
sender address when sending datagrams.

In the case of a DNS server and an interface with multiple addresses
that means outgoing answers will always be sent from the primary
address if the server does not take specific measures to answer
queries received on an alias address also *from* that alias address.

I guess that is the primary reason why BIND binds to all addresses
it finds at startup individually - to get this function "for free" by the
underlying OS.

Now recently I stumbled over AdGuard Home - a filtering recursive
nameserver written in golang - sending replies from the wrong
address when alias addresses are involved. Naturally I opened
the folks responsible a ticket:

https://github.com/AdguardTeam/AdGuardHome/issues/3015

Their answer: "we *do* keep track of the address a query was sent to,
that problem was solved long ago."

Yet, clearly, my installation on Free/HardenedBSD 12.1 (OPNsense)
behaves differently. My question to you on this list: since they do
their main development work on Linux, is there a remote possibility
that our API is sufficiently different for their code to run, but not to
work as intended?

Their code in question is here:
https://github.com/AdguardTeam/dnsproxy/blob/1163404e605c3dfbeab360fc3540fc290f61a321/proxyutil/udp_unix.go#L47

I am familiar with the socket API in C (and could always fetch a copy
of "Stevens" from my shelf), but don't know enough about golang
to make any progress from here.

Anyone who can help?

Thanks!
Patrick
--
punkt.de GmbH
Patrick M. Hausen
.infrastructure

Kaiserallee 13a
76133 Karlsruhe

Tel. +49 721 9109500

https://infrastructure.punkt.de
***@punkt.de

AG Mannheim 108285
GeschÀftsfÌhrer: JÌrgen Egeling, Daniel Lienert, Fabian Stein
Peter Jeremy via freebsd-net
2021-05-11 10:38:21 UTC
Permalink
This post might be inappropriate. Click to display it.
Patrick M. Hausen
2021-05-11 11:40:44 UTC
Permalink
Hi!
Post by Peter Jeremy via freebsd-net
Post by Patrick M. Hausen
I am facing a problem that is perfectly explained by the semantics
of the socket interface for UDP, if one assumes that the application
in question binds to INADDR_ANY and does not specifically set the
sender address when sending datagrams.
...
Post by Patrick M. Hausen
https://github.com/AdguardTeam/dnsproxy/blob/1163404e605c3dfbeab360fc3540fc290f61a321/proxyutil/udp_unix.go#L47
So, they say that they retrieve "the net interface IP the packet was
sent to (dst addr) from the socket's OOB data" and I agree that's what
the referenced code does. I hadn't heard of that behaviour before and
went digging...
Thank you. I received some code with internal debugging added from the
AdGuard core team and will try that today or tomorrow. If I read the quote
from the documentation correctly, on possible explanation would be them
calling recvmsg() but forgetting to setsockopt()?

Kind regards,
Patrick
--
punkt.de GmbH
Patrick M. Hausen
.infrastructure

Kaiserallee 13a
76133 Karlsruhe

Tel. +49 721 9109500

https://infrastructure.punkt.de
***@punkt.de

AG Mannheim 108285
GeschÀftsfÌhrer: JÌrgen Egeling, Daniel Lienert, Fabian Stein
Peter Jeremy via freebsd-net
2021-05-12 12:06:09 UTC
Permalink
This post might be inappropriate. Click to display it.
Patrick M. Hausen
2021-05-14 21:40:04 UTC
Permalink
Hi Peter and everyone else following,
Post by Peter Jeremy via freebsd-net
1) The Go code isn't enabling IPPROTO_IP.IP_RECVDSTADDR on the socket.
2) There's a FreeBSD kernel bug that mean setting IP_RECVDSTADDR
isn't being correctly reflected into the recvmsg control message.
3) The control message isn't being correctly plumbed through from
recvmsg(2) to the Go RecvMsg() return.
Note that a lot of the relevant Go library code is BSD- or FreeBSD-
specific so it's also possible that there is a bug in the Go library
code.
do you have some spare time and would you be so kind to look at our discussion
here: https://github.com/AdguardTeam/AdGuardHome/issues/3015

Andrey from the AdGuard team references this golang issue:
https://github.com/golang/go/issues/8329

Which references this FreeBSD issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246

What I as a sysadmin can observe is that the test code Andrey gave me
binds to *.53 on IPv4 and IPv6 although I start it with `-l 0.0.0.0` which is
clearly an IPv4 "any" address.

I am not 100% familiar with the API but as I understand you can treat
IPv4 as IPv6 via the socket interface by using an IPv4-mapped IPv6
address. So far so good.
But then of course you have an AF_INET6 socket and it seems that
FreeBSD does not allow setting IPv4 specific options via setsockopt()
because it's an IPv6 socket. Correct?

Why can you have a single socket on both address families, anyway?
IPv4 and IPv6 are as "related" as IP and IPX - if you go dual stack,
treat them both separately - no?

Any light you can shed on this issue greatly appreciated.

Thanks,
Patrick
--
punkt.de GmbH
Patrick M. Hausen
.infrastructure

Kaiserallee 13a
76133 Karlsruhe

Tel. +49 721 9109500

https://infrastructure.punkt.de
***@punkt.de

AG Mannheim 108285
GeschÀftsfÌhrer: JÌrgen Egeling, Daniel Lienert, Fabian Stein
Peter Jeremy via freebsd-net
2021-05-16 11:18:55 UTC
Permalink
Hi Patrick,
Post by Patrick M. Hausen
do you have some spare time and would you be so kind to look at our discussion
here: https://github.com/AdguardTeam/AdGuardHome/issues/3015
https://github.com/golang/go/issues/8329
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246
I've skimmed through all three issues.
Post by Patrick M. Hausen
What I as a sysadmin can observe is that the test code Andrey gave me
binds to *.53 on IPv4 and IPv6 although I start it with `-l 0.0.0.0` which is
clearly an IPv4 "any" address.
I am not 100% familiar with the API but as I understand you can treat
IPv4 as IPv6 via the socket interface by using an IPv4-mapped IPv6
address. So far so good.
Yes.
Post by Patrick M. Hausen
But then of course you have an AF_INET6 socket and it seems that
FreeBSD does not allow setting IPv4 specific options via setsockopt()
because it's an IPv6 socket. Correct?
That's my reading of the FreeBSD issue.
Post by Patrick M. Hausen
Why can you have a single socket on both address families, anyway?
IPv4 and IPv6 are as "related" as IP and IPX - if you go dual stack,
treat them both separately - no?
This is getting outside my expertise but my understanding is that
the idea behind using IPv4-mapped addressed is to simplify building
dual-stack applications, particularly during the early introduction
of IPv6. The main benefit is that it made it possible to support
both IPv4 and IPv6 without needing 2 sockets - which means you
can stick to doing an accept() on a blocking socket, rather than
needing to use poll() or select() etc with a pair of non-blocking
sockets.

I'm not sure how to solve your problem, sorry.
--
Peter Jeremy
Lutz Donnerhacke
2021-05-16 11:59:31 UTC
Permalink
Post by Peter Jeremy via freebsd-net
This is getting outside my expertise but my understanding is that
the idea behind using IPv4-mapped addressed is to simplify building
dual-stack applications, particularly during the early introduction
of IPv6. The main benefit is that it made it possible to support
both IPv4 and IPv6 without needing 2 sockets - which means you
can stick to doing an accept() on a blocking socket, rather than
needing to use poll() or select() etc with a pair of non-blocking
sockets.
Correct. IPv4-mapped addresses exists only for this purpose, they do not
have any meaning outside of this API. Unfortunatly the API is incomplete,
but still heavily used. For this purpose the API might be stretched over its
limits.

Loading...