Discussion:
[Bug 256393] Issue with recreation of ppp/tun interfaces
b***@freebsd.org
2021-06-03 02:49:52 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Mark Linimon <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|***@FreeBSD.org |***@FreeBSD.org
Keywords| |regression
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 04:42:20 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Kubilay Kocak <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Flags| |mfc-stable13?,
| |mfc-stable12-,
| |mfc-stable11-
Keywords| |needs-qa
Status|New |Open

--- Comment #1 from Kubilay Kocak <***@FreeBSD.org> ---
@Nathan

- What stable/13 revision? (uname -a) Is there a known working previous/last
revision?
- Any relevent error logs (console or messages) with respect to 'IPCP fails'
etc?
- Any chance of a bisection of sys/dev/tun ?
- Any additional info with sysctl debug.if_tun_debug=1 ?
- Any additional logging with ppp.conf set log all
- Can you include the relevent ppp.conf (sanitized) as an attachment
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 17:45:07 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #2 from Nathan Whitehorn <***@FreeBSD.org> ---
(In reply to Kubilay Kocak from comment #1)

Here are a whole bunch of logs:
FreeBSD xxxxxxxxxxxxxxx 13.0-STABLE FreeBSD 13.0-STABLE #1
stable/13-n245818-ae23d302479: Wed Jun 2 10:27:55 EDT 2021
***@terel.tachypleus.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64

From the ppp log (key portion is "Error: ipcp_InterfaceUp: unable to set ip
address"):
Jun 3 17:10:07 conference syslogd: last message repeated 1 times
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: deflink: RecvConfigAck(2)
state = Ack-Sent
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: IPADDR[6] 192.168.42.1
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: COMPPROTO[6] 16 VJ slots
with slot compression
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: deflink: State change
Ack-Sent --> Opened
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: deflink: LayerUp.
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: myaddr 192.168.42.1 hisaddr =
192.168.42.2
Jun 3 17:10:07 conference ppp[8508]: tun0: Warning: iface add:
ioctl(SIOCAIFADDR, 192.168.42.1 -> 192.168.42.2): File exists
Jun 3 17:10:07 conference ppp[8508]: tun0: Error: ipcp_InterfaceUp: unable to
set ip address
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: deflink: LayerDown:
192.168.42.1
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: deflink: SendTerminateReq(3)
state = Opened
Jun 3 17:10:07 conference ppp[8508]: tun0: IPCP: deflink: State change Opened
--> Closing
Jun 3 17:10:07 conference ppp[8508]: tun0: LCP: deflink: SendIdent(4) state =
Opened
Jun 3 17:10:07 conference ppp[8508]: tun0: LCP: MAGICNUM c6d8cf0b
Jun 3 17:10:07 conference ppp[8508]: tun0: LCP: TEXT user-ppp 3.4.2
Jun 3 17:10:08 conference ppp[8508]: tun0: Warning: ipv4_Input: IPCP not open
- packet dropped
Jun 3 17:10:10 conference syslogd: last message repeated 3 times
Jun 3 17:10:10 conference ppp[8508]: tun0: IPCP: deflink: SendTerminateReq(3)
state = Closing
Jun 3 17:10:11 conference ppp[8508]: tun0: Warning: ipv4_Input: IPCP not open
- packet dropped
Jun 3 17:10:13 conference syslogd: last message repeated 5 times
Jun 3 17:10:14 conference ppp[8508]: tun0: IPCP: deflink: SendTerminateReq(3)
state = Closing
Jun 3 17:10:14 conference ppp[8508]: tun0: Warning: ipv4_Input: IPCP not open
- packet dropped


Jun 3 17:43:15 conference syslogd: last message repeated 1 times
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: deflink: RecvConfigAck(2)
state = Ack-Sent
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: IPADDR[6] 192.168.42.1
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: COMPPROTO[6] 16 VJ slots
with slot compression
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: deflink: State change
Ack-Sent --> Opened
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: deflink: LayerUp.
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: myaddr 192.168.42.1 hisaddr =
192.168.42.2
Jun 3 17:43:15 conference ppp[8811]: tun0: Warning: iface add:
ioctl(SIOCAIFADDR, 192.168.42.1 -> 192.168.42.2): File exists
Jun 3 17:43:15 conference ppp[8811]: tun0: Error: ipcp_InterfaceUp: unable to
set ip address
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: deflink: LayerDown:
192.168.42.1
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: deflink: SendTerminateReq(3)
state = Opened
Jun 3 17:43:15 conference ppp[8811]: tun0: IPCP: deflink: State change Opened
--> Closing
Jun 3 17:43:15 conference ppp[8811]: tun0: LCP: deflink: SendIdent(4) state =
Opened
Jun 3 17:43:15 conference ppp[8811]: tun0: LCP: MAGICNUM d57f3e26
Jun 3 17:43:15 conference ppp[8811]: tun0: LCP: TEXT user-ppp 3.4.2
Jun 3 17:43:15 conference ppp[8811]: tun0: Warning: ipv4_Input: IPCP not open
- packet dropped
Jun 3 17:43:18 conference syslogd: last message repeated 3 times

Nothing in dmesg with tun_debug set except this:
tun0: tunpoll
tun0: tunpoll waiting

I can try to bisect, but it might be a little while as this is an important
production machine.
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:01:19 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #3 from Nathan Whitehorn <***@FreeBSD.org> ---
I at least found the proximate problem:

***@conference:/home/nwhitehorn # netstat -rn -f inet
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 128.135.X.X UGS vtnet0
127.0.0.1 link#2 UH lo0
128.135.X.X/25 link#1 U vtnet0
128.135.X.X link#1 UHS lo0
169.254.169.254 128.135.X.X UGHS vtnet0
192.168.42.1 127.0.0.1 UH lo0 <- this is the IP that
the tunnel will use once established, and it didn't go away when the tunnel
interface did

If I delete the route to the tunnel endpoint on the loopback interface that
appeared:

***@conference:/home/nwhitehorn # route delete 192.168.42.1
delete host 192.168.42.1

Then the connection re-establishes itself without issue.

The route in question seems to have been added by routed(8), replacing the
link-level default route, though I am not sure why it is not going away when
routed(8) exits and the interface disappears or why routed(8) is re-writing it:


***@conference:/home/nwhitehorn # netstat -rn -f inet
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 128.135.X.X UGS vtnet0
127.0.0.1 link#2 UH lo0
128.135.X.X/25 link#1 U vtnet0
128.135.X.X link#1 UHS lo0
169.254.X.X 128.135.X.X UGHS vtnet0
192.168.42.0/24 192.168.42.2 UGS tun0
192.168.42.1 link#3 UHS lo0 <- Original loopback
route
192.168.42.2 link#3 UH tun0
***@conference:/home/nwhitehorn # service routed start
Starting routed.
***@conference:/home/nwhitehorn # netstat -rn -f inet
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 128.135.X.X UGS vtnet0
10.7.114.0/24 192.168.42.2 UG tun0
127.0.0.1 link#2 UH lo0
128.135.X.X/25 link#1 U vtnet0
128.135.X.X link#1 UHS lo0
157.132.6.8/29 192.168.42.2 UG tun0
169.254.X.X 128.135.158.134 UGHS vtnet0
192.168.1.0/24 192.168.42.2 UG tun0
192.168.2.0/24 192.168.42.2 UG tun0
192.168.3.0/24 192.168.42.2 UG tun0
192.168.5.0/24 192.168.42.2 UG tun0
192.168.42.0/24 192.168.42.2 UGS tun0
192.168.42.1 127.0.0.1 UH lo0 <- Rewritten by routed(8)
192.168.42.2 link#3 UH tun0
192.168.176.0/24 192.168.42.2 UG tun0

On the 12.2 system on the other end of this link, the same loopback host route
is present, but it doesn't seem to prevent re-IP'ing the link:

$ netstat -rn -f inet
Routing tables

Internet:
Destination Gateway Flags Netif Expire
192.168.42.1 link#5 UHS tun0
192.168.42.2 127.0.0.1 UH lo0

My suspicion is that the actual problem is that some of the routing-table
verification code that went in recently is preventing adding an IP address with
an address to which there is already a route on another interface.
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:27:46 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Eugene Grosbein <***@freebsd.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org,
| |***@freebsd.org,
| |***@FreeBSD.org

--- Comment #4 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Nathan Whitehorn from comment #3)
My suspicion is that the actual problem is that some of the routing-table verification code that went in recently is preventing adding an IP address with an address to which there is already a route on another interface.
If such "verification" was really added, it is wrong and must be corrected.
Only PINNED routes should remain intact, other routes should be replaceable by
daemons like ppp or net/mpd5 or alike even if they were already added by some
routing daemons.

Adding some more eyes to the CC of this PR.
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:30:46 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Eugene Grosbein <***@freebsd.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.freebsd.org/bu
| |gzilla/show_bug.cgi?id=2231
| |29
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:36:26 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #5 from Nathan Whitehorn <***@FreeBSD.org> ---
One workaround to this is to just delete the logic in routed that adds this
loopback route. The code is original from SGI in 1996 and I think never did
anything useful on FreeBSD -- it certainly doesn't seem to now -- but didn't
cause harm until whatever kernel change is preventing setting an IP on this
interface.

That said, while I'm happy to commit the attached patch to routed, the
underlying behavior in the kernel, whatever it is, that is making SIOCAIFADDR
fail seems bogus to me.
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:37:10 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #6 from Nathan Whitehorn <***@FreeBSD.org> ---
Created attachment 225533
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=225533&action=edit
Patch to routed to avoid creating pointless loopback routes
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:40:22 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #7 from Nathan Whitehorn <***@FreeBSD.org> ---
I should also mention that I am no longer confident that this issue wasn't in
13.0-RELEASE. routed(8) is what triggers it here and routed(8) does not work at
all in 13.0-RELEASE because of changes to the routing code in the kernel, so my
test on the release was not valid.
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-03 18:51:30 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Alexander V. Chernikov <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Assignee|***@FreeBSD.org |***@FreeBSD.org
--
You are receiving this mail because:
You are the assignee for the bug.
b***@freebsd.org
2021-06-04 00:02:03 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Rodney W. Grimes <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 01:29:23 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #13 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Alexander V. Chernikov from comment #11)
That said, I'm all up with removing the loopback manipulation piece from routed(8).
Does it mean that system behaviour will keep changed comparing to stable/12 and
earlier branches? I mean, if one assigns /30 network with such mask to a P2P
interface (tun, ng etc.) then packets to local IP will not be delivered over
loopback but travel over P2P interface corresponding to /30 route?
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 08:34:16 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #14 from Alexander V. Chernikov <***@FreeBSD.org> ---
(In reply to Eugene Grosbein from comment #13)
I mean, if one assigns /30 network with such mask to a P2P interface (tun, ng etc.) then packets to local IP will not be delivered over loopback but travel over P2P interface corresponding to /30 route?
Kernel behaviour is the same. It will work as expected unless you run
routed(8).

Potentially we can counter-action routed(8) actions by adding PINNED to the
loopback/destination routes, however the desired state is that routed(8) does
not touch delete loopback routes installed by the kernel.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 10:32:46 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

Rodney W. Grimes <***@FreeBSD.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@FreeBSD.org

--- Comment #15 from Rodney W. Grimes <***@FreeBSD.org> ---
(In reply to Alexander V. Chernikov from comment #14)
Let me again assert, it is WRONG and causing troubles for 25 years of router
code to have FreeBSD's kernel attempt to maintain these loopback routes. That
is the domain of a routing daemon, and here again we have yet another we have
to fix the routing daemon cause we did something stupid in the FreeBSD kernel.

PLEASE rip out the stupid attempt for the kernel to implement a routing policy
and restore this to the domain of routing code which has a far better chance of
knowing what goes on.

PLEASE STOP ripping out code from routing daemons that is actually the exact
correct way to maintain loop back routes.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 15:00:08 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #16 from Nathan Whitehorn <***@FreeBSD.org> ---
I'm not sure what the routing daemon is doing currently is sensible, but it at
least does seem like nothing in the routing table should prevent setting the IP
of an interface.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 17:40:31 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #17 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Alexander V. Chernikov from comment #14)

I'm talking not about kernel behaviour only but abouth the whole complex of
generally used scenarios. Considering also the comment of rgrimes@, let's think
about following cases:

1) Some routing daemon installs to FIB some /32 route learned dynamically. It
may have its reasons and it should not fail unless there is already such PINNED
route in the FIB. Later some PPP daemon tries to assign that address to its
interface as address of local or remote side and it should not fail with EEXIST
but override non-PINNED route. It should fail with EEXIST if PINNED route
exists already.

2) Same in case of a routing daemon doing same things but route(8) instead of
another daemon trying to create a route or ifconfig(8) trying to assign same
address, they both should fail only due to existing PINNED route. They should
not fail otherwise and silently override possibly pre-existing non-PINNED route
including one installed by still running routing daemon.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 20:33:56 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #18 from Alexander V. Chernikov <***@FreeBSD.org> ---
(In reply to Nathan Whitehorn from comment #12)

Interesting.

Are you absolutely sure it's stock 13-S?
Any custom sysctl's / multiple fibs ?

The logs above show commit
https://cgit.freebsd.org/src/commit/sys/net/route/route_ifaddrs.c?h=stable/13&id=ae23d302479
, but does not explicitly specify the hostname (in that line).

Currently loopback routes in 13-S are PINNED:
https://cgit.freebsd.org/src/tree/sys/net/route/route_ifaddrs.c?h=stable/13&id=ae23d302479#n163
, so it shouldn't be possible for routed(8) to delete such routes.

PINNED logic also should take precedence over any other non-PINNED routes,
allowing to replace these on route addition during interface address
assignment.

Could you by any chance show the following:

1) netstat -4rnW, netstat -4onW
2) route -n monitor
3) if possible, run `dtrace -i 'fbt:kernel:*:return /arg1==17/ {stack()}'`

4) Run commmand-to-fail: `ifconfig tun0 192.168.42.1 192.168.42.2`

5) netstat -4rnW, netstat -4onW
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 20:52:16 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #19 from Alexander V. Chernikov <***@FreeBSD.org> ---
(In reply to Rodney W. Grimes from comment #15)
Do I understand correctly that you're suggesting that loopback routes should be
installed by the routing daemons instead of kernel?
If yes, I'm not sure how one would handle non-router cases (e.g. a server with
a single interface). I'm also not sure how can this work with modern routing
software. IIRC frr does not care about any route which is not RTF_GATEWAY. It
is certainly possible to configure such routes in bird, but it has to be done
on per-prefix basis.
Could you share a bit more details on what is the proposed alternative?
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 21:08:55 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #20 from Alexander V. Chernikov <***@FreeBSD.org> ---
(In reply to Eugene Grosbein from comment #17)
1) Some routing daemon installs to FIB some /32 route learned dynamically. It may have its reasons and it should not fail unless there is already such PINNED route in the FIB. Later some PPP daemon tries to assign that address to its interface as address of local or remote side and it should not fail with EEXIST but override non-PINNED route. It should fail with EEXIST if PINNED route exists already.
"and it should not fail unless there is already such PINNED route in the FIB" -
currently we have 2-level priority system (PINNED and non-PINNED,
https://cgit.freebsd.org/src/tree/sys/net/route/route_ctl.c?h=stable/13&id=ae23d302479#n467
). Routes within a single priority are treated equally. If something tries to
insert a route, which already exists, the following options are possible:
1) different priorities for the current/inserted route -> fail if lower,
replace if higher
2) same priorities, one of nexthop is not multipath-capable (interface or
temporary route) -> EEXIST
3) same priorities, both nexthops are multipath-capable -> extend/form
multipath group

Other than the clarification above, the described behaviour is the current
behavior.
2) Same in case of a routing daemon doing same things but route(8) instead of another daemon trying to create a route or ifconfig(8) trying to assign same address, they both should fail only due to existing PINNED route. They should not fail otherwise and silently override possibly pre-existing non-PINNED route including one installed by still running routing daemon.
IIRC route(8) does not use RTF_PINNED during addition, so route installation
may fail even w/o PINNED route. I'm not sure if that's something that we should
change.
Also, currently, override triggers a couple of rtsock notifications to allow
routing daemons to track the changes, so it's not exactly "silent".

The rest describes the current system behaviour.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-04 21:31:54 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #21 from Rodney W. Grimes <***@FreeBSD.org> ---
(In reply to Alexander V. Chernikov from comment #19
Do I understand correctly that you're suggesting that loopback routes should be installed by the routing daemons instead of kernel?
Suggest? No, my words are stronger than that. The KERNEL should NOT implement
ANY routing policies. A loopback route IS a routing policy.

Further loopback routes are a micro-optimazation that was originally done to
short circuit the MTU of 1500 on ethernet, and much short in the days of IMP's
and slip lines to use the larger MTU of the loopback interfaces. A BSD system
can run perfectly fine with NONE of these loopback routes, they are nothing
more than an optimization.
If yes, I'm not sure how one would handle non-router cases (e.g. a server with a single interface).
Well this use to be handled by a simple static route, but someone couldnt
handle the fact that the route goes away if you down the interface and thought
that the kernel should maintain this route for them. This is arguable a lack
of skill or understanding that if you take an interface down ALL routes are
gona go away, and you need to re install them.
I'm also not sure how can this work with modern routing software. IIRC frr does not care about any route which is not RTF_GATEWAY. It is certainly possible to configure such routes in bird, but it has to be done on per-prefix basis.
I'll discuss this with the FRR folks, but I do believe that software already
knows how to maintain loopback routes. Usually on a "router" you do NOT want
these routes in place, as this hides interface errors for locally sent packets
to a local address.
Could you share a bit more details on what is the proposed alternative? Well I think part of why we are here right now is that routed is trying to maintain these routes and it is conflicting/having issue with what the kernel is doing. I also know of older routing code that maintained these without issue. And finally these routes are a micro optimazation that are simply not needed in most cases.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-05 08:29:02 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #22 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Alexander V. Chernikov from comment #20)
IIRC route(8) does not use RTF_PINNED during addition, so route installation may fail even w/o PINNED route. I'm not sure if that's something that we should change. Also, currently, override triggers a couple of rtsock notifications to allow routing daemons to track the changes, so it's not exactly "silent".
The rest describes the current system behaviour.
Thank you very much for clarificaion.

If there is daemon-installed route (learned dynamically), there should be a way
to overrided it with manually installed route. I'm sure "route delete/route
add" is racy and won't work because of daemon re-installing its route.

Will "route change" do the job reliably in such case?
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-05 08:35:25 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #23 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Rodney W. Grimes from comment #21)
Post by b***@freebsd.org
Suggest? No, my words are stronger than that. The KERNEL should NOT implement
ANY routing policies. A loopback route IS a routing policy.

I'm not sure I understand what "routing policies" mean in general.
Nevertheless, I'd like we have some knob (sysctl?) to optionally preserve
current behaviour when kernel assigns loopback routes, so packet filters still
deal with local traffic as going through loopback interface, BPF "sees" it on
lo0 too etc.

Too many things will break if we just remove loopback routes unconditionally.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-05 17:00:18 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #24 from Rodney W. Grimes <***@FreeBSD.org> ---
(In reply to Eugene Grosbein from comment #23)
Post by b***@freebsd.org
Too many things will break if we just remove loopback routes unconditionally.
I agree, that would be necessary. Though as is you should already be filtering
this on both interfaces as it is possible to down lo0 and have your filter stop
working, and the traffic contine to flow, as I have stated you do NOT need
these routes at all, they are a micro-optimization.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-05 22:38:18 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #25 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Rodney W. Grimes from comment #24)
I agree, that would be necessary. Though as is you should already be filtering this on both interfaces
Almost never had a reason to filter traffic flowing through loopback with
exceptionally rare case.
they are a micro-optimization
It heavily depends on local condition and there may be lots of local traffic.

Also, I do not understand how a router can deal without delivering traffic
locally for its local IPs in many cases like running squid proxy or web/smtp
server or even routinely running ping to check if distinct uplink is still
alive.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-05 23:11:06 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #26 from Rodney W. Grimes <***@FreeBSD.org> ---
(In reply to Eugene Grosbein from comment #25)
Also, I do not understand how a router can deal without delivering traffic locally for its local IPs in many cases like running squid proxy or web/smtp server or even routinely running ping to check if distinct uplink is still alive.
You seem to not understand that traffic works just FINE without the loopback
route, packets sent locally to the local ip address of an interface are
delivered without any issue, the different is that the MTU used for the packets
well be the MTU of the ethernet/ppp/slip/whatever interface rather than the MTU
of the loopback device.

Further in your specific case of a ping check... that is near worthless to a
lo0 route as it can't tell when the carrier has dropped on an interface, and
infact this is one of the reasons I absolutely HATE this loopback route by the
kernel code and have ripped it out of my systems.

I think somehow you think that IP does not work correctly without the loopback
routes, that is a not the case. I repeat, all the interface loopback routes
are not needed, the system functions fine without them. all be it with a 10x
packet count due to MTU delta for lo0 vs an ethernet.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-06 06:29:51 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #27 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Rodney W. Grimes from comment #26)
Further in your specific case of a ping check... that is near worthless to a lo0 route as it can't tell when the carrier has dropped on an interface
No, no, I was thinking about incoming ICMP echo replies delivery. Some external
resource is checked by its IP address, of course. Not even remote IP of PPP
interface if we need to check global connectivity - we ping some global
resource with source IP of distinct link.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-06 14:36:21 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #28 from Rodney W. Grimes <***@FreeBSD.org> ---
(In reply to Eugene Grosbein from comment #27)
No, no, I was thinking about incoming ICMP echo replies delivery. Some external resource is checked by its IP address, of course. Not even remote IP of PPP interface if we need to check global connectivity - we ping some global resource with source IP of distinct link.
How would that in any way involve any loopback route???? You seem to be
confused on exactly when these loopback routes come into play. An incoming
ICMP echo reply is going to be picked off as "to this host" at ip_input
processing and the route table wont even be looked at.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-07 06:11:03 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #29 from Kubilay Kocak <***@FreeBSD.org> ---
Hey folks, just a heads up that the temperature of this issue is coming across
higher than it probably should be.

Let's please focus on the substance of the issue "out there" with everyone on
the same side looking at it together to figure it out.

Alternatively, if there's *key* design decision(s) to be made with respect to
'resolving' the issue, or a ton of questions that need to be thrown back and
forth, either do so succinctly (as questions and answers) here, which is fine,
or hash it out as a broader discussion on the mailing lists until an outcome is
reached and then reference the list discussion and outcome here.

Thanks!
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-07 06:41:19 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #30 from Eugene Grosbein <***@freebsd.org> ---
(In reply to Kubilay Kocak from comment #29)

I'm just fine with current "temperature" :-) Аnd I see nothing wrong with
discussion in comments here.
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-07 07:01:11 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #31 from Kubilay Kocak <***@FreeBSD.org> ---
(In reply to Eugene Grosbein from comment #30)

I know you're both tough, might look different to the community from the
outside though :)

Just a reminder to be mindful is all.

Carry on!
--
You are receiving this mail because:
You are on the CC list for the bug.
b***@freebsd.org
2021-06-07 14:38:39 UTC
Permalink
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256393

--- Comment #32 from Nathan Whitehorn <***@FreeBSD.org> ---
(In reply to Alexander V. Chernikov from comment #18)

It is stock 13-STABLE, with no custom sysctls or fibs. I'll try to set a
maintenance window to test the things you asked for some time in the next few
days.
--
You are receiving this mail because:
You are on the CC list for the bug.
Loading...