No IPFW binary compat across versions ?

Discussion:

Arnaud Lacombe

2011-09-05 18:31:08 UTC

Hi,

It would seem that the ipfw binary from a 7.4 install is not
compatible with the in-kernel implementation of ipfw from 8-STABLE.
The following command returns junk:

# uname -a
FreeBSD 8.2-STABLE FreeBSD 8.2-STABLE #0: Mon Sep 5 13:26:22 EDT 2011

# ipfw show
65535 79609572473438209 9528064790723821568 count ip from any to any
# ipfw show
65535 80229898894966785 10211949650127093760 skipto 20069 ip from any to any
# ipfw show
65535 81461570961408001 11566216593050435584 divert 20069 ip from any to any
# ipfw show
65535 81826462792941569 11967688918742073344 tee 20069 ip from any to any
# ipfw show
65535 85659197118611457 16196106284901072896 altq ?<301> ip from any to any
# ipfw show
65535 86531998897668097 17160025365645623296 ** unrecognized action 64
len 19 ip from any to any
# ipfw show
65535 127737816351244289 7272335601653776384 ** unrecognized action
158 len 19 ip from any to any

However as the 8-STABLE kernel have COMPAT_FREEBSD7 enabled, I would
not expect this to happen...

Thanks,
- Arnaud

K. Macy

2011-09-05 19:14:34 UTC

Permalink

-STABLE only implies that the ABI does not change during that release
line. It makes no guarantees when moving from one branch to the next.

Post by Arnaud Lacombe
Hi,
It would seem that the ipfw binary from a 7.4 install is not
compatible with the in-kernel implementation of ipfw from 8-STABLE.
# uname -a
FreeBSD 8.2-STABLE FreeBSD 8.2-STABLE #0: Mon Sep 5 13:26:22 EDT 2011
# ipfw show
65535 79609572473438209 9528064790723821568 count ip from any to any
# ipfw show
65535 80229898894966785 10211949650127093760 skipto 20069 ip from any to any
# ipfw show
65535 81461570961408001 11566216593050435584 divert 20069 ip from any to any
# ipfw show
65535 81826462792941569 11967688918742073344 tee 20069 ip from any to any
# ipfw show
65535 85659197118611457 16196106284901072896 altq ?<301> ip from any to any
# ipfw show
65535 86531998897668097 17160025365645623296 ** unrecognized action 64
len 19 ip from any to any
# ipfw show
65535 127737816351244289 7272335601653776384 ** unrecognized action
158 len 19 ip from any to any
However as the 8-STABLE kernel have COMPAT_FREEBSD7 enabled, I would
not expect this to happen...
Thanks,
- Arnaud
_______________________________________________
http://lists.freebsd.org/mailman/listinfo/freebsd-net

Arnaud Lacombe

2011-09-05 20:18:27 UTC

Permalink

Hi,

Post by K. Macy
-STABLE only implies that the ABI does not change during that release
line. It makes no guarantees when moving from one branch to the next.

IIUC, FreeBSD does not provide binary backward compatibility between
version at all, is that it ?

Thanks,
- Arnaud

Post by K. Macy

Arnaud Lacombe

2011-09-05 21:44:48 UTC

Permalink

Hi,

Post by Arnaud Lacombe
Hi,

Post by K. Macy
-STABLE only implies that the ABI does not change during that release
line. It makes no guarantees when moving from one branch to the next.

IIUC, FreeBSD does not provide binary backward compatibility between
version at all, is that it ?

I guess the answer is "NO", on the same system:

# netstat -m
netstat: memstat_mtl_find: zone mbuf_jumbo_pagesize not found

It would seem that COMPAT_FREEBSD7 is just as useful as is eyes powder
on a scare-crow...

- Arnaud

Post by Arnaud Lacombe
Thanks,
- Arnaud

Post by K. Macy

Adrian Chadd

2011-09-05 23:50:27 UTC

Permalink

That's not what the COMPAT_* hooks are for.

They're for backwards compatibility of normal userland binaries, not
binaries which use a FreeBSD-specific kernel ABI.
Adrian

Arnaud Lacombe

2011-09-06 00:18:30 UTC

Permalink

Hi,

Post by Adrian Chadd
That's not what the COMPAT_* hooks are for.
They're for backwards compatibility of normal userland binaries, not
binaries which use a FreeBSD-specific kernel ABI.

What do you define as "normal" and where do you draw the line ?

Post by Adrian Chadd
From my point of view, I should be able to run a FreeBSD 9.0 kernel

(when released) on top of a FreeBSD 5 userland without such issues.
That's what backward compatibility is. _Every_ piece of the ABI
exposed by the kernel[0] should be kept compatible, even funky
behavior userland might ends up relying on.

However, if as you say, you (the committers folks) willingly break the
exposed ABI, well, sorry, but that can no longer be called "backward
compatible"...

That said, this has turned out of context.

Now, to come back to my original issue, how am I supposed to track
down cross-release issue ? swap storage ? [sic...]

Regards,
- Arnaud

[0]: ... with very few exception, such are security issue when there
is no other choices available.

Doug Barton

2011-09-06 00:45:36 UTC

Permalink

Post by Adrian Chadd
From my point of view, I should be able to run a FreeBSD 9.0 kernel
(when released) on top of a FreeBSD 5 userland without such issues.

Unfortunately your expectation is completely unrealistic. We do our best
to maintain backward compatibility but sometimes improvements require
breaking the KBI/ABI.

Also, we have never supported running a kernel from RELENG_N on anything
older than the latest version of RELENG_{N-1}.

hth,

Doug

--
Nothin' ever doesn't change, but nothin' changes much.
-- OK Go

Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price. :) http://SupersetSolutions.com/

Arnaud Lacombe

2011-09-07 04:16:22 UTC

Permalink

Hi,

Post by Doug Barton

Post by Adrian Chadd
From my point of view, I should be able to run a FreeBSD 9.0 kernel
(when released) on top of a FreeBSD 5 userland without such issues.

Unfortunately your expectation is completely unrealistic.

Completely unrealistic ? Seriously, you've got to be kidding me ?

I just downloaded a 4 (_four_) years old OpenWRT image, built the
latest Linux kernel in development and it worked just fine. netstat(8)
works just fine, I can pick a random iptables tutorial on the net, and
it still works fine. Those binaries are 4 years old and still work, oh
and I can go back at will to the original 2.6.19 without issue.

Now, let's see FreeBSD. Beside the issues included in that thread, the
latest development are the following: if you boot a FreeBSD 8-STABLE
kernel, with a 7.4 userland, then reboot on a 7-STABLE kernel, the
system will no longer boot, even in single user. Why ? because
fsck_ufs(8) crashes on a SIGFPE. I'll avoid commenting further.

Anyway, now I'll be obliged to re-install (well, find the motivation
first) if I want to be able to track down the mbuf corruption in
FreeBSD 7-STABLE I already reported.

Happy bikeshed coloring,

- Arnaud

Post by Doug Barton
We do our best
to maintain backward compatibility but sometimes improvements require
breaking the KBI/ABI.
Also, we have never supported running a kernel from RELENG_N on anything
older than the latest version of RELENG_{N-1}.
hth,
Doug
--
Nothin' ever doesn't change, but nothin' changes much.
-- OK Go
Breadth of IT experience, and depth of knowledge in the DNS.
Yours for the right price. :) http://SupersetSolutions.com/

K. Macy

2011-09-07 05:05:42 UTC

Permalink

Post by Arnaud Lacombe
Hi,

Post by Doug Barton

Post by Adrian Chadd
From my point of view, I should be able to run a FreeBSD 9.0 kernel
(when released) on top of a FreeBSD 5 userland without such issues.

Unfortunately your expectation is completely unrealistic.

Interesting. I remember binaries routinely breaking with minor glibc
changes, such that in the linux community it was accepted that dynamic
linking was "a bad thing". However, it is entirely possible that
they've grown up in the mean time while we've stood still.

Adrian Chadd

2011-09-07 05:56:51 UTC

Permalink

Try running an updated udev on an older kernel, because you're using
PVM virtualisation but don't want to deal with the latest kernels.
Useless, but all the current distributions use the latest udev, so
require up to date kernels.

A bunch of linux stuff "works" across large swaths of time because a
bunch of those things are text files in /proc.

FreeBSD could do this for a variety of things, but:

* someone has to do the legwork (define a public API and versioning
scheme for it), then write the code, then convert the utilities, then
handle backwards compatibility (eg by providing read-only methods for
earlier API versions, defining what the default behaviour should be
for older versions that are missing the newer features)
* someone has to champion getting it into the tree
* someone has to keep it up to date

Just please keep in mind the current method (hi KVM access) allows for
things like reading the current routing table from a crashdump using
the same tools you're using on a live system. You'd likely have to
come up with a unified API for accessing both live data and crashed
kernel data. That's not out of the question, it'll just take some
time.)

If you'd like to do it, I bet everyone will cheer you on. Honest :)

Adrian

Bruce Evans

2011-09-06 03:07:20 UTC

Permalink

This post might be inappropriate. Click to display it.

Julian Elischer

2011-09-06 04:02:47 UTC

Permalink

Post by Arnaud Lacombe
Hi,

Post by K. Macy
-STABLE only implies that the ABI does not change during that release
line. It makes no guarantees when moving from one branch to the next.

IIUC, FreeBSD does not provide binary backward compatibility between
version at all, is that it ?

# netstat -m
netstat: memstat_mtl_find: zone mbuf_jumbo_pagesize not found
It would seem that COMPAT_FREEBSD7 is just as useful as is eyes powder
on a scare-crow...
- Arnaud

That is not quite correct.

WE do not guarantee interfaces that are used for system configuration etc.

Interfaces used in normal data processing applications we try very
hard to keep stable,
It is assumed that applications that use the system administration
interfaces will come with the system.

for example I can run a "make world" in a chroot populated with
freebsd 1.1 and it actually works (after a few minor tweeks)
(actually I last tested this at 8.1 ).. required a change of PID_MAX
in the kernel config file and a.out compatibility

K. Macy

2011-09-06 10:54:05 UTC

Permalink

Hi,

Post by Arnaud Lacombe
Hi,

Post by K. Macy
-STABLE only implies that the ABI does not change during that release
line. It makes no guarantees when moving from one branch to the next.

IIUC, FreeBSD does not provide binary backward compatibility between
version at all, is that it ?

Netstat depends on specific structures in kernel memory, ipfw depends
on specific ioctls whose arguments may vary. Netstat in particular
breaks on head on a quasi daily basis because of changes to networking
structures. STABLE implies STABLE for things that depend on libc and
are not dependent on obscure kernel interfaces. There is some work
going on (off and on) to decouple netstat from kvm which would fix
that. N-1 compatibility for ipfw is probably possible, anything more
than that is probably infeasible from an interface tracking
standpoint.