Splitting Mellanox ConnectX-4 interface using breakout cables

[Andrey V. Elsukov]

Hi All,

we are wondering, is it possible to use such configuration under
FreeBSD? I.e. split one mce interface 40G => 4x10G or 100G => 4x25G?

[David Horn]

I was under the impression that these Mellanox 40G->4x10G breakout cables
were for the Mellanox switch side (not Mellanox NIC) to provide more
flexible utilization options for the switch. I have never heard of doing
this from the 40G NIC (on any OS).

Please reply if you have seen this working on *any* OS from the Mellanox
40G NIC, as I am curious as well and have a both ConnectX-3 and -4 HW.

-Dave

Post by Andrey V. Elsukov
Hi All,
we are wondering, is it possible to use such configuration under
FreeBSD? I.e. split one mce interface 40G => 4x10G or 100G => 4x25G?
--
WBR, Andrey V. Elsukov

[Andrey V. Elsukov]

Post by David Horn
I was under the impression that these Mellanox 40G->4x10G breakout
cables were for the Mellanox switch side (not Mellanox NIC) to provide
more flexible utilization options for the switch. I have never heard of
doing this from the 40G NIC (on any OS).
Please reply if you have seen this working on *any* OS from the Mellanox
40G NIC, as I am curious as well and have a both ConnectX-3 and -4 HW.

We have several Intel-XL710-QDA1 cards that work like this:

***@pci0:6:0:0: class=0x020000 card=0x00028086 chip=0x15848086 rev=0x01
hdr=0x00
***@pci0:6:0:1: class=0x020000 card=0x00008086 chip=0x15848086 rev=0x01
hdr=0x00
***@pci0:6:0:2: class=0x020000 card=0x00008086 chip=0x15848086 rev=0x01
hdr=0x00
***@pci0:6:0:3: class=0x020000 card=0x00008086 chip=0x15848086 rev=0x01
hdr=0x00

This is one card with QSPF28 splited into 4x10G.

[Matt Joras]

On Sep 13, 2017 6:43 AM, "David Horn" <***@gmail.com> wrote:

I was under the impression that these Mellanox 40G->4x10G breakout cables
were for the Mellanox switch side (not Mellanox NIC) to provide more
flexible utilization options for the switch. I have never heard of doing
this from the 40G NIC (on any OS).

Please reply if you have seen this working on *any* OS from the Mellanox
40G NIC, as I am curious as well and have a both ConnectX-3 and -4 HW.

-Dave

Post by Andrey V. Elsukov
Hi All,
we are wondering, is it possible to use such configuration under
FreeBSD? I.e. split one mce interface 40G => 4x10G or 100G => 4x25G?
--
WBR, Andrey V. Elsukov

I was also under the impression that they were exclusively used on the
switch for Mellanox. Mellanox's documentation seems to back this up:

"Important: The 40GbE split options is supported only on Mellanox
switches and not supported on Mellanox adapters (e.g. ConnectX-3) when
equipped with 40GbE ports. In case you wish to limit the 40GbE port on
the adapter to 10GbE you can use QSA or similar 40GbE-10GbE cables
(Refer to Mellanox.com - here)" [1]

That doesn't mention their newer NICs, however, though I don't see a
mention of the feature on their product sheets.

Matt

[1] https://community.mellanox.com/docs/DOC-1450