Intel X520 in an R630 rejects a 10GBASE-T copper SFP+ with comp_codes_10g=0x00 despite allow_unsupported_sfp=1
We are consolidating a pair of R630s onto 10G and the cabling to the top of the rack is copper, so instead of pulling fibre I put 10GBASE-T SFP+ modules into the X520 cards. The switch end takes them without a word. The servers refuse to.
- Dell PowerEdge R630, Intel X520 (82599), dual port
- FS SFP-10GM-T-30, Dell-coded, one module per server
- out-of-tree ixgbe from Intel, built through DKMS
- /etc/modprobe.d/ixgbe.conf with the override set for both ports
# cat /etc/modprobe.d/ixgbe.conf
options ixgbe allow_unsupported_sfp=1,1
# dmesg
ixgbe: failed to load because an unsupported SFP+ module type was detected
# 10G compliance codes read back from the module
comp_codes_10g=0x00What I have already tried:
modprobe ixgbe allow_unsupported_sfp=1by hand as well as the modprobe.d entry- rebuilt the initramfs and cold booted the machine, not just a module reload
- moved the module to the second port and then to the second server, same result
The interesting part is that compliance byte: the module reports nothing at all for 10G. Does the driver test that before it ever looks at the override, and is there anything to be done short of buying optics with the right coding?
Also available in: English
العربية
Deutsch
Español
Français
हिन्दी
Bahasa Indonesia
Italiano
日本語
한국어
Nederlands
Polski
Português
Русский
Türkçe
Tiếng Việt
简体中文
Comments 6
You are not fighting a whitelist, you are fighting the order of the checks.
SFF-8472 has no compliance bit for 10GBASE-T. There simply is no code point for it, so an honest copper SFP+ reports all-zero 10G compliance codes, which is exactly your
comp_codes_10g=0x00. ixgbe reads that byte, finds nothing it recognises as a 10G module and gives up right there, before it gets anywhere near theallow_unsupported_sfpoverride. That is why the flag works fine for an unqualified optical module or a DAC and does nothing whatsoever for your copper ones.There is a community patch against Intel's out-of-tree ixgbe that moves the compliance test: when the administratr has explicitly set
allow_unsupported_sfp=1, a module reporting all-zero 10G compliance codes gets classified as SR instead of being thrown out early. It was submitted upstream and is still unmerged, so you apply it to the DKMS source by hand and keep it with your build. Whoever wrote it reported 10 Gb/s full duplex on a pair of servers afterwards, and someone else confirmed the same patch makes HLX-SFPX copper modules work in an X520.Two cavaets before you do it. An optic Intel has not qualified is outside their compatibility guarantee, so this becomes your problem, not theirs. And the 10GBASE-T PHY is a hot part - in a server cage with no airflow of its own it will sit well above anything optical in the neighbouring slot, so watch the module temperature once the link is up.
Two things to pin down before you go patching anything.
First, print the parameter as the kernel actually sees it,
/sys/module/ixgbe/parameters/allow_unsupported_sfp, on a machine that has been through a cold boot rather than a module reload. If that does not read back exactly what stands in your conf file, something is loading the driver before your config is in play and the rest of the debugging is wasted.Second, which ixgbe is loaded?
ethtool -iis no use to you while the drivver never finishes loading and the interfaces are absent, so post whatmodinfo ixgbereports and the version of the DKMS package you built.And where does that
comp_codes_10g=0x00come from - is that the driver telling you, or did you dump the module EEPROM yourself?Parameter is
1,1in the file and/sys/module/ixgbe/parameters/allow_unsupported_sfpreads back1,1after a cold boot, so it is applied, not silently ignored. Initramfs was rebuilt before the boot. Same line in dmesg either way.The compliance codes I read out of the module myself from the SFF-8472 data, not from the driver - the 10G compliance byte is zero, everything else in the ID fields looks sane. Same module in the switch port links at 10G, so it is not a dead module.
Worth adding on the practical side: with DKMS every kernel update rebuilds from the sources on disk, so the patch has to live in that source tree, not in a build directory you cleaned out afterwards. Check the port comes back after the first kernel bump rather than finding out during a reboot window.
The wider lesson from this class of problem is that driver vintage decides more than module coding does. Same story on an X710 with a passive DAC: one cable, one port, happy under Ubuntu 24.04 and dead under TrueNAS SCALE with
Link detected: noandSpeed: Unknown, because that build carried the i40e out of the 6.6.44-production kernel. On 25.04-BETA.1, where i40e is taken from 6.12.9-production, the twinax came up by itself - nothing else touched, card still on firmware 9.20. Optics were fine on that port under either system, which is what pinned the fault on how the older driver handles passive copper.ethtool -ion both sides of the comparison would have saved a lot of cable swapping.Different module type, same driver, and a trap worth ruling out while you are in there. Dell R720 with an X520 daughter card, Cisco 10G multimode LC modules refused, interfaces simply missing. The option was in modprobe.d, it was in GRUB too, and nothing changed - because the host boots via EFI and that GRUB command line was never used.
On an EFI-booted Proxmox the parameter belongs in
/etc/kernel/cmdlineasixgbe.allow_unsupported_sfp=1, followed bypve-efiboot-tool refresh. In that case even that did not fix it and they ended up buying Intel-branded modules, so treat it as something to eliminate rather than as a cure.The other thing from that mess: both ends have to be happy with the optic independently. A module the swwitch accepts can still be rejected by the host, which is where you already are.
Applied the patch to the DKMS source and rebuilt. Both ports come up at 10 Gb/s full duplex and have stayed up under load since.
Working, but I would not call it solved. It is an unmerged patch that I now carry across every kernel update, and the drvier presents the port as SR, which will confuse whoever looks at this box after me. The module also runs distinctly hotter than the optics in the next cage over and that slot has no airflow worth the name. For the next batch of servers I will pull fibre and stop arguing with the driver.