Supermicro E300-9A on pfSense Plus 22.05: ix2 and ix3 stay at no carrier with a DAC that links fine on a USW-Aggregation
My firewall is a Supermicro E300-9A running pfSense Plus 22.05, and both 10G SFP+ ports refuse to come up. Neither ix2 nor ix3 ever shows carrier, whatever I put in the cage.
Hardware:
- Supermicro E300-9A, pfSense Plus 22.05
- Ubiquiti DAC-SFP10-0.5M and a 10Gtek passive twinax cable
- Supermicro AXS85-192-M3 fibre modules as the alternative
- Ubiquiti USW-Aggregation on the switch side
# ifconfig ix2
ix2:
media: Ethernet autoselect
status: no carrierix3 looks the same.
Tried so far:
- both cables work on the USW-Aggregation between other devices, so they are not dead
- swapped copper for the AXS85-192-M3 fibre modules, same no carrier on both ports
- rebooted the appliance several times, including inserting a module while it was running
Is there something in this box that has to be kicked before the cages start working, or am I looking at two dead ports?
Also available in: English
العربية
Deutsch
Español
Français
हिन्दी
Bahasa Indonesia
Italiano
日本語
한국어
Nederlands
Polski
Português
Русский
Türkçe
Tiếng Việt
简体中文
Comments 4
Warm reboots will not get you anywhere - those ports latch a media state and never re-probe on a restart. Shut the appliance down properly, pull the power adapter out for a couple of minutes, then start it again with the module already inserted. That is what brought both ports back here, and another person described identical behaviour on an Intel X552 port, which is why I think it is a stale media state rather than a pfSense problem.
If you insert a module while the system is already running, bounce the interface instead of rebooting:
That makes the driver look at the cage again. It is not a permanent fix for anything, but it saves a reboot when you are swapping modules on the bench. Check the result with
ifconfig -arather than the front panel.Do the full power removal first and confirm both cages with a DAC before you touch the switch side. Debugging one thing at a time matters here, because "no carrier with every module" and "link comes up at the wrong speed" are usually two separate faults that happen to be in the same cable run.
Full power reemoval did it. Shut down, adapter out, waited a couple of minutes, powered back on - both ports came up. Looped a DAC between ix2 and ix3 and got a clean 10G link, and the AXS85-192-M3 pair also does 10G between the two ports, so the cages and the modules are fine.
The switch side is another story. Toward the USW-Aggregation the link only ever negoitates at 1G, and if I force 10G on either end it goes down and stays down. So half the problem is gone and the annoying half is still here.
The 1G fallback half looks very familiar. I chased the same symptom on a TL-SG3428X and a TL-SX3008F: restart a server hanging off one of those SFP+ ports and it came back negotiated at 1G no matter what the switch port was configured for. Intel X520-DA2, Mellanox and HP adapters, Intel E10GSFPSR and 10GTek optics, firmware updates, several driver versions on Linux and Windows, port profiles - none of it changed anything. A switch reboot, or toggling the port speed away from 10G and back, restored the 10G link until the next host reset.
What actually fixed it was changing the optics rather than anything on the host: TP-Link SM5110-SR modules on the switch end and the link came back at 10G evry time. Someone else confirmed the same on an SG3428XMPP. The reading was that the switch mis-negotiates with some third-party modules after a host-side link reset.
Different vendor on your side, but the shape matches. Before you buy a set of anything, borrow a single Ubiquiti-branded module and test one port on the aggregation switch.
On forcing 10G: doing it on one end only makes things worse, not better. The peer is still trying to negotiate and a fixed setting gives it nothing to negotiate against, so the link simply stays down - which is exactly the behaviour you describe. Fix speed and duplex on both ends, or on neither.
Two related cases from the MikroTik world, in case they ring a bell. On an RB4011 a Finisar FTLF8524P2BNV-BR was detected with sfp-rx-loss and sfp-tx-fault both showing no, and the interface still said no-link, because a 1G SFP in an SFP+ cage has to be pinned rather than negotiated:
and, again, on both ends. The second one was a CCR1072 where auto-negotiation stayed DONE after a link loss and the driver never restarted it; disabling autoneg and fixing the speed brought the link back, at the price of propre link-down detection.