ISP BiDi SFP links in a Cisco switch but gives no carrier on the X710 ports of a pfSense box
Home setup. The fibre handoff from my provider is a 1G BiDi module and I would like it in the firewall itself instead of kepeing a switch in front of the box just to convert media.
- Protectli Vault 6 running pfSense, X710 ports, ixl driver
- the provider-supplied 1G BiDi module, an EZConn ETB43315-7S34-FR
- the same module links without any fuss in Cisco switches
- a Cisco 10G LR FTLX1474D3BCL-CS in the same cage also links fine
With the BiDi in ixl1 the port never leaves no carrier, and the log is not shy about the reason:
ixl1: no carrier
i40e link status timeout, I40E_ERR_TIMEOUT
Link failed because an unqualified module was detectedThere is also an error getting PHY capabilities in the same burst of messages.
Tried so far:
ifconfig ixl1 media 1000baseLX- refused, the port will not take that media typesysctl dev.ixl.1.advertise_speed- set it, no effect whatsoeverhw.ixl.unsupported_sfp=1in loader.conf plus a reboot - nothing, the unqualified module line still appears
So the card sees a module, decides it does not like it, and that ends the conversation. Is there a way to run a 1G optic on these ports at all, or is the honest answer to keep a switch in front of the firewall?
Also available in: English
العربية
Deutsch
Español
Français
हिन्दी
Bahasa Indonesia
Italiano
日本語
한국어
Nederlands
Polski
Português
Русский
Türkçe
Tiếng Việt
简体中文
Comments 6
You have two separate problems stacked on top of each other and they have to be cleared in order.
First, the firmware. 6.1 is old enough that the card does not present modules properly at all, which is why your power readings are blank. Going to 9.40 is what makes them show up with real RX and TX power. Do that before anything else, otherwise you are debugging blind and every later test is worthless.
Second, and this is the part that actually keeps the port down: on the X710 the module qualification lives in NIC firmware, not in the driver, so none of the knobs you tried can reach it. That is why
hw.ixl.unsupported_sfp=1does nothing here, whysysctl dev.ixl.1.advertise_speedis quietly ignored, and whyifconfig ixl1 media 1000baseLXis refused - the port will not select 1G media for a module it has already rejected withLink failed because an unqualified module was detected.What worked for me was recoding the optics with Intel vendor data. I used FS SFP-GE-BX modules, wrote Intel vendor data into them with the FS Box, and after that they came up on autoselect with no media forcing and no sysctls at all. Note that this changes the module, not the card, and the provider's EZConn is not yours to reprogram - buy your own BX pair with the right wavelengths and recode those.
If you would rather not touch EEPROMs, then use optics the card already accepts. Your Cisco 10G LR is one example, and an FS RJ45 1G SFP+ is another if the provider can hand you copper anywhere in the path.
Before anything else: which NIC firmware is on that X710? These cards shipped with builds old enough to change how a module is presented in the first place, and it matters more than any of the sysctls you haev been poking at.
Second question in the same breath: with the BiDi sitting in ixl1, does the port status show any RX and TX power for it, or is the module simply listed as present with no readings? Those two answers together tell you whether the card is refusing a module it can read, or failing before it ever gets that far.
Firmware is 6.1, whatever the box shipped with - I have never touched it.
And no, there are no redings. With the prvoider BiDi in ixl1 the module is listed as present but RX and TX power are blank. The Cisco 10G LR in the same cage does report power, which I assume is exactly the difference you were pointing at.
Worth knowing before you order a programmer for this job: the FS Box only programs FS modules. People who fed it something from another vendor have had their account locked for a week, so it is not a general purpose recoder, it is a tool for one vendor's optics.
If you need something that will take arbitrary modules, Flexoptix and the other programmers do the same job and cost considerably more, while the Ubiquiti wizard sits at the cheap end of the range. For a sinlge pair of BX modules the least painful route is usually to buy the optics from the same vendor as the box you already own.
Similar symptom, entirely different cause, so do not skip the boring checks.
I had a Supermicro AOC-STGN-i2S under pfSense with a 1G Ubiquiti UF-MM-1G in ix1, permanently in no carrier.
ifconfig -v ix1listed only autoselect and 10Gbase-SR as media, andifconfig ix1 media 1000baseTXcaem back with a device-not-configured error, which looks exactly like a driver refusing to run 1G on a 10G port. I moved the card into an Ubuntu live image: no link there either, neither with a DAC nor with the module. Then I put in a second, identical Unifi module and it linked instantly, under Ubuntu and under pfSense both. The first one was simply dead.A spare copy of the module and a second host cost you twenty minutes and eliminate half the theories. In your case the unqualified module message names the culprit fairly clearly, so you are probably fine, but it is cheap insurance before you spend money on a prgrammer.
Result, in the order it happened.
Firmware first, 6.1 to 9.40. Modules immediately started reporting RX and TX power, but the provider BiDi still refused to link and the unqualified module line was still in the log - so the firmware update buys you visibility and nothing else.
Then a pair of FS SFP-GE-BX recoded to Intel vendor data. Both ends up on autoselect, no media forcing, no
hw.ixl.unsupported_sfp, nothing left in loader.conf. I rolled baack the sysctls I had set while flailing and the link stayed up regardless.The provider module went into the drawer as a spare, since it is not mine to reprogram anyway, and the switch in front of the firewall is gone.