Programmers, coding boards, switches, NICs and other hardware
We are wiring an FPGA capture card into an existing server and I am stuck at the very first step: the port simply never comes up. The setup: Mellanox MCX414-GCAT (ConnectX-4 EN, QSFP28, 50GbE capable) in the host FS Q-8LCA020 breakout AOC, QSFP+ on one end, eight LC ends on the other FPGA card on…
Standing up 25G uplinks from a QFX5120-48Y towards a second switch and a couple of new servers. Before I hand the segment over I want the link settings written down properly, and I cannot find where auto-negotiation lives on these ports. Juniper QFX5120-48Y 25G optics in the SFP28 cages et…
We are refreshing the core of a small office and put in a pair of Aruba CX 6200F. The link between the two switches is meant to be a 10G trunk on the SFP+ uplinks, and I cannot get either box to admit that 10G exists on that port. 2x Aruba CX 6200F Switch Series, trunk on port 1/1/25 Cisco…
A good half of my work is putting one company's optics into another company's switch, so I spend a lot of time asking a module what it thinks it is. My toolbox for that is currently one command deep, and I would like to know what else is worth putting on the bench machine. Linux boxes with SFP+ and…
Small office, we are dropping the ISP-supplied box and terminating the fibre straight on the router. The handoff is AON, the provider only asks for a VLAN tag, no PPPoE and no ONT registration. What is in the rack: TP-Link ER7412-M2 V1, firmware 1.0.1, fibre into port 11 (SFP WAN) a TP-Link…
We are pushing Wi-Fi into an outbuilding on a campus site. The only fibre going that way is a single strand, so the plan was to hang an AP-375 out there and feed it over a BiDi pair we already had on the shelf. Aruba AP-375, ArubaOS 8 Cisco BiDi SFP in the AP cage, its matching partner wavelength…
A dual-port ConnectX-5 (MT27800, firmware 16.35.1012, MLNXOFED 5.8 on BC-Linux 8.2) connected to a switch with a DAC would not recover after the cable had been connected and disconnected about fifteen times: the port stayed in State: Polling with Physical state ETHANFSMABILITYDETECT, and mlxlink…
On a Mellanox SN2410 (MLNX-OS 3.6.4112) a 100G port connected to a Chelsio adapter through a 2 m unequalised QSFP copper cable would not link when FEC was disabled; no HSS lane signal was detected on the adapter side. The poster required FEC-off operaation and asked whether the switch supports it…
I look after a two-building site and the access switches were replaced with Aruba 2540s a while back. The fibre between the buildings has not been touched, but since the swap the uplink refuses to come up. Aruba 2540 on our side, ports 51 and 52 used for the uplinks Aruba J4858D 1000SX in the 2540…
Mellanox-branded Q-4S28AO03 3 m QSFP28 AOC 1x4 breakout cables (100G with FEC) that worked under Onyx 3.9.1908 stopped linking after an upgrade to Onyx 3.10.x. The ports reported Cable speed not enabled and, with auto speed, Negotiation failure; supported speeds showed 1G/10G/25G but the actual…
Adding a 100G link between two racks on an X690 that has been quietly doing 10G access work for a couple of years. The uplink cage will not take the speed at all. ExtremeSwitching X690-48x-2q-4c 1 m QSFP28 direct attach cable, Q28+CR41m port 57, one of the QSFP28 uplink cages The cable comes up as…
A Mellanox ConnectX-4 Lx (MCX4121A-ACAAx, firmware 14.32.1900) in a Ryzen desktop (Fedora 41 and Windows 10) detected the interface but reported Cable is unplugged with two FS.com SFP28-25GLR-31 modules and a 40 m single-mode LC duplex patch; a MikroTik router recognised the same optics. The poster…
A Mellanox ConnectX-3 40GbE NIC connected with Mellanox QSFP passive copper cables (MC2210126-005) to a Dell PowerConnect 8164 (firmware 5.1.4.5) produced a link-up event for a few seconds followed by a permanent link-down. The same cables worked NIC-to-NIC, and Dell refused support because the…
Bringing up a 10G uplink from a PA-3220 to a new core switch. The module went into a free cage on the firewall, the switch side is configured and waiting, and the interface simply refuses to come up. The port LED blinks, which is what made me assume the module was at least being powered. Palo Alto…
Standing up a small traffic-test bench next to the production fabric, and the tester ports will not come up over copper at all. HP 5900 40G switch, JC772A HP JG328A 40G QSFP+ direct attach cable Ixia Xcellon-Multis QSFP 40/10GE load module Spirent TestCenter 40G module, tried as a second opinion HP…
Bench work: re-coding a small batch of SFP+ modules so they carry the vendor string and part number the customer's kit expects. The edit itself is trivial in a hex editor, the write goes through, the read back matches byte for byte what I wrote - and the host still throws the module out. generic…
Installing a new MES2324FB at a site to replace old hardware. The uplink is 1G fiber to an Alcatel 6224, Foxgate modules from the same batch as at neighboring sites. Setup: Eltex MES2324FB, firmware 4.0.4.7, also tried 4.0.6 - no difference 1G SFP Foxgate on both ends of the link far end - Alcatel…
I look after a small office network and put in a DrayTek VigorSwitch FX2120 to get a 10G core. Both 10G links, to the router and to the NAS, came up on their own within seconds. What I cannot do is get a single 1G device onto the switch through the SFP+ cages. Switch: DrayTek VigorSwitch FX2120…
We are a small regional ISP rebuilding the optics budget for next year. Every 10G spare on our shelf is coded for exactly one platform, so the spares pool is roughly three times larger than our failure rate justifies. The idea on the table is to buy re-codable modules and code them ourselves as…
Replacing a customer's incumbent aggregation pair with CloudEngine kit, and the single link that refuses to come up is the 100GE feed into their packet broker. Everything else on the switch went in without drama. Huawei CloudEngine 6800, 100GE port 100GBASE-SR4 QSFP28 modules, Huawei parts, one at…
Commissioning a 10G handoff for a customer on a 3930 that was already on site. The module is Ciena's own, pulled from our stock, and the port does come up - but the operational state is UCTF instead of Ena and the uncertified transceiver alarm sits there permanently. The acceptance test includes a…
I am putting together a switchless Fibre Channel setup at home: one box holds the storage and has to act as the FC target, the other two are plain initiaators. No FC switch in the middle, just direct cables between HBAs. Hardware: QLogic QLE2694 in the storage box, the one that should be the target…
All 10G ports on a Netgear M5300-28G3 were used, so the poster asked whether a Cisco 3 m SFP+ DAC (H10GB-CU3M) or Finisar FTLX8571D3BCL 10G SR modules could be used in the switch's 1G SFP ports to add another 10G-capable device at reduced speed. Hardware involved: Netgear M5300-28G3, Cisco…
We pick up a 1GE carrier handoff on a Ciena 3930 and I am putting together a spares kit for it. The module quoted for that circuit is XCVR-A10Y31, a 1000BASE-LX/LC part: 1310 nm, 10 km single mode. Nothing exotic about it, and I have a drawer of generic optics with exactly that specification…
I am specifying a fibre ring for a customer's plant network and the switch choice is already fixed: Moxa EDS-518A. The BOM went out with generic 1G optics, because our stock room is full of them, and now the customer's own engineer insists that only Moxa-branded modules may be fitted. switches…
I look after a small pool of FreeBSD boxes and I am adding 10G to one of them. The card went in, the machine boots fine, but the OS refuses to make an interface out of it. Server on FreeBSD 12.1-RELEASE-p1, stock GENERIC kernel QLogic FastLinQ QL41112 dual-port 10GbE SFP+ adapter SFP+ optics in…
I run a backup storage system in a data center. A pair of DD9900 in HA, network ports on the NDC QLogic. After a reboot or reseating the module, port ethMa stops seeing the transceiver, and it's not a one-off - it's a stable, repeating pattern. Dell DD9900, DD OS 7.2.0.95, HA configuration NDC…
We run a pair of ThinkSystem NE1032 RackSwitch units as top of rack in a small colo cage. The Lenovo-branded optics we inherited only cover half the ports, so the rest got filled with generic SFP+ modules and a couple of short DACs out of the same batch we already use on other vendors' switches…
I look after a mixed Zyxel estate on one site and I am putting together a vrey basic optical health check for the fibre uplinks: temperature and Rx power per port, so I can see a link degrading before the users report it. What is involved: Zyxel GS1910, older access switch with SFP uplinks Zyxel…
Small office rack: there is a 40G card in the file server and a switch with spare uplink ports, so instead of buying another box I tried to hang the server straight off the switch. Both ends see the cable, the link never comes up. Huawei S6730-H24X6C, QSFP28 uplink ports QNAP LAN-40G2SF-MLX in the…
We are adding capacity to a small FC fabric and bought compatible 16G short-wave SFP+ from an optics supplier instead of paying the switch vendor price. The switch refuses them. Brocade G610, Fabric OS 9.x generic-coded 16G SWL SFP+, correct wavelength and reach according to the label Brocade-coded…
Second hand HPE 5130 for the rack at home, and it arrived with four HPE JD092B X130 10G SFP+ SR modules in the box. I bought new OM4 LC patch cords, wired one SFP+ port to the 10G card in the server, and got absolutely nothing. HPE FlexNetwork 5130 24G PoE+ 4SFP+ EI (JG936A) 4x HPE JD092B X130 10G…
I run a Turris Omnia NG at home and the last thing sitting on the metallic WAN port is a two metre hop to the ISP router. I would like to move that hop into the SFP+ cage and free the copper port for the lab side. The official Turris SFP+ copper module (RTROM01-RTSF-10G) costs about what a small…
Building a storage box at home and picked up a dual 10GbE Solarflare SFN7122F (SFC9120) because it was cheap. The card itself looks healthy, the system sees it and both ports enumerate, but not one of my existing multimode SFP+ modules will bring up a link in it. Solarflare SFN7122F, dual port…
We recode a small batch of modules for customer hosts most months, nothing exotic: read the original page, write the vendor string the host wants, put the module in the switch and move on. One module out of the current batch refuses every write while reading back perfectly, and before I bin it I…
We hand customer circuits off on MRV OptiSwitch demarcation boxes - mostly OS906, with a few OS904 and OS912 left on older builds. When a ticket comes in about a circuit that is down or flapping, I want two things over SSH before anyone drives out: what module is actually sitting in the port…
We run packet capture on a handful of Napatech cards and I am putting together the optics order for the next rack. Before spending anything I want to understand what the qualified module list actually means in practice, because the wording leaves room for interpretation. Napatech NT40E3 and NT20E3…
A FreeNAS box with on-board Intel X552 SFP+ ports (Xeon D) and an ESXi 6.7 host with a Mellanox ConnectX-3 (MT27500, Dell Y3KKR) were cabled with two Mellanox MCP2104-X003B passive DACs from 2014. Looping each device to itself brought the ports up, but between the two systems ESXi showed…
Two SRX1500 sit in separate data centres a few kilometres apart, joined by dark fiber pairs we own end to end. They have to come up as a chassis cluster, which means the control link has to cross that fiber instead of a patch cord inside one rack. The setup: two Juniper SRX1500, identical hardware…
We took over a pair of ZS3-2 controllers together with the shelves behind them, and I am the one who has to order optics before they go back into service. The controller service manual is very short on this point: it lists the NIC and HBA options, and for the 10GbE SFP+ card it names exactly two…
We are building out two racks and the switching is settled: a pair of M4500-48XF8C as the leaves with M4350 underneath. What is not settled is the optics BOM. NETGEAR does not put its own label on 25G or 100G modules, and I would rather not discover after delivery that the SFP28 ports dislike what…
We run a pair of FortiGate 101F units for a 40-person office, nothing exotic. port17 carries an SFP uplink to the core switch, port19 is an RJ45 run to the server room, both of them inside the shared RJ45/SFP block (ports 17-20). In the maintenance window last month we took the pair up to FortiOS…
One physical server on our ERS 8600 talks to nothing, and the switch is quite convinced everything is fine. The port has been in this state since the server was moved from copper to fibre. Avaya ERS 8600, server on a 1G SFP port in slot 3, port 12 server NIC with its own SFP, fibre patch through…
I run a small Proxmox box at home and wanted a proper 10G path to the storage node, so I dropped in a used Supermicro AOC-STGN-i1S. It is the plain Intel 82599 design, board marked E157872, and I assumed this would be the boring part of the build. It is not. Supermicro AOC-STGN-i1S, Intel X520-DA1…
The poster tried a Mellanox MFA2P10-Axxx 25G SFP28 AOC between a MikroTik switch and a Supermicro server with an Intel X722 (ixl driver) under pfSense. The cable linked fine switch-to-switch but on the X722 ifconfig -vvvm showed the module recognised (10GBase-SFI, 1000Base-KX listed) yet no carrier…
Bringing up a pair of GPU boxes against our existing 400G fabric, and I have already managed to buy the wrong cable once, so I would rather get the theory straight before ordering again. NIC: ConnectX-7 MCX715105AS-WEAT, single 400G port, OSFP cage Switch: 400G QSFP-DD ports In hand: a passive 400G…
A DS14-MK4 shelf came my way for almost nothing and I would like to hang it off an ordinary tower server as bulk storage at home. It is fully populated with its original carriers, it powers up, fans spin, front panel looks alive - but I cannot get a single disk to appear on the host. NetApp…
I do small batches of module recoding for integrators, and until now everything on my bench has been SFP-shaped: read the 256 bytes at A0h with the serial ID in the first half, read A2h for the diagnostics and thresholds, write the identification block, done. A customer has now turned up with 100G…
Running a small virtualization setup: two Proxmox 6 nodes and a target on FreeNAS 11.2. While the HBAs were connected to the target directly, everything ran for months without a single error. Put a used Brocade 200e between them to avoid crossing cables, and that's when it started. two Proxmox 6…
We run a pair of racks that hand off to a carrier-facing Cisco NCS, and one of the 100G server uplinks has never come up since the build. Same result after moving the server to another cabinet with a different patch panel, so I have stopped treating it as a one-off. Supermicro server…
