Switchless 16G FC homelab: qla2xxx qlini_mode=disabled is ignored on RHEL 8, no target LUNs
I am putting together a switchless Fibre Channel setup at home: one box holds the storage and has to act as the FC target, the other two are plain initiaators. No FC switch in the middle, just direct cables between HBAs.
Hardware:
- QLogic QLE2694 in the storage box, the one that should be the target
- QLogic QLE2690 and QLE2692 in the two initiator boxes
- FTLF8529P4BCV-QL 16G SFP+ LC optics, QLogic coded, 850 nm short reach
- 3 m OM3 multimode LC-LC cords between the ports
The optical side gives me no trouble at all. Ports come up, the cards see each other, everything behaves as long as both ends stay initiators. The problem is flipping the storage box into target mode. I put the module option in place, rebuilt the initramfs, rebooted, and the parameter simply never takes:
# cat /etc/modprobe.d/qla2xxx.conf
options qla2xxx qlini_mode="disabled"
# after rebuilding the initramfs and rebooting
# cat /sys/module/qla2xxx/parameters/qlini_mode
enabledtargetcli starts fine, but with initiator mode still on there is no FC fabric to hang a backstore off, so nothing gets exported and the initiators see an empty link.
What I tried:
- the option in /etc/modprobe.d and, separately, on the kernel command line
- regenerating the initramfs and the boot configuration after every change
- swapping which card sits in the storage box, in case it was a QLE2694 quirk
Is this something about the driver build on the distro I am running, which is RHEL 8 based, or am I missing a step in how the option is applied?
Comments 4
You are not missing a step, the option cannot work where you are running it. RHEL 8 and later ship a qla2xxx build with the ability to disable initiator mode taken out, so wherever you put
qlini_mode="disabled"it is ignored and the card stays an initiator. Nothing wrong with the QLE2694, the optics or the fibre.Two ways out: run the target host on a distribution that still keeps target mode in qla2xxx, or build the module yourself, which I would not do on a box meant to hold data. I went the first way and put the target on Fedora Server 39.
After that the sequence is the boring one:
in modprobe.d, then rebuild the initramfs and the GRUB configuration, reboot, and check the ports with
Install targetcli and sysfsutils, enable the FC ports, then map your LVM volumes off the NVMe as block backstores, bind them to the port WWNs and give each initiator its own ACL. The initiator boxes can stay exactly as they are, the restriction only bites on the target side.
One buying note for anyone doing this later: prefer the Dell branded QLE2690, QLE2692 and QLE2694, because their documentation actually covers target mode, which saves a lot of guessing.
Two things to pin down before anyone guesses.
First, where does your qla2xxx actually come from: the in-box module that ships with the distro kernel, or an out-of-tree build you pulled from the vendor and compiled yourself? On a RHEL 8 base those two behave very differently here, and it matters a lot more than the modprobe syntax.
Second, post
systool -c fc_host -vfrom the storage box. It tells us whether the pors actually come up and at what speed, so we can separate a driver question from a physical one.Also confirm the optics on both ends. FTLF8529P4BCV-QL is an 850 nm short reach part, so it needs multimode fibre - on single mode you would get nothing at all, and I want that ruled out before we talk about target mode.
In-box driver: stock kernel and stock qla2xxx straight from the distro repositories, nothing pulled from the vendor, nothing compiled by hand.
systool -c fc_host -vshows both ports on the QLE2694 online at 16G, and the same FTLF8529P4BCV-QL modules are on both ends over the 3 m OM3 cords, so the physical layer really is not the issue - as initiator the whole thing works.sysfs still reports qlini_mode as enabled after every rebuild and reobot, no matter whether I set it in modprobe.d or on the kernel command line. sysfsutils and targetcli are installed, targetcli itself starts without complaining, there is just nothing FC shaped to configure in it.
Adjacent trap worth carrying with you when you move the target host to another distro: make sure the parameter lands where that box's boot path actually reads it.
Classic version of this is a Proxmox host that would not accept its optiics. The admin put
ixgbe.allow_unsupported_sfp=1into modprobe.d, then into the GRUB config, and neither did a thing, because the machine boots through EFI and never touches GRUB. There the parameter has to go into/etc/kernel/cmdlinefollowed bypve-efiboot-tool refresh.Same failure signature as yours: the config file says one thing, sysfs says another, and you lose an evening suspecting the hardware. Whatever the hots, read the value back out of
/sys/module/...after each reboot and trust that rather than the file you edited.