About 15% packet loss through an S+RJ10 copper SFP+ on a CRS518-16XS-2XQ while the direct 100G path is clean
We push traffic from a lab server into a CRS518-16XS-2XQ over a 100G QSFP28 uplink, and it leaves the switch through a MikroTik S+RJ10 copper SFP+ into a plain 1G RJ45 host. The receiving side is missing a large share of the packets and I cannot pin it on anything obvious.
Setup:
- MikroTik CRS518-16XS-2XQ, 100G QSFP28 uplink from the traffic source
- MikroTik S+RJ10 copper SFP+ in one of the cages, 1G RJ45 device on the far end
- capture running on the receiving host
What the capture says:
100G QSFP28 uplink -> CRS518-16XS-2XQ -> S+RJ10 -> 1G RJ45 host
capture on the 1G host: ~15% of the packets never arrive
same source, direct 100G connection: nothing missing
switch CPU load: 1%Tried so far:
- connected the same source directly at 100G, no loss at all, so the sender itself is fine
- brought the switch CPU down, it now sits at 1% while the loss still hapens
- reseated the S+RJ10 and swapped the patch cord to the 1G device
The copper module is my main suspect at this point, but the link is clean and the interface shows no errors at all. Is the S+RJ10 known to eat traffic like this, or should I be looking somewhere else inside the switch?
Also available in: English
العربية
Deutsch
Español
Français
हिन्दी
Bahasa Indonesia
Italiano
日本語
한국어
Nederlands
Polski
Português
Русский
Türkçe
Tiếng Việt
简体中文
Comments 5
400-500 Mbps average with a bursty sender is the whole story. Your traffic is not spread evenly: short bursts leave the source faster than 1 Gbps, and everything above that line has to sit in the egress buffer of the port until the 1G side drains it. When the buffer is full, the switch drops. That is exactly what the rx-overflow movement is telling you, and it is why the direct 100G connection shows nothing - there is no speed step down there to buffer against.
The transceiver is innocent. Anything in that cage, copper or fibre, would behave the same, because the drop happens on the 100G to 1G step down and not inside the module.
Two things to do. The real fix is on the sender: pace it so the packets are spread evenly instead of being written in bursts. Once the source stops producing bursts above the egress rate, the loss disappears.
On the switch you can make the buffer situation less hostile:
That buys headroom and lets a burst ride out longer, but it does not remove the cause - if the sender bursts hard enough for long enuogh, no buffer size will save you. Keep watching the switch QoS statistics and the rx-overflow counters after the change, so you can see whether you are still hitting the ceiling or only touching it occasionally.
The general lesson is worth keeping: a port with a clean link, no errors and a healthy module can still drop a double-digit share of traffic purely because of a speed step down between ports.
Before blaming the module, look at what the port counters actually say. Run
on the 100G ingress port and on the cage with the S+RJ10 in it, and go specifically for the rx-overflow line rather than the usual rx/tx error counters. A copper SFP+ that is genuinely broken announces itself as FCS errors or a flapping link, not as a tidy 15% haircut off an otherwise healthy stream.
Second question: what is the average rate through that path, and do you have any idea about the peaks? Losing one packet in seven with the CPU at 1% smells far more like the egress port running out of buffer than like a transceiver fault.
Counters first: no errors on either port, the link stays up the whole time and the module reports nothing unusual. rx-overflow is the only place where the numbers move at all.
On rate, the path averages 400-500 Mbps, so on paper it is nowhere near saturating the 1G side. I have no peak measurement, but the traffic is bursty by nature - the sender writes a chunk and then goes quiet for a while. CPU is still 1% while packets go missing.
Different failure, same lesson about trusting counters over intuition. I had a CRS354-48G-4S+2Q+RM on SwOS 2.18 with steadily growing Rx FCS Errors on both QSFP+ ports, and Rx MAC Errors at a lower rate. Both ports sat at 40G full duplex with MTU 1500, and on the far side were ESXi hosts with Mellanox ConnectX-3 Pro CX324A cards in them.
The interesting part: the NIC side reported nothing whatsoever.
Clean. Known-good cables changed nothing, and the 10G SFP+ ports of the same box stayed error free the whole time. I never got a real diagnosis - moving the switch from SwOS to RouterOS made the counters vanish, which I count as hiding the problem rather than solving it.
The method that survived: clear the counters, re-read them over a fixed interval and see whether the errors track traffic volume. In your case they will track the bursts, in mine they tracked nothing useful, and that difference alone tells you which side to keep digging on.
One thing to keep in mind while you experiment on that port: do not reach for forced speed and duplex as a way out. The documented behaviour of the MikroTik copper modules, S-RJ01 and S+RJ10 alike, is that they only work with auto-negotiation enabled - pin the rate statically and the link does not come up at all. Practice partly contradicts that, since a few RB5009 and RB4011 owners report the opposite and only got an S-RJ01 stable by forcing 1G ful duplex, so it is a "try both on your own hardware" thing rather than a rule. Either way it is a detour from your actual problem, which is on the buffer side.
The other S+RJ10 detail worth knowing for later: it draws noticeably more power than a normal optic and runs hot, so it is not recommended in a passively cooled device without extra airflow. If that module ever starts misbehaving in a warm chassis, temperature is the firrst thing I would check.