Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LLDPD announces 100GbaseLR4 instead of SR4 #477

Open
thiell opened this issue Sep 10, 2021 · 3 comments
Open

LLDPD announces 100GbaseLR4 instead of SR4 #477

thiell opened this issue Sep 10, 2021 · 3 comments
Labels
bug

Comments

@thiell
Copy link

thiell commented Sep 10, 2021

We have a server with 100G SR4 optics which is detected by LLDPD as LR4 and announced as such, making the switch complain. I'm wondering if this is a bug with lldpd or a firmware bug. We ethtool -m p1p1 shows SR4, and not LR4, I'm opening this ticket in case it's an issue with lldpd. Thanks!

# lldpcli -vv
lldpcli 1.0.12
  Built on 2021-08-20T20:31:48Z

Additional output formats:   TEXT, KV, JSON, XML

C compiler command: gcc -std=gnu99  -fdiagnostics-show-option -fdiagnostics-color=auto -pipe -Wall -W -Wextra -Wformat -Wformat-security -Wfatal-errors -Winline -Wpointer-arith -fno-omit-frame-pointer -Wno-cast-align -Wno-unused-parameter -Wno-missing-field-initializers -Wno-sign-compa
Linker command:     /bin/ld -m elf_x86_64  -Wl,-z,relro -Wl,-z,now  -fPIE -pie -Wl,-z,relro

Bug description

Steps to reproduce the problem

  1. Run lldpd
  2. Run lldpcli with "show interfaces ports p1p1 details"

Expected outcome

SR4 should be detected instead of LR4.

Current outcome

lldpcli reports LR4 instead of SR4 optics for this server.

[root@oak-rtr-scg-2 ~]# lldpcli 
[lldpcli] # show interfaces ports p1p1 details 
-------------------------------------------------------------------------------
LLDP interfaces:
-------------------------------------------------------------------------------
Interface:    p1p1
  Administrative status: RX and TX
  Chassis:     
    ChassisID:    mac b0:26:28:bb:04:a2
    SysName:      oak-rtr-scg-2.sunet
    SysDescr:     CentOS Linux 7 (Core) Linux 3.10.0-1127.10.1.el7.x86_64 #1 SMP Wed Jun 3 14:28:03 UTC 2020 x86_64
    MgmtIP:       10.0.0.116
    MgmtIface:    2
    MgmtIP:       fe80::b226:28ff:febb:4a2
    MgmtIface:    2
    Capability:   Bridge, off
    Capability:   Router, off
    Capability:   Wlan, off
    Capability:   Station, on
  Port:        
    PortID:       mac b8:59:9f:30:7c:1c
    PortDescr:    p1p1
    PMD autoneg:  supported: yes, enabled: yes
      Adv:          1000Base-X, HD: no, FD: yes
      MAU oper type: 100GbaseLR4 - 100GBASE-R PCS/PMA over 4 WDM lane single mode fiber, long reach
  TTL:          120
-------------------------------------------------------------------------------

Switch connected to it is complaining about that with:

Sep  8 23:14:25: %1-3-LLDP: [0xd0a70554] lldp_util.c(4927) 2910 %% Local Port 23 receive a mismatch TLV in Auto-negotiation Enable
Sep  8 23:14:25: %1-3-LLDP: [0xd0a70554] lldp_util.c(4914) 2909 %% Local Port 23 receive a mismatch TLV in Auto-negotiation Support
Sep  8 23:14:25: %1-3-LLDP: [0xd0a70554] lldp_util.c(4849) 2908 %% Local Port 23 receive a mismatch TLV in MAU type

Full output of ethtool -m p1p1:

[root@oak-rtr-scg-2 ~]# ethtool -m p1p1
	Identifier                                : 0x11 (QSFP28)
	Extended identifier                       : 0x8c
	Extended identifier description           : 2.5W max. Power consumption
	Extended identifier description           : CDR present in TX, CDR present in RX
	Extended identifier description           : High Power Class (> 3.5 W) not enabled
	Connector                                 : 0x0c (MPO Parallel Optic)
	Transceiver codes                         : 0x80 0x00 0x00 0x00 0x00 0x00 0x00 0x00
	Transceiver type                          : 100G Ethernet: 100G Base-SR4 or 25GBase-SR
	Encoding                                  : 0x05 (64B/66B)
	BR, Nominal                               : 25500Mbps
	Rate identifier                           : 0x00
	Length (SMF,km)                           : 0km
	Length (OM3 50um)                         : 70m
	Length (OM2 50um)                         : 0m
	Length (OM1 62.5um)                       : 0m
	Length (Copper or Active cable)           : 50m
	Transmitter technology                    : 0x00 (850 nm VCSEL)
	Laser wavelength                          : 850.000nm
	Laser wavelength tolerance                : 15.000nm
	Vendor name                               : FS
	Vendor OUI                                : 00:02:c9
	Vendor PN                                 : QSFP28-SR4-100G
	Vendor rev                                : A4
	Vendor SN                                 : S1908052750
	Revision Compliance                       : SFF-8636 Rev 2.5/2.6/2.7
	Module temperature                        : 47.15 degrees C / 116.87 degrees F
	Module voltage                            : 3.2802 V
	Alarm/warning flags implemented           : Yes
	Laser tx bias current (Channel 1)         : 7.058 mA
	Laser tx bias current (Channel 2)         : 7.058 mA
	Laser tx bias current (Channel 3)         : 7.058 mA
	Laser tx bias current (Channel 4)         : 7.186 mA
	Transmit avg optical power (Channel 1)    : 1.7630 mW / 2.46 dBm
	Transmit avg optical power (Channel 2)    : 1.6254 mW / 2.11 dBm
	Transmit avg optical power (Channel 3)    : 1.5026 mW / 1.77 dBm
	Transmit avg optical power (Channel 4)    : 1.5178 mW / 1.81 dBm
	Rcvr signal avg optical power(Channel 1)  : 1.1667 mW / 0.67 dBm
	Rcvr signal avg optical power(Channel 2)  : 1.1072 mW / 0.44 dBm
	Rcvr signal avg optical power(Channel 3)  : 1.1786 mW / 0.71 dBm
	Rcvr signal avg optical power(Channel 4)  : 1.1635 mW / 0.66 dBm
	Laser bias current high alarm   (Chan 1)  : Off
	Laser bias current low alarm    (Chan 1)  : Off
	Laser bias current high warning (Chan 1)  : Off
	Laser bias current low warning  (Chan 1)  : Off
	Laser bias current high alarm   (Chan 2)  : Off
	Laser bias current low alarm    (Chan 2)  : Off
	Laser bias current high warning (Chan 2)  : Off
	Laser bias current low warning  (Chan 2)  : Off
	Laser bias current high alarm   (Chan 3)  : Off
	Laser bias current low alarm    (Chan 3)  : Off
	Laser bias current high warning (Chan 3)  : Off
	Laser bias current low warning  (Chan 3)  : Off
	Laser bias current high alarm   (Chan 4)  : Off
	Laser bias current low alarm    (Chan 4)  : Off
	Laser bias current high warning (Chan 4)  : Off
	Laser bias current low warning  (Chan 4)  : Off
	Module temperature high alarm             : Off
	Module temperature low alarm              : Off
	Module temperature high warning           : Off
	Module temperature low warning            : Off
	Module voltage high alarm                 : Off
	Module voltage low alarm                  : Off
	Module voltage high warning               : Off
	Module voltage low warning                : Off
	Laser tx power high alarm   (Channel 1)   : Off
	Laser tx power low alarm    (Channel 1)   : Off
	Laser tx power high warning (Channel 1)   : Off
	Laser tx power low warning  (Channel 1)   : Off
	Laser tx power high alarm   (Channel 2)   : Off
	Laser tx power low alarm    (Channel 2)   : Off
	Laser tx power high warning (Channel 2)   : Off
	Laser tx power low warning  (Channel 2)   : Off
	Laser tx power high alarm   (Channel 3)   : Off
	Laser tx power low alarm    (Channel 3)   : Off
	Laser tx power high warning (Channel 3)   : Off
	Laser tx power low warning  (Channel 3)   : Off
	Laser tx power high alarm   (Channel 4)   : Off
	Laser tx power low alarm    (Channel 4)   : Off
	Laser tx power high warning (Channel 4)   : Off
	Laser tx power low warning  (Channel 4)   : Off
	Laser rx power high alarm   (Channel 1)   : Off
	Laser rx power low alarm    (Channel 1)   : Off
	Laser rx power high warning (Channel 1)   : Off
	Laser rx power low warning  (Channel 1)   : Off
	Laser rx power high alarm   (Channel 2)   : Off
	Laser rx power low alarm    (Channel 2)   : Off
	Laser rx power high warning (Channel 2)   : Off
	Laser rx power low warning  (Channel 2)   : Off
	Laser rx power high alarm   (Channel 3)   : Off
	Laser rx power low alarm    (Channel 3)   : Off
	Laser rx power high warning (Channel 3)   : Off
	Laser rx power low warning  (Channel 3)   : Off
	Laser rx power high alarm   (Channel 4)   : Off
	Laser rx power low alarm    (Channel 4)   : Off
	Laser rx power high warning (Channel 4)   : Off
	Laser rx power low warning  (Channel 4)   : Off
	Laser bias current high alarm threshold   : 14.996 mA
	Laser bias current low alarm threshold    : 3.002 mA
	Laser bias current high warning threshold : 12.998 mA
	Laser bias current low warning threshold  : 4.496 mA
	Laser output power high alarm threshold   : 3.5328 mW / 5.48 dBm
	Laser output power low alarm threshold    : 0.0617 mW / -12.10 dBm
	Laser output power high warning threshold : 2.8160 mW / 4.50 dBm
	Laser output power low warning threshold  : 0.1444 mW / -8.40 dBm
	Module temperature high alarm threshold   : 70.00 degrees C / 158.00 degrees F
	Module temperature low alarm threshold    : -10.00 degrees C / 14.00 degrees F
	Module temperature high warning threshold : 60.00 degrees C / 140.00 degrees F
	Module temperature low warning threshold  : 0.00 degrees C / 32.00 degrees F
	Module voltage high alarm threshold       : 3.6352 V
	Module voltage low alarm threshold        : 2.9696 V
	Module voltage high warning threshold     : 3.4672 V
	Module voltage low warning threshold      : 3.1304 V
	Laser rx power high alarm threshold       : 3.5328 mW / 5.48 dBm
	Laser rx power low alarm threshold        : 0.0398 mW / -14.00 dBm
	Laser rx power high warning threshold     : 2.8160 mW / 4.50 dBm
	Laser rx power low warning threshold      : 0.0795 mW / -11.00 dBm

Additional information

  • Output of lldpd -vv:
lldpd 1.0.12
  Built on 2021-08-20T20:31:48Z

Additional LLDP features:    LLDP-MED, Dot1, Dot3, Custom TLV
Additional protocols:        CDP, FDP, EDP, SONMP
SNMP support:                yes
Old kernel support:          no (Linux 2.6.39+)
Privilege separation:        enabled
Privilege separation user:   _lldpd
Privilege separation group:  _lldpd
Privilege separation chroot: /var/run/lldpd
Configuration directory:     /etc

C compiler command: gcc -std=gnu99  -fdiagnostics-show-option -fdiagnostics-color=auto -pipe -Wall -W -Wextra -Wformat -Wformat-security -Wfatal-errors -Winline -Wpointer-arith -fno-omit-frame-pointer -Wno-cast-align -Wno-unused-parameter -Wno-missing-field-initializers -Wno-sign-compa
Linker command:     /bin/ld -m elf_x86_64  -Wl,-z,relro -Wl,-z,now  -fPIE -pie -Wl,-z,relro
  • Output of ps -fp $(pgrep -d, -x lldpd):
UID         PID   PPID  C STIME TTY          TIME CMD
_lldpd    96150      1  0 15:09 ?        00:00:00 /usr/sbin/lldpd
_lldpd    96152  96150  0 15:09 ?        00:00:00 /usr/sbin/lldpd
  • Output of uname -sro:
Linux 3.10.0-1127.10.1.el7.x86_64 GNU/Linux
@thiell thiell added the bug label Sep 10, 2021
@akkornel
Copy link

Hello! I'm Stéphane's coworker, and the administrator of the switch that generated the warning. It warned that the LLDP TLVs for Autonegotiation Support, Autonegotiation Enable, and MAU Type did not match between the switch and the server.

Indeed, if there was an MAU type mismatch, it's highly unlikely that the physical link would even be able to come up, which made me suspect either a bug somewhere (in the transceiver, the OS, or lldpd).

After some searching, it looks like this is because lldpd is making an assumption about the MAU for 40, 50, and 100GBASE: https://github.com/lldpd/lldpd/blob/master/src/daemon/interfaces-linux.c#L662

@akkornel
Copy link

I'm very much not a kernel person, but I've been poking around the Linux source with https://elixir.bootlin.com/linux/latest/source, and I have a guess why this is happening. Is it because lldpd is using the pre-netlink ethtool API, which doesn't have enough bits to hold all of the possible values for ethtool_link_mode_bit_indices?

I looked through the documentation on https://lldpd.github.io, but I didn't see any mention of this behavior. A complete rewrite to use netlink instead of ethtool would be a lot of work, so maybe it's better for now to mention this in the man page or other documentation?

@vincentbernat
Copy link
Member

lldpd is using ETHTOOL_GLINKSETTINGS which can use all the bits. So, it should be possible to correctly map all information. https://github.com/lldpd/lldpd/blob/master/src/daemon/interfaces-linux.c#L555 should be updated and used instead of speed to get the appropriate information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vincentbernat @thiell @akkornel