MACsec · Link security

Key and transport failures

Diagnose independent failures using their evidence, make a focused repair, then verify both security and forwarding.

Example platform scope. CLI examples use supported Catalyst 9500 physical ports on IOS XE 17.18.x. The example port is TenGigabitEthernet1/0/1; substitute the actual supported port on your model. Verify hardware, license and release support in the linked guide. Configuration fragments state their prerequisites; all output is illustrative, with unrelated fields omitted.

Keep the fault scenarios independent

Each scenario below starts from the working direct link in Protected link configuration. R1 and R2 use PSK MKA, CKN 01, mandatory protection and GCM-AES-128. The transport examples explicitly replace the direct cable with a separately qualified Ethernet service. They are not extra faults stacked on the same run.

Baseline for independent fault casesIllustrative topology

Restore the baseline before studying the next failure. All excerpts are illustrative; no operational failure injection is implied.

Keep management access and record both peers' time, configuration references, MKA state and counter deltas. Do not print operational CAKs into shared logs. A pending session is a symptom, not a unique diagnosis.

Case 1: the CAK differs at R2

Illustrative failure · R1

R1# show mka sessions
Total MKA Sessions....... 1
 Secured Sessions... 0
 Pending Sessions... 1

R1# show mka statistics interface TenGigabitEthernet1/0/1
MKPDU Failures
 MKPDU Rx Validation.............. 7

Validation failures indicate received MKA messages did not pass the relevant checks; they do not uniquely identify a wrong CAK. In this independent case, a secure comparison against the intended secret store confirms that R2 has the wrong CAK under the correct CKN 01. Correct only that confirmed mismatch. If the CKN itself differs, repair the identifier as well; do not expect a common association to form from unrelated identifiers.

Configuration repair · R2 · public example key only

configure terminal
key chain LINK-KEY macsec
 key 01
  key-string 8c40a71d25f6913b70e2c94a56d809be
end

Illustrative restored state · R1

R1# show mka sessions
Total MKA Sessions....... 1
 Secured Sessions... 1
 Pending Sessions... 0

Use the intended private secret in a real repair. Repeat active SAK/confidentiality and traffic-counter checks; the historical failure total need not return to zero. Newly observed validation failures should stop increasing.

Case 2: the next key is not active

Illustrative failure · R2 · known rollover window

R2# show mka sessions
Total MKA Sessions....... 1
 Secured Sessions... 0
 Pending Sessions... 1

In this case both configured key strings match, but the devices disagree on time and the old CAK has expired on one side. Inspect clock/NTP and the scheduled key lifetimes using management access. Correct the trusted time source or the incorrectly scheduled lifetime, provision a common next key at both ends and restore an overlapping transition window. Do not assume changing the SAK can repair an inactive CAK.

Illustrative restored state · R2 · after coordinated time/key repair

R2# show mka sessions interface TenGigabitEthernet1/0/1 detail
Status: SECURED - Secured MKA Session with MACsec
CAK Name (CKN)........... 02
Latest SAK Status........ Rx & Tx

The active identifier has moved to the intended next credential. Confirm the same association at R1 and ongoing protected traffic. For certificate authentication, perform the analogous time/validity checks at the EAP stage; an expired certificate is not a PSK mismatch.

Case 3: the service filters EAPOL

Here the direct cable is replaced with an Ethernet service, and the service does not pass the required link-local EAPOL traffic. The endpoints' known-good credentials have not changed.

Illustrative failure · R1 and R2 · same observation interval

R1# show mka statistics interface TenGigabitEthernet1/0/1
MKPDU Statistics
 MKPDUs Transmitted......... 20
 MKPDUs Validated & Rx...... 0

R2# show mka statistics interface TenGigabitEthernet1/0/1
MKPDU Statistics
 MKPDUs Transmitted......... 30
 MKPDUs Validated & Rx...... 0

The different local totals are not an error by themselves. Both transmit while neither validates peer MKA. Check physical receive counters, filtering, EAPOL destination handling and both service edges before blaming the CAK. A qualified wire observation or provider evidence distinguishes messages never delivered from messages delivered but invalid.

The focused repair is to restore the known-good direct link, or have the service correctly transport the documented EAPOL and MACsec profiles. There is no universal “carrier fix” CLI. Do not add arbitrary custom EtherTypes at one end; any supported customization must be agreed and verified end to end.

Illustrative restored state · R1 · after transport repair

R1# show mka sessions
Total MKA Sessions....... 1
 Secured Sessions... 1
 Pending Sessions... 0

R1# show mka statistics interface TenGigabitEthernet1/0/1
MKPDU Statistics
 MKPDUs Transmitted......... 45
 MKPDUs Validated & Rx...... 12

Validated messages now arrive and the session is secured. Also test protected data transport and size: permitting EAPOL alone is not a complete repair.

Case 4: protected data fails after keying succeeds

In one independent service test, MKA is secured but larger data frames fail. In another, a reordered service drives late drops. Keep those observations separate.

Illustrative failure A · R1 · MKA works, larger test fails

R1# show mka sessions
Total MKA Sessions....... 1
 Secured Sessions... 1
 Pending Sessions... 0
R1# ping 192.0.2.2 size 1400 source 192.0.2.1
!!!!!
Success rate is 100 percent (5/5)
R1# ping 192.0.2.2 size 1500 source 192.0.2.1
.....
Success rate is 0 percent (0/5)

The size boundary suggests an MTU/frame-size problem but is not proof by itself. Confirm the actual on-wire size and the limiting hop. Repair the supported service frame budget or agree a smaller endpoint IP MTU; do not disable MACsec as a size workaround. Retest the intended largest supported packets.

Illustrative restored A · R1 · after qualifying the larger frame budget

R1# ping 192.0.2.2 size 1500 source 192.0.2.1
!!!!!
Success rate is 100 percent (5/5)

Illustrative failure B · R2 · reordered service interval

R2# show macsec interface TenGigabitEthernet1/0/1
Replay window : 0
Receive Secure Channels
 SA Statistics
  Valid pkts 2000
  Invalid pkts 0
  Late pkts 38

After independently confirming bounded legitimate reordering, the design can restore ordered transport, or accept a measured nonzero window under its security policy. A window of 64 below is an example decision, not a default recommendation or a duplicate-rejection guarantee.

Configuration repair B · R1 and R2 · qualified reordered transport only

configure terminal
interface TenGigabitEthernet1/0/1
 macsec replay-protection window-size 64
end

Illustrative restored B · R2 · later snapshot after 100 valid frames

R2# show macsec interface TenGigabitEthernet1/0/1
Replay window : 64
Receive Secure Channels
 SA Statistics
  Valid pkts 2100
  Invalid pkts 0
  Late pkts 38

Valid traffic advances while the late total no longer increases over the observation interval. Compare against the prior valid count and the peer's encrypted count. A zero new-drop interval does not guarantee the service will never reorder beyond 64. Return to window zero when returning to the ordered direct-link baseline.

Close with an evidence-based restoration

For every repair, confirm the expected peer/credential, secured MKA, active receive/transmit SAKs, intended confidentiality/replay/access policy and bidirectional traffic deltas. If authentication succeeded but the test still fails, verify VLAN/SVI/forwarding state independently. Preserve the failure and restored-state evidence with timestamps; do not erase cumulative counters merely to make a display look clean.

Key Takeaways

Keep key, time, control-transport, frame-size and reordering faults as independent scenarios.
Pending MKA and validation failures require corroboration before a key is changed.
CAK schedules and certificate validity depend on correct trusted time.
EAPOL reachability does not by itself prove protected data transport or a sufficient frame budget.
Restore the intended protection policy and verify bidirectional traffic instead of allowing accidental cleartext.

Technical references and example notes

IEEE 802.1AE · IEEE 802.1X and MKA · Platform configuration and verification · Topology-symbol attribution.

Diagrams, frame sequences and command results are constructed teaching examples, not recorded captures or lab measurements.

← Session and counter checks