ELAM (Embedded Logic Analyzer Module) is an engineering tool that gives us the ability to look inside Cisco ASICs and understand how a packet is being forwarded. ELAM is “embedded” within the forwarding pipeline and can capture a packet in real time without affecting performance or control plane resources. It can help answer questions like:
- Did the packet reach the forwarding engine?
- On what port and VLAN was the packet received?
- What did the packet look like (L2 – L4 data)?
- How was the packet altered and where was it sent?
- And much more…
ELAM is extremely powerful, granular, and non-intrusive. It is an extremely valuable troubleshooting tool for Cisco TAC engineers who work on hardware switching platforms.
ELAM challenges
ELAM was designed as a diagnostic tool for internal use. The CLI syntax utilizes internal code names for Cisco ASICs and interpreting the ELAM data requires some hardware specific architecture and forwarding knowledge. Many of these details cannot be explained because they expose the internal Cisco proprietary features that make Cisco devices best in class.For these reasons, ELAM is not a customer-supported feature and has remained a diagnostic tool for internal use. There are no external configuration guides and the syntax and operation may change from version to version without any notice. So, with such a firm disclaimer and challenges, why are we discussing ELAM now?
First, it is very common that a TAC engineer will utilize ELAM to isolate an issue. TAC may even request you to perform ELAM if the issue is intermittent. It’s important to understand that these steps are non-intrusive and how they can help provide a root cause analysis.
Also, there are times when there are no other tools are available that can help isolate an issue (i.e., no configuration changes allowed during production hours for SPAN, ACL hits, intrusive debugs, etc…). There may not be time to get TAC on the line and ELAM can be an extremely helpful tool to have in your back pocket as a last resort.
Because we had some TAC cases in SWE core already and SWE core is simple L2 network I was able to cover ELAM basics and specifics for our purpose.
ELAM supported hardware
ELAM unfortunately is not supported on every Cisco platform. As far as I know ELAM is supported on Nexus7K, N77K, N5672K (because it uses N6K software) and others. ELAM is not supported on N5K! In N5K you have to use etheanalyzer with ACL log to hit the CPU for dataplane capture.
ELAM workflow
As previously discussed, ELAM is dependent on the underlining hardware and therefore the CLI syntax will be dependent on the hardware in use. However, each platform will follow a similar workflow as shown below. Please refer to the ELAM Examples section to see how this workflow is applied on different platforms.
- Identify the expected ingress-forwarding engine (FE). When platforms have more than one FE, it is critical to identify which FE makes the forwarding decision for the packet you are trying to capture. Configure the ELAM on the correct FE.
- Configure the ELAM trigger. You will need to configure a trigger with details specific to the packet you are trying to capture. Common triggers include a source and destination IP address or L4 port numbers. ELAM allows for multiple fields to be specified and will perform a logical AND on all fields configured.
- Start the ELAM
- Wait for the ELAM to trigger and display the result.
DBUS vs RBUS
DBUS and RBUS are two mechanisms how to analyse elam capture. DBUS (Data bus) contains information where the packet was received and it’s L2-L4 information. The RBUS contains the forwarding decision made by the FE. Viewing the RBUS helps determine if the frame was altered and where it was sent.
Local Target Logic (LTL)
The LTL is an index used to represent a port or group of ports. The source LTL index and the destination LTL index tell us where the frame was received and where it was sent. Different platforms and supervisors will have different commands to decode the LTL values.
More information about ELAM can be find at https://supportforums.cisco.com/document/9880486/understanding-elam