Tips about ASR9K interfaces

I don’t know if it can be useful for someone, but I share :

To know about optic budget :

RP/0/RSP0/CPU0:ASR9K-01#show controllers tenGigE 0/0/0/2 phy
Tue Dec  9 18:10:41.280 CET

SFP EEPROM  port: 2
        Xcvr Type: SFP
        Xcvr Code: SFP-10G-ER
        Encoding: 64B66B
        Bit Rate: 10300 Mbps
        Link Reach 9u fiber (Km): 40000 meter
        Link Reach 9u fiber (100m): 25500 meter
        Link Reach 9u fiber (100m): 25500 meter
        Vendor Name: CISCO-FINISAR  
        Vendor OUI: 00.90.65
        Vendor Part Number: FTLX1671D3BCL-C4 (rev.: A   )
        Laser wavelength: 1550 nm (fraction: 0.00 nm)
        Optional SFP Signal: Rate Sel, LOS
        Vendor Serial Number: FNS181206PL    
        Date Code (yy/mm/dd): 14/03/17  lot code:  
        Diagnostic Monitoring: DOM, Int. Cal.,
        Enhanced Options: SW RX LOS Mon., SW TX Fault Mon, SW TX Disable, Alarm/Warning Flags
        Thresholds:                    Alarm High         Warning High          Warning Low            Alarm Low
              Temperature:            +75.000 C             +70.000 C              +0.000 C              -5.000 C
                  Voltage:           3.630 Volt            3.465 Volt            3.135 Volt            2.970 Volt
                     Bias:        100.000 mAmps          95.000 mAmps          55.000 mAmps          50.000 mAmps
           Transmit Power:  5.012 mW (7.00 dBm)   2.512 mW (4.00 dBm)   0.339 mW (-4.70 dBm)   0.135 mW (-8.70 dBm)
           Receive Power:  1.585 mW (2.00 dBm)   0.794 mW (-1.00 dBm)   0.026 mW (-15.80 dBm)   0.011 mW (-19.79 dBm)
        Temperature: 41.539
        Voltage: 3.319 Volt
        Tx Bias: 81.050 mAmps
        Tx Power:  1.693 mW (2.29 dBm)
        Rx Power:  0.008 mW (-21.02 dBm)
        Oper. Status/Control: Rx Rate Select, 

More to see on the right…

Stats and more specifics errors :

RP/0/RSP0/CPU0:ASR9K-01#show controllers tenGigE 0/0/0/2 ? 
  all       Show all the information
  bert      Show BERT status
  control   Show configuration and control information(cisco-support)
  internal  Show internal information
  mac       Show mac information
  phy       Show phy information
  regs      Show registers information
  stats     Show stats information
  xgxs      Show xgxs information
  |         Output Modifiers
RP/0/RSP0/CPU0:ASR9K-01#show controllers tenGigE 0/0/0/2 stats
Tue Dec  9 18:10:54.905 CET
Statistics for interface TenGigE0/0/0/2 (cached values):

    Input total bytes           = 583691
    Input good bytes            = 583691

    Input total packets         = 4563
    Input 802.1Q frames         = 0
    Input pause frames          = 0
    Input pkts 64 bytes         = 0
    Input pkts 65-127 bytes     = 4108
    Input pkts 128-255 bytes    = 71
    Input pkts 256-511 bytes    = 269
    Input pkts 512-1023 bytes   = 61
    Input pkts 1024-1518 bytes  = 52
    Input pkts 1519-Max bytes   = 2 

    Input good pkts             = 4548
    Input unicast pkts          = 157
    Input multicast pkts        = 4347
    Input broadcast pkts        = 59

    Input drop overrun          = 0
    Input drop abort            = 0
    Input drop invalid VLAN     = 0
    Input drop invalid DMAC     = 0
    Input drop invalid encap    = 0
    Input drop other            = 0

    Input error giant           = 0
    Input error runt            = 0
    Input error jabbers         = 0
    Input error fragments       = 0
    Input error CRC             = 12
    Input error collisions      = 0
    Input error symbol          = 232214
    Input error other           = 3

    Input MIB giant             = 2
    Input MIB jabber            = 0
    Input MIB CRC               = 12


    Output total bytes          = 757008
    Output good bytes           = 757008

    Output total packets        = 5682
    Output 802.1Q frames        = 0
    Output pause frames         = 0
    Output pkts 64 bytes        = 59
    Output pkts 65-127 bytes    = 5239
    Output pkts 128-255 bytes   = 93
    Output pkts 256-511 bytes   = 162
    Output pkts 512-1023 bytes  = 71
    Output pkts 1024-1518 bytes = 58
    Output pkts 1519-Max bytes  = 0

    Output good pkts            = 5682
    Output unicast pkts         = 332
    Output multicast pkts       = 5350
    Output broadcast pkts       = 0

    Output drop underrun        = 0
    Output drop abort           = 0
    Output drop other           = 0

    Output error other          = 0

If you read it carefully you must have seen that my link have some problems… Which ones ?

I will probably write another article to talk only about NP, FIA, …

Etherate : the beginning of an Open Aurora Tango test (L2 tester)…

For those of you which perform some Aurora Tango tests (BERT, RFC 2544) you may know that a couple of these testers are really expensive ! The principle of these test is to place a tester in loopback and another is test mode. By means of this, you are able to qualify an Ethernet link (direct, Q-in-Q, VPLS, Xconnect).

During my readings of mailing-lists such as cisco-nsp or others, someone posts an URL to a really awesome Linux program : etherate. With modern CPU platform you are able to supply a throughput until 1G, and this program give you the possibility to test your L2 circuits.

You can do some tests by using it such as :


root@Loop_Host# etherate -r

And :

root@Tx_Host# etherate

To give some features :

root@pluton:~/COMPILE/Etherate-master# ./etherate -h
Usage info; [Mode] [Destination] [Source] [Options] [Other]
[Mode] By default run in transmit mode, not receive
    -r    Change to receive (listening) mode.
    -d    Without this we default to 00:00:5E:00:00:02
        as the TX host and :01 as the RX host.
        Specify a custom desctination MAC address, 
        -d 11:22:33:44:55:66
        Specify a custom source MAC address, -s 11:22:33:44:55:66
    -i    Set interface by name. Without this option we guess which
        interface to use.
    -I    Set interface by index. Without this option we guess which
        interface to use.
    -l    List interface indexes (then quit) for use with -i option.
    -s    Without this we default to 00:00:5E:00:00:01
        as the TX host and :02 as the RX host.
    -a    Ack mode, have the receiver ack each frame during the test
        (This will significantly reduce the speed of the test).
    -b    Number of bytes to send, default is 0, default behaviour
        is to wait for duration.
        Only one of -t, -c or -b can be used, both override -t,
        -b overrides -c.
    -c    Number of frames to send, default is 0, default behaviour
        is to wait for duration.
    -e    Set a custom ethertype value the default is 0x0800 (IPv4).
    -f    Frame payload size in bytes, default is 1500
        (1514 bytes is the expected size on the wire with headers).
    -m    Max bytes per/second to send, -m 125000 (1Mbps).
    -t    Transmition duration, integer in seconds, default is 30.
    -v    Add an 802.1q VLAN tag. By default none is in the header.
        If using a PCP value with -p a default VLAN of 0 is added.
    -p    Add an 802.1p PCP value from 1 to 7 using options -p 1 to
        -p 7. If more than one value is given, the highest is used.
        Default is 0 if none specified.
        (If no 802.1q tag is set the VLAN 0 will be used).
    -q    Add an outter Q-in-Q tag. If used without -v, 1 is used
        for the inner VLAN ID.
    -o    Add an 802.1p PCP value to the outer Q-in-Q VLAN tag.
        If no PCP value is specified and a Q-in-Q VLAN ID is,
        0 will be used. If no outer Q-in-Q VLAN ID is supplied this
        option is ignored. -o 1 to -o 7 like the -p option above.
    -x    Display examples.
    -V|--version Display version
    -h|--help Display this help text

I really hope and try to help so that this project goes to his end. It could be really great to have L2 tests as this one as we have with bwping and iperf on L3/L4 layer of OSI model.

EVC : Ethernet Virtual Connections

It is been a while that I read some papers about EVC, flexible matching, … But I have not found a good paper explaining EVC and a way to understanding well except to cisco configuration guide.


This figure is exactly what I searched for. This example is simple but enough to understand well how it works.

My misunderstanding was about “ingress” and “symmetric” : I didn’t understand, but now with the figure that it is clear that :

  1. encapsulation dot1q 10 : match all frames where a dot1Q tag is present with value 10
  2. “rewrite ingress tag pop 1 symmetric” : We pop 1 tag on this frame
  3. Give this frame to the MPLS xconnect without any tag

When a frame comes from the xconnect :

  1. It arrives without any tag
  2. With “symmetric”, we push the 10 dot1q tag
  3. put in TX the frame through the GigabitEthernet 0/2

You can ask me : “Ok this is really complicated your configuration, why don’t you use xxxx” ?
Where xxx is :

int gi 0/2
switchport trunk encapsulation dot1q
switchport trunk allow vlan 10
switchport mode trunk

int vl 10
xconnect 33 encapsulation mpls

I will reply : “Have you declare your vlan 100 on your switch ? What does it involve for the scalability ?”
=> Yes, you understand well : you are limited by the number of vlans.
=> With EVC : the “encapsulation dot1q 10” is local to the port. So no limitation.
=> Yes it is really great !!!!!

I urge you to read these papers :

BFD : Bidirectional Forwarding Detection (RFC5880)

BFD is a mecanism which give you access to a fast switchover for IGP, EBGP, … It is a RFC protocol : RFC5880.

BFD is a UDP protocol.

  • BFD control : UDP / 3784
  • BFD echo : UDP / 3785

BFD is in use on a segment, when a protocol needs it (such as : OSPF). It is configured by interface and called in protocol configuration.
When BFD detect a failure it informs upper protocols and helps to make a better / quick convergence.

You can configure BFD like :


Then you enable it on IGP configuration :

router ospf 1
 network area 0
 bfd all-interfaces

BFD initialization works with a 4-state’s automate. These states are :

  • DOWN
  • INIT
  • UP

ADMIN-DOWN : as it says the system want to hold the session DOWN by an administrative point of view : “I want to keep this session DOWN, so calm down and shut up as long as I say!”

DOWN : says that the session has just been created, administratively or operationnaly it maintains the state to DOWN until we receive a BFD DOWN control datagram from the other side : “We are agree that the session is DOWN, we can go on and try to initialize it!”

INIT : We come from DOWN state. Both systems are communicating. Until we receive an INIT or UP, we keep in INIT and send BFD INIT datagram.

UP : Ok both sides are agreed on INIT/UP we finalize the session and go together to UP states. We are UP until we detect it fails or ADMIN DOWN state is said to me ! So at TX interval we send Control packet with UP state until we detect a problem or been administratively set to DOWN.

Authentication :

BFD is aware of different authentication methods such as simple passowrd, MD5, SHA1.

Timers negotiation :

Timers are continually negotiated during the session by means of sending BFD Control datagram where :

  • Interval : Max rate (minimum interval), we are able to send
  • Min_RX : Minimum interval we are able to compute
  • Min Echo RX : Minimum interval we are able to compute for the Echo function
  • Multiplier : is a normal value which is used to calculate the Dead-Timer (HoldOn, …) : number of BFD datagrams lost in a row before we say that the session is DOWN. Dead Timer is : mult x max(desired TX interval , Required RX received interval)

So each side can adjust their timers and this at any time of the session life.

Demultiplexing :

To identify between multiple sessions between two systems, a discriminator (A unique ID). When we receive a BFD datagram with my discriminator, I know it is for __this session and I can compute it correctly.

Echo function :

This function is a way to detect and transmit packets/test forwarding plane : We send BFD echo packet with :

  • IP SRC = Me
  • IP DST = Me !
  • UDP / 3785
  • BFD information to demultiplex the session involved.

We can use Echo function only when BFD Control session is UP. After this, we can slow down sending BFD Control packet and only base failure detection by means of Echo function. We detect failure by using number of failure on a row as BFD Control Packets.

We can use BFD without this function only with BFD Control.

R1 (conf-if)# no bfd echo

Lab test :

You can find a cloudshark pcap trace here :

It consists of an BFD session initialization (you can find in the pcap by adding the filter : ‘bfd’). Then an outage has been produced by removing vlan on SW1 on the trunk (frame 249 : “Diag : Echo function failed”). And then allow it again on the trunk to reform the BFD session).

Checks :

R1#show bfd neighbors  details 

IPv4 Sessions
NeighAddr                              LD/RD         RH/RS     State     Int                              1/1          Up        Up        Et0/1.13
Session state is UP and using echo function with 500 ms interval.
Session Host: Software
Handle: 1
Local Diag: 0, Demand mode: 0, Poll bit: 0
MinTxInt: 1000000, MinRxInt: 1000000, Multiplier: 3
Received MinRxInt: 1000000, Received Multiplier: 3
Holddown (hits): 0(0), Hello (hits): 1000(30)
Rx Count: 31, Rx Interval (ms) min/max/avg: 1/1000/857 last: 681 ms ago
Tx Count: 32, Tx Interval (ms) min/max/avg: 1/999/840 last: 363 ms ago
Elapsed time watermarks: 0 0 (last: 0)
Registered protocols: OSPF 
Uptime: 00:00:26
Last packet: Version: 1                  - Diagnostic: 0
             State bit: Up               - Demand bit: 0
             Poll bit: 0                 - Final bit: 0
             C bit: 0                                   
             Multiplier: 3               - Length: 24
             My Discr.: 1                - Your Discr.: 1
             Min tx interval: 1000000    - Min rx interval: 1000000
             Min Echo interval: 500000  

Or on ASR9K or CRS plateform :

show bfd session 

Hope this help,