Parser for MultiHaul TG Radios

The output of a TG ‘show’ command takes some effort to parse and tokenise. The module parse_show_tg.py is dedicated to doing just this. The main API is class SikShowTg. For further information, refer to Implementation Details.

Usage Example

Trivial example

>>> from batchscanner.sikcli import SikCli
>>> cli = SikCli('192.168.0.1', username='admin', password='admin')
>>> show_dump = cli.send('show')

>>> from batchscanner.parsers.parse_show_tg import SikShowTg
>>> tokens = SikShowTg(show_dump, 'TG @192.168.0.1')
>>> tokens
Interfaces:
name      port    status    dup    speed
dn1       eth1    up        FD     1Gbps
dn1       eth2    down
dn1       eth3    down

IP:
name      address        pref    vlan    gateway
dn1       192.168.0.1    24              192.168.0.100

Inventory:
name      sn          model                hw_rev    sw_ver
dn1       FB18483274  MH-N366-CCP-PoE-MWB  A1        1.3.1-2436

Node:
name      popdn    sync               mode    sched    ptx    freq    pol    gol
dn1       False    gps-sync-holdover  BU      Long     auto   60480   even   1|1

Sectors:
name      sec    admin    cfg_f    pol    gol    act_f       sync      Tmdm    Trf
dn1       1      up       60480    even   1|1    60480 (ok)  internal  50      45
dn1       2      up       64800    even   1|1    64800 (ok)  internal  49      50
dn1       3      up       60480    even   1|1    60480 (ok)  internal  50      49
dn1       4      up       64800    even   1|1    64800 (ok)  internal  50      52

Links:
name      remote    admin    role    status        uptime         type    cfg_lsec    cfg_rsec    act_lsec    act_rsec    rssi    snr    mcs_tx    mcs_rx
dn1       cn1       up       init    up            00051:29:54:34 cn      3           1           3           1           -65     11     9         10

System:
name      product    uptime          datetime             location    sw_active    sw_passive    gps_mode    gps_sats
dn1       MH-N366    00055:07:55:38  2023-08-17 06:30:32  London      1.3.1        2.1.2         3D          11

SikShowTg Class Information

class batchscanner.parsers.parse_show_tg.SikShowTg(show_dump, identifier, silent=True)

A class to parse the output of a TG ‘show’ dump, and represent the derived parameters (tokens).

__init__(show_dump, identifier, silent=True)

Parses a show_dump: a text dump of a TG ‘show’ command.

Parameters:

show_dump (str) – a text dump of a TG ‘show’ command
identifier (str) – an arbitrary label (typically some unique identifier) used in parsing error messages.
silent (bool) – If True, do not print out any errors

If parsing is successful, attributes (listed below) provide the derived parameters (tokens). Each attribute is an instance of SikShowTgSection, which is a container for one or more identical atom classes. Each of these atom classes is a subclass of SikShowTgAtom.

interfaces: Tokens from the interfaces section of ‘show’

inventory: Tokens from the inventory section of ‘show’

ip: Tokens from the ip section of ‘show’

links: Tokens from both the radio-common->links and radio-common->links sections of ‘show’

name: The radio’s name (str)

node: Tokens from both the radio-common->node_config and radio-common->node_config sections of ‘show’

sectors: Tokens from both the radio-common->sectors_config and radio-common->sectors_config’ sections of ‘show’

system: Tokens from the system section of ‘show’

Implementation Details

Parsing

Parsing the output of a TG ‘show’ command is implemented in method _deyamlify(). In a nutshell, it converts the output into a standard YAML, and then uses a standard YAML parser to convert int a a list of dictionaries.

_deyamlify Class Information

SikShowTg._deyamlify()

This function converts the output of a TG ‘show’ command to standard YAML. It then uses a standard YAML library to render the converted output as a list of dictionaries. Each dictionary is single-key, and the key equals one of the main sections of the ‘show’ output: interfaces, inventory, system, …

There is no need to explicitly call this function, as it is automatically called when instantiating class :class:SikShowTg.

Conversion to Standard YAML

The output of a TG ‘show’ command is quasi-YAML. The following substitutions are made sequentially, in order to convert to standard YAML:

Change	Into	Comment
‘: ‘	‘:’	Resolves cases where the string ‘: ‘ appears as part of a description.
‘key value;’	‘key: value’	Simple mapping (key, value)
‘key;’	‘key:’	A mapping…
‘name {’	‘name:’	Opening of section
‘}’	‘’	Closing of section

Example:

**Original**                                 **Converted**
ip {                                         - ip:
  ipv4 {                                        - ipv4:
    address {                                     - address:
      ip 172.19.40.10;                               - ip: 172.19.40.10
      prefix-length 24;                              - prefix-length: 24
      c-vlan 20;                                     - c-vlan: 20
    }
    address {                                     - address:
      ip 192.168.0.100;                              - ip: 192.168.0.100
      prefix-length 24;                              - prefix-length: 24
    }
    default-gateway 172.19.40.1;                  - default-gateway: 172.19.40.1
  }
  ipv6 {                                        - ipv6:
    link-local fe80::6:7bff:fe2d:3790;            - link-local: fe80::6:7bff:fe2d:3790
   }
}

Parsing the Standard YAML

The converted output (standard YAML) is parsed using the PyYAML, library into Python lists and dictionaries. This typically comprises a list of dictionaries, where each dictionary has a single element. For instance, the ipv4 subsection (in the example above) is parsed into:

[{'address': [{'ip': '172.19.40.10'}, {'prefix-length': 24}, {'c-vlan': 20}]},
 {'address': [{'ip': '192.168.0.100'}, {'prefix-length': 24}]},
 {'default-gateway': '172.19.40.1'}
]

In addition, within a list, the keys of the dictionaries are not necessarily unique. In the example above, two dictionaries in the list have the same key: address. Duplicate keys are prevalent in earlier TG (embedded) software versions (up to and including 2.0.0). As of software version 2.1.1 (with the addition of SNMP support), most keys are unique. Here’s the equivalent of the above example in version 2.1.1:

[{'address 172.19.40.10': [{'prefix-length': 24}, {'c-vlan': 20}]},
 {'address 192.168.0.100': [{'prefix-length': 24}]},
 {'default-gateway': '172.19.40.1'}
]

In order to effectively parse the above type of lists of dictionaries, two helper functions are heavily used throughout the code: _gkv() and _vbkild().

Tokenising

The key attributes of class SikShowTg are each an instance of class SikShowTgSection: essentially a container for one or more atoms of the same types.

Atoms are class instances for storing tokensied data for specific section of the output of the ‘show’ command. For example, SikShowTgAtomLink contains tokens pertaining to the ‘radio-dn links’ and ‘radio-common links’ sections of the ‘show’ output. There may be multiple instances of SikShowTgAtomLink (grouped in container SikShowTgSection, each corresponding to a different configured link).

SikShowTgSection Class Information

class batchscanner.parsers.parse_show_tg.SikShowTgSection(identifier, name, section='')

A Container for atoms: the subclasses of TgAtoms. This container is implemented as a abc.MutableSequence. Essentially a list with some added functionality.

__init__(identifier, name, section='')

Initialises an instance of the class.

Parameters:

name (str) – an arbitrary identifier, typically the name of the radio
section (str) – an arbitrary identifier, typically the name of the section within the ‘show’ dump

tocsv()

Dumps container (instance of SikShowTgSection) as csv

Returns:

Returns a tuple of 2 strings:

Comma-separated field names
Lines of comma-separated values, where each line corresponds to one atom in the container, and lines are separated with a newline character.

Return type:

tuple(str, str)

Atoms

The different types of atoms are subclasses of TgShowAtom, which provide convenient functions, such as converting an atom to csv.

The following table shows the correspondence between the attributes of SikShowTg, and the type of atoms they contain.

Atom classes within the main `SikShowTg` Attributes
SikShowTg Attribute	Atoms contained within the SikShowTgSection
`interfaces`	`SikShowTgAtomInterface`
`inventory`	`SikShowTgAtomInventory`
`ip`	`SikShowTgAtomIp`
`links`	`SikShowTgAtomLink`
`node`	`SikShowTgAtomNode`
`sectors`	`SikShowTgAtomSector`
`system`	`SikShowTgAtomSystem`

SikShowTgAtom* Classes Information

A parser for the output of a MultiHaul TG ‘show’

class batchscanner.parsers.parse_show_tg.SikShowTgAtom

A base class for atoms: classes representing tokens extracted from the TG ‘show’ dump.

tokens = {}: A dictionary mapping names of attributes (as used in code) to labels (as shown to user). Assigned in each subclass.

tocsv(): Docstring here

todict(): Docstring here

class batchscanner.parsers.parse_show_tg.SikShowTgAtomSystem(system): Tokens derived from the system section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomInterface(interface): Tokens derived from the interfaces section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomInventory(interface): Tokens derived from the inventory section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomIp(ip_data): Tokens derived from the ip section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomNode(common, dn): Tokens derived from the radio-common->node-config and radio-dn->node-config sections of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomSectorCommon(common): Tokens derived from the radio-common->sector-config section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomSectorDn(dn, node): Tokens derived from the radio-dn->sector-config section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomSector(common, dn): Tokens derived by merging tokens from SikShowTgAtomSectorDn and SikShowTgAtomSectorCommon.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomLinkCommon(common): Tokens derived from the radio-common->links section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomLinkDn(dn): Tokens derived from the radio-dn->sector-config section of the ‘show’ dump.

class batchscanner.parsers.parse_show_tg.SikShowTgAtomLink(common, dn): Tokens derived by merging tokens from SikShowTgAtomLinkDn and SikShowTgAtomLinkCommon.

Helper Functions:

batchscanner.parsers.parse_show_tg._gkv(in_dict)

gkv is an acronym for Get Key and Value (from dictionary).

Return the key and value an item in a dictionary (with a single item). If the dictionary has multiple items, the key and value of the first item are returned. If the dictionary is empty (or in_dict is not a dictionary), returns a tuple of empty strings: (‘’, ‘’).

Parameters:: in_dict (dict) – A dictionary with a single element: {key, value}
Returns:: (key, value) of the first item in in_dict
Return type:: tuple(str, Any)

batchscanner.parsers.parse_show_tg._vbkild(list_of_dicts, key_id, default=None)

vbkild is an acronym for get Value By Key In List of Dictionaries, where the dictionaries are assumed to have a single item.

Given a list of single-element dictionaries {key: value}, return value of the first dictionary where key contains key_id. If list_of_dicts is not a list of dictionaries, or if no dictionary with the right key is found: return default.

Parameters:

list_of_dicts (list[dict]) – A list of single-element dictionaries
key_id (str) – A substring for matching the key of each dictionary.
default (Any) – value to return if no dictionary with the right key is found.

Returns:

The value of the skdict {key:value} whose key contains key_id

Return type:

Any