Torrus SNMP Discovery User Guide

Introduction

In many (but not only) cases Torrus is used for SNMP monitoring. It provides powerful tools which automate the process of devices' MIB discovery.

The discovery tools consist of two programs: torrus devdiscover performs the SNMP discovery, based on the discovery instructions XML file. The result of its work is a new Torrus datasource configuration file. Another utility, torrus genddx, is a program that generates the basic discovery instructions file based on a set of commandline options.

The device discovery XML, or as we call them DDX files, are usually resided in /etc/torrus/conf/discovery/ directory. This is the default path to search for them when the absolute path is not given.

torrus genddx: Discovery instructions generator

  Usage: torrus genddx options...
  Options:
     --host=hostname         router hostname
     --hostfile=filename     space-separated router hostnames file
     --out=outfile           output file.         [routers.ddx]
     --discout=filename      discovery output file [routers.xml]
     --domain=domain         optional DNS domain name
     --version=v             SNMP version         [2c]
     --community=string      SNMP read community  [public]
     --port=number           SNMP port            [161]
     --retries=number        SNMP retries         [2]
     --timeout=number        SNMP timeout         [10]
     --subtree=string        Subtree name         [/Routers]
     --datadir=path          data-dir parameter [/var/lib/torrus/collector_rrd]
     --holtwinters           Enable Holt-Winters analysis

This utility generates devdiscover instructions XML file (DDX) based on commandline options and a plain list of SNMP agents' hostnames. Hostnames are specified with one or many --host=hostname options, or a plain text file with space-separated hostnames.

Each host may have a symbolic name. This symbolic name is used as the host-level subtree name. The symbolic name follows the hostname with a semicolon.

By default, the devices are placed into /Routers/ subtree hierarchy. You may change the subtree name with the --subtree option. Single slash as subtree name would cause host-level subtrees placed at the top of the datasources tree.

By default, genddx specifies the discovery output file as <routers.xml>, and it would be placed in site XML configuration directory. You most probably will change this by using the --discout option.

Examples:

  torrus genddx --out=ch-langenthal.ddx \
    --discout=ch-langenthal.xml \
    --host=192.168.34.35:Langenthal_PE1 \
    --host=192.168.34.36:Langenthal_PE2 \
    --host=192.168.34.37:Langenthal_CE_Stadtverwaltung \
    --community=blahBlah \
    --subtree=/MPLS/CH/Bern/Langenthal

  torrus devdiscover --in=ch-langenthal.ddx

Note: genddx is designed as a one-time utility. You may run it to create a typical discovery file with basic set of options. Then the discovery configuration XML would be the primary source of information, and it should be maintained by manual editing, or by some other automated means. Alternatively you may use ttproclist utility and generate highly customized discovery instruction files based on static templates and lists of SNMP nodes. See torrus_ttproclist(8) manpage for more details and examples.

torrus devdiscover: SNMP discovery tool

  Usage: torrus devdiscover --in=filename.ddx options...
  Options:
   --in=filename.ddx       discovery instructions XML file
   --mkdir                 create data-dir directories
   --limit=regexp          limit the discovery by output files
   --verbose               print extra information
   --debug                 print debugging information
   --snmpdebug             print SNMP protocol details

This utility performs the SNMP discovery of devices listed in the input DDX file. The output XML file is the Torrus datasources configuration. Output file name is taken from output-file parameter in the DDX. If the output file is not an absolute path, the file is placed in the site XML configuration directory (/etc/torrus/xmlconfig).

devdiscover is accompanied by a number of MIB- or vendor-specific modules, each responsible for finding specific SNMP variables in the SNMP device, and for generating a piece of Torrus configuration XML file responsible for that specific MIB. The list of supported generic MIBs and vendors is constantly growing. It is quite easy to create new discovery modules, and the internals are documented in Torrus SNMP Device Discovery Developer's Guide.

The output file automatically includes all required prerequisite generic and vendor template definition files.

Behaviour of devdiscover is controlled by variables in /etc/torrus/conf/devdiscover-siteconfig.pl file. Default values for those variables reside in /usr/share/torrus/conf_defaults/devdiscover-config.pl.

For large installations, it is recommended to place RRD files into a hashed directory structure. You can enable this feature by setting

  $Torrus::DevDiscover::hashDataDirEnabled = 1;

in your devdiscover-siteconfig.pl file. Then launching devdiscover with --mkdir option would automatically create the subdirectories inside data-dir.

The XML files produced by devdiscover may be automatically changed with some local site-specific scripts. See XUpdate usage example in Torrus User Guide.

DDX, the discovery instructions file

The input file for devdiscover is an XML file. Its DTD is available in Torrus distribution in snmp-discovery.dtd.

A typical place to store the discovery XML is /etc/torrus/conf/discovery/.

Example:

    <?xml version="1.0" encoding="UTF8"?>
    <snmp-discovery>
      <file-info>
        <format-version>
    1.0
        </format-version>
      </file-info>
      <creator-info>
    Torrus version 0.1.4d
    This file was generated by command:
    /usr/local/torrus-0.1/bin/genddx \
     --discout=share/torrus/xmlconfig/myrouters.xml \
     --community=blahblah  --host=10.0.0.1  --host=10.0.1.1
    On Tue Dec  2 17:43:30 2003
      </creator-info>
      <param name="data-dir" value="/var/lib/torrus/collector_rrd"/>
      <param name="domain-name" value=""/>
      <param name="host-subtree" value="/Routers"/>
      <param name="output-file" value="myrouters.xml"/>
      <param name="rrd-hwpredict" value="no"/>
      <param name="snmp-community" value="blahblah"/>
      <param name="snmp-port" value="161"/>
      <param name="snmp-retries" value="2"/>
      <param name="snmp-timeout" value="10"/>
      <param name="snmp-version" value="2c"/>
      <host>
        <param name="snmp-host" value="10.0.0.1"/>
        <param name="symbolic-name" value="10.0.0.1"/>
      </host>
      <host>
        <param name="snmp-host" value="10.0.1.1"/>
        <param name="symbolic-name" value="10.0.1.1"/>
      </host>
    </snmp-discovery>

XML elements

• snmp-discovery
  
Mandatory. The top-level element.
• file-info
  
Mandatory. It must contain the element format-version.
• format-version
  
Mandatory. It must contain a format version currently supported by devdiscover. Currently supported version is 1.0.
• creator-info
  
Optional. May contain the information about the file creator: a human author name, or a program.
• param
  
Some parameters are mandatory. This element defines a global or host-specific parameter. Mandatory attribute name identifies the parameter, and the value is taken eother from value attribute, or from the textual content of the element.

param element should be the child element of snmp-discovery for global parameters or host for host-level parameters. Host-level parameters override the values of global parameters.

These parameters are for devdiscover only. They are not Torrus configuration parameters, although some of them are directly copied into the generated configuration file.

• host
  
Mandatory. Defines a host to run SNMP discovery.

Common parameters

• collector-period, collector-timeoffset, collector-dispersed-timeoffset, collector-timeoffset-min, collector-timeoffset-max, collector-timeoffset-step, monitor-period, monitor-timeoffset
  
Optional. When defined, these parameters override those from snmp-defaults template in snmp-defs.xml.
• output-file
  
Mandatory. Specifies the output filename for generated Torrus configuration. If it's a relative path, the file is placed in /etc/torrus/xmlconfig directory.
• output-bundle
  
Optional. Specifies a comma-separated list of XML file names for bundle output. Each bundle file will contain <include> statement for each corresponding output-file.
• snmp-port, snmp-community, snmp-version, snmp-timeout, snmp-retries, snmp-host
  
Mandatory. These parameters define the SNMP agent properties and are copied one-to-one to the output configuration.
• domain-name
  
Optional. Defines a DNS domain name to be appended to snmp-host.
• data-dir
  
Mandatory. Defines the directory path where RRD files for this host are stored. In case if hashed directories are enabled, data-dir specifies the base path under which the hashed subdirectories are created.
• symbolic-name
  
Optional. Determines the host-specific subtree name. If not specified, the subtree is named by system-id, or by snmp-host if system ID is not defined.
• system-id
  
Optional. If defined, it overrides the default value of system ID, which is equal to the value of snmp-host.
• snmp-oids-per-pdu
  
Optional. When defined, these parameters overwrites the value from the template and vendor-specific discovery modules.
• snmp-check-sysuptime
  
Optional. Default: yes. Devdiscover sets this parameter to no when it finds SNMPv2-MIB::sysUpTime variable unavailable in the device.
• host-subtree
  
Optional. Defines the datasourse tree path under which the host-specific subtree is created.
• rrd-hwpredict
  
Optional. Valid values: yes, no. Determines if Holt-Winters forecasting should be enabled for the given host.
• disable-devtypes
  
Optional. Comma-separated list of discovery device types that need to be excluded from discovery process. For example, ATMEL appliances conflict with Empire SystemEdge MIBs, and you need to disable EmpireSystemedge module in order to run the discovery on those appliances.
• only-devtypes
  
Optional comma-separated list of device types. If defined, only the specified types will be used for device probing.
• host-aliases
  
Optional comma-separated list of alias paths for a given host. Aliases must be unique for each host.
• custom-host-templates
  
Optional comma-separated list of template names that will be applied to the resulting XML configuration at the host level.
• host-copy-params
  
Optional comma-separated list of parameter names that should be copied from the discovery configuration file to the generated Torrus configuration at the host level.
• selectors
  
Optional comma-separated list of object selectors. They are explained in more detail below.
• disable-snmpcollector
  
When set to yes, this parameter disables SNMP collection for this host. It is useful for creating custom views, in conjunction with RFC2863_IF_MIB::only-interfaces parameter.

Parameters for RFC2863_IF_MIB

This discovery module is responsible for agent's interfaces table and interface counters. Recognized optional parameters are:

• RFC2863_IF_MIB::suppress-hc-counters
  
Some agents do not implement 64-bit counters correctly. When this parameter is set to yes, 64-bit interface counters are not used for the host. For Cisco IOS devices, the CiscoIOS discovery module tries to detect such situation automatically.
• RFC2863_IF_MIB::subtree-name
  
Defines the child subtree name within host-level subtree where interface counters are located. Default: Interface_Counters.
• RFC2863_IF_MIB::exclude-interfaces
  
Comma-separated list of interface names which should be excluded from configuration. Spaces are allowed between the names and commas. The names should be the ones that are used for interface subtrees.
• RFC2863_IF_MIB::only-interfaces
  
If defined, this parameter specifies the list of interfaces that will be included in the discovery results. The names should match the interface subtree names. Caution: if specified incorrectly, this parameter may disable discovery for all interfaces on a device.
• RFC2863_IF_MIB::tokenset-members
  
This parameter defines which interfaces' InOut_bps leaves to add to which tokensets. The value is a semicolon-separated (;) list of entries of form tset:interface,interface, where tset is a name of the tokenset, and the interface is the subtree name of the corresponding interface. The token sets must be defined elsewhere in Torrus configuration. Example:

  <host>
    <param name="snmp-host" value="192.168.49.131"/>
    <param name="RFC2863_IF_MIB::tokenset-members">
     clusters: 10_1, 10_2;
     uplinks: 1_1, 1_2
    </param>
  </host>

• RFC2863_IF_MIB::errors-monitor
  
When set to yes, all interface error and discard counters are assigned to a monitor called RFC2863_IF_MIB-errors. It is up to you to define the monitor somewhere in the configuration.
• RFC2863_IF_MIB::copy-params
  
Optional comma-separated list of parameter names that would be copied to the output Torrus configuration at interface level. For each parameter param, the value for a particular interface intf will be taken from parameter RFC2863_IF_MIB::param::intf in the discovery configuration file. Example:

  <host>
    <param name="snmp-host" value="192.168.49.131"/>
    <param name="RFC2863_IF_MIB::copy-params" value="intf-error-threshold"/>
    <param name="RFC2863_IF_MIB::intf-error-threshold::10_1" value="300"/>
    <param name="RFC2863_IF_MIB::intf-error-threshold::2_1" value="900"/>
  </host>

• RFC2863_IF_MIB::ifindex-map-hint
  
Optional. For a device that doesn't have a corresponding vendor-specific discovery module, this parameter would advice devdiscover how to build ifTable index mapping. By default, the interfaces are mapped by ifDescr variables. For many vendors, this would not give reliable mapping. Valid values are: ifName, ifPhysAddress, and ifIndex. The first two suggest the mapping by corresponding SNMP variables, and the third one should be used for devices with fixed ifIndex layout: in this case, the ifIndex values are recorded as they are at the moment of discovery. This option is ignored if the device is supported by a vendor specific discovery module.
• RFC2863_IF_MIB::subtree-name-hint
  
Optional. By default, per-interface subtrees are named after the values of ifDescr SNMP variables. If this option is set to ifName, this SNMP variable would be used for subtree naming. This option is ignored if the device is supported by a vendor specific discovery module.

Other generic MIB parameters

• RFC2790_HOST_RESOURCES::sysperf-subtree-name
  
Defines the child subtree name within host-level subtree where system CPU and memory statistics are located. Default: System_Performance.

Vendor parameters

• CiscoGeneric::disable-cpu-stats
  
When set to yes, Cisco CPU usage statistics are excluded from discovery.
• CiscoGeneric::disable-memory-pools
  
When set to yes, Cisco memory pool usage statistics are excluded from discovery. For Cisco series 7500, see the note in the Torrus Vendor Support List.
• CiscoGeneric::disable-sensors
  
When set to yes, Cisco temperature sensors are excluded from discovery.
• CiscoGeneric::use-fahrenheit
  
When set to yes, Cisco temperature sensors are recorded and displayed in degrees Fahrenheit.
• CiscoGeneric::file-per-sensor
  
When set to yes, Cisco temperature sensor values are stored in a separate RRD file per sensor ID. This is useful for modular big routers like Cisco GSR.
• CiscoGeneric::sensor-monitor
  
Comma-separated list of monitor names to be applied to the sensor datasource.
Regular expression that should match the sensor descriptions that would have the CiscoGeneric::sensor-monitor applied. Default: .*.
• CiscoIOS::disable-membuf-stats
  
When set to yes, Cisco memory buffer statistics are excluded from discovery.
• CiscoIOS::disable-ipsec-stats
  
When set to yes, IPSec statistics are excluded from discovery.
• CiscoCatOS::suppress-noname-ports
  
When set to yes, only those Catalyst ports are included in configuration which have a port name (set up by set port name command).
• CiscoIOS_MacAccounting::bgponly
  
When set to yes, Cisco MAC accounting statistics will be enabled for those MACs which correspond to BGP peers only.
• bgp-as-description-NNNN
  
Specifies a description for an autonomous system number. Currently it's used with Cisco MAC accounting only.
• peer-ipaddr-description-AAA_AAA_AAA_AAA
  
Specifies a description for a peer by its IP address. Dots in IP address should be replaced with underscores. Currently it's used with Cisco MAC accounting only.
• Paradyne::slot-name
  
Mandatory for Paradyne DSLAM devices. It should uniquely identify the slot within the device. Any sequence of word characters is allowed.

See also: Torrus Vendor Support List for the list of supported MIBs and vendors.

Object selectors

Selectors are a common mechanism for applying customizations to some discovery objects. For example, you may want to apply a monitor to byte counters of interfaces that have a special word in the description. Or apply Holt-Winters prediction to a certain subset of interfaces.

Selectors are defined in a DDX file as regular parameters. The parameter selectors defines a list of selector names. Each selector is identifed by its name and type. The type of selector defines which objects will be searched: IF-MIB interfaces, or Cisco temperature sensors, or something else. The parameters are described below.

• selectors
  
Comma-separated list of selector names. Further below we use S as selector name, A as attribute name, and T as action name.
• S-selector-type
  
Type of objects to be selected. Supported values are: RFC2863_IF_MIB, CiscoCPU, CiscoSensor.
• S-selector-expr
  
An RPN (reverse Polish notation) expression that defines the object attributes to be checked and relation between them. Attribute names should be specified in curly braces ({}). The rest of the syntax is described in RPN expressions in Torrus guide. The attribute names are specific to the selector type and are described below.
• S-A
  
For a given selector name S and attribute name A, this parameter defines the attribute value that should be compared to the object's properties. In most cases it defines a regular expression that should match the value of a corresponding object's property.
• S-selector-actions
  
For a given selector name S, this parameter defines a comma-separated list of actions to be applied for objects where S-selector-expr returns true. The action names are specific to the selector type and are described below.
• S-T-arg
  
For a given selector S and action T, this parameter defines an argument that should be passed to the action. Each action defines its own meaning of the argument.

RFC2863_IF_MIB selector

This type of selector selects the network interfaces on a SNMP device. The following attribute names are supported:

• ifSubtreeName
  
Interface-level subtree name as you see it in the discovery results.
• ifComment
  
Comment string that is defined in comment parameter. It is usually derived from an interface description.
• ifType
  
Numeric IANA interface type, as returned by ifType SNMP variable.

ifType is compared numerically. ifSubtreeName and ifComment are regular expressions.

The following actions are supported:

• InBytesMonitor, OutBytesMonitor
  
The argument defines the monitor name to be applied to the input or output bytes counter.
• InErrorsMonitor, OutErrorsMonitor
  
The argument defines the monitor name to be applied to error and discard counters that are discovered for a given interface.
• HoltWinters
  
No argument needed. This action enables Holt-Winters prediction for given interface's counters.
• NoPacketCounters, NoErrorCounters
  
No argument needed. The action disables the packet or error counters for a given interface to be collected.
• TokensetMember
  
The argument is a comma-separated list of tokenset names where InOutBbs graphs would be added. This action complements RFC2863_IF_MIB::tokenset-members.
• Parameters
  
The argument defines additional configuration parameters to be placed at the interface level. The value should be one or several param=value pairs separated by semicolon (;). Parameters defined in RFC2863_IF_MIB::copy-params may override the ones defined in this action.
• DocsisUpSNRMonitor
  
For a DOCSIS CMTS, this action assigns a new monitor to the upstream Signal/Noise ratio gauge, instead of the default monitor.
• DocsisUpSNRTokenset
  
For a DOCSIS CMTS, this action adds the Signal/Noise ratio graph to a specified tokenset.
• DocsisUpFECCorMonitor
  
For a DOCSIS CMTS, this action assigns a monitor to the Correctable counter of the upstream FEC statistics.
• DocsisUpFECUncorMonitor
  
For a DOCSIS CMTS, this action assigns a monitor to the Uncorrectable counter of the upstream FEC statistics.
• DocsisDownUtilMonitor
  
For a DOCSIS CMTS, this action assigns a monitor to the downstream UsedBytes counter.
• DocsisMacModemsMonitor
  
For Cisco uBR, this action assigns a monitor to the Modems_Registered gauge in the MAC layer stats.
• DocsisUpUtilMonitor, DocsisUpSlotsMonitor
  
For Cisco uBR, this action assigns a monitor to upstream utilization and free contention timeslots gauges correspondingly.

CiscoCPU selector

A selector of this type selects CPU statistics Cisco router or switch.

The attributes supported are: CPUName and CPUDescr, and their values are regular expressions that should match the CPU name or description, as discovered by devdiscover.

The action supported is TokensetMember, and its argument specifies the tokenset where to place the CPU statistics graph.

CiscoSensor selector

A selector of this type selects temperature sensors on a Cisco router or switch.

The only attribute supported is SensorDescr, and its value is a regular expression that should match the sensor description, as discovered by devdiscover.

Actions supported: Monitor, TokensetMember.

Examples

The following example applies a monitor called temp60degrees to all inlet sensors on a Cisco device:

  <host>
    <param name="snmp-host" value="router1"/>
    <param name="output-file" value="router1.xml"/>
    <param name="selectors" value="inlet"/>
    <param name="inlet-selector-type" value="CiscoSensor"/>
    <param name="inlet-selector-expr"  value="{SensorDescr}"/>
    <param name="inlet-SensorDescr" value="Inlet"/>
    <param name="inlet-selector-actions"  value="Monitor"/>
    <param name="inlet-Monitor-arg" value="temp60degrees"/>
  </host>

The following example enables Holt-Winters prediction and specifies some parameters to all FastEthernet interfaces which have the string ``SRV-ID'' in their descriptions:

  <host>
    <param name="snmp-host" value="router2"/>

    <param name="selectors"  value="FastEthHW"/>
    <param name="FastEthHW-selector-type"     value="RFC2863_IF_MIB"/>

    <param name="FastEthHW-selector-expr"    
           value="{ifSubtreeName},{ifComment},AND"/>
    <param name="FastEthHW-ifSubtreeName"     value="^FastEthernet"/>
    <param name="FastEthHW-ifComment"         value="SRV-ID"/>

    <param name="FastEthHW-selector-actions"
           value="HoltWinters,Parameters"/>

    <param name="FastEthHW-Parameters-arg"
       value="rrd-create-hw-alpha=0.2; rrd-create-hw-beta=0.01"/>
  </host>

The following example sets up the monitors defined in examples/docsis-monitors.xml:

    <param name="selectors" value="docs"/>
    <param name="docs-selector-type" value="RFC2863_IF_MIB"/>
    <param name="docs-selector-expr"  value="{ifSubtreeName}"/>
    <param name="docs-ifSubtreeName"  value="^Cable"/>
    <param name="docs-selector-actions"> 
          DocsisUpSNRMonitor,
          DocsisUpFECCorMonitor,
          DocsisUpFECUncorMonitor,
          DocsisDownUtilMonitor,
          DocsisMacModemsMonitor,
          DocsisUpUtilMonitor,
          DocsisUpSlotsMonitor,
          InErrorsMonitor,
          OutErrorsMonitor
    </param>
    <param name="docs-DocsisUpSNRMonitor-arg" 
           value="docsis-snr-1,docsis-snr-2,docsis-snr-3"/>

    <param name="docs-DocsisUpFECCorMonitor-arg" 
           value="docsis-feccor-1,docsis-feccor-2"/>

    <param name="docs-DocsisUpFECUncorMonitor-arg" 
           value="docsis-fecuncor-1,docsis-fecuncor-2,docsis-fecuncor-3"/>

    <param name="docs-DocsisDownUtilMonitor-arg" 
           value="docsis-downutl-1,docsis-downutl-2,docsis-downutl-3"/>

    <param name="docs-DocsisMacModemsMonitor-arg" 
           value="docsis-modems-1,docsis-modems-2"/>

    <param name="docs-DocsisUpUtilMonitor-arg" 
           value="docsis-uputil-1,docsis-uputil-2,docsis-uputil-3"/>
    
    <param name="docs-DocsisUpSlotsMonitor-arg" 
           value="docsis-upslots-1,docsis-upslots-2,docsis-upslots-3"/>
    
    <param name="docs-InErrorsMonitor-arg" 
           value="docs-inerrors-1,docs-inerrors-2"/>

    <param name="docs-OutErrorsMonitor-arg" 
           value="docs-outerrors-1,docs-outerrors-2"/>

Author

Copyright (c) 2002-2005 Stanislav Sinyagin <ssinyagin@yahoo.com>