# Copyright (C) 2002-2009 Stanislav Sinyagin # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA. # $Id: snmpdiscovery.pod.in,v 1.1 2010-12-27 00:04:35 ivan Exp $ # Stanislav Sinyagin # # =head1 Torrus SNMP Discovery User Guide =head2 Introduction In many (but not only) cases Torrus is used for SNMP monitoring. It provides powerful tools which automate the process of devices' MIB discovery. The discovery tools consist of two programs: C performs the SNMP discovery, based on the discovery instructions XML file. The result of its work is a new Torrus datasource configuration file. Another utility, C, is a program that generates the basic discovery instructions file based on a set of commandline options. The device discovery XML, or as we call them DDX files, are usually resided in F<@siteconfdir@/discovery/> directory. This is the default path to search for them when the absolute path is not given. =head2 C: Discovery instructions generator Usage: torrus genddx options... Options: --host=hostname router hostname --hostfile=filename space-separated router hostnames file --out=outfile output file. [routers.ddx] --discout=filename discovery output file [routers.xml] --domain=domain optional DNS domain name --version=v SNMP version [2c] --community=string SNMP read community [public] --port=number SNMP port [161] --retries=number SNMP retries [2] --timeout=number SNMP timeout [10] --subtree=string Subtree name [/Routers] --datadir=path data-dir parameter [@defrrddir@] --holtwinters Enable Holt-Winters analysis This utility generates C instructions XML file (DDX) based on commandline options and a plain list of SNMP agents' hostnames. Hostnames are specified with one or many C<--host=hostname> options, or a plain text file with space-separated hostnames. Each host may have a symbolic name. This symbolic name is used as the host-level subtree name. The symbolic name follows the hostname with a semicolon. By default, the devices are placed into C subtree hierarchy. You may change the subtree name with the I<--subtree> option. Single slash as subtree name would cause host-level subtrees placed at the top of the datasources tree. By default, C specifies the discovery output file as , and it would be placed in site XML configuration directory. You most probably will change this by using the C<--discout> option. Examples: torrus genddx --out=ch-langenthal.ddx \ --discout=ch-langenthal.xml \ --host=192.168.34.35:Langenthal_PE1 \ --host=192.168.34.36:Langenthal_PE2 \ --host=192.168.34.37:Langenthal_CE_Stadtverwaltung \ --community=blahBlah \ --subtree=/MPLS/CH/Bern/Langenthal torrus devdiscover --in=ch-langenthal.ddx B C is designed as a one-time utility. You may run it to create a typical discovery file with basic set of options. Then the discovery configuration XML would be the primary source of information, and it should be maintained by manual editing, or by some other automated means. Alternatively you may use C utility and generate highly customized discovery instruction files based on static templates and lists of SNMP nodes. See L manpage for more details and examples. =head2 C: SNMP discovery tool Usage: torrus devdiscover --in=filename.ddx options... [ddx files] Options: --in=filename.ddx discovery instructions XML file(s) --mkdir create data-dir directories --limit=regexp limit the discovery by output files --forcebundle always write the bundle file --fallback=integer maximum age of XML file to fall back to --threads=integer number of parallel discovery threads --verbose print extra information --debug print debugging information --snmpdebug print SNMP protocol details This utility performs the SNMP discovery of devices listed in the input DDX file. The output XML file is the Torrus datasources configuration. Output file name is taken from C parameter in the DDX. If the output file is not an absolute path, the file is placed in the site XML configuration directory (F<@sitexmldir@>). C is accompanied by a number of MIB- or vendor-specific modules, each responsible for finding specific SNMP variables in the SNMP device, and for generating a piece of Torrus configuration XML file responsible for that specific MIB. The list of supported generic MIBs and vendors is constantly growing. It is quite easy to create new discovery modules, and the internals are documented in I. The output file automatically includes all required prerequisite generic and vendor template definition files. Behaviour of C is controlled by variables in F<@siteconfdir@/devdiscover-siteconfig.pl> file. Default values for those variables reside in F<@cfgdefdir@/devdiscover-config.pl>. For large installations, it is recommended to place RRD files into a hashed directory structure. You can enable this feature by setting $Torrus::DevDiscover::hashDataDirEnabled = 1; in your F file. Then launching C with C<--mkdir> option would automatically create the subdirectories inside C. The XML files produced by C may be automatically changed with some local site-specific scripts. See XUpdate usage example in I. =head2 DDX, the discovery instructions file The input file for C is an XML file. Its DTD is available in Torrus distribution in F. A typical place to store the discovery XML is F<@siteconfdir@/discovery/>. Example: 1.0 Torrus version 0.1.4d This file was generated by command: /usr/local/torrus-0.1/bin/genddx \ --discout=share/torrus/xmlconfig/myrouters.xml \ --community=blahblah --host=10.0.0.1 --host=10.0.1.1 On Tue Dec 2 17:43:30 2003 =head3 XML elements =over 4 =item * C Mandatory. The top-level element. =item * C Mandatory. It must contain the element C. =item * C Mandatory. It must contain a format version currently supported by devdiscover. Currently supported version is C<1.0>. =item * C Optional. May contain the information about the file creator: a human author name, or a program. =item * C Some parameters are mandatory. This element defines a global or host-specific parameter. Mandatory attribute C identifies the parameter, and the value is taken eother from C attribute, or from the textual content of the element. C element should be the child element of C for global parameters or C for host-level parameters. Host-level parameters override the values of global parameters. These parameters are for C only. They are not Torrus configuration parameters, although some of them are directly copied into the generated configuration file. =item * C Mandatory. Defines a host to run SNMP discovery. =back =head3 Common parameters =over 4 =item * C, C, C, C, C, C, C, C Optional. When defined, these parameters override those from C template in F. =item * C Mandatory. Specifies the output filename for generated Torrus configuration. If it's a relative path, the file is placed in F<@sitexmldir@> directory. =item * C Optional. Specifies a comma-separated list of XML file names for bundle output. Each bundle file will contain EincludeE statement for each corresponding C. =item * C, C, C, C, C, C, C, C, C, C, C, C, C, C, C Mandatory SNMP session parameters. Authentication parameters depend on the SNMP version used. See the I for details. These parameters define the SNMP agent properties and are copied one-to-one to the output configuration. =item * C Optional. Defines a DNS domain name to be appended to C. =item * C and C Optional parameters specifying the local socket binding address and port. =item * C Mandatory. Defines the directory path where RRD files for this host are stored. In case if hashed directories are enabled, C specifies the base path under which the hashed subdirectories are created. =item * C Optional. Determines the host-specific subtree name. If not specified, the subtree is named by C, or by C if system ID is not defined. =item * C Optional. If defined, it overrides the default value of system ID, which is equal to the value of C. =item * C Optional. Defines the host-specific reference for I. Default value is equal to C. =item * C Optional. When defined, these parameters overwrites the value from the template and vendor-specific discovery modules. =item * C Optional. Default: C. Devdiscover sets this parameter to C when it finds SNMPv2-MIB::sysUpTime variable unavailable in the device. =item * C Optional. If defined, it sets the SNMP maximum message size different from default. The default value is 1470 (defined in Net::SNMP). =item * C Optional. Defines the datasourse tree path under which the host-specific subtree is created. =item * C Optional. Valid values: C, C. Determines if Holt-Winters forecasting should be enabled for the given host. =item * C Optional. Comma-separated list of discovery device types that need to be excluded from discovery process. For example, ATMEL appliances conflict with Empire SystemEdge MIBs, and you need to disable C module in order to run the discovery on those appliances. =item * C Optional comma-separated list of device types. If defined, only the specified types will be used for device probing. =item * C Optional comma-separated list of alias paths for a given host. Aliases must be unique for each host. =item * C Optional comma-separated list of template names that will be applied to the resulting XML configuration at the host level. =item * C Optional comma-separated list of XML files that will be listed in C statements in the resulting XML output. =item * C Optional comma-separated list of parameter names that should be copied from the discovery configuration file to the generated Torrus configuration at the host level. =item * C Optional comma-separated list of object selectors. They are explained in more detail below. =item * C When set to C, this parameter disables SNMP collection for this host. It is useful for creating custom views, in conjunction with C parameter. =item * C Semicolon-separated (;) list of pairs of tokenset names and descriptions. Each tokenset name is followed by colon (:) and the description text: important-graphs: Important Graphs; unimportant-graphs: Unimportant Graphs =item * C if set to C, the legend is not shown. It usually has uptime, software version, and contact information for the SNMP device. =item * C Sets the comment string for the host. =item * C Default: C. When set to C, the link to the recursive view at the host level is omitted. =item * C If defined, this should be a comma-separated list of template registry entries that override the default template references. The overlaying templates should be referred in C<%Torrus::DevDiscover::templateOverlays>. For example, we want to redefine the interface counter template. Then in the DDX file, we set Then in F we set %Torrus::DevDiscover::templateOverlays = { 'my_ifcounters' => { 'RFC2863_IF_MIB::iftable-octets' => { 'name' => 'my-iftable-octets', 'source' => 'mycustom-rfc2863.if-mib.xml' }, 'RFC2863_IF_MIB::ifxtable-hcoctets' => { 'name' => 'my-ifxtable-hcoctets', 'source' => 'mycustom-rfc2863.if-mib.xml' }, }}; =back =head3 Parameters for C This discovery module is responsible for agent's interfaces table and interface counters. Recognized optional parameters are: =over 4 =item * C Some agents do not implement 64-bit counters correctly. When this parameter is set to C, 64-bit interface counters are not used for the host. For Cisco IOS devices, the C discovery module tries to detect such situation automatically. =item * C Defines the child subtree name within host-level subtree where interface counters are located. Default: C. =item * C Defines the comment string for interface counters subtree. Default: I. =item * C If set to C, interfaces with ifAdminStatus set to other than up(1) are included in the discovery results. By default, such interfaces are not listed. =item * C If set to C, the interfaces with ifOperStatus status set to notPresent(6) are included in the discovery results. By default, such interfaces are not listed. =item * C If set to C, the interfaces with ifOperStatus equal to down(2) are excluded from the discovery results. =item * C Comma-separated list of interface names which should be excluded from configuration. Spaces are allowed between the names and commas. The names should be the ones that are used for interface subtrees. =item * C If defined, this parameter specifies the list of interfaces that will be included in the discovery results. The names should match the interface subtree names. Caution: if specified incorrectly, this parameter may disable discovery for all interfaces on a device. =item * C This parameter defines which interfaces' C leaves to add to which tokensets. The value is a semicolon-separated (;) list of entries of form I, where I is a name of the tokenset, and the I is the subtree name of the corresponding interface. The token sets must be defined elsewhere in Torrus configuration. Example: clusters: 10_1, 10_2; uplinks: 1_1, 1_2 Alternatively, this parameter can be defined at the global level, and then each interface should be pretended by the SNMP host name (the same as in C parameter) and slash sign (/): clusters: RTR01/Ethernet0_0, RTR01/FastEthernet2_1; =item * C Optional comma-separated list of parameter names that would be copied to the output Torrus configuration at interface level. For each parameter I, the value for a particular interface I will be taken from parameter C::I> in the discovery configuration file. Example: =item * C Optional. For a device that doesn't have a corresponding vendor-specific discovery module, this parameter would advice C how to build C index mapping. By default, the interfaces are mapped by C variables. For many vendors, this would not give reliable mapping. Valid values are: C, C, and C. The first two suggest the mapping by corresponding SNMP variables, and the third one should be used for devices with fixed C layout: in this case, the C values are recorded as they are at the moment of discovery. This option is ignored if the device is supported by a vendor specific discovery module. =item * C Optional. By default, per-interface subtrees are named after the values of C SNMP variables. If this option is set to C, this SNMP variable would be used for subtree naming. This option is ignored if the device is supported by a vendor specific discovery module. =item * C Optional. By default, per-interface I is derived from C. This parameter can change the reference table for nodeid. Valid values are: C, C, C, C. =item * C If set to C, all the interface statistics are skipped from the XML configuration output. Still the interface table is examined and the results may be used by other discovery modules. =item * C If set to C, the bandwidth usage percentage will be shown for those interfaces where two parameters are defined (in megabits per second): C and C. These interface parameters may be set by C discovery parameter, or by selectors with the action C, or by setting C. =item * C Defines the values (in megabits per second) for C and C interface parameters. Semicolon-separated list of (I:I:I) values, for example: ethernet0_0:10:10; ethernet0_1:20:20; =item * C, C, C, C Defines traffic summary graphs. A summary graph presents a sum of traffic from several interfaces. The interfaces can belong to different hosts, but then these hosts should have the same C parameter value. C defines a list of summary names, C specifies the full path in the datasource tree to display the graph, optional C defines a descriptive comment for the graph, and C lists the interfaces that comprise this sum. The interfaces can be defined at the host level in the form C or at the global level in the form C. Example: =back =head3 Other generic MIB parameters =over 4 =item * C Defines the child subtree name within host-level subtree where system CPU and memory statistics are located. Default: C. =item * C If set to C, only the DOCSIS upstream statistics are enabled, and downstream and MAC layer stats are skipped. =back =head3 Vendor parameters =over 4 =item * C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C, C When set to C, the corresponding statistics are included or excluded from the graphs. =item * C When set to C, the data will be written to filenames based on the name bundle instead of by the ID of the bundle. =item * C When set to C, wireless client machines are not tracked. =item * C When set to C, Cisco CPU usage statistics are excluded from discovery. =item * C When set to C, Cisco memory pool usage statistics are excluded from discovery. For Cisco series 7500, see the note in the I. =item * C When set to C, Cisco temperature sensors are excluded from discovery. =item * C When set to C, Cisco temperature sensors are recorded and displayed in degrees Fahrenheit. =item * C When set to C, Cisco temperature sensor values are stored in a separate RRD file per sensor ID. This is useful for modular big routers like Cisco GSR. =item * C if set to C, Cisco power supply statistics are not collected. =item * C Name of the power supply monitor for Cisco devices. =item * C When set to C, Cisco memory buffer statistics are included to the discovery. Many IOS releases report faulty information, thus these stats are disabled by default. =item * C When set to C, IPSec statistics are excluded from discovery. =item * C If set to C, BGP statistics are not included. =item * C When set to C, the interfaces VlanXXX are added to statistics. By default they are not included, because some hardware platforms give irrelevant counter statistics. =item * C When set to C, the interfaces "unrouted Vlan XXX" are added to statistics. By default they are not included, because some hardware platforms provide no statistics for such interfaces. =item * C When set to C, only those Catalyst ports are included in configuration which have a port name (set up by C command). =item * C When set to C, Cisco MAC accounting statistics will be enabled for those MACs which correspond to BGP peers only. =item * C Specifies a description for an autonomous system number. Currently it's used with Cisco MAC accounting and Cisco BGP statistics only. =item * C Specifies a description for a peer by its IP address. Dots in IP address should be replaced with underscores. Currently it's used with Cisco MAC accounting and Cisco BGP statistics only. =item * C Works the same way as C and assigns MAC accounting peers to tokensets. =item * C When set to C, the Committed Access Rate statistics are disabled from discovery. =item * C When set to C, the VPDN (Virtual Private Dial-up Network) statistics are disabled from discovery. =item * C If set to C, the Hw Serial and Revision strings are not shown in device comment. =item * C, C, C, C, C, C If set to C, the corresponding statistics are excluded from Cisco SCE graphs. =item * C, C, C When set to C, disables the corresponding statistics category. =item * C, C When set to C, enables the user of fahrenheit temperature, or writes multiple rrd files. =item * C, C, C, C, C, C When set to C, disables the corresponding statistics category for JunOS devices. =item * C, C, C, C When set to C, disable the corresponding statistics category for Liebert devices. =item * C When set to C, the temperature readings will be in fahrenheit. =item * C, C, C Set a numeric value for the maximum temperature, humidity, or dew point on NetBotz sensor graphs. The maximum is displayed with a dark red line. =item * C Mandatory for Paradyne DSLAM devices. It should uniquely identify the slot within the device. Any sequence of word characters is allowed. =back See also: I for the list of supported MIBs and vendors. =head2 Object selectors Selectors are a common mechanism for applying customizations to some discovery objects. For example, you may want to apply a monitor to byte counters of interfaces that have a special word in the description. Or apply Holt-Winters prediction to a certain subset of interfaces. Selectors are defined in a DDX file as regular parameters. The parameter C defines a list of selector names. Each selector is identifed by its name and type. The type of selector defines which objects will be searched: IF-MIB interfaces, or Cisco temperature sensors, or something else. The parameters are described below. =over 4 =item * C Comma-separated list of selector names. Further below we use C as selector name, C as attribute name, and C as action name. =item * C Type of objects to be selected. Supported values are: C, C, C. =item * C An RPN (reverse Polish notation) expression that defines the object attributes to be checked and relation between them. Attribute names should be specified in curly braces ({}). The rest of the syntax is described in I guide. The attribute names are specific to the selector type and are described below. =item * C For a given selector name C and attribute name C, this parameter defines the attribute value that should be compared to the object's properties. In most cases it defines a regular expression that should match the value of a corresponding object's property. =item * C For a given selector name C, this parameter defines a comma-separated list of actions to be applied for objects where C returns true. The action names are specific to the selector type and are described below. =item * C For a given selector C and action C, this parameter defines an argument that should be passed to the action. Each action defines its own meaning of the argument. =back =head3 RFC2863_IF_MIB selector This type of selector selects the network interfaces on a SNMP device. The following attribute names are supported: =over 4 =item * C, C, ... Interface-level subtree name as you see it in the discovery results. You can compose a complex expression of subtree matches, by createing attributes with different numbers at the end. =item * C Comment string that is defined in C parameter. It is usually derived from an interface description. =item * C Numeric IANA interface type, as returned by C SNMP variable. =back C is compared numerically. C and C are regular expressions. C accepts multiple expressions separated by space, and the selector matches if any of these expressions matches the subtree name. The following actions are supported: =over 4 =item * C, C The argument defines the monitor name to be applied to the input or output bytes counter. =item * C, C, C, C The argument defines the monitor name to be applied to discard and error counters that are discovered for a given interface. =item * C The argument defines the value for C parameter. See the manual for F for more details. =item * C No argument needed. This action enables Holt-Winters prediction for given interface's counters. =item * C, C, C No argument needed. The action disables the packet, discard or error counters for a given interface to be collected. =item * C The argument is a comma-separated list of tokenset names where C graphs would be added. This action complements C. =item * C The argument defines additional configuration parameters to be placed at the interface level. The value should be one or several C pairs separated by semicolon (;). Parameters defined in C may override the ones defined in this action. =item * C, C Analagous to C, but the parameters are applied to InBytes and OutBytes leaves of the selected interface. =item * C For a DOCSIS CMTS, this action assigns a new monitor to the upstream Signal/Noise ratio gauge, instead of the default monitor. =item * C For a DOCSIS CMTS, this action adds the Signal/Noise ratio graph to a specified tokenset. =item * C For a DOCSIS CMTS, this action assigns a monitor to the C counter of the upstream FEC statistics. =item * C For a DOCSIS CMTS, this action assigns a monitor to the C counter of the upstream FEC statistics. =item * C For a DOCSIS CMTS, this action assigns a monitor to the downstream C counter. =item * C For Cisco uBR, this action assigns a monitor to the C gauge in the MAC layer stats. =item * C, C For Cisco uBR, this action assigns a monitor to upstream utilization and free contention timeslots gauges correspondingly. =back =head3 CiscoCPU selector A selector of this type selects CPU statistics Cisco router or switch. The attributes supported are: C and C, and their values are regular expressions that should match the CPU name or description, as discovered by C. The action supported is C, and its argument specifies the tokenset where to place the CPU statistics graph. =head3 CiscoSensor selector A selector of this type selects temperature sensors on a Cisco router or switch. The only attribute supported is C, and its value is a regular expression that should match the sensor description, as discovered by C. Actions supported: C, C. =head3 ALU_SAP selector This selector type is designed for SAP entries in Alcatel-Lucent ESS and SR routers. Attributes supported: C (regexp), C (regexp), C (exact match), C (exact match). Actions supported: C (excludes a selected SAP from the datasources). =head3 Examples The following example applies a monitor called C to all inlet sensors on a Cisco device: The following example enables Holt-Winters prediction and specifies some parameters to all FastEthernet interfaces which have the string "SRV-ID" in their descriptions: The following example sets up the monitors defined in F: DocsisUpSNRMonitor, DocsisUpFECCorMonitor, DocsisUpFECUncorMonitor, DocsisDownUtilMonitor, DocsisMacModemsMonitor, DocsisUpUtilMonitor, DocsisUpSlotsMonitor, InErrorsMonitor, OutErrorsMonitor =head1 Author Copyright (c) 2002-2005 Stanislav Sinyagin Essinyagin@yahoo.comE