I currently have a script to glean QoS data from differing cisco routers and this is working well but missing the bandwidth data for each class.
I can see that the data is available in that querying:
enterprises.9.9.166.1.9.1.1.1.1608 = INTEGER: 425
Returns the correct bandwidth for this particular class [425kb]. I have seen this index elsewhere:
enterprises.9.9.166.1.5.1.1.2.6933270.5456067 = Gauge32: 1608
With '6933270' being one of the indexes associated with the interface I am interested in.
How though do I 'learn' the second index '5456067' or is there another way to derive the class bandwidth?
I have scoured Google which has me at this point but I am unable to get any closer to the second index. Multiple snmpwalks grepping the second index show no light either in that I can find no way to relate to this from existing known data.
Thanks
I think you get the wrong oid entry. enterprises.9.9.166.1.5.1.1.2 stands for cisco cbQosConfigIndex from mib, if you want to get the bandwidth, you should use the 1.3.6.1.4.1.9.9.166.1.9.1.1.2 which means QueueingBandwidthUnits instead.
Related
I have been reading through documentation and have been able to get Graphite to receive data I have been sending it. I can definitely see the benefit in tracking things like concurrent users and network load.
But now I have been tasked with implementing it on the client to show things like RAM, and CPU usage. In addition to this, I must also track actions (users buying things). Maybe I am missing a large chunk of the picture here, but I am not sure how I would do these things. Do I need a timestamp? I've also seen plugins for pie charts and such and this would indicate I could perhaps create graphs from devices with different statistics.
What am I missing?
I don't think you're missing anything.
Any data you send on to, say, InfluxDB (as that's what I've used the most) will be timestamped automatically when it arrives - unless you specify an explicit one yourself.
If you're showing, for example, CPU load you can write your query to pick up the latest value, or perhaps an average (mean value) over time, or the max or min over a period of time as appropriate.
Pie charts can also be used successfully to graph relationships over time.
The key is to create a specific query (I use SQL directly) to craft the data used for the panel type. There are excellent examples in the documentation.
Can anyone tell me why NMS implementations parse and save MIB items in a database?
I know one of the reasons is when they receive a trap and want to analyze it, then they use the parsed MIB. What else they do with parsed MIB?
For example, when the NMS sends a SNMP GET request to an agent, the programmer must specify which OIDs are being requested?
Does the the parsed MIB have a another purpose or do we parse MIBs only for analyzing SNMP traps?
You are on the right track - you parse the MIB at all in order to make it human-readable. That is for both traps (informs) and polled values. But if you parse it out to a text file, that's a huge amount of data to read/grep through to find out the description, message, possible values, related OIDs, etc.
Added to this is that there isn't just one MIB. There are dozens or hundreds that an NMS may be interested in. Since, on a host, you only add the MIBs that you want that host to respond to, the NMS has to have a copy of every MIB that ever device it is monitoring may have on IT so that it can understand the response the host returns.
So you parse each MIB and store it in a db to make it faster to search and to have everything all in one place. That could be so that you can find the messages associated with varbinds, or what all the possible enumerations are, etc.
Just to be clear, parsing the MIB isn't the same as doing an SNMPWalk on a host. SNMPWalk just gives you the current response to each OID in sequence.
I am using NET-SNMP (V5.6.1.1) on windows to read my MIB with snmpget & snmpwalk. When I try accessing the MIB I can only see some of the elements. I know the MIB is good since my colleague can extract the same revision of the MIB from the repository and can see all elements within the MIB. We are using the same SNMP command syntax to query the data. I have compared the MIB and snmp.conf files between his machine and mine and they are identical, so can only assume that it is due to a difference in the configuration of our respective PCs. I’ve also checked for any differences in the Environment Variables between our machines, but can see nothing obvious. Is there anything in the machine configuration that might explain why I can only see part of the MIB?
Edit: The MIB is implemented as a single bespoke executable, with the data held in a number of tables, for example:
mibTableA.parameter1
mibTableA.parameter2
mibTableA.parameter3
mibTableB.parameter4
mibTableB.parameter5
mibTableC.parameter6
mibTableC.parameter7
mibTableC.parameter8
None of these tables are dependant upon the availability of system hardware, etc. These tables can also be accesses via an RTA interface using PSQL queries, and using the RTA interface on both my machine and my colleague’s machines, I can see all the tables/parameters. Yet, for instance, accessing the MIB via SNMP I can only see the mibTableA on my machine.
First you need to identify which are the missing ones on your box. Show some examples in your question so that others can guess what can be the cause.
Second, SNMP query result is indeed machine dependent. For example, if your machine has less network adapters than your friend's, then it is reasonable some objects are missing.
I found the problem. There are some scalar fields in the MIB which define the table sizes and these had not been initialised correctly, but instead were picking up old values stored in tables in a C:\Documents and Settings\user\Application Data folder. Hence the difference in behaviour between my machine and my colleagues.
I have a web app that uses Guids as the PK in the DB for an Employee object and an Association object.
One page in my app returns a large amount of data showing all Associations all Employees may be a part of.
So right now, I am sending to the client essentially a bunch of objects that look like:
{assocation_id: guid, employees: [guid1, guid2, ..., guidN]}
It turns out that many employees belong to many associations, so I am sending down the same Guids for those employees over and over again in these different objects. For example, it is possible that I am sending down 30,000 total guids across all associations in some cases, of which there are only 500 unique employees.
I am wondering if it is worth me building some kind of lookup index that I also send to the client like
{ 1: Guid1, 2: Guid2 ... }
and replacing all of the Guids in the objects I send down with those ints,
or if simply gzipping the response will compress it enough that this extra effort is not worth it?
Note: please don't get caught up in the details of if I should be sending down 30,000 pieces of data or not -- this is not my choice and there is nothing I can do about it (and I also can't change Guids to ints or longs in the DB).
Your wrote at the end of your question the following
Note: please don't get caught up in the details of if I should be
sending down 30,000 pieces of data or not -- this is not my choice and
there is nothing I can do about it (and I also can't change Guids to
ints or longs in the DB).
I think it's your main problem. If you don't solve the main problem you will be able to reduce the size of transferred data to 10 times for example, but you still don't solve the main problem. Let us we think about the question: Why so many data should be sent to the client (to the web browser)?
The data on the client side are needed to display some information to the user. The monitor is not so large to show 30,000 total on one page. No user are able to grasp so much information. So I am sure that you display only small part of the information. In the case you should send only the small part of information which you display.
You don't describe how the guids will be used on the client side. If you need the information during row editing for example. You can transfer the data only when the user start editing. In the case you need transfer the data only for one association.
If you need display the guids directly, then you can't display all the information at once. So you can send the information for one page only. If the user start to scroll or start "next page" button you can send the next portion of data. In the way you can really dramatically reduce the size of transferred data.
If you do have no possibility to redesign the part of application you can implement your original suggestion: by replacing of GUID "{7EDBB957-5255-4b83-A4C4-0DF664905735}" or "7EDBB95752554b83A4C40DF664905735" to the number like 123 you reduce the size of GUID from 34 characters to 3. If you will send additionally array of "guid mapping" elements like
123:"7EDBB95752554b83A4C40DF664905735",
you can reduce the original size of data 30000*34 = 1020000 (1 MB) to 300*39 + 30000*3 = 11700+90000 = 101700 (100 KB). So you can reduce the size of data in 10 times. The usage of compression of dynamic data on the web server can reduce the size of data additionally.
In any way you should examine why your page is so slowly. If the program works in LAN, then the transferring of even 1MB of data can be quick enough. Probably the page is slowly during placing of the data on the web page. I mean the following. If you modify some element on the page the position of all existing elements have to be recalculated. If you would be work with disconnected DOM objects first and then place the whole portion of data on the page you can improve the performance dramatically. You don't posted in the question which technology you use in you web application so I don't include any examples. If you use jQuery for example I could give some example which clear more what I mean.
The lookup index you propose is nothing else than a "custom" compression scheme. As amdmax stated, this will increase your performance if you have a lot of the same GUIDs, but so will gzip.
IMHO, the extra effort of writing the custom coding will not be worth it.
Oleg states correctly, that it might be worth fetching the data only when the user needs it. But this of course depends on your specific requirements.
if simply gzipping the response will compress it enough that this extra effort is not worth it?
The answer is: Yes, it will.
Compressing the data will remove redundant parts as good as possible (depending on the algorithm) until decompression.
To get sure, just send/generate the data uncompressed and compressed and compare the results. You can count the duplicate GUIDs to calculate how big your data block would be with the dictionary compression method. But I guess gzip will be better because it can also compress the syntactic elements like braces, colons, etc. inside your data object.
So what you are trying to accomplish is Dictionary compression, right?
http://en.wikibooks.org/wiki/Data_Compression/Dictionary_compression
What you will get instead of Guids which are 16 bytes long is int which is 4 bytes long. And you will get a dictionary full of key value pairs that will associate each guid to some int value, right?
It will decrease your transfer time when there're many objects with the same id used. But will spend CPU time before transfer to compress and after transfer to decompress. So what is the amount of data you transfer? Is it mb / gb / tb? And is there any good reason to compress it before sending?
I do not know how dynamic is your data, but I would
on a first call send two directories/dictionaries mapping short ids to long GUIDS, one for your associations and on for your employees e.g. {1: AssoGUID1, 2: AssoGUID2,...} and {1: EmpGUID1, 2:EmpGUID2,...}. These directories may also contain additional information on the Associations and Employees instances; I suspect you do not simply display GUIDs
on subsequent calls just send the index of Employees per Association { 1: [2,4,5], 3:[2,4], ...}, the key being the association short id and the ids in the array value, the short ids of the employees. Given your description building the reverse index: Employee to Associations may give better result size wise (but higher processing)
Then its all down to associative arrays manipulations which is straightforward in JS.
Again, if your data is (very) dynamic server side, the two directories will soon be obsolete and maintaining synchronization may cost you a lot.
I would start by answering the following questions:
What are the performance requirements? Are there size requirements? Speed requirements? What is the minimum performance that is truly needed?
What are the current performance metrics? How far are you from the requirements?
You characterized the data as possibly being mostly repeats. Is that the normal case? If not, what is?
The 2 options you listed above sound reasonable and trivial to implement. Try creating a look-up table and see what performance gains you get on actual queries. Try zipping the results (with look-ups and without), and see what gains you get.
In my experience if you're not TOO far from the goal, performance requirements are often trial and error.
If those options don't get you close to the requirements, I would take a step back and see if the requirements are reasonable in the time you have to solve the problem.
What you do next depends on which performance goals are lacking. If it is size, you're starting to be limited if you're required to send the entire association list ever time. Is that truly a requirement? Can you send the entire list once, and then just updates?
I'm querying a bunch of information from cisco switches using SNMP. For instance, I'm pulling information on neighbors detected using CDP by doing an snmpwalk on .1.3.6.1.4.1.9.9.23
Can I use this OID across different cisco models? What pitfalls should I be aware of? To me, I'm a little uneasy about using numeric OIDs - it seems like I should be using a MIB database or something and using the named OIDs, in order to gain cross-device compatibility, but perhaps I'm just imagining the need for that.
Once a MIB has been published it won't move to a new OID. Doing so would break network management tools and cause support calls, which nobody wants. To continue your example, the CDP MIB has been published at Cisco's SNMP Object Navigator.
For general code cleanliness it would be good to define the OIDs in a central place, especially since you don't want to duplicate the full OID for every single table you need to access.
The place you need to be most careful is a unique MIB in a product which Cisco recently acquired. The OID will change, if nothing else to move it into their own Enterprise OID space, but the MIB may also change to conform to Cisco's SNMP practices.
It is very consistent.
Monitoring tools depend on the consistency and the MIBs produced by Cicso rarely change old values and usually only implement new ones.
Check out the Cisco OID look up tool.
Notice how it doesn't ask you what product the look up is for.
-mw
The OIDs can vary with hardware but also with firmware version for the same hardware as, over time, the architecture of the management functions can change and require new MIBs. It is worth checking whether any of the OIDs you intend to use are in deprecated MIBs, or become so in the life of the application, as this indicates not only that the MIB could one day be unsupported but also that there is likely to be improved, richer data or access to data. It is also good practice to test management apps against a sample upgraded device as part of the routine testing of firmware updates before widespread deployment.
An example of a change of OID due to a MIB being deprecated is at
http://www.cisco.com/en/US/tech/tk648/tk362/technologies_configuration_example09186a0080094aa6.shtml
"This document shows how to copy a
configuration file to and from a Cisco
device with the CISCO-CONFIG-COPY-MIB.
If you start from Cisco IOS® software
release 12.0, or on some devices as
early as release 11.2P, Cisco has
implemented a new means of Simple
Network Management Protocol (SNMP)
configuration management with the new
CISCO-CONFIG-COPY-MIB. This MIB
replaces the deprecated configuration
section of the OLD-CISCO-SYSTEM-MIB. "
I would avoid putting in numeric OIDs and instead use 'OID names' and leave that hard work (of translating) to whatever SNMP API you are using.
If that is not possible, then it is okay to use OIDs as they should not change per the SNMP MIB guidelines. Unless the device itself changes but that requires a new MIB anyway which can't reuse old OIDs.
This is obvious, but be sure to look at the attributes of the SNMP MIB variable. Be sure not to query variables that have a status of 'obsolete'.
Jay..
In some cases, using the names instead of the numerical representations can be a serious performance hit due to the need to read and parse the MIB files to get the numerical representations of the OIDs that the lower level libraries need.
For instance, say your using a program to collect something every minute, then loading the MIBs over and over is very inefficient.
As stated by others, once published, the name to numerical mapping will never change, so the fact that you're hard-coding stuff into your programs is not really a problem.
If you have access to command line SNMP tools, check out 'snmptranslate' for a nice tool to get back and forth from text to numerical OIDs.
I think that is a common misconception (about MIB reload each time you resolve a name).
Most of the SNMP APIs (such as AdventNet, CMU) load the MIBS at startup and after that there is no 'overhead' of loading MIBs everytime you ask for a 'translation' from name to oid and vice versa. What's more, some of them cache the results and at that point, there is no difference between name lookups and directly coding the OID.
This is a bit similar to specifying an "IP Address" versus a 'hostname'.