I am attempting to understand SNMP (in general, and v3). The goal is to include an snmp agent in an embedded device running an RTOS.
I've already been through over a dozen RFCs with at least another dozen more to go. Each one creates more questions than it answers. (1052, 1065, 1067, 1155, 1156, 1157, 1212, 1213, 1592, 1905, 2578, 2579, 2580, 3410, 3411, 3412, 3413, 3414, 3415, 3416, 3417, 3418, 3584... )
I implemented mDNS-SD and 802.1X EAPOL with just a couple RFCs and it wasn't this confusing.
Many of the reviews of books I considered all complain of the same inconsistent and vagueness of the material. I bought a couple books that had better reviews.
Searching online isn't getting anywhere largely because the keywords aren't finding things I want answers to. So I must not even know the best keywords to search with.
Eventually, I decided to just try to reverse engineer what's going on, I installed WireShark on a Linux PC, and the snmpd and snmp tools, so I could sniff it. Here is what I have, and can't align what I see with what I read.
This is a v3 sniff, It's a reply to the first request from a manager. This question is just zeroing in on one of the things that I want to understand. I can't decode and examine a plaintext PDU, because I can't get a request in v2 or v1.
Wireshark shows this reply to a manager. It's apparently the first step in whatever authentication it to be used.
The book I have shows this as the protocol on the wire. And I am trying to parse out the variable bindings.
Here are the variable bindings from Wireshark
A "sequence" that is 15 bytes long (x30 x0f)
This, from the RFC, says that the list is a SEQUENCE of VarBinds, where each VarBind is the object name, and the value in ObjectSyntax. So it's looking okay so far.
Here is the next segment inside the SEQUENCE (Wireshark highlighted all 14 bytes)
An object ID that is 10 bytes long (x06, x0a)
Here is the actual object:
The objectName is the object ID, and it is x2b x6 x1 x6 x3 xf x1 x1 xx4 x0 or (1.3).6.1.6.3.15.1.1.4.0
Given that this is ISO, ORG, DOD, INTERNET, 6?... I have to assume "6" is an object under internet branch I've not yet come across. Likely something to do with the v3 security.
Next, is the value.
This is a type x41 (65), with a length of 1, and a value of 7.
Well, in "ObjectSyntax" what is x41? I can't find it defined anywhere.
For that matter, all these RFCs use words for identifiers, and I can find only a fraction of what their actual numeric values are.
Wireshark knew what it was... It's saying "Counter32"... is that what x41 is supposed to be? If so, it's nowhere near 32 bits. It's only one byte. Again, I'd like to find it's definition.
Also, somewhere, (I can't even recall which RFC) it said the reply to an OID request is to append the value to the requested object, not replace the zero (example: request: 1.3.6.1.4.300.1 -> reply 1.3.6.1.4.300.1.15 so it is a value of 15 ). This OID has a trailing zero, nad I'm not sure why.
Can anyone point me to some useful, concise, condensed information explaining this material? Every RFC requires that I go back and read some previous (and sometimes obsoleted) RFC, and I've now got over 25 of them already. I don't think it should take this many RFCs to be able to write an "simple" snmp agent. A month of researching, and most of what I have to show for it is how to read MIB files. Although that take some mental gymnastics too.
"Simple" is rather deceptive (as more than one book reviewer has stated).
RFC 1157 specifies that SNMP messages are encoded with "a subset of the basic encoding rules of ASN.1". I don't think the official basic encoding rules (BER) specification is available for free, but it's not hard to find explainers online (here's one I found with a simple search). To your question about the 0x41 byte, this is a BER identifier. The 2 most-significant bits (01) tell you the "class" (i.e. something like a namespace) is "application". The "form" bit (0) tells you that it's a primitive type (i.e. not a sequence). Finally the "tag" is 1. Consulting the SNMPv2-SMI MIB (RFC 2578) you can find this definition:
Counter32 ::=
[APPLICATION 1]
IMPLICIT INTEGER (0..4294967295)
You also asked about why a 32-bit integer is encoded with a single byte. This requires you to distinguish between the scope of the SNMP standard versus the ASN.1 standard. ASN.1 only has a single INTEGER type, which 1) has an unlimited range, 2) is always signed (two's complement), and 3) should be encoded in the least number of octets possible. This actually means that a Counter32 (or any other 32-bit unsigned integer type) might use up to 5 bytes for its encoding (see this answer I gave to a question about that).
Finally, you asked about the way the replies are modifying the requested OID. I was confused about this for a long time, but when I figured it out, I realized it's actually pretty simple. I think the best place to start is with this excerpt from RFC 1157:
Each instance of any object type defined in the MIB is identified in
SNMP operations by a unique name called its "variable name." In
general, the name of an SNMP variable is an OBJECT IDENTIFIER of the
form x.y, where x is the name of a non-aggregate object type defined
in the MIB and y is an OBJECT IDENTIFIER fragment that, in a way
specific to the named object type, identifies the desired instance.
This naming strategy admits the fullest exploitation of the semantics
of the GetNextRequest-PDU (see Section 4), because it assigns names
for related variables so as to be contiguous in the lexicographical
ordering of all variable names known in the MIB.
The type-specific naming of object instances is defined below for a
number of classes of object types. Instances of an object type to
which none of the following naming conventions are applicable are
named by OBJECT IDENTIFIERs of the form x.0, where x is the name of
said object type in the MIB definition.
For example, suppose one wanted to identify an instance of the
variable sysDescr The object class for sysDescr is:
iso org dod internet mgmt mib system sysDescr
1 3 6 1 2 1 1 1
Hence, the object type, x, would be 1.3.6.1.2.1.1.1 to which is
appended an instance sub-identifier of 0. That is, 1.3.6.1.2.1.1.1.0
identifies the one and only instance of sysDescr.
So, to summarize, the OID that comes from the MIB doesn't refer to a concrete object, but to the "object type". Each concrete object (i.e. "instance") is identified by a suffix of one or more sub-identifiers (i.e. the y in this explanation). For singleton objects, this suffix is always 0. However, I think most SNMP objects are found in tables, not in singleton objects. I don't actually know of a good explanation of this in the standards, so I'll give it my best shot.
Like any table, SNMP tables are made up of rows and columns. In SNMP, however, the rows are called "entries", and each entry defines a custom type to describe the columns. Here's a simple example from the IF-MIB:
ifTable OBJECT-TYPE
SYNTAX SEQUENCE OF IfEntry
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION
"A list of interface entries. The number of entries is
given by the value of ifNumber."
::= { interfaces 2 }
ifEntry OBJECT-TYPE
SYNTAX IfEntry
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION
"An entry containing management information applicable to a
particular interface."
INDEX { ifIndex }
::= { ifTable 1 }
IfEntry ::=
SEQUENCE {
ifIndex InterfaceIndex,
ifDescr DisplayString,
ifType IANAifType,
ifMtu Integer32,
ifSpeed Gauge32,
ifPhysAddress PhysAddress,
ifAdminStatus INTEGER,
ifOperStatus INTEGER,
ifLastChange TimeTicks,
ifInOctets Counter32,
ifInUcastPkts Counter32,
ifInNUcastPkts Counter32, -- deprecated
ifInDiscards Counter32,
ifInErrors Counter32,
ifInUnknownProtos Counter32,
ifOutOctets Counter32,
ifOutUcastPkts Counter32,
ifOutNUcastPkts Counter32, -- deprecated
ifOutDiscards Counter32,
ifOutErrors Counter32,
ifOutQLen Gauge32, -- deprecated
ifSpecific OBJECT IDENTIFIER -- deprecated
}
So, ifTable has an OID of 1.3.6.1.2.1.2.2, and ifEntry has an OID of 1.3.6.1.2.1.2.2.1. Each item in IfEntry also has its own definition, which includes the OID relative to ifEntry. Generally they match up with the entry's data type, so, for example, ifIndex, as the first column in IfEntry, has an OID of ifEntry.1. Confusingly, when you do a simple Get-Next walk, you will traverse in column-major order, meaning you will get all the ifIndexes, followed by all the ifDescrs, and so on.
So, with all that explained, I'm now prepared to explain the instance identifiers for these tables. Notice above that ifEntry defines
INDEX { ifIndex }
This means, first, that each row is guaranteed to have a unique ifIndex, and, more importantly, that the ifIndex is used as the instance identifier for the entire entry. For example, you can pick any column in the IfEntry data type, let's say ifOperStatus (1.3.6.1.2.1.2.2.1.8), and use Get-Next to find the first instance of that column. Let's say its OID is 1.3.6.1.2.1.2.2.1.8.1, and it's value is 1 (up). The last sub-identifier tells you that it belongs to the row whose ifIndex is 1. To find the name of that interface, you can then query ifDescr.1, and to find its speed setting, you can query ifSpeed.1, and so forth. In this case, it is possible to query ifIndex.1, which will just return 1, but in many tables, the INDEX columns are not-accessible, meaning you can only find out what instances there are by walking some other column. Some tables also use multiple indices, or use OCTET STRING or even OBJECT IDENTIFIER rather than INTEGER typed indices. The rules for encoding and decoding those are in RFC 2578 section 7.7.
Without thinking too hard about it I created a column of type [UUID] and successfully stored "strings" (as noted in the documentation, and generally referred to as a separate type altogether) returned from DB::generateKey in it.
Feels like I've done something I shouldn't have.
Can anyone share some light on this. Thanks in advance.
Mostly they return different types.
For clarity, DB::generateKey is equivalent to Uuid::generate |> toString
According to the standard library docs, it's the return type.
Uuid::generate() -> UUID
Generate a new UUID v4 according to RFC 4122
DB::generateKey() -> Str
Returns a random key suitable for use as a DB key
I believe the UUID type is a bitstring representation, that is, a specific sequence of bits in memory.
Whereas the Str type is a string representation of a UUID.
I've recently encountered all sorts of wrappers in Google's protobuf package. I'm struggling to imagine the use case. Can anyone shed the light: what problem were these intended to solve?
Here's one of the documentation links: https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/string-value (it says nothing about what can this be used for).
One thing that will be different in behavior between this, and simple string type is that this field will be written less efficiently (a couple extra bytes, plus a redundant memory allocation). For other wrappers, the story is even worse, since the repeated variants of those fields will be written inefficiently (official Google's Protobuf serializer doesn't support packed encoding for non-numeric types).
Neither seems to be desirable. So, what's this all about?
There's a few reasons, mostly to do with where these are used - see struct.proto.
StringValue can be null, string often can't be in a language interfacing with protobufs. e.g. in Go strings are always set; the "zero value" for a string is "", the empty string, so it's impossible to distinguish between "this value is intentionally set to empty string" and "there was no value present". StringValue can be null and so solves this problem. It's especially important when they're used in a StructValue, which may represent arbitrary JSON: to do so it needs to distinguish between a JSON key which was set to empty string (StringValue with an empty string) or a JSON key which wasn't set at all (null StringValue).
Also if you look at struct.proto, you'll see that these aren't fully fledged message types in the proto - they're all generated from message Value, which has a oneof kind { number_value, string_value, bool_value... etc. By using a oneof struct.proto can represent a variety of different values in one field. Again this makes sense considering what struct.proto is designed to handle - arbitrary JSON - you don't know what type of value a given JSON key has ahead of time.
In addition to George's answer, you can't use a Protobuf primitive as the parameter or return value of a gRPC procedure.
I am writing an extension to my companies existing SNMP MIB. I have a whole list of objects, with the same properties on each. I want to be able to get and set these through SNMP.
So for example, consider my object has name, desc, arg0, arg1. What I want is to be able to refer to these as:
fullpath.objects.ObjectA.name
fullpath.objects.ObjectA.desc
fullpath.objects.ObjectA.arg0
fullpath.objects.ObjectB.name
fullpath.objects.ObjectB.desc
fullpath.objects.ObjectB.arg0
However the leaf nodes appear to have to have unique names, so I am unable to define this.
I can use a SNMP table to produce:
fullpath.objects.table.name.1
fullpath.objects.table.desc.1
fullpath.objects.table.arg0.1
fullpath.objects.table.name.2
fullpath.objects.table.desc.2
fullpath.objects.table.arg0.2
But there is nowhere to look up that 2 means ObjectB. This leaves it open to user error looking up the wrong value and setting the wrong thing.
At the moment the best solution I can see is:
fullpath.objects.ObjectAName
fullpath.objects.ObjectADesc
fullpath.objects.ObjectAArg0
fullpath.objects.ObjectBName
fullpath.objects.ObjectBDesc
fullpath.objects.ObjectBArg0
which involves defining name for every object (there are 20 or so of them). The set of objects is fixed, so this is ok...just not very tidy.
Is there some way to define names for index in the table?
Is there some way of defining a container type?
Is there some way of allowing leaf nodes to be non-unique?
Any other ideas?
You should definitely use SNMP tables to accomplish what is required. This is the only way.
MIB Object names must be unique within entire MIB file.
You can easily use object of OCTET STRING type as Table index. So each byte/symbol/char of OCTET STRING value will be translated to corresponding numeric ASCII code in OID.
I ended up just using a naming convention and adding each of the settings directly into the MIB.
Not really the answer I wanted, but it means that all of the settings show up in the MIB, and that reduces the chance of users setting the wrong setting.
I have a list of objects that are returned as a sequence, I would like to retrieve the keys of each object so as to be able to display the object correctly. At the moment I try data?first?keys which seems to get something like the queries that return the objects (Not sure how to explain that last sentence either but img below shows what I'm trying to explain).
The objects amount of objects returned are correct (7) but displaying the keys for each object is my aim. The macro that attempts this is here (from the apache ofbiz development book chapter 8).
Seems like it my sequence is a list of hashes and as explained by Daniel Dekany this post:
The original problem is that, someHash[key] expects a
string as key. Because, the hash type of FTL, by definition, maps
string keys to arbitrary values. It's not the same as Java's Map.
(Note that to further complicate the matters, in FTL
someSequenceOrString[index] expects an integer index. So, the [] thing
is used for that too.) Now someBeanWrappedMap(key) has technically
nothing to do with all the []-s, it's just a method call, so it
accepts all kind of keys. If you have a Map with non-string keys, you
must use that.
Thanks D Dekany if you're on stack, this ended my half day frustration with the ftl template.