I am attempting to understand SNMP (in general, and v3). The goal is to include an snmp agent in an embedded device running an RTOS.
I've already been through over a dozen RFCs with at least another dozen more to go. Each one creates more questions than it answers. (1052, 1065, 1067, 1155, 1156, 1157, 1212, 1213, 1592, 1905, 2578, 2579, 2580, 3410, 3411, 3412, 3413, 3414, 3415, 3416, 3417, 3418, 3584... )
I implemented mDNS-SD and 802.1X EAPOL with just a couple RFCs and it wasn't this confusing.
Many of the reviews of books I considered all complain of the same inconsistent and vagueness of the material. I bought a couple books that had better reviews.
Searching online isn't getting anywhere largely because the keywords aren't finding things I want answers to. So I must not even know the best keywords to search with.
Eventually, I decided to just try to reverse engineer what's going on, I installed WireShark on a Linux PC, and the snmpd and snmp tools, so I could sniff it. Here is what I have, and can't align what I see with what I read.
This is a v3 sniff, It's a reply to the first request from a manager. This question is just zeroing in on one of the things that I want to understand. I can't decode and examine a plaintext PDU, because I can't get a request in v2 or v1.
Wireshark shows this reply to a manager. It's apparently the first step in whatever authentication it to be used.
The book I have shows this as the protocol on the wire. And I am trying to parse out the variable bindings.
Here are the variable bindings from Wireshark
A "sequence" that is 15 bytes long (x30 x0f)
This, from the RFC, says that the list is a SEQUENCE of VarBinds, where each VarBind is the object name, and the value in ObjectSyntax. So it's looking okay so far.
Here is the next segment inside the SEQUENCE (Wireshark highlighted all 14 bytes)
An object ID that is 10 bytes long (x06, x0a)
Here is the actual object:
The objectName is the object ID, and it is x2b x6 x1 x6 x3 xf x1 x1 xx4 x0 or (1.3).6.1.6.3.15.1.1.4.0
Given that this is ISO, ORG, DOD, INTERNET, 6?... I have to assume "6" is an object under internet branch I've not yet come across. Likely something to do with the v3 security.
Next, is the value.
This is a type x41 (65), with a length of 1, and a value of 7.
Well, in "ObjectSyntax" what is x41? I can't find it defined anywhere.
For that matter, all these RFCs use words for identifiers, and I can find only a fraction of what their actual numeric values are.
Wireshark knew what it was... It's saying "Counter32"... is that what x41 is supposed to be? If so, it's nowhere near 32 bits. It's only one byte. Again, I'd like to find it's definition.
Also, somewhere, (I can't even recall which RFC) it said the reply to an OID request is to append the value to the requested object, not replace the zero (example: request: 1.3.6.1.4.300.1 -> reply 1.3.6.1.4.300.1.15 so it is a value of 15 ). This OID has a trailing zero, nad I'm not sure why.
Can anyone point me to some useful, concise, condensed information explaining this material? Every RFC requires that I go back and read some previous (and sometimes obsoleted) RFC, and I've now got over 25 of them already. I don't think it should take this many RFCs to be able to write an "simple" snmp agent. A month of researching, and most of what I have to show for it is how to read MIB files. Although that take some mental gymnastics too.
"Simple" is rather deceptive (as more than one book reviewer has stated).
RFC 1157 specifies that SNMP messages are encoded with "a subset of the basic encoding rules of ASN.1". I don't think the official basic encoding rules (BER) specification is available for free, but it's not hard to find explainers online (here's one I found with a simple search). To your question about the 0x41 byte, this is a BER identifier. The 2 most-significant bits (01) tell you the "class" (i.e. something like a namespace) is "application". The "form" bit (0) tells you that it's a primitive type (i.e. not a sequence). Finally the "tag" is 1. Consulting the SNMPv2-SMI MIB (RFC 2578) you can find this definition:
Counter32 ::=
[APPLICATION 1]
IMPLICIT INTEGER (0..4294967295)
You also asked about why a 32-bit integer is encoded with a single byte. This requires you to distinguish between the scope of the SNMP standard versus the ASN.1 standard. ASN.1 only has a single INTEGER type, which 1) has an unlimited range, 2) is always signed (two's complement), and 3) should be encoded in the least number of octets possible. This actually means that a Counter32 (or any other 32-bit unsigned integer type) might use up to 5 bytes for its encoding (see this answer I gave to a question about that).
Finally, you asked about the way the replies are modifying the requested OID. I was confused about this for a long time, but when I figured it out, I realized it's actually pretty simple. I think the best place to start is with this excerpt from RFC 1157:
Each instance of any object type defined in the MIB is identified in
SNMP operations by a unique name called its "variable name." In
general, the name of an SNMP variable is an OBJECT IDENTIFIER of the
form x.y, where x is the name of a non-aggregate object type defined
in the MIB and y is an OBJECT IDENTIFIER fragment that, in a way
specific to the named object type, identifies the desired instance.
This naming strategy admits the fullest exploitation of the semantics
of the GetNextRequest-PDU (see Section 4), because it assigns names
for related variables so as to be contiguous in the lexicographical
ordering of all variable names known in the MIB.
The type-specific naming of object instances is defined below for a
number of classes of object types. Instances of an object type to
which none of the following naming conventions are applicable are
named by OBJECT IDENTIFIERs of the form x.0, where x is the name of
said object type in the MIB definition.
For example, suppose one wanted to identify an instance of the
variable sysDescr The object class for sysDescr is:
iso org dod internet mgmt mib system sysDescr
1 3 6 1 2 1 1 1
Hence, the object type, x, would be 1.3.6.1.2.1.1.1 to which is
appended an instance sub-identifier of 0. That is, 1.3.6.1.2.1.1.1.0
identifies the one and only instance of sysDescr.
So, to summarize, the OID that comes from the MIB doesn't refer to a concrete object, but to the "object type". Each concrete object (i.e. "instance") is identified by a suffix of one or more sub-identifiers (i.e. the y in this explanation). For singleton objects, this suffix is always 0. However, I think most SNMP objects are found in tables, not in singleton objects. I don't actually know of a good explanation of this in the standards, so I'll give it my best shot.
Like any table, SNMP tables are made up of rows and columns. In SNMP, however, the rows are called "entries", and each entry defines a custom type to describe the columns. Here's a simple example from the IF-MIB:
ifTable OBJECT-TYPE
SYNTAX SEQUENCE OF IfEntry
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION
"A list of interface entries. The number of entries is
given by the value of ifNumber."
::= { interfaces 2 }
ifEntry OBJECT-TYPE
SYNTAX IfEntry
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION
"An entry containing management information applicable to a
particular interface."
INDEX { ifIndex }
::= { ifTable 1 }
IfEntry ::=
SEQUENCE {
ifIndex InterfaceIndex,
ifDescr DisplayString,
ifType IANAifType,
ifMtu Integer32,
ifSpeed Gauge32,
ifPhysAddress PhysAddress,
ifAdminStatus INTEGER,
ifOperStatus INTEGER,
ifLastChange TimeTicks,
ifInOctets Counter32,
ifInUcastPkts Counter32,
ifInNUcastPkts Counter32, -- deprecated
ifInDiscards Counter32,
ifInErrors Counter32,
ifInUnknownProtos Counter32,
ifOutOctets Counter32,
ifOutUcastPkts Counter32,
ifOutNUcastPkts Counter32, -- deprecated
ifOutDiscards Counter32,
ifOutErrors Counter32,
ifOutQLen Gauge32, -- deprecated
ifSpecific OBJECT IDENTIFIER -- deprecated
}
So, ifTable has an OID of 1.3.6.1.2.1.2.2, and ifEntry has an OID of 1.3.6.1.2.1.2.2.1. Each item in IfEntry also has its own definition, which includes the OID relative to ifEntry. Generally they match up with the entry's data type, so, for example, ifIndex, as the first column in IfEntry, has an OID of ifEntry.1. Confusingly, when you do a simple Get-Next walk, you will traverse in column-major order, meaning you will get all the ifIndexes, followed by all the ifDescrs, and so on.
So, with all that explained, I'm now prepared to explain the instance identifiers for these tables. Notice above that ifEntry defines
INDEX { ifIndex }
This means, first, that each row is guaranteed to have a unique ifIndex, and, more importantly, that the ifIndex is used as the instance identifier for the entire entry. For example, you can pick any column in the IfEntry data type, let's say ifOperStatus (1.3.6.1.2.1.2.2.1.8), and use Get-Next to find the first instance of that column. Let's say its OID is 1.3.6.1.2.1.2.2.1.8.1, and it's value is 1 (up). The last sub-identifier tells you that it belongs to the row whose ifIndex is 1. To find the name of that interface, you can then query ifDescr.1, and to find its speed setting, you can query ifSpeed.1, and so forth. In this case, it is possible to query ifIndex.1, which will just return 1, but in many tables, the INDEX columns are not-accessible, meaning you can only find out what instances there are by walking some other column. Some tables also use multiple indices, or use OCTET STRING or even OBJECT IDENTIFIER rather than INTEGER typed indices. The rules for encoding and decoding those are in RFC 2578 section 7.7.
I've recently encountered all sorts of wrappers in Google's protobuf package. I'm struggling to imagine the use case. Can anyone shed the light: what problem were these intended to solve?
Here's one of the documentation links: https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/string-value (it says nothing about what can this be used for).
One thing that will be different in behavior between this, and simple string type is that this field will be written less efficiently (a couple extra bytes, plus a redundant memory allocation). For other wrappers, the story is even worse, since the repeated variants of those fields will be written inefficiently (official Google's Protobuf serializer doesn't support packed encoding for non-numeric types).
Neither seems to be desirable. So, what's this all about?
There's a few reasons, mostly to do with where these are used - see struct.proto.
StringValue can be null, string often can't be in a language interfacing with protobufs. e.g. in Go strings are always set; the "zero value" for a string is "", the empty string, so it's impossible to distinguish between "this value is intentionally set to empty string" and "there was no value present". StringValue can be null and so solves this problem. It's especially important when they're used in a StructValue, which may represent arbitrary JSON: to do so it needs to distinguish between a JSON key which was set to empty string (StringValue with an empty string) or a JSON key which wasn't set at all (null StringValue).
Also if you look at struct.proto, you'll see that these aren't fully fledged message types in the proto - they're all generated from message Value, which has a oneof kind { number_value, string_value, bool_value... etc. By using a oneof struct.proto can represent a variety of different values in one field. Again this makes sense considering what struct.proto is designed to handle - arbitrary JSON - you don't know what type of value a given JSON key has ahead of time.
In addition to George's answer, you can't use a Protobuf primitive as the parameter or return value of a gRPC procedure.
Am trying to read the lotus notes document using VB6.I can able to read the values of the but suddenly type mismatch error is throwed.When i reintialise the vb6 variable it works but stops after certain point.
ex; address field in lotus notes
lsaddress=ImsField(doc.address)
private function ImsField(pValue)
ImsField=pValue(0)
end function
Like this I am reading the remaining fields but at certain point the runtime error "13" type mismatch error throwed.
I have to manually reintialize by
set doc=view.getdocumentbykey(doclist)
The type mismatch error occurs for a certain field. The issue should be a data type incompatibility. Try to figure out which field causes the error.
Use GetItemValue() instead of short notation for accessing fields and don't use ImsField():
lsaddress=doc.GetItemValue("address")(0)
The type mismatch is occurring because you are encountering a case where pValue is not an array. That will occur when you attempt to reference a NotesItem that does not exist. I.e., doc.MissingItem.
You should not use the shorthand notation doc.itemName. It is convenient, but it leads to sloppy coding. You should use getItemValue as everyone else is suggesting, and also you should check to see if the NotesItem exists. I.e.,
if doc.hasItem("myItem") then
lsaddress=doc.getItemValue("myItem")(0)
end if
Notes and Domino are schema-less. There are no data integrity checks other than what you write yourself. You may think that the item always has to be there, but the truth is that there is nothing that will ever guarantee that, so it is always up to you to write your code so that it doesn't assume anything.
BTW: There are other checks that you might want to perform besides just whether or not the field exists. You might want to check the field's type as well, but to do that requires going one more level up the object chain and using getFirstItem instead of getItemValue, which I'm not going to get into here. And the reason, once again, is that Notes and Domino are schema-less. You might think that a given item must always be a text list, but all it takes is someone writing sloppy code in an one-time fix-it agent and you could end up having a document in which that item is numeric!
Checking your fields is actually a good reason (sometimes) to encapsulate your field access in a function, much like the way you have attempted to do. The reason I added "sometimes" above is that your code's behavior for a missing field isn't necessarily always going to be the same, but for cases where you just want to return a default value when the field doesn't exist you can use something like this:
lsaddress ImsField("address","")
private function ImsField(fieldName,defaultValue)
if doc.hasItem(fieldName) then
lsaddress=doc.getItemValue(fieldName)(0)
else
lsaddress=defaultValue
end if
end function
Type mismatch comes,
When you try to set values from one kind of datatype variable to different datatype of another variable.
Eg:-
dim x as String
Dim z as variant
z= Split("Test:XXX",":")
x=z
The above will through the error what you mentioned.
So check the below code...
lsaddress = ImsField(doc.address)
What is the datatype of lsaddress?
What is the return type of ImsField(doc.address)?
If the above function parameter is a string, then you should pass the parameter like (doc.address(0))
I'm trying to determine the relationship between default values and the has_foo() methods that are declared in various programmatic interfaces. In particular, I'm trying to determine under what circumstances (if any) you can "tell the difference" between a field explicitly set to the default value, and an unset value.
If I explicitly set a field (e.g. "Bar.foo") to its default value (e.g., zero), then is Bar::has_foo() guaranteed return true for that data structure? (This appears to be true for the C++ generated code, from a quick inspection, but that doesn't mean it's guaranteed.) If this is true, then it's possible to distinguish between an explicitly set default value and an unset prior to serialization.
If I explicitly set a field to its default value (e.g., zero), and then serialize that object and send it over the wire, will the value be sent or not? If it is not, then clearly any code that receives this object can't distinguish between an explicitly set default value and an unset value. I.e., it won't be possible to distinguish these two cases after serialization -- Bar::has_foo() will return false in both cases.
If it's not possible to tell the difference, what is the recommended technique for encoding a protobuf field if I want to encode a "nullable" optional value? A couple options come to mind, but neither seem great: (a) add an extra boolean field that records whether the field is set or not, or (b) use a "repeated" field even though I semantically want an optional field -- this way I can tell the difference between no value (length-zero list) or a set value (length-one list).
The following applies for 'proto2' syntax, not 'proto3' :
The notion of a field being set or not is a core feature of Protobuf. If you set a field to a value (any value), then the corresponding has_xxx method must return true, otherwise you have a bug in the API.
If you do not set a field and then serialize the message, no value is sent for that field. The receiving side will parse the message, discover which values where included, and set the corresponding "has_xxx" values.
Exactly how this is implemented in the wire-format is documented here: http://code.google.com/apis/protocolbuffers/docs/encoding.html. The short version is that message are encoded as a sequence of key-value pairs, and only fields which are explicitly set are included in the encoded message.
Default values only come into play when you attempt to read an unset field.
endOfMibView is an SNMPexception, right? if endOfMibView occurs in SNMPv3, what will be the value of the error-status parameter in the PDU? Is there any RFC regarding this matter?
In SNMPv1, "noSuchName" is set to the error-status.
Thanks for answering!
Actually, no... endOfMibView is an exception added to the varbind value itself when it's hit. This is done so that an endOfMibView can occur along side a real variable as well. In the SNMPv2 PDU structure, documented in RFC3416, you'll find the definition.
If it set the error-status you'd have to refetch the rest of the variables since it would be a true error and would stop the entire processing sequence. Having it as an in-line exception means that the other variable fetches can proceed still.