The Official documentation about map type says:
map<key_type, value_type> map_field = N;
...where the key_type can be any integral or string type (so, any
scalar type except for floating point types and bytes). The value_type
can be any type.
I want to define a map<string, repeated string> field, but it seems illegal on my libprotoc 3.0.0, which complains Expected ">". So I wonder if there is any way to put repeated string into map.
A Possible workaround could be:
message ListOfString {
repeated string value = 1;
}
// Then define:
map<string, ListOfString> mapToRepeatedString = 1;
But ListOfString here looks redundant.
I had the same need, and got the same error. I do not believe this is possible. Here is the relevant BNF definitions from the language specification.
https://developers.google.com/protocol-buffers/docs/reference/proto3-spec
messageType = [ "." ] { ident "." } messageName
mapField = "map" "<" keyType "," type ">" mapName "=" fieldNumber [ "["fieldOptions "]" ] ";"
type = "double" | "float" | "int32" | "int64" | "uint32" | "uint64"
| "sint32" | "sint64" | "fixed32" | "fixed64" | "sfixed32" | "sfixed64"
| "bool" | "string" | "bytes" | messageType | enumType
messageName = ident
ident = letter { letter | decimalDigit | "_" }
field = [ "repeated" ] type fieldName "=" fieldNumber [ "[" fieldOptions "]" ] ";"
"repeated" keyword only appears in the field definition. The map definition requires a "type", which does not include the repeated keyword.
That means there are a few options.
You could create a wrapper around the repeated value as you indicated.
There is the older way people defined define maps, which is more burdensome but is equivalent. This is the backwards compatible example from the language guide.
https://developers.google.com/protocol-buffers/docs/proto3#maps
message MapFieldEntry {
key_type key = 1;
repeated value_type value = 2;
}
repeated MapFieldEntry map_field = N;
You would need to convert the data to a map yourself, but this should be fairly trivial in most languages. In Java:
List<MapFieldEntry> map_field = // Existing list from protobuf.
Map<key_type, List<value_type>> = map_field.stream()
.collect(Collectors.toMap(kv -> kv.key, kv -> kv.value));
Use google.protobuf.ListValue
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#listvalue
This is an untyped list collection from their well known types.
I think it should be as follows.
message ListOfString {
repeated string what_ever_name = 1;
}
// Then define:
map<string, ListOfString> what_ever_name = 1;
Remember what_ever_name should be same in both places.
Related
I want to define a message that can have 2 fields (field A AND field B) XOR one other field (field C alone). I saw I can use the keyword oneof to set the XOR, but only between two fields.
how can I express my needs?
Ideally I want something like (not working)
syntax = "proto3";
message M {
oneof name {
{
string a = 1;
string b = 2;
}
string c = 3;
}
}
Only way I know of is to put the two fields a and b into separate submessage, which you can then put inside the oneof:
syntax = "proto3";
message A {
string a = 1;
string b = 2;
}
message M {
oneof name {
A a = 1;
string c = 3;
}
}
Alternatively you can put all fields into M without oneof. Describe the logic in the comments and check it manually in application code.
In the Protocol Buffers Version 3 Language Specification
The EBNF syntax for an option is
option = "option" optionName "=" constant ";"
optionName = ( ident | "(" fullIdent ")" ) { "." ident }
constant = fullIdent | ( [ "-" | "+" ] intLit ) | ( [ "-" | "+" ] floatLit ) | strLit | boolLit
ident = letter { letter | decimalDigit | "_" }
fullIdent = ident { "." ident }
strLit = ( "'" { charValue } "'" ) | ( '"' { charValue } '"' )
charValue = hexEscape | octEscape | charEscape | /[^\0\n\\]/
hexEscape = '\' ( "x" | "X" ) hexDigit hexDigit
octEscape = '\' octalDigit octalDigit octalDigit
charEscape = '\' ( "a" | "b" | "f" | "n" | "r" | "t" | "v" | '\' | "'" | '"' )
Or in plain English, an option may be assigned a dotted.notation.identifier, an integer, a float, a boolean, or a single- or double-quoted string, which MUST NOT have "raw" newline characters.
And yet, I'm encountering .proto files in various projects such as grpc-gateway and googleapis, where the rhs of the assignment is not quoted and spans multiple lines. For example in googleapis/google/api/http.proto there is this service definition in a comment block:
// service Messaging {
// rpc UpdateMessage(Message) returns (Message) {
// option (google.api.http) = {
// patch: "/v1/messages/{message_id}"
// body: "*"
// };
// }
// }
In other files, the use of semicolons (and occasionally commas) as separators seems somewhat arbitrary, and I have also seen keys repeated, which in JSON or JavaScript would result in loss of data due to overwriting.
Are there any canonical extensions to the language specification, or are people just Microsofting? (Yes, that's a verb now.)
I posted a similar question on the Protocol Buffers Google Group, and received a private message from a fellow at Google stating the following
This syntax is correct and valid for setting fields on a proto option field which is itself a field referencing a message type. This form is based on the TextFormat spec which I'm unclear if its super well documented, but here's an implementation of it: https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.text_format
When I have time, I will try to unpack what I learn from analyzing TextFormat.
update
I received an answer on the Groups forum
I think for better or worse, "what protoc implements" takes precedence over whatever the spec says. The spec came later and as far as I know we have not put a lot of effort into ensuring that it comprehensively matches the format that protoc expects. I believe the syntax you are looking at is missing from the .proto file format spec but is mentioned here as the "aggregate syntax."
The link above is to a section titled Custom Options in the Language Guide (proto2) page. If you scroll all the way to the end of that section, there is the following snippet that mentions TextFormat:
message FooOptions {
optional int32 opt1 = 1;
optional string opt2 = 2;
}
extend google.protobuf.FieldOptions {
optional FooOptions foo_options = 1234;
}
// usage:
message Bar {
optional int32 a = 1 [(foo_options).opt1 = 123, (foo_options).opt2 = "baz"];
// alternative aggregate syntax (uses TextFormat):
optional int32 b = 2 [(foo_options) = { opt1: 123 opt2: "baz" }];
}
In Go, when assigning multiple values to an array, braces {....} is used. What is this braces? Is it anonymous struct?
package main
import "fmt"
func main() {
var string_array [4]string = [4]string {"X", "Y", "Z", "W"}
var int_array [5]int = [5]int {1,2,3}
fmt.Println(string_array)
fmt.Println(int_array)
}
{"X", "Y", "Z", "W"} is the same as below and Go runtime is doing an implicit conversion?
type anonymous struct {
_0 string
_1 string
_2 string
_3 string
}
var anon anonymous = anonymous{"X", "Y", "Z", "W"}
Why not using ["X", "Y", "Z", "W"] if it is an array?
multiple assignment from array or slice
Go array initialization
Where in the Golang specification is this syntax explained?
It's documented in the Go language specification under "Composite literals" (emphasis mine):
Composite literals construct values for structs, arrays, slices, and maps and create a new value each time they are evaluated. They consist of the type of the literal followed by a brace-bound list of elements. Each element may optionally be preceded by a corresponding key.
Where "brace-bound" refers to the values being delimited with { and } as per your posted code.
From the grammar spec:
CompositeLit = LiteralType LiteralValue .
LiteralType = StructType | ArrayType | "[" "..." "]" ElementType |
SliceType | MapType | TypeName .
LiteralValue = "{" [ ElementList [ "," ] ] "}" . <-- This production-rule, right here
ElementList = KeyedElement { "," KeyedElement } .
KeyedElement = [ Key ":" ] Element .
Key = FieldName | Expression | LiteralValue .
FieldName = identifier .
Element = Expression | LiteralValue .
I am trying to understand this Elm construct:
type Item = Item { name : String, data : String }
It resembles a record, but it behaves very differently.
It is useful for defining recursive data models.
Unlike type alias Item = {...}, it does not provide a "constructor".
I cannot find it in Elm Syntax guides.
I cannot figure out how to access its fields:
> item = Item { name = "abc", data = "def" }
Item { name = "abc", data = "def" } : Repl.Item
> item.name
-- TYPE MISMATCH --------------------------------------------- repl-temp-000.elm
`item` does not have a field named `name`.
6| item.name
^^^^^^^^^ The type of `item` is:
Item
Which does not contain a field named `name`.
How is this construct called?
How do you access the contained fields?
It is a Union Type with a single constructor which happens to take a Record as its only type parameter.
The fact that the type name and constructor name are both Item is a common idiom, but holds no significance. It could easily be any other valid constructor name:
type Item = Foo { name : String, data : String }
For practical purposes, it can be useful to use a type alias for the internal record type so you can more succinctly pull values out. If you move things around a little bit:
type alias ItemContents = { name : String, data : String }
type Item = Item ItemContents
You could provide a function that returns the internal contents:
getItemContents : Item -> ItemContents
getItemContents (Item contents) = contents
And now it could be used like this REPL example:
> item = Item { name = "abc", data = "def" }
Item { name = "abc", data = "def" } : Repl.Item
> contents = getItemContents item
{ name = "abc", data = "def" } : Repl.ItemContents
> contents.name
"abc" : String
The Official documentation about map type says:
map<key_type, value_type> map_field = N;
...where the key_type can be any integral or string type (so, any
scalar type except for floating point types and bytes). The value_type
can be any type.
I want to define a map<string, repeated string> field, but it seems illegal on my libprotoc 3.0.0, which complains Expected ">". So I wonder if there is any way to put repeated string into map.
A Possible workaround could be:
message ListOfString {
repeated string value = 1;
}
// Then define:
map<string, ListOfString> mapToRepeatedString = 1;
But ListOfString here looks redundant.
I had the same need, and got the same error. I do not believe this is possible. Here is the relevant BNF definitions from the language specification.
https://developers.google.com/protocol-buffers/docs/reference/proto3-spec
messageType = [ "." ] { ident "." } messageName
mapField = "map" "<" keyType "," type ">" mapName "=" fieldNumber [ "["fieldOptions "]" ] ";"
type = "double" | "float" | "int32" | "int64" | "uint32" | "uint64"
| "sint32" | "sint64" | "fixed32" | "fixed64" | "sfixed32" | "sfixed64"
| "bool" | "string" | "bytes" | messageType | enumType
messageName = ident
ident = letter { letter | decimalDigit | "_" }
field = [ "repeated" ] type fieldName "=" fieldNumber [ "[" fieldOptions "]" ] ";"
"repeated" keyword only appears in the field definition. The map definition requires a "type", which does not include the repeated keyword.
That means there are a few options.
You could create a wrapper around the repeated value as you indicated.
There is the older way people defined define maps, which is more burdensome but is equivalent. This is the backwards compatible example from the language guide.
https://developers.google.com/protocol-buffers/docs/proto3#maps
message MapFieldEntry {
key_type key = 1;
repeated value_type value = 2;
}
repeated MapFieldEntry map_field = N;
You would need to convert the data to a map yourself, but this should be fairly trivial in most languages. In Java:
List<MapFieldEntry> map_field = // Existing list from protobuf.
Map<key_type, List<value_type>> = map_field.stream()
.collect(Collectors.toMap(kv -> kv.key, kv -> kv.value));
Use google.protobuf.ListValue
https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#listvalue
This is an untyped list collection from their well known types.
I think it should be as follows.
message ListOfString {
repeated string what_ever_name = 1;
}
// Then define:
map<string, ListOfString> what_ever_name = 1;
Remember what_ever_name should be same in both places.