Get list of proto messages - protocol-buffers

I'm a newbie to protobufs and I can't seem to get it.
I have a proto file like this.
message Address{
string Home=1;
State state=2;
string Name=3;
enum State{
STATE_UNKNOWN=0;
STATE_ARCHIVED=1;
}
}
And I've added data to the message that looks like this.
Address{
Home:"Cornfield";
State: STATE_UNKNOWN;
Name:"Corner";
}
Address{
Home:"Ham";
State: STATE_UNKNOWN;
Name:"Hammer";
}
data = Address.getfielddescriptor()
The field descriptor method can't return a list of values like
data=['Cornfield','Ham']
How would I do this?

For you to be able to use a list you need to define a field as repeated. So somewhere you need to define something like an Address Book where you store all your addresses:
message Address {
string home = 1;
State state = 2;
string name = 3;
enum State {
STATE_UNKNOWN = 0;
STATE_ARCHIVED = 1;
}
}
// Your address book message
message AddressBook {
repeated addresses= 1;
}
Next in python you use this as followed:
address_book = AddressBook()
addr = address_book.addresses.add()
addr.home = "Cornfield"
addr.state = STATE_UNKNOWN
addr.name = "Corner"
# You can also first create an address object and extend the list
addr2 = Address()
addr2.home = "Ham"
addr2.state = STATE_UNKNOWN
addr2.name = "Hammer"
address_book.addresses.extend(addr2)
# You can use the list of addresses like any other list:
# By index: address_book.addresses[0]
# or in a list: for addr in address_book.addresses:
Other methods of extending the address book can be found in the protobuf documentation here.

Related

How to organize proto file to re-use message if possible?

I recently started working with protobuf in my golang project. I created below simple protobuf file. I have three different endpoints.
GetLink takes CustomerRequest as an input parameter and returns back CustomerResponse
GetBulkLinks takes BulkCustomerRequest as an input parameter and returns back BulkCustomerResponse
StreaLinks takes StreamRequest as an input parameter and returns back CustomerResponse
I am wondering if there is any way we can improve below proto file because CustomerRequest and BulkCustomerRequest has mostly everything in common except resources field so there is a duplication. And same goes with StreamRequest input parameter as it only takes clientId as the input parameter. Is there anything in protocol buffer which can reuse stuff from another message type?
Is there any better or efficient way to organize below proto file which reuses message accordingly?
syntax = "proto3";
option go_package = "github.com/david/customerclient/gen/go/data/v1";
package data.v1;
service CustomerService {
rpc GetLink(CustomerRequest) returns (CustomerResponse) {};
rpc GetBulkLinks(BulkCustomerRequest) returns (BulkCustomerResponse) {};
rpc StreaLinks(StreamRequest) returns (CustomerResponse) {};
}
message CustomerRequest {
int32 clientId = 1;
string resources = 2;
bool isProcess = 3;
}
message BulkCustomerRequest {
int32 clientId = 1;
repeated string resources = 2;
bool isProcess = 3;
}
message StreamRequest {
int32 clientId = 1;
}
message CustomerResponse {
string value = 1;
string info = 2;
string baseInfo = 3;
string link = 4;
}
message BulkCustomerResponse {
map<string, CustomerResponse> customerResponse = 1;
}
Is there anything in protocol buffer which can reuse stuff from another message type?
Composition.
However keep in mind that request and response payloads may change over time. Even if it looks like they have something in common today, they may diverge tomorrow. After all they are used in different RPCs. Then excessive coupling achieves the opposite effect and becomes technical debt.
Since your schema has literally no more than three fields for each message, I would leave everything as is. Anyway, if you really must, you may consider the following:
Extract the GetLink and GetBulkLinks common fields in a separate message and compose with that:
message CustomerRequestParams {
int32 clientId = 1;
bool isProcess = 2;
}
message CustomerRequest {
CustomerRequestParams params = 1;
string resources = 2;
}
message BulkCustomerRequest {
CustomerRequestParams params = 1;
repeated string resources = 2;
}
StreamRequest looks just fine with repeating clientId. Arguably a stream is conceptually different from a unary RPC, so just keep them separated. And it's just one field.

In a proto3 message is there a way to mark a field as not required for requests and required for response?

I have the following proto3 message structure:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
google.protobuf.Timestamp createTime = 4;
}
I am using the "same" structure for some requests and responses on my app. What I want to achieve is to mark somehow (if possible) the createTime field as not required, in case we are talking about a request object, and required in case we are taking about a response object.
Is it possible to do this without creating a separate message ?
Thanks
To my knowledge, it's not possible and I'd discourage pursuing solutions other than defining distinct message types: one which includes the optional field and one which does not.
One way to solve this is to define a message that includes the mandatory fields and another than extends it:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
}
message SomeRequest {
BaseBuildContent content = 1;
}
message SomeResponse {
BaseBuildContent content = 1;
google.protobuf.Timestamp createTime = 2;
}
NOTE Protobuf style guide recommends message names be PascalCased and field names be snake_cased.

Sending ListBlog response in unary streaming

I am trying to return list of items from grpc without streaimg API and not finding any solution or example , I know we use this kind of proto file when using streaming
message ListBlogResponse {
Blog blog = 1;
}
and then Blog
message Blog {
string id = 1;
string author_id = 2;
string title = 3;
string content = 4;
}
but I want to send response once without using any streaming some thing like this :
return &api.ListBlogResponse{
Blog: items,
}, nil
what will be the protofile for this ?
You need a message containing multiple blogs:
message BlogEntries {
repeated Blog blog = 1;
}
Then you can return:
return &BlogEntries{Blog:entries},nil

Terraform - Adding Validation for type = map(object()) in variables.tf

First thanks for this post Adding a default field for type = map(object()) in variavles.tf, this answered the first part of the struggle that I had in getting default values to work with type map(object()). The last part that I am trying to get working is how to do validation of the input values.
terraform {
experiments = [module_variable_optional_attrs]
}
variable "dns_server" {
description = "Add DNS Servers for domain resolution. You can configure a maximum of two servers. Only one can be preferred 'true'."
type = map(object({
preferred = optional(bool)
server = optional(string)
}))
default = {
default = {
preferred = false
server = "198.18.1.1"
}
}
validation {
condition = (
can(regexall("^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}$", var.dns_server["server"]))
)
error_message = "The DNS Server is not a valid IPv4 Address."
}
}
locals {
dns_server = {
for k, v in var.dns_server : k => {
preferred = coalesce(v.preferred, false)
server = coalesce(v.server, "198.18.1.1")
}
}
}
The defaults value in the variable field I know isn't used but I am using it as a placeholder for the terraform docs output.
I also know what I have above for validation is not correct because if the user used the default server IPv4 that wouldn't be set until the locals definition. I just don't know of a way to do the validation because I my trusty google search hasn't pulled up any similar examples.
The code is located here if you need more details about how it is being used:
https://github.com/scotttyso/terraform-aci-fabric/tree/main/test
If I comment out the validation everything else is working fine. Thanks in advance.
Is this what you are after?
variable "mapobject" {
type = map(object({
cidr_block = string
destination_type = string
}
))
validation {
condition = alltrue([
for o in var.mapobject : contains(["CIDR_BLOCK","NETWORK_SECURITY_GROUP","SERVICE_CIDR_BLOCK"],o.destination_type)]) error_message = "All destination_types must be one of CIDR_BLOCK,NETWORK_SECURITY_GROUP or SERVICE_CIDR_BLOCK!"
}
}
With a variable assignment of
mapobject = {
"r0" = {cidr_block = "10.1.1.0/24",destination_type = "CIDR_BLOCK" }
}
The validation succeeds, where as the following fails (as required)
mapobject = {
"r0" = {cidr_block = "10.1.1.0/24",destination_type = "CIRD_BLOCK" }
}
Error: Invalid value for variable
on main.tf line 86:
86: variable "mapobject" {
All destination_types must be one of CIDR_BLOCK,NETWORK_SECURITY_GROUP or
SERVICE_CIDR_BLOCK!
This was checked by the validation rule at main.tf:93,2-12.
If it is, then kudos goes here: https://discuss.hashicorp.com/t/validate-list-object-variables/18291/2

How do I read the Received Date from Outlook MSG files -without- the Outlook API?

I need to read stuff from an Outlook msg file. Currently I'm using a class from CodeProject.com project to accomplish this, since deploying VSTO and Outlook on a server is not an option.
This class gets To, From, CC, Subject, Body, and everything else I need from the msg file, except Date information (such as Received Date and Sent Date).
There is some (really, really low-level) documentation on how to get stuff out of msg files on MSDN, but it's a little beyond the scope of this project and doesn't mention dates at all.
Ideally I'd be able to have a drop-in replacement for the class I am using now (OutlookStorage.cs in the previously mentioned CodeProject) or be able to modify the existing class a bit. To modify, I would need the correct 4 character hexidecimal prop identifier for received date. For instance, Subject is listed as PR_SUBJECT = "0037" and Body is listed as PR_BOY = "1000".
If you're using OutlookStorage.cs from CodeProject, then add the following:
private const string PR_RECEIVED_DATE="007D";
private const string PR_RECEIVED_DATE_2 = "0047";
...
/// <summary>
/// Gets the date the message was received.
/// </summary>
public DateTime ReceivedDate
{
get
{
if (_dateRevieved == DateTime.MinValue)
{
string dateMess = this.GetMapiPropertyString(OutlookStorage.PR_RECEIVED_DATE);
if (String.IsNullOrEmpty(dateMess))
{
dateMess = this.GetMapiPropertyString(OutlookStorage.PR_RECEIVED_DATE_2);
}
_dateRevieved = ExtractDate(dateMess);
}
return _dateRevieved;
//return ExtractDate(dateMess);
}
}
private DateTime _dateRevieved = DateTime.MinValue;
private DateTime ExtractDate(string dateMess)
{
string matchStr = "Date:";
string[] lines = dateMess.Split(new String[] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);
foreach (string line in lines)
{
if (line.StartsWith(matchStr))
{
string dateStr = line.Substring(matchStr.Length);
DateTime response;
if (DateTime.TryParse(dateStr, out response))
{
return response;
}
}
}
return DateTime.MinValue;
}
I think the Aspose library will do what you want, ok it a 3rd party lib so may not be what you want. There are a few vbs scripts around that get basic infomation out of msg files that could be translated.
Got a hint from this:
string fullFileName = "c:\message.msg";
DateTime dateRevieved = new DateTime();
StreamReader sr = new StreamReader(fullFileName, Encoding.Default);
string full = sr.ReadToEnd();
string date;
int iStart;
int iLast;
string caption;
//This -should- handle all manner of screwage
//The ONLY way it would not is if someone guessed the -exact- to-the-second
//time that they send the message, put it in their subject in the right format
while (true) { //not an infinite loop, I swear!
caption = "Date:";
if (full.IndexOf("Date:") > -1) { //full shortens with each date is removed
string temp = "";
iStart = full.LastIndexOf(caption);
temp = full.Remove(0, iStart + caption.Length);
full = full.Substring(0, iStart);
iLast = temp.IndexOf("\r\n");
if (iLast < 0) {
date = temp;
} else {
date = temp.Substring(0, iLast);
}
date = date.Trim();
if (date.Contains(subject) || subject.Contains(date)) {
continue; //would only happen if someone is trying to screw me
}
try {
dateRevieved = DateTime.Parse(date); //will fail if not a date
break; //if not a date breaks out of while loop
} catch {
continue; //try with a smaller subset of the msg
}
} else {
break;
}
}
This is kind of a hack compared to the ways you can get other things from msg files using something this lovely project. Still, it's stood up to everything I have thrown against it, and as noted the -only- way to fool it is to put the exact to-the-second date in the subject line in the proper format.
to combine your two posts I would suggest the following solution:
To modify, I would need the correct 4 character hexidecimal prop identifier for recieved date. For instance, Subject is listed as PR_SUBJECT = "0037" and Body is listed as PR_BOY = "1000".
Look for "007D".
Use the method you posted in your second post on the received data to eliminate the problem when the same (date) string is inside the subject.
I have to mention that this method doesn't seem to work on internal eMails: In mails I receive from colleagues, there is no substg1.0_007Dxxxx-Property.
Here, the date seems to be hidden in substg1.0_0047xxxx.
All the best!
inno

Resources