protobuf map value is object or array of object - protocol-buffers

I have
message {
message leadsEntity{
optional string name = 1;
}
optional string description = 1;
map <string, leadsEntity> entity = 2;
}
Now I can have 2 types of responses-
// 1st case
{
description : "this is test",
entity : {
key1 : { name : "name1" },
key2 : { name : "name2" }
}
}
Or
// 2nd case
{
description : "this is test",
entity : {
key1 : { name : "name1" },
key2 : [ { name : "name2" } ] //the value in array now
}
}
Now the above given proto message works perfectly for 1st case.
But how to handle 2nd case? where the value is an object array?

I assume by the use of optional that you prefer proto2
Your existing proto is invalid.
Your response types are going to make your life difficult because the entity values are a union of types.
Perhaps something like this:
syntax = "proto2";
message X {
message LeadsEntity{
optional string name = 1;
}
message LeadsEntities {
repeated LeadsEntity leads_entity = 1;
}
message Value {
oneof These {
LeadsEntity leads_entity = 1;
LeadsEntities leads_entities = 2;
}
}
optional string description = 1;
map <string, Value> entity = 2;
}
I gave the root message the name X
I added LeadsEntities to hold the repeated LeadsEntity
I added Value to hold the oneof
from google.protobuf.json_format import MessageToJson
import x_pb2
x = x_pb2.X()
x.description = "this is a test"
x.entity["key1"].leads_entity.name = "name1"
le = x.entity["key2"].leads_entities.leads_entity.add()
le.name = "name2"
print(MessageToJson(x))
Yields:
{
"description": "this is a test",
"entity": {
"key1": {
"leadsEntity": {
"name": "name1"
}
},
"key2": {
"leadsEntities": {
"leadsEntity": [
{
"name": "name2"
}
]
}
}
}
}

Related

Convert string in struct to []string

I have a struct as below:
type TourData struct {
ArtistID int //artist ID
RelationID string //key for relations
City string
Country string
TourDates []string
}
type MyRelation struct {
ID int `json:"id"`
DatesLocations map[string][]string `json:"datesLocations"`
}
which contains this data from a csv file:
1,nagoya-japan,Nagoya,Japan,
1,penrose-new_zealand,Penrose,New_Zealand,
1,dunedin-new_zealand,Dunedin,New_Zealand,
2,playa_del_carmen-mexico,Playa Del Carmen,Mexico,
2,papeete-french_polynesia,Papeete,French_Polynesia,
MyRelations is populated from an API which contains:
"index": [
{
"id": 1,
"datesLocations": {
"dunedin-new_zealand": [
"10-02-2020"
],
"nagoya-japan": [
"30-01-2019"
],
"penrose-new_zealand": [
"07-02-2020"
]
}
},
{
"id": 2,
"datesLocations": {
"papeete-french_polynesia": [
"16-11-2019"
],
"playa_del_carmen-mexico": [
"05-12-2019",
"06-12-2019",
"07-12-2019",
"08-12-2019",
"09-12-2019"
]
}
}
The dates come from another struct. The code I have used to populate this struct is as below:
var oneRecord TourData
var allRecords []TourData
for _, each := range csvData {
oneRecord.ArtistID, _ = strconv.Atoi(each[0])
oneRecord.RelationID = each[1]
oneRecord.City = each[2]
oneRecord.Country = each[3]
oneRecord.TourDates = Relations.Index[oneRecord.ArtistID-1].DatesLocations[each[1]]
allRecords = append(allRecords, oneRecord)
}
jsondata, err := json.Marshal(allRecords) // convert to JSON
json.Unmarshal(jsondata, &TourThings)
I need to group all the 1s together then the 2s etc. I thought to create another struct, and populate from this one but not having much luck - any ideas?
To clarify I would want say TourData.City to equal:
[Nagoya,Penrose,Dunedin]
[Playa Del Carmen, Papeete]
At the moment if I was to print TourData[0].City I would get Nagoya.
I have tried creating another struct to be populated from the TourData struct with the following fields:
type TourDataArrays struct {
ArtistID int
City []string
Country []string
TourDates [][]string
}
and then populate the struct using the code below:
var tourRecord TourDataArrays
var tourRecords []TourDataArrays
for i := 0; i < len(Relations.Index); i++ {
for j := 0; j < len(allRecords); j++ {
if allRecords[i].ArtistID == i+1 {
tourRecord.City = append(tourRecord.City, allRecords[j].City)
}
}
tourRecords = append(tourRecords, tourRecord)
}
However this is adding all the cities to one array i.e
[Nagoya, Penrose, Dunedin, Playa Del Carmen, Papeete].
If I understand your requirements correctly you needed to declare city as a string array as well. (And Country to go with it).
Check out this solution : https://go.dev/play/p/osgkbfWV3c5
Note I have not deduped country and derived city and country from one field in the Json.

elasticsearch sort by price with currency

I have data
{
"id": 1000,
"price": "99,01USA",
},
{
"id": 1001,
"price": "100USA",
},
{
"id": 1002,
"price": "780USA",
},
{
"id": 1003,
"price": "20USA",
},
How I sort order by price (ASC , DESC)
You can alter it a little to parse price to integer and then sort it
You can create a dynamic sort function that sorts objects by their value that you pass:
function dynamicSort(property) {
var sortOrder = 1;
if(property[0] === "-") {
sortOrder = -1;
property = property.substr(1);
}
return function (a,b) {
/* next line works with strings and numbers,
* and you may want to customize it to your needs
*/
var result = (a[property] < b[property]) ? -1 : (a[property] > b[property]) ? 1 : 0;
return result * sortOrder;
}
}
So you can have an array of objects like this:
var People = [
{Name: "Name", Surname: "Surname"},
{Name:"AAA", Surname:"ZZZ"},
{Name: "Name", Surname: "AAA"}
];
...and it will work when you do:
People.sort(dynamicSort("Name"));
People.sort(dynamicSort("Surname"));
People.sort(dynamicSort("-Surname"));
Actually this already answers the question. Below part is written because many people contacted me, complaining that it doesn't work with multiple parameters.
Multiple Parameters
You can use the function below to generate sort functions with multiple sort parameters.
function dynamicSortMultiple() {
/*
* save the arguments object as it will be overwritten
* note that arguments object is an array-like object
* consisting of the names of the properties to sort by
*/
var props = arguments;
return function (obj1, obj2) {
var i = 0, result = 0, numberOfProperties = props.length;
/* try getting a different result from 0 (equal)
* as long as we have extra properties to compare
*/
while(result === 0 && i < numberOfProperties) {
result = dynamicSort(props[i])(obj1, obj2);
i++;
}
return result;
}
}
Which would enable you to do something like this:
People.sort(dynamicSortMultiple("Name", "-Surname"));
Subclassing Array
For the lucky among us who can use ES6, which allows extending the native objects:
class MyArray extends Array {
sortBy(...args) {
return this.sort(dynamicSortMultiple(...args));
}
}
That would enable this:
MyArray.from(People).sortBy("Name", "-Surname");

Run Elasticsearch processor on all the fields of a document

I am trying to trim and lowercase all the values of the document that is getting indexed into Elasticsearch
The processors available has the field key is mandatory. This means one can use a processor on only one field
Is there a way to run a processor on all the fields of a document?
There sure is. Use a script processor but beware of reserved keys like _type, _id etc:
PUT _ingest/pipeline/my_string_trimmer
{
"description": "Trims and lowercases all string values",
"processors": [
{
"script": {
"source": """
def forbidden_keys = [
'_type',
'_id',
'_version_type',
'_index',
'_version'
];
def corrected_source = [:];
for (pair in ctx.entrySet()) {
def key = pair.getKey();
if (forbidden_keys.contains(key)) {
continue;
}
def value = pair.getValue();
if (value instanceof String) {
corrected_source[key] = value.trim().toLowerCase();
} else {
corrected_source[key] = value;
}
}
// overwrite the original
ctx.putAll(corrected_source);
"""
}
}
]
}
Test with a sample doc:
POST my-index/_doc?pipeline=my_string_trimmer
{
"abc": " DEF ",
"def": 123,
"xyz": false
}

Terraform output object with multiple attributes for each of `for` resources?

I have terraform with a resource being created with for. As is typical, each instance of this resource has several attributes. At the moment I have a series of map outputs for this resource group but each consists of only a single key-value pair. I would like my terraform output to include a list or map of maps or objects with all of the attributes grouped by resource instance. How do I do this without using flatten; zipmap etc to construct them from my current outputs? This example is with aws_route53_record but this is a generic query:
Current code
output "r53record_zonal_fqdn" {
value = {
for entry in aws_route53_record.zonal :
entry.name => entry.fqdn
}
}
output "r53record_zonal_records" {
value = {
for entry in aws_route53_record.zonal :
entry.name => entry.records
}
}
output "r53record_zonal_zone_id" {
value = {
for entry in aws_route53_record.zonal :
entry.name => entry.zone_id
}
}
As you would expect, this renders three maps with aws_route53_record.zonal.name as the key and the other attribute(s) as the value.
What I would like is to have these outputs grouped by resource with a predefined key for each value, e.g. (pseudocode):
output "r53record_zonal_zone_id" {
value = {
for entry in aws_route53_record.zonal : {
instance[count.index] {
"name" = entry.name
"fqdn" = entry.fqdn
"records" = entry.records
"zone_id" = entry.zone_id
}
}
}
}
Producing a map or list of maps for each instance.
How can this or something like it be done?
I created a random route53_record resource block with two "name" arguments in for_each loop and tried to output something close to what you were looking for.
Assuming "mydomain.com" is the domain in Route53 as example....
resource "aws_route53_record" "zonal" {
for_each=toset(["site1","site2"])
name = each.key
zone_id = "ABCDZONEIDSTRING"
type = "A"
ttl = "300"
records = ["192.168.1.10"]
}
output "test" {
value = {
for dns, details in aws_route53_record.zonal:
dns => ({"fqdn" = details.fqdn , "zone_id" = details.zone_id , "records" = details.records})
}
}
this will create output in this fashion..
test = {
"site1" = {
"fqdn" = "site1.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "Z0630117NTQNSYTXYQ4Z"
}
"site2" = {
"fqdn" = "site2.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "Z0630117NTQNSYTXYQ4Z"
}
}
Suppose you passed the name values with domain name, like below...
for_each=toset(["site1.mydomain.com","site2.mydomain.com"])
the output would look like
test = {
"site1.mydomain.com" = {
"fqdn" = "site1.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "ABCDMYZONEIDSTRING"
}
"site2.mydomain.com" = {
"fqdn" = "site2.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "ABCDMYZONEIDSTRING"
}
}

How to check an empty JSONArray in swiftyJSON

I have a JSON that have a JSONArray as a value in one of the json inside it. here is the example of it.
[
{
"id": 1,
"symptoms" : [{\"key\":\"sample1\",\"value\":5},{\"key\":\"sample2\",\"value\":5}]
},
{
"id": 2,
"symptoms" : [{\"key\":\"sample3\",\"value\":1}]
},
{ "id": 3,
"symptoms" : []
},
{
"id": 4,
"symptoms": [{\"key\":\"sample4\",\"value\":1}]
}
]
So what I am doing is that I am parsing the inner JSON and place it in a String Array. But whenever I look up to symptoms it skips the empty JSONArray. So whenever i print the String Array it goes like this (with the given sample on top) ["sample1", "sample2", "sample3", "sample4"]. But i want to do is to append an "" to the String Array whenever the JSONArray is empty so it should be like this ["sample1", "sample2", "sample3", "", "sample4"]. Anyone can help me with this? Here is my code
var arrayHolder: [String] = []
var idHolder: [Int] = []
for item in swiftyJSON.arrayValue {
idHolder.append(item["id"].intValue)
//for the inner JSON
let innerJSON = JSON(data: item["symptoms"].dataUsingEncoding(NSUTF8StringEncoding)!)
for symptoms in innerJSON.arrayValue {
arrayHolder.append(symptoms["key"].stringValue)
}
}
print(idHolder) // [1,2,3,4]
print(arrayHolder) // ["sample1","sample2","sample3","sample4"]
Just check if innerJSON is empty:
for item in swiftyJSON.arrayValue {
idHolder.append(item["id"].intValue)
//for the inner JSON
let innerJSON = item["symptoms"].arrayValue // non need to create a new JSON object
if innerJSON.isEmpty {
arrayHolder.append("")
} else {
for symptoms in innerJSON {
arrayHolder.append(symptoms["key"].stringValue)
}
}
}

Resources