Protobuf memory error when adding 6 array messages - protocol-buffers

I am getting this error: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 67112960 bytes) in C:\xampp\htdocs\... when i adding 6 array message in my proto file. I am using php protobuf protocol buffers for serialization.
syntax = "proto3";
message data {
.data_step_one test = 2;
}
message data_step_one {
.data_step_two test1 = 1;
}
message data_step_two {
.data_step_three test2 = 2;
}
message data_step_three {
.data_step_four test3 = 2;
}
message data_step_four {
.data_step_five test4 = 14;
}
message data_step_five {
.data_step_six test5 = 1;
}
message data_step_six {
string something = 1;
}
When i remove string (string something = 1;) from last six array
message data_step_six {
}
its worked without that error. Just i got that error when i add something in six message. I am new in this protobuf message, so don't understand where was the problem and how will i solve it.
Updated More:
This is my original data for post method (I followed this for create proto messages):
1 {
1: 1
2: 58480067980758310
4: 0
}
2 {
1 {
1: 112
2 {
1: id:a-7769307577821621420
2 {
14 {
1 {
1: mid:b-6430467773153509964
2 {
1: 1
2: mcacc.com
3: Emma Ava
10: mcacc.com
}
3 {
1: 1
2: scssd.com
}
7: 1627577996674
8: Test
9 {
2 {
1: 0
2: <div dir='auto'>This is test.</div>
}
7: 1
}
11: ^a
11: ^all
11: ^f_bt
11: ^f_cl
11: ^f_btns
11: ^pfg
18: 1627577996674
36 {
6: 0
}
37 {
4: 0
}
42: 0
52: s:23cbdb9f4cd2713a|#mid:b-6430467773153509964|0
}
3: 1
}
}
}
}
}
3 {
1: 4
2: 518600
5 {
1: 30
2: 0
}
7: 1
8 {
1: ^wa_ready
1: ^f
1: ^t_z
2: ^r
}
}
4 {
2: 1
3: 1627577996728
4: 1
5: 57
}
5: 2
This is proto message i created:
syntax = "proto3";
message rs_request {
.unknown_int_data unknown_int = 1;
.unknown_int_data1 unknown_int1 = 2;
.unknown_int_data2 unknown_int2 = 3;
.unknown_int_data3 unknown_int3 = 4;
bool unknown_int4 = 5;
}
message unknown_int_data {
bool unint = 1;
fixed32 unknown = 2;
bool unknown1 = 4;
}
message unknown_int_data1 {
.unknown_int_datas unknown2 = 1;
}
message unknown_int_datas {
int32 unknown3 = 1;
.unknown_int_datas1 unknown4 = 2;
}
message unknown_int_datas1 {
string threadaId = 1;
.unknown_int_datass unknowns = 2;
}
message unknown_int_datass {
.unknown_int_datas2 unknown5 = 14;
}
message unknown_int_datas2 {
.unknown_int_datas3 unknown6 = 1;
bool unknown7 = 3;
}
message unknown_int_datas3 {
string msgaId = 1;
.unknown_int_datas4 unknown9 = 2;
.unknown_int_datas5 unknown10 = 3;
fixed32 unknown11 = 7;
string subject = 8;
.unknown_int_datas6 unknown12 = 9;
string unknown13 = 11;
fixed32 unknown14 = 18;
.unknown_int_datas7 unknown15 = 36;
.unknown_int_datas8 unknown16 = 37;
int32 unknown17 = 42;
string unknown18 = 52;
}
message unknown_int_datas4 {
bool unknown19 = 1;
string from_domain = 2;
string from_name = 3;
string from_domain_again = 10;
}
message unknown_int_datas5 {
bool unknown20 = 1;
string to_domain = 2;
}
message unknown_int_datas6 {
.unknown_int_datas9 unknown21 = 2;
bool unknown22 = 7;
}
message unknown_int_datas9 {
bool unknown23 = 1;
string msg = 2;
}
message unknown_int_datas7 {
bool unknown24 = 6;
}
message unknown_int_datas8 {
bool unknown25 = 4;
}
message unknown_int_data2 {
bool unknowsthree = 1;
uint32 unknowsthree1 = 2;
.unknowsthree_data unknowsthree2 = 5;
bool unknowsthree3 = 7;
.unknowsthree_datas unknowsthree4 = 8;
}
message unknowsthree_data {
int32 unknowsthree5 = 1;
bool unknowsthree6 = 2;
}
message unknowsthree_datas {
string unknowsthree7 = 1;
string unknowsthree8 = 2;
}
message unknown_int_data3 {
bool unknown26 = 2;
fixed32 unknown27 = 3;
bool unknown28 = 4;
int32 unknown29 = 5;
}
and this is request model json for post (mcRequestModel.php):
"unknown_int1" => [
"unknown2" => [
"unknown4" => [
"unknowns" => [
"unknown5" => [
"unknown6" => [
"unknown9" => [
"from_domain" => "mcacc.com",
"from_name" => "Emma Ava",
"from_domain_again" => "mcacc.com"
],
"unknown10" => [
"to_domain" => "scssd.com"
],
"subject" => "Test",
"unknown12" => [
"unknown21" => [
"msg" => "<div dir='auto'>This is test.</div>"
]
]
]
]
]
]
]
]
Trying to get buffer compiler using php-protobuf:
$mcRequest = new mcRequestModel();
$class = new \rs_request();
$class->parseFromJsonStream($mcRequest);
var_dump($class);
When i remove all data from array unknown_int_datas3 i got buffer compiler arrays larger output but not getting compiler output when not remove data from unknown_int_datas3and just getting Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 67112960 bytes) in C:\xampp\htdocs\...

It's possible that the compiler is challenged by the levels of nesting in your messages.
I don't use PHP but I pasted your proto into Marc Gravell's protogen tool, selected PHP and it was able to compile.
The nesting in your messages is unusual and I was unable to understand why you would want to do this. Some context in your question to explain what you're trying to achieve may be useful.
With your current structure, in JSON, a data message would be:
{
test: {
test1: {
test2: {
test3: {
test4: {
test5: {
something: "...."
}
}
}
}
}
}
}
Google's style guide recommends PascalCase messages names (i.e. DataStepTwo) and snake_case field names (which you are doing test)
Your messages should probably use 1 consistently for the field numbers. Each of your messages only have 1 field so, unless you're intentionally avoiding collisions you may wish to avoid using 2 and 14.

Related

Protobuf different messages for the same field

is it possible to create a field that can hold different types of messages? Let's say I wanted to send a request to create a car, but it can have different types of windows in it, and each type requires different fields to be present:
message TintedWindows {
double tintPercent = 1;
string tintColor = 2;
}
message ArmoredWindows {
string armorClass = 1;
}
message CreateCarRequest {
string color = 1;
int wheels = 2;
int doors = 3;
Windows??? windows = 4;
}
Is there a way of doing that smoothly or I have to add all those messages and check which one is empty and which one is not:
message CreateCarRequest {
string color = 1;
int wheels = 2;
int doors = 3;
TintedWindows tinted = 4;
ArmoredWindows armored = 5;
}
then do: (pseudocode)
windowsInterface windows;
if req.tinted != null && req.armored == null {
windows = newTinted(req.tinted)
} else if req.tinted == null && req.armored != null {
windows = newArmored(req.armored)
} else {
throw error
}
I'm curently doing that using the second method and it's a bit cumbersome.
You can use Oneof.
message CreateCarRequest {
string color = 1;
uint32 wheels = 2;
uint32 doors = 3;
oneof windows {
TintedWindows tinted = 4;
ArmoredWindows armored = 5;
}
}
There is no int type (see scalar), you'll need to choose one of the available types, probably uint32 for number of wheels and doors.

How to represent a mix of enum and oneof in protobuf

I am trying to create a protocol buffer message with fields that are either a message or one of a choice of some constant (like an enum). Something that is logically equivalent to:
message Error {
oneof error_type {
EMPTY_METHOD_NAME = 0
ExecutionError execution_error = 1;
}
message ExecutionError {
string value = 1;
}
}
Essentially, I would like a field that can represent either an error type that is just a name with no fields, or an error type that has fields. How would I do this with protobuf3?
See Enumerations in the Language Guide (proto3)
message Error {
oneof error_type {
Foo foo = 1;
ExecutionError execution_error = 2;
}
enum Foo {
X = 0;
Y = 1;
Z = 2;
}
message ExecutionError {
string value = 1;
}
}

How to calculate multi-operator of list in dart?

I have a list of num and string of operator above:
final List<dynamic> items = [5.0, "-", 2, "*", 3];
I want to calculate the value inside of it. I can do like this:
final List<dynamic> items = [5.0, "-", 2, "*", 3]; //correct is -1
num result = 0;
String op = "+";
for (final item in items) {
if (item is String) {
op = item;
}
if (item is num) {
switch (op) {
case "+":
result = result + item;
break;
case "-":
result = result - item;
break;
case "*":
result = result * item;
break;
case "/":
result = result / item;
break;
}
}
}
print(result); // incorrect result: 9
As above code, If it is just an "+" or "-" operator it would return the correct answer, but because of the order of operators "/" and "*" doing like above return incorrect result.
Can anyone suggest any algorithm?
Here is one way of doing it, the code works but I didn't put the tests that would be necessary to check for bad input.
// functions for basic operations
num multiply(num leftOp, num rightOp) => leftOp * rightOp;
num divide(num leftOp, num rightOp) => leftOp / rightOp;
num add(num leftOp, num rightOp) => leftOp + rightOp;
num substract(num leftOp, num rightOp) => leftOp - rightOp;
void main() {
final List<dynamic> items = [
2,
"*",
5,
"/",
2,
"-",
3,
"+",
8
]; //correct is 10
num result = 0;
// Copy the list in a temporary list
var calc = [...items];
// set the precedence order of the operators
// create 2 groups of equal importance
var operators = [
{
"*": multiply,
"/": divide,
},
{
"+": add,
"-": substract,
}
];
// loop until all operators have produced result
while (calc.length > 1) {
for (var opPrecedence in operators) {
// find first operator in a group, starting from left
var pos = 0;
do {
pos = calc.indexWhere((e) => opPrecedence.containsKey(e));
if (pos >= 0) {
num leftOp = calc[pos - 1];
num rightOp = calc[pos + 1];
var operation = opPrecedence[calc[pos]];
result = operation!(leftOp, rightOp);
// remove the 2 operands and replace with result
calc.removeAt(pos);
calc.removeAt(pos);
calc[pos - 1] = result;
}
} while (pos >= 0);
}
}
// what should be left is the final result
print(calc[0]);
}
You can create your own simple calculator parser for this purpose and use it.
Building (generation) of the parser will take less than one second.
You can always modify it to suit your needs and generate an updated parser.
Below is an example of definition such a parser.
It contains (also for example) a value calculation for your data (as an expression).
This script will generate a parser into example/calculator.dart.
File tool/calculator_builder.dart:
import 'package:parser_builder/branch.dart';
import 'package:parser_builder/bytes.dart';
import 'package:parser_builder/char_class.dart';
import 'package:parser_builder/character.dart';
import 'package:parser_builder/combinator.dart';
import 'package:parser_builder/error.dart';
import 'package:parser_builder/expression.dart';
import 'package:parser_builder/fast_build.dart';
import 'package:parser_builder/parser_builder.dart';
import 'package:parser_builder/sequence.dart';
Future<void> main(List<String> args) async {
final context = Context();
return fastBuild(context, [_parse, _expression_], 'example/calculator.dart',
header: __header, publish: {'parse': _parse});
}
const __header = '''
import 'package:source_span/source_span.dart';
import 'package:tuple/tuple.dart';
void main() {
final List items = [5.0, "-", 2, "*", 3];
final source = items.join(' ');
final result = parse(source);
print(result);
}
num _calculate(num left, String operator, num right) {
switch (operator) {
case '+':
return left + right;
case '-':
return left - right;
case '*':
return left * right;
case '/':
return left / right;
case '~/':
return left ~/ right;
default:
throw StateError('Unknown operator: \$operator');
}
}
''';
const _additive = Named(
'_additive',
BinaryExpression(
_multiplicative, _additiveOperator, _multiplicative, _calculate));
const _additiveOperator =
Named('_additiveOperator', Terminated(Tags(['+', '-']), _ws));
const _calculate = ExpressionAction<num>(
['left', 'op', 'right'], '_calculate({{left}}, {{op}}, {{right}})');
const _closeParen = Named('_closeParen', Terminated(Tag(')'), _ws));
const _decimal = Named(
'_decimal',
Terminated(
Map1(
Recognize(
Tuple3(Digit1(), Tag('.'), Digit1()),
),
ExpressionAction<num>(['x'], 'num.parse({{x}})')),
_ws));
const _expression = Ref<String, num>('_expression');
const _expression_ = Named('_expression', _additive);
const _integer = Named(
'_integer',
Terminated(
Map1(Digit1(), ExpressionAction<num>(['x'], 'int.parse({{x}})')), _ws));
const _isWhitespace = CharClass('#x9 | #xA | #xD | #x20');
const _multiplicative = Named('_multiplicative',
BinaryExpression(_primary, _multiplicativeOperator, _primary, _calculate));
const _multiplicativeOperator =
Named('_multiplicativeOperator', Terminated(Tags(['*', '/', '~/']), _ws));
const _openParen = Named('_openParen', Terminated(Tag('('), _ws));
const _parse = Named('_parse', Delimited(_ws, _expression, Eof<String>()));
const _primary = Named(
'_primary',
Nested(
'expression',
Alt3(
_decimal,
_integer,
Delimited(_openParen, _expression, _closeParen),
)));
const _ws = Named('_ws', SkipWhile(_isWhitespace));
If you run example/calculator.dart you will get the result.
-1.0
You can modify the build script and generate it as a library (eg as lib/calculator.dart).
Then import this library and use it.
It requires some programming skills from you, but there is nothing difficult about it.
Everything is very simple.
By combined ManuH68 answer I did:
void main() {
test('Return correct', () {
const correctResult = 10;
final calc = <dynamic>[2, "*", 5, "/", 2, "-", 3, "+", 8];
num result = 0;
// clean "*" and "/" first
while (calc.contains("*") || calc.contains("/")) {
final pos = calc.indexWhere((e) => e == "*" || e == "/");
num leftOp = calc[pos - 1];
num rightOp = calc[pos + 1];
switch (calc[pos]) {
case "*":
result = leftOp * rightOp;
break;
case "/":
result = leftOp / rightOp;
break;
}
calc.removeAt(pos);
calc.removeAt(pos);
calc[pos - 1] = result;
}
// Then After calculated "*" and "/" perform calculate on "+" and "-"
while (calc.length > 1) {
final pos = calc.indexWhere((e) => e == "-" || e == "+");
num leftOp = calc[pos - 1];
num rightOp = calc[pos + 1];
switch (calc[pos]) {
case "-":
result = leftOp - rightOp;
break;
case "+":
result = leftOp + rightOp;
break;
}
calc.removeAt(pos);
calc.removeAt(pos);
calc[pos - 1] = result;
}
expect(calc[0], correctResult);
});
}
this might not clean but this is what I founded that work

Swift variable accessibility

I am doing this for a class, so I am not looking for someone to code this for me, just for a little guidance. Using Swift3 in Xcode 8. I fixed the previous errors and I think I'm making progress but...
class Cat {
var catName, catBreed, catColor: String
var catAge, catWeight: Int
init(name:String, age:Int, weight:Int, breed:String, color:String)
{
catName = name
catAge = age
catWeight = weight
catBreed = breed
catColor = color
}
func calculateAge (catAge: Int) -> Int
{
var humanYears = 0
if catAge == 1 {
humanYears = 15
} else if catAge == 2 {
humanYears = 24
} else if catAge > 2 {
humanYears = (24 + (catAge * 4))
}
print ("Your cat is \(humanYears) human years old!")
return humanYears
}
func createCats(name: String, age: Int, weight: Int, breed: String, color: String) -> String
{
let humanYears = calculateAge(catAge: age)
let catInfo : String = "\(name) is a \(color) \(breed) who weights \(weight) named \(name), and he is \(humanYears) human years old."
print (catInfo)
return catInfo
}
}
Cat.createCats()
I get an error at Cat.createCats() stating use of instance member 'createCats' on type 'Cat'; did you mean to use a value of type 'Cat' instead?
I have tried putting Cat in the (), I have tried naming all of my variables in there...
As I see in your code , you are re-declaring the variable "humanYears".
One is immediate after Class and second is in the function "calculateAge" . Edit your function "calculate" as below :
func calculateAge (catAge: Int) -> Int {
if catAge == 1 {
humanYears = 15
} else if catAge == 2 {
humanYears = 24
} else if catAge > 2 {
humanYears = (24 + (catAge * 4))
}
return humanYears
}
I figured out what I was doing wrong, thanks to you guys and another post on Stack Overflow that I searched up. Just in case it helps anyone here, I will post my final code below. My biggest issue was that I was trying to call the function before I created an instance of the class. This is not the finished version of the solution, but I was excited that I got it working so I wanted to post it.
Thanks again for all of your help,
class Cat
{
func calculateAge (catAge: Int) -> String
{
var humanYears = 0
if catAge == 1 {
humanYears = 15
} else if catAge == 2 {
humanYears = 24
} else if catAge > 2 {
humanYears = (24 + (catAge * 4))
}
return "Your cat is \(humanYears) human years old!"
}
func createCats(name: String = "Fluffy", age: Int = 1, weight: Int = 3, breed: String = "tabby", color: String = "brown") -> String
{
var catInfo : String
let humanYears = calculateAge(catAge: age)
catInfo = "\(name) is a \(color) \(breed) who weights \(weight) pounds named \(name), and he is \(humanYears) human years old."
print (catInfo)
return catInfo
}
}
let firstCat = Cat()
firstCat.createCats()
let secondCat = Cat()
secondCat.createCats(name: "bob")

Windows Phone 7 UserExtendedProperties

After being directed here:
http://msdn.microsoft.com/en-us/library/microsoft.phone.info.userextendedproperties.getvalue%28v=VS.92%29.aspx
I tried the following code on my sideloaded application. All that was written out was the 000001 value.
I knew that would be the case in the emulator, but I was hoping to get a real value when it was on my phone. Any ideas?
int ANIDLength = 32;
int ANIDOffset = 2;
string result = string.Empty;
object anid;
if (UserExtendedProperties.TryGetValue("ANID", out anid))
{
if (anid != null && anid.ToString().Length >= (ANIDLength + ANIDOffset))
{
result = anid.ToString().Substring(ANIDOffset, ANIDLength);
}
else
{
result = "000001";
}
}
You need to remove 1 from the calculation of your length check to account for the substring operation working on a zero based index:
if (anid != null && anid.ToString().Length >= (ANIDLength + ANIDOffset - 1))
This works on my machine:
private string GetAnid()
{
object anidValue;
int ANIDLength = 32;
int ANIDOffset = 2;
if (UserExtendedProperties.TryGetValue("ANID", out anidValue))
{
if (anidValue != null && anidValue.ToString().Length >= (ANIDLength + ANIDOffset - 1))
{
return anidValue.ToString().Substring(ANIDOffset, ANIDLength);
}
else
{
return "???";
}
}
else
{
return "???";
}
}

Resources