I have a proto with imported Data types. something similar to:
import "google/protobuf/wrappers.proto";
message someMessage
.google.protobuf.StringValue someString = 1;
.google.protobuf.Int64Value someInt = 2;
.google.protobuf.DoubleValue someDouble = 3;
previously my proto looked like:
message someMessage {
string someString = 1;
Prior to adding data types, the code below worked just fine. Now when I run, I get an error similar to com.google.protobuf.InvalidProtocolBufferException: Invalid wrapper type: google.protobuf.StringValue
DynamicMessage.Builder dynamicMessageBuilder = DynamicMessage.newBuilder(someDescriptor);
JsonFormat.parser().merge(stringPayload, dynamicMessageBuilder);
byte[] blah = dynamicMessageBuilder.build().toByteArray();
How do I get my proto messages to build correctly with typed data?
Hello The answer actually lied in a different part of the code:
FileDescriptorProto myFile = DescriptorProtos.FileDescriptorSet.parseFrom(is).getFile(0);
someDescriptor = Descriptors.FileDescriptor.buildFrom(myFile, new Descriptors.FileDescriptor[]{}).findMessageTypeByName("myMessage");
We ended up doing something like this. Note that the actual solution is dynamic and not hard coded, but it is not as easily unerstood.
Descriptors.FileDescriptor valuetypes = Descriptors.FileDescriptor.buildFrom(fileDescriptorProtoList.get(0), new Descriptors.FileDescriptor[]{});
someDescriptor = Descriptors.FileDescriptor.buildFrom(fileDescriptorProtoList.get(1), new Descriptors.FileDescriptor[]{valueTypes}).findMessageTypeByName("myMessage");
Also note this is important when generating the descriptor files
<includeDependenciesInDescriptorSet>true</includeDependenciesInDescriptorSet>
Related
I have been trying to upload an image in chunks with client side streaming using grpcurl. The service is working without error except that at the server, image data received is 0 bytes.
The command I am using is:
grpcurl -proto image_service.proto -v -d # -plaintext localhost:3010 imageservice.ImageService.UploadImage < out
This link mentions that the chunk data should be base64 encode and so the contents of my out file are:
{"chunk_data": "<base64 encoded image data>"}
This is exactly what I am trying to achieve, but using grpcurl.
Please tell what is wrong in my command and what is the best way to achieve streaming via grpcurl.
I have 2 more questions:
Does gRPC handles the splitting of data into chunks?
How can I first send a meta-data chunk (ImageInfo type) and then the actual image data via grpcurl?
Here is my proto file:
syntax = "proto3";
package imageservice;
import "google/protobuf/wrappers.proto";
option go_package = "...";
service ImageService {
rpc UploadImage(stream UploadImageRequest) returns (UploadImageResponse) {}
}
message UploadImageRequest {
oneof data {
ImageInfo info = 1;
bytes chunk_data = 3;
};
}
message ImageInfo {
string unique_id = 1;
string image_type = 2;
}
message UploadImageResponse {
string url = 1;
}
Interesting question. I've not tried streaming messages with (the excellent) grpcurl.
The documentation does not explain how to do this but this issue shows how to stream using stdin.
I recommend you try it that way first to ensure that works for you.
If it does, then bundling various messages into a file (out) should also work.
Your follow-on questions suggest you're doing this incorrectly.
chunk_data is the result of having split the file into chunks; i.e. each of these base64-encoded strings should be a subset of your overall image file (i.e. a chunk).
your first message should be { "info": "...." }, subsequent messages will be { "chunk_data": "<base64-encoded chunk>" } until EOF.
I tried creating a method in Postman and got really close but am having issues with the signature. We are trying to query the IP ranges for VPCs to add to a WAF rule, in order to allow traffic to a secure application.
Postman allows a pre-request script, in Javascript, and supports a handful of included JS libraries, including CryptoJS.
The code here creates exactly the request that Ali Cloud says needs to be signed. It signs with HMAC-SHA1 from CryptoJS and encodes to base 64.
All of the variables are included in the request parameters. I'm not sure what else it could be complaining about.
var dateIso = new Date().toISOString();
var randomString = function(length) {
var text = "";
var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
for(var i = 0; i < length; i++) {
text += possible.charAt(Math.floor(Math.random() * possible.length));
}
return text;
}
var accesskeyid = "LTAI4GC7VEijsm5bV3zwcZxZ"
var action = "DescribePublicIPAddress"
var format = "XML"
var regionid = "cn-shanghai-eu13-a01"
var signaturemethod = "HMAC-SHA1"
var signaturenonce = randomString(16)
var signatureversion = "1.0"
var timestamp = dateIso.replace(/:/gi, "%253A")
var version = "2016-04-28"
pm.environment.set("AccessKeyId", accesskeyid)
pm.environment.set("Action", action)
pm.environment.set("Format", format)
pm.environment.set("RegionID", regionid)
pm.environment.set("SignatureMethod", signaturemethod)
pm.environment.set("SignatureNonce", signaturenonce)
pm.environment.set("SignatureVersion", signatureversion)
pm.environment.set("Timestamp", dateIso)
pm.environment.set("Version", version)
var request = "GET&%2F&" + "AccessKeyID%3D" + accesskeyid + "%26Action%3D" + action + "%26Format%3D" + format + "%26RegionID%3D" + regionid + "%26SignatureMethod%3D" + signaturemethod + "%26SignatureNonce%3D" + signaturenonce + "%26SignatureVersion%3D" + signatureversion + "%26Timestamp%3D" + timestamp + "%26Version%3D" + version
pm.environment.set("Request", request)
var hash = CryptoJS.HmacSHA1(request, "spwH5dNeEm4t4dlpqvYWVGgf7aEAxB&")
var base64 = CryptoJS.enc.Base64.stringify(hash)
var encodesig = encodeURIComponent(base64)
pm.environment.set("Signature", encodesig);
console.log(base64)
console.log(request)
The console output shows:
Signature: XbVi12iApzZ0rRgJLBv0ytJJ0LY=
Parameter string to be signed:
GET&%2F&AccessKeyID%3DLTAI4GC7VEijsm5bV3zwcZvC%26Action%3DDescribePublicIPAddress%26Format%3DXML%26RegionID%3Dcn-shanghai-eu13-a01%26SignatureMethod%3DHMAC-SHA1%26SignatureNonce%3DiP1QJtbasQNSOxVY%26SignatureVersion%3D1.0%26Timestamp%3D2020-06-01T15%253A38%253A12.266Z%26Version%3D2016-04-28
Request sent:
GET https://vpc.aliyuncs.com/?AccessKeyID=LTAI4GC7VEijsm5bV3zwcZvC&Action=DescribePublicIPAddress&Format=XML&RegionID=cn-shanghai-eu13-a01&SignatureMethod=HMAC-SHA1&SignatureNonce=iP1QJtbasQNSOxVY&SignatureVersion=1.0&Timestamp=2020-06-01T15:38:12.266Z&Version=2016-04-28&Signature=XbVi12iApzZ0rRgJLBv0ytJJ0LY%3D
Response Received:
<?xml version='1.0' encoding='UTF-8'?><Error><RequestId>B16D216F-56ED-4D16-9CEC-633C303F2B61</RequestId><HostId>vpc.aliyuncs.com</HostId><Code>IncompleteSignature</Code><Message>The request signature does not conform to Aliyun standards. server string to sign is:GET&%2F&AccessKeyID%3DLTAI4GC7VEijsm5bV3zwcZvC%26Action%3DDescribePublicIPAddress%26Format%3DXML%26RegionID%3Dcn-shanghai-eu13-a01%26SignatureMethod%3DHMAC-SHA1%26SignatureNonce%3DiP1QJtbasQNSOxVY%26SignatureVersion%3D1.0%26Timestamp%3D2020-06-01T15%253A38%253A12.266Z%26Version%3D2016-04-28</Message><Recommend><![CDATA[https://error-center.aliyun.com/status/search?Keyword=IncompleteSignature&source=PopGw]]></Recommend></Error>
When I check the "server string to sign" from the response and the parameter string that was signed in a compare, they are identical.
It looks like everything is built as needed but the signature is still barking. Guessing I missed something simple but haven't found it yet.
Note: The accesskeyID and key posted are for example purposes and not a real account so this code will not copy and paste to execute in Postman.
PS - I learned quite a bit from the other few threads on this topic, which is how I got to this point. akostadinov was super helpful on another thread.
I believe you have double encoded &. I have implemented other Alibaba Cloud REST APIs successfully. Could you please check this.
Following is the expected string to sign format:
GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeVpcs&Format%3DXML&
SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&TimeStamp%3D2016-02-23T12%253A46%
253A24Z&Version%3D2014-05-15
A bit late to the party, but as this is the first result when googling for the IncompleteSignature error, I thought I might comment and hopefully save someone else the grief I have been through.
For me, the subtle detail that I missed in the official documentation here is that the key used for the signature requires an ampersand & to be added to the end, before being used.
As soon as I caught that, everything else worked perfectly.
I am having an issue with the IONotificationCreatePort function in IOKit:
var NotificationPort = IONotificationPortCreate(MasterPort)
IONotificationPortSetDispatchQueue(NotificationPort, DispatchQueue)
gives the following compiler error when NotificationPort is used in the function call in the second line
'Unmanaged IONotificationPort' is not identical to
'IONotificationPort'
if I use the following code based on the information in the Using Swift with Cocoa and Objective-C document, it compiles but generates a runtime error
var NotificationPort = IONotificationPortCreate(MasterPort).takeRetainedValue()
IONotificationPortSetDispatchQueue(NotificationPort, DispatchQueue)
Thread 1: EXC_BAD_ACCESS(code=1, address=0xwhatever)
So I think I have the run time error figured out, the IONotificationPort object does not have takeRetainedValue method
The crux of the problem as I see it, is that the IONotificationPortCreate function creates an IONotificationPort object and returns the reference to it.
I have looked all over the place and there is lots of information about and ways to pass references into a function call from Swift but nowhere can I find how to deal with references as a return value.
Can Swift call an object by reference?
Or am I way off the mark here????
Here is the objective C code that I am trying to convert to swift:
_notificationPort = IONotificationPortCreate(masterPort);
IONotificationPortSetDispatchQueue(_notificationPort, _controllerQueue);
Here is the complete code snippet from my swift file:
//Get IOKit Master Port
var MasterPort: mach_port_t = 0
let BootstrapPort: mach_port_t = 0
var MasterPortReturnCode: kern_return_t = 0
MasterPortReturnCode = IOMasterPort(BootstrapPort, &MasterPort)
println("Master port returned as \(MasterPort) with return code of \(MasterPortReturnCode)")
//Set up notification port and send queue
let DispatchQueue = dispatch_queue_create("com.apparata.AVB_Browser", DISPATCH_QUEUE_SERIAL)
var NotificationPort = IONotificationPortCreate(MasterPort)
IONotificationPortSetDispatchQueue(NotificationPort, DispatchQueue)
I want to retrieve bibtex data (for building a bibliography) by sending a DOI (Digital Object Identifier) to http://www.crossref.org from within matlab.
The crossref API suggests something like this:
curl -LH "Accept: text/bibliography; style=bibtex" http://dx.doi.org/10.1038/nrd842
based on this source.
Another example from here suggests the following in ruby:
open("http://dx.doi.org/10.1038/nrd842","Accept" => "text/bibliography; style=bibtex"){|f| f.each {|line| print line}}
Although I've heard ruby rocks I want to do this in matlab and have no clue how to translate the ruby message or interpret the crossref command.
The following is what I have so far to send a doi to crossref and retrieve data in xml (in variable retdat), but not bibtex, format:
clear
clc
doi = '10.1038/nrd842';
URL_PATTERN = 'http://dx.doi.org/%s';
fetchurl = sprintf(URL_PATTERN,doi);
numinputs = 1;
www = java.net.URL(fetchurl);
is = www.openStream;
%Read stream of data
isr = java.io.InputStreamReader(is);
br = java.io.BufferedReader(isr);
%Parse return data
retdat = [];
next_line = toCharArray(br.readLine)'; %First line contains headings, determine length
%Loop through data
while ischar(next_line)
retdat = [retdat, 13, next_line];
tmp = br.readLine;
try
next_line = toCharArray(tmp)';
if strcmp(next_line,'M END')
next_line = [];
break
end
catch
break;
end
end
%Cleanup java objects
br.close;
isr.close;
is.close;
Help translating the ruby statement to something matlab can send using a script such as that posted to establish the communication with crossref would be greatly appreciated.
Edit:
Additional constraints include backward compatibility of the code (back at least to R14) :>(. Also, no use of ruby, since that solves the problem but is not a "matlab" solution, see here for how to invoke ruby from matlab via system('ruby script.rb').
You can easily edit urlread for what you need. I won't post my modified urlread function code due to copyright.
In urlread, (mine is at C:\Program Files\MATLAB\R2012a\toolbox\matlab\iofun\urlread.m), as the least elegant solution:
Right before "% Read the data from the connection." I added:
urlConnection.setRequestProperty('Accept','text/bibliography; style=bibtex');
The answer from user2034006 lays the path to a solution.
The following script works when urlread is modified:
URL_PATTERN = 'http://dx.doi.org/%s';
doi = '10.1038/nrd842';
fetchurl = sprintf(URL_PATTERN,doi);
method = 'post';
params= {};
[string,status] = urlread(fetchurl,method,params);
The modification in urlread is not identical to the suggestion of user2034006. Things worked when the line
urlConnection.setRequestProperty('Content-Type','application/x-www-form-urlencoded');
in urlread was replaced with
urlConnection.setRequestProperty('Accept','text/bibliography; style=bibtex');
I trying to use ImageInfo and Jython to get information from a image on my harddrive.
I have imported the module fine but keep getting this error:
TypeError: setInput(): expected 2 args; got 1
And this is the code I am trying to use:
filename = "C:\\image.jpg"
img = ImageInfo.setInput(filename)
Could anyone point out what I am doing wrong.
Cheers
Eef
The missing argument Jython complains about is the ImageInfo object itself, which doesn't exist yet. You must construct it first. So:
filename = "C:\\image.jpg"
ii = ImageInfo()
img = ii.setInput(filename)
or
filename = "C:\\image.jpg"
img = ImageInfo().setInput(filename)
may work also.