I am trying to translate a message generated using C# to JAVA. As a first step, i generated proto file and this is what i got
package Om.Business.Scanner;
message ScannerActivityDetail {
optional string ActivityId = 1;
optional string ContextId = 2;
optional int32 ActivityStart = 3;
optional bcl.DateTime ActivityEnd = 4;
}
How do i interpret bcl.DateTime in java world?
I am using protobuf-net and trying to de-serialize message generated by C# app.
Thanks in advance for your help.
Looking at bcl.proto, it should be pretty straightforward. Create a Map<DateTime.TimeSpanScale, TimeUnit> in the obvious way, then:
public static Date toDate(bcl.DateTime proto) {
TimeUnit unit = SCALE_TO_UNIT_MAP.get(proto.getScale());
if (unit == null) {
throw new IllegalArgumentException("Invalid scale: " + proto.getScale());
}
long millis = unit.toMillis(proto.getValue());
return new Date(millis);
}
You could use Joda Time's DateTime type in exactly the same way, as it has a constructor accepting a long too. (You might want to think about which time zone to specify though...)
Related
I recently started working with protobuf in my golang project. I created below simple protobuf file. I have three different endpoints.
GetLink takes CustomerRequest as an input parameter and returns back CustomerResponse
GetBulkLinks takes BulkCustomerRequest as an input parameter and returns back BulkCustomerResponse
StreaLinks takes StreamRequest as an input parameter and returns back CustomerResponse
I am wondering if there is any way we can improve below proto file because CustomerRequest and BulkCustomerRequest has mostly everything in common except resources field so there is a duplication. And same goes with StreamRequest input parameter as it only takes clientId as the input parameter. Is there anything in protocol buffer which can reuse stuff from another message type?
Is there any better or efficient way to organize below proto file which reuses message accordingly?
syntax = "proto3";
option go_package = "github.com/david/customerclient/gen/go/data/v1";
package data.v1;
service CustomerService {
rpc GetLink(CustomerRequest) returns (CustomerResponse) {};
rpc GetBulkLinks(BulkCustomerRequest) returns (BulkCustomerResponse) {};
rpc StreaLinks(StreamRequest) returns (CustomerResponse) {};
}
message CustomerRequest {
int32 clientId = 1;
string resources = 2;
bool isProcess = 3;
}
message BulkCustomerRequest {
int32 clientId = 1;
repeated string resources = 2;
bool isProcess = 3;
}
message StreamRequest {
int32 clientId = 1;
}
message CustomerResponse {
string value = 1;
string info = 2;
string baseInfo = 3;
string link = 4;
}
message BulkCustomerResponse {
map<string, CustomerResponse> customerResponse = 1;
}
Is there anything in protocol buffer which can reuse stuff from another message type?
Composition.
However keep in mind that request and response payloads may change over time. Even if it looks like they have something in common today, they may diverge tomorrow. After all they are used in different RPCs. Then excessive coupling achieves the opposite effect and becomes technical debt.
Since your schema has literally no more than three fields for each message, I would leave everything as is. Anyway, if you really must, you may consider the following:
Extract the GetLink and GetBulkLinks common fields in a separate message and compose with that:
message CustomerRequestParams {
int32 clientId = 1;
bool isProcess = 2;
}
message CustomerRequest {
CustomerRequestParams params = 1;
string resources = 2;
}
message BulkCustomerRequest {
CustomerRequestParams params = 1;
repeated string resources = 2;
}
StreamRequest looks just fine with repeating clientId. Arguably a stream is conceptually different from a unary RPC, so just keep them separated. And it's just one field.
I have the following proto3 message structure:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
google.protobuf.Timestamp createTime = 4;
}
I am using the "same" structure for some requests and responses on my app. What I want to achieve is to mark somehow (if possible) the createTime field as not required, in case we are talking about a request object, and required in case we are taking about a response object.
Is it possible to do this without creating a separate message ?
Thanks
To my knowledge, it's not possible and I'd discourage pursuing solutions other than defining distinct message types: one which includes the optional field and one which does not.
One way to solve this is to define a message that includes the mandatory fields and another than extends it:
message BaseBuildContent {
string locale = 1;
string buildVersion = 2;
string buildLabel = 3;
}
message SomeRequest {
BaseBuildContent content = 1;
}
message SomeResponse {
BaseBuildContent content = 1;
google.protobuf.Timestamp createTime = 2;
}
NOTE Protobuf style guide recommends message names be PascalCased and field names be snake_cased.
Given the following proto definitions:
syntax = "proto3";
import "google/protobuf/descriptor.proto";
option java_package = "com.example.dto";
option java_multiple_files = true;
extend google.protobuf.FieldOptions {
Projector projector = 50002;
}
message Projector {
string name = 1;
string class = 2;
bool default = 3;
}
message SearchRequest {
string query = 1 [(projector) = {name: "queryProjector", class: "foobar"}];
int32 page_number = 2;
int32 result_per_page = 3;
}
How can I access the field extension?
As far as I understand extension still work in proto3, but are generally replaced by the Any type?
I came as this far:
final Descriptors.Descriptor descriptor = SearchRequest.getDescriptor();
final Descriptors.FieldDescriptor query = descriptor.findFieldByName("query");
Is this the right way? Whats the next step?
As stated here https://github.com/google/protobuf/issues/1460
Custom options are still supported. It's the only place where you can use extensions in proto3. It works the same way as in proto2. Languages that don't support proto2 may provide a special API to access custom options as they don't support extensions.
so custom options seem to be still supported and you should get them using
descriptor.findFieldByName("query").getOptions().getAllFields();
That would return you a map of your custom options (as fields)
final Map<Descriptors.FieldDescriptor, Object> allFields;
whereas the value would be the type of your option, Projector in your case.
The FileDescriptor for this custom option (projector) seems to be generated as a public static in a class named after your *.proto file using its camelCase name.
If your proto file is called search_service_v1.proto you might find the custom option directly as follows:
final DescriptorProtos.FieldOptions options descriptor.findFieldByName("query").getOptions();
final Object field = options.getField(SearchServiceV1.projector.getDescriptor());
And you'll get your extension by
final Projector projector = Projector.class.cast(field);
Right now I have an application where my iPhone app sends a request that is handled in .NET/C#, serialized to XML, and parsed on the app in objective-c. The current response class structure has one base class (BaseResponse) and many (over 25) subclasses for each type of request that corresponds to different things that need to be returned. Right now I'm looking to see if protobuf would be faster and easier than XML. From what I understand, a .proto file for this class structure is:
Message BaseResponse {
Required Int field1 = 1;
Optional SubResponse1 sub1= 2;
Optional SubResponse2 sub2 = 3;
Etc....
}
Message SubResponse1 {
....
}
Message SubResponse2 {
....
}
Etc for each sub response.
My question is: if I have over 25 of these optional elements (of which only 1 will be non null), does that completely wipe away the size and performance benefit of using protobuf? Does protobuf make sense for this application?
No, it does not impact the performance benefit - you'll just need to check which one is non-null in the objective-C code. Since protobuf only serializes the non-null values it will still be very efficient over the wire. The protobuf specification itself doesn't actually include inheritance, so you are right to say that you need to spoof it via encapsulation - but since you mention C#, note that what you have described (including how the data appears on the wire, i.e. it will be 100% comptible) can be done directly via inheritance if you use protobuf-net as the C# implementation - which should be possible with your existing model. For example:
[ProtoContract]
[ProtoInclude(2, typeof(SubResponse1))]
[ProtoInclude(3, typeof(SubResponse2))]
public class BaseResponse
{
// note Name and IsRequired here are optional - only
// included to match your example
[ProtoMember(1, IsRequired = true, Name="field1")]
public int Field1 { get; set; }
/*...*/
}
[ProtoContract]
public class SubResponse1 : BaseResponse
{/*...*/}
[ProtoContract]
public class SubResponse2 : BaseResponse
{/*...*/}
You can get the .proto via:
var proto = Serializer.GetProto<BaseResponse>();
Which gives:
message BaseResponse {
required int32 field1 = 1 [default = 0];
// the following represent sub-types; at most 1 should have a value
optional SubResponse1 SubResponse1 = 2;
optional SubResponse2 SubResponse2 = 3;
}
message SubResponse1 {
}
message SubResponse2 {
}
I need to read stuff from an Outlook msg file. Currently I'm using a class from CodeProject.com project to accomplish this, since deploying VSTO and Outlook on a server is not an option.
This class gets To, From, CC, Subject, Body, and everything else I need from the msg file, except Date information (such as Received Date and Sent Date).
There is some (really, really low-level) documentation on how to get stuff out of msg files on MSDN, but it's a little beyond the scope of this project and doesn't mention dates at all.
Ideally I'd be able to have a drop-in replacement for the class I am using now (OutlookStorage.cs in the previously mentioned CodeProject) or be able to modify the existing class a bit. To modify, I would need the correct 4 character hexidecimal prop identifier for received date. For instance, Subject is listed as PR_SUBJECT = "0037" and Body is listed as PR_BOY = "1000".
If you're using OutlookStorage.cs from CodeProject, then add the following:
private const string PR_RECEIVED_DATE="007D";
private const string PR_RECEIVED_DATE_2 = "0047";
...
/// <summary>
/// Gets the date the message was received.
/// </summary>
public DateTime ReceivedDate
{
get
{
if (_dateRevieved == DateTime.MinValue)
{
string dateMess = this.GetMapiPropertyString(OutlookStorage.PR_RECEIVED_DATE);
if (String.IsNullOrEmpty(dateMess))
{
dateMess = this.GetMapiPropertyString(OutlookStorage.PR_RECEIVED_DATE_2);
}
_dateRevieved = ExtractDate(dateMess);
}
return _dateRevieved;
//return ExtractDate(dateMess);
}
}
private DateTime _dateRevieved = DateTime.MinValue;
private DateTime ExtractDate(string dateMess)
{
string matchStr = "Date:";
string[] lines = dateMess.Split(new String[] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);
foreach (string line in lines)
{
if (line.StartsWith(matchStr))
{
string dateStr = line.Substring(matchStr.Length);
DateTime response;
if (DateTime.TryParse(dateStr, out response))
{
return response;
}
}
}
return DateTime.MinValue;
}
I think the Aspose library will do what you want, ok it a 3rd party lib so may not be what you want. There are a few vbs scripts around that get basic infomation out of msg files that could be translated.
Got a hint from this:
string fullFileName = "c:\message.msg";
DateTime dateRevieved = new DateTime();
StreamReader sr = new StreamReader(fullFileName, Encoding.Default);
string full = sr.ReadToEnd();
string date;
int iStart;
int iLast;
string caption;
//This -should- handle all manner of screwage
//The ONLY way it would not is if someone guessed the -exact- to-the-second
//time that they send the message, put it in their subject in the right format
while (true) { //not an infinite loop, I swear!
caption = "Date:";
if (full.IndexOf("Date:") > -1) { //full shortens with each date is removed
string temp = "";
iStart = full.LastIndexOf(caption);
temp = full.Remove(0, iStart + caption.Length);
full = full.Substring(0, iStart);
iLast = temp.IndexOf("\r\n");
if (iLast < 0) {
date = temp;
} else {
date = temp.Substring(0, iLast);
}
date = date.Trim();
if (date.Contains(subject) || subject.Contains(date)) {
continue; //would only happen if someone is trying to screw me
}
try {
dateRevieved = DateTime.Parse(date); //will fail if not a date
break; //if not a date breaks out of while loop
} catch {
continue; //try with a smaller subset of the msg
}
} else {
break;
}
}
This is kind of a hack compared to the ways you can get other things from msg files using something this lovely project. Still, it's stood up to everything I have thrown against it, and as noted the -only- way to fool it is to put the exact to-the-second date in the subject line in the proper format.
to combine your two posts I would suggest the following solution:
To modify, I would need the correct 4 character hexidecimal prop identifier for recieved date. For instance, Subject is listed as PR_SUBJECT = "0037" and Body is listed as PR_BOY = "1000".
Look for "007D".
Use the method you posted in your second post on the received data to eliminate the problem when the same (date) string is inside the subject.
I have to mention that this method doesn't seem to work on internal eMails: In mails I receive from colleagues, there is no substg1.0_007Dxxxx-Property.
Here, the date seems to be hidden in substg1.0_0047xxxx.
All the best!
inno