Say I'm modeling a Business resource in Protobuf. It represents any generic kind of "Business" - a bakery, a restaurant, a bank, etc...
Concept, in JSON
In JSON I might model the structure as follows:
{
id: "...",
name: "...",
address: "...",
properties: {
type: "...",
...
}
The properties attribute is a different structure for each type of business.
For a bakery it might be:
properties: {
type: 'Bakery',
serves_coffee: true,
gluten_free: true
}
For a bank it might be:
properties: {
type: "Bank",
max_denomination_offered: 100000,
}
Protobuf
The main Business can be modeled as follows in Protobuf:
message Business {
string id = 1;
string name = 2;
string address = 3;
??? properties = 4
}
How do I model a generic Properties message that can have varying shape?
Is there some sort of "generic" object I can use? Or do I have to create BankProperties, BakeryProperties, etc... and add them in as:
Google.Protobuf.WellKnownTypes.Any properties = 4;
Is there an established or well-accepted pattern for this sort of thing?
Thanks!
What will your receiving code do? If it only knows about specific kinds of Business anyway, it would make sense to just list them in the .proto:
message BakeryProperties {
bool serves_coffee = 1;
bool serves_cake = 2;
}
message BankProperties {
int32 max_denomination = 1;
}
message Business {
string id = 1;
string name = 2;
string address = 3;
oneof details {
BakeryProperties bakery = 100;
BankProperties bank = 101;
}
}
That way you get the actual benefits of a schema: any code that knows how do deal with bakeries, deals with them in the same way. If the code doesn't know about the particular type, it can still use the common properties.
But if you are just going to e.g. display the properties, maybe what you want is just a generic key-value mapping:
map<string,string> properties = 10;
or
message Property {
string name = 1;
oneof {
string strvalue = 2;
int32 intvalue = 3;
}
}
repeated Property properties = 10;
Related
Is there a way to reuse protobuf structs across multiple proto files?
in the client.proto file I have the following
message SingleOperation {
string command = 1;
}
message ClientBatch {
string unique_id = 1;
repeated SingleOperation requests = 2;
int64 sender = 3;
}
Then in network.proto file, I have the following
message MemPool {
int32 sender = 1;
string unique_id = 2;
SingleOperation op = 3;
}
In the network.proto file, I need to reuse the SingleOperation message.
However, I do not see an option to do this.
Does protobuff support some sort of packaging to support struct reuse?
I am trying to create a protocol buffer message with fields that are either a message or one of a choice of some constant (like an enum). Something that is logically equivalent to:
message Error {
oneof error_type {
EMPTY_METHOD_NAME = 0
ExecutionError execution_error = 1;
}
message ExecutionError {
string value = 1;
}
}
Essentially, I would like a field that can represent either an error type that is just a name with no fields, or an error type that has fields. How would I do this with protobuf3?
See Enumerations in the Language Guide (proto3)
message Error {
oneof error_type {
Foo foo = 1;
ExecutionError execution_error = 2;
}
enum Foo {
X = 0;
Y = 1;
Z = 2;
}
message ExecutionError {
string value = 1;
}
}
I have two ArrayLists of different Data classes as given below:
class Record{
var id: Long = 0
var RecordId: Int = 0
var Record: String? = null
var title: String? = null
var description: String? = null
var longDate: Long = 0
}
class Type{
var id: Long = 0
var typeId: Int = 0
var subTypeId: Int = 0
var typeString: String? = null
var longDate: Long = 0
}
var recordsList: ArrayList<Record>
var typesList: ArrayList<Type>
Now, I want a merged list of these two which will be sorted based on a common field in both the Objects i.e. longDate. I have tried .associate , sortedBy, sortedWith(compareBy<>) etc. but could not achieve the desired result.
Here, also there is one point to note is that while comparing the two lists it is possible that one on them may be empty.
This will produce a List<Any> with all items sorted by longDate:
(recordsList + typesList)
.sortedBy {
when (it) {
is Record -> it.longDate
is Type -> it.longDate
else -> error("")
}
}
Or you might consider creating an interface that has val longDate: Long that both of these classes implement. Then you wouldn't need the when expression, and your List would be of the type of the interface.
Something like this should work, but I personally think that it is quite the code smell. There is no guarantee that Record.longDate is truly the same type as Type.longDate (we know that it is, since we create the model, but the compiler would never know).
val result = (recordsList + typesList).sortedBy {
when(it){
is Record -> it.longDate
is Type -> it.longDate
else -> error("incompatible list element $it")
}
}
And it would work something like this: (I've removed some parameters from the models as they don't really count here)
fun main() {
val recordsList = listOf(Record().apply { longDate = 5 }, Record().apply { longDate = 3})
val typesList = listOf(Type().apply { longDate = 3 }, Type().apply { longDate = 2 })
val result = (recordsList + typesList).sortedBy {
when(it){
is Record -> it.longDate
is Type -> it.longDate
else -> error("incompatible list element $it")
}
}
result.forEach{
println(it.toString())
}
}
class Record{
var longDate: Long = 0
override fun toString(): String {
return "Record(longDate=$longDate)"
}
}
class Type{
var longDate: Long = 0
override fun toString(): String {
return "Type(longDate=$longDate)"
}
}
This will output:
Type(longDate=2)
Record(longDate=3)
Type(longDate=3)
Record(longDate=5)
Doing it in a more generic way, so that you can create a fun where you state which property to be used from each object type would most likely use reflection, which I'd avoid at all costs.
So I would definitely consider if one object can inherit the other, or create an interface, or anything else.
I'll end with 2 questions: why no constructors? why ArrayList and not list?
In my proto file, I want to define a map as a custom option, tried a few things but none is working.
my metadata proto file:
syntax = "proto2";
import "google/protobuf/descriptor.proto";
package com.util;
option java_package = "com.util";
message MyMeta {
optional bool needValidation = 1;
map<string, string> fileMap = 2;
}
extend google.protobuf.FieldOptions {
optional MyMeta meta = 80412;
}
my proto file
syntax = "proto3";
package com.test;
import "util/meta.proto";
import "google/protobuf/timestamp.proto";
message MyMeta {
int32 id = 1 [(com.util.meta).needValidation = false, /*looking for a way to set a map (com.meta).tableOptions = {"t,raw_orders"}]*/;
}
Is this possible?
This works:
message MyMeta {
int32 id = 1 [
(com.util.meta) = {
needValidation: false,
fileMap: [
{
key: "t",
value: "raw_orders"
}
];
}];
}
The protobuf map is syntactic sugar for a repeated field of map entry messages with key and value fields. 1
In your case, your meta message
message MyMeta {
optional bool needValidation = 1;
map<string, string> fileMap = 2;
}
is equivalent to
message MyMeta {
optional bool needValidation = 1;
message FileMapEntry {
string key = 1;
string value = 2;
}
repeated FileMapEntry fileMap = 2;
}
Of course, it would be much nicer if there was a more obvious way to specify a map value in an option.
Seems it is not possible. To make my logic, I created a string like "StoreOrders:raw_orders_test,OrderItems:raw_order_items_test
message MyMeta {
int32 id = 1 [(com.util.meta).needValidation = false, (com.meta).tableOptions = "TableA:valueA,TableB:valueB;
}
and in my jav code and splitting the string to create a hash map.
I am trying to convert my code as clean as possible using the Kotlin's built-in functions. I have done some part of the code using for loops. But I want to know the efficient built-in functions to be used for this application
I have two array lists accounts and cards.
My goal is to search a specific card with the help of its card-number, in the array list named cards.
Then I have to validate the pin. If the pin is correct, by getting that gift card's customerId I have to search the account in the array list named accounts. Then I have to update the balance of the account.
These are the class which I have used
class Account{
constructor( )
var id : String = generateAccountNumber()
var name: String? = null
set(name) = if (name != null) field = name.toUpperCase() else { field = "Unknown User"; println("invalid details\nAccount is not Created");}
var balance : Double = 0.0
set(balance) = if (balance >= 0) field = balance else { field = 0.0 }
constructor(id: String = generateAccountNumber(), name: String?,balance: Double) {
this.id = id
this.balance = balance
this.name = name
}
}
class GiftCard {
constructor( )
var cardNumber : String = generateCardNumber()
var pin: String? = null
set(pin) = if (pin != null) field = pin else { field = "Unknown User"; println("Please set the pin\nCard is not Created");}
var customerId : String = ""
set(customerId) = if (customerId != "") field = customerId else { field = "" }
var cardBalance : Double = 0.0
set(cardBalance) = if (cardBalance > 0) field = cardBalance else { field = 0.0; println("Card is created with zero balance\nPlease deposit") }
var status = Status.ACTIVE
constructor(cardNumber: String = generateCardNumber(),
pin: String,
customerId: String,
cardBalance: Double = 0.0,
status: Status = Status.ACTIVE){
this.cardNumber = cardNumber
this.pin = pin
this.customerId = customerId
this.cardBalance = cardBalance
this.status = status
}
}
This is the part of code, I have to be changed :
override fun closeCard(cardNumber: String, pin: String): Pair<Boolean, Boolean> {
for (giftcard in giftcards) {
if (giftcard.cardNumber == cardNumber) {
if (giftcard.pin == pin) {
giftcard.status = Status.CLOSED
for (account in accounts)
account.balance = account.balance + giftcard.cardBalance
giftcard.cardBalance = 0.0
return Pair(true,true)
}
\\invalid pin
return Pair(true,false)
}
}
\\card is not present
return Pair(false,false)
}
Both classes are not very idiomatic. The primary constructor of a Kotlin class is implicit and does not need to be defined, however, you explicitly define a constructor and thus you add another one that is empty.
// good
class C
// bad
class C {
constructor()
}
Going further, Kotlin has named arguments and default values, so make use of them.
class Account(
val id: String = generateAccountNumber(),
val name: String = "Unknown User",
val balance: Double = 0.0
)
Double is a very bad choice for basically anything due to its shortcomings, see for instance https://www.floating-point-gui.de/ Choosing Int, Long, heck even BigDecimal would be better. It also seems that you don’t want the balance to ever go beneath zero, in that case consider UInt and ULong.
Last but not least is the mutability of your class. This can make sense but it also might be dangerous. It is up to you to decide upon your needs and requirements.
enum class Status {
CLOSED
}
#ExperimentalUnsignedTypes
class Account(private var _balance: UInt) {
val balance get() = _balance
operator fun plusAssign(other: UInt) {
_balance += other
}
}
#ExperimentalUnsignedTypes
class GiftCard(
val number: String,
val pin: String,
private var _status: Status,
private var _balance: UInt
) {
val status get() = _status
val balance get() = _balance
fun close() {
_status = Status.CLOSED
_balance = 0u
}
}
#ExperimentalUnsignedTypes
class Main(val accounts: List<Account>, val giftCards: List<GiftCard>) {
fun closeCard(cardNumber: String, pin: String) =
giftCards.find { it.number == cardNumber }?.let {
(it.pin == pin).andAlso {
accounts.forEach { a -> a += it.balance }
it.close()
}
}
}
inline fun Boolean.andAlso(action: () -> Unit): Boolean {
if (this) action()
return this
}
We change the return type from Pair<Boolean, Boolean> to a more idiomatic Boolean? where Null means that we did not find anything (literally the true meaning of Null), false that the PIN did not match, and true that the gift card was closed. We are not creating a pair anymore and thus avoid the additional object allocation.
The Boolean.andAlso() is a handy extension function that I generally keep handy, it is like Any.also() from Kotlin’s STD but only executes the action if the Boolean is actually true.
There's probably a million different ways to do this, but here's one that at least has some language features I feel are worthy to share:
fun closeCard(cardNumber: String, pin: String): Pair<Boolean, Boolean> {
val giftCard = giftcards.find { it.cardNumber == cardNumber }
?: return Pair(false, false)
return if (giftCard.pin == pin) {
giftCard.status = Status.CLOSED
accounts.forEach {
it.balance += giftCard.cardBalance
}
Pair(true, true)
} else
Pair(true, false)
}
The first thing to notice if the Elvis operator - ?: - which evaluates the right side of the expression if the left side is null. In this case, if find returns null, which is equivalent to not finding a card number that matches the desired one, we'll immediately return Pair(false, false). This is the last step in your code.
From there one it's pretty straight forward. If the pins match, you loop through the accounts list with a forEach and close the card. If the pins don't match, then we'll go straight to the else branch. In kotlin, if can be used as an expression, therefore we can simply put the return statement before the if and let it return the result of the last expression on each branch.
PS: I won't say this is more efficient than your way. It's just one way that uses built-in functions - find and forEach - like you asked, as well as other language features.
PPS: I would highly recommend to try and find another way to update the lists without mutating the objects. I don't know your use cases, but this doesn't feel too thread-safe. I didn't post any solution for this, because it's outside the scope of this question.