I have two chaincodes - let's call them A and B - and I am trying to get A to invoke a method on B, in a setup that has privacy enabled. An example of the sort of call I'm trying to make is shown below.
func (e *ChaincodeA) someFuncOnChaincodeA(stub *shim.ChaincodeStub, args []string) ([]byte, error) {
//Do stuff
newArgs := []string{"somevalue1","somevalue2"}
msg, err := stub.InvokeChaincode(chaincodeBName,"someFuncOnChaincodeB",args)
if err != nil{
fmt.Println(err.Error())
}
return msg, err
}
However, whenever I try to run this, it gives me the following error messages before killing my chaincode:
[72047168]Error chaincode-chaincode interactions not supported for
with privacy enabled.
Sending ERROR Error starting Simple chaincode:
Error handling message:
[72047168-5f5a-4017-862a-1329660e2076]Chaincode handler FSM cannot
handle message (COMPLETED) with payload size (0) while in state: ready
Process finished with exit code 0
Evidently privacy interferes with chaincode-chaincode communications. Is there any way around this, to enable communications while maintaining privacy? Or is it a best-practice to put absolutely everything into a single gigantic chaincode?
Additionally, why does privacy interfere with chaincode-chaincode communications? I don't understand exactly why this occurs.
It appears that this is a known issue with hyperledger, that is being worked on. As such there is no current workaround, but there will likely be one in the future when this issue is dealt with.
Relevent Issue
Related
I created a TOPIC in google pubsub, and created a SUBSCRIPTION inside the TOPIC, with the following settings
then I wrote a puller in go, using its Receive to pull and acknowledge published messages
package main
import (
...
)
func main() {
ctx := context.Background()
client, err := pubsub.NewClient(ctx, config.C.Project)
if err != nil {
// do things with err
}
sub := client.Subscription(config.C.PubsubSubscription)
err := sub.Receive(ctx, func(ctx context.Context, msg *pubsub.Message) {
msg.Ack()
})
if err != context.Canceled {
logger.Error(fmt.Sprintf("Cancelled: %s", err.Error()))
}
if err != nil {
logger.Error(fmt.Sprintf("Error: %s", err.Error()))
}
}
Nothing fancy, its working well, but then after a while (~ after 3 hours idle), it stops receiving new published messages, no error(s), nothing. Am i missing something?
In general, there can be several reasons why a subscriber may stop receiving messages:
If a subscriber does not ack or nack messages, the flow control limits can be reached, meaning no more messages can be delivered. This does not seem to be the case in your particular instance given that you immediately ack messages.
If another subscriber starts up for the same subscription, it could be receiving the messages. In this scenario, one would expect the subscriber to receive a subset of the messages rather than no messages at all.
Publishers just stop publishing messages and therefore there are no messages to receive. If you restart the subscriber and it starts receiving messages again, this probably isn't the case. You can also verify that a backlog is being built up by looking at the Stackdriver metric for subscription/backlog_bytes.
If your problem does not fall into one of those categories, it would be best to reach out to Google Cloud support with your project name, topic name, and subscription name so that they can narrow down the issue to either your user code, the client library itself, or the service.
I was experiencing something similar and I was pretty sure there was not another subscriber pulling those messages.
Try this: go to the topic, create a new bogus subscription (name it whatever you want, because you'll just delete it later). Right after I did that both the fake subscription (which I was using the python sample code client to subscribe to) and the real one was receiving messages again. Strange solution, but maybe it kicked the topic awake again.
Hopefully someone from Google could give us some insight into what's happening here, but I'm definitely not paying them enough to get direct support.
Few changes will help you to investigate the issue better:
- Check error from Receive
- Use separate context for Receive
ctx := context.Background()
err := sub.Receive(ctx, func(ctx context.Context, msg *pubsub.Message) {
msg.Ack()
})
if err != nil {
log.Fatal(err)
}
Does your code work before? I have problems with PubSub since today. Methods like get_topic(), create_topic() in Python PubSub library stop working, but I don't have any problems with sending and pulling messages. Yesterday everything was working fine but today not...
I currently have a MQTT code that can subscribe to a topic, print out the messages received, then publish further instructions to a new topic. The subscribing/printing is completed in one Goroutine, and the publishing is done in another Goroutine. Here is my code:
var wg, pg sync.WaitGroup
// All messages are handled here - printing published messages and publishing new messages
var f MQTT.MessageHandler = func(client MQTT.Client, msg MQTT.Message) {
wg.Add(1)
pg.Add(1)
go func() {
defer wg.Done()
fmt.Printf("%s\n", msg.Payload())
//fmt.Println(os.Getpid())
}()
go func(){
defer pg.Done()
message := ""
//Changing configurations
if strings.Contains(string(msg.Payload()), "arduinoLED") == true {
message = fmt.Sprintf("change configuration")
}
if strings.Contains(string(msg.Payload()), "NAME CHANGED") == true{
message = fmt.Sprintf("change back")
}
// Publish further instructions to "sensor/instruction"
token := client.Publish("sensor/instruction", 0, false, message)
//fmt.Println(os.Getpid())
token.Wait()
}()
}
func main() {
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
opts := MQTT.NewClientOptions().AddBroker("tcp://test.mosquitto.org:1883")
opts.SetDefaultPublishHandler(f)
// Topic to subscribe to for sensor data
topic := "sensor/data"
opts.OnConnect = func(c MQTT.Client) {
if token := c.Subscribe(topic, 0, f); token.Wait() && token.Error() != nil {
panic(token.Error())
}
}
// Creating new client
client := MQTT.NewClient(opts)
if token := client.Connect(); token.Wait() && token.Error() != nil {
panic(token.Error())
} else {
fmt.Printf("Connected to server\n")
}
wg.Wait()
pg.Wait()
<-c
}
The commented out os.Getpid() line is to check which process I am running that Goroutine on. Right now they both display the same number (which means both are running on the same process?).
My question is: How can I run the two Goroutines on separate processes? Is there a way?
Edit: If this cannot be done, I want to write this code using channels. Here is the code I have for that:
var f MQTT.MessageHandler = func(client MQTT.Client, msg MQTT.Message) {
sensorData := make(chan []byte)
wg.Add(1)
pg.Add(1)
go func() {
defer wg.Done()
//fmt.Printf("%s\n", msg.Payload())
sensorData <- string(msg.Payload())
fmt.Println(<-sensorData) //currently not printing anything
}()
go func(){
defer pg.Done()
message := ""
//Changing configurations
if strings.Contains(<-sensorData, "arduinoLED") == true{
message = fmt.Sprintf("change configuration")
}
if strings.Contains(<-sensorData, "NAME CHANGED") == true{
message = fmt.Sprintf("change back")
}
// Publish further instructions to "sensor/instruction"
token := client.Publish("sensor/instruction", 0, false, message)
token.Wait()
}()
}
However, I am not able to print out any data using channels. What am I doing wrong?
You might be coming from Python, right? ;-)
It has the module named
multiprocessing
in its stdlib, and this might well explain why you have used
this name in the title of your question and why you apparently
are having trouble interpreting what #JimB meant by saying
If you need a separate process, you need to exec it yourself
"Multiprocessing" in Python
The thing is, Python's multiprocessing is a quite high-level
thing which hides under its hood a whole lot of stuff.
When you spawn a multiprocessing.Process and make it run
a function, what really happens is this:
The Python interpreter creates another operating system's
process (using
fork(2) on Unix-like systems
or CreateProcess on Windows) and arranges
for it to execute a Python interpter, too.
The crucial point is that you will now have two processes
running two Python interpters.
It is arranged for that Python interpterer in the
child process to have a way to communicate with the Python
interpreter in the parent process.
This "communication link" necessarily involves some form
of IPC #JimB referred to.
There is simply no other way to communicate data and actions
between separate processes exactly because a commodity
contemporary OS provides strict process separation.
When you exchange Python objects between the processes, the two communicating Python
interpreters serialize and deserialize them behind your back
before sending them over their IPC link and after receiving
them from there, correspondingly.
This is implemented using the pickle module.
Back to Go
Go does not have any direct solution which would closely
match Python's multiprocessing, and I really doubt it could
have been sensibly implemented.
The chief reason for this mostly stems from the fact Go
is quite more lower level than Python, and hence it does not
have the Python's luxury of making sheer assumptions about
the types of values it manages, and it also strives to have
as few hidden costs in its constructs as possible.
Go also strives to steer clear of "framework-style" approaches
to solve problems, and use "library-style" solutions when
possible. (A good rundown of the "framework vs library"
is given, for instance, here.)
Go has everything in its standard library to implement
something akin to Python's multiprocessing but there is no
ready-made frakework-y solution for this.
So what you could do for this is to roll along these lines:
Use os/exec to run another copy of your own process.
Make sure the spawned process "knows" it's started
in the special "slave" mode—to act accordingly.
Use any form of IPC to communicate with the new process.
Exchanging data via the standard I/O streams
of the child process is supposedly
the simplest way to roll (except when you need to exchange
opened files but this is a harder topic, so let's not digress).
Use any suitable package in the encoding/ hierarchy — such as binary, gob, xml — to serialize
and deserialize data when exchanging.
The "go-to" solution is supposedly encoding/gob
but encoding/json will also do just fine.
Invent and implement a simple protocol to tell the
child process what to do, and with which data,
and how to communicate the results back to master.
Does it really worth the trouble?
I would say that no, it doesn't—for a number of reasons:
Go has nothing like the dreaded GIL,
so there's no need to sidestep it to achieve real parallelism
when it is naturally possible.
Memory safety is all in your hands, and achieving it is
not really that hard when you dutifully obey the principle
that what is sent over a channel is now owned by
the receiver. In other words, sending values over a channel
is also the transfer of ownership of those values.
The Go toolchain has integrated race detector, so you
may run your test suite with the -race flag and create evaluation
builds of your program using go build -race for the same
purpose: when a program instrumented in such a way runs,
the race detector crashes it as soon as it detects any
unsynchronized read/write memory access.
The printout resulting from that crash includes
explanatory messages on what, and where went wrong,
with stack traces.
IPC is slow, so the gains may well be offset by the losses.
All-in-all, I see no real reason to separate processes unless
you're writing something like an e-mail processing server
where this concept comes naturally.
Channel is used for communicating between goroutines, you shouldn't use it in same goroutine like this code:
sensorData <- string(msg.Payload())
fmt.Println(<-sensorData) //currently not printing anything
If you like to test printing by channel, you can use buffered channel in same goroutine to avoid blocking, like this:
sensorData := make(chan []byte, 1)
Cheers
I have been using golang to automate some deploy processes and I had to use exec package to call some bash scripts.
I used exec.Command("/home/rodrigo/my-deploy.sh").CombinedOutput() and I saw his implementation
func (c *Cmd) CombinedOutput() ([]byte, error) {
if c.Stdout != nil {
return nil, errors.New("exec: Stdout already set")
}
if c.Stderr != nil {
return nil, errors.New("exec: Stderr already set")
}
var b bytes.Buffer
c.Stdout = &b
c.Stderr = &b
err := c.Run()
return b.Bytes(), err
}
I realized you can't assign c.Stdout when using CombinedOutput() and I think that's ok but the way it is informed to the api caller is not correct.
CombinedOutput() return an error when you are using it in a bad way, so if you are going to use CombinedOutput() then you shouldn't assign c.Stderr or c.Stdout previously, if you do that then you are going to receive an error.
But this error is not because your script throw an error, it is because you are using the api wrong, in that case I believe you should get a panic because a bad api usage should not being handled (I think).
I come from Java World and when you are using some method in the wrong way then you receive an RuntimeException, for example.
public void run(Job job) throws NotCompletedJob {
if (job.getId() != null) {
throw new IllegalArgumentException("This job should not have id");
}
job.setId(calculateId());
job.run();
}
With this signature I can know I'm wrong calling run(obj); with a Job that has an id and in fact I can distinguish if there is an error with my script or I'm using api in a wrong way.
NotCompletedJob is a checked exception so I must handle it but IllegalArgumentException is not so I could get it anytime. Catching IllegalArgumentException or any other RuntimeException is not always considered a good practice because they are indicating you have an error from programmers's point of view and it is not a possible expected error like NotCompletedJob.
Having said that, How can I differentiate between a programming error (bad api usages for example) from an expected error (script doesn't finished ok) with current CombinedOutput() implementation ?
To clarify my concern, I'm not saying that is wrong the current implementation of CombinedOuput, but I don't understand how the caller could distinguish if it is an error of the command being executed or an error caused from his bad api usage.
I believe that a best approach would be to panicking when the caller is using the api in a wrong way, as the same case when the caller is passing a nil reference to a function that expect a non nil reference (in fact this is the current behaviour).
I come from Java World and when you are using some method in the wrong
way then you receive an RuntimeException.
You are in the Go world now. Therefore, that argument is invalid. Abandon Java.
The Go Programming Language Specification
Handling panics
Two built-in functions, panic and recover, assist in reporting and
handling run-time panics and program-defined error conditions.
func panic(interface{})
func recover() interface{}
While executing a function F, an explicit call to panic or a run-time
panic terminates the execution of F. Any functions deferred by F are
then executed as usual. Next, any deferred functions run by F's caller
are run, and so on up to any deferred by the top-level function in the
executing goroutine. At that point, the program is terminated and the
error condition is reported, including the value of the argument to
panic. This termination sequence is called panicking.
The Go Blog
Defer, Panic, and Recover
The convention in the Go libraries is that even when a package uses
panic internally, its external API still presents explicit error
return values.
Go Code Review Comments
This page collects common comments made during reviews of Go code, so
that a single detailed explanation can be referred to by shorthands.
This is a laundry list of common mistakes, not a style guide.
Don't Panic
See https://golang.org/doc/effective_go.html#errors. Don't use panic
for normal error handling. Use error and multiple return values.
Effective Go
Errors
The usual way to report an error to a caller is to return an error as
an extra return value.
Your Go server program is concurrently handling 100,000 clients. If an error occurs, report and handle it; always check for errors. DON'T crash all 100,000 clients with a panic. Go packages should not panic.
Read the Go documentation and the Go standard library code.
Go provides two ways of handling errors, but I'm not sure which one to use.
Assuming I'm implementing a classic ForEach function which accepts a slice or a map as an argument. To check whether an iterable is passed in, I could do:
func ForEach(iterable interface{}, f interface{}) {
if isNotIterable(iterable) {
panic("Should pass in a slice or map!")
}
}
or
func ForEach(iterable interface{}, f interface{}) error {
if isNotIterable(iterable) {
return fmt.Errorf("Should pass in a slice or map!")
}
}
I saw some discussions saying panic() should be avoided, but people also say that if program cannot recover from error, you should panic().
Which one should I use? And what's the main principle for picking the right one?
You should assume that a panic will be immediately fatal, for the entire program, or at the very least for the current goroutine. Ask yourself "when this happens, should the application immediately crash?" If yes, use a panic; otherwise, use an error.
Use panic.
Because your use case is to catch a bad use of your API. This should never happen at runtime if the program is calling your API properly.
In fact, any program calling your API with correct arguments will behave in the same way if the test is removed. The test is there only to fail early with an error message helpful to the programmer that did the mistake. Ideally, the panic might be reached once during development when running the testsuite and the programmer would fix the call even before committing the bad code, and that incorrect use would never reach production.
See also this reponse to question Is function parameter validation using errors a good pattern in Go?.
I like the way it's done in some libraries where on top of a regular method DoSomething, its "panicky" version is added with MustDoSomething. I'm relatively new to go, but I've already seen it in several places, notably sqlx.
In general, if you want to expose your code to someone else, you should either have Must- and a regular version of the method, or your methods/functions should give the client a chance to recover the way they want and so error should be available to them in a go-idiomatic way.
Having said that, I agree that if your API/library is used inappropriately, it's Ok to panic as well. As a matter of fact, I've also seen methods like MustGetenv() that will panic if a critical env.var is missing. Fail-fast mechanism basically.
If some mandatory requirement is not provided or not there while starting the service (eg. database connection, some service configuration which is required) then you should use panic.
There should be return error for any user response or server side error.
Ask yourself these questions:
Do you expect the exceptional situation to occur, regardless how well would you code your app? Do you think it should be useful to make the user aware of such condition as part of the normal usage of your app? Handle it as an error, because it concerns the application as working normally.
Should that exceptional situation NOT occur if you code appropriately (and somewhat defensively)? (example: dividing by zero, or accessing an array element out of bounds) Is your app totally clueless under that error? Panic.
Do you have your API and want to ensure users use it appropriately? Panic. Your API will seldom recover if used incorrectly.
Use error whenever possible
Only use panic when your code could end up in a bad state that would be prone to crashing; something truly unexpected. The example above with ForEach() is an exported func that accepts an interface so it should expect someone will improperly call it. And if it is improperly called, you know why you cannot continue and you know how to handle that error. isNotIterable is literally binary and easy to control.
But error is not like a try/catch
Even if you try to justify panic/recover by looking at throw/catch from other languages, you still use errors. We know you are trying the function because you are calling it, we know there was an error because err != nil, and just like checking the type of exception thrown you can check the type of error returned with errors.Is(err, ErrNotIterable)
So should you use panic for errors in concurrency?
The answer is still most likely no. Errors are still the preferred way in Go and you can use a wait group to shut down the goroutines:
ctx, cancel := context.WithTimeout(context.Background(), time.Minute*5)
// automatically cancel in 5 min
defer cancel()
errGroup, ctx := errgroup.WithContext(ctx)
errGroup.Go(func() error {
// do crazy stuff; you can still check for errors
if ... {
return fmt.Errorf("critical error, stopping all goroutines now")
}
// code completed without issues
return nil
})
err = errGroup.Wait()
Even using the structure of the original example, you still have better control with errors than panics:
func ForEach(iterable interface{}, f interface{}) error {
if isNotIterable(iterable) {
return fmt.Errorf("expected something iterable but got %v", reflect.ValueOf(iterable).String())
}
switch v.Kind() {
case reflect.Map:
...
case reflect.Array, reflect.Slice:
...
default:
return fmt.Errorf("isNotIterable is false but I do not know how to iterate through %v", reflect.ValueOf(iterable).String())
}
But error feels very verbose
Yes, that is the point. When an error is returned, it is at that point to do something about it. You are giving the calling code options rather than making the decision to start shutting down and killing the application unless you recover(). If you are just returning the same error all the way up the call stack then error will seem inferior to panic, but this is due to not addressing issues when they happen.
So when to use panic?
When your code is on a collision course to crash and you cannot assume your way out of it. Another is when the code assumes something that is no longer true and having to check the integrity in every function from here on out would be tedious (and might impact performance). Still, you would use panic() only to get out of the layers of uncertainty... then still handle errors:
func ForEach(iterable interface{}, f interface{}) error {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("cannot iterate due to unexpected runtime error %v", r)
return
}
}()
...
// perhaps a broken pipe in a global var
// or an included module threw a panic at you!
}
But if you are still not convinced... Here is the Go FAQ
We believe that coupling exceptions to a control structure, as in the try-catch-finally idiom, results in convoluted code. It also tends to encourage programmers to label too many ordinary errors, such as failing to open a file, as exceptional.
Go takes a different approach. For plain error handling, Go's multi-value returns make it easy to report an error without overloading the return value. A canonical error type, coupled with Go's other features, makes error handling pleasant but quite different from that in other languages.
A panic typically means something went unexpectedly wrong. Mostly used to fail fast on errors that shouldn’t occur during normal operation, or that we aren’t prepared to handle gracefully. So in this case just return the error, you don't want your program to panic.
I think none of the previous answers are correct:
By default, if we don't know what to do with the "error" code must panic following best programming patterns:
https://en.wikipedia.org/wiki/Fail-fast
Putting it more formally, our "Turing Machine" is broken and we need to come back to an "stable state" or "reset state". More info at https://en.wikipedia.org/wiki/Reset_(computing)
For example in web (micro)services that means returning a 40X error (panic caused by input from user) or 50X error (panic caused by something else - hardware, network, assert error, ...)
If we know what to do with the "error", then we do not have an error in first place, but an uncomfortable return value. This is a normal execution condition and probably not an error. Normally this correspond to the happy vs non-happy path modeling.
In a summary, the err return value is mostly a wrong idea, even if the GO community has adopted it as a religion. Using error return values is just a patchy way to speed up program execution since it require fewer CPU instructions to be implemented, but most of the time, except for low-level services, it is useless and promote dirty code. (note that GO was designed to implement those low-level services as an "easy-C", but it was adopted for high-level (Level 7) application programs when an error must fail fast to avoid continuing with undefined states that can potentially cause money being lost of fatal casualties. In case of doubt, default to panic.
Don't use panic for normal error handling. Use error and multiple return values. See https://golang.org/doc/effective_go.html#errors.
Speaking as a new go enthusiast trying to work with the go way of error handling. To be clear - I like exceptions.
I have a server that accepts a connection , processes a set of requests and replies to them. I found that I can do
if err != nil{
panic(err)
}
in the deep down processing code
and have
defer func() {
if err := recover(); err != nil {
log.Printf("%s: %s", err, debug.Stack()) // line 20
}
}()
in the client connection code (each connection is in a goroutine). This nicely wraps everything up, forcefully closes the connection (other defers fire) and my server continues to hum along.
But this feels an awful lot like a throw/catch scenario - which golang states it doesn't support. Questions
is this stable. ie recovering a panic is an OK thing to do as an
ongoing way of life. Its not intended to just slightly defer an
immediate shutdown
I looked for a discussion on this topic and did not find it anywhere - any pointers?
I feel that the answer is 'yes it works' and can be used inside you own code, but panic should NOT be used by a library intended for wider use. The standard and polite way for a library to behave is by error returns
Yes, you can do what you suggest. There are some situations within the standard packages where panic/recover is used for handling errors. The official Go blog states:
For a real-world example of panic and recover, see the json package
from the Go standard library. It decodes JSON-encoded data with a set
of recursive functions. When malformed JSON is encountered, the parser
calls panic to unwind the stack to the top-level function call, which
recovers from the panic and returns an appropriate error value (see
the 'error' and 'unmarshal' methods of the decodeState type in
decode.go).
Some pointers:
Use error for your normal use cases. This should be your default.
If your code would get clearer and simpler by using a panic/recover (such as with a recursive call stack), then use it for that particular case.
Never let a package leak panics. Panics used within a package should be recovered within the package and returned as an error.
Recovering from a panic is stable. Don't worry about continuing execution after a recover. You can see such behavior in standard library such as with the net/http package which recovers from panics within handlers to prevent the entire http server to go crash when panicing on a single request.
Generally most methods won't panic, they will return an error instead, and there's a bit of an overhead of using defer.
So yes, it does work, but the "proper" / "go" way is to return an error instead of using panic / recover.