I'm now working with Golang to connect IBM Websphere MQ-ESB and libraries which I use to communicate with MQ-ESB are
https://github.com/ibm-messaging/mq-golang
https://github.com/ibm-messaging/mq-golang-jms20
Generally, it can send the messages to MQ-ESB , but when something wrong happen the error MQ Connection Broken. This cause my app could not any longer send the messages to the MQ-ESB. Restarting service is the way to solve this issue (but it is not the way solve). Anyone has ideas?
Thanks
This is code create mq connection
func NewIBMMQConnection(mqConnConfig *mqjms.ConnectionFactoryImpl) jms20subset.JMSContext {
if !viper.GetBool("mq.openconnection") {
return &mqjms.ContextImpl{}
}
logx.WithFields(logrus.Fields{
"queue manager": viper.GetString("mq.qManager"),
"host": viper.GetString("mq.host"),
"port": viper.GetInt("mq.port"),
"channel": viper.GetString("mq.qChannel"),
}).Infof("[CONFIG] [MDM IBMMQ]")
conn, exception := mqConnConfig.CreateContext()
if exception != nil {
if exception.GetLinkedError() != nil {
logx.Fatalf("new mdm mq error: %s", exception.GetLinkedError())
}
}
return conn
}
func NewIBMMQConfig() *mqjms.ConnectionFactoryImpl {
return &mqjms.ConnectionFactoryImpl{
QMName: viper.GetString("mq.qManager"),
Hostname: viper.GetString("mq.host"),
PortNumber: viper.GetInt("mq.port"),
ChannelName: viper.GetString("mq.qChannel"),
UserName: viper.GetString("mq.login"),
Password: viper.GetString("mq.pass"),
}
}
This is the code in main.go that instantiate the connection
func main() {
db := newGormDB()
defer closeDB(db)
mq := ibmmq.NewIBMMQConnection(ibmmq.NewIBMMQConfig())
defer mq.Close()
ibmmq := ibmmq.New(mq)
...
...
go startServer()
shutdown()
}
This is code produce message
func (i *IBMMQ) ProduceMSGToMQ(ctx context.Context, msg string) error {
logx.WithContext(ctx).Infof("Producing Message queueName: message: %s", msg)
err := i.producer.SendString(i.queueCDDEMoeny, msg)
if err != nil {
logx.WithSeverityError(ctx).Errorf("Send msg to mq error: %s", err.GetErrorCode()+"-"+err.GetReason()+"-"+err.GetLinkedError().Error())
return errors.New("Send msg to mq error: " + err.GetErrorCode() + "-" + err.GetReason() + "-" + err.GetLinkedError().Error())
}
return nil
}
I'm not a Go language expert, only an IBM MQ expert, but here goes (no pun intended!).
I don't see where you call your ProduceMSGToMQ function, but I imagine something like this:-
error := ibmmq.ProduceMSGToMQ(...)
if error != nil && (error.GetReason() == "MQRC_CONNECTION_BROKEN") {
mq := ibmmq.NewIBMMQConnection(ibmmq.NewIBMMQConfig())
}
Hope that helps a little.
We also had to restart service to restore connection on MQ client.
But the problem was resolved after setting the option to reconnect.
mqcno := ibmmq.NewMQCNO()
...
...
mqcno.Options = ibmmq.MQCNO_CLIENT_BINDING
mqcno.Options |= ibmmq.MQCNO_RECONNECT_Q_MGR
Related
AWS SDK Version: v1.38.19
Go Version: go1.15.7
alpine 3.7
I am using the standard queue
I am initializing the SQS connection at once in my application like this;
// Connection connection to the SQS
var Connection *sqs.SQS
// InitSQS initialize the AWS SQS connection
func InitSQS() {
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
Connection = sqs.New(sess, &aws.Config{
Region: aws.String("eu-west-1"),
DisableSSL: aws.Bool(true),
})
}
I am disabling the SSL because; I am having memory and CPU leaks when I am going with the SSL in my application (my application isn't open for the rest of the world btw it's an internal service for my other applications).
Here is the config I use to read a message from SQS:
func ConsumeUpdateMessage(db *database.MySQLWrap, sqsApi queue.SQSAPI) error {
result, err := sqsApi.ReceiveMessage(&sqs.ReceiveMessageInput{
AttributeNames: []*string{
aws.String(sqs.MessageSystemAttributeNameSentTimestamp),
},
MessageAttributeNames: []*string{
aws.String(sqs.QueueAttributeNameAll),
},
QueueUrl: &qURL,
MaxNumberOfMessages: aws.Int64(10),
WaitTimeSeconds: aws.Int64(20),
})
if err != nil {
return fmt.Errorf("error on receiving the message from queue: %s", err.Error())
}
for _, msg := range result.Messages {
// business logic
}
return err
}
this is how I am calling the ConsumeUpdateMessage method;
// InitializeUpdateMessage ..
func InitializeUpdateMessage(db *database.MySQLWrap, sqsApi queue.SQSAPI) {
go func() {
for {
time.Sleep(500 * time.Millisecond)
err := ConsumeUpdateMessage(db, sqsApi)
if err != nil {
log.Error(err)
continue
}
}
}()
}
but sometimes my subscriber return error like this;
*awserr.baseError: RequestError: send request failed
caused by: Post "http://sqs.eu-west-1.amazonaws.com/": dial tcp xx.x.xx.xxx:80: i/o timeout
(Note: I put xx instead of sharing the IP Address)
I made my search over the forums and other places but I can't find a solution for these two-issue,
Memory leak when I using the connection with SSL (idea: but since my application is an internal service I think I don't have to use with SSL)
i/o timeout
I am trying to connect to the Darwin Push port feed located here. When I try to connect following the instructions from wiki I get 1 of 2 errors.
If I leave out the subscription options I get an authentication error,
and if I include them it says I cannot create a durable subscription to a queue.
If someone could just tell me which error is closer to being connected that would be a great help.
package main
import (
"fmt"
"github.com/Showmax/go-fqdn"
"github.com/go-stomp/stomp"
"github.com/go-stomp/stomp/frame"
)
var serverAddr = "darwin-dist-44ae45.nationalrail.co.uk:61613"
var messageCount = 10
var topic = "topic/darwin.pushport-v16"
var username = "####"
var password = "###"
var stop = make(chan bool)
fqdn, err := fqdn.FqdnHostname()
if err != nil {
panic(err)
}
var connOptions []func(*stomp.Conn) error = []func(*stomp.Conn) error{
stomp.ConnOpt.Login(username, password),
stomp.ConnOpt.Host(serverAddr),
stomp.ConnOpt.Header("client-id", fmt.Sprintf("%v-%v", username, fqdn)),
}
conn, err := stomp.Dial("tcp", serverAddr, connOptions...)
if err != nil {
println("cannot connect to server", err.Error())
return
}
subOptions := []func(*frame.Frame) error{
stomp.SubscribeOpt.Header("activemq.subscriptionName", fqdn),
stomp.SubscribeOpt.Header("durable-subscription-name", fqdn),
}
sub, err := conn.Subscribe(topic, stomp.AckClient, subOptions...)
if err != nil {
println("cannot subscribe to", topic, err.Error())
return
}
for i := 1; i <= messageCount; i++ {
msg := <-sub.C
fmt.PrintLn(msg)
}
}
println("receiver finished")
}
All I get is:
2020/09/04 16:55:35 received ERROR; Closing underlying connection
2020/09/04 16:55:35 Subscription 1: topic/darwin.pushport-v16: ERROR message:Invalid Subscription: cannot durably subscribe to a Queue destination!
Expected: Message #1
Actual: org.apache.activemq.transport.stomp.ProtocolException: Invalid Subscription: cannot durably subscribe to a Queue destination!
at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSubscribe(ProtocolConverter.java:581)
at ...
If I remove the subscription headers:
subOptions := []func(*frame.Frame) error{
// stomp.SubscribeOpt.Header("activemq.subscriptionName", fqdn),
// stomp.SubscribeOpt.Header("durable-subscription-name", fqdn),
}
I get this:
2020/09/04 17:00:13 received ERROR; Closing underlying connection
2020/09/04 17:00:13 Subscription 1: topic/darwin.pushport-v16: ERROR message:User REDACTED is not authorized to read from: queue://topic/darwin.pushport-v16
Expected: Message #1
Actual: java.lang.SecurityException: User REDACTED is not authorized to read from: queue://topic/darwin.pushport-v16
at org.apache.activemq.security.AuthorizationBroker.addConsumer(AuthorizationBroker.java:159)
at org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:104)
at org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:104)
at
I have tried both of these approaches with and without the topic/ proceeding the topic.
Any help would be greatly appreciated.
To take Tim Bish's answer to its logical conclusion, you need to add a leading slash to your topic description, so that it reads /topic/darwin.pushport-v16 rather than just topic/darwin.pushport-v16. Then it will be correctly identified as a topic (rather than defaulting to a queue) and should "just work".
The destination prefix in ActiveMQ for STOMP clients for Queues and Topics are /queue/ or /topic/ respectively and the default when there is no prefix or in your case an invalid one is to use a Queue.
I'm trying to implement a websocket proxy server for GlassFish. If I try to connect more than one client I'm getting error:
ReadMessage Failed: websocket: close 1007 Illegal UTF-8 Sequence.
I'm sure the GlassFish server sending right data, because the same server works properly with another proxy server implemented with node.js.
func GlassFishHandler(conn *websocket.Conn){
defer conn.Close()
conn.SetReadDeadline(time.Now().Add(1000 * time.Second))
conn.SetWriteDeadline(time.Now().Add(1000 * time.Second))
fmt.Println("WS-GOLANG PROXY SERVER: Connected to GlassFish")
for {
messageType, reader, err := conn.NextReader()
if err != nil {
fmt.Println("ReadMessage Failed: ", err) // <- error here
} else {
message, err := ioutil.ReadAll(reader)
if (err == nil && messageType == websocket.TextMessage){
var dat map[string]interface{}
if err := json.Unmarshal(message, &dat); err != nil {
panic(err)
}
// get client destination id
clientId := dat["target"].(string)
fmt.Println("Msg from GlassFish for Client: ", dat);
// pass through
clients[clientId].WriteMessage(websocket.TextMessage, message)
}
}
}
}
Summing up my comments as an answer:
When you are writing to the client, you are taking the clientId from the GlassFish message, fetching the client from a map, and then writing to it - basically clients[clientId].WriteMessage(...).
While your map access can be thread safe, writing is not, as this can be seen as:
// map access - can be safe if you're using a concurrent map
client := clients[clientId]
// writing to a client, not protected at all
client.WriteMessage(...)
So what's probably happening is that two separate goroutines are writing to the same client at the same time. You should protect your client from it by adding a mutex in the WriteMessage method implementation.
BTW actually instead of protecting this method with a mutex, a better, more "go-ish" approach would be to use a channel to write the message, and a goroutine per client that consumes from the channel and writes to the actual socket.
So in the client struct I'd do something like this:
type message struct {
msgtype string
msg string
}
type client struct {
...
msgqueue chan *message
}
func (c *client)WriteMessage(messageType, messageText string) {
// I'm simplifying here, but you get the idea
c.msgqueue <- &message{msgtype: messageType, msg: messageText}
}
func (c *client)writeLoop() {
go func() {
for msg := ragne c.msgqueue {
c.actuallyWriteMessage(msg)
}
}()
}
and when creating a new client instance, just launch the write loop
Is it actually possible to do RPC calls from a server to a client with the net/rpc package in Go? If no, is there a better solution out there?
I am currently using thrift (thrift4go) for server->client and client->server RPC functionality. By default, thrift does only client->server calls just like net/rpc. As I also required server->client communication, I did some research and found bidi-thrift. Bidi-thrift explains how to connect a java server + java client to have bidirectional thrift communication.
What bidi-thrift does, and it's limitations.
A TCP connection has an incomming and outgoing communication line (RC and TX). The idea of bidi-thrift is to split RS and TX and provide these to a server(processor) and client(remote) on both client-application and server-application. I found this to be hard to do in Go. Also, this way there is no "response" possible (the response line is in use). Therefore, all methods in the service's must be "oneway void". (fire and forget, call gives no result).
The solution
I changed the idea of bidi-thrift and made the client open two connections to the server, A and B. The first connection(A) is used to perform client -> server communication (where client makes the calls, as usual). The second connection(B) is 'hijacked', and connected to a server(processor) on the client, while it is connected to a client(remote) on the server. I've got this working with a Go server and a Java client. It works very well. It's fast and reliable (just like normal thrift is).
Some sources.. The B connection (server->client) is set up like this:
Go server
// factories
framedTransportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
// create socket listener
addr, err := net.ResolveTCPAddr("tcp", "127.0.0.1:9091")
if err != nil {
log.Print("Error resolving address: ", err.Error(), "\n")
return
}
serverTransport, err := thrift.NewTServerSocketAddr(addr)
if err != nil {
log.Print("Error creating server socket: ", err.Error(), "\n")
return
}
// Start the server to listen for connections
log.Print("Starting the server for B communication (server->client) on ", addr, "\n")
err = serverTransport.Listen()
if err != nil {
log.Print("Error during B server: ", err.Error(), "\n")
return //err
}
// Accept new connections and handle those
for {
transport, err := serverTransport.Accept()
if err != nil {
return //err
}
if transport != nil {
// Each transport is handled in a goroutine so the server is availiable again.
go func() {
useTransport := framedTransportFactory.GetTransport(transport)
client := worldclient.NewWorldClientClientFactory(useTransport, protocolFactory)
// Thats it!
// Lets do something with the connction
result, err := client.Hello()
if err != nil {
log.Printf("Errror when calling Hello on client: %s\n", err)
}
// client.CallSomething()
}()
}
}
Java client
// preparations for B connection
TTransportFactory transportFactory = new TTransportFactory();
TProtocolFactory protocolFactory = new TBinaryProtocol.Factory();
YourServiceProcessor processor = new YourService.Processor<YourServiceProcessor>(new YourServiceProcessor(this));
/* Create thrift connection for B calls (server -> client) */
try {
// create the transport
final TTransport transport = new TSocket("127.0.0.1", 9091);
// open the transport
transport.open();
// add framing to the transport layer
final TTransport framedTransport = new TFramedTransport(transportFactory.getTransport(transport));
// connect framed transports to protocols
final TProtocol protocol = protocolFactory.getProtocol(framedTransport);
// let the processor handle the requests in new Thread
new Thread() {
public void run() {
try {
while (processor.process(protocol, protocol)) {}
} catch (TException e) {
e.printStackTrace();
} catch (NullPointerException e) {
e.printStackTrace();
}
}
}.start();
} catch(Exception e) {
e.printStackTrace();
}
I came across rpc2 which implements it. An example:
Server.go
// server.go
package main
import (
"net"
"github.com/cenkalti/rpc2"
"fmt"
)
type Args struct{ A, B int }
type Reply int
func main(){
srv := rpc2.NewServer()
srv.Handle("add", func(client *rpc2.Client, args *Args, reply *Reply) error{
// Reversed call (server to client)
var rep Reply
client.Call("mult", Args{2, 3}, &rep)
fmt.Println("mult result:", rep)
*reply = Reply(args.A + args.B)
return nil
})
lis, _ := net.Listen("tcp", "127.0.0.1:5000")
srv.Accept(lis)
}
Client.go
// client.go
package main
import (
"fmt"
"github.com/cenkalti/rpc2"
"net"
)
type Args struct{ A, B int }
type Reply int
func main(){
conn, _ := net.Dial("tcp", "127.0.0.1:5000")
clt := rpc2.NewClient(conn)
clt.Handle("mult", func(client *rpc2.Client, args *Args, reply *Reply) error {
*reply = Reply(args.A * args.B)
return nil
})
go clt.Run()
var rep Reply
clt.Call("add", Args{5, 2}, &rep)
fmt.Println("add result:", rep)
}
RPC is a (remote) service. Whenever some computer requests a remote service then it is acting as a client asking the server to provide the service. Within this "definition" the concept of a server calling client RPC has no well defined meaning.
Does anyone know how to create a SOCKET as returned by WSASocket() function in go programming language?
Using a normal syscall.Socket type syscall.Bind results in:
WSAENOTSOCK - Error 10038 - An operation was attempted on something that is not a socket. The specified socket parameter refers to a file, not a socket.
Thanks
We don't use such low level API, we use net.Dial. ex.
func main() {
var (
host = "127.0.0.1"
port = "9998"
remote = host + ":" + port
msg string = "test"
)
con, error := net.Dial("tcp4", remote)
if error != nil {
fmt.Printf("Host not found: %s\n", error)
os.Exit(1)
} else {
defer con.Close()
}
in, error := con.Write([]byte(msg))
if error != nil {
fmt.Printf("Error sending data: %s, in: %d\n", error, in)
os.Exit(2)
}
fmt.Println("Connection OK")
}
Or, you could trace the code $GOROOT/src/pkg/net/dial.go