I need to publish a task by name (creating a message on a queue with task name passed in headers, passing some args).
I've done this in Celery (Python) like this:
celery_app.send_task("task_name", args=("arg1", "arg2",))
Here's my code in Go to do the same thing:
headers := make(amqp.Table)
headers["argsrepr"] = []string{"arg1", "arg2"}
headers["task"] = "task_name"
body := ""
jsonBody, _ := json.Marshal(body)
err = ch.Publish(
"", // exchange
"queue_name", // routing key
false, // mandatory
false, // immediate
amqp.Publishing {
DeliveryMode: amqp.Persistent,
ContentType: "application/json",
Body: jsonBody,
Headers: headers,
})
But I get this error:
Failed to publish a message: table field "argsrepr" value [%!t(string=arg1) %!t(string=arg2)] not supported
I'm using "github.com/streadway/amqp" library to talk with my rabbitmq node. It seems that "send task by name" is not implemented in this library.
Here is an illustration of the symptoms. The fmt format string is apparently expecting boolean values.
package main
import "fmt"
func main() {
s := fmt.Sprintf("%t %t", "arg1", "arg2")
fmt.Println(s)
t := fmt.Sprintf("%t %t", true, false)
fmt.Println(t)
}
Output:
%!t(string=arg1) %!t(string=arg2)
true false
Related
I am learning RabbitMQ through building a small application in GoLang - following this example here: https://www.rabbitmq.com/tutorials/tutorial-one-go.html. My project has the following structure:
project
└───cmd
│ └───api
│ │ main.go
│ └───internal
│ │ rabbitmq.go
And in cmd/internal/rabbitmq.go I have the following code - (errs are delt with):
import (
...
amqp "github.com/rabbitmq/amqp091-go"
)
func NewRabbitMQ() (*RabbitMQ, error) {
// Initialise connection to RabbitMQ
conn, err := amqp.Dial("amqp://guest:guest#localhost:5672/")
// Initialise Channel
ch, err := conn.Channel()
// Initialise Queue
q, err := ch.QueueDeclare(
"hello", // name
false, // durable
false, // delete when unused
false, // exclusive
false, // no-wait
nil, // arguments
)
// Set Context
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
body := "Hello World!"
err = ch.PublishWithContext( // errors here
ctx, // context
"", // exchange
q.Name, // routing key
false, // mandatory
false, // immediate
amqp.Publishing{
ContentType: "text/plain",
Body: []byte(body),
})
return &RabbitMQ{}, nil
}
As far as I can tell, from the documentation, this is how it should be implemented, so I'm not sure why it is erroring.
I've tried googling to find help with this issue but to no avail. I am using Go 1.19, maybe it is an issue with that?
Thanks to #oakad, this was a simple fix! The issue was the version of RabbitMQ I was using was an older one so I just need to change my go.mod from:
require github.com/rabbitmq/amqp091-go v1.1.0
To:
require github.com/rabbitmq/amqp091-go v1.4.0
EDIT: Adrian's suggestion makes sense, so I moved my code into a function and called the function from my cobra block:
package cmd
import (
"fmt"
"log"
"os"
"io"
"github.com/spf13/cobra"
"github.com/spf13/viper"
input "github.com/tcnksm/go-input"
)
var configureCmd = &cobra.Command{
Use: "configure",
Short: "Configure your TFE credentials",
Long: `Prompts for your TFE API credentials, then writes them to
a configuration file (defaults to ~/.tgc.yaml`,
Run: func(cmd *cobra.Command, args []string) {
CreateConfigFileFromPrompts(os.Stdin, os.Stdout)
},
}
func CreateConfigFileFromPrompts(stdin io.Reader, stdout io.Writer) {
ui := &input.UI{
Writer: stdout,
Reader: stdin,
}
tfeURL, err := ui.Ask("TFE URL:", &input.Options{
Default: "https://app.terraform.io",
Required: true,
Loop: true,
})
if err != nil {
log.Fatal(err)
}
viper.Set("tfe_url", tfeURL)
tfeAPIToken, err := ui.Ask(fmt.Sprintf("TFE API Token (Create one at %s/app/settings/tokens)", tfeURL), &input.Options{
Default: "",
Required: true,
Loop: true,
Mask: true,
MaskDefault: true,
})
if err != nil {
log.Fatal(err)
}
viper.Set("tfe_api_token", tfeAPIToken)
configPath := ConfigPath()
viper.SetConfigFile(configPath)
err = viper.WriteConfig()
if err != nil {
log.Fatal("Failed to write to: ", configPath, " Error was: ", err)
}
fmt.Println("Saved to", configPath)
}
So what can I pass to this method to test that the output is as expected?
package cmd
import (
"strings"
"testing"
)
func TestCreateConfigFileFromPrompts(t *testing.T) {
// How do I pass the stdin and out to the method?
// Then how do I test their contents?
// CreateConfigFileFromPrompts()
}
func TestCreateConfigFileFromPrompts(t *testing.T) {
var in bytes.Buffer
var gotOut, wantOut bytes.Buffer
// The reader should read to the \n each of two times.
in.Write([]byte("example-url.com\nexampletoken\n"))
// wantOut could just be []byte, but for symmetry's sake I've used another buffer
wantOut.Write([]byte("TFE URL:TFE API Token (Create one at example-url.com/app/settings/tokens)"))
// I don't know enough about Viper to manage ConfigPath()
// but it seems youll have to do it here somehow.
configFilePath := "test/file/location"
CreateConfigFileFromPrompts(&in, &gotOut)
// verify that correct prompts were sent to the writer
if !bytes.Equal(gotOut.Bytes(), wantOut.Bytes()) {
t.Errorf("Prompts = %s, want %s", gotOut.Bytes(), wantOut.Bytes())
}
// May not need/want to test viper's writing of the config file here, or at all, but if so:
var fileGot, fileWant []byte
fileWant = []byte("Correct Config file contents:\n URL:example-url.com\nTOKEN:exampletoken")
fileGot, err := ioutil.ReadFile(configFilePath)
if err != nil {
t.Errorf("Error reading config file %s", configFilePath)
}
if !bytes.Equal(fileGot, fileWant) {
t.Errorf("ConfigFile: %s not created correctly got = %s, want %s", configFilePath, fileGot, fileWant)
}
}
As highlighted by #zdebra in comments to his answer, the go-input package is panicing and giving you the error: Reader must be a file. If you are married to using that package, you can avoid the problem by disabling the masking option on the ui.Ask for your second input:
tfeAPIToken, err := ui.Ask(fmt.Sprintf("TFE API Token (Create one at %s/app/settings/tokens)", tfeURL), &input.Options{
Default: "",
Required: true,
Loop: true,
//Mask: true, // if this is set to True, the input must be a file for some reason
//MaskDefault: true,
})
The reader and the writer need to be set up before the tested function is called. After is called, the result is written into the writer where it should be verified.
package cmd
import (
"strings"
"testing"
)
func TestCreateConfigFileFromPrompts(t *testing.T) {
in := strings.NewReader("<your input>") // you can use anything that satisfies io.Reader interface here
out := new(strings.Builder) // you could use anything that satisfies io.Writer interface here like bytes.Buffer
CreateConfigFileFromPrompts(in, out)
// here you verify the output written into the out
expectedOutput := "<your expected output>"
if out.String() != expectedOutput {
t.Errorf("expected %s to be equal to %s", out.String(), expectedOutput)
}
}
I started using AWS SAM and for now I only have some unit tests, but I want to try to run integration tests in a pre traffic hook function.
Unfortunately there seems to be no code example for Golang, all I could find was for Javascript.
From this example I pieced together that I have to use the code deploy SDK and call PutLifecycleEventHookExecutionStatus, but the specifics remain unclear. The aws code example repo for go has no examples for code deploy either.
More Information about the topic that I am looking for is available here https://github.com/awslabs/serverless-application-model/blob/master/docs/safe_lambda_deployments.rst#pretraffic-posttraffic-hooks.
I want to start out by testing a lambda function that simply queries DynamoDB.
Something like this works:
package main
import (
"context"
"encoding/json"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/codedeploy"
)
type CodeDeployEvent struct {
DeploymentId string `json:"deploymentId"`
LifecycleEventHookExecutionId string `json:"lifecycleEventHookExecutionId"`
}
func HandleRequest(ctx context.Context, event CodeDeployEvent) (string, error) {
// add some tests here and change status flag as needed . . .
client := codedeploy.New(session.New())
params := &codedeploy.PutLifecycleEventHookExecutionStatusInput{
DeploymentId: &event.DeploymentId,
LifecycleEventHookExecutionId: &event.LifecycleEventHookExecutionId,
Status: "Succeeded",
}
req, _ := client.PutLifecycleEventHookExecutionStatusRequest(params)
_ = req.Send()
}
I got around to implement this and want to share my complete solution.
After figuring out how to use it, I decided against using it, because there are a couple of drawbacks.
there is no way to expose a new version of the canary to a dedicated portion of the user base, that means sometimes they'll hit the new or the old version
invoking functions that publish to sns will trigger all downstream actions, which might get the new or the old version of the downstream services, which would cause a lot of problems in case of breaking APIs
IAM changes affect both version immediately, possibly breaking the old version.
Instead, I deploy everything to a pre prod account, run my integration and e2e tests and if they succeed I'll deploy to prod
the cdk code to create a canary deployment:
const versionAlias = new lambda.Alias(this, 'Alias', {
aliasName: "alias",
version: this.lambda.currentVersion,
})
const preHook = new lambda.Function(this, 'LambdaPreHook', {
description: "pre hook",
code: lambda.Code.fromAsset('dist/upload/convert-pre-hook'),
handler: 'main',
runtime: lambda.Runtime.GO_1_X,
memorySize: 128,
timeout: cdk.Duration.minutes(1),
environment: {
FUNCTION_NAME: this.lambda.currentVersion.functionName,
},
reservedConcurrentExecutions: 5,
logRetention: RetentionDays.ONE_WEEK,
})
// this.lambda.grantInvoke(preHook) // this doesn't work, I need to grant invoke to all functions :s
preHook.addToRolePolicy(new iam.PolicyStatement({
actions: [
"lambda:InvokeFunction",
],
resources: ["*"],
effect: iam.Effect.ALLOW,
}))
const application = new codedeploy.LambdaApplication(this, 'CodeDeployApplication')
new codedeploy.LambdaDeploymentGroup(this, 'CanaryDeployment', {
application: application,
alias: versionAlias,
deploymentConfig: codedeploy.LambdaDeploymentConfig.ALL_AT_ONCE,
preHook: preHook,
autoRollback: {
failedDeployment: true,
stoppedDeployment: true,
deploymentInAlarm: false,
},
ignorePollAlarmsFailure: false,
// alarms:
// autoRollback: codedeploy.A
// postHook:
})
My go code of the pre hook function. PutLifecycleEventHookExecutionStatus tells code deploy if the pre hook succeeded or not. Unfortunately in case you fail the deployment message, the message you get in the cdk deploy output is utterly useless, so you need to check the pre/post hook logs.
In order to actually run the integration test I simply invoke the lambda and check if an error occurred.
package main
import (
"encoding/base64"
"fmt"
"log"
"os"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/codedeploy"
lambdaService "github.com/aws/aws-sdk-go/service/lambda"
)
var svc *codedeploy.CodeDeploy
var lambdaSvc *lambdaService.Lambda
type codeDeployEvent struct {
DeploymentId string `json:"deploymentId"`
LifecycleEventHookExecutionId string `json:"lifecycleEventHookExecutionId"`
}
func handler(e codeDeployEvent) error {
params := &codedeploy.PutLifecycleEventHookExecutionStatusInput{
DeploymentId: &e.DeploymentId,
LifecycleEventHookExecutionId: &e.LifecycleEventHookExecutionId,
}
err := handle()
if err != nil {
log.Println(err)
params.Status = aws.String(codedeploy.LifecycleEventStatusFailed)
} else {
params.Status = aws.String(codedeploy.LifecycleEventStatusSucceeded)
}
_, err = svc.PutLifecycleEventHookExecutionStatus(params)
if err != nil {
return fmt.Errorf("failed putting the lifecycle event hook execution status. the status was %s", *params.Status)
}
return nil
}
func handle() error {
functionName := os.Getenv("FUNCTION_NAME")
if functionName == "" {
return fmt.Errorf("FUNCTION_NAME not set")
}
log.Printf("function name: %s", functionName)
// invoke lambda via sdk
input := &lambdaService.InvokeInput{
FunctionName: &functionName,
Payload: nil,
LogType: aws.String(lambdaService.LogTypeTail), // returns the log in the response
InvocationType: aws.String(lambdaService.InvocationTypeRequestResponse), // synchronous - default
}
err := input.Validate()
if err != nil {
return fmt.Errorf("validating the input failed: %v", err)
}
resp, err := lambdaSvc.Invoke(input)
if err != nil {
return fmt.Errorf("failed to invoke lambda: %v", err)
}
decodeString, err := base64.StdEncoding.DecodeString(*resp.LogResult)
if err != nil {
return fmt.Errorf("failed to decode the log: %v", err)
}
log.Printf("log result: %s", decodeString)
if resp.FunctionError != nil {
return fmt.Errorf("lambda was invoked but returned error: %s", *resp.FunctionError)
}
return nil
}
func main() {
sess, err := session.NewSession()
if err != nil {
return
}
svc = codedeploy.New(sess)
lambdaSvc = lambdaService.New(sess)
lambda.Start(handler)
}
I just started reading about RabbitMQ and I'm trying to send large number of messages in a for loop. The problem is that it just doesn't work.
package main
import (
"fmt"
"github.com/streadway/amqp"
"strconv"
)
func main() {
var connectionString = "amqp://guest:guest#localhost:5672/"
conn, _ := amqp.Dial(connectionString)
defer conn.Close()
ch, _ := conn.Channel()
defer ch.Close()
q, _ := ch.QueueDeclare(
"user_actions", // name
true, // durable
false, // delete when unused
false, // exclusive
false, // no-wait
nil, // arguments
)
for i := 0; i < 10000; i++ {
body := "Hello from Go! " + strconv.Itoa(i)
ch.Publish(
"", // exchange
"hello", // routing key
false, // mandatory
false, // immediate
amqp.Publishing {
ContentType: "text/plain",
Body: []byte(body),
})
fmt.Println("Sent: "+body)
}
}
I even tried reducing the number of iterations and even tried sending messages outside of the loop but it just doesn't work. What am I doing wrong?
The provided code seems fine except that you are using the default exchange and providing a route name different than your queue name.
Chances are you'll want to use the queue name as the routing name. Try to replace hello with user_actions in ch.Publish function.
I want to send a Slack notification with an attached file. This is my current code:
package Message
import (
"fmt"
"os"
"github.com/ashwanthkumar/slack-go-webhook"
)
func Message(message string, cannalul string, attash bool) {
f, err := os.Open(filename)
if err != nil {
return false
}
defer f.Close()
_ = f
fullName := "myServer"
webhookUrl := "https://hooks.slack.com/services/......."
attachment1 := slack.Attachment {}
//attachment1.AddField(slack.Field { Title: "easySmtp", Value: "EasySmtp" }).AddField(slack.Field { Title: "Status", Value: "Completed" })
if attash {
attachment1.AddField(slack.Field { Title: "easySmtp", Value: fullName})
}
payload := slack.Payload {
Text: message,
Username: "worker",
Channel: cannalul,
IconEmoji: ":grin:",
Attachments: []slack.Attachment{attachment1},
}
err := slack.Send(webhookUrl, "", payload)
if len(err) > 0 {
fmt.Printf("error: %s\n", err)
}
}
My code works, but I don't know how I can add an attached file in my current code. How I can do this?
You can not attach a file to an attachment through a webhook in Slack. That functionality does not exist in Slack.
If its just text you can add the content as part of the message or another attachments (up to a limit of currently 500,000 characters, which will soon be reduced to 40,000 - see here for reference).
Or you can directly upload a file to a channel with the API method files.upload.