Deploy serverless functions to digitalocean in golang - go

I am trying to build a CRUD in digital ocean serverless functions. I am testing with sqlite3 and insert into the table. When I tried to deploy it to productions, I am getting errors. Here is my code:
package main
import (
"database/sql"
"fmt"
_ "github.com/mattn/go-sqlite3"
)
func Main(args map[string]interface{}) map[string]interface{} {
db, err := sql.Open("sqlite3", "./data.db")
checkErr(err)
// insert
stmt, err := db.Prepare("INSERT INTO userinfo(username, departname, created) values(?,?,?)")
checkErr(err)
res, err := stmt.Exec("rumi", "CSE", "2012-12-09")
checkErr(err)
id, err := res.LastInsertId()
checkErr(err)
fmt.Println(id)
db.Close()
msg := make(map[string]interface{})
msg["body"] = id
return msg
}
Errors I am getting:
➜ functions git:(master) ✗ doctl serverless deploy . --remote-build
Deploying '/home/rumi/go/src/github.com/manjurulhoque/digitalocean-cloud-functions/functions'
to namespace 'fn-b799454253a-a40440-4639-937f-05102a48c06e'
on host 'https://fa45as-sgp1-18b45c02afgc.doserverless.co'
Submitted action 'blog/createBlog' for remote building and deployment in runtime go:default (id: b3d5421ee5656bb44c4295421eebb44c642cf)
Submitted action 'sample/hello' for remote building and deployment in runtime go:default (id: b3d5421ee5656bb44c4295421eebb44c642cf)
Submitted action 'blog/db' for remote building and deployment in runtime go:default (id: edcc9eefce9f4aa58c9eefce9f2aa5e6)
Transcript of remote build session for action 'blog/db':
Output of failed build in /tmp/slices/builds/fn-b79956253a-a4080-465639-95637f-05102a48c06e/blog_db/2022-10-22T04-23-08.642Z/packages/blog/db
initializing modules
go: creating new go.mod: module exec
go: to add module requirements and sums:
go mod tidy
building
db.go:6:2: no required module provides package github.com/mattn/go-sqlite3; to add it:
go get github.com/mattn/go-sqlite3
Deployed functions ('doctl sbx fn get <funcName> --url' for URL):
- blog/createBlog
- sample/hello
Failures:
Error: While deploying action 'blog/db': './build.sh' exited with code 1
project.yml
environment: {}
parameters: {}
packages:
- name: blog
environment: {}
parameters: {}
annotations: {}
functions:
- name: db
binary: false
main: ''
runtime: 'go:default'
web: true
parameters: {}
environment: {}
annotations: {}
limits: {}
I didn't find any good resource though to connect to DB. Any help would be appreciated.

In DO, every function itself is an app. So you need to create go.mod and go.sum in each function directoy.
Below is one of my project structure
Note: I wrote an article on how to setup golang app for DO
https://medium.com/#manzurulhoque/use-package-in-digital-ocean-do-serverless-functions-using-golang-cb5200ab22ee

Related

Build error in golang package compilation

I'm designing an IBM DB2 warehouse database using Go ORM. I include the dependencies
import ( _ "github.com/ibmdb/go_ibm_db" "github.com/jinzhu/gorm" )
and create the database
func Init() \*gorm.DB {
db, err := gorm.Open("ibm_db", dbURL)
if err != nil {
log.Print("Error db init Database.init 19")
panic(err.Error())
}
migrate(db)
return db
}
but I receive this error:
Build Error: go build -o /home/t/skola/programmering/hackYourWorld/api/\__debug_bin -gcflags all=-N -l .
# github.com/ibmdb/go_ibm_db/api
/home/t/go/pkg/mod/github.com/ibmdb/go_ibm_db#v0.4.2/api/api_unix.go:13:11: fatal error: sqlcli1.h: No such file or directory 13 |
// #include \<sqlcli1.h\> | ^\~\~\~\~\~\~\~\~\~\~ compilation terminated. (exit status 2)
I'm running a ubuntu docker container with go1.18.1 and include the package according to the documentation: go get -d github.com/ibmdb/go_ibm_db\
I tried installing it but can't find any possible problem.

Golang using aws lambda error: InvokeInput not declared by package lambda

I'm trying to call another lambda function using the following code:
sess := session.Must(
session.NewSessionWithOptions(
session.Options{
SharedConfigState: session.SharedConfigEnable,
},
),
)
svc := lambda.New(sess, &aws.Config{Region: aws.String("ap-east-1")})
result, err := svc.Invoke(&lambda.InvokeInput{
FunctionName: aws.String(os.Getenv("testLambdaFunc")),
Payload: []byte(req.Body),
})
But there are two errors
New not declared by package lambda
and
InvokeInput not declared by package lambda
I've tried to initialize the go.mod file, but it doesn't fix both of the errors.
Any ideas?
The imports of my main.go file:
"fmt"
"os"
"pkg/log"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
And the go.mod file
module somefunc
go 1.16
require (
github.com/aws/aws-lambda-go v1.27.0
github.com/aws/aws-sdk-go v1.40.59
pkg/log v0.0.0-00010101000000-000000000000
)
replace pkg/log => ./../../go/common/pkg/log
Based on your code snipped, it looks as if you are using the AWS SDK for Go V2. It is recommended to use the AWS SDK for Go V2 (please review the Migrating to the AWS SDK for Go V2 documentation).
Please initialize a Go Modules project (as described on the SDK's Github page):
mkdir YOUR-PROJECT
cd YOUR-PROJECT
go mod init YOUR-PROJECT
Add the dependencies as follows:
go get github.com/aws/aws-sdk-go-v2/aws
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/service/lambda
The above should give you a working project where the dependencies for the SDK packages will resolve.
The corresponding V2 code will look something like the following:
cfg, err := config.LoadDefaultConfig(context.TODO(),
config.WithRegion("us-east-1"),
)
if err != nil {
[...]
}
svc := lambda.NewFromConfig(cfg)
result, err := svc.Invoke(context.TODO(),
&lambda.InvokeInput{
FunctionName: aws.String(os.Getenv("testLambdaFunc")),
Payload: []byte(req.Body),
},
)

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed Error ID: 6191efcd

I am getting the following error since this morning but nothing changed regarding my code (just added a log.println()... Even getting back to a previous version that was deployed doesn't solve the issue)
trigger.go (entry point)
// The function that runs with the cloud function itself
func HandleUserCreateEvent(ctx context.Context, e my_project.FirestoreEvent) error {
log.Println("-------------------- oldValue --------------------")
log.Printf("Name: %s\n", e.OldValue.Name)
log.Printf("CreateTime: %v\n", e.OldValue.CreateTime)
log.Printf("Fields: %v\n", e.OldValue.Fields)
log.Printf("UpdateTime: %v\n", e.OldValue.UpdateTime)
log.Println("-------------------- newValue --------------------")
log.Printf("Name: %s\n", e.Value.Name)
log.Printf("CreateTime: %v\n", e.Value.CreateTime)
log.Printf("Fields: %v\n", e.Value.Fields)
log.Printf("UpdateTime: %v\n", e.Value.UpdateTime)
log.Println("-------------------- jsonValue -------------------")
jsonB, _ := json.Marshal(e.Value)
log.Printf("Json: %v\n", string(jsonB))
log.Println("---------------------- DONE ----------------------")
// My code
return nil
deploy.sh (having also a deploy.bat which was working as well as the sh script below)
#!/usr/bin/env bash
# Deployment automation
name="HandleUserCreateEvent"
projectId="my_project"
collection="UsersCollection"
# Must call go vendor as go modules are ignored
go mod vendor
yes Y | gcloud functions deploy ${name} \
--trigger-event providers/cloud.firestore/eventTypes/document.create \
--trigger-resource "projects/${projectId}/databases/(default)/documents/${collection}/{pushId}" \
--region europe-west1 \
--runtime go113 \
--allow-unauthenticated \
--memory 128
As this was working perfectly, now it doesn't anymore (as stated, I just added the jsonValue part of the code you see above)
Error
MacBook-Pro-de-Emixam23:my-project-elasticsearch-creater emixam23$ ./deploy.sh
Deploying function (may take a while - up to 2 minutes)...
....................................................failed.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: # serverless_function_app/main
src/serverless_function_app/main/main.go:24:38: cannot use my_project_elasticsearch_creater.HandleUserCreateEvent (type func(context.Context, my_project_elasticsearch.FirestoreEvent) error) as type func(http.ResponseWriter, *http.Request) in argument to funcframework.RegisterHTTPFunction; Error ID: 6191efcd
Any idea? I really think that gcloud has some issues...
Update -> gcloud is really broken
I just redeployed an empty function, with only a simple struct as package with no dependencies (go modules empty)
type FirestoreEvent struct {
OldValue interface{} `json:"oldValue"`
Value interface{} `json:"value"`
UpdateMask struct {
FieldPaths []string `json:"fieldPaths"`
} `json:"updateMask"`
}
Still the same issue
C:\Workspace\go\src\gitlab-group\my-project-elasticsearch-creater>gcloud --version
Google Cloud SDK 298.0.0
app-engine-go
app-engine-python 1.9.91
bq 2.0.58
cloud-datastore-emulator 2.1.0
core 2020.06.19
gsutil 4.51
I tried the exact same at the end of the same day and it worked...
I believe it was a Google Cloud Platform issue related, no idea why we were not able to deploy it during a couple of hours.

Why does proto marshal then unmarshal fail when it is run outside of project containing vendor directory?

I have a main.go file that uses the proto files in pkg/models to Marshal and Unmarshal a proto struct like this:
// Convert to string
protoStr := proto.MarshalTextString(proto)
// Unmarshal string back to proto struct
var proto2 models.Stuff
err := proto.UnmarshalText(protoStr, &proto2)
The setup is here: https://github.com/chuyval/qqs/tree/master/q2
The project contains a vendor directory that only has the github.com/golang/protobuf repo checked out. (run glide install to create vendor if it doesnt exist)
The main.go program works fine when running go run main.go from inside the project.
When I move the main.go file one level up to the parent directory, and run the same command go run main.go at the parent level, it reports the following error:
line 2: unknown field name "value_list" in models.Stuff
When I delete the vendor directory in the project directory, and run go run main.go at the parent level, I get no error.
Why would having a vendor directory in the project repository make it error out?
Another thing to note, if I run the same main.go application inside of the dependent repo, it works every time (with or without the vendor repository).
Sample code:
package main
import (
"github.com/chuyval/qqs/q2/pkg/models"
"github.com/golang/protobuf/proto"
"log"
)
func main () {
stuff := createProtoStuff()
log.Printf("Stuff: %+v", stuff)
// Convert to string
stuffStr := proto.MarshalTextString(stuff)
// Unmarshal string back to proto struct
var stuff2 models.Stuff
err := proto.UnmarshalText(stuffStr, &stuff2)
if err != nil {
log.Printf("It didnt work. Error: %s", err.Error())
} else {
log.Printf("It worked. Proto: %+v", stuff2)
}
}
func createProtoStuff() *models.Stuff {
someValueList := []*models.SomeValue{&models.SomeValue{Id: "Some Value ID"}}
valueList := &models.SomeValueList{SomeValue: someValueList}
stuffValueList := &models.Stuff_ValueList{
ValueList: valueList,
}
stuff := &models.Stuff{
Id: "Stuff List Id",
Stuff: stuffValueList,
}
return stuff
}
Software Versions
glide version 0.13.1
go version go1.10.3 darwin/amd64
protoc version libprotoc 3.6.0

Golang on OpenShift, I can't find local module

I want to use an OpenShift test environment for my Golang applications.
I made a test application:
myproj/
------web.go
------/mylib/
-------------mylib.go
web.go is standard OpenShift file:
package main
import (
"fmt"
"net/http"
"os"
"runtime"
"./mylib"
)
func main() {
http.HandleFunc("/", hello)
bind := fmt.Sprintf("%s:%s", os.Getenv("HOST"), os.Getenv("PORT"))
fmt.Printf("listening on %s...", bind)
err := http.ListenAndServe(bind, nil)
if err != nil {
panic(err)
}
}
func hello(res http.ResponseWriter, req *http.Request) {
str := mylib.Lib();
fmt.Fprintf(res, "hello, %s from %s", str, runtime.Version())
}
and I created "mylib"
package mylib
func Lib() string {
return "world"
}
and when I run "go run web.go" everything works fine on my local computer. But when I try to upload this code to OpenShift I get the following error:
remote: -----> Using Go 1.1.2
remote: -----> Running: go get -tags openshift ./...
remote: can't load package: /var/lib/openshift/5354e6fd4382ec2dca000223/app-root/runtime/repo/.openshift/g/src/github.com/smarterclayton/goexample/web.go:8:2: local import "./mylib" in non-local package
remote: An error occurred executing 'gear postreceive' (exit code: 1)
remote: Error message: CLIENT_ERROR: Failed to execute: 'control build' for /var/lib/openshift/5354e6fd4382ec2dca000223/go
What does this mean? Why can't Golang find this package? I can't write all code in one file. How should I write the application for OpenShift?
I know this question is old but i had the same problem and it was difficult to find the solution, so i decided to ask in order to help who will run on the same problem.
The solution is very simple and can be found in readme of the go cartdrige repo on github: github.com/smarterclayton/openshift-go-cart
You have to create a file named .godir and put here the name of the main package of your server.
For example if you put myserver you can use:
package main
import "myserver/mylib"
func main() {
mylib.DoStuff()
}
Basically when you push on openshift the repo is copied in the directory placed in .godir before the build

Resources