I'm trying to deploy an API written in go to GCP, Cloud Run, but the gcloud run deploy command always fails. Google Log says: unable to open tcp connection with host '<ip address>:1433'
Here are the messages from Google Log in relation to the Google SQL connection and the final error:
Dockerfile
FROM golang:1.19
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . ./
RUN go build -v -o simpleapps-backend
EXPOSE 8080
CMD ["/app/simpleapps-backend"]
client.go
package db
import (
"backend/models"
"log"
"gorm.io/driver/sqlserver"
"gorm.io/gorm"
)
var Instance *gorm.DB
var err error
func Connect(connectionString string) {
Instance, err = gorm.Open(sqlserver.Open(connectionString), &gorm.Config{})
//connectionString = "Server=<ipAddress>,1433;Database=<name>;User Id=<userId>;Password=<password>"
if err != nil {
log.Fatal(err)
}
log.Println("Connected to Database!")
}
func Migrate() {
Instance.AutoMigrate(&models.User{})
log.Println("Database Migration Completed!")
}
The app locally runs successfully.
Related
Golang vesion: 1.18.5
Configuration lib:
"github.com/spf13/viper"
github repo
I'm writing a AWS Lambda using Go. It sits behind an AWS APIGateway as a REST API. It sends out POST requests to an external API. I want to config that external API's URL and a few other header params in a configuration file. Found that spf13/viper library can be used for that requirement.
Here's my source tree:
api-root
├──config
├ ├── application.yaml
└──lambda
├── main.go
Here's my main.go,
package main
import (
"fmt"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
"github.com/spf13/viper"
"my-api/pkg/handlers"
"os"
)
func main() {
lambda.Start(Handler)
}
func Handler(request events.APIGatewayProxyRequest) (*events.APIGatewayProxyResponse, error) {
// Config
v := viper.New()
v.SetConfigName("application") // config file name without extension
v.SetConfigType("yaml")
v.AddConfigPath(".")
v.AddConfigPath("../config/")
v.AutomaticEnv() // read value ENV variable
err := v.ReadInConfig()
if err != nil {
fmt.Println("fatal error config file: default \n", err)
os.Exit(1)
}
env := v.GetString("app.env")
fmt.Println("Environment : ", env)
return handlers.Join(request)
}
Here's my go build command,
GOARCH=amd64 GOOS=linux go build -mod=vendor -o ./bin/<lambda-function-name> -ldflags="-s -w" ./lambda/main.go
And when I run it locally using sam
sam local start-api -t deployment/template.yml
And then hit the local endpoint using postman, it gives me the following error,
fatal error config file: default
Config File "application" Not Found in "[/var/task /var/config]"
However, when I create a normal go app and run it I get the expected params successfully. Here's my normal go app main.go,
package main
import (
"fmt"
"github.com/spf13/viper"
"os"
)
func main() {
v := viper.New()
v.SetConfigName("application") // config file name without extension
v.SetConfigType("yaml")
v.AddConfigPath(".")
v.AddConfigPath("./config/")
v.AutomaticEnv() // read value ENV variable
err := v.ReadInConfig()
if err != nil {
fmt.Println("fatal error config file: default \n", err)
os.Exit(1)
}
en := v.GetString("app.env")
fmt.Println("Environment : ", en)
}
when I run the app I see the expected print in the console,
➜ go run main.go
Environment : dev
What should I do to get viper working in the lambda?
I had the same problem.
I solved it by creating a layer in Lambda (sam)
AWSTemplateFormatVersion: '2010-09-09'
...
Resources:
Function:
Type: AWS::Serverless::Function
Properties:
...
Layers:
- !Ref Files
Files:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: config-files-layer
ContentUri: <<YAML FILES DIR>>
CompatibleRuntimes:
- ...
And changing
v.AddConfigPath(".")
to
v.AddConfigPath("/opt")
More: Creating and sharing Lambda layers
I had trouble getting my React app to run on GitHub pages, so I chose to try and serve the files on my Go backend which is on Heroku. At first I was serving the React app through the main Go backend, which did serve the React app successfully but none of my other routes would work after that in the Go app, the routes needed for my React app to operate.
So I chose to create a new Heroku Go app and separate the backend and frontend on different Heroku Go apps. The frontend Go app is running fine, but the backend will intermittently work. I understand Heroku free apps go into a sleep state with a period of inactivity, but I am literally talking about the app being online for a few minutes, and then all of sudden just switching back to the default mode of saying "no such app"
Frontend Heroku Go app:
// Route: Delete Graph from database
func RouteDefault(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
http.ServeFile(w, r, "static/index.html")
}
func main() {
port := os.Getenv("PORT")
if port == "" {
port = "9000" // Default port if not specified
}
// HTTPRouter Settings and Routes
router := httprouter.New()
router.GET("/", RouteDefault)
// if not found look for a static file
static := httprouter.New()
static.ServeFiles("/*filepath", http.Dir("static"))
router.NotFound = static
handler := cors.AllowAll().Handler(router)
fmt.Println(http.ListenAndServe(":"+port, handler))
}
Backend Heroku Go app:
func main() {
// BasicAuth username and password
user := ""
pass := ""
port := os.Getenv("PORT")
if port == "" {
port = "9000" // Default port if not specified
}
DefaultUser()
// HTTPRouter Settings and Routes
router := httprouter.New()
router.POST("/login/", BasicAuth(RouteLogin, user, pass))
router.POST("/upload/", JWTAuth(RouteUpload))
router.POST("/graph/", JWTAuth(RouteGetGraph))
router.GET("/autologin/", JWTAuth(RouteAutoLogin))
handler := cors.AllowAll().Handler(router)
fmt.Println(http.ListenAndServe(":"+port, handler))
}
Frontend: https://grafulatordemo.herokuapp.com/
Backend: https://grafulator.herokuapp.com/
run the command: heroku apps -A and share the relevant output so it can be troubleshooted
The problem was on heroku's end, contacting their support team resolved it.
I want to get list of all the docker images that I have uploaded on a particular project.
I am using the official SDK https://pkg.go.dev/cloud.google.com/go
As per my understanding the listing should be under the container module. But, I am not able to find any method called ListImages OR ListRepositories that can server the purpose.
I checked out the artifactregistry module, but it seems that it is only useful in case I push my images to artifact registry.
What is the correct way to get listing of docker images (project wise), in golang ?
I don't think we have a client library for the containerregistry. Since it's just an implementation of the Docker Registry API according to this thread at least
Have you tried with the Registry package ? https://pkg.go.dev/github.com/docker/docker/registry
For the repository you can use hostname/project-id. Where hostname is gcr.io or eu.gcr.io or us.gcr.io depending on how your repositories in GCR are configured.
golang approach (tested after running: gcloud auth configure-docker):
package main
import (
"context"
"fmt"
"log"
"github.com/google/go-containerregistry/pkg/authn"
gcr "github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/google"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
auth, err := google.NewGcloudAuthenticator()
if err != nil {
log.Fatal(err)
}
fmt.Println(auth)
registry, err := gcr.NewRegistry("gcr.io")
if err != nil {
log.Fatal(err)
}
ctx := context.Background()
repos, err := remote.Catalog(ctx, registry, remote.WithAuthFromKeychain(authn.DefaultKeychain))
if err != nil {
log.Fatal(err)
}
for _, repo := range repos {
fmt.Println(repo)
}
}
response:
&{0xc0000000}
project-id/imagename
REST API Approach:
you can list the images like this (replace the gcr.io with your gcr endpoint such as gcr.io, us.gcr.io, asia.gcr.io etc.):
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" "https://gcr.io/v2/_catalog"
response:
{"repositories":["project-id/imagename"]}
you can find some related details regarding the token fetching from this link:
How to list images and tags from the gcr.io Docker Registry using the HTTP API?
PS: you will have to filter out the project specific images from response if you have access to multiple projects
I am getting error [ERROR] when trying to pull image through test container. Reason CI machines in your organisation have access to a common registry server and is not allowed to talk to external web. Testcontainer java has something like for this use case
Private registry image name
// Referring directly to an image on a private registry - image name will vary
final MySQLContainer<?> mysql = new MySQLContainer<>(
DockerImageName.parse("registry.mycompany.com/mirror/mysql:8.0.24")
.asCompatibleSubstituteFor("mysql")
)
what's the go equivalent to override image registry urls that testcontainer-go uses?
Code
req := testcontainers.ContainerRequest{
Image: "amazon/dynamodb-local:1.15.0",
ExposedPorts: []string{"8000" + "/tcp"},
ReaperImage: artifact_path,
}
container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
// auto-start the container
Started: true,
})
[ERROR]
2021/09/28 20:21:11 Failed to pull image: Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers), will retry
Have you tried to put the fully qualified image name in the container request and passing your registry credentials?
func TestFoo(t *testing.T) {
authConfig := types.AuthConfig{
Username: os.Getenv("DOCKER_USER"),
Password: os.Getenv("DOCKER_PWD"),
}
json, err := json.Marshal(authConfig)
assert.Nil(t, err)
req := testcontainers.ContainerRequest{
Image: "docker.my-company.org/my-namespace/dynamodb-local:1.15.0",
ExposedPorts: []string{"8000" + "/tcp"},
RegistryCred: base64.URLEncoding.EncodeToString(json),
}
container, err := testcontainers.GenericContainer(context.Background(), testcontainers.GenericContainerRequest{
ContainerRequest: req,
// auto-start the container
Started: true,
})
assert.Nil(t, err)
assert.NotNil(t, container)
}
As far as I see in the code, there is no replacer function as in the Java version, but I do not think it makes sense, as in Java they have MySQL containers, as opposite what the Go version does, which does not have specialised containers. Therefore, in the Java version it makes sense to replace the default image for the mysql container.
I'm using a Go API and I need to connect it to my Cassandra DB on a Google Cloud server.
This is my Go connection code:
func SetupDBConnection() {
cluster := gocql.NewCluster("XX.XXX.XXX.XXX")
cluster.Keyspace = "afterstr"
cluster.Consistency = gocql.Quorum
s, err := cluster.CreateSession()
if err != nil {
fmt.Println(err)
}
}
But, when I run this code, I have this error:
gocql: unable to create session: control: unable to connect to initial hosts: dial tcp xx.xxx.xxx.xxx:9042: i/o timeout
This is my Google Cloud configuration:
I also modified the cassandra.yaml file in my server to change the listen address and the RPC address to put my IP with the port I use to run my Go program that I opened.
(It is opened 3 times because I was just testing something)
Try:
cluster.Authenticator = gocql.PasswordAuthenticator{Username: username, Password: password}