I have been trying to upload to cloudinary via golang sdk. but sometimes it seems to fail after several requests (sometimes it works, other times it doesn't).
I run the app in localhost (windows) and without vpn.
error message:
"https://api.cloudinary.com/v1_1/dd6mdlpgj/auto/upload": dial tcp: lookup api.cloudinary.com: no such host
type cloudinaryStorageRepository struct {
cld *cloudinary.Cloudinary
}
func NewCloudinaryStorageRepository(cloudName string, apiKey string, apiSecret string) domain.StorageRepository {
cld, err := cloudinary.NewFromParams(cloudName, apiKey, apiSecret)
if err != nil {
panic(err)
}
return &cloudinaryStorageRepository{cld}
}
func (c *cloudinaryStorageRepository) Upload(ctx context.Context, storage *domain.Storage) (string, error) {
file, err := storage.File.Open()
defer func(file multipart.File) {
err := file.Close()
if err != nil {
logrus.Error(err)
}
}(file)
if err != nil {
logrus.Error(err)
return "", err
}
uploaded, err := c.cld.Upload.Upload(ctx, file, uploader.UploadParams{
Folder: storage.Folder,
PublicID: storage.Name,
})
if err != nil {
logrus.Error(err)
return "", err
}
return uploaded.SecureURL, nil
}
If I understand you correctly, you can try to reset your DNS on your computer or use another computer to run the code because it might be a misconfiguration of your local host.
Related
I am working an application written in golang,for which one of the capability will be to SSH into a device and execute a shell script there. With my current implementation,a normal script is getting executed.The problem lies in the execution of a script that requires user input: for e.g the script asks the user, can we proceed with the installation: and the user has to type in Y or N. This is the part where it is failing. Here is my implementation:
func main() {
cmdOutput,err := ExecuteScriptSSH()
if err != nil{
fmt.Printf("Error is %s",err)
}
fmt.Printf("Command Output is %s", cmdOutput)
}
func ExecuteScriptSSH() (string, error) {
script := "#!/bin/sh\n\n ls -l \n\n date"
params := make(map[string]interface{})
params["Username"] = "uname"
params["Password"] = "pwd"
params["IPAddress"] = "ip"
params["Port"] = "22"
connection := NewConnection(params)
client, err := connection.ConnectNonSecure()
if err != nil {
fmt.Printf("unable to Connect:%v ", err)
}
defer client.Close()
ss, err := client.NewSession()
if err != nil {
fmt.Printf("unable to create SSH session:%v ", err)
}
//Converting the Script to string to a shell script file by writing to it.
d1 := []byte(script)
err = os.WriteFile("/tmp/script.sh", d1, 0777)
if err != nil {
fmt.Printf("Error constructing Script file %v", err)
}
// opening the script file
scriptFile, err2 := os.OpenFile("/tmp/script.sh", os.O_RDWR|os.O_CREATE, 0777)
if err2 != nil {
fmt.Printf("Error opening Script file is %s", err)
}
cmdOutput := &bytes.Buffer{}
ss.Stdout = cmdOutput
ss.Stdin = scriptFile
interpreter := "sh"
err = ss.Run(interpreter)
if err != nil {
fmt.Printf("Error in Executing Script file %v", err.Error())
}
return cmdOutput.String(), err
}
type Params struct {
Username string
Password string
IPAddress string
Port string
}
func NewConnection(data map[string]interface{}) *Params {
var params Params
fmt.Printf("Data is %v",data)
err := mapstructure.Decode(data,¶ms)
if err != nil {
panic(err)
}
fmt.Printf("Mapped Connection details %v",params)
return ¶ms
}
func (p *Params) ConnectNonSecure() (*ssh.Client, error){
config := &ssh.ClientConfig{
User: p.Username,
Auth: []ssh.AuthMethod{
ssh.Password(p.Password),
},
// Non-Production-only
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}
fmt.Printf("Connection details %v",p)
// Connect to the remote server and perform a handshake
client, err := ssh.Dial("tcp", p.IPAddress+":"+p.Port, config)
if err != nil {
log.Fatalf("unable to connect to: %v %s", err,p.IPAddress)
}
return client, err
}
for simplicity sake I have added a very simple script,but actually the script is huge and requires user confirmation.Any idea on how to solve this issue?
I am facing an issue where in, whenever I try to upload a file to a remote sftp server, I get an error saying "failed to send packet header: EOF". This occurs when I try to perform the uploading step from my own hosted EC2 instance. While locally, everything works fine.
Sftp client is initiated as follow.
// Connect to server
var authMethods []ssh.AuthMethod
// Use password authentication if password provided
if pass != "" {
authMethods = append(authMethods, ssh.Password(pass))
}
config := ssh.ClientConfig{
User: user,
Auth: authMethods,
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}
conn, err := ssh.Dial("tcp", addr, &config)
if err != nil {
return nil, tearDown, errors.Wrap(err, fmt.Sprintf("failed to connect to %s", addr))
}
tearDown = func() {
_ = conn.Close()
}
// Create new SFTP client
sc, err := sftp.NewClient(conn)
if err != nil {
return nil, tearDown, errors.Wrap(err, "Unable to start SFTP subsystem")
}
tearDown = func() {
fmt.Println("defer is called. closing connection now .... ")
_ = conn.Close()
_ = sc.Close()
}
return sc, tearDown, nil
And instance of sc is attached to a struct and passed around the codebase
Function invoked while uploading file is as follow.
file, err := s.sc.OpenFile(remoteFilePath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC)
defer func() {
if file == nil {
return
}
cErr := file.Close()
if cErr != nil {
fmt.Println(fmt.Sprintf("error while closing file %v", cErr))
}
}()
if err != nil {
fmt.Println(fmt.Sprintf("error while opening file %v", err))
return err
}
_, err = file.Write(data)
if err != nil {
fmt.Println(fmt.Sprintf("error while writing to file %v", err))
return err
}
return nil
Can someone guide me as in where is the error coming from?
I've accomplished implementing TCP reverse proxy in GoLang. But unfortunately couldn't come up with implementing caching to a TCP reverse proxy. Is it possible to do so, if yes, is there any resource out there? Is caching possible on a TCP (Transport Layer of Network)?
Here's the simple TCP reverse proxy in Golang.
package main
import (
"io"
"log"
"net"
)
//Proxy struct
type Proxy struct {
laddr, raddr *net.TCPAddr
lconn, rconn io.ReadWriteCloser
errorSignal chan bool
}
// New Create a new Proxy instance.
func New(lconn *net.TCPConn, laddr, raddr *net.TCPAddr) *Proxy {
return &Proxy{
lconn: lconn,
laddr: laddr,
raddr: raddr,
errorSignal: make(chan bool),
}
}
//TCPAddressResolver resolves an address and returns to a struct having ip and port.
func TCPAddressResolver(addr string) (tcpAddress *net.TCPAddr, err error) {
tcpAddress, err = net.ResolveTCPAddr("tcp", addr)
return
}
func main() {
listenerAddress, err := TCPAddressResolver(":8080")
if err != nil {
log.Fatalf("Failed to resolve local address: %v", err)
}
remoteAddress, err := TCPAddressResolver(":3000")
if err != nil {
log.Fatalf("Failed to resolve remote address: %v", err)
}
listener, err := net.ListenTCP("tcp", listenerAddress)
if err != nil {
log.Fatalf("Failed to open local port to listen: %v", err)
}
log.Printf("Simple Proxy started on: %d and forwards to port %d", listenerAddress.Port, remoteAddress.Port)
for {
conn, err := listener.AcceptTCP()
if err != nil {
log.Fatalf("Failed to accept connection: %v", err)
continue
}
var p *Proxy
// HTTP is a stateless protocol thus a proxy needs to reinitiate the new next incoming call (conn)
// each time it finishes handling the previous one.
p = New(conn, listenerAddress, remoteAddress)
p.Start()
}
}
//Start initiates transmission of data to and from the remote to client side.
func (p *Proxy) Start() {
defer p.lconn.Close()
var err error
p.rconn, err = net.DialTCP("tcp", nil, p.raddr)
if err != nil {
log.Fatalf("Remote connection failure: %v", err)
}
defer p.rconn.Close()
go p.CopySrcDst(p.lconn, p.rconn)
go p.CopySrcDst(p.rconn, p.lconn)
//Wait for everything to close -- This one blocks the routine.
<-p.errorSignal
log.Printf("Closing Start routine \n")
}
func (p *Proxy) err(err error) {
if err != io.EOF {
log.Printf("Warning: %v: Setting error signal to true", err)
}
p.errorSignal <- true
}
//CopySrcDst copies data from src to dest
func (p *Proxy) CopySrcDst(src, dst io.ReadWriteCloser) {
buff := make([]byte, 1024)
for {
n, err := src.Read(buff)
if err != nil {
// Reading error.
p.err(err)
return
}
dataFromBuffer := buff[:n]
n, err = dst.Write(dataFromBuffer)
if err != nil {
// Writing error.
p.err(err)
return
}
}
}
You are asking how to save data read from an io.Reader. That's different from caching.
The easiest approach is to tee the reader into a buffer.
While you are at it, you might as well use io.Copy instead of the similar code in the question. The code in the question does not handle the case when read returns n > 0 and a non-nil error.
Use an error group to coordinate waiting for the goroutines and collecting error status.
var g errgroup.Group
var rbuf, lbuf bytes.Buffer
g.Go(func() error {
_, err := io.Copy(lconn, io.TeeReader(p.rconn, &rbuf))
return err
})
g.Go(func() error {
_, err := io.Copy(rconn, io.TeeReader(p.lconn, &lbuf))
return err
})
if err := g.Wait(); err != nil {
// handle error
}
// rbuf and lbuf have the contents of the two streams.
The name of the programming language is "Go", not "Golang" or "GoLang".
Using docker sources how to pull image from custom registry? As a result of using such code
// Prepare auth registry for usage
func (app *App) PrepareRegistry() error {
app.AuthConfig = types.AuthConfig{
Username: Username,
Password: Password,
ServerAddress: DefaultServer,
}
resp, err := app.Client.RegistryLogin(context.Background(), app.AuthConfig)
if err != nil {
panic(err)
}
fmt.Println(resp.Status)
if resp.IdentityToken != "" {
app.AuthConfig.IdentityToken = resp.IdentityToken
}
app.AuthConfigEncoded, err = command.EncodeAuthToBase64(app.AuthConfig)
return err
}
func (app *App) ImagePull() error {
opts := types.ImagePullOptions{
All: true,
RegistryAuth: app.AuthConfigEncoded,
PrivilegeFunc: registryAuthentication(app.Name),
}
responseBody, err := app.Client.ImagePull(context.Background(), app.Name, opts)
defer responseBody.Close()
if err != nil {
return err
}
return nil
}
I am still getting the error
Login Succeeded
panic: Error response from daemon: Get https://registry-1.docker.io/v2/shalakhin/blender/tags/list: unauthorized: incorrect username or password
While ServerAddress is registry.gitlab.com, not registry-1.docker.io
Did you check identity token? That might cause an authentication problem.
A suggestion:
Docker client
This works fine, as I can see you did not specify the endpoint. I think you should add this info.
authConfig := types.AuthConfig{
Username: "username",
Password: "password",
}
encodedJSON, err := json.Marshal(authConfig)
if err != nil {
panic(err)
}
authStr := base64.URLEncoding.EncodeToString(encodedJSON)
out, err := cli.ImagePull(ctx, "alpine", types.ImagePullOptions{RegistryAuth: authStr})
if err != nil {
panic(err)
}
I am beginner at Go, I had wrote small server to testing and deploy it on heroku platform. I have /logout request, which almost works, but sometimes I see something like this:
PANIC: write tcp 172.17.110.94:36641->10.11.189.195:9951: use of closed network connection
I don't know why it happens, and why sometimes it works perfectly.
My steps:
I send 1st POST request to /token-auth with body then generate token and send as response.
At 2nd I do /logout GET request with that token, and set token to Redis store
Here is full code of my redil_cli.go
package store
import (
"github.com/garyburd/redigo/redis"
)
type RedisCli struct {
conn redis.Conn
}
var instanceRedisCli *RedisCli = nil
func Connect() (conn *RedisCli) {
if instanceRedisCli == nil {
instanceRedisCli = new(RedisCli)
var err error
//this is works!!!
instanceRedisCli.conn, err = redis.Dial("tcp", "lab.redistogo.com:9951")
if err != nil {
panic(err)
}
if _, err := instanceRedisCli.conn.Do("AUTH", "password"); err != nil {
//instanceRedisCli.conn.Close()
panic(err)
}
}
return instanceRedisCli
}
func (redisCli *RedisCli) SetValue(key, value string, expiration ...interface{}) error {
_, err := redisCli.conn.Do("SET", key, value)
if err == nil && expiration != nil {
redisCli.conn.Do("EXPIRE", key, expiration[0])
}
return err
}
func (redisCli *RedisCli) GetValue(key string) (interface{}, error) {
data, err := redisCli.conn.Do("GET", key)
if err != nil{
panic(err)
}
return data, err
}
After that my function that checks Authorization header will panic while trying to do GetValue(key string) method
func (redisCli *RedisCli) GetValue(key string) (interface{}, error) {
data, err := redisCli.conn.Do("GET", key)
if err != nil{
panic(err)
}
return data, err
}
Can anyone point me, what I doing wrong?