Paths in Heroku deployment - heroku

I am trying to deploy a test golang app to Heroku:
My procfile looks like this:
web: todo
But when running my app I get the following error:
2017-04-10T15:24:07.128780+00:00 app[web.1]: panic: could not locate box "./static"
My main.go file contains:
package main
import (
"net/http"
"github.com/kitensei/go-todoist/server"
"github.com/GeertJohan/go.rice"
"os"
"log"
"fmt"
"path"
"strconv"
)
var boxPrefix = getenv("BOXPATH", "")
func main() {
dir, err := os.Getwd()
if err != nil {
log.Fatal(err)
}
fmt.Println("DIRECTORY CWD: " + dir)
ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := path.Dir(ex)
fmt.Println("EXECUTABLE PATH: " + exPath)
exists, err := exists(boxPrefix + "static")
if err != nil {
panic(err)
}
fmt.Println("CHECK IF (" +boxPrefix + "static) EXISTS: " + strconv.FormatBool(exists))
server.RegisterHandlers()
http.Handle("/", http.FileServer(rice.MustFindBox(boxPrefix + "static").HTTPBox()))
http.ListenAndServe(":8080", nil)
}
func getenv(key, fallback string) string {
value := os.Getenv(key)
if len(value) == 0 {
return fallback
}
return value
}
func exists(path string) (bool, error) {
_, err := os.Stat(path)
if err == nil {
return true, nil
}
if os.IsNotExist(err) {
return false, nil
}
return true, err
}
Can someone give me an hint ?
EDIT: Here the output when I try to find the CWD and set the box path accordingly
2017-04-10T17:23:57.000000+00:00 app[api]: Build succeeded
2017-04-10T17:24:06.341977+00:00 heroku[web.1]: Starting process with command `go-todoist`
2017-04-10T17:24:08.592486+00:00 app[web.1]: DIRECTORY CWD: /app
2017-04-10T17:24:08.595689+00:00 app[web.1]: panic: could not locate box "static"
2017-04-10T17:24:08.595691+00:00 app[web.1]:
2017-04-10T17:24:08.595692+00:00 app[web.1]: goroutine 1 [running]:
2017-04-10T17:24:08.595718+00:00 app[web.1]: github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go%2erice.MustFindBox(0x71e08d, 0x6, 0x0)
2017-04-10T17:24:08.595721+00:00 app[web.1]: /tmp/tmp.Pa1igbANNl/.go/src/github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go.rice/box.go:110 +0x94
2017-04-10T17:24:08.595724+00:00 app[web.1]: main.main()
2017-04-10T17:24:08.595741+00:00 app[web.1]: /tmp/tmp.Pa1igbANNl/.go/src/github.com/kitensei/go-todoist/main.go:21 +0x1a8
2017-04-10T17:24:08.660495+00:00 heroku[web.1]: Process exited with status 2
2017-04-10T17:24:08.685016+00:00 heroku[web.1]: State changed from starting to crashed
2017-04-10T17:24:08.686277+00:00 heroku[web.1]: State changed from crashed to starting
2017-04-10T17:24:09.023202+00:00 heroku[web.1]: Starting process with command `go-todoist`
2017-04-10T17:24:10.743837+00:00 app[web.1]: DIRECTORY CWD: /app
2017-04-10T17:24:10.746355+00:00 app[web.1]: panic: could not locate box "static"
2017-04-10T17:24:10.746357+00:00 app[web.1]:
2017-04-10T17:24:10.746360+00:00 app[web.1]: goroutine 1 [running]:
2017-04-10T17:24:10.746361+00:00 app[web.1]: github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go%2erice.MustFindBox(0x71e08d, 0x6, 0x0)
2017-04-10T17:24:10.746361+00:00 app[web.1]: /tmp/tmp.Pa1igbANNl/.go/src/github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go.rice/box.go:110 +0x94
2017-04-10T17:24:10.746363+00:00 app[web.1]: main.main()
2017-04-10T17:24:10.746367+00:00 app[web.1]: /tmp/tmp.Pa1igbANNl/.go/src/github.com/kitensei/go-todoist/main.go:21 +0x1a8
2017-04-10T17:24:10.791994+00:00 heroku[web.1]: Process exited with status 2
2017-04-10T17:24:10.817345+00:00 heroku[web.1]: State changed from starting to crashed
vagrant#precise64:/code/src/github.com/kitensei/go-todoist$ heroku run bash -a go-todoist-gpr
/usr/local/heroku/lib/heroku/jsplugin.rb:119: warning: Insecure world writable dir /code/bin in PATH, mode 040777
Running bash on ⬢ go-todoist-gpr... up, run.9334 (Free)
~ $ cd /app
~ $ pwd
/app
~ $ ls -lA
total 48
-rw------- 1 u52460 dyno 289 Apr 10 17:23 .gitignore
drwx------ 3 u52460 dyno 4096 Apr 10 17:24 .heroku
drwx------ 2 u52460 dyno 4096 Apr 10 17:24 .profile.d
-rw------- 1 u52460 dyno 15 Apr 10 17:24 Procfile
-rw------- 1 u52460 dyno 26 Apr 10 17:23 README.md
-rw------- 1 u52460 dyno 326 Apr 10 17:23 app.json
drwx------ 2 u52460 dyno 4096 Apr 10 17:24 bin
-rw------- 1 u52460 dyno 592 Apr 10 17:23 main.go
drwx------ 2 u52460 dyno 4096 Apr 10 17:23 server
drwx------ 3 u52460 dyno 4096 Apr 10 17:23 static
drwx------ 2 u52460 dyno 4096 Apr 10 17:23 task
drwx------ 3 u52460 dyno 4096 Apr 10 17:23 vendor
~ $
Trying to run the app on the Heroku bash:
~ $ go-todoist
DIRECTORY CWD: /app
EXECUTABLE PATH: /app/bin
CHECK IF (static) EXISTS: true
panic: could not locate box "static"
goroutine 1 [running]:
github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go%2erice.MustFindBox(0xc4200f0cf0, 0xb, 0x5)
/tmp/tmp.FmcU3dgw8Y/.go/src/github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go.rice/box.go:110 +0x94
main.main()
/tmp/tmp.FmcU3dgw8Y/.go/src/github.com/kitensei/go-todoist/main.go:34 +0x42c
The same with an absolute path:
~ $ go-todoist
DIRECTORY CWD: /app
EXECUTABLE PATH: /app/bin
CHECK IF (/app/static) EXISTS: true
panic: given name/path is absolute
goroutine 1 [running]:
github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go%2erice.MustFindBox(0xc4200f0cf0, 0xb, 0x5)
/tmp/tmp.FmcU3dgw8Y/.go/src/github.com/kitensei/go-todoist/vendor/github.com/GeertJohan/go.rice/box.go:110 +0x94
main.main()
/tmp/tmp.FmcU3dgw8Y/.go/src/github.com/kitensei/go-todoist/main.go:34 +0x42c

Related

Is there a equvilant of Python's `os.system()` in Golang? [duplicate]

This question already has answers here:
Executing a Bash Script from Golang
(4 answers)
Closed 3 months ago.
I want to create a wrapper program that can wrapper whatever shell commands user provides, like:
./wrapper "cmd1 && cmd2"
In Python, I can call os.system("cmd1 && cmd2"). But Golang's exec.Command needs a list for command and args. Is there way in Golang to archive the same as Python's os.system()?
os/exec https://pkg.go.dev/os/exec
package main
import (
"fmt"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("/usr/bin/bash", "-c", os.Args[1])
output, err := cmd.CombinedOutput()
if err != nil {
panic(err)
}
fmt.Println(string(output))
}
$ go run main.go "ls -alh && pwd"
total 4.0K
drwxr-xr-x 2 xxxx xxxx 120 Nov 14 11:12 .
drwxrwxrwt 17 root root 420 Nov 14 11:42 ..
-rw-r--r-- 1 xxxx xxxx 217 Nov 14 11:42 main.go
/tmp/stk

fork/exec /proc/self/exe: operation not permitted

I am following : Containers From Scratch • Liz Rice • GOTO 2018
and test the source code lizrice/containers-from-scratch locally to learn containers.
But with code below , I am not able to fork a child process on Ubuntu 1804, below are my main.go
package main
import (
"fmt"
"os"
"os/exec"
"syscall"
)
// go run main.go run <cmd> <args>
func main() {
switch os.Args[1] {
case "run":
run()
case "child":
child()
default:
panic("help")
}
}
func run() {
fmt.Printf("In <Run> Running %v \n", os.Args[2:])
cmd := exec.Command("/proc/self/exe", append([]string{"child"}, os.Args[2:]...)...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.SysProcAttr = &syscall.SysProcAttr{
Cloneflags: syscall.CLONE_NEWUTS,
}
must(cmd.Run())
}
func child() {
fmt.Printf("In <child> Running %v \n", os.Args[2:])
cmd := exec.Command(os.Args[2], os.Args[3:]...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
syscall.Sethostname([]byte("container"))
must(cmd.Run())
}
func must(err error) {
if err != nil {
panic(err)
}
}
I tried to run it :
go run main1.go run /bin/bash
Error below occurred:
In <Run> Running [/bin/bash]
panic: fork/exec /proc/self/exe: operation not permitted
goroutine 1 [running]:
main.must(...)
/home/jia/cs_study_plan/docker/containers-from-scratch/main1.go:72
main.run()
/home/jia/cs_study_plan/docker/containers-from-scratch/main1.go:44 +0x27f
main.main()
/home/jia/cs_study_plan/docker/containers-from-scratch/main1.go:17 +0x4d
exit status 2
Any suggestion how can I fix this permission issue in my main.go ?
Thank you #Peter for suggestions.
I could run my app use 'go build' and 'sudo'
Here is what I do
1. go to directory of main1.go
2. go mod init main1 <it must be the same name as main1.go>
3. go mod tidy <it is optional step>
4. gp build
5. main1 is build in current directory
6. run command :
sudo ./main1 run /bin/bash
7. child forked successfully , as I can tell from my output
In <Run> Running [/bin/bash]
In <child> Running [/bin/bash]
ps
PID TTY TIME CMD
15774 pts/0 00:00:00 sudo
15775 pts/0 00:00:00 main1
15781 pts/0 00:00:00 exe
15786 pts/0 00:00:00 bash
16047 pts/0 00:00:00 ps
Also I tried to disable 'AppArmor' in my Ubunut , see if that is what blocks main1.go from fork child process
$sudo systemctl status apparmor
[sudo] password for jia:
● apparmor.service - AppArmor initialization
Loaded: loaded (/lib/systemd/system/apparmor.service; enabled; vendor preset: enabled)
Active: active (exited) since Thu 2022-06-09 08:31:21 CST; 2h 3min ago
Docs: man:apparmor(7)
http://wiki.apparmor.net/
Main PID: 660 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4915)
CGroup: /system.slice/apparmor.service
Jun 09 08:31:21 ub1804 apparmor[660]: * Starting AppArmor profiles
Jun 09 08:31:21 ub1804 apparmor[660]: Skipping profile in /etc/apparmor.d/disable: usr.bin.firefox
Jun 09 08:31:21 ub1804 apparmor[660]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
Jun 09 08:31:21 ub1804 systemd[1]: Starting AppArmor initialization...
Jun 09 08:31:21 ub1804 apparmor[660]: ...done.
Jun 09 08:31:21 ub1804 systemd[1]: Started AppArmor initialization.
$sudo systemctl stop apparmor
$go run main1.go run echo hello
In <Run> Running [echo hello]
panic: fork/exec /proc/self/exe: operation not permitted
NO luck , if anyone knows what stops my main1.go to fork child , please let me know , thanks in advance
Your problem appears because Ubuntu has sudo preinstalled so your user has restricted rights. According to proc documentation, it provides an interface for kernel data structures and represents a pseudo-filesystem. The /proc is mounted by the system itself as read-only, but you are trying to create a new process inside this file system which is read-only for your current user.
You can read more about it in the man https://man7.org/linux/man-pages/man5/proc.5.html
TL;DR: you need to run your program as a root.

how can I update a file in tarball?

How can I update a file in tarball with Go?
I can only find some way to append file into a tar file like this:
file, err := os.OpenFile(path, os.O_WRONLY, 0644)
if _, err = file.Seek(-1024, os.SEEK_END); err != nil {
log.Fatalln(err)
}
if err != nil { return err }
defer file.Close()
tw := tar.NewWriter(file)
hdr := &tar.Header{
Name: "manifest.json",
Mode: 0644,
Size: int64(len(content)),
}
if err := tw.WriteHeader(hdr); err != nil {
log.Fatal(err)
}
if _, err := tw.Write(content); err != nil {
log.Fatal(err)
}
if err := tw.Close(); err != nil {
log.Fatal(err)
}
After finding solutions by google ,it seemed I have some commands like tar uvf ... or try to extract the tarball and make a new one by golang or command tar.
As tar document says , command tar uf can update a file in tar ball directly, but after using tar uvf , I found a very interesting thing like below:
a.tar is a docker image made by docker image save
I want to change manifest.json file in thie docker image tarball by tar uvf a.tar manifest.json
before update, the content of tarball seems OK.
[root#master1 a]# ls
a.tar
[root#master1 a]# tar xf a.tar
tar: manifest.json: implausibly old time stamp 1970-01-01 08:00:00
tar: repositories: implausibly old time stamp 1970-01-01 08:00:00
[root#master1 a]# ls
2080c332033e330242f9ea2043e18c7ea98de2ee1b1b3921b1a33bcb84ac4cd5
2acf0fc34f8cc199bde6f74a7d85382461ff69cf4618e5964e7e1d19d0fe1d5c
3962220e6895f5a8c88034ae1bc7926b9bb40ecdc441196cd01c7cf36e36b7db.json
a7d478578679c443a1d4e695154dbbe8cad5525ac46fa80e925c2a080d0b620f
a.tar
c2103589e99f907333422ae78702360ad258a8f0366c20e341c9e0c53743e78a.json
da70bf0a87b384edc6cfc5cff3a33ba087de7b81f914df73181bfc2869314ebb
manifest.json
repositories
then I try to update manifest.json by tar uvf a.tar manifest.json
[root#master1 b]# ls
a.tar
[root#master1 b]# tar xf a.tar
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
[root#master1 b]# ls -al
total 65972
drwxr-xr-x 6 root root 4096 Apr 26 15:28 .
drwxr-xr-x 4 root root 4096 Apr 26 15:28 ..
drwxr-xr-x 2 root root 4096 Sep 10 2019 2080c332033e330242f9ea2043e18c7ea98de2ee1b1b3921b1a33bcb84ac4cd5
drwxr-xr-x 2 root root 4096 Dec 31 2019 2acf0fc34f8cc199bde6f74a7d85382461ff69cf4618e5964e7e1d19d0fe1d5c
-rw-r--r-- 1 root root 2429 Dec 31 2019 3962220e6895f5a8c88034ae1bc7926b9bb40ecdc441196cd01c7cf36e36b7db.json
drwxr-xr-x 2 root root 4096 Dec 31 2019 a7d478578679c443a1d4e695154dbbe8cad5525ac46fa80e925c2a080d0b620f
-rw-r--r-- 1 root root 67517440 Apr 26 15:28 a.tar
-rw-r--r-- 1 root root 2467 Sep 10 2019 c2103589e99f907333422ae78702360ad258a8f0366c20e341c9e0c53743e78a.json
drwxr-xr-x 2 root root 4096 Sep 10 2019 da70bf0a87b384edc6cfc5cff3a33ba087de7b81f914df73181bfc2869314ebb
As you can see, manifest.json and repositories disappeared . And sometimes , there will be a var directory. This is very strange .
Thanks a lot if someone can tell me why.
After trying use command 7za d and 7za a, I change a file in a tarball successfully .

How to extract kubernetes pod command execution result attributes

I am connecting to pod via client-Go and I want to get the properties of the file directory
func GetPodFiles(c *gin.Context) {
client, _ := Init.ClusterID(c)
path := c.DefaultQuery("path", "/")
cmd := []string{
"sh",
"-c",
fmt.Sprintf("ls -l %s", path),
}
config, _ := Init.ClusterCfg(c)
req := client.CoreV1().RESTClient().Post().
Resource("pods").
Name("nacos-0").
Namespace("default").SubResource("exec").Param("container", "nacos")
req.VersionedParams(
&v1.PodExecOptions{
Command: cmd,
Stdin: false,
Stdout: true,
Stderr: true,
TTY: false,
},
scheme.ParameterCodec,
)
var stdout, stderr bytes.Buffer
exec, err := remotecommand.NewSPDYExecutor(config, "POST", req.URL())
if err != nil {
response.FailWithMessage(response.InternalServerError, err.Error(), c)
return
}
err = exec.Stream(remotecommand.StreamOptions{
Stdin: nil,
Stdout: &stdout,
Stderr: &stderr,
})
if err != nil {
response.FailWithMessage(response.InternalServerError, "Error obtaining file", c)
return
}
fmt.Println(stdout.String())
}
Execution Result Output
total 0
lrwxrwxrwx 1 root root 7 Jun 1 2018 bin -> usr/bin
drwxr-xr-x 5 root root 360 Feb 16 16:39 dev
lrwxrwxrwx 1 root root 8 Jun 1 2018 sbin -> usr/sbin
drwxr-xr-x 2 root root 6 Apr 11 2018 srv
Expect the result
"data": [
{
"perm": "drwxr-xr-x",
"mod_time": "2022-03-02 15:02:15",
"kind": "d",
"name": "temp",
"size": ""
},
]
Is there a good way or a golang third-party library to handle it. Please let me know. Thank you
In a Kubernetes pod you can execute the stat linux command instead of ls command.
$ stat yourFileOrDirName
The output of this command by default is like this:
File: yourFileOrDirName
Size: 346 Blocks: 0 IO Block: 4096 directory
Device: 51h/82d Inode: 40431 Links: 1
Access: (0755/drwxr-xr-x) Uid: ( 1000/ username) Gid: ( 1000/ groupname)
Access: 2022-03-02 11:59:07.384821351 +0100
Modify: 2022-03-02 11:58:48.733821177 +0100
Change: 2022-03-02 11:58:48.733821177 +0100
Birth: 2021-12-21 11:12:05.571841723 +0100
But you can tweak its output like this:
$ stat --printf="%n,%A,%y,%s" yourFileOrDirName
where %n - file name, %A - permission bits and file type in human readable form, %y - time of last data modification human-readable, %s - total size, in bytes. You can also choose any character as a delimiter instead of comma.
the output will be:
yourFileOrDirName,drwxr-xr-x,2022-03-02 11:58:48.733821177 +0100,346
See more info about the stat command here.
After you get such output, I believe you can easily 'convert' it to json format if you really need it.
Furthermore, you can run the stat command like this:
$ stat --printf="{\"data\":[{\"name\":\"%n\",\"perm\":\"%A\",\"mod_time\":\"%y\",\"size\":\"%s\"}]}" yourFileOrDirName
Or as #mdaniel suggested, since the command does not contain any shell variables, nor a ', the cleaner command is:
stat --printf='{"data":[{"name":"%n","perm":"%A","mod_time":"%y","size":"%s"}]}' yourFileOrDirName
and get the DIY json output:
{"data":[{"name":"yourFileOrDirName","perm":"drwxrwxr-x","mod_time":"2022-02-04 15:17:27.000000000 +0000","size":"4096"}]}

How to uncompress a file generated by the Go compress/gzip package using the command line?

I have a file that was generated using the Go compress/gzip package with code like
payload := bytes.NewBuffer(nil)
gw := gzip.NewWriter(payload)
tw := tar.NewWriter(gw)
...
tw.Close()
gw.Close()
How can I unzip this file from the command line on Mac? I tried gunzip but it fails
$ gunzip test.gz
gunzip: test.gz: not in gzip format
Also tried following without luck
$ tar -xvf test.gz
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
Your issue is that you are never writing to your tar writer, in your example code. In order to produce a valid targz file with content, you need to keep in mind that:
Add at least one file needs to be in the tarball
Each file requires headers
You'll need to use tar.WriteHeader to create the header for each file, and you can then simply write the content of the files as bytes through a call to tar.Write.
You can then untar it using tar -xvf test.tgz like you mentioned in your previous example.
Here is a sample code that I quickly wrote on my machine for the sake of demonstration:
package main
import (
"compress/gzip"
"archive/tar"
"os"
"fmt"
"time"
)
func main() {
// Create targz file which will contain other files.
file, err := os.Create("test.tgz")
if err != nil {
panic(err)
}
gw := gzip.NewWriter(file)
defer gw.Close()
tw := tar.NewWriter(gw)
defer tw.Close()
// Create file(s) in targz
if err := addFile(tw, "myfile.test", "example content"); err != nil {
panic(err)
}
}
func addFile(tw *tar.Writer, fileName, content string) error {
header := &tar.Header{
Name: fileName,
Size: int64(len(content)),
Mode: 0655,
ModTime: time.Now(),
}
err := tw.WriteHeader(header)
if err != nil {
return fmt.Errorf("could not write header for file %q: %w", fileName, err)
}
_, err = tw.Write([]byte(content))
if err != nil {
return fmt.Errorf("could not write content for file %q: %w", fileName, err)
}
return nil
}
And here is the result:
$> go run main.go
$> ls -la
total 5
drwxr-xr-x 12 ullaakut staff 384 Feb 12 05:34 ./
drwxr-xr-x 29 ullaakut staff 928 Jan 28 14:56 ../
-rw-r--r--# 1 ullaakut staff 6148 Dec 25 13:01 .DS_Store
-rw-r--r-- 1 ullaakut staff 888 Feb 12 05:34 main.go
-rw-r--r-- 1 ullaakut staff 121 Feb 12 05:34 test.tgz
$> tar -zxvf test.tgz
x myfile.test
$> cat myfile.test
example content
As you can see, the archive is extracted and uncompressed just fine :)

Resources