check binary integrity in golang - go

i try to create integrity protection of my application , this is my actual code :
package main
import (
"os"
"io"
"crypto/sha256"
"fmt"
)
var OriginalSign string
func checkSUM() string {
hasher := sha256.New()
f, err := os.Open(os.Args[0])
if err != nil {
os.Exit(0)
}
defer f.Close()
if _, err = io.Copy(hasher, f); err != nil {
os.Exit(0)
}
return fmt.Sprintf("%x", hasher.Sum(nil))
}
func main() {
signature := checkSUM()
fmt.Println(OriginalSign)
fmt.Println(signature)
if signature != OriginalSign {
fmt.Println("binary is compromised")
}
}
i compiled with this command :
C:\Users\admin\go\src\localhost\lic>go build -ldflags="-s -w -X main.OriginalSig
n=8636cdeef255e52c6fd3f391fd7d75fbaf7c6e830e0e7ac66a645093c7efcbc7" -o checksum.
exe checksum.go
C:\Users\admin\go\src\localhost\lic>checksum.exe
8636cdeef255e52c6fd3f391fd7d75fbaf7c6e830e0e7ac66a645093c7efcbc7
d29440d3467f6176a6af0dcb61ea696cb318db3a6f1680b5b8f7890e165d8d7e
binary is compromised
how i can do this corectly in go ? i need to know signature of final binary file and check if is compromited.

I can't see how to hook into tool buildid in a program but it can (kind of) detect changes to a binary
buildid does seem to store a "contentid" of the binary which is the essence of the original question
Here's a bash script that shows this (sorry I don't do MS Windows)
#
# delete any old binaries
rm -f t
# do an initial build
go build t.go
# show it works
./t
# get the original buildid
ORIG=$(go tool buildid t)
# now tamper with it!
perl -p -i -e 's/testing/porkpie/' t
# run again, show the tamper
./t
# now regenerate the buildid
go tool buildid -w t
# get the buildid after the regeneration
LATER=$(go tool buildid t)
# show the original and the post-tampering buildid - they are different
echo "$ORIG"
echo "$LATER"
Here's the do nothing t.go
package main
import (
"fmt"
)
func main() {
fmt.Println("testing 123")
}
Here's the output
testing 123
porkpie 123
koB1H61TwQSHTQGiI4PP/-o93sSzqt1ltMhBJn4pR/2wvL4J9vF4vGUGjdbsyd/y-0uRBmxfJdrbAfsE1lr
koB1H61TwQSHTQGiI4PP/-o93sSzqt1ltMhBJn4pR/2wvL4J9vF4vGUGjdbsyd/UeLetY1pBF54B_4Y8-Nj
So the go tool buildid can store a hash in with the binary and (kind of) detect tampering. But I couldn't work out how to get the contentid from a normal call inside a normal program

Related

"file is not gofmted" in go report card

I have a Go project, where I wanted to generate a Go report card (https://goreportcard.com/)
One of the things that this report card is that it runs
gofmt -s
On all files.
My repo contains around 25 Go files, the only flag that is raised is this one, on six files:
Line 1: warning: file is not gofmted with -s (gofmt)
I've been googling around on gofmt, but I really can't find what this actually means.
Here is an example of a file that raises the error:
package services
import (
"github.com/heyjoakim/devops-21/models"
log "github.com/sirupsen/logrus"
)
var d = GetDBInstance()
// GetUserID returns user ID for username
func GetUserID(username string) (uint, error) {
var user models.User
getUserIDErr := d.db.First(&user, "username = ?", username).Error
if getUserIDErr != nil {
log.WithFields(log.Fields{
"err": getUserIDErr,
"username": username,
}).Error("Error in GetUserID")
}
return user.UserID, getUserIDErr
}
and here is a file that does not raise the error:
package services
import (
"strconv"
"github.com/heyjoakim/devops-21/models"
log "github.com/sirupsen/logrus"
)
func UpdateLatest(latest int) {
var c models.Config
err := d.db.First(&c, "key = ?", "latest").Error
if err != nil {
log.WithField("err", err).Error("Latest does not exist: DB err")
c.ID = 0
c.Key = "latest"
c.Value = strconv.Itoa(latest)
d.db.Create(&c)
} else {
err := d.db.Model(&models.Config{}).Where("key = ?", "latest").Update("Value", latest).Error
if err != nil {
log.WithField("err", err).Error("UpdateLatest: DB err")
}
}
}
I really don't see why one raises some error on line 1, and the other doesn't?
What does this flag mean?
The command gofmt -s myfile.go prints the formatted file to stdout. The -s flag applies simplifications to the formatted file.
Run gofmt -s -d myfile.go to view the differences between the original file and the formatted file.
Run gofmt -s -w myfile.go to update the file to the desired formatting.
Replace myfile.go with . to operate on all files in the directory.
The documentation for the gofmt command is here.
Your files are not formatted. Run gofmt -s -w . in the directory to fix the files.

Unable to use google/apis/annotations

I try to use REST over gRPC using google api annotations.
Unfortunately, I'm facing a protoc issue telling me annotations.proto is not exists or had errors.
I tried several fixes in vain.
I even tried to reinstall a complete stack in case I did something wrong.
I will detail you as much as possible the full lines and files I have set from my fresh install.
From a fresh go install VM I type these shell lines :
$ mkdir sources/golang
$ echo 'export GOPATH=$HOME/sources/golang' >> $HOME/.zshrc
$ source ~/.zshrc
$ cd sources/golang
$ mkdir src
$ cd src
$ export PATH=$PATH:$GOPATH/bin
$ go get -u google.golang.org/grpc
$ go get -u github.com/golang/protobuf/protoc-gen-go
$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger
$ mkdir -p test/proto/test
$ cd test/proto/test
$ vim test.proto
In My test.proto file, I wrote very basic lines :
syntax = "proto3";
package main;
import "google/api/annotations.proto";
service Tester {
rpc Test (Request) returns (Reply) {
option (google.api.http) = { get: "/v1/test" };
}
}
message Request {
string Message = 1;
}
message Reply {
string Message = 1;
}
Then
$ cd $GOPATH/src
$ vim main.go
In my main.go, very basic too :
package main
import (
tc "test/proto/test"
"context"
"fmt"
"log"
"net"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
)
func main() {
err := StartServer("tcp", "127.0.0.1:50051")
if err != nil {
fmt.Printf("Error!! %s", err)
}
}
type Server struct {
tc.UnimplementedTesterServer
}
func (s *Server) Test(ctx context.Context, in *tc.Request) (*tc.Reply, error) {
return &tc.Reply{Message: ""}, nil
}
func StartServer(protocol string, port string) error {
lis, err := net.Listen(protocol, port)
if err != nil {
fmt.Printf("failed to listen: %v", err)
}
s := grpc.NewServer()
tc.RegisterTesterServer(s, &Server{})
error := s.Serve(lis)
if error != nil {
fmt.Printf("failed to serve: %v", error)
return error
}
return nil
}
Finally, I try to compile my proto files :
$ protoc --proto_path=.:$GOPATH/src --go_out=plugins=grpc:. proto/*/*.proto
And I systematically have the following error :
proto/test/test.proto:5:1: Import "github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis/google/api/annotations.proto" was not found or had errors.
When I see files get by go get ..., they are placed under $GOPATH/pkg/mod/
For example the googleapis' annotations.proto file is under : $GOPATH/pkg/mod/github.com/grpc-ecosystem/grpc-gateway#v1.12.1/third_party/googleapis/google/api/annotations.proto
Maybe is it the cause?
Mixing up your protobuf commands and your go commands is making this more complicated than it needs to be. Just focus on the protobuf to handle the error.
You are importing "google/api/annotations.proto"
Your proto_path is --proto_path=.:$GOPATH/src
So that means that when you execute protoc you should have the file located at ./google/api/annotations.proto or $GOPATH/src/google/api/annotations.proto
I faced the same problem and I was trying to automate a solution that would work on any Linux with as little dependencies as possible.
The official docs for grpc_gateway states:
You will need to provide the required third party protobuf files to the protoc compiler. They are included in this repo under the third_party/googleapis folder, and we recommend copying them into your protoc generation file structure. If you've structured your proto files according to something like the Buf style guide, you could copy the files into a top-level ./google folder.
In other words: download it somehow and make it work.
Also, since this quote is also a quote in the original source, it is not very visible which made me skip it the first 2 times I read it looking for a solution, which is bad.
So, as I argued on the comments of #SeanF answer, using go get is not a maintainable option since it saves the project in a folder whose name contains the latest version, which would cause headaches to maintain when the version changes.
So the best option is actually to clone the grpc-gateway project:
git clone https://github.com/grpc-ecosystem/grpc-gateway
protoc -I grpc-gateway/ \
-I grpc-gateway/third_party/googleapis \
--go_out=plugins=grpc:. proto/*/*.proto
I also wrote a gist with my solution which depends only on docker and bash and, thus, should be stable:
https://gist.github.com/VinGarcia/43dfa71b412c16b9365c224d3760af5e
There is a problem with it, where I am using go.mod but discarding it after every execution, which is not good. But it works well enough for now and it might be helpful to some people.
Regards

Go: embed bash script in binary

I'm trying to cross compile a Go program that will execute a bash script. Is it possible to embed the bash script in the binary?
I've referenced:
Golang serve static files from memory
Not sure if this applies to executing bash scripts though. Am I missing something here? Some clarification or pointers will be very helpful, thanks!
Since bash can execute scripts from stdin, you just send your script to the bash command via command.Stdin.
An example without go embed:
package main
import (
"bytes"
"fmt"
"os/exec"
"strings"
)
var script = `
echo $PWD
pwd
echo "-----------------------------------"
# print process
ps aux | grep code
`
func main() {
c := exec.Command("bash")
c.Stdin = strings.NewReader(script)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
With go 1.16 embed (https://golang.org/pkg/embed/):
package main
import (
"bytes"
_ "embed"
"fmt"
"os/exec"
"strings"
)
//go:embed script.sh
var script string
func main() {
c := exec.Command("bash")
c.Stdin = strings.NewReader(script)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
Bonus
Passing parameters to your script with -s -.
The following example will pass -la /etc to the script.
package main
import (
"fmt"
"os/exec"
"strings"
)
func main() {
// pass parameters to your script as a safe way.
c := exec.Command("sh", "-s", "-", "-la", "/etc")
// use $1, $2, ... $# as usual
c.Stdin = strings.NewReader(`
echo $#
ls $1 "$2"
`)
b, e := c.Output()
if e != nil {
fmt.Println(e)
}
fmt.Println(string(b))
}
Playground: https://go.dev/play/p/T1lMSrXcOIL
You can actually directly interface with the system shell in Go. Depending on what's in your bash script you can probably convert everything completely to go. For example things like handling files, extracting archives, outputting text, asking for user input, downloading files, and so much more can be done natively in Go. For anything you absolutely need the shell for you can always use golang.org/pkg/os/exec.
I wrote a snippet that demonstrates a really simple Go based command shell. Basically it pipes input, output, and error between the user and the shell. It can be used interactively or to directly run most shell commands. I'm mentioning it here mostly to demonstrate Go's OS capabilities. Check it out: github.com/lee8oi/goshell.go
Did you try writing the stream-data (according to the reference go-bindata provides a function that returns []byte) into a temporary file?
see: http://golang.org/pkg/io/ioutil/#TempFile
you can then execute it with a syscall
http://golang.org/pkg/syscall/#Exec
where the first argument needs to be a shell.

Printing output to a command window when golang application is compiled with -ldflags -H=windowsgui

I have an application that usually runs silent in the background, so I compile it with
go build -ldflags -H=windowsgui <gofile>
To check the version at the command line, I wanted to pass a -V flag to the command line to get the string holding the version to be printed to the command prompt then have the application exit. I added the flag package and code. When I test it with
go run <gofile> -V
...it prints the version fine. When I compile the exe, it just exits, printing nothing. I suspect it's the compilation flag causing it to not access the console and sending my text into the bit bucket.
I've tried variations to print to stderr and stdout, using println and fprintf and os.stderr.write, but nothing appears from the compiled application. How should I try printing a string to the command prompt when compiled with those flags?
The problem is that when a process is created using an executable which has the "subsystem" variable in its PE header set to "Windows", the process has its three standard handles closed and it is not associated with any console—no matter if you run it from the console or not. (In fact, if you run an executable which has its subsystem set to "console" not from a console, a console is forcibly created for that process and the process is attached to it—you usually see it as a console window popping up all of a sudden.)
Hence, to print anything to the console from a GUI process on Windows you have to explicitly connect that process to the console which is attached to its parent process (if it has one), like explained here for instance. To do this, you call the AttachConsole API function. With Go, this can be done using the syscall package:
package main
import (
"fmt"
"syscall"
)
const (
ATTACH_PARENT_PROCESS = ^uint32(0) // (DWORD)-1
)
var (
modkernel32 = syscall.NewLazyDLL("kernel32.dll")
procAttachConsole = modkernel32.NewProc("AttachConsole")
)
func AttachConsole(dwParentProcess uint32) (ok bool) {
r0, _, _ := syscall.Syscall(procAttachConsole.Addr(), 1, uintptr(dwParentProcess), 0, 0)
ok = bool(r0 != 0)
return
}
func main() {
ok := AttachConsole(ATTACH_PARENT_PROCESS)
if ok {
fmt.Println("Okay, attached")
}
}
To be truly complete, when AttachConsole() fails, this code should probably take one of these two routes:
Call AllocConsole() to get its own console window created for it.
It'd say this is pretty much useless for displaying version information as the process usually quits after printing it, and the resulting user experience will be a console window popping up and immediately disappearing; power users will get a hint that they should re-run the application from the console but mere mortals won't probably cope.
Post a GUI dialog displaying the same information.
I think this is just what's needed: note that displaying help/usage messages in response to the user specifying some command-line argument is quite often mentally associated with the console, but this is not a dogma to follow: for instance, try running msiexec.exe /? at the console and see what happens.
One problem with the solutions already posted here is that they redirect all output to the console, so if I run ./myprogram >file, the redirection to file gets lost. I've written a new module, github.com/apenwarr/fixconsole, that avoids this problem. You can use it like this:
import (
"fmt"
"github.com/apenwarr/fixconsole"
"os"
)
func main() {
err := fixconsole.FixConsoleIfNeeded()
if err != nil {
fmt.Fatalf("FixConsoleOutput: %v\n", err)
}
os.Stdout.WriteString(fmt.Sprintf("Hello stdout\n"))
os.Stderr.WriteString(fmt.Sprintf("Hello stderr\n"))
}
Answer above was helpful but alas it did not work for me out of the box. After some additional research I came to this code:
// go build -ldflags -H=windowsgui
package main
import "fmt"
import "os"
import "syscall"
func main() {
modkernel32 := syscall.NewLazyDLL("kernel32.dll")
procAllocConsole := modkernel32.NewProc("AllocConsole")
r0, r1, err0 := syscall.Syscall(procAllocConsole.Addr(), 0, 0, 0, 0)
if r0 == 0 { // Allocation failed, probably process already has a console
fmt.Printf("Could not allocate console: %s. Check build flags..", err0)
os.Exit(1)
}
hout, err1 := syscall.GetStdHandle(syscall.STD_OUTPUT_HANDLE)
hin, err2 := syscall.GetStdHandle(syscall.STD_INPUT_HANDLE)
if err1 != nil || err2 != nil { // nowhere to print the error
os.Exit(2)
}
os.Stdout = os.NewFile(uintptr(hout), "/dev/stdout")
os.Stdin = os.NewFile(uintptr(hin), "/dev/stdin")
fmt.Printf("Hello!\nResult of console allocation: ")
fmt.Printf("r0=%d,r1=%d,err=%s\nFor Goodbye press Enter..", r0, r1, err0)
var s string
fmt.Scanln(&s)
os.Exit(0)
}
The key point: after allocating/attaching the console, there is need to get stdout handle, open file using this handle and assign it to os.Stdout variable. If you need stdin you have to repeat the same for stdin.
You can get the desired behavior without using -H=windowsgui; you'd basically create a standard app (with its own console window), and hide it until the program exits.
func Console(show bool) {
var getWin = syscall.NewLazyDLL("kernel32.dll").NewProc("GetConsoleWindow")
var showWin = syscall.NewLazyDLL("user32.dll").NewProc("ShowWindow")
hwnd, _, _ := getWin.Call()
if hwnd == 0 {
return
}
if show {
var SW_RESTORE uintptr = 9
showWin.Call(hwnd, SW_RESTORE)
} else {
var SW_HIDE uintptr = 0
showWin.Call(hwnd, SW_HIDE)
}
}
And then use it like this:
func main() {
Console(false)
defer Console(true)
...
fmt.Println("Hello World")
...
}
If you build a windowless app you can get output with PowerShell command Out-String
.\\main.exe | out-string
your build command may look like:
cls; go build -i -ldflags -H=windowsgui main.go; .\\main.exe | out-string;
or
cls; go run -ldflags -H=windowsgui main.go | out-string
No tricky syscalls nor kernel DLLs needed!

Open a file in the same directory as the .go source file in Go

When in a source file $PWD/dir/src.go I use
os.Open("myfile.txt")
it looks for myfile.txt in $PWD (which looks normal).
Is there way to tell Go to look for myfile.txt in the same directory as src.go ? I need something like __FILE__ in Ruby.
Go is not an interpreted language so looking for a file in the same location as the source file doesn't make any sense. The go binary is compiled and the source file doesn't need to be present for the binary to run. Because of that Go doesn't come with an equivalent to FILE. The runtime.Caller function returns the file name at the time the binary was compiled.
I think perhaps if we understood why you actually wanted this functionality we could advise you better.
A possible substitute skeleton:
func __FILE__() (fn string) {
_, fn, _, _ = runtime.Caller(0)
return
}
Details here.
Use package osext
It's providing function ExecutableFolder() that returns an absolute path to folder where the currently running program executable reside (useful for cron jobs). It's cross platform.
Online documentation
package main
import (
"github.com/kardianos/osext"
"fmt"
"log"
)
func main() {
folderPath, err := osext.ExecutableFolder()
if err != nil {
log.Fatal(err)
}
fmt.Println(folderPath)
}
You can also get full executable path (similar to __FILE__):
package main
import (
"github.com/kardianos/osext"
"fmt"
)
func main() {
exeAbsolutePath, _ := osext.Executable()
fmt.Println(exeAbsolutePath)
}

Resources