This seems like a really simple issue but for the life of me I cannot figure it out.
All I want to do is unzip and extract the contents of a tar.gz file. On godoc there seems to be a reference to a function that does exactly this. (https://godoc.org/golang.org/x/build/internal/untar).
There are no examples though and I can't seem to figure it out. Worse yet, though, I can't even figure out how to get access to the function.
Right now I have:
package main
import (
"io"
"os"
//???
)
func main() {
f, err := os.Open("foo.tar.gz")
if err != nil {
panic(err)
}
defer f.Close()
var freader io.ReadCloser = f
err = untar.Untar(freader, ".") // <- Compiler doesn't recognize
if err != nil {
panic(err)
}
freader.Close()
}
You can't. Packages with internal as part of their name can't be used by any package outside of the same repo. In other words, golang.org/x/build/internal/untar is for golang.org/x/build only, and the compiler won't allow you to reference it from anywhere else.
You can, if you really like, copy the source of that package into a new file, and use it from there, but you can't use it directly. This gives the developers of the internal untar package complete freedom to break their interface without having to worry about other users.
Related
Is there a way to check if a public function/struct is used outside of the package in which it's declared? I'm not writing a public go module that's consumed anywhere else, and simply want to scan whether func Foo() it's used anywhere in my codebase outside of the package in which it's declared.
I'm using GoLand but any programmatic solution would do.
Simplest solution: manually rename Foo() to Foo2(). Build/compile your project: if there are no compilation errors, it's not referenced in your code. Same check also works with any identifiers and with any IDEs (this doesn't use any of the IDE's features).
Obviously if you already have a Foo2 identifier, this will fail. But the idea is to rename it to a non-existing identifier...
You can scan a particular package to see all the available function in it.
In this main.go, app the root package name and there is another package in database directory under the package name database.
By running the code you will found all the function name available inside database package
package main
import (
"fmt"
"app/database"
"go/ast"
"go/parser"
"go/token"
"os"
)
// Name of the package you want to scan
const subPackage = "database"
func main() {
set := token.NewFileSet()
packs, err := parser.ParseDir(set, subPackage, nil, 0)
if err != nil {
fmt.Println("Failed to parse package:", err)
os.Exit(1)
}
funcs := []*ast.FuncDecl{}
for _, pack := range packs {
for _, f := range pack.Files {
for _, d := range f.Decls {
if fn, isFn := d.(*ast.FuncDecl); isFn {
funcs = append(funcs, fn)
}
}
}
}
fmt.Println("All the functions in the package:",subPackage)
for _, fn := range funcs {
fmt.Println(fn.Name.Name)
}
// database Package is called/used
database.Connection()
}
This will get all function declarations in the stated subpackage as an ast.FuncDecl. This isn't an invokable function; it's just a representation of its source code of it.
If you wanted to do anything like call these functions, you'd have to do something more sophisticated. After gathering these functions, you could gather them and output a separate file that calls each of them, then run the resulting file.
I have a file with serialized array in PHP.
The content of the file locks like this
a:2:{i:250;s:7:"my_catz";s:7:"abcd.jp";a:2:{s:11:"category_id";i:250;s:13:"category_name";s:7:"my_catz";}}
The array unserialized is this
(
[250] => my_catz
[abcd.jp] => Array
(
[category_id] => 250
[category_name] => my_catz
)
)
Now, i want to get the content of the file in GO, unserialize it convert it to an array.
In GO i can get the content of the file using
dat, err := os.ReadFile("/etc/squid3/compiled-categories.db")
if err != nil {
if e.Debug {
log.Printf("error reading /etc/squid3/compiled-categories.db: ", err)
}
}
And unserialized it using github.com/techoner/gophp library
package categorization
import (
"fmt"
"os"
"github.com/techoner/gophp"
"log"
"errors"
)
type Data struct {
Website string
Debug bool
}
func (e Data) CheckPersonalCategories() (int,string) {
if e.Debug {
log.Printf("Checking Personal Categories")
}
if _, err := os.Stat("/etc/squid3/compiled-categories.db"); errors.Is(err, os.ErrNotExist) {
if e.Debug {
log.Printf("/etc/squid3/compiled-categories.db not exit: ", err)
}
return 0,""
}
dat, err := os.ReadFile("/etc/squid3/compiled-categories.db")
if err != nil {
if e.Debug {
log.Printf("error reading /etc/squid3/compiled-categories.db: ", err)
}
}
out, _ := gophp.Unserialize(dat)
fmt.Println(out["abcd.jp"])
return 0,""
}
But I can't access to the array, for example, when I try access to array key using out["abcd.jp"] i get this error message
invalid operation: out["abcd.jp"] (type interface {} does not support indexing)
The result of out is
map[250:my_catz abcd.jp:map[category_id:250 category_name:my_catz]]
Seams that is unserializing
Don't make assumptions about what is and isn't succeeding in your code. Error responses are the only reliable way to know whether a function succeeded. In this case the assumption may hold, but ignoring errors is always a mistake. Invest time in catching errors and at least panic them - don't instead waste your time ignoring errors and then trying to debug unreliable code.
invalid operation: out["abcd.jp"] (type interface {} does not support indexing)
The package you're using unfortunately doesn't provide any documentation so you have to read the source to understand that gophp.Unserialize returns (interface{}, error). This makes sense; php can serialize any value, so Unserialize must be able to return any value.
out is therefore an interface{} whose underlying value depends on the data. To turn an interface{} into a particular value requires a type assertion. In this case, we think the underlying data should be map[string]interface{}. So we need to do a type assertion:
mout, ok := out.(map[string]interface{})
Before we get to the working code, one more point I'd like you to think about. Look at the code below: I started it from your code, but the resemblance is very slight. I took out almost all the code because it was completely irrelevant to your question. I added the input data to the code to make a minimal reproduction of your code (as I asked you to do and you declined to do). This is a very good use of your time for 2 reasons: first, it makes it a lot easier to get answers (both because it shows sufficient effort on your part and because it simplifies the description of the problem), and second, because it's excellent practice for debugging. I make minimal reproductions of code flows all the time to better understand how to do things.
You'll notice you can run this code now without any additional effort. That's the right way to provide a minimal reproducible example - not with a chunk of mostly irrelevant code which still can't be executed by anybody.
The Go Plaground is a great way to demonstrate go-specific code that others can execute and investigate. You can also see the code below at https://go.dev/play/p/QfCl08Gx53e
package main
import (
"fmt"
"github.com/techoner/gophp"
)
type Data struct {
Website string
Debug bool
}
func main() {
var dat = []byte(`a:2:{i:250;s:7:"my_catz";s:7:"abcd.jp";a:2:{s:11:"category_id";i:250;s:13:"category_name";s:7:"my_catz";}}`)
out, err := gophp.Unserialize(dat)
if err != nil {
panic(err)
}
if mout, ok := out.(map[string]interface{}); ok {
fmt.Println(mout["abcd.jp"])
}
}
myprogram/
|
|-main.go
|-dir1/
|-data/
|-datafile.json
|-runner.go
|-runner_test.go
In runner.go, I have a simple function that reads the datafile.json. Something like
func GetPayload() (string, err) {
dBytes, dErr := ioutil.ReadFile("dir1/data/datafile.json")
if dErr != nil { return nil, dErr}
return dBytes, nil
}
I'm using Go in a Lambda with a structure similar to above. When the Lambda runs in its actual environment, it starts at main.go, and then invokes GetPayload() from runner.go. However, I have a test in a simple worker node machine in runner_test.go that also hits GetPayload() .
During "normal" execution (from main.go) - this works OK. However, when GetPayload() is invoked from runner_test.go, it errors, saying
open dir1/data/datafile.json no such file or directory
This makes sense, because during the test, the working directory is the directory that houses runner_test.go, which is data/, so there is no dir1 as a child of it. I've been trying to play with using os.Getwd() and getting the paths from there like:
pwd, _ := os.Getwd()
dBytes, dErr := ioutil.ReadFile(pwd + "dir1/data/datafile.json")
But again, that won't work, because for runner_test.go pwd is user/myname/myprogram/dir1, but from main.go, it turns up as user/myname/myprogram.
Any idea how I can find and open datafile.json from within GetPayload() in any environment? I could pass an optional parameter to GetPayload() but if possible, it'd be great to avoid that.
If the file is static (meaning that it doesn't need to change after you build the program), you can embed it into the built program. This means you no longer have to worry about run-time file paths.
import (
"embed"
)
//go:embed data/*
var dataFiles embed.FS
func GetPayload() (string, err) {
dBytes, dErr := dataFiles.ReadFile(dataFiles, "data/datafile.json")
if dErr != nil { return nil, dErr}
return dBytes, nil
}
Now the files in your data/ directory are embedded in this variable dataFiles which acts as a read-only file system.
For more info:
Read more about embed in the package documentation overview.
Read my answer about "when to use embed"
For data files that your program needs during runtime, either use a fixed directory and refer to that, or accept a command line argument or some sort of configuration that tells you where the file is.
When running unit tests, the wd is the directory containing the test file. One convention is to use a testdata/ directory under the directory containing the test, and put all data files there. That way you can refer to that file from the test by using testdata/datafile.json.
You can use a copy of the file you need during runtime as your test file, or you can use a symlink from the runtime data file to the test file under the testdata/ dir.
For data files that your program needs during runtime, either use a fixed
directory and refer to that
Someone made this suggestion, which I agree with. To that end, you can use
something like this:
package main
import (
"os"
"path/filepath"
)
func main() {
d, err := os.UserCacheDir()
if err != nil {
panic(err)
}
d = filepath.Join(d, "file.json")
f, err := os.Open(d)
if err != nil {
panic(err)
}
defer f.Close()
os.Stdout.ReadFrom(f)
}
https://golang.org/pkg/os#UserCacheDir
https://golang.org/pkg/os#UserConfigDir
I have recently looked into Go plugins instead of manually loading .so files myself.
Basically, I have a game server application, and I want to be able to load plugins (using plugins package) when the server starts. Then, in the plugin itself, I want to be able to call exported functions that are a part of the server.
Say I have this plugin, which is compiled to example_plugin.so using go build -buildmode=plugin:
package main
import "fmt"
func init() {
fmt.Println("Hello from plugin!")
}
Then say I have this server application, which loads the plugin (and ultimately calls the "init" function under the hood):
package main
import (
"fmt"
"plugin"
)
func main() {
fmt.Println("Server started")
if _, err := plugin.Open("example_plugin.so"); err != nil {
panic(err)
}
}
// some API function that loaded plugins can call
func GetPlayers() {}
The output is:
Server started
Hello from plugin!
This works as expected, however I want to be able to call that GetPlayers function (and any other exported functions in the server application, ideally) from the plugin (and any other plugins.) I was thinking about making some sort of library consisting of interfaces containing API functions that the server implements, however I have no idea where to start. I know I will probably need to use a .a file or something similar.
For clarification, I am developing this application for use on Linux, so I am fine with a solution that only works on Linux.
Apologies if this is poorly worded, first time posting on SO.
As mentioned in the comments, there is a Lookup function. In the documentation for the module they have the following example:
// A Symbol is a pointer to a variable or function.
// For example, a plugin defined as
//
// var V int
//
// func F() { fmt.Printf("Hello, number %d\n", V) }
//
// may be loaded with the Open function and then the exported package
// symbols V and F can be accessed
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("plugin_name.so")
if err != nil {
panic(err)
}
v, err := p.Lookup("V")
if err != nil {
panic(err)
}
f, err := p.Lookup("F")
if err != nil {
panic(err)
}
*v.(*int) = 7
f.(func())() // prints "Hello, number 7"
}
I think the most confusing lines here are
*v.(*int) = 7
f.(func())() // prints "Hello, number 7"
The first one of them performs a type assertion to *int to assert that v is indeed a pointer to int. That is needed since Lookup returns an interface{} and in order to do anything useful with a value, you should clarify its type.
The second line performs another type assertion, this time making sure that f is a function with no arguments and no return values, after which, immediately calls it. Since function F from the original module was referencing V (which we've replaced with 7), this call will display Hello, number 7.
When in a source file $PWD/dir/src.go I use
os.Open("myfile.txt")
it looks for myfile.txt in $PWD (which looks normal).
Is there way to tell Go to look for myfile.txt in the same directory as src.go ? I need something like __FILE__ in Ruby.
Go is not an interpreted language so looking for a file in the same location as the source file doesn't make any sense. The go binary is compiled and the source file doesn't need to be present for the binary to run. Because of that Go doesn't come with an equivalent to FILE. The runtime.Caller function returns the file name at the time the binary was compiled.
I think perhaps if we understood why you actually wanted this functionality we could advise you better.
A possible substitute skeleton:
func __FILE__() (fn string) {
_, fn, _, _ = runtime.Caller(0)
return
}
Details here.
Use package osext
It's providing function ExecutableFolder() that returns an absolute path to folder where the currently running program executable reside (useful for cron jobs). It's cross platform.
Online documentation
package main
import (
"github.com/kardianos/osext"
"fmt"
"log"
)
func main() {
folderPath, err := osext.ExecutableFolder()
if err != nil {
log.Fatal(err)
}
fmt.Println(folderPath)
}
You can also get full executable path (similar to __FILE__):
package main
import (
"github.com/kardianos/osext"
"fmt"
)
func main() {
exeAbsolutePath, _ := osext.Executable()
fmt.Println(exeAbsolutePath)
}