I have two different files: (1) /common/handler.go and (2) main.go.
In the (/common/handler.go) file, I have declared
var db *sql.DB
var err error
as global variables (at the top level, below import). Now, I want to use these two variables in my main.go file because I have this line of code in my main() function:
db, err = sql.Open("mysql","username:password#tcp(127.0.0.1:3306)/test123")
What should I do in order for Go to understand that I was referring to that db and err variables? Am I even doing this the right way? If not, please let me know the best solution.
The reason I split into these two files because I want to make the code cleaner.
Basically, main() in main.go contains
router := mux.NewRouter()
router.HandleFunc("/", common.login)
....
and handler.go contains all the messy stuff.
Thanks,
"I want to use these two variables in my main.go" You can't because they are unexported, ergo you need to export them.
"What should I do in order for Go to understand that I was referring to that db and err variables?" You need to qualify them. It's no different from you telling Go with sql.Open that you're referring to the function Open that's declared in the package database/sql.
Also I recommend you take the Tour of Go.
Related
There's a lot of examples in test code that are shadowing package names in test libraries like testify.
For example:
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSomething(t *testing.T) {
assert := assert.New(t)
require := require.New(t)
var a string = "Hello"
var b string = "Hello"
assert.Equal(a, b, "The two words should be the same.")
}
Shouldn't it be an explicit decision and rare occurrence to override the value of a package namespace with a local variable?
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSomething(t *testing.T) {
assertions := assert.New(t)
req := require.New(t)
var a string = "Hello"
var b string = "Hello"
assert.Equal(a, b, "The two words should be the same.")
}
I'd expect the proper behavior to be aliasing the package like pkgassert which I doesn't feel idiomatic, or ensuring that variables are not set to the same name as the package: assertions := assert.New(t).
Other examples:
Ok: buf := bytes.Buffer{}
Bad: bytes := bytes.Buffer{}
Note: I'm not focused on variable naming patterns, as Go tends to recommend single letter, or short variables in the narrowed scope you might use the buffer for. Assume the longer name is acceptable and desired due to the function length.
If this is a desired behavior in testing packages I'd like to know why?
It appears to violate a key principle of idiomatic Go in "no magic" and being readable.
If there are undesired side effects from this I'd like to know of that as well.
My assumption is that is means after import that any calls to the package in the same scope wouldn't work due to variable shadowing
Edit
Scope Shadowing in Go was a nice read and gave some great points on usage. The main takeaway I had from this and from Stack commenters:
It can enhance readability. (I think this is probably due to Go preferring small concise naming for packages, making package names in Go variable length.)
Parallel Test Issues was something I recall reading a while back. I didn't understand the recommendation to "hide" the loop variable by putting in:
for _, tc := range tests {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
// Here you test tc.value against a test function.
// Let's use t.Log as our test function :-)
t.Log(tc.value)
})
}
}
With a better understanding of shadowing, I'm seeing this "hiding" was talking about shadowing the caller scope variable of tc by assigning inside the loop, thereby shadowing.
This seems like a good use case, though I still feel it is being "clever" rather if not commented, and violates the idiomatic Go goals of being clear and readable with no "magic".
After reviewing all the feedback and writing I've seen I'm going to summarize the findings here.
If you are investigating this behavior after seeing Go linting alerts from a tool like Goland, hopefully this answer will save you same time and make clear that it's ok, idiomatic, and only in some cases of reusing the actual package calls after initializing would any issues be experienced.
Shadowing Package Names With Local Variables
This is a matter of opinion, there is no official statement indicating this is considered non-idiomatic code.
Opinions seem to overall lean towards avoiding shadowing unless intentionally done for clarity & readability, but this is again opinion, not a requirement.
Once overridden, the package would be unavailable in that scope as the package name has been shadowed by the local variable.
I've observed that this behavior seems more likely in Go than some languages, because Go prompts package names that are short, not multi-word, and easy to type. It seems inevitable that assert := assert.New(t) could result when this happens.
Consistency, as always in matters like this, is the key. If a team determines to never shadow the package name, then assertions := assert.New(t) could be used, or as mentioned the package could be aliased such as import tassert "github.com/stretchr/testify/assert". This is an opinion standard, and either way is legal and considered idiomatic.
In the case of testing, this is commonly done as further usage of the package after the initial variable set isn't used further. Example below using the is is package. Package author comment on this (again this is opinionated, but it doesn't break anything to use it with is := is.New(t).
package package_test
import (
"testing"
"package"
iz "github.com/matryer/is"
)
func TestFuncName(t *testing.T) {
is := iz.New(t)
got := FuncName()
want := ""
is.Equal(got,want) // AssertMessage
}
Shadowing Behavior Can Address Goroutine Variable Issues
Shadowing is a known behavior and documented clearly in Effective Go.
The bug is that in a Go for loop, the loop variable is reused for each iteration, so the req variable is shared across all goroutines. That's not what we want.
It may seem odd to write req := req but it's legal and idiomatic in Go to do this. You get a fresh version of the variable with the same name, deliberately shadowing the loop variable locally but unique to each goroutine.
This can be solved without shadowing by passing in the variable as a parameter into the embedded func as well, simplifying the code. Both are legal and considered idiomatic per the guide.
Shadowing for Tests
Related to this goroutine behavior, but in the context of a testing, parallel tests run in goroutines, and therefore benefit from this pattern as well.
The tc := tc statement is required in the parallel test example because the closure runs in a goroutine. More generally, the tc := tc idiom used in the context of closures where the closure can be executed after the start of the next loop iteration - credit #gopher in comments
The gist on Be Careful With Table Driven Tests mentions this behavior as well in the How To Solve This.
Detecting
The check go vet can check for shadowing, but is not considered part of the stable package, and was removed here related to GitHub Issue 29260 and GitHub Issue 34053.
The change seems to have been prompted for compatibility purposes due to the requirement of unsafeptr flag. Additional comments seem to point towards also requiring better heuristics before allowing in go vet without being experimental.
Go Vet Shadow
Instructions on running via go vet: Package Variables With Instructions:
go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow
go vet -vettool=$(which shadow)
For those wanting to lint this, golangci-lint does have an option to configure shadowing checks in the configuration (search for check-shadowing as no bookmark to that text block`.
Further Reading
Scope and Shadowing in Go
Scopes and Closures
Wow, that is awful code. While it might not be a compiler enforced rule, if I see this:
import "github.com/stretchr/testify/assert"
Then in that file, as far as I'm concerned, any identifier spelled assert is the top level import from that package, period. Anyone who breaks that pattern, in the name of being "clever" or whatever, is just asking for trouble. People's muscle memory is trained to recognize top level imports, so please, do not do stuff like this. If you want to avoid thinking of a new name, just remove some letters:
asrt := assert.New(t)
or add a letter:
nAssert := assert.New(t)
or alias the import:
import tAssert "github.com/stretchr/testify/assert"
Just please, do not follow the poor example given in that code.
As I come from a C and C++ background and golang is semantically different I have some discussions about function names "New" vs "Open" and what the programmer expects what to happen under the hood. I have created a package where the New function opens file descriptors to temporary files. I'm not sure this is intended behavior for APIs written in Go.
Here is a snippet:
// Not directly convenient New allocates file descriptors
deb := New()
deb.AddFile("/tmp/myfile")
deb.Write("/tmp/mypackage.deb")
deb.Close()
// Idiomatic it opens file descriptors but we have to provide context to open
info := &Props{Name: "mypackage"}
deb := info.Open("/tmp/mypackage.deb")
deb.AddFile("/tmp/myfile")
deb.Close()
Go's community is still finding its idioms and patterns, so don't consider anything dogma. Based on what I've seen in a year and a half, I don't think it's wrong to touch disk in a New function if it has godoc comments explaining the behavior. It would be unusual, and probably make more sense to touch the disk in d.Write(). Some common patterns I've seen with New functions:
are only exported when designed for use by other packages
makes sense when invoked from other packages with the package name prefixed: d := deb.New(...) (since you have a variable named deb, your package is something else)
returns an instance of the package's main business logic, to be used further
accept dependencies as arguments instead of constructing its own dependencies (hopefully any behavioral dependencies are interfaces)
often a main() function assembles those dependencies, calls d := deb.New(deps), and executes the behavior d.AddFile(); d.Write(); d.Close()
an example signature using an interface:
package deb
// New accepts blah returning a *Deb. It returns nil if blah.
func New(name string, to io.WriteCloser) *Deb {...}
I really like Go but makes me crazy about if-err hell and when I have sync datatypes in Go code with other languages. For C/C++ I can easily deal such stuff with macros, while Go developers say the idiomatic solution for Go is code generation, but I didn't find any out-of-the-box solution.
So basically, what I need is something like
Read the source, for every type usage check if it is listed in special config file. If it is, then change it with the one from config.
Read the source, for every function check if it is listed in config file. If it is then, change it with the code snippet from config by template and add neccessary import if it's missing.
Probably, add some polymorphism based on return values to prevent type casts.
Maybe, add (err error) logic. Not sure it's a good idea.
Like this
code.go
func getConn(id platform.UUID) (res1 string, res2 platform.res) {
res1 = driver_native_res(id)
res2 = driver_native_res(id)
return
}
code-gen.go
import (
"linux"
)
func getConn(id uint64) (res1 string, res2 int32, err error) {
res1, err = linux.GetResAsString(id)
if err != nil {
return
}
res2, err = linux.GetRes(id)
if err != nil {
return
}
return
}
I know about go AST, but seems like it's not very fast to implement such features with it. I hope there is some easier solution.
As you have discovered, there are no macros and are unlikely to be. There may be generics at some point which could be helpful. In the meantime, there are a few options for code generation. You can use go generate for this:
// add a magic comment to your go file
// which needs some extra generated code
// this calls a tool called stringer to generate,
// but it could be yacc or some homemade tool.
//go:generate stringer -type=Pill
// call go generate to generate the methods:
go generate
https://blog.golang.org/generate
But it really requires you to have a tool to generate the code you want like stringer.
Or you could just use text/template, build your own template, and run a simple tool that substitutes values into this template (from a config file, or perhaps arguments on the command line).
mytool generate -type=thing -field=type...
... mytool loads a tmplate file,
runs it through text/template and outputs it.
This is pretty straightforward and you could easily build a custom system with it, though you'll probably want to generate once, then use the code. Here are some examples of this approach:
http://clipperhouse.github.io/gen/
https://github.com/fragmenta/fragmenta
Finally, you can use tools like grpc which generate structs in multiple languages in order to ease cross-language communication, which sounds like exactly the use case you are looking for:
https://grpc.io/docs/quickstart/go.html
I'd look at something like grpc first.
I am currently writing ExampleFuncs in my Go test scripts. For example:
package hello
import "testing"
func ExampleGetSymbol() {
data := GetSymbol("AAPL")
fmt.Println(len(data.Data))
// Output: 21
}
Now, this gives me two benefits:
This example is executed when I am running go test and
It appears in the godoc documentation under func GetSymbol
One thing bothers me and I am wondering if there is anything I should do about it.
For the user that's trying to learn from this example, the line
data := GetSymbol("AAPL")
should actually be
data := hello.GetSymbol("AAPL")
but since the test is in the same scope as the package, I cannot use it like this.
So I guess the distilled version of my question would be:
Is there a way to allow package.field notation inside the package scope?
Thanks in advance
Rather than putting this in package hello, put it in package hello_test. You're allowed to have both hello and hello_test packages in the same directory, and it allows (requires) you to create your examples the way you're suggesting.
Incidentally, this also causes you to write your test cases (at least for this file) only to the public API. This is often a good thing. But if you need to write to private functions, you can split your tests into separate files, some in the hello package and some in hello_test.
BTW, documentation for this is slightly buried. You can find it in the "Test packages" section of the go cmd documentation.
package demo
type People struct {
Name string
Age uint
}
type UserInfo struct {
Address string
Hobby []string
NickNage string
}
another package:
import "demo"
in this package, how can I get all the types exported from the demo package?
Using go/importer:
pkg, err := importer.Default().Import("time")
if err != nil {
fmt.Println("error:", err)
return
}
for _, declName := range pkg.Scope().Names() {
fmt.Println(declName)
}
(note, this returns an error on the Go Playground).
Go retains no master list of structs, interfaces, or variables at the package level, so what you ask is unfortunately impossible.
Drat, I was hoping that Jsor's answer was wrong, but I can't find any way to do it.
All is not lost though: If you have the source to 'demo', you could use the parser package to fish out the information you need. A bit of a hack though.
Do you come from some scripting language? It looks so.
Go has good reasons to propagate to let not slip 'magic' into your code.
What looks easy at the beginning (have access to all structs, register them automatically somewhere, "saving" coding) will end up in debugging and maintenance nightmare when your project gets larger.
Then you will have to document and lookup all your lousy conventions and implications.
I know what I am talking about, because I went this route several times with ruby and nodejs.
Instead, if you make everything explicit you get some feature, like renaming the People struct to let
the compiler tell you where it is used in your whole code base (and the go compiler is faster than you).
Such possibilities are invaluable for debugging, testing and refactoring.
Also it makes your code easy to reason about for your fellow coders and yourself several months after you have written it.