Say I have got code for a project spread across multiple files. Is there any other way to reference such code within files in the project without going through the whole build and install process? For example within a single project, I might have an implementation of a type Foo in file A and I might want to use such type in file B within the same project. Is there anyway to go about doing that
It's not clear to me what exactly you're asking about, but this part of the language specs about "Declarations and scope" might be related and/or helpful. Another, more prosaic source of related information might probably be "How to Write Go Code".
EDIT - Expanding on now expanded question:
If "package a" declares type Foo - then, b/c Foo starts with an upper case letter - Foo is exported by a. Then in "package b" one would write import "a" and can then refer to that type as a.Foo inside "package b".
Related
I'm moderately new to common lisp, but have extended experience with other 'separate compilation' languages (think C/C++/FORTRAN and such)
I know how to do an ASDF system definition. I know how to separate stuff in packages. I'm using SBCL, by the way.
The question is this: what's the best practice for splitting code (large packages) between .lisp files? I mean, in C there are include files, while lisp lives with the current image state. So with multiple files I need to handle dependencies or serial order in the system definition. But without something like forward declarations it's painful.
Simple example on what I want to do: I have, for example, two defstructs that are part of the same bigger data structure (like struct1 is a parent of some set of struct2). Some functions works on one, some other works on the other and some other use both.
So I would have: a packages.lisp, a fun1.lisp (with the first defstruct and related functions), a fun2.lisp (with the other defstruct and functions) and a funmix.lisp (with functions that use both). In an ideal world everything is sealed and compiling these in this order would be fine. As most of you know, this in practice almost never happen.
If I need to use struct2 functions from the struct1 ones I would need to either reorder or add a dependency. But then if there's some kind of back call (that can't be done with a closure) I would have struct1.lisp depending on struct2.lisp and vice-versa which is obviously not valid. So what? I could break the loop putting the defstruct in a separate file (say, structs.lisp) but what if either of the struct's function need to access the common functions in the third file? I would like to avoid style notes.
What's the common way to solve this, i.e. keeping loosely related code in the same file but still be able to interface to other ones. Is the correct solution to seal everything in a compilation unit (a single file)? use a package for every file with exports?
Lisp dependencies are simple, because in many cases, a Lisp implementation doesn't need to process the definition of something in order to compile its use.
Some exceptions to the rule are:
Macros: macros must be loaded in order to be expanded. There is a compile-time dependency between a file which uses macro and the file which defines them.
Packages: a package foo must be defined in order to use symbols like foo:bar or foo::priv. If foo is defined by a defpackage form in some foo.lisp file, then that file has to be loaded (either in source or compiled form).
Constants: constants defined with defconstant should be seen before their use. Similar remarks apply to inline functions, compiler macros.
Any custom things in a "domain specific language" which enforces definition before use. E.g. if Whizbang Inference Engine needs rules to be defined when uses of the rules are compiled, you have to arrange for that.
For certain diagnostics to be suppressed like calls to undefined functions, the defining and using files must be taken to be as a single compilation unit. (See below.)
All the above remarks also have implications for incremental recompilation.
When there is dependency like the above between files so that one is a prerequisite of the other, when the prerequisite is touched, the dependent one must be recompiled.
How to split code into files is going to be influenced by all the usual things: cohesion, coupling and what have you. Common-Lisp-specific reasons to keep certain things together in one file is inlining. The call to a function which is in the same file as the caller may be inlined. If your program supports any in-service upgrade, the granularity of code loading is individual files. If some functions foo and bar should be independently redefinable, don't put them in the same file.
Now about compilation units. Suppose you have a file foo.lisp which defines a function called foo and bar.lisp which calls (foo). If you just compile bar.lisp, you will likely get a warning that an undefined function foo has been called. You could compile foo.lisp first and then load it, and then compile bar.lisp. But that will not work if there is a circular reference between the two: say foo.lisp also calls (bar) which bar.lisp defines.
In Common Lisp, you can defer such warnings to the end of a compilation unit, and what defines a compilation unit isn't a single file, but a dynamic scope established by a macro called with-compilation-unit. Simply put, if we do this:
(with-compilation-unit
(compile-file "foo.lisp") ;; contains (defun foo () (bar))
(compile-file "bar.lisp")) ;; contains (defun bar () (foo))
If a compile-file isn't surrounded by with-compilation-unit then there is a compilation unit spanning that file. Otherwise, the outermost nesting of the with-compilation-unit macro determines the scope of what is in the compilation unit.
Warnings about undefined functions (and such) are deferred to the end of the compilation unit. So by putting foo.lisp and bar.lisp compilation into one unit, we suppress the warnings about either foo or bar not being defined and we can compile the two in any order.
Build systems use with-compilation-unit under the hood, as appropriate.
The compilation unit isn't about dependencies but diagnostics. Above, we don't have a compile time dependency. If we touch foo.lisp, bar.lisp doesn't have to be recompiled or vice versa.
By and large, Lisp codebases don't have a lot of hard dependencies among the files. Incremental compilation often means that just the affected files that were changed have to be recompiled. The C or C++ problem that everything has to be rebuilt because a core header file was touched is essentially nonexistent.
but what if
No matter how you first organize your code, if you change it significantly you are going to have to refactor. IMO there is no ideal way of grouping dependencies in advance.
As a rule of thumb it is generally safe to define generic functions first, then types, then actual methods, for example. For non-generic functions, you can cut circular dependencies by adding forward declarations:
(declaim (ftype function ...))
Having too much circular dependency is a bit of a code smell.
Is the correct solution to seal everything in a compilation unit
Yes, if you group the definitions in the same compilation unit (the same file), the file compiler will be able to silence the style notes until it reaches the end of file: at this point it knows if there are still missing references or if all the cross-references are resolved.
But then if there's some kind of back call (that can't be done with a closure)
If you have a specific example in mind please share, but typically you can define struct1 and its functions in a way that can be self-contained; maybe it can accept a map that binds event names to callbacks:
(make-struct-1 :callbacks (list :on-empty one-is-empty
:on-full one-is-full))
Similarly, struct2 can accept callbacks too (Dependency Injection) and the main struct ties them using closures (?).
Alternatively, you can design your data-structures so that they signal conditions, and the in the caller code you intercept them to bind things together.
I have a Go project in JetBrains goland where all files are runnable yet independent of each other.
But to make every runnnable, I need to make them as package main.
And I have several "Vertex" defined elsewhere in other file and Goland complain about it.
But it is still runnable, and that's purely complaint from Goland.
Question -
Is there a better way to organized the files?
If not, is there a way to turn off the complaint from Goland?
Working with multiple files that declare the main() function in the same directory is not recommended in general, mainly due to problems similar to yours.
However, there are several ways to solve this.
You can use build constraints, also known as build tags, to separate the binaries at build time. When using them, the IDE will also need to be adjusted using the Settings/Preferences | Build Tags & Vendoring. And, depending how you build your application, you might also need to adjust the build command to add the corresponding tags to it.
The other option, which I'd recommend in this case, is to move each main() defining file into a structure such as this:
/repository_root
/cmd
/command1
command1.go (file holds the `main()` func)
/command2
command2.go (file holds the `main()` func)
/command3
command3.go (file holds the `main()` func)
/some
/package
some_file.go
some_other_file.go
....
some_other_file.go
As an example of this layout, you can have a look at Delve, which uses a similar structure, but only has a single "command" in the cmd folder.
Lastly, sometimes it's possible to remove the duplication and move it to a common file which holds the data type, but it's not always ideal and can make the build command more complex, since you need to specify all the files that should be included in the build process.
Edit:
And you can read more on how to organize your Go packages/applications here
These articles will explain how to organize your Go packages:
https://rakyll.org/style-packages/
https://medium.com/#benbjohnson/standard-package-layout-7cdbc8391fc1#.ds38va3pp
https://peter.bourgon.org/go-best-practices-2016/#repository-structure
To understand more about the design philosophy for Go packages: https://www.goinggo.net/2017/02/design-philosophy-on-packaging.html
I am reading mixed opinions on how to name structs and the files that contain them. I'm also having a hard time finding detail on multi-word structs.
What is the most standard way to name my structs and the files that contain them in the following project?
I have an executable project with 2 structs:
Foo
FooBar
I want to declare the structs in their own file so I can create test files. Each will get imported into main.go.
Do I name the structs PascalCase? if not, how?
How should the go files be named?
edit:
This doc is why I went with Pascal Case first, but it is a one word struct and doesn't show it being used in a separate file. https://tour.golang.org/moretypes/2
The effective go blog post in the official Golang blog helps a lot with this.
The Package Names section and the Mixed Caps as suggested by icza are specially helpful.
I also had this discussion with the community in the official slack channel a couple months ago, and most agreed that files should be always lowercase. If the names are too big, you can use underscores , be be careful: the go build will ignore files starting with _, and don't use specific suffixes, like _test.go and OS/arch names, like _amd64.go or _linux.go. The build tool will only use those last ones in the specific OSs/architectures. This is specified the the build docs.
So if you are creating a struct called FooBar, your file can either be foobar.go or foo_bar.go.
TL;DR:
Do test written within a package end up in the final exported package? Do they add any garbage or weight to a compiled binary?
Longer version:
Let's say that I have a foo Go package:
pkg/
foo/
bar.go
bar_test.go
I'm aware of the black box vs white box approaches to testing in go. A short recap is that I can either:
have bar_test.go declare a foo_test package, or
have it part of the main foo package.
Approach 1 provides better isolation because it allows to focus on the public API of the package, as one only access the exported identifiers of foo. Also, when application code imports the foo package with import "pkg/foo", only the files containing the main foo package are compiled. That's nice. [1]
However, there are cases where putting the tests in foo is a convenient compromise. I don't particularly like it myself, but I can see it in several codebases and I understand why at times it's necessary.
My question is about what happens to these tests. Since they're part of the package foo, when foo is imported somewhere, I'd expect the tests to be brought along. Or is the compiler smart enough to strip them?
[1] this is not a question on what approach is "better". It's doesn't really matter. Please abstain from commenting with "that approach is not better".
Yes, the Go tool is smart enough not to pass them to the compiler.
https://github.com/golang/go/issues/10184
I've project done in C++ where I used #define macros to give a name of the project, which I used in several places, I don't change this name often but sometimes I may need to change this, then I change this macro and rebuild my code. Now I'm converting this code to Go. Can someone suggest me how to implement this in Go? I'm not interested in using global variables for this purpose because I've many such macros and I doubt this cause my project to occupy more cpu and effect the performance.
Luckily, Go does not support macros.
There are two venues in Go to implement what is done using macros
in other programming languages:
"Meta-programming" is done using code generation.
"Magic variables/constants" are implemented using "symbol substitutions"
at link time.
It appears, the latter is what you're after.
Unfortunately, the help on this feature is nearly undiscoverable
on itself, but it explained in the output of
$ go tool link -help
To cite the relevant bit from it:
-X definition
add string value definition of the form importpath.name=value
So you roll like this:
In any package, where it is convenient,
you define a string constant the value of which you'd like to change at build time.
Let's say, you define constant Bar in package foo.
You pass a special flag to the go build or go install invocation for the linking phase at compile time:
$ go install -ldflags='-X foo.Bar="my super cool string"'
As the result, the produced binary will have the constant foo.Bar
set to the value "my super cool string" in its "read-only data" segment,
and that value will be used by the program's code.
See also the go help build output about the -ldflags option.
Go doesn't support Macros.
but you can use a constants inside a package and refer it where ever you need.
package constant
// constants.go file
const (
ProjectName = "My Project"
Title = "Awesome Title"
)
and in your program
package main
import "<path to project>/constant" // replace the path to project with your path from GOPATH
func main() {
fmt.Println(constant.ProjectName)
}
The project structure would be
project
|- constant
| |- constants.go
|-main.go