After being unable to run stack haddock on my project because one package causes it to hang (see How to show progress of `stack haddock`? ) I tried to skip that package with
stack haddock --skip haskell-src-exts
But it still tries to haddock haskell-src-exts anyway (and hangs). What is --skip supposed to do?
--skip was introduced in stack 1.6.1 to avoid building a specific component of a project. A component corresponded to a test suite, an executable or a benchmark suite. As backpack support is implemented, this will have to include libraries, too.
From stack build --help
--skip ARG Skip given component, can be specified multiple times
The docs go into more detail.
--skip, to skip building components of a local package. It allows
you to skip test suites and benchmark without specifying other components
(e.g. stack test --skip long-test-suite will run the tests without the
long-test-suite test suite). Be aware that skipping executables won't work
the first time the package is built due to
an issue in cabal.
This option can be specified multiple times to skip multiple components.
For example, a package my-package might have a library, an executable my-executable, and two test suites (unit-tests and integration-tests). To build and run tests, you may run stack test. Adding --skip integration-tests will cause the integration-tests component not to be built (nor run).
Unfortunately, the current version of stack (1.9 branch) does not support skipping haddock for individual dependencies, though there is something like this due in the near future. I encountered this exact same problem (with haskell-src-exts). For now, I suggest skipping all haddock dependencies (--no-haddock-deps).
You may wish to invoke haddock with stack exec -- haddock if you wish to pass arguments to haddock which do not work with the stack flag --haddock-arguments.
Related
I'm trying to use stack install to build and install some executables. My package also has an internal library that the test suite and benchmarks depend on, but none of the executables do. But when I do stack install, it attempts to build the internal library. I haven't been able to find a way to stop that, trying various combinations of targets and --skip. Is it possible?
My root package.yaml is laid out like
name: my-lib
library:
...
internal-libraries:
test-lib:
...
dependencies:
- my-lib
tests:
test:
...
dependencies:
- test-lib
...
benchmarks:
bench:
...
dependencies:
- test-lib
...
And the package.yamls for the executables are just
name: some-exe
executables:
some-exe:
...
dependencies:
- my-lib
...
Nothing lists test-lib in its dependencies. test-lib only exists so that the benchmark can use some utilities defined in the test suite; until recently it was just part of the test suite, not a separate library, and stack install wasn't trying to build it.
An option I've found is to create a separate package for just the internal library. I've created a directory test-pkg and moved the everything from the internal-libraries line down into the package.yaml in there. I also created symlinks to the source directories for my test suite and benchmarks, and assorted other relevant things found in the root of my original package.
Then I can do
stack install `stack ide targets 2>&1 | grep :exe:`
To install all the binaries.
I don't like this very much, but it seems to work.
I have a repository with a group of nested go packages organized as follows:
$GOPATH/src/
- mypackage/common/utils.go
- mypackage/app1/main.go
- mypackage/app2/main.go
...
It compiles to a handful of binaries. For releasing, I'm cross-compiling for a multitude of platforms/archs (I deploy repeatedly with a different GOOS and GOARCH). I'm trying to write the compilation results to a directory of my choice, but I'm fighting with the toolchain.
I can:
Combine GOBIN and go install when compiling for my own architecture
(i.e. not cross compiling):
# build + output all binaries in /somedir/bin
# this works great when compiling for my local architecture.
GOBIN=/somedir/bin go install mypackage/app1 mypackage/app2 mypackage/app3
But unfortunately, GOBIN conflicts with cross-compilation e.g:
# Throws: "cannot install cross-compiled binaries when GOBIN is set"
GOBIN=/somedir/bin GOARCH=amdm64 GOOS=darwin go install mypackage/app1 mypackage/app2
Use go build with GOOS=X and GOARCH=Y for each subpackage
# cross compile one of the binaries
cd /output/darwin-386-bin/ && \
GOOS=darwin GOARCH=386 go build mypackage/app1
# but if given multiple binaries, there is no output. (as documented)
GOOS=darwin GOARCH=386 go build mypackage/app1 mypackage/app2
When go build is given multiple packages, it no longer emits binaries -- it just checks that the code compiles. To emit all the binaries, it seems I have to run go build once for each subpackage, so it takes longer, esp when building with -a.
Another possible issue with using go build like this, is that it's potentially mixing binaries and intermediary results across multiple architectures in the same workspace. But maybe that's just a matter of taste. Hopefully the toolchain keeps cached results separate for different architectures.
How is this handled in practice? Should I treat each subpackage as an individual package, despite their shared common code (would it then be safe to build them all in parallel)?
Related articles:
go build vs go install: https://groups.google.com/forum/#!topic/golang-nuts/s8Csz3-7EXA (Little info on cross-compilation, or multipackage setups)
cross compile with go 1.5.: https://dave.cheney.net/2015/08/22/cross-compilation-with-go-1-5 (does not specifically address multipackage / common code compilation. Also, the statement about having to recompile the standard library into /usr/local/go/pkg seems to no longer hold in 1.9.1, as far as I can tell)
Go build multiple nested packages: Go build multiple/nested packages? . ( The accepted answer, go build ./... behaves the same as passing multiple packages in a single build command, and therefore outputs no binaries. The thread also does not cover cross compilation)
What I'm about to suggest feels like a hack at best, but it might work for you if you're cross-compiling stuff in a container or an isolated build environment.
You can drop the GOBIN from the install command:
# I'm not on this platform, so this is a cross compile
GOOS=darwin GOARCH=amd64 go install mypackage/app1 mypackage/app2
Assuming your package is in $GOPATH/src/mypackage the above command will install the two binaries to:
$GOPATH/bin/darwin_amd64/{app1, app2}
and the .a compile dependency files in:
$GOPATH/pkg/darwin_amd64/mypackage/{app1,app2,common}
If you're running this in a for loop for all the platforms you plan on supporting, one nuisance with this process is that when you pass GOOS=x GOARCH=y matching your local architecture (i.e. not cross compiling), then the executables will be placed directly in $GOPATH/bin/ and not $GOPATH/bin/x_y/.
(packages on the other hand always end up in $GOPATH/pkg/x_y/, cross compilation or not).
To determine your local architecture, you can follow steps in this answer: https://stackoverflow.com/a/35669816/5556676 .
Simulating GOBIN=/foo/bin GOOS=x GOARCH=y go install mypackage/app{1,2,3}
Changing $GOPATH has little effect on where go install writes output. With some go commands, like go get, you tweak where packages are installed by adding a new component at the front of $GOPATH. But in go 1.9, go install will always copy binaries in the bin/ folder that's a sibling of the src/ which contains the packages to install.
Instead of using GOBIN=/foo/bin, you can pretend your source files are in /foo/src/mypackage (you may use a symlink), and then do GOPATH=/foo GOOS=x GOARCH=y go install mypackage/app{1,2,3}. This will put binaries in /foo/bin, because of the behavior I describe in the previous paragraph.
It's probably simpler to just grab the binaries from where you expect them to be though, than mucking with copying source trees around.
I committed a change to Haskell's opaleye project that I need for a project I am working on. The change is in version 0.6.7003.1, which has not propagated to the nixos repository yet (nixos.org shows its current version to be 0.6.7001.0).
Since this needed dependency differs from the the resolver's LTS version, I have marked the extra dependency in stack.yaml:
packages:
- git#github.com:tomjaguarpaw/haskell-opaleye.git
- commit: cf3296c5ffef58d36dd6b386ae53dff519ee47e9
I have also marked this version in the build-depends of my project.cabal file:
build-depends: opaleye >= 0.6.7003.1 && < 1
When I then attempted to build, I got the following error:
$ stack build
Error: While constructing the build plan, the following exceptions were encountered:
In the dependencies for taskmaster-0.1.0.0:
opaleye must match >=0.6.7003.1 && <1, but the stack configuration has no specified version (latest matching version is 0.6.7003.1)
needed since taskmaster is a build target.
Some different approaches to resolving this:
* Consider trying 'stack solver', which uses the cabal-install solver to attempt to find some working build configuration. This can be convenient when dealing with many
complicated constraint errors, but results may be unpredictable.
* Recommended action: try adding the following to your extra-deps in /home/matthew/backup/taskmaster/taskmaster/stack.yaml:
opaleye-0.6.7003.1#sha256:914ac99c6f7ceea050df843ac31c33be0f6340bc0f05753c8fdfc18074fa9e5b
Plan construction failed.
(I do not understand where this long (>40 char) sha256 hash comes from).
I followed the advice and added the following to my stack.yaml.
extra-deps:
- opaleye-0.6.7003.1#sha256:914ac99c6f7ceea050df843ac31c33be0f6340bc0f05753c8fdfc18074fa9e5b
When I run $ stack build to build my project, the compiler attempts to build without showing any dependency errors. However, it strikes a type error that it would not if it were using the version of Opaleye containing my change. And, when I run $ stack ghci and import a relevant opaleye module, my change is not present. It appears that somehow stack is still using an older opaleye version. How can I get stack to use the more recent version of opaleye containing my change?
My attempts seem to have exhausted the options mentioned in the haskell-stack documentation.
The hackage option is usually better than the github extra-dep.
The correct syntax for what you initially tried would be
extra-deps:
- git: https://github.com/tomjaguarpaw/haskell-opaleye.git
commit: cf3296c5ffef58d36dd6b386ae53dff519ee47e9
and the github: tomjaguarpaw/haskell-opaleye version is just a shortcut.
The sha256 hash is the sha256sum of the cabal file corresponding to that version (and revision) of opaleye.
Given that version of opaleye is already on hackage, you should just be able to add the line suggested by stack. You may wish to try a stack clean after you make the change (though it shouldn't be necessary).
If you present more information, I'll update this answer.
It turned out that stack's extra-deps suggestion was incorrect. I noticed an alternative format under this stack github issue, gave it a try, and it happened to work for me. The working extra-deps format is listed below.
extra-deps:
- github: tomjaguarpaw/haskell-opaleye
commit: cf3296c5ffef58d36dd6b386ae53dff519ee47e9
Just got started with Nix (version 2.2.1), and while installing darcs (version 2.14.1) i encountered my first problem: I get the following error message (preceded by the callstack):
Setup: Encountered missing dependencies:
base >=4.9 && <4.12,
network >=2.6 && <2.8,
stm >=2.1 && <2.5,
zip-archive ==0.3.*
I have the haskell tool stack installed as well as a global ghc (though both should not be needed to build darcs i think).
I also had no problem with installing darcs with 'apt'
Am i making a classic nix beginner mistake or whats going on here?
Nix is very different from package managers like 'apt'. Derivations (which are like packages) are designed to be built in an isolated environment, where the derivation is responsible for providing its own dependencies by referencing other derivations. Because of this, you do not need to explicitly install anything in order to build a package.
Note also that although Nixpkgs uses the Cabal library to build Haskell packages, installing a package via Nix is quite different from installing with cabal-install. In fact it is closer to Stack, because Nixpkgs defines its haskellPackages based on stackage and it avoids cabal-style dependency resolution. It does however let you use the Cabal solver to check whether the dependencies match the versions specified in the cabal files. This check can be disabled using the doJailbreak function in Nixpkgs.
I don't think we need to get into the details of Haskell packaging in Nixpkgs though, because you should be able to get a pre-built darcs from the nixos-18.09 channel. The Nix expression from the nixos-unstable produces exactly your error message.
I recommend you to use the latest release channel, nixos-18.09, because nixos-unstable will break regularly. See the Nix manual for changing your channel configuration.
How can I put my Go binary into a Debian package? Since Go is statically linked, I just have a single executable--I don't need a lot of complicated project metadata information. Is there a simple way to package the executable and resource files without going through the trauma of debuild?
I've looked all over for existing questions; however, all of my research turns up questions/answers about a .deb file containing the golang development environment (i.e., what you would get if you do sudo apt-get install golang-go).
Well. I think the only "trauma" of debuild is that it runs lintian after building the package, and it's lintian who tries to spot problems with your package.
So there are two ways to combat the situation:
Do not use debuild: this tool merely calls dpkg-buildpackage which really does the necessary powerlifting. The usual call to build a binary package is dpkg-buildpackage -us -uc -b. You still might call debuild for other purposes, like debuild clean for instance.
Add the so-called "lintian override" which can be used to make lintian turn a blind eye to selected problems with your package which, you insist, are not problems.
Both approaches imply that you do not attempt to build your application by the packaging tools but rather treat it as a blob which is just wrapped to a package. This would require slightly abstraining from the normal way debian/rules work (to not attempt to build anything).
Another solution which might be possible (and is really way more Debian-ish) is to try to use gcc-go (plus gold for linking): since it's a GCC front-end, this tool produces a dynamically-linked application (which links against libgo or something like this). I, personally, have no experience with it yet, and would only consider using it if you intend to try to push your package into the Debian proper.
Regarding the general question of packaging Go programs for Debian, you might find the following resources useful:
This thread started on go-nuts by one of Go for Debian packagers.
In particular, the first post in that thread links to this discussion on debian-devel.
The second thread on debian-devel regarding that same problem (it's a logical continuation of the former thread).
Update on 2015-10-15.
(Since this post appears to still be searched and found and studied by people I've decided to update it to better reflec the current state of affairs.)
Since then the situation with packaging Go apps and packages got improved dramatically, and it's possible to build a Debian package using "classic" Go (the so-called gc suite originating from Google) rather than gcc-go.
And there exist a good infrastructure for packages as well.
The key tool to use when debianizing a Go program now is dh-golang described here.
I've just been looking into this myself, and I'm basically there.
Synopsis
By 'borrowing' from the 'package' branch from one of Canonical's existing Go projects, you can build your package with dpkg-buildpackage.
install dependencies and grab a 'package' branch from another repo.
# I think this list of packages is enough. May need dpkg-dev aswell.
sudo apt-get install bzr debhelper build-essential golang-go
bzr branch lp:~niemeyer/cobzr/package mypackage-build
cd mypackage-build
Edit the metadata.
edit debian/control file (name, version, source). You may need to change the golang-stable dependency to golang-go.
The debian/control file is the manifest. Note the 'build dependencies' (Build-Depends: debhelper (>= 7.0.50~), golang-stable) and the 3 architectures. Using Ubuntu (without the gophers ppa), I had to change golang-stable to golang-go.
edit debian/rules file (put your package name in place of cobzr).
The debian/rules file is basically a 'make' file, and it shows how the package is built. In this case they are relying heavily on debhelper. Here they set up GOPATH, and invoke 'go install'.
Here's the magic 'go install' line:
cd $(GOPATH)/src && find * -name '*.go' -exec dirname {} \; | xargs -n1 go install
Also update the copyright file, readme, licence, etc.
Put your source inside the src folder. e.g.
git clone https://github.com/yourgithubusername/yourpackagename src/github.com/yourgithubusername/yourpackagename
or e.g.2
cp .../yourpackage/ src/
build the package
# -us -uc skips package signing.
dpkg-buildpackage -us -uc
This should produce a binary .deb file for your architecture, plus the 'source deb' (.tgz) and the source deb description file (.dsc).
More details
So, I realised that Canonical (the Ubuntu people) are using Go, and building .deb packages for some of their Go projects. Ubuntu is based on Debian, so for the most part the same approach should apply to both distributions (dependency names may vary slightly).
You'll find a few Go-based packages in Ubuntu's Launchpad repositories. So far I've found cobzr (git-style branching for bzr) and juju-core (a devops project, being ported from Python).
Both of these projects have both a 'trunk' and a 'package' branch, and you can see the debian/ folder inside the package branch. The 2 most important files here are debian/control and debian/rules - I have linked to 'browse source'.
Finally
Something I haven't covered is cross-compiling your package (to the other 2 architectures of the 3, 386/arm/amd64). Cross-compiling isn't too tricky in go (you need to build the toolchain for each target platform, and then set some ENV vars during 'go build'), and I've been working on a cross-compiler utility myself. Eventually I'll hopefully add .deb support into my utility, but first I need to crystallize this task.
Good luck. If you make any progress then please update my answer or add a comment. Thanks
Building deb or rpm packages from Go Applications is also very easy with fpm.
Grab it from rubygems:
gem install fpm
After building you binary, e.g. foobar, you can package it like this:
fpm -s dir -t deb -n foobar -v 0.0.1 foobar=/usr/bin/
fpm supports all sorts of advanced packaging options.
There is an official Debian policy document describing the packaging procedure for Go: https://go-team.pages.debian.net/packaging.html
For libraries: Use dh-make-golang to create a package skeleton. Name your package with a name derived from import path, with a -dev suffix, e.g. golang-github-lib-pq-dev. Specify the dependencies ont Depends: line. (These are source dependencies for building, not binary dependencies for running, since Go statically links all source.)
Installing the library package will install its source code to /usr/share/golang/src (possibly, the compiled libraries could go into .../pkg). Building depending Go packages will use the artifacts from those system-wide locations.
For executables: Use dh-golang to create the package. Specify dependencies in Build-Depends: line (see above regarding packaging the dependencies).
I recently discovered https://packager.io/ - I'm quite happy with what they're doing. Maybe open up one of the packages to see what they're doing?