How to improve Golang compilation speed? - performance

I'm trying to find a way to make the compilation of a Go program faster. It is currently about 30 seconds, which makes it slow to work with the project.
When I run go build -v, I see that most of the time is spent compiling go-sqlite3 (which links to the C sqlite lib). However, since this lib never changes, I'm wondering if it's possible to prevent the build tool from recompiling this every time?

Try go install -a github.com/mattn/go-sqlite3 which will install the compiled-against-Go-1.3 package into your $GOPATH.
Right now, you likely have an older version installed under $GOPATH/pkg/ and therefore Go is recompiling it for every build.

This is likely due to you upgrading to go 1.3
I had to remove $GOPATH/pkg to get rid of old (incompatible) binaries
and then it was able to cache compilation results again

In Go 1.10 no need to run go install etc. Just use go build. The new version uses a build cache to determine which packages need to be re-compiled.
Check out: https://tip.golang.org/doc/go1.10

Related

Issue with using a modified version of a Go dependency

So here is the situation:
I have a fork of go-ipfs. It depends on go-ipfs-config. I need to modify go-ipfs-config and make go-ipfs depend on my modified version.
I forked the go-ipfs-config made my changes and made sure to update the path to be that of my forked version as can be seen here. I confirmed that this still builds successfully by running go build
Then I updated go.mod in go-ipfs to use my modified version. I used the replace directive to signify this intention which can be seen here
This is where things gets absolutely bunkers and I am no longer sure what is going on.
When i do go mod tidy to fetch the dependency i get the following output:
go: finding module for package github.com/dadepo/go-ipfs-config
go: found github.com/dadepo/go-ipfs-config in github.com/dadepo/go-ipfs-config v0.5.3
The crazy thing is that v0.5.3 does not exist in github.com/dadepo/go-ipfs-config!
Also the following line get added to go.mod :
github.com/dadepo/go-ipfs-config v0.5.3 // indirect
Which can even be seen here
I have run commands like go clean -modcache and go clean -r etc but does not seem to fix things!
Does anybody know what I am doing wrong? And also how to achieve the goal of making my version of a project depend on another modified version of its dependency?
Ok, so this is as a result of me not being aware of couple of things going on in the Go lang toolchain.
Apparently https://proxy.golang.org is a thing! It is a service operated by google that caches modules. So If you made a release, deleted it, chances are that the version is already cached in https://proxy.golang.org. This was exactly what happened in my case. I had made a 0.5.3 release, deleted it, but it is not really gone as the Google cache already got a hold of it.
So in case you are seeing versions that should not exist. This should be the first place you check. This documentation link also sheds some more light on the proxy and how it can be tweaked.
I found this out based on the conversation I had on the issue I opened reporting this behaviour. If you are curious, you can check it out here.

Running go mod vendor updates my libraries

I'm trying to vendor my go-modules and am using go 1.11.5. However, when i run go mod vendor, it appears that under the hood go mod tidy is being called first. This updates my go libraries. Unfortunately, I am dependent on an older version of a certain library.
Is there any way I can vendor based on go.sum instead?
Any help much appreciated!
I think the problem I was having is that my go program specified version 1.2.3 of a specific library, but a dependency of my program pulled in version 1.2.4, resulting in a bump when I did a go vendor.
Hostile environment here! I had no idea ;). I will think twice before asking anything in the future.
Peace!

if packagist says master build failing, If I install package with composer, will it work?

I'm kinda new to using composer (but I think it is awesome) so please pardon the noob question.
Folks are requesting exports fancier than csv so I thought I'd install phpOffice excel using composer. But, the master is 'build failing' and the develop is 'build error'. An I correct in assuming the master build failed means I should not install this with composer because it won't work?
In researching this I also found phpOffice spreadsheet, but that appears to be in development still.
Any other alternatives if I cannot use phpOffice excel due to the build status issue?
Regardless of what package you are talking about, if the builds for any of the branches are failing it just means that the builds for any of the branches are failing, that's it.
The master and develop branches may be well ahead of the latest tag, and chances are the maintainers are experimenting with it, and working towards a passing build again.
If you trust the maintainers not to release broken software, then it's safe to use one of the tagged versions, so for example, run
$ composer require phpoffice/phpexcel:^1.8.1
to install the latest stable version.
Note It appears that phpoffice/phpexcel:1.8.1 has been released on 2015-05-01; quite some time has passed since. Whether or not this package works for you, then, is something you have to find out for yourself.
For reference, see:
https://packagist.org/packages/phpoffice/phpexcel

Getting packages for GO language

The below command is taking ages with no output or anything, is there an alternative way to download packages for go language.? I am new to golang.
go get -u github.com/gogits/gogs/
PS: my net connection is not that slow, downloading through git takes around 1 min, but i can't individually do that for all dependencies.
Edit 1: Small packages like go get github.com/tools/godep downloads and install flawlessly, i have the problem only with github.com/gogits/gogs/ . Its stuck there for an hour. Even a download progress would have been helpful.
go get command is working but its terribly slow for packages that have lots of dependencies. the -v flag(not in doc) helped me to get a verbose output of what is happening.
Took almost an hour to finish. Culprit might possibly be github or my ISP.
Verbose output should had made default by the developers.
*sorry for my English.
You can use GoDep which is a powerful Go dependency manager: https://github.com/tools/godep
GoDep also helps you to have a predictable build since it freezes dependencies in your app
Very easy to get started, just follow instruction on its Github page.

Dependency solution when make/compile error from source code

Very often we need to install software from its source code. Most of the time I just hit "make world" or "make all" then it will work like a charm. But some other time we see make errors, and we need to install other packages in order to let the make go through. This is particularly a problem for compiling low-level systems, such as a Linux kernel or Xen hypervisor.
I have one experience with Xen 3.4. Maybe it has been documented in some corner documents, but it depends on udev-125 to work properly. The weird thing is it functions well most of the time when udev version is 160+, it only breaks in certain cases! It took me a few MONTHS to find out it was because of the wrong udev version!
To make developers' life easier, when a source code is made successfully in one machine, is there some tools to record the list of packages and versions of that machine? Such a 'snapshot' should be shipped with the source code as well, so that when someone meets the make error they at least have a successful 'snapshot' for reference.
Is there such a tool already?
If your software depends on a specific version of a dependency, you should write a check for your configure script/cmakefile/etc. that tests the version of the dependency and bails out if the wrong version was found.
Comparing the output of config.log (a file created by a configure script) can also help diagnose problems like you encountered.

Resources