The question is simple.
I want to download a package (mingw-w64-x86_64-libshout) with all the dependencies on it recursively (dependencies of dependencies of ...).
I don't want to install the package in msys2 posix platform. Just to download it.
Using quasi-msys2 (which I'm the author of):
git clone https://github.com/holyblackcat/quasi-msys2
cd quasi-msys2
make get-deps mingw-w64-x86_64-gcc | xargs make cache-download
Where mingw-w64-x86_64-gcc is the package name.
The packages are downloaded to ./cache.
Caveat: it only supports packages prefixed with mingw-??-, not the ones from the MSYS environment. By default the MINGW64 environment is used (which owns the packages prefixed with mingw-w64-x86_64-, which is what you need), but it can be changed using e.g. echo UCRT64 >msystem.txt after cloning the repo.
It doesn't use pacman, and will work on other operating systems without MSYS2 installed.
Related
I am trying to install the libguestfs package for golang but couldn't find the way to do so. I went through this but it didn't work for me and I believe it's empty. I see only test files here
Following is the content of the README
$GOPATH
-------
http://golang.org/doc/code.html
Note that $GOPATH is set (by the ./run script) to the
libguestfs/golang directory so that "installs" happen in the local
build tree.
From this I could figure out that I need to build the package but again for that, I couldn't find some help or proper document.
Is there any way to do the installation? I am using RHEL 7.7
Libguestfs (the C library) is packaged already in RHEL 7, so I would suggest first of all installing that using yum install libguestfs. That's the version that Red Hat supports.
We don't ship the golang bindings, not even in RHEL 8. Furthermore the bindings in all languages are generated from a mechanical description of the API and thus not stored directly inside our git repository. That's why you could only find test files in git.
However we do distribute the generated files in the tarballs, so an easy way is to grab the latest tarball from https://download.libguestfs.org/1.42-stable/ and look in the golang/src/libguestfs.org/guestfs subdirectory of the tarball.
The harder way is to generate them from git. These steps worked for me on RHEL 7.7, YMMV:
sudo yum-builddep libguestfs
git clone https://github.com/libguestfs/libguestfs
cd libguestfs
git submodule init
git submodule update
./bootstrap
./autogen.sh SUPERMIN=/usr/bin/supermin5
make
and again look in the golang/ subdirectory.
I'm reasonably sure that golang bindings from one version should work with another version of the C library. We did test this at one point in time, so if it's no longer true then it may have regressed as these things sometimes do.
The haskell-platform (2014.2.0.0) ships MingGW, but without mingw-get. However, to compile the setlocale bindings for haskell I need locale.h, which is part of mingwrt.
In a usual mingw installation I would use mingw-get to install mingwrt, but since it's not included in the haskell-platform that doesn't work out of the box.
What would be the usual way of installing mingwrt without mingw-get?
Install MSYS[^1]
Download version 1.0.11 of MSYS. You'll need the following files:
MSYS-1.0.11.exe
msysDTK-1.0.1.exe
msysCORE-1.0.11-bin.tar.gz
The files are all hosted on haskell.org as they're quite hard to find in the official MinGW/MSYS repo
Run MSYS-1.0.11.exe followed by msysDTK-1.0.1.exe. The former asks you if you want to run a normalization step. You can skip that.
Unpack msysCORE-1.0.11-bin.tar.gz into D:\msys\1.0. Note that you can't do that using an MSYS shell, because you can't overwrite the files in use, so make a copy of D:\msys\1.0, unpack it there, and then rename the copy back to D:\msys\1.0.
[^1]:Setting up a Haskell development environment on Windows
I am trying to include ZMQ sockets in a Go app but both zmq4 and gozmq (the referred ZMQ binding libraries for Go) are giving me problems. I would like to understand why zmq4 specifically isn't importable on my system.
I am running a Windows 8 system and I used the windows installer from the ZMQ website for version 4.0.3. I am primarily concerned about getting zmq4 set up and here is the result of my "go get" query on the github library's location:
> go get github.com/pebbe/zmq4
# github.com/pebbe/zmq4
polling.go:4:17: fatal error: zmq.h: No such file or directory
compilation terminated.
This issue is not alleviated by cloning the Github repository - the error remains the same.
I know the issue has to do with the C library zmq.h that is located in the "include" folder of my ZMQ installation, but whether the dependency is held up by a pathing issue or an external tool issue is a mystery to me.
A similar error has come up in regards to node.js and is the solution I see others referred to, outside of node scripting, but it was unsuccessful in my case.
I've so far included the path to the "include" folder in my PATH environment variable and previously placed zmq.h inside of the zmq4 top-level folder. I don't have much of an arsenal otherwise to understand this problem because I am new to C and C-importing packages in Go
I wanted to do the same thing, but on Windows 7, and here is what I had to do.
Since the Go bindings are using cgo to integrate with zeromq, you need zeromq built with gcc. There are no pre-built binaries, so you'll have to build them yourself, with mingw or similar, but this process is easier than it may sound, and nicely described on the zeromq site.
As #photoionized pointed out, C_INCLUDE_PATH and LIBRARY_PATH need to be set when building the Go bindings.
(In my case, I ran into a problem when compiling libzmq with IN6_ADDR not being defined. The only solution I found was, inspired by this issue, to manually add the line #include <in6addr.h> to the windows.hpp file.)
The Windows installer version of ZeroMQ won't work with zmq4, you need to compile from source with gcc, I recommend using MSYS2.
Install and update MSYS2 following the instructions from
http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/
Start the mingw32_shell.bat or mingw64_shell.bat based on Go arch (32bit or 64bit)
pacman -S mingw-w64-(x86_64|i686)-toolchain make (x86_64 for 64bit, i686 for 32bit)
cd into zeromq src folder (C:\ path starts with /c/ inside the shell)
./configure
make
make install
CGO_CFLAGS=-I/usr/local/include CGO_LDFLAGS=-L/usr/local/lib go get github.com/pebbe/zmq4
Copy the following dlls and put them next to your go program (.exe):
/usr/local/bin/libzmq.dll
/mingw(32|64)/bin/libgcc*.dll
/mingw(32|64)/bin/libwinpthread*.dll
/mingw(32|64)/bin/libstdc++*.dll
Here's updated steps for #user2172816's MSYS2 solution:
Install and update MSYS2 following the instructions from http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/
Start the mingw32_shell.bat or mingw64_shell.bat based on Go arch (32bit or 64bit)
pacman -S mingw-w64-(x86_64|i686)-toolchain make (x86_64 for 64bit, i686 for 32bit)
Add C:\msys64\mingw64\bin to your Path (pkg-config is there)
Restart the msys2 shell to get the new Path
Download and unzip libsodium source: https://github.com/jedisct1/libsodium/releases
cd into libsodium folder (C:\ path starts with /c/ inside the shell)
./configure --build=(x86_64|i686)-w64-mingw32
make
make install
Add /usr/local/lib to PKG_CONFIG_PATH (export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig)
cd into zeromq src folder
./configure --build=(x86_64|i686)-w64-mingw32
Add
#ifdef ZMQ_HAVE_MINGW32
#include <winsock2.h>
#include <windows.h>
#include "netioapi.h"
#endif
To the top of src/tcpaddress.cpp
make
make install
CGO_CFLAGS=-I/usr/local/include CGO_LDFLAGS=-L/usr/local/lib go get github.com/pebbe/zmq4
CGO_CFLAGS=-I/usr/local/include CGO_LDFLAGS=-L/usr/local/lib go build in your project directory
Copy the following dlls and put them next to your go program (.exe):
/usr/local/bin/libzmq.dll
/mingw(32|64)/bin/libgcc*.dll
/mingw(32|64)/bin/libwinpthread-*.dll
/mingw(32|64)/bin/libstdc++*.dll
/usr/local/bin/libsodium-*.dll
maybe? /usr/local/bin/libsodium-*.def
An updated answer using MSYS2.
Install MSYS2 MSYS2 installation guide.
Make sure to choose the correct installation 32bit or 64bit.
Open the appropriate shell MSYS2 MinGW 64-bit or MSYS2 MinGW 32-bit. All further steps assume you are using this shell.
Update packages following instructions at the installation guide.
Install libtool pacman -Sy libtool.
Download zmq source code to a location of your choice.
Navigate to the zmq source folder.
To generate the configure file, run the autogen tool by running ./autogen.sh.
In the probable case that step 8 fails:
Find the file at fault (probably version.sh).
Replace line endings by (replace file by the actual filename).
cp file file.bak
tr -d '\r' <file.bak> file
If this fails you'll have to dive in the code and find the problem.
Run the configure tool ./configure.
In the probable case of failure. Comment out empty else clauses in the configure file.
Add Go to Path: PATH=${PATH}:<go bin directory>.
Install Go Package: CGO_CFLAGS=-I/usr/local/include CGO_LDFLAGS=-L/usr/local/lib go get github.com/pebbe/zmq4
To install ZMQ in windows: Problem in Installing Golang ZMQ for windows - fatal error: czmq.h: No such file or directory
First of all, install the msys64. Download the software from https://www.msys2.org/ and install it on C:\msys64.
Then add C:\msys64\mingw64\bin to PATH environment variable of the windows.
Then run the following commands (in CMD) one by one.
pacman -Su
pacman -S --needed base-devel mingw-w64-x86_64-toolchain
pacman -S base-devel gcc vim cmake
pacman -S mingw-w64-x86_64-libsodium
pacman -S mingw-w64-x86_64-zeromq
Finally, run the Go install command:
go get github.com/pebbe/zmq4
Finished.
I'm working on a support library for a large Python project which heavily uses relative imports by appending various project directories to sys.path.
Using The Hitchhiker's Guide to Packaging as a template I attempted to create a package structure which will allow me to do a local install, but can easily be changed to a global install later if desired.
One of the dependencies of my package is the pyasn1 package for the encoding and decoding of ASN.1 annotated objects. I have to include the pyasn1 library separately as the version supported by the CentOS 6.3 default repositories is one major version back and has known bugs that will break my custom package.
The top-level of the library structure is as follows:
MyLibrary/
setup.py
setup.cfg
LICENSE.txt
README.txt
MyCustomPackage/
pyasn1-0.1.6/
In my setup configuration file I define the install directory for my library to be a local directory called .lib. This is desirable as it allows me to do absolute imports by running the command import site; site.addsitedir("MyLibrary/.lib") in the project's main application without requiring our engineers to pass command line arguments to the setup script.
setup.cfg
[install]
install-lib=.lib
setup.py
setup(
name='MyLibrary',
version='0.1a',
package_dir = {'pyasn1': 'pyasn1-0.1.6/pyasn1'},
packages=[
'MyCustomPackage',
'pyasn1',
'pyasn1.codec',
'pyasn1.compat','
pyasn1.codec.ber',
'pyasn1.codec.cer',
'pyasn1.codec.der',
'pyasn1.type'
],
license='',
long_description=open('README.txt').read(),
data_files = []
)
The problem I've run into with doing the installation this way is that when my package tries to import pyasn1 it imports the global version and ignores the locally installed version.
As a possible workaround I have tried installing the pyasn1 package under a different name than the global package (eg pyasn1_0_1_6) by doing package_dir = {'pyasn1_0_1_6':'pyasn1-0.1.6/pyasn1'}. However, this fails since the imports used internally to the pyasn1 package do not use the pyasn1_0_1_6 name.
Is there some way to either a) force Python to import a locally installed package over a globally installed one or b) force a package to install under a different name?
Use virtualenv to ensure that your application runs in a fully known configuration which is independent from the OS version of libraries.
EDIT: a quick (unix) solution is setting the PYTHONPATH environment variable, which works just like PATH for Python modules (module loaded from first path in which is found, so simply append you directory at the beginning of the PYTHONPATH). Anwyay, I strongly recommend you to proceed with virtualenv, since it was specifically engineered for handling situations like the one you are facing.
Rationale
The process is easily automatable if you write a setuptools script specifying dependencies with install_requires. For a complete example, refer to this one I wrote
Setup
Note that you can easily insert the steps below in a setup.sh shell script.
First create a virtualenv and enter it:
$ virtualenv $name
$ cd $name
Activate it:
$ source bin/activate
Now cd to your project directory and run the installer script:
$ cd $my_project_dir
$ python ./setup.py --prefix $path_to_virtualenv
Note the --prefix $path_to_virtualenv, which is used to tell the script to install in the virtualenv instead of system-wide. Call this after activating the virtualenv. Note that all the depencies are automatically downloaded and installed in the virtualenv.
Then you are done. When you want to leave the virtualenv, issue:
$ deactivate
On subsequent calls, you will only need to activate the virtualenv (step 2), maybe using a runawesomeproject.sh if you really want.
As noted on the virtualenv website, you should use virtualenv >= 1.9, as the previous versions did not download dependencies via HTTPS. If you consider plain HTTP to be sufficient, then any version should do.
You might also try relocatable virtualenvs: setup it and copy the folder to your host. Anyway, note that this feature is still experimental.
How can I put my Go binary into a Debian package? Since Go is statically linked, I just have a single executable--I don't need a lot of complicated project metadata information. Is there a simple way to package the executable and resource files without going through the trauma of debuild?
I've looked all over for existing questions; however, all of my research turns up questions/answers about a .deb file containing the golang development environment (i.e., what you would get if you do sudo apt-get install golang-go).
Well. I think the only "trauma" of debuild is that it runs lintian after building the package, and it's lintian who tries to spot problems with your package.
So there are two ways to combat the situation:
Do not use debuild: this tool merely calls dpkg-buildpackage which really does the necessary powerlifting. The usual call to build a binary package is dpkg-buildpackage -us -uc -b. You still might call debuild for other purposes, like debuild clean for instance.
Add the so-called "lintian override" which can be used to make lintian turn a blind eye to selected problems with your package which, you insist, are not problems.
Both approaches imply that you do not attempt to build your application by the packaging tools but rather treat it as a blob which is just wrapped to a package. This would require slightly abstraining from the normal way debian/rules work (to not attempt to build anything).
Another solution which might be possible (and is really way more Debian-ish) is to try to use gcc-go (plus gold for linking): since it's a GCC front-end, this tool produces a dynamically-linked application (which links against libgo or something like this). I, personally, have no experience with it yet, and would only consider using it if you intend to try to push your package into the Debian proper.
Regarding the general question of packaging Go programs for Debian, you might find the following resources useful:
This thread started on go-nuts by one of Go for Debian packagers.
In particular, the first post in that thread links to this discussion on debian-devel.
The second thread on debian-devel regarding that same problem (it's a logical continuation of the former thread).
Update on 2015-10-15.
(Since this post appears to still be searched and found and studied by people I've decided to update it to better reflec the current state of affairs.)
Since then the situation with packaging Go apps and packages got improved dramatically, and it's possible to build a Debian package using "classic" Go (the so-called gc suite originating from Google) rather than gcc-go.
And there exist a good infrastructure for packages as well.
The key tool to use when debianizing a Go program now is dh-golang described here.
I've just been looking into this myself, and I'm basically there.
Synopsis
By 'borrowing' from the 'package' branch from one of Canonical's existing Go projects, you can build your package with dpkg-buildpackage.
install dependencies and grab a 'package' branch from another repo.
# I think this list of packages is enough. May need dpkg-dev aswell.
sudo apt-get install bzr debhelper build-essential golang-go
bzr branch lp:~niemeyer/cobzr/package mypackage-build
cd mypackage-build
Edit the metadata.
edit debian/control file (name, version, source). You may need to change the golang-stable dependency to golang-go.
The debian/control file is the manifest. Note the 'build dependencies' (Build-Depends: debhelper (>= 7.0.50~), golang-stable) and the 3 architectures. Using Ubuntu (without the gophers ppa), I had to change golang-stable to golang-go.
edit debian/rules file (put your package name in place of cobzr).
The debian/rules file is basically a 'make' file, and it shows how the package is built. In this case they are relying heavily on debhelper. Here they set up GOPATH, and invoke 'go install'.
Here's the magic 'go install' line:
cd $(GOPATH)/src && find * -name '*.go' -exec dirname {} \; | xargs -n1 go install
Also update the copyright file, readme, licence, etc.
Put your source inside the src folder. e.g.
git clone https://github.com/yourgithubusername/yourpackagename src/github.com/yourgithubusername/yourpackagename
or e.g.2
cp .../yourpackage/ src/
build the package
# -us -uc skips package signing.
dpkg-buildpackage -us -uc
This should produce a binary .deb file for your architecture, plus the 'source deb' (.tgz) and the source deb description file (.dsc).
More details
So, I realised that Canonical (the Ubuntu people) are using Go, and building .deb packages for some of their Go projects. Ubuntu is based on Debian, so for the most part the same approach should apply to both distributions (dependency names may vary slightly).
You'll find a few Go-based packages in Ubuntu's Launchpad repositories. So far I've found cobzr (git-style branching for bzr) and juju-core (a devops project, being ported from Python).
Both of these projects have both a 'trunk' and a 'package' branch, and you can see the debian/ folder inside the package branch. The 2 most important files here are debian/control and debian/rules - I have linked to 'browse source'.
Finally
Something I haven't covered is cross-compiling your package (to the other 2 architectures of the 3, 386/arm/amd64). Cross-compiling isn't too tricky in go (you need to build the toolchain for each target platform, and then set some ENV vars during 'go build'), and I've been working on a cross-compiler utility myself. Eventually I'll hopefully add .deb support into my utility, but first I need to crystallize this task.
Good luck. If you make any progress then please update my answer or add a comment. Thanks
Building deb or rpm packages from Go Applications is also very easy with fpm.
Grab it from rubygems:
gem install fpm
After building you binary, e.g. foobar, you can package it like this:
fpm -s dir -t deb -n foobar -v 0.0.1 foobar=/usr/bin/
fpm supports all sorts of advanced packaging options.
There is an official Debian policy document describing the packaging procedure for Go: https://go-team.pages.debian.net/packaging.html
For libraries: Use dh-make-golang to create a package skeleton. Name your package with a name derived from import path, with a -dev suffix, e.g. golang-github-lib-pq-dev. Specify the dependencies ont Depends: line. (These are source dependencies for building, not binary dependencies for running, since Go statically links all source.)
Installing the library package will install its source code to /usr/share/golang/src (possibly, the compiled libraries could go into .../pkg). Building depending Go packages will use the artifacts from those system-wide locations.
For executables: Use dh-golang to create the package. Specify dependencies in Build-Depends: line (see above regarding packaging the dependencies).
I recently discovered https://packager.io/ - I'm quite happy with what they're doing. Maybe open up one of the packages to see what they're doing?