How to build SO library for ARM architecture - go

I need to build a shared library in Go. For this purpose, I used CGO and then built SO lib with options
go build -o libUtil.so -buildmode=c-shared main.go
Now, I need to do the same, but for ARM architecture. When I do not use CGO, I only do export GOARCH=arm
and this is enough to succeed. However, when I use CGO, I can not build SO library.
I suspect, that I need to install arm build tools, but I don`t know how to do that and how to configure my GO environment to use these tools. I hope, somebody might help me.
OS is Linux.

It may be different in you own *.nix system, but in common case your settings shoul de like below:
export CC=arm-linux-gnueabi-gcc //choose your arm compiler binary
export GOARCH=arm
export CGO_ENABLED=1
then you cab build:
go build -o yourLib.so -buildmode=c-shared main.go

Related

Cross-compiling for linux arm/7: clang error: argument unused during compilation '-marm'

What is the reason and how to solve it? Please..
cmd : CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 go build
output:
# runtime/cgo
clang: error: argument unused during compilation: '-marm' [-Werror,-Wunused-command-line-argument]
OS: macOS Big Sur
Golang verson: go1.17 darwin/amd64
Read the fine manual:
When cross-compiling, you must specify a C cross-compiler for cgo to use. You can do this by setting the generic CC_FOR_TARGET or the more specific CC_FOR_${GOOS}_${GOARCH} (for example, CC_FOR_linux_arm) environment variable when building the toolchain using make.bash, or you can set the CC environment variable any time you run the go tool.
So you need to specify the cross-compiler for CGo, e.g. like so:
 CC=arm-linux-gnueabihf-gcc CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 go build
For C++ code also add CXX=arm-linux-gnueabihf-g++.
You'll need to have gcc-arm-linux-gnueabihf and libc6-dev-armhf-cross packages installed for the above to work (on Linux, don't know about Mac).

Compile Go for windows/arm and arm64 with buildmode=c-shared

I need to compilet a go library via C to a DLL that can be used through PInvoke on ARM/ARM64 Windows. I find many open issues, topics and discussions about this and it sounds like it might partially work. But if I try it like this:
export CC="arm-none-eabi-gcc"
export CXX="arm-none-eabi-g++"
export GOOS="windows"
export GOARCH="arm"
export GOARM=7
export CGO_ENABLED="1"
go build -ldflags="-s -w" -o my_library.dll -buildmode c-shared
I get the result buildmode c-shares is not supported on windows/arm. So it seems to still be not supported.
Another problem is that I need to set CGO_ENABLED and route the compiling through a C/C++-Toolchain as I have to add a C-file generated by SWIG. I tried the above on Ubunu 20.04 with the toolchain of the package gcc-arm-none-eabi.
I'm no C/C++/Go-pro - but the same works for nearly all other platforms like Windows, Linux, Android, Mac and iOS. The latter also is based on ARM64, so I do not really understand why this is not possible - though I value the difficulties with all this.
So, if someone with more in-depth-knowledge can help me here that would be great.
Just to clarifiy: I do not want/need to compile Go itself for ARM/ARM64. I need to compile a Go-program for that platform (to use my library from .Net on e.g. the Surface or a Hololens).
Update from 04.08.2021:
Go 1.17rc2 should include windows arm64 now. And I got the hint to use Zig for cross-compiling. So I've changed my build pipeline to something like this (I'm using Azure Devops in a Ubuntu VM):
go get -v golang.org/dl/go1.17rc2
/home/vsts/go/bin/go1.17rc2 download
/home/vsts/go/bin/go1.17rc2 version
sudo snap install zig --classic --beta
zig version
export CC="zig cc -target aarch64-windows-gnu"
export CXX="zig c++ -target aarch64-windows-gnu"
export GOOS="windows"
export GOARCH="arm64"
export GOARM=7
export CGO_ENABLED="1"
/home/vsts/go/bin/go1.17rc2 build -ldflags="-s -w" -o storj_uplink.dll -buildmode c-shared -tags extended
I then get this error:
2021-08-03T19:24:52.0641737Z # runtime/cgo
2021-08-03T19:24:52.0642803Z info: Usage: zig [command] [options]
2021-08-03T19:24:52.0643335Z
2021-08-03T19:24:52.0643827Z Commands:
2021-08-03T19:24:52.0643940Z
2021-08-03T19:24:52.0644276Z build Build project from build.zig
2021-08-03T19:24:52.0645203Z init-exe Initialize a `zig build` application in the cwd
2021-08-03T19:24:52.0645768Z init-lib Initialize a `zig build` library in the cwd
2021-08-03T19:24:52.0645950Z
2021-08-03T19:24:52.0646407Z ast-check Look for simple compile errors in any set of files
2021-08-03T19:24:52.0646936Z build-exe Create executable from source or object files
2021-08-03T19:24:52.0647468Z build-lib Create library from source or object files
2021-08-03T19:24:52.0647994Z build-obj Create object from source or object files
2021-08-03T19:24:52.0648390Z fmt Reformat Zig source into canonical form
2021-08-03T19:24:52.0648753Z run Create executable and run immediately
2021-08-03T19:24:52.0649088Z test Create and run a test build
2021-08-03T19:24:52.0649551Z translate-c Convert C code to Zig code
2021-08-03T19:24:52.0649707Z
2021-08-03T19:24:52.0650109Z ar Use Zig as a drop-in archiver
2021-08-03T19:24:52.0650576Z cc Use Zig as a drop-in C compiler
2021-08-03T19:24:52.0651070Z c++ Use Zig as a drop-in C++ compiler
2021-08-03T19:24:52.0651549Z dlltool Use Zig as a drop-in dlltool.exe
2021-08-03T19:24:52.0652033Z lib Use Zig as a drop-in lib.exe
2021-08-03T19:24:52.0652495Z ranlib Use Zig as a drop-in ranlib
2021-08-03T19:24:52.0652670Z
2021-08-03T19:24:52.0652962Z env Print lib path, std path, cache directory, and version
2021-08-03T19:24:52.0653531Z help Print this help and exit
2021-08-03T19:24:52.0653879Z libc Display native libc paths file or validate one
2021-08-03T19:24:52.0654250Z targets List available compilation targets
2021-08-03T19:24:52.0654579Z version Print version number and exit
2021-08-03T19:24:52.0655062Z zen Print Zen of Zig and exit
2021-08-03T19:24:52.0655220Z
2021-08-03T19:24:52.0655445Z General Options:
2021-08-03T19:24:52.0655565Z
2021-08-03T19:24:52.0655982Z -h, --help Print command-specific usage
2021-08-03T19:24:52.0656154Z
2021-08-03T19:24:52.0656502Z error: unknown command: -E
2021-08-03T19:25:03.2047129Z # golang.org/x/sys/windows
2021-08-03T19:25:03.2048568Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/types_windows.go:1620:24: undefined: JOBOBJECT_BASIC_LIMIT_INFORMATION
2021-08-03T19:25:03.2049594Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3020:38: undefined: WSAData
2021-08-03T19:25:03.2050606Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3096:51: undefined: Servent
2021-08-03T19:25:03.2051572Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3110:50: undefined: Servent
2021-08-03T19:25:04.7947309Z ##[error]Bash exited with code '1'.
Basically "unknown command: -E" which is described here. But from my understanding this should work already. And furthermore this blog post does it directly with zig, too.
Second update from 04.08.2021
Go is now calling zig! The workaround using a bash-script is working. Now I get the following error:
2021-08-04T11:54:47.2530981Z # golang.org/x/sys/windows
2021-08-04T11:54:47.2532284Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/types_windows.go:1620:24: undefined: JOBOBJECT_BASIC_LIMIT_INFORMATION
2021-08-04T11:54:47.2533180Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3020:38: undefined: WSAData
2021-08-04T11:54:47.2534002Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3096:51: undefined: Servent
2021-08-04T11:54:47.2534797Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3110:50: undefined: Servent
2021-08-04T11:54:57.4223210Z # runtime/cgo
2021-08-04T11:54:57.4224911Z /snap/zig/3678/lib/libc/mingw/secapi/vsprintf_s.c:39:10: warning: implicit declaration of function '__ms_vsnprintf' is invalid in C99 [-Wimplicit-function-declaration]
2021-08-04T11:54:57.4225714Z return __ms_vsnprintf (_DstBuf, _Size, _Format, _ArgList);
2021-08-04T11:54:57.4226223Z ^
2021-08-04T11:54:57.4226624Z 1 warning generated.
2021-08-04T11:54:57.4227534Z /snap/zig/3678/lib/libc/mingw/math/arm/s_trunc.c/snap/zig/3678/lib/libc/mingw/math/arm/s_truncf.c:24:10: fatal error: '../bsd_private_base.h' file not found
2021-08-04T11:54:57.4228188Z #include "../bsd_private_base.h"
2021-08-04T11:54:57.4228651Z ^~~~~~~~~~~~~~~~~~~~~~~
2021-08-04T11:54:57.4229332Z :26:10: fatal error: '../bsd_private_base.h' file not found
2021-08-04T11:54:57.4229850Z #include "../bsd_private_base.h"
2021-08-04T11:54:57.4230310Z ^~~~~~~~~~~~~~~~~~~~~~~
2021-08-04T11:54:57.4230966Z 1 error generated.
2021-08-04T11:54:57.4231397Z 1 error generated.
2021-08-04T11:54:57.4522549Z ##[error]Bash exited with code '1'.
Update from 05.08.2021:
I finally found a toolchain that at least does not throw an error. But now it quits silently without generating a DLL. Not sure what happens now, though. This is my call:
go get -v golang.org/dl/go1.17rc2
/home/vsts/go/bin/go1.17rc2 download
/home/vsts/go/bin/go1.17rc2 version
wget https://developer.arm.com/-/media/Files/downloads/gnu-a/10.3-2021.07/binrel/gcc-arm-10.3-2021.07-x86_64-arm-none-linux-gnueabihf.tar.xz
tar -xf gcc-arm-10.3-2021.07-x86_64-arm-none-linux-gnueabihf.tar.xz
cd gcc-arm-10.3-2021.07-x86_64-arm-none-linux-gnueabihf
cd bin
export PATH=$PATH:$(pwd)
cd ..
cd ..
cd uplink-c
export CC="arm-none-linux-gnueabihf-gcc -v"
export CXX="arm-none-linux-gnueabihf-g++ -v"
export GOOS="windows"
export GOARCH="arm64"
export GOARM=7
export CGO_ENABLED="1"
home/vsts/go/bin/go1.17rc2 build -ldflags="-s -w" -o storj_uplink.dll -buildmode c-shared -tags extended -v
I do not want/need to compile Go itself for ARM/ARM64. I need to compile a Go-program for that platform
That should work, using Go 1.17 beta
Its documentation do mention:
Windows
Go 1.17 adds support of 64-bit ARM architecture on Windows (the windows/arm64 port).
This port supports cgo.
The OP topperdel refers in the comments to CL 326310:
cmd/internal/sys: mark windows/arm64 as c-shared-capable
The platform supports c-shared now, so flip this on.
I've given this a small smoke test using WireGuard Tunnel Library, and it was able to pass packets and generally function well.
Since the WireGuard Tunnel Library uses quite a bit of Go functionality under the hood, I think it's a decent test that a lot of things that should be working are working. So this commit enables it.
In order to get all tests passing, we make a few small changes, such as
passing -Wno-dll-attribute-on-redeclaration to clang and avoiding
loading shared libraries into Powershell on arm.
As illustrated by those issues, this is this is still a work in progress:
issue 46502 ("runtime: several tests are failing on windows-arm64-aws builder due to redeclaration warnings (upgraded to errors in testing)"),
issue 46701 ("Powershell on arm64/arm cannot load arm64/arm binaries because it is an intel process")
The test part is now (June 13th, 2021) closed with golang/go commit 1ed0d12:
runtime: testprogcgo: don't call exported Go functions directly from Go
Instead route through a C function, to avoid declaration conflicts
between the declaration needed in the cgo comment and the declaration
generated by cgo in _cgo_export.h.
This is not something user code will ever do, so no need to make it
work in cgo.
I mentioned in the comments
From ziglang/zig issue 7342, zig should be supported now.
Example, with Go 1.17 : "Zig Makes Go Cross Compilation Just Work" from Loris Cro (VP of community #ziglang):
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 \
CC="zig cc -target x86_64-linux" \
CXX="zig c++ -target x86_64-linux" \
go build --tags extended
You would need to adapt the arch targets to your need
Note: a bash shell session is needed. So git bash on Windows, for instance.

Is it possible to do configure-make-make install?

I have some Go source files: one.go, two.go,main.go
I build them to C static library for using in my cross-platform application compilation.
There are 4 types of application compilation:
Compilation on Windows
Compilation on Ubuntu
Compilation on Mac
Cross-compilation for Windows from Ubuntu
For 1,2,3 I use:
go build -buildmode c-archive -o libxyz.a .
For 4, I use:
GOOS="windows" GOARCH="amd64" CGO_ENABLED="1" CXX="x86_64-w64-mingw32-g++" CC="x86_64-w64-mingw32-gcc" go build -buildmode c-archive -o libxyz.a .
Is it somehow possible to do that by the configure-make-make install procedure?
(There is no configure.ac file in the xyz Go source package)
Also, it possible to use pkg-config in that?
Is it somehow possible to do that by the configure-make-make install procedure [?]
Yes.
Should you do it, is this easy, is this sensible, does this provide any benefit? No, no, no, no.
If you want to type less: Write a small shell script (or a tiny Makefile if you insist on using make for whatever reason).

Flags needed to create static binaries in golang

I was building a Docker image using scratch as base.
The following build command:
RUN go build -o /go/bin/myapp
created a binary that kept failing when executed:
standard_init_linux.go:211: exec user process caused "no such file or directory"
By trial and error I found out that I needed to build as follows:
RUN CGO_ENABLED=0 go build -o /go/bin/myapp -a -ldflags '-extldflags "-static"' .
Why are both CGO_ENABLED=0 and -ldflags '-extldflags "-static"' necessary?
Don't both options create static binaries? (i.e. binaries that will need absolutely nothing in terms of libraries from the environments they will run on?)
Just before we start, a heads up by Russ Cox's: Comment
Read this as well: Comment
Yes, I agree with Volker's comment that some systems don't really allow static binaries.
Read on: Compile packages and dependencies
-a
force rebuilding of packages that are already up-to-date.
-ldflags '[pattern=]arg list'
arguments to pass on each go tool link invocation.
Read: go tool link
-extldflags flags
Set space-separated flags to pass to the external linker.
Hence, it tries to rebuild all the packages (dependencies as well) with CGO disabled and also -static means do not link against shared libraries.
Some of the points related to static linking are explained well: Linking golang statically

How are GOARCH and GOOS used during go toolchain compilation?

I am trying to reconcile why make.bash accepts GOARCH, GOOS and other target variables when it also supports cross compiling to all supported architectures by default. I am compiling this for amd64/linux but will be using the compiler to cross-compile apps for arm/linux in an embedded context. Buildroot builds Go with the following command
cd /builddir/build/host-go-1.10.2/src &&
GOROOT_BOOTSTRAP=/builddir/host/lib/go-1.4.3
GOROOT_FINAL=/builddir/host/lib/go
GOROOT="/builddir/build/host-go-1.10.2"
GOBIN="/builddir/build/host-go-1.10.2/bin"
GOARCH=arm
GOARM=7
GOOS=linux
CC=/usr/bin/gcc
CXX=/usr/bin/g++
GO_ASSUME_CROSSCOMPILING=1
CC_FOR_TARGET="/builddir/host/bin/arm-linux-gnueabihf-gcc"
CXX_FOR_TARGET="/builddir/host/bin/arm-linux-gnueabihf-g++"
CGO_ENABLED=1
./make.bash
Which works and goes on to build other apps just fine. My question is why are these target variables relevant at this stage? Wouldn't that only be relevant to the applications being compiled with this program?
The Go compiler and linker are Go programs. make.bash needs to know which architecture and which operating system to build the Go toolchain for.
See Downloads - The Go Programming Language for examples of the different architectures and operating systems.
See Installing Go from source.

Resources