Compile go program with C dependency for Windows - go

Building go works fine for pure go project with pure go dependencies. But when building a project with a C dependency, it fails on Windows:
go build -a -o bin/xyz.exe ./xyz/main.go
go: downloading gopkg.in/confluentinc/confluent-kafka-go.v1 v1.4.2
go: downloading github.com/confluentinc/confluent-kafka-go v1.4.2
# gopkg.in/confluentinc/confluent-kafka-go.v1/kafka
In file included from C:\Users\VssAdministrator\go\pkg\mod\gopkg.in\confluentinc\confluent-kafka-go.v1#v1.4.2\kafka\00version.go:24:
./librdkafka/rdkafka.h:83:10: fatal error: sys/socket.h: No such file or directory
#include <sys/socket.h> /* for sockaddr, .. */
^~~~~~~~~~~~~~
compilation terminated.
mingw32-make: *** [Makefile:10: build-windows] Error 2
##[error]Cmd.exe exited with code '2'.
Finishing: CmdLine
As can be seen from the output above, I'm using a Makefile, and my azure-pipelines.yml looks like this:
...
- script: 'make package-windows'
...
Here's my Makefile:
build-windows:
go build -a -o bin/xyz.exe ./xyz/main.go
I also tried setting GOOS and GOARCH, to no avail:
build-windows:
GOOS=windows GOARCH=amd64 go build -a -o bin/xyz.exe ./xyz/main.go
So how can I build this?

This doesn't have much to do with Go - it seems you're doing everything correctly in that regard. The issue is that the library you're using needs sys/socket.h which simply doesn't exist on Windows (see Using sys/socket.h functions on windows).
Your options are similar to what's mentioned in the other answer:
Try to build using Cygwin.
Modify the library to use Winsock instead of sys/socket.h.
Find a different library, one that supports Windows (the author of the library you're using specifically said Windows is not supported).

Related

Compile Go for windows/arm and arm64 with buildmode=c-shared

I need to compilet a go library via C to a DLL that can be used through PInvoke on ARM/ARM64 Windows. I find many open issues, topics and discussions about this and it sounds like it might partially work. But if I try it like this:
export CC="arm-none-eabi-gcc"
export CXX="arm-none-eabi-g++"
export GOOS="windows"
export GOARCH="arm"
export GOARM=7
export CGO_ENABLED="1"
go build -ldflags="-s -w" -o my_library.dll -buildmode c-shared
I get the result buildmode c-shares is not supported on windows/arm. So it seems to still be not supported.
Another problem is that I need to set CGO_ENABLED and route the compiling through a C/C++-Toolchain as I have to add a C-file generated by SWIG. I tried the above on Ubunu 20.04 with the toolchain of the package gcc-arm-none-eabi.
I'm no C/C++/Go-pro - but the same works for nearly all other platforms like Windows, Linux, Android, Mac and iOS. The latter also is based on ARM64, so I do not really understand why this is not possible - though I value the difficulties with all this.
So, if someone with more in-depth-knowledge can help me here that would be great.
Just to clarifiy: I do not want/need to compile Go itself for ARM/ARM64. I need to compile a Go-program for that platform (to use my library from .Net on e.g. the Surface or a Hololens).
Update from 04.08.2021:
Go 1.17rc2 should include windows arm64 now. And I got the hint to use Zig for cross-compiling. So I've changed my build pipeline to something like this (I'm using Azure Devops in a Ubuntu VM):
go get -v golang.org/dl/go1.17rc2
/home/vsts/go/bin/go1.17rc2 download
/home/vsts/go/bin/go1.17rc2 version
sudo snap install zig --classic --beta
zig version
export CC="zig cc -target aarch64-windows-gnu"
export CXX="zig c++ -target aarch64-windows-gnu"
export GOOS="windows"
export GOARCH="arm64"
export GOARM=7
export CGO_ENABLED="1"
/home/vsts/go/bin/go1.17rc2 build -ldflags="-s -w" -o storj_uplink.dll -buildmode c-shared -tags extended
I then get this error:
2021-08-03T19:24:52.0641737Z # runtime/cgo
2021-08-03T19:24:52.0642803Z info: Usage: zig [command] [options]
2021-08-03T19:24:52.0643335Z
2021-08-03T19:24:52.0643827Z Commands:
2021-08-03T19:24:52.0643940Z
2021-08-03T19:24:52.0644276Z build Build project from build.zig
2021-08-03T19:24:52.0645203Z init-exe Initialize a `zig build` application in the cwd
2021-08-03T19:24:52.0645768Z init-lib Initialize a `zig build` library in the cwd
2021-08-03T19:24:52.0645950Z
2021-08-03T19:24:52.0646407Z ast-check Look for simple compile errors in any set of files
2021-08-03T19:24:52.0646936Z build-exe Create executable from source or object files
2021-08-03T19:24:52.0647468Z build-lib Create library from source or object files
2021-08-03T19:24:52.0647994Z build-obj Create object from source or object files
2021-08-03T19:24:52.0648390Z fmt Reformat Zig source into canonical form
2021-08-03T19:24:52.0648753Z run Create executable and run immediately
2021-08-03T19:24:52.0649088Z test Create and run a test build
2021-08-03T19:24:52.0649551Z translate-c Convert C code to Zig code
2021-08-03T19:24:52.0649707Z
2021-08-03T19:24:52.0650109Z ar Use Zig as a drop-in archiver
2021-08-03T19:24:52.0650576Z cc Use Zig as a drop-in C compiler
2021-08-03T19:24:52.0651070Z c++ Use Zig as a drop-in C++ compiler
2021-08-03T19:24:52.0651549Z dlltool Use Zig as a drop-in dlltool.exe
2021-08-03T19:24:52.0652033Z lib Use Zig as a drop-in lib.exe
2021-08-03T19:24:52.0652495Z ranlib Use Zig as a drop-in ranlib
2021-08-03T19:24:52.0652670Z
2021-08-03T19:24:52.0652962Z env Print lib path, std path, cache directory, and version
2021-08-03T19:24:52.0653531Z help Print this help and exit
2021-08-03T19:24:52.0653879Z libc Display native libc paths file or validate one
2021-08-03T19:24:52.0654250Z targets List available compilation targets
2021-08-03T19:24:52.0654579Z version Print version number and exit
2021-08-03T19:24:52.0655062Z zen Print Zen of Zig and exit
2021-08-03T19:24:52.0655220Z
2021-08-03T19:24:52.0655445Z General Options:
2021-08-03T19:24:52.0655565Z
2021-08-03T19:24:52.0655982Z -h, --help Print command-specific usage
2021-08-03T19:24:52.0656154Z
2021-08-03T19:24:52.0656502Z error: unknown command: -E
2021-08-03T19:25:03.2047129Z # golang.org/x/sys/windows
2021-08-03T19:25:03.2048568Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/types_windows.go:1620:24: undefined: JOBOBJECT_BASIC_LIMIT_INFORMATION
2021-08-03T19:25:03.2049594Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3020:38: undefined: WSAData
2021-08-03T19:25:03.2050606Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3096:51: undefined: Servent
2021-08-03T19:25:03.2051572Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3110:50: undefined: Servent
2021-08-03T19:25:04.7947309Z ##[error]Bash exited with code '1'.
Basically "unknown command: -E" which is described here. But from my understanding this should work already. And furthermore this blog post does it directly with zig, too.
Second update from 04.08.2021
Go is now calling zig! The workaround using a bash-script is working. Now I get the following error:
2021-08-04T11:54:47.2530981Z # golang.org/x/sys/windows
2021-08-04T11:54:47.2532284Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/types_windows.go:1620:24: undefined: JOBOBJECT_BASIC_LIMIT_INFORMATION
2021-08-04T11:54:47.2533180Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3020:38: undefined: WSAData
2021-08-04T11:54:47.2534002Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3096:51: undefined: Servent
2021-08-04T11:54:47.2534797Z /home/vsts/go/pkg/mod/golang.org/x/sys#v0.0.0-20210112091331-59c308dcf3cc/windows/zsyscall_windows.go:3110:50: undefined: Servent
2021-08-04T11:54:57.4223210Z # runtime/cgo
2021-08-04T11:54:57.4224911Z /snap/zig/3678/lib/libc/mingw/secapi/vsprintf_s.c:39:10: warning: implicit declaration of function '__ms_vsnprintf' is invalid in C99 [-Wimplicit-function-declaration]
2021-08-04T11:54:57.4225714Z return __ms_vsnprintf (_DstBuf, _Size, _Format, _ArgList);
2021-08-04T11:54:57.4226223Z ^
2021-08-04T11:54:57.4226624Z 1 warning generated.
2021-08-04T11:54:57.4227534Z /snap/zig/3678/lib/libc/mingw/math/arm/s_trunc.c/snap/zig/3678/lib/libc/mingw/math/arm/s_truncf.c:24:10: fatal error: '../bsd_private_base.h' file not found
2021-08-04T11:54:57.4228188Z #include "../bsd_private_base.h"
2021-08-04T11:54:57.4228651Z ^~~~~~~~~~~~~~~~~~~~~~~
2021-08-04T11:54:57.4229332Z :26:10: fatal error: '../bsd_private_base.h' file not found
2021-08-04T11:54:57.4229850Z #include "../bsd_private_base.h"
2021-08-04T11:54:57.4230310Z ^~~~~~~~~~~~~~~~~~~~~~~
2021-08-04T11:54:57.4230966Z 1 error generated.
2021-08-04T11:54:57.4231397Z 1 error generated.
2021-08-04T11:54:57.4522549Z ##[error]Bash exited with code '1'.
Update from 05.08.2021:
I finally found a toolchain that at least does not throw an error. But now it quits silently without generating a DLL. Not sure what happens now, though. This is my call:
go get -v golang.org/dl/go1.17rc2
/home/vsts/go/bin/go1.17rc2 download
/home/vsts/go/bin/go1.17rc2 version
wget https://developer.arm.com/-/media/Files/downloads/gnu-a/10.3-2021.07/binrel/gcc-arm-10.3-2021.07-x86_64-arm-none-linux-gnueabihf.tar.xz
tar -xf gcc-arm-10.3-2021.07-x86_64-arm-none-linux-gnueabihf.tar.xz
cd gcc-arm-10.3-2021.07-x86_64-arm-none-linux-gnueabihf
cd bin
export PATH=$PATH:$(pwd)
cd ..
cd ..
cd uplink-c
export CC="arm-none-linux-gnueabihf-gcc -v"
export CXX="arm-none-linux-gnueabihf-g++ -v"
export GOOS="windows"
export GOARCH="arm64"
export GOARM=7
export CGO_ENABLED="1"
home/vsts/go/bin/go1.17rc2 build -ldflags="-s -w" -o storj_uplink.dll -buildmode c-shared -tags extended -v
I do not want/need to compile Go itself for ARM/ARM64. I need to compile a Go-program for that platform
That should work, using Go 1.17 beta
Its documentation do mention:
Windows
Go 1.17 adds support of 64-bit ARM architecture on Windows (the windows/arm64 port).
This port supports cgo.
The OP topperdel refers in the comments to CL 326310:
cmd/internal/sys: mark windows/arm64 as c-shared-capable
The platform supports c-shared now, so flip this on.
I've given this a small smoke test using WireGuard Tunnel Library, and it was able to pass packets and generally function well.
Since the WireGuard Tunnel Library uses quite a bit of Go functionality under the hood, I think it's a decent test that a lot of things that should be working are working. So this commit enables it.
In order to get all tests passing, we make a few small changes, such as
passing -Wno-dll-attribute-on-redeclaration to clang and avoiding
loading shared libraries into Powershell on arm.
As illustrated by those issues, this is this is still a work in progress:
issue 46502 ("runtime: several tests are failing on windows-arm64-aws builder due to redeclaration warnings (upgraded to errors in testing)"),
issue 46701 ("Powershell on arm64/arm cannot load arm64/arm binaries because it is an intel process")
The test part is now (June 13th, 2021) closed with golang/go commit 1ed0d12:
runtime: testprogcgo: don't call exported Go functions directly from Go
Instead route through a C function, to avoid declaration conflicts
between the declaration needed in the cgo comment and the declaration
generated by cgo in _cgo_export.h.
This is not something user code will ever do, so no need to make it
work in cgo.
I mentioned in the comments
From ziglang/zig issue 7342, zig should be supported now.
Example, with Go 1.17 : "Zig Makes Go Cross Compilation Just Work" from Loris Cro (VP of community #ziglang):
CGO_ENABLED=1 GOOS=linux GOARCH=amd64 \
CC="zig cc -target x86_64-linux" \
CXX="zig c++ -target x86_64-linux" \
go build --tags extended
You would need to adapt the arch targets to your need
Note: a bash shell session is needed. So git bash on Windows, for instance.

CMake: The C Compiler is not able to compile a simple test program

I am trying to cross-compile the Azure IoT SDK C for a Mips processor. Cross-compiling an older version of the same SDK using an older version of CMake (2.8.12.2) works just fine, so I doubt it's the code itself. I am guessing it's the Mips GCC compiler.
Error message:
CMake Error at /usr/share/cmake-3.10/Modules/CMakeTestCCompiler.cmake:52 (message):
The C compiler
"/usr/local/mipsisa32r2el/r23/bin/mipsisa32r2el-axis-linux-gnu-gcc"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /home/axis/azure-iot-sdk-c/cmake/iotsdk_linux/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/make" "cmTC_2cc84/fast"
/usr/bin/make -f CMakeFiles/cmTC_2cc84.dir/build.make CMakeFiles/cmTC_2cc84.dir/build
make[1]: Entering directory '/home/axis/azure-iot-sdk-c/cmake/iotsdk_linux/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_2cc84.dir/testCCompiler.c.o
/usr/local/mipsisa32r2el/r23/bin/mipsisa32r2el-axis-linux-gnu-gcc --sysroot=/usr/local/mipsisa32r2el/r23 -o CMakeFiles/cmTC_2cc84.dir/testCCompiler.c.o -c /home/axis/azure-iot-sdk-c/cmake/iotsdk_linux/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_2cc84
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2cc84.dir/link.txt --verbose=1
/usr/local/mipsisa32r2el/r23/bin/mipsisa32r2el-axis-linux-gnu-gcc --sysroot=/usr/local/mipsisa32r2el/r23 -rdynamic CMakeFiles/cmTC_2cc84.dir/testCCompiler.c.o -o cmTC_2cc84
/usr/local/mipsisa32r2el/r23/lib/gcc/mipsisa32r2el-axis-linux-gnu/4.7.2/../../../../mipsisa32r2el-axis-linux-gnu/bin/ld: this linker was not configured to use sysroots
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_2cc84.dir/build.make:97: recipe for target 'cmTC_2cc84' failed
make[1]: *** [cmTC_2cc84] Error 1
make[1]: Leaving directory '/home/axis/azure-iot-sdk-c/cmake/iotsdk_linux/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_2cc84/fast' failed
make: *** [cmTC_2cc84/fast] Error 2
Unfortunately, I am stuck with the Mips GCC compiler I have. Is there a way to disable this test-program check?
Solution was to add these to the toolchain-file:
SET (CMAKE_C_COMPILER_WORKS 1)
SET (CMAKE_CXX_COMPILER_WORKS 1)
CMake tries to compile an executable using "standard" (as per what CMake thinks is standard) compiler options and tries to run that executable, so to see if the compiler is working. The executable is simple like int main(int argc, char *argv[]) { return argc - 1; }.
You can't do that when cross-compiling. Because usually you can't link with a proper C standard library, you don't have printf, or _start or _exit or similar, passing arguments to main is implementation-defined, or you need a special linker script, or there's no emulator for your architecture, so can't run cross-compiled source on the host, etc... Simply: you usually can't run the cross-compiled executable on the host, and most of the time even the compilation is hard enough to do.
The common solution is to set before project():
set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")
So that CMake will try to compile a static library not an executable, as explained in cmake docs CMAKE_TRY_COMPILE_TARGET_TYPE. This avoids running the linker and is intended for cross-compiling.
You can set CMAKE_C_COMPILER_WORKS and it will omit the check in CMakeTestCCompiler.cmake, but CMAKE_TRY_COMPILE_TARGET_TYPE is a more proper solution.
Well this problem is really annoying, i faced this issue for 2 day now I got solution.
Let me explain my issue first
When I delete NDK and Cmake from Sdk folder and Run my application then grable install NDK and Cmake again.
Only that time Application run and when try to run again i get this error The C Compiler is not able to compile a simple test program.
Before i was using ndkVersion "22.0.7026061" then change to thisndkVersion "21.1.6352462" and IT WORKED.
I think this is NDK problem and most of answer are outdated try this i hope this will HELP.
If using CMake GUI, you can add a boolean entry named
CMAKE_CXX_COMPILER_FORCED
then set it to True.
This will skip checking process for this build.
Ran into the same problem while re-compiling a project. Turned out that the compiler had been updated in the meantime.
After removing the build directory and creating it again, the compilation completed without errors.

Vala examples compile errors

Installed Vala for Windows64 from the MSYS2 project. Started running various examples from https://wiki.gnome.org/Projects/Vala/Examples. Some examples produce errors like
... the "Clutter Samples:Animated Actors"
D:\Projects\Vala\Examples>valac.exe --version
Vala 0.36.1-dirty
D:\Projects\Vala\Examples>valac.exe --pkg clutter-1.0 clutter.vala -o clutter.exe
D:/Projects/Vala/Examples/clutter.vala.c:7:29: fatal error: clutter/clutter.h: No such file or direc
tory
#include <clutter/clutter.h>
^
compilation terminated.
error: cc exited with status 1
Compilation failed: 1 error(s), 0 warning(s)
... and the "Vala GTK+ 3.x Examples: Basic Sample"
D:\Projects\Vala\Examples>valac.exe --version
Vala 0.36.1-dirty
D:\Projects\Vala\Examples>valac.exe --pkg gtk+-3.0 gtk+3-hello.vala -o gtk+3-hello.exe
D:/Projects/Vala/Examples/gtk+3-hello.vala.c:17:21: fatal error: gtk/gtk.h: No such file or director
y
^
compilation terminated.
error: cc exited with status 1
Compilation failed: 1 error(s), 0 warning(s)
...and the Vala Collections: libgee sample
D:\Projects\Vala\Examples>valac.exe --version
Vala 0.36.1-dirty
D:\Projects\Vala\Examples>valac.exe --pkg=gee-0.8 libgee-collections.vala -o libgee-collections.exe
error: Package `gee-0.8' not found in specified Vala API directories or GObject-Introspection GIR di
rectories
Compilation failed: 1 error(s), 0 warning(s)
I realize pkg:gee-0.8 is not installed, but I did find 0.20.0-1 using pacman. I dont know how to reference this library?
$ pacman -Ss gee -v
Root : /
Conf File : /etc/pacman.conf
DB Path : /var/lib/pacman/
Cache Dirs: /var/cache/pacman/pkg/
Hook Dirs : /usr/share/libalpm/hooks/ /etc/pacman.d/hooks/
Lock File : /var/lib/pacman/db.lck
Log File : /var/log/pacman.log
GPG Dir : /etc/pacman.d/gnupg/
Targets : gee
mingw32/mingw-w64-i686-libgee 0.20.0-1
A collection library providing GObject-based interfaces and classes for
commonly used data structures (mingw-w64)
mingw64/mingw-w64-x86_64-libgee 0.20.0-1
A collection library providing GObject-based interfaces and classes for
commonly used data structures (mingw-w64)
Obviously my installation is incorrect/incomplete. What do I need to change or install to fix these type of issues?
Thanks for your help!
Running valac.exe directly can be a problem.
Usually when using msys2 to compile something you first enter the msys2 shell with the MSYSTEM environment variable set up correctly. (there are scripts and in newer versions executables in the msys2 root folder that can do that for you).
For example when you set MSYSTEM=MINGW64 (caveat: MINGW64 must be upper case!) and then run mingtty.exe or bash.exe --login you can then check your PKG_CONFIG_PATH var:
$ echo $PKG_CONFIG_PATH
/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig
Be sure to understand the difference of the three MSYSTEM settings:
MSYS - Only used to bootstrap the core msys2 system and for packaging purposes
MINGW32 / MINGW64 - The "normal" environment (32-Bit / 64-Bit) you work in when using msys2.
The pacman package names for the different MSYSTEM environment have prefixes. I.e. the packages that start with mingw-w64-x86_64 are used in the MSYSTEM=MINGW64 environment.
vala uses pkg-config (be sure to have mingw64/mingw-w64-x86_64-pkg-config installed) when you specify something with --pkg and pkg-config then searches for .pc files (like gee-0.8.pc).
The mingw-w64-x86_64-libgee package installs its pc file as /mingw64/lib/pkgconfig/gee-0.8.pc.
With gee-0.8 the 0.8 is the interface version number, not the release version number. So install release 0.20.0 and the relevant interface files should become available. That also applies to Clutter and GTK+3.

cannot implicitly include runtime/cgo in a shared library

Following my earlier question, now I'm getting this error when trying to perform all the same steps from this article, since I've upgraded from Go 1.6.1 to Go 1.7.1 (I cannot go back to Go 1.6.1 because the linker crashes when tries to compile some shared libraries).
What I did:
Installed go in ~/.go/go (this is later referred to as GOROOT).
Compiled libstd.so:
GOROOT=~/.go/go GOPATH=~/tests go install -buildmode=shared -linkshared std
Compiled calc library:
GOROOT=~/.go/go GOPATH=~/tests go install -a -x -buildmode=shared -linkshared calc
Tried to compile app:
GOROOT=~/.go/go GOPATH=~/tests go build -a -x -linkshared -o app cashier
and received this error:
~/.go/go/pkg/tool/linux_amd64/link: cannot implicitly include runtime/cgo in a shared library
I tried to repeat the steps with CGO_ENABLED=0 prepended to the environment, but nothing would build this way. Giving me this error:
imports runtime/cgo: C source files not allowed when not using cgo or SWIG: gcc_fatalf.c gcc_linux_amd64.c gcc_mmap.c gcc_util.c
Is this a known bug? Is making shared libraries meant to be supported?
The only way I've been able to make working DLLs is to use buildmode=c-archive and write C stubs for all the go functions I want to export. But I was working on Windows for this, I haven't had to mess with this on Linux. It is a path to investigate.

error adding symbols while compiling gdcm

I wanted to compile gdcm from source code on eOS 0.3/Ubuntu 14.04, and add python support. Therefore I installed swig, and afterwards ran ccmake (for configuring the make file) and make. Unfortunately I get the error:
../../bin/libgdcmMEXD.a: error adding symbols
How can I fix that?
I would try to build using shared libs instead. From the ccmake interface you should see something like
GDCM_BUILD_SHARED_LIBS ON
Then rebuild:
$ make clean && make

Resources