Ubuntu. vscode 1.62.1. go1.17.3. vscode go extension v0.29.0. delve v1.7.1.
I'm trying to a build a small app that uses Cgo, using vscode and vscode-go. Only one module imports "C".
My project structure has the root directory containing the "go.mod" and "main.go" files, and I have subpackages in subfolders. I also have "include" and "lib" directories that contain the C artifacts.
This is what I have so far in the C module:
package voltage
// #cgo CFLAGS: -g -Wall -Iinclude
// #cgo LDFLAGS: -Llib/linux -lvibesimple -lcurl -lssl -lvibecrypto -lvibeictk -lvibeserver
// #include <stdio.h>
// #include <errno.h>
// #include "veapi.h"
import "C"
func Encrypt(datatype string, data string) (result string) {
return
}
func Decrypt(datatype string, data string) (result string) {
return
}
In the "Problems" view, it shows the following two issues:
go list failed to return CompiledGoFiles. This may indicate failure to perform cgo processing; try building at the command line. See https://golang.org/issue/38990.
And:
could not import C (cgo preprocessing failed) (compile)
I read the cited issue, but I'm not sure what to do with that information.
I'm not sure how to move forward here.
The C compiler is not executed in the source directory but in a temporary directory only containing intermediate files, such as your go files compiled as static libraries (.a). Therefore, the LDFLAG -Llib/linux points to an unexisting directory.
To solve this issue, just replace that flag with -L${SRCDIR}/lib/linux.
Directly from the cgo docs:
When the cgo directives are parsed, any occurrence of the string ${SRCDIR} will be replaced by the absolute path to the directory containing the source file. This allows pre-compiled static libraries to be included in the package directory and linked properly.
The cgo tool will always invoke the C compiler with the source file's directory in the include path; i.e. -I${SRCDIR} is always implied.
Related
I have a really simple setup: A .go file (test.go) and a .c file (PMDK.c). I include the .c file in Go as follows:
test.go:
package main
/*
#include "PMDK.c"
#cgo pkg-config: libpmem
*/
import "C"
func main() {
C.helloWorld()
}
When I run go run test.go, it builds for exactly one time. Whatever changes I do to PMDK.c, my program has the exact same behavior every time.
I also tried go build test.go, which leads to the same result.
Finally , following CGo not compiling C files in same directory, I just did go build. This didn't work, since I had to create a .mod file (go build test.go). Then, the problem was that the three functions in PMDK.c (helloWorld and two others) were supposedly defined multiple times.
I can't make it build my changes.
By the way, if I copy/move the source files to another directory and build them there, the changes apply (only for once, again).
The heart of the problem is that your setup is wrong: your Go code should #include, in the Cgo prefix, only the headers for any C code you want compiled separately. For instance:
package main
/*
#include "pmdk.h"
*/
import "C"
func main() {
C.helloWorld()
}
You can put C code into the prefix:
package main
/*
#include <stdio.h>
void helloWorld() {
printf("hello world from C\n");
}
*/
import "C"
...
But if you put your C code into a separate file (prog.c, etc.), you should create a small header file that simply declares each function, and #include that header file from both the C code and the Go code.1
Running:
go build
will then compile the C code if it has changed, and build the Go code if it has changed, and link together the two as appropriate. If you #include the C code directly into the Go package, as you did, go build will both build the C code and build the Go code that includes the C code, which is why you get the duplicate definitions.
Whatever C code you embed in the Cgo header should appear nowhere else. This is a good place to put small "plumbing adapters", if you have some existing C code that mostly works with Go, but needs some tweaks.
1This is a general technique in C for making sure that the header file declarations of functions agree with the C source definition of the same functions. That is, the header fifth.h might say:
void func(int arg1, char *arg2);
and, separately, the C code will read:
#include "fifth.h"
void func(int zorg, char *evil) {
// ...
}
and the C compiler will check that the declaration matches the definition.
I want to make a simple example of calling C code from Go with CGO. But for some reason I can't achieve desired. Compilation fails with the following error:
go build main.go
# awesomeProject1/print
duplicate symbol '_do_print' in:
$WORK/b002/_x002.o
$WORK/b002/_x003.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The code:
// print/print.c
#include <stdio.h>
void do_print(char * x){
printf("%s", x);
}
// print/print.go
package print
// #include <print.c>
import "C"
func DoPrint() {
C.do_print(C.CString("Hello!"))
}
// main.go
package main
import "awesomeProject1/print"
func main() {
print.DoPrint()
}
If I make do_print function static it compiles but I wouldn't be able to do that for 3rd party code I want to integrate with later.
Am I missing some essential piece from documentation? Tutorials are all alike and claim to work where my example fails. Please help!
Go version 1.16.4
There are two things going on here:
go build compiles *.c in addition to *.go1
#include <print.c> is exactly equivalent to overwriting the include statement with the contents of print.c
As a result, you are compiling the contents of print.c twice: once when print.c is compiled by CC, and once when print.go is compiled by CGo. Thus, the object files for print.c and print.go each contain all of the exported symbols defined in print.c. So you get two copies of do_print. Making do_print static works because a function declared as static will not be exported by CC.
Including a .c file (e.g. #include <file.c>) is pretty much always a bad idea. If you have a legitimate reason to #include function bodies, the convention is to use a header (.h) file. This is what C++ templating libraries do (like Boost, IIRC). Because C files normally are not included, and H files normally are included, including a C file subverts expectations and thus is likely to cause confusion and problems.
1 IIRC, Go will compile C files in a package if any of the Go files in the package import "C". That is, if no Go files in a package use CGo, Go will ignore C files in that package.
I have 2 project, the first, name as A, there is a submodule a imported sqlite3(github.com/mattn/go-sqlite3). Another B project import A's submodule a, and in another submodule b, it also import the same sqlite3.
Both A and B put there imports under vendor dir(managed by govendor). My Golang version is go version go1.12 linux/amd64.
While build B (go build main.go), throwing following errors(too many, part of them):
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/tmp/go-link-281256755/000029.o: In function `callbackTrampoline':
/tmp/go-build/_cgo_export.c:25: multiple definition of `callbackTrampoline'
/tmp/go-link-281256755/000005.o:/tmp/go-build/_cgo_export.c:25: first defined here
/tmp/go-link-281256755/000029.o: In function `stepTrampoline':
...
/home/xxx/go/src/gitlab.xxxxxxxxx.com/xxxxxxxxx-tools/A/vendor/github.com/mattn/go-sqlite3/sqlite3.go:129: multiple definition of `_sqlite3_result_text'
/tmp/go-link-281256755/000009.o:/home/xxx/go/src/gitlab.xxxxxxxxx.com/xxxxxxxxx-tools/A/vendor/github.com/mattn/go-sqlite3/sqlite3.go:129: first defined here
/tmp/go-link-281256755/000033.o: In function `_sqlite3_result_blob':
...
But building A works well. To testing the error, I started following demo, also with vendor inited by govendor, and build ok.
package main
import (
"database/sql"
"fmt"
"gitlab.xxxxxxxxx.com/xxxxxxxxxxxxxxx/A/a"
_ "github.com/mattn/go-sqlite3"
)
func main() {
fmt.Println(a.ModuleVariable) // use submodule `a` just like B is doing
_, _ = sql.Open(`sqlite3`, `test.db`) // use sqlite too
}
I think the compiler first compile A's sqlite3, objects are created under /tmp/go-link-281256755/000005.o (but no this dir after building), then compile B's importing of sqlite3 and also create a object contains the same-name function, then the compiler find 2 same-name symbols, the linking failed.
How to fix these situation? Is there any golang env settings to avoid these?
After I remove the sqlite3 package under vendor both of A and B, they both use the sqlite3 under ~/go/src/github.com/mattn/go-sqlite3/, they all build ok. But I can't do these, due to project A's deploy platform, I must put all dependencies under vendor, is there any another option to use multiple import with the same package?
For the cgo's issue of linking error "multiple definition of ...", the (work-around) solution is dependent on the nature of the linked C codes:
If two Go packages link to the same C codes (libraries), you should pass option --allow-multiple-definition to the linker (see ld man page), either via command options
go build --ldflags '-extldflags "-Wl,--allow-multiple-definition"',
or via #cgo directive in Go source of the packages linking to C codes:
//#cgo LDFLAGS: -Wl,--allow-multiple-definition
import "C"
If two Go packages link to different C codes containing some functions & variables with the same names, you should refactor those C codes:
Make sure to put keyword static to all declarations whose usage is within that C object only (not intended to be linked to Go nor to other C objects).
Find some way to do name mangling or put those duplicated identifiers into different namespaces (like in C++). It would be better if cgo supported some mechanism to do automatic namespacing using Go package name, but until now (2020) you must do it yourself. The C preprocessor's "token pasting" operator ## may help in this namespacing task. Eg.
//File: my_package1.h
#define NS(id) my_package1_ ## id
void NS(my_function1)(int);
void NS(my_function2)(float);
char NS(my_shared_var);
If you have any C function definition in the Go source, like in this question, you must move those definitions into a separate C source file under the same package folder, leaving only declarations in the Go source.
I am trying to write a go program in windows by importing a dynamic link library that I built. In order to build the go program using these custom .dll files, I am using cgo's CFLAGS and LDFLAGS in go source file. The code looks like this:
/*
#cgo CFLAGS: -I${SRCDIR}/lib
#cgo LDFLAGS: -L${SRCDIR}/lib -lkeyboard
#include <keyboard.h>
*/
import "C"
import "fmt"
func main() {
fmt.Printf("Hello word\n")
C.InitKeyboard()
fmt.Printf("\nEnter: ")
for {
r := C.GetCharacter()
fmt.Printf("%c", r)
if r == 'q' {
break
}
}
C.Closeboard()
}
My directory structure:
src
|_hello.go
|_lib
----|_keyboard.h
----|_keyboard.c
----|_libkeyboard.dll
The header (.h) and dynamic link library (.dll) are kept in a folder called 'lib' which is present in the same level as .go program file, this code compiles/builds but when I run the built .exe file it throws an error 'libkeyboard.dll not found', but works when I place the libkeyboard.dll is the same location as the compiled .exe file.
I want to understand why is it able to fetch the library file from the mentioned path (./lib) during compile time but failing to do the same during runtime.
I know creating PATH variable will help it to locate but why does it not work without setting PATH when I've already given the library files location CFLAGS and LDFLAGS.
And how do I make it work for static library file 'libkeyboard.a' because as I observe the built .exe file searches for libkeyboard.dll file and does not accept the libkeyboard.a file.
There are three files, (m.c,m.h, and **main.c*).
File m.h
// m.h
int m();
File m.c
// m.c
#include <stdio.h>
#include "m.h"
int m(){
printf("Hello,m!\n");
return 0;
}
File main.c
// main.c
#include "m.h"
int main(){
return m();
}
While I prefer a shared library (m.dll), I've made the CMakeLists.txt file:
PROJECT("app1")
ADD_LIBRARY(m SHARED m.c)
ADD_EXECUTABLE(myexe main.c)
TARGET_LINK_LIBRARIES(myexe m)
The CMake configuration is done and generated done. Opening app1.sln and building with Visual Studio, it crashes as
LNK1104:Can't open file "Debug\m.lib"
It only works as STATIC at ADD_LIBRARY(). Why doesn't it work on Windows?
If I got another shared library (mylib.dll), how could I invoke its functions in my main.c and CMakeLists.txt files?
There are differences between dynamic library linking on different platforms which also needs some additional code. The good news is, that CMake can help you with this. I found the following blog post by Gernot Klingler very useful:
Creating and using shared libraries with different compilers on different operating systems
In short you need some "export prefix" defined for whatever is declared in m.h. Otherwise the build process will not generate an "import library" for statically linking named m.lib (see also CMAKE_IMPORT_LIBRARY_SUFFIX).
Here is your code with the modifications needed:
m.h
#include "m_exports.h"
int M_EXPORTS m();
m.c
#include "m.h"
#include <stdio.h>
int m(){
printf("Hello,m!\n");
return 0;
}
CMakeLists.txt
cmake_minimum_required(VERSION 3.0)
include(GenerateExportHeader)
PROJECT("app1")
INCLUDE_DIRECTORIES("${CMAKE_CURRENT_BINARY_DIR}")
ADD_LIBRARY(m SHARED m.c m.h m_exports.h)
GENERATE_EXPORT_HEADER(m
BASE_NAME m
EXPORT_MACRO_NAME M_EXPORTS
EXPORT_FILE_NAME m_exports.h
STATIC_DEFINE SHARED_EXPORTS_BUILT_AS_STATIC)
ADD_EXECUTABLE(myexe main.c)
TARGET_LINK_LIBRARIES(myexe m)
Additional References
GenerateExportHeader macro
cmake and GenerateExportHeader
How do I get CMake to create a dll and its matching lib file?
MSDN: Walkthrough: Creating and Using a Dynamic Link Library (C++)
Using WINDOWS_EXPORT_ALL_SYMBOLS might help. See an introductory article for details. In short, invoke CMake like this:
cmake -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=TRUE -DBUILD_SHARED_LIBS=TRUE
Add this in CMakeLists.txt.
if(MSVC)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS TRUE)
set(BUILD_SHARED_LIBS TRUE)
endif()