Let's supose my package workspace has github.com/yada/yada third party package. Inside this package there is a yoda.go.h header I would like to reuse (not sure if it's a good idea, but that's a hole new question). How do I import the header from a dependecy package into my own package?
package main
// #cgo pkg-config: my-dep other-dep
// #include <someHeader.h>
// #include <otherHeader.h>
// #include github.com/yada/yada/yoda.go.h // doesn't work :(
import "C"
Apart from being a good idea or not, I still would like to know if it's possible.
PS: If you think it's really a bad idea, what should I do instead?
Use CGO CFLAGS directive to reference additional include path.
//#cgo CFLAGS: -I $GOPATH/src/github.com/yada/yada/
...
//#include "yoda.go.h"
import "C"
CORRECTION:
The go tool does not expand $GOPATH variable during build. Instead, the full path should be used there. Corrected code:
//#cgo CFLAGS: -I /full/path/to/src/github.com/yada/yada/
//#include "yoda.go.h"
Probably not a good idea to try and reference it directly, since it's not an exported entity, and is subject to change or removal.
If you really need that header, you'll have to reference it directly in your local filesystem. (of course you're free to copy into your project too)
Related
Ubuntu. vscode 1.62.1. go1.17.3. vscode go extension v0.29.0. delve v1.7.1.
I'm trying to a build a small app that uses Cgo, using vscode and vscode-go. Only one module imports "C".
My project structure has the root directory containing the "go.mod" and "main.go" files, and I have subpackages in subfolders. I also have "include" and "lib" directories that contain the C artifacts.
This is what I have so far in the C module:
package voltage
// #cgo CFLAGS: -g -Wall -Iinclude
// #cgo LDFLAGS: -Llib/linux -lvibesimple -lcurl -lssl -lvibecrypto -lvibeictk -lvibeserver
// #include <stdio.h>
// #include <errno.h>
// #include "veapi.h"
import "C"
func Encrypt(datatype string, data string) (result string) {
return
}
func Decrypt(datatype string, data string) (result string) {
return
}
In the "Problems" view, it shows the following two issues:
go list failed to return CompiledGoFiles. This may indicate failure to perform cgo processing; try building at the command line. See https://golang.org/issue/38990.
And:
could not import C (cgo preprocessing failed) (compile)
I read the cited issue, but I'm not sure what to do with that information.
I'm not sure how to move forward here.
The C compiler is not executed in the source directory but in a temporary directory only containing intermediate files, such as your go files compiled as static libraries (.a). Therefore, the LDFLAG -Llib/linux points to an unexisting directory.
To solve this issue, just replace that flag with -L${SRCDIR}/lib/linux.
Directly from the cgo docs:
When the cgo directives are parsed, any occurrence of the string ${SRCDIR} will be replaced by the absolute path to the directory containing the source file. This allows pre-compiled static libraries to be included in the package directory and linked properly.
The cgo tool will always invoke the C compiler with the source file's directory in the include path; i.e. -I${SRCDIR} is always implied.
I have a really simple setup: A .go file (test.go) and a .c file (PMDK.c). I include the .c file in Go as follows:
test.go:
package main
/*
#include "PMDK.c"
#cgo pkg-config: libpmem
*/
import "C"
func main() {
C.helloWorld()
}
When I run go run test.go, it builds for exactly one time. Whatever changes I do to PMDK.c, my program has the exact same behavior every time.
I also tried go build test.go, which leads to the same result.
Finally , following CGo not compiling C files in same directory, I just did go build. This didn't work, since I had to create a .mod file (go build test.go). Then, the problem was that the three functions in PMDK.c (helloWorld and two others) were supposedly defined multiple times.
I can't make it build my changes.
By the way, if I copy/move the source files to another directory and build them there, the changes apply (only for once, again).
The heart of the problem is that your setup is wrong: your Go code should #include, in the Cgo prefix, only the headers for any C code you want compiled separately. For instance:
package main
/*
#include "pmdk.h"
*/
import "C"
func main() {
C.helloWorld()
}
You can put C code into the prefix:
package main
/*
#include <stdio.h>
void helloWorld() {
printf("hello world from C\n");
}
*/
import "C"
...
But if you put your C code into a separate file (prog.c, etc.), you should create a small header file that simply declares each function, and #include that header file from both the C code and the Go code.1
Running:
go build
will then compile the C code if it has changed, and build the Go code if it has changed, and link together the two as appropriate. If you #include the C code directly into the Go package, as you did, go build will both build the C code and build the Go code that includes the C code, which is why you get the duplicate definitions.
Whatever C code you embed in the Cgo header should appear nowhere else. This is a good place to put small "plumbing adapters", if you have some existing C code that mostly works with Go, but needs some tweaks.
1This is a general technique in C for making sure that the header file declarations of functions agree with the C source definition of the same functions. That is, the header fifth.h might say:
void func(int arg1, char *arg2);
and, separately, the C code will read:
#include "fifth.h"
void func(int zorg, char *evil) {
// ...
}
and the C compiler will check that the declaration matches the definition.
I want to make a simple example of calling C code from Go with CGO. But for some reason I can't achieve desired. Compilation fails with the following error:
go build main.go
# awesomeProject1/print
duplicate symbol '_do_print' in:
$WORK/b002/_x002.o
$WORK/b002/_x003.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The code:
// print/print.c
#include <stdio.h>
void do_print(char * x){
printf("%s", x);
}
// print/print.go
package print
// #include <print.c>
import "C"
func DoPrint() {
C.do_print(C.CString("Hello!"))
}
// main.go
package main
import "awesomeProject1/print"
func main() {
print.DoPrint()
}
If I make do_print function static it compiles but I wouldn't be able to do that for 3rd party code I want to integrate with later.
Am I missing some essential piece from documentation? Tutorials are all alike and claim to work where my example fails. Please help!
Go version 1.16.4
There are two things going on here:
go build compiles *.c in addition to *.go1
#include <print.c> is exactly equivalent to overwriting the include statement with the contents of print.c
As a result, you are compiling the contents of print.c twice: once when print.c is compiled by CC, and once when print.go is compiled by CGo. Thus, the object files for print.c and print.go each contain all of the exported symbols defined in print.c. So you get two copies of do_print. Making do_print static works because a function declared as static will not be exported by CC.
Including a .c file (e.g. #include <file.c>) is pretty much always a bad idea. If you have a legitimate reason to #include function bodies, the convention is to use a header (.h) file. This is what C++ templating libraries do (like Boost, IIRC). Because C files normally are not included, and H files normally are included, including a C file subverts expectations and thus is likely to cause confusion and problems.
1 IIRC, Go will compile C files in a package if any of the Go files in the package import "C". That is, if no Go files in a package use CGo, Go will ignore C files in that package.
I have 2 project, the first, name as A, there is a submodule a imported sqlite3(github.com/mattn/go-sqlite3). Another B project import A's submodule a, and in another submodule b, it also import the same sqlite3.
Both A and B put there imports under vendor dir(managed by govendor). My Golang version is go version go1.12 linux/amd64.
While build B (go build main.go), throwing following errors(too many, part of them):
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/tmp/go-link-281256755/000029.o: In function `callbackTrampoline':
/tmp/go-build/_cgo_export.c:25: multiple definition of `callbackTrampoline'
/tmp/go-link-281256755/000005.o:/tmp/go-build/_cgo_export.c:25: first defined here
/tmp/go-link-281256755/000029.o: In function `stepTrampoline':
...
/home/xxx/go/src/gitlab.xxxxxxxxx.com/xxxxxxxxx-tools/A/vendor/github.com/mattn/go-sqlite3/sqlite3.go:129: multiple definition of `_sqlite3_result_text'
/tmp/go-link-281256755/000009.o:/home/xxx/go/src/gitlab.xxxxxxxxx.com/xxxxxxxxx-tools/A/vendor/github.com/mattn/go-sqlite3/sqlite3.go:129: first defined here
/tmp/go-link-281256755/000033.o: In function `_sqlite3_result_blob':
...
But building A works well. To testing the error, I started following demo, also with vendor inited by govendor, and build ok.
package main
import (
"database/sql"
"fmt"
"gitlab.xxxxxxxxx.com/xxxxxxxxxxxxxxx/A/a"
_ "github.com/mattn/go-sqlite3"
)
func main() {
fmt.Println(a.ModuleVariable) // use submodule `a` just like B is doing
_, _ = sql.Open(`sqlite3`, `test.db`) // use sqlite too
}
I think the compiler first compile A's sqlite3, objects are created under /tmp/go-link-281256755/000005.o (but no this dir after building), then compile B's importing of sqlite3 and also create a object contains the same-name function, then the compiler find 2 same-name symbols, the linking failed.
How to fix these situation? Is there any golang env settings to avoid these?
After I remove the sqlite3 package under vendor both of A and B, they both use the sqlite3 under ~/go/src/github.com/mattn/go-sqlite3/, they all build ok. But I can't do these, due to project A's deploy platform, I must put all dependencies under vendor, is there any another option to use multiple import with the same package?
For the cgo's issue of linking error "multiple definition of ...", the (work-around) solution is dependent on the nature of the linked C codes:
If two Go packages link to the same C codes (libraries), you should pass option --allow-multiple-definition to the linker (see ld man page), either via command options
go build --ldflags '-extldflags "-Wl,--allow-multiple-definition"',
or via #cgo directive in Go source of the packages linking to C codes:
//#cgo LDFLAGS: -Wl,--allow-multiple-definition
import "C"
If two Go packages link to different C codes containing some functions & variables with the same names, you should refactor those C codes:
Make sure to put keyword static to all declarations whose usage is within that C object only (not intended to be linked to Go nor to other C objects).
Find some way to do name mangling or put those duplicated identifiers into different namespaces (like in C++). It would be better if cgo supported some mechanism to do automatic namespacing using Go package name, but until now (2020) you must do it yourself. The C preprocessor's "token pasting" operator ## may help in this namespacing task. Eg.
//File: my_package1.h
#define NS(id) my_package1_ ## id
void NS(my_function1)(int);
void NS(my_function2)(float);
char NS(my_shared_var);
If you have any C function definition in the Go source, like in this question, you must move those definitions into a separate C source file under the same package folder, leaving only declarations in the Go source.
Can I link to individual obj file using #pragma ? I tried #pragma comment(lib, "disk_file_sink.obj") but it doesn't seem to work. And it works if I specify this file in Linker/Input/Additional Dependencies.
I have the same problem as yours and I look up many website but I do not get the result so I use an another method to resolve this problem.That was I did.
I use the command lib to compact the objs to library. You can use lib /out:xxx.lib one.obj two.obj or lib /out:xxx.lib *.obj. The command can add to project property.