Is there a way to build a Go + C application that:
From main package X, import packages Y and Z.
Package M exports a go callback F.
Packages X and Y are both built with accompanying C files, both want
to call F from C source code.
Generally speaking I'm trying to figure out how to call a callback from accompanying C files in other modules which are used to build a final application. I coudn't figure out how to achieve this or something similar. I'm also interested in convoluted solutions.
I don't see a way to call a Go function across packages, but all cgo packages are linked into the same binary and can call each other. This means that you can export M.F to a C function in package M and call that C function from packages Y and Z.
m/m.go:
package m
// void F();
import "C"
import "fmt"
//export F
func F() {
fmt.Println("m.f")
}
m/m.h:
void m_f();
m/m.c:
#include <stdio.h>
#include "_cgo_export.h"
#include "m.h"
void m_f() {
printf("m_f\n")
F();
}
y/y.go:
package y
// The LDFLAGS lines below are needed to prevent linker errors
// since not all packages are present while building intermediate
// packages. The darwin build tag is used as a proxy for clang
// versus gcc because there doesn't seem to be a better way
// to detect this.
// #cgo darwin LDFLAGS: -Wl,-undefined -Wl,dynamic_lookup
// #cgo !darwin LDFLAGS: -Wl,-unresolved-symbols=ignore-all
// #include "y.h"
import "C"
import (
"fmt"
_ "m"
)
func Y() {
fmt.Println("y.Y")
C.y()
}
y/y.h:
void y();
y/y.c:
#include <stdio.h>
#include "../m/m.h"
void y() {
printf("y.C.y\n");
m_f();
}
Here is an example, that will accept any go callback (not thread-safe).
b.go:
package b
// typedef void (*cbFunc) ();
// void do_run(cbFunc);
// void goCallback();
import "C"
//export goCallback
func goCallback() {
if goCallbackHolder != nil {
goCallbackHolder()
}
}
var goCallbackHolder func()
func Run(callback func()) {
goCallbackHolder = callback
C.do_run(C.cbFunc(C.goCallback))
}
b.c:
#include "_cgo_export.h"
void do_run(void (*callback)())
{
callback();
}
I couldn't make it to work in a simple fashion IMO.
Given main package X that imports Y and Z, both having to call (from C source code) F declared in package M,
I had to:
Create a small wrapper W1 for F in Y and export it to be called from Y's C source.
Create a small wrapper W2 for F in Z and export it to be called from Z's C source.
In Y CGO CPPFLAGS define -DCALLBACK=W1
In Z CGO CPPFLAGS define -DCALLBACK=W2
From C source code, anywhere, I'm now able to refer to F as CALLBACK (yeah, internally it's all different stuff, which I refer to using a single name at one end to call a single function at the other end).
This is convoluted, but it's working, although configuring such macros and producing little wrappers
is not being ideal. If anyone could detail a simpler procedure I would be glad. Everything
I tried ended up with duplicated symbols or non-visible declarations.
Related
Is it possible to call a static object (.so) file from Go?
I've been searchign Google and I keep hitting upon the claim that I can do
lib, _ := syscall.LoadLibrary("...")
But trying this gives an error
undefined: syscall.LoadLibrary
and searching through Godocs I cannot find reference to this function in the syscall package.
Is it possible to load a library and call its functions?
On a POSIX platform, you could use cgo to call dlopen and friends:
// #cgo LDFLAGS: -ldl
// #include <dlfcn.h>
import "C"
import fmt
func foo() {
handle := C.dlopen(C.CString("libfoo.so"), C.RTLD_LAZY)
bar := C.dlsym(handle, C.CString("bar"))
fmt.Printf("bar is at %p\n", bar)
}
As #JimB said, you should just use CGO, and put the linking to the dynamic/static library there. as per this example:
// #cgo LDFLAGS: -lpng
// #include <png.h>
import "C"
...
var x:= C.png_whatever() // whatever the API is
Read more here: http://blog.golang.org/c-go-cgo
The answer by #Martin Törnwall explains how to use dlopen() for function lookup. Adding this answer to include sample code for how to actually call that function as well. (Using the approach suggested in the comments).
The idea is to write a wrapper function in C language for each function the shared library, which accepts a void* pointer (pointer to the function returned by dlopen()), converts it into an appropriate function pointer, and then call it.
Suppose we have a function named str_length in libfoo.so to calculate the length of a string, then the resulting Go code would be:
package main
import (
"fmt"
)
/*
#cgo LDFLAGS: -ldl
#include <dlfcn.h>
typedef int (*str_length_type)(char*); // function pointer type
int str_length(void* f, char* s) { // wrapper function
return ((str_length_type) f)(s);
}
*/
import "C"
func main() {
handle := C.dlopen(C.CString("libfoo.so"), C.RTLD_LAZY)
str_length_ptr := C.dlsym(handle, C.CString("str_length"))
result := C.str_length(str_length_ptr, C.CString("Hello World!"))
fmt.Println(result) // prints 12
}
In Go, I have a compile error due to incompatible types which I cannot explain. I'm using the "C" module. A minimum example consists of the following 2 files:
package module
import "C"
type T struct {
X C.int
}
and a main program
package main
import (
"fmt"
"sandbox/module"
)
import "C"
func f() *module.T {
var x C.int = 42
return &module.T{X: x}
}
func main() {
fmt.Printf("value: %d", f().X)
}
This fails to compile with the message
./main.go:12: cannot use x (type C.int) as type module.C.int in field value.
For some reason the compiler thinks that C.int is not equal to module.C.int.
It must have something to do with the C module and the fact that the code is spread across 2 modules because it suddenly works if I switch from C.int to plain int.
Why does this piece of code not compile? What would be the proper solution to make it compile without squashing all code together in one module?
I'm using the latest Go compiler 1.9.2 on Ubuntu 16.04.
Command cgo
Go references to C
Cgo translates C types into equivalent unexported Go types. Because
the translations are unexported, a Go package should not expose C
types in its exported API: a C type used in one Go package is
different from the same C type used in another.
You say, "For some reason the compiler thinks that C.int is not equal to module.C.int." As the cgo command documentation explains, unexported Go types in different packages are not equal.
Don't expose C types in an exported API. For example,
module.go:
package module
type T struct {
X int
}
main.go:
package main
import (
"fmt"
"sandbox/module"
)
import "C"
func f() *module.T {
var x C.int = 42
return &module.T{X: int(x)}
}
func main() {
fmt.Printf("value: %d\n", f().X)
}
Output:
value: 42
I am trying to understand cgo. I can call C functions from go and the other way also. I tried doing something different and it didn't work.
go code:
// main.go
package main
/*
extern void myfunction(GoSlice s);
*/
import "C"
func main() {
var s [5]byte
C.myfunction(s[:])
}
C code:
// main.c
#include "_cgo_export.h"
#include <stdint.h>
void myfunction(GoSlice s)
{
for (int i = 0; i < s.len; i++)
printf("%hhu ", ((uint8_t *)s.data)[i]);
}
The compilation fails because GoSlice is not defined at the time of the C function declaration in main.go. Also, I can't include _cgo_export.h in main.go to get the definition. Is there any way I can get this working ? Note that I can get the functionality working using C compatible data types. But I want to directly use go data type here.
I want to malloc large number of objects in to memory.(about 100 million objects) because the gc of golang is not effective enough,so i need to use c/c++ to malloc memory and use std::vector to hold objects.
this is my code,i want use std container in cgo:
package main
import (
"fmt"
)
/*
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <vector>
using namespace std;
void dosome(){
vector<int> ivec; // empty vector
for (vector<int>::size_type ix = 0; ix != 10; ++ix)
ivec[ix] = ix; // disaster: ivec has no elements
}
*/
// #cgo LDFLAGS: -lstdc++
import "C"
//import "fmt"
func main() {
C.dosome()
var input string
fmt.Scanln(&input)
}
and have error message below:
go run stddemo.go
# command-line-arguments
./stddemo.go:13:10: fatal error: 'vector' file not found
#include <vector>
^
1 error generated.
how can i set the include path or is there another idea?
While you can use C++ with CGo, you can't embed that code inside the .go file, since it ultimately gets built with a C compiler.
Instead, place your dosome function in a separate .cpp file in the same directory as the .go file, and declare your function to use C linkage. For example:
extern "C" {
void dosome() {
vector<int> ivec;
...
}
}
If you include a prototype for the function in the CGo comment in the .go file so you can call it from Go.
Since you have multiple files now, you can't use the go run foo.go shorthand any more (since it only compiles a single file). Instead, you will need to use go run package or go build package, where your code is located at $GOPATH/src/package.
Uhh I think your conclusions are a bit too fast. GC cost is driven by two things: The more garbage your program produces, the more the GC will have to run. Second: The more pointers there are to scan, the longer a single GC will take.
That is to say: as long as you put your 100 million things into a go slice and keep them there: the GC won't have to run much, because there's no garbage. And second: if your things don't contain pointers, GC time, should it still occur, will be fine.
So, my question is: do your things have pointers?
if you just want to call someone's golang code this is a quick ram inefficient way:
package main
import "C"
import "fmt"
import "unsafe"
func intArrayFromC (src unsafe.Pointer, sz int) []uint64 {
dest := make([]uint64, sz)
copy(dest, (*(*[1000000000]uint64)(unsafe.Pointer(src)))[:sz:sz])// big number dose not affect ram.
return dest
}
//export doPrint
func doPrint(src unsafe.Pointer, sz int){
var numbers []uint64 = intArrayFromC(src, sz);
for i := 0; i < len(numbers); i++ {
fmt.Printf("%d index: %d\n", numbers[i], i)
}
}
func main() {}
and the c++ code:
#include "print.h"
#include <string.h>
#include <vector>
int main() {
std::vector<GoUint64> numbers{99,44,11,00,2,33,44};
while (1) {
doPrint(numbers.data(), numbers.size());
}
return 0;
}
Is it possible to call a static object (.so) file from Go?
I've been searchign Google and I keep hitting upon the claim that I can do
lib, _ := syscall.LoadLibrary("...")
But trying this gives an error
undefined: syscall.LoadLibrary
and searching through Godocs I cannot find reference to this function in the syscall package.
Is it possible to load a library and call its functions?
On a POSIX platform, you could use cgo to call dlopen and friends:
// #cgo LDFLAGS: -ldl
// #include <dlfcn.h>
import "C"
import fmt
func foo() {
handle := C.dlopen(C.CString("libfoo.so"), C.RTLD_LAZY)
bar := C.dlsym(handle, C.CString("bar"))
fmt.Printf("bar is at %p\n", bar)
}
As #JimB said, you should just use CGO, and put the linking to the dynamic/static library there. as per this example:
// #cgo LDFLAGS: -lpng
// #include <png.h>
import "C"
...
var x:= C.png_whatever() // whatever the API is
Read more here: http://blog.golang.org/c-go-cgo
The answer by #Martin Törnwall explains how to use dlopen() for function lookup. Adding this answer to include sample code for how to actually call that function as well. (Using the approach suggested in the comments).
The idea is to write a wrapper function in C language for each function the shared library, which accepts a void* pointer (pointer to the function returned by dlopen()), converts it into an appropriate function pointer, and then call it.
Suppose we have a function named str_length in libfoo.so to calculate the length of a string, then the resulting Go code would be:
package main
import (
"fmt"
)
/*
#cgo LDFLAGS: -ldl
#include <dlfcn.h>
typedef int (*str_length_type)(char*); // function pointer type
int str_length(void* f, char* s) { // wrapper function
return ((str_length_type) f)(s);
}
*/
import "C"
func main() {
handle := C.dlopen(C.CString("libfoo.so"), C.RTLD_LAZY)
str_length_ptr := C.dlsym(handle, C.CString("str_length"))
result := C.str_length(str_length_ptr, C.CString("Hello World!"))
fmt.Println(result) // prints 12
}