OpenCV in Go without SWIG and third-parties lib - go

Main goal: Make OpenCV work in Go without SWIG and third party lib (an application to compare image in linux using Go)
I am new in all the kits (OpenCv Go and linux)
Can image detection (feature2d etc) can be done by C-api only? There is no convenient way to call C++ code and C-api is not updated(?)
I have followed How to use C++ in Go? but I failed.
When I make, I got the following errors
makefile:5: /usr/local/go/bin/src/Make.amd64: No such file or directory
makefile:6: /usr/local/go/bin/src/Make.pkg: No such file or directory
makefile:8: * missing separator. Stop.
The makefile is as followed
GOROOT=/usr/local/go/bin
GOARCH=amd64
TARG=foo
CGOFILES=foo.go
include $(GOROOT)/src/Make.$(GOARCH)
include $(GOROOT)/src/Make.pkg
foo.o:foo.cpp
g++ $(_CGO_CFLAGS_$(GOARCH)) -fPIC -O2 -o $# -c $(CGO_CFLAGS) $<
cfoo.o:cfoo.cpp
g++ $(_CGO_CFLAGS_$(GOARCH)) -fPIC -O2 -o $# -c $(CGO_CFLAGS) $<
CGO_LDFLAGS+=-lstdc++
$(elem)_foo.so: foo.cgo4.o foo.o cfoo.o
gcc $(_CGO_CFLAGS_$(GOARCH)) $(_CGO_LDFLAGS_$(GOOS)) -o $# $^ $(CGO_LDFLAGS)
Thanks a lot

You can't call C++ code without either writing C wrappers (+ cgo) yourself or using SWIG, that's just the way it is sadly.
That post you linked is extremely outdated and can't be used anymore.
On the other hand, you can always start rewriting opencv in pure go, the speed differences won't be that massive, specially if you learn how to use unsafe for the speed-critical parts.
disclaimer using unsafe is not advised since, well, it's unsafe.

You can do this, I've ported a very trivial subset of OpenCV into Go for my own purposes. In general, the process is to allocate everything on the heap and return it as a typedef'd void*. For example:
typedef void* gocv_matrix;
From there, a lot of your work is passthrough functions. One very important note is that your header files must be in pure C and must only (recursively) include headers that are pure C. This means your headers are going to be mostly prototypes/forward declarations.
So a few Matrix methods in your header mat.h may look like
gocv_matrix newMatrix();
void add(gocv_matrix m1, gocv_matrix m2, gocv_matrix dst);
void destroy(gocv_matrix m);
Then your implementation in mat.cxx will look something like
//include all relevant C++ OpenCV headers directly
gocv_matrix newMatrix() {
cv::Matrix *mat = new cv::Matrix();
return (gocv_matrix)mat;
}
void add(gocv_matrix m1, gocv_matrix m2, gocv_matrix dst) {
cv::Matrix *a = (cv::Matrix *)m1;
cv::Matrix *b = (cv::Matrix *)m2;
cv::Matrix *dstMat = (cv::Matrix *)dst;
(*dstMat) = (*a)+(*b);
}
void destroy(gocv_matrix m) {
cv::Matrix *a = (cv::Matrix *)(m1);
delete a;
}
(Disclaimer: the exact code here isn't verified for correctness, this is just the gist).
A few special notes:
Make sure you have a destroy method that you actually call or you'll leak memory.
Since C and C++ constants aren't the same as Go constants, you'll have to declare them as var instead of const.
Some of OpenCV's constants are included in headers which aren't pure C, which makes it extremely difficult to define them within Go. I noticed this most with some image processing subpackages.
Note the lack of templated generics. In general you're either foregoing templates entirely, defining a different type for each possible instance, or picking one (probably double, maybe an int size for displaying images) and sticking with it.
Note that you can't use overloaded operators this way. So a+b*c is b.Mul(c).Add(a). In theory you could invent some expression parser that takes in a string like "a+(b*c)" and a list of matrices, and then does some call batching, but if you were at that point in development you wouldn't be asking this question.
This is normal with cgo in general, but you'll probably be using unsafe a lot, especially if you want to work directly with the raw backing data of the matrix. You can reduce this somewhat by making your Go-level Mytype type a simple struct that contains a C.mytype instead of actually converting it.
Honestly, you should probably just use SWIG, since this is basically already what it does for you anyway, in addition to extra niceties like generating actual Go constants for you in most cases instead of sketchy var magic.

Related

gcc/clang: How to force ordering of items on the stack?

Consider the following code:
int a;
int b;
Is there a way to force that a precedes b on the stack?
One way to do the ordering would be to put b in a function:
void foo() {
int b;
}
...
int a;
foo();
However, that would generally work only if b isn't inlined.
Maybe there's a different way to do that? Putting an inline assembler between the two declarations may do a trick, but I am not sure.
Your initial question was about forcing a function call to not be inlined.
To improve on Jordy Baylac's answer, you might try to declare the function within the block calling it, and perhaps use a statement expr:
#define FOO_WITHOUT_INLINING(c,i) ({ \
extern int foo (char, int) __attribute__((noinline)); \
int r = foo(c,i); \
r; })
(If the type of foo is unknown, you could use typeof)
However, I still think that your question is badly formulated (and is meaningless, if one avoid reading your comments which should really go inside the question, which should have mentioned your libmill). By definition of inlining, a compiler can inline any function as it wants without changing the semantics of the program.
For example, a user of your library might legitimately compile it with -flto -O2 (both at compiling and at linking stage). I don't know what would happen then.
I believe you might redesign your code, perhaps using -fsplit-stack; are you implementing some call/cc in C? Then look inside the numerous existing implementations of it, and inside Gabriel Kerneis CPC.... See also setcontext(3) & longjmp(3)
Perhaps you might need to use somewhere the return_twice (and/or nothrow) function attribute of GCC, or some _Pragma like GCC optimize
Then you edited your question to change it completely (asking about order of variables on the call stack), still without mentioning in the question your libmill and its go macro (as you should; comments are volatile so should not contain most of the question).
But the C compiler is not even supposed to have a call stack (an hypothetical C99 conforming compiler could do whole program optimization to avoid any call stack) in the compiled program. And GCC is certainly allowed to put some variables outside of the call stack (e.g. only in registers) and it is doing that. And some implementations (IA64 probably) have two call stacks.
So your changed question is completely meaniningless: a variable might not sit on the stack (e.g. only be in a register, or even disappear completely if the compiler can prove it is useless after some other optimizations), and the compiler is allowed to optimize and use the same call stack slot for two variables (and GCC is doing such an optimization quite often). So you cannot force any order on the call stack layout.
If you need to be sure that two local variables a & b have some well defined order on the call stack, make them into a struct e.g.
struct { int _a, _b; } _locals;
#define a _locals._a
#define b _locals._b
then, be sure to put the &_locals somewhere (e.g. in a volatile global or thread-local variable). Since some versions of GCC (IIRC 4.8 or 4.7) had some optimization passes to reorder the fields of non-escaping struct-s
BTW, you might customize GCC with your MELT extension to help about that (e.g. introduce your own builtin or pragma doing part of the work).
Apparently, you are inventing some new dialect of C (à la CPC); then you should say that!
below there is a way, using gcc attributes:
char foo (char, int) __attribute__ ((noinline));
and, as i said, you can try -fno-inline-functions option, but this is for all functions in the compilation process
It is still unclear for me why you want function not to be inline-d, but here is non-pro solution I am proposing:
You can make this function in separate object something.o file.
Since you will include header only, there will be no way for the compiler to inline the function.
However linker might decide to inline it later at linking time.

why does gcc(default version on openSUSE 11.3) give an error on the statement int *p=malloc(sizeof(int));?

malloc returns a void pointer.so why is it not working for me without typecasting the return value?
The error pretty clear said that gcc is not allowing conversion from void* to int*.
In C, you don't have to cast. In fact it's a bad idea to cast there since it can cause certain subtle errors.
However, casting is required in C++ so that would be my first guess, that you're somehow invoking the C++ compiler. Perhaps your source files are *.cpp or *.C both of which may be auto-magigically treated as C++ rather than C.
See here for more detail:
C++ source files conventionally use one of the suffixes ‘.C’, ‘.cc’, ‘.cpp’, ‘.CPP’, ‘.c++’, ‘.cp’, or ‘.cxx’; C++ header files often use ‘.hh’, ‘.hpp’, ‘.H’, or (for shared template code) ‘.tcc’; and preprocessed C++ files use the suffix ‘.ii’. GCC recognizes files with these names and compiles them as C++ programs even if you call the compiler the same way as for compiling C programs (usually with the name gcc).
The fact that it knows you're trying to convert void* to int* means that you have a valid malloc prototype in place so I can't see it being anything other than the imposition of C++ rules.
Without code I can't help you properly, but you can try this:
p = (int*)malloc(sizeof(int));
Give more info about what you want to do and what you are allocating.

Where Is gcvt or gcvtf Defined in gcc Source Code?

I'm working on some old source code for an embedded system on an m68k target, and I'm seeing massive memory allocation requests sometimes when calling gcvtf to format a floating point number for display. I can probably work around this by writing my own substitute routine, but the nature of the error has me very curious, because it only occurs when the heap starts at or above a certain address, and it goes away if I hack the .ld linker script or remove any set of global variables (which are placed before the heap in my memory map) that add up to enough byte size so that the heap starts below the mysterious critical address.
So, I thought I'd look in the gcc source code for the compiler version I'm using (m68k-elf-gcc 3.3.2). I downloaded what appears to be the source for this version at http://gcc.petsads.us/releases/gcc-3.3.2/, but I can't find the definition for gcvt or gcvtf anywhere in there. When I search for it, grep only finds some documentation and .h references, but not the definition:
$ find | xargs grep gcvt
./gcc/doc/gcc.info: C library functions `ecvt', `fcvt' and `gcvt'. Given va
lid
./gcc/doc/trouble.texi:library functions #code{ecvt}, #code{fcvt} and #code{gcvt
}. Given valid
./gcc/sys-protos.h:extern char * gcvt(double, int, char *);
So, where is this function actually defined in the source code? Or did I download the entirely wrong thing?
I don't want to change this project to use the most recent gcc, due to project stability and testing considerations, and like I said, I can work around this by writing my own formatting routine, but this behavior is very confusing to me, and it will grind my brain if I don't find out why it's acting so weird.
Wallyk is correct that this is defined in the C library rather than the compiler. However, the GNU C library is (nearly always) only used with Linux compilers and distributions. Your compiler, being a "bare-metal" compiler, almost certainly uses the Newlib C library instead.
The main website for Newlib is here: http://sourceware.org/newlib/, and this particular function is defined in the newlib/libc/stdlib/efgcvt.c file. The sources have been quite stable for a long time, so (unless this is a result of a bug) chances are pretty good that the current sources are not too different from what your compiler is using.
As with the GNU C source, I don't see anything in there that would obviously cause this weirdness that you're seeing, but it's all eventually a bunch of wrappers around the basic sprintf routines.
It is in the GNU C library as glibc/misc/efgcvt.c. To save you some trouble, the code for the function is:
char *
__APPEND (FUNC_PREFIX, gcvt) (value, ndigit, buf)
FLOAT_TYPE value;
int ndigit;
char *buf;
{
sprintf (buf, "%.*" FLOAT_FMT_FLAG "g", MIN (ndigit, NDIGIT_MAX), value);
return buf;
}
The directions for obtain glibc are here.

Getting GCC to compile without inserting call to memcpy

I'm currently using GCC 4.5.3, compiled for PowerPC 440, and am compiling some code that doesn't require libc. I don't have any direct calls to memcpy(), but the compiler seems to be inserting one during the build.
There are linker options like -nostdlib, -nostartfiles, -nodefaultlibs but I'm unable to use them as I'm not doing the linking phase. I'm only compiling. With something like this:
$ powerpc-440-eabi-gcc -O2 -g -c -o output.o input.c
If I check the output.o with nm, I see a reference to memcpy:
$ powerpc-440-eabi-nm output.o | grep memcpy
U memcpy
$
The GCC man page makes it clear how to remove calls to memcpy and other libc calls with the linker, but I don't want the compiler to insert them in the first place, as I'm using a completely different linker (not GNU's ld, and it doesn't know about libc).
Thanks for any help you can provide.
There is no need to -fno-builtins or -ffreestanding as they will unnecessarily disable many important optimizations
This is actually "optimized" by gcc's tree-loop-distribute-patterns, so to disable the unwanted behavior while keeping the useful builtin capabilities, you can just use:
-fno-tree-loop-distribute-patterns
Musl-libc uses this flag for its build and has the following note in their configure script (I looked through the source and didn't find any macros, so this should be enough)
# Check for options that may be needed to prevent the compiler from
# generating self-referential versions of memcpy,, memmove, memcmp,
# and memset. Really, we should add a check to determine if this
# option is sufficient, and if not, add a macro to cripple these
# functions with volatile...
# tryflag CFLAGS_MEMOPS -fno-tree-loop-distribute-patterns
You can also add this as an attribute to individual functions in gcc using its optimize attribute, so that other functions can benefit from calling mem*()
__attribute__((optimize("no-tree-loop-distribute-patterns")))
size_t strlen(const char *s){ //without attribute, gcc compiles to jmp strlen
size_t i = -1ull;
do { ++i; } while (s[i]);
return i;
}
Alternatively, (at least for now) you may add a confounding null asm statement into your loop to thwart the pattern recognition.
size_t strlen(const char *s){
size_t i = -1ull;
do {
++i;
asm("");
} while (s[i]) ;
return i;
}
Gcc emits call to memcpy in some circumstance, for example if you are copying a structure.
There is no way to change GCC behaviour but you can try to avoid this by modifying your code to avoid such copy. Best bet is to look at the assembly to figure out why gcc emitted the memcpy and try to work around it. This is going to be annoying though, since you basically need to understand how gcc works.
Extract from http://gcc.gnu.org/onlinedocs/gcc/Standards.html:
Most of the compiler support routines used by GCC are present in libgcc, but there are a few exceptions. GCC requires the freestanding environment provide memcpy, memmove, memset and memcmp. Finally, if __builtin_trap is used, and the target does not implement the trap pattern, then GCC will emit a call to abort.
You need to disable a that optimization with -fno-builtin. I had this problem once when trying to compile memcpy for a C library. It called itself. Oops!
You can also make your binary a "freestanding" one:
The ISO C standard defines (in clause 4) two classes of conforming implementation. A conforming hosted implementation supports the whole standard [...]; a conforming freestanding implementation is only required to provide certain library facilities: those in , , , and ; since AMD1, also those in ; and in C99, also those in and . [...].
The standard also defines two environments for programs, a freestanding environment, required of all implementations and which may not have library facilities beyond those required of freestanding implementations, where the handling of program startup and termination are implementation-defined, and a hosted environment, which is not required, in which all the library facilities are provided and startup is through a function int main (void) or int main (int, char *[]).
An OS kernel would be a freestanding environment; a program using the facilities of an operating system would normally be in a hosted implementation.
(paragraph added by me)
More here. And the corresponding gcc option/s (keywords -ffreestanding or -fno-builtin) can be found here.
This is quite an old question, but I've hit the same issue, and none of the solutions here worked.
So I defined this function:
static __attribute__((always_inline)) inline void* imemcpy (void *dest, const void *src, size_t len) {
char *d = dest;
const char *s = src;
while (len--)
*d++ = *s++;
return dest;
}
And then used it instead of memcpy. This has solved the inlining issue for me permanently. Not very useful if you are compiling some sort of library though.

Detect writable static data

I just discovered that some parts of the code I am working on incorrectly uses writeable static data where it could/should use constant data.
Short of doing a dumb search-and-replace for "static" -> "static const", is there any way to preventing all 'static' data from being writeable, much like how constant string data can be made explicitly writeable?
I am using the GCC toolchain, development target is x86.
There's probably writable static data in some of the libraries you use. (Such as the standard C and C++ libraries). Making that const would be bad.
It's probably better to go through your code and change things manually.
You can use nm to get a list of symbols in your .o files. In the nm output, the first column gives the type of symbol; the letters B, C, D, G or S indicate writable data. The last column gives the (mangled) variable name. It's possible to write a little script to parse the nm output and look for these.
I guess the better way is adding the "const" to all variables you have. You could use a "#define static static const" (note that it would break wherever you already changed it) but I don't recommend doing so (will make your code much less readable and possibly will break some things, and you won't be able to declare static functions).

Resources