How to interface more complex shared library? - ruby

I'm trying to dynamically call functions of external library on Linux/Unix system.
I have some success with dl library but only when primitive C types are used and arguments are passed by value:
require 'dl/import'
module LibM
extend DL::Importer
dlload 'libm.so'
extern 'double sin(double)'
end
puts LibM.sin(3.14159265358979323846 / 2) # 1.0
However how can be interfaced functions using more complex types like C structs or if arguments are pointers where results of the call are stored ?
module LibX11
extend DL::Importer
dlload 'libX11.so.6'
extern 'Display *XkbOpenDisplay (char *display_name, int *event_rtrn, int *error_rtrn, int *major_in_out, int *minor_in_out, int *reason_rtrn)'
end
Display is a big struct, at event_rtrn is stored some result etc.
I've taken a look at DL::CStructBuilder and it looks can do the job, but as the documentation is very brief and no working examples found I'm lost here how it has to be properly used.
I have to add standard Ruby 1.9 modules have to be used (if possible) as installation of additional gems on the target machine is prohibited.

I am facing with interfacing with C libraries in this period (I am writing a wrapper for libgtop), and I choosed to use ffi, which is quite well documented (although the documentation sometimes is a bit outdated) and above all its mailing list is full of people who give you an helping hand when you are in trouble.
So I propose you a solution which uses ffi:
require 'ffi'
module LibX11
extend FFI::Library
ffi_lib 'libX11.so.6'
# Display *XkbOpenDisplay (char *display_name, int *event_rtrn, int *error_rtrn, int *major_in_out, int *minor_in_out, int *reason_rtrn)
attach_function :XkbOpenDisplay, [:pointer, :pointer, :pointer, :pointer, :pointer, :pointer], :pointer
end
Then you have to describe the Display struct layout, like written here:
class Display < FFI::Struct
layout :value, :double,
:other_value, :char,
...
end
And then you do something like this:
p1 = FFI::MemoryPointer.new(:char)
p2 = FFI::MemoryPointer.new(:int)
p3 = FFI::MemoryPointer.new(:int)
p4 = FFI::MemoryPointer.new(:int)
p5 = FFI::MemoryPointer.new(:int)
p6 = FFI::MemoryPointer.new(:int)
# write to pointer if needed, otherwise it is a null pointer
# p1.write_char('a')
# p2.write_int(1)
# ...
struct_pointer = LibX11.XkbOpenDisplay(p1, p2, p3, p4, p5, p6)
# read the struct
struct = Display.new(display_struct_pointer)
p Hash[ s.members.map { |m| [ m, s[m] ] } ]
I didn't tested the code, but it should be roughly correct. Let me now if it isn't.
After some research about DL on ruby 2.0 I found that it is deprecated and replaced with fiddle, you would consider to use it rather than DL if you can't use FFI. Fiddle seems to be available for ruby 1.9.3 and 1.9.2 too

Related

Documenting Python coroutines with Sphinx autodoc

I'm starting to document my first asyncio-based project with Sphinx. I noticed some projects have this "coroutine" prefix before some methods and I'd like to do the same in my project's documentation, but I can't figure out how.
For example, aiohttp's HTTP client reference shows this:
class aiohttp.ClientSession(...):
coroutine request(...)
That project seems to use a coroutinemethod directive to achieve this, but I document all my functions and classes inline using docstrings and this directive only works if you write the docs in a reStructuredText document.
Does anybody know how to achieve this result with autodoc?
Edit: I will also accept answers that explain how to make a Sphinx extension to do this if Sphinx doesn't support it. Bonus points if someone can point my to some kind of way that automatically detects whether the method is a coroutine or not using inspect.iscoroutinefunction().
Edit: I'm looking at the "pyspecific" Sphinx extension in the CPython project for inspiration. However, I need to change autodoc's behavior, not add new directives. After a bit of research, it looks like autodoc has an autodoc-process-signature event which can be used to customize the function signature, but it doesn't seem to have the object used by the "pyspecific" extension.
This functionality is not already built-in to Sphinx. However, pull request #1826 adds support for generators, coroutine functions, coroutine methods with built-in detection of coroutines in autodoc's autofunction and automethod directives.
Here is a patch that I applied locally to enable this in Sphinx 1.4 (requires disabling sphinx-build's -W "turn warnings into errors" option):
# Recipe stolen from open PR (https://github.com/sphinx-doc/sphinx/pull/1826).
from inspect import iscoroutinefunction
from sphinx import addnodes
from sphinx.domains.python import (
PyClassmember,
PyModulelevel,
)
from sphinx.ext.autodoc import FunctionDocumenter as _FunctionDocumenter
from sphinx.ext.autodoc import MethodDocumenter as _MethodDocumenter
class PyCoroutineMixin(object):
"""Helper for coroutine-related Sphinx custom directives."""
def handle_signature(self, sig, signode):
ret = super(PyCoroutineMixin, self).handle_signature(sig, signode)
signode.insert(0, addnodes.desc_annotation('coroutine ', 'coroutine '))
return ret
class PyCoroutineFunction(PyCoroutineMixin, PyModulelevel):
"""Sphinx directive for coroutine functions."""
def run(self):
self.name = 'py:function'
return PyModulelevel.run(self)
class PyCoroutineMethod(PyCoroutineMixin, PyClassmember):
"""Sphinx directive for coroutine methods."""
def run(self):
self.name = 'py:method'
return PyClassmember.run(self)
class FunctionDocumenter(_FunctionDocumenter):
"""Automatically detect coroutine functions."""
def import_object(self):
ret = _FunctionDocumenter.import_object(self)
if not ret:
return ret
obj = self.parent.__dict__.get(self.object_name)
if iscoroutinefunction(obj):
self.directivetype = 'coroutine'
self.member_order = _FunctionDocumenter.member_order + 2
return ret
class MethodDocumenter(_MethodDocumenter):
"""Automatically detect coroutine methods."""
def import_object(self):
ret = _MethodDocumenter.import_object(self)
if not ret:
return ret
obj = self.parent.__dict__.get(self.object_name)
if iscoroutinefunction(obj):
self.directivetype = 'coroutinemethod'
self.member_order = _MethodDocumenter.member_order + 2
return ret
def setup(app):
"""Sphinx extension entry point."""
# Add new directives.
app.add_directive_to_domain('py', 'coroutine', PyCoroutineFunction)
app.add_directive_to_domain('py', 'coroutinemethod', PyCoroutineMethod)
# Customize annotations for anything that looks like a coroutine.
app.add_autodocumenter(FunctionDocumenter)
app.add_autodocumenter(MethodDocumenter)
# Return extension meta data.
return {
'version': '1.0',
'parallel_read_safe': True,
}

Elixir NIF- Hello World example on x64 Mac OSX

Hi I'm trying to get the Hello World example for Erlang NIF (Native Implemented Function) shown here
http://www.erlang.org/doc/man/erl_nif.html
to work from Elixir on OSX 64bit.
First I create the C-code:
/* niftest.c */
#include "erl_nif.h"
static ERL_NIF_TERM hello(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
{
return enif_make_string(env, "Hello world!", ERL_NIF_LATIN1);
}
static ErlNifFunc nif_funcs[] =
{
{"hello", 0, hello}
};
ERL_NIF_INIT(niftest,nif_funcs,NULL,NULL,NULL,NULL)
Then I successfully compile it using gcc for 64 bit architecture as suggested here
Erlang NIF Test -- OS X Lion
gcc -undefined dynamic_lookup -dynamiclib niftest.c -o niftest.so -I /usr/local/Cellar/erlang/R14B02/lib/erlang/usr/include
which produces the necessary file niftest.so that I should be able to interface-into from Erlang/Elixir. My Elixir (niftest.ex) looks like this (inspired by a more complex example reported here):
defmodule Niftest do
#onload :init
def init() do
:erlang.load_nif("./niftest", 0)
:ok
end
def hello() do
"NIF library not loaded"
end
end
Now with niftest.so and niftest.ex in the same directory I fire up elixir using iex and type in Niftest.hello and all I get back is: "NIF library not loaded"
Am I missing a vital step? - please help!
The load of the library is failing silently. You can assert that it succeeds using:
:ok = :erlang.load_nif("./niftest", 0)
This results in an error:
** (MatchError) no match of right hand side value:
{:error, {:bad_lib, 'Library module name \'niftest\' does not match calling module \'\'Elixir.Niftest\'\''}}
niftest.ex:4: Niftest.init/0
That happens because a NIF lib can only be called from its "owning" module. The name of that module is the first argument to the ERL_NIF_INIT macro, so you can fix this by changing that call and recompiling:
ERL_NIF_INIT(Elixir.Niftest,nif_funcs,NULL,NULL,NULL,NULL)
There is also a typo in the load hook. It should be:
#on_load :init

ruby query interface not working?

I'm having problems with the WIN32OLE#ole_query_interface function.
I use the same source code, as mentioned in the doc of the function:
ie = WIN32OLE.new('InternetExplorer.Application')
ie_web_app = ie.ole_query_interface('{0002DF05-0000-0000-C000-000000000046}')
puts "ie_web_app: = #{ie_web_app.ole_type}"
and the output is:
# => ie_web_app: = IWebBrowser2
but I'd expect the output to be IWebBrowserApp. What am I missing?
BTW: I'm working on a 64bit Win7 and I have now used 2 interpreters to test:
ruby 2.0.0 p353 (2013-11-22) [x64-mingw32]
ruby 1.9.3 p484 (2013-11-22) [i386-mingw32]
here's the coclass from the tlb for reference:
uuid(0002DF01-0000-0000-C000-000000000046), helpstring("Internet Explorer Application.") ] coclass InternetExplorer {
[default] interface IWebBrowser2;
interface IWebBrowserApp;
[default, source] dispinterface DWebBrowserEvents2;
[source] dispinterface DWebBrowserEvents; };
and the relevant part of IWebBrowserApp:
[
uuid(0002DF05-0000-0000-C000-000000000046),
helpstring("Web Browser Application Interface."),
hidden,
dual
]
dispinterface IWebBrowserApp {
The coclass says it implements IWebBrowser2 as its default interface. I suspect that ruby's .ole_type method is using that to generate a printable name. Can you call IWebBrowserApp methods on the resultant interface? If so, it's an IWebBrowserApp, regardless of what gets printed.

CLR IL-significance of square bracket on .locals init

I am trying to generate a dynamic assembly using Reflection & Emit in .NET. I am getting an error, "Common Language Runtime detected an invalid program." I created another program which has the functionality I want using hard-coded types. The functionality I am trying to write will ultimately use dynamic types, but I can use ILDasm to see the IL I need to generate. I am comparing the IL I am generating with the IL which the compiler generates. In the .locals init declaration of one method I see there is an extra item in the compiler-generated code,
compiler-generated:
.locals init ([0] class [System.Core]System.Linq.Expressions.ParameterExpression CS$0$0000,
[1] class [System.Core]System.Linq.Expressions.ParameterExpression[] CS$0$0001)
mine:
.locals init (class [System.Core]System.Linq.Expressions.ParameterExpression V_0,
class [System.Core]System.Linq.Expressions.ParameterExpression[] V_1)
I don't understand the significance of the "[0]" and "[1]" in the compiler-generated code. Can anyone tell me what it means?
As a more general question, I can follow most ILDasm output without too much trouble. But every so often I run across a problematic expression. For instance, in this line from ILDasm
callvirt instance class [EntityFramework]System.Data.Entity.ModelConfiguration.EntityTypeConfiguration`1<!!0> [EntityFramework]System.Data.Entity.DbModelBuilder::Entity<class DynamicEdmxTrial.HardFooAsset>()
the "!!0" probably refers to the generic type of the Entity<>, but I don't know for sure, and I wonder if there is a key to ILDasm output that would explain its more obscure output to me.
The specification is freely available here. It takes a little getting used to, but most details are easily found once you figure out the structure.
!! is listed in II.7.1 Types:
Type ::= | Description | Clause
‘!’ Int32 | Generic parameter in a type definition, | §II.9.1
| accessed by index from 0 |
| ‘!!’ Int32 | Generic parameter in a method | §II.9.2
| definition, accessed by index from 0 |
...
In other words, inside a method that C# would call f<T, U>(), !!0 would be T, and !!1 would be U.
However, the [0] is a good question. The spec does not seem to address it. The .locals directive is described in II.15.4.1.3 The .locals directive, which lists the syntax as
MethodBodyItem ::= ...
| .locals [ init ] ‘(’ LocalsSignature ‘)’
LocalsSignature ::= Local [ ‘,’ Local ]*
Local ::= Type [ Id ]
There is nothing that seems to allow [0] there unless it is part of a Type, and Type does not allow anything starting with [ either. My guess is that this is an undocumented peculiarity specific to Microsoft's implementation, intended to help the human reader see that location 0 is local variable CS$0$0000, for when the generated instructions access local variables by index.
Experimenting with ILAsm shows that this is exactly what it means. Taking a simple C# program:
static class Program {
static void Main() {
int i = 0, j = 1;
}
}
and compiling and then disassembling it (csc test.cs && ildasm /text test.exe >test.il) shows:
....
.locals init (int32 V_0,
int32 V_1)
IL_0000: nop
IL_0001: ldc.i4.0
IL_0002: stloc.0
IL_0003: ldc.i4.1
IL_0004: stloc.1
IL_0005: ret
....
Modifying the .locals to
.locals init ([0] int32 V_0, [0] int32 V_1)
gives a useful warning message:
test.il(41) : warning : Local var slot 0 is in use
And indeed, declaring variables of different types, then reordering them using [2], [1], [0], assembling and immediately disassembling the result, shows that the variables got reordered.

How to stop ImageMagick in Ruby (Rmagick) evaluating an # sign in text annotation

In an app I recently built for a client the following code resulted in the variable #nameText being evaluated, and then resulting in an error 'no text' (since the variable doesn't exist).
To get around this I used gsub, as per the example below. Is there a way to tell Magick not to evaluate the string at all?
require 'RMagick'
#image = Magick::Image.read( '/path/to/image.jpg' ).first
#nameText = '#SomeTwitterUser'
#text = Magick::Draw.new
#text.font_family = 'Futura'
#text.pointsize = 22
#text.font_weight = Magick::BoldWeight
# Causes error 'no text'...
# #text.annotate( #image, 0,0,200,54, #nameText )
#text.annotate( #image, 0,0,200,54, #nameText.gsub('#', '\#') )
This is the C code from RMagick that is returning the error:
// Translate & store in Draw structure
draw->info->text = InterpretImageProperties(NULL, image, StringValuePtr(text));
if (!draw->info->text)
{
rb_raise(rb_eArgError, "no text");
}
It is the call to InterpretImageProperties that is modifying the input text - but it is not Ruby, or a Ruby instance variable that it is trying to reference. The function is defined here in the Image Magick core library: http://www.imagemagick.org/api/MagickCore/property_8c_source.html#l02966
Look a bit further down, and you can see the code:
/* handle a '#' replace string from file */
if (*p == '#') {
p++;
if (*p != '-' && (IsPathAccessible(p) == MagickFalse) ) {
(void) ThrowMagickException(&image->exception,GetMagickModule(),
OptionError,"UnableToAccessPath","%s",p);
return((char *) NULL);
}
return(FileToString(p,~0,&image->exception));
}
In summary, this is a core library feature which will attempt to load text from file (named SomeTwitterUser in your case, I have confirmed this -try it!), and your work-around is probably the best you can do.
For efficiency, and minimal changes to input strings, you could rely on the selectivity of the library code and only modify the string if it starts with #:
#text.annotate( #image, 0,0,200,54, #name_string.gsub( /^#/, '\#') )

Resources