ruby native extension: undefined symbol - ruby

I'm attempting to create a ruby native extension, but when I run rake which uses ext/example_project/extconf.rb to build my project and run my tests under test/, I get the following error when the tests are run:
./home/jbuesking/.rbenv/versions/2.3.0/bin/ruby: symbol lookup error:
/home/jbuesking/repositories/example_project/lib/example_project/example_project.so: undefined symbol: some_function
I'm pretty sure my files are not being linked correctly and that I need to alter my extconf.rb and/or Rakefile in some way, but I'm not sure how.
I've created a simple repository that demonstrates the issue over on GitHub. It'll fail with the same error if you clone it and run rake from the projects root.
Some additional information:
I used the ruby gem hoe to create the project via sow example_project
The failing function is attempting to call a function defined in the subdirectory ext/example_project/c_example_project. My actual project uses a git submodule from the ext/example_project directory, which in turn sets up the submodule as a subdirectory. The submodule is a c project with a flattened structure (all files in the root directory). Note: That wording may be confusing, but the key point is that there's a nested c project defined at ext/example_project/c_example_project which has methods I'm trying to call.
Let me know if any clarification is needed, and I'll do my best to provide it.

So, there are some interesting issues you have here. By default, mkmf doesn't actually support specifying multiple directories for building sources.
There is a workaround, as seen here (Takehiro Kubo's comment about setting objs):
https://www.ruby-forum.com/topic/4224640
Basically, you construct the $objs global in your extconf.rb file yourself.
Using your github code, here's what I added to the extconf.rb and got to work
extconf.rb
globs = [".", "c_example_project"].map do |directory|
File.join(File.dirname(__FILE__), directory)
end.join(",")
$objs = Dir.glob("{#{globs}}/*.c").map do |file|
File.join(File.dirname(file), "#{File.basename(file, ".c")}.o")
end
Notice I'm actually constructing an absolute path to each of the c sources, for some reason rake-compiler was freaking out if we were just globbing with {.,c_example_project}/*.c, presumably since it's running the extconf.rb file from another directory.
In addition, your tests/c extensions have a few errors in them. Making the following change in example_project.c fixes the test failure:
static VALUE example_project_c_code_function()
{
- return some_function();
+ VALUE _string = rb_str_new2(some_function());
+ int _enc = rb_enc_find_index("UTF-8");
+ rb_enc_associate_index(_string, _enc);
+ return _string;
}
Explanation
Basically even though you're checking the c_example_project.h header in your extconf.rb, you're not actually generating the object file where some_function is defined. So, when linking the final dynamic library that ruby loads up, there's no definition for some_function and you get your error.

I don't have any experience with building native extensions, but from mkmf source code it looks like you can only specify one source directory. I moved both files from c_example_project to the parent directory and everything was linked properly. I think that is how you should do it. All common gems (like pg, nokogiri etc) have such code structure, all *.c and *.h files are in one directory.
You can always create Makefile yourself, but that would require too much work to maintain.
PS. Your project compiled successfully, but you there is segmentation fault, because you should return proper ruby string object in some_function and not pointer to a string.

photoionized has a great answer, but $obj can be an array instead of an Enumerator.It seems to be okay to simply use an absolute path.
$objs = Dir.glob([".c", "libfoobar/*.c"], base: __dir__)
.map { |f| File.expand_path(f, __dir__) }
.map { |f| f.sub(/\.c$/, ".o") }

Related

Puppet: generate statement fails when trying to retrieve default path of an executable

I have built a stanza to remove a ruby gem package from our servers. The problem is that the ruby gem executable is installed in different paths on the servers, so on one server it could be in /opt/ruby/bin/gem on other servers it's in /usr/local/rvm/rubies/ruby-2.0.0-p353/bin/gem
My stanza uses the generate function in puppet to pull out the default ruby gem installation as follows:
$ruby_gem_location = generate('which', 'gem')
exec { "remove-remote_syslog":
command => "gem uninstall remote_syslog",
path => "$ruby_gem_location:/opt/ruby/bin:/usr/bin:/usr/sbin",
onlyif => "$ruby_gem_location list|grep remote_syslog"
}
When I run puppet agent I get the following error:
Generators must be fully qualified at ****redacted*
I have also tried to provide a default path for the which command as follows:
$ruby_gem_location = generate('/usr/bin/which', 'gem')
and now the error says : Could not evaluate: Could not find command '/usr/bin/gem
I checked the target server and the gem command is in
/usr/local/rvm/rubies/ruby-2.0.0-p353/bin/gem
What am I doing wrong?
How can I pull out the default ruby gem location on our servers?
Thank you in advance
Your code
$ruby_gem_location = generate('/usr/bin/which', 'gem')
will generate a full path to your gem command (if it succeeds). From the result you describe, I think it is generating '/usr/bin/gem', which is perhaps a symlink to the real gem command. You are putting that into your command path instead of just the directory part, and that will not be helpful. It is not, however, the source of the error message you report.
The real problem here is that generate(), like all DSL fucntions, runs during catalog building. I infer from your results that you are using a master / agent setup, so generate() is giving you a full path to gem -- evidently /usr/bin/gem -- on the master. Since the whole point is that different servers have gem installed in different places, this is unhelpful. The actual error message arises from an attempt to execute your onlyif command with the wrong path to gem.
Your best way forward is probably to create a custom fact with which each node can report the appropriate location of the gem binary. You can then use that fact's value in your Exec, maybe:
exec { "remove-remote_syslog":
command => "$::ruby_gem_path uninstall remote_syslog",
onlyif => "$::ruby_gem_path list | grep remote_syslog"
}
Note that you don't need a path attribute if you give a complete path to the executable in the first place.
Details on creating the $::ruby_gem_path custom fact depend on a number of factors, and in their full generality they are rather too broad for SO, but PL provides good documentation.

Creating Ruby Main (command line utility) program with multiple files

I am trying to use the main gem for making command line utilities. This was presented in a recent Ruby Rogues podcast.
If I put all the code in one file and require that file, then rspec gives me an error, as the main dsl regards rpsec as a command line invocation of the main utility.
I can break out a method into a new file and have rspec require that file. Suppose you have this program, but want to put the do_something method in a separate file to test with rspec:
require 'main'
def do_something(foo)
puts "foo is #{foo}"
end
Main {
argument('foo'){
required # this is the default
cast :int # value cast to Fixnum
validate{|foo| foo == 42} # raises error in failure case
description 'the foo param' # shown in --help
}
do_something(arguments['foo'].value)
}
What is the convenient way to distribute/deploy a ruby command line program with multiple files? Maybe create a gem?
You are on the right track for testing - basically you want your "logic" in separate files so you can unit test them. You can then use something like Aruba to do an integration test.
With multiple files, your best bet is to distribute it as a RubyGem. There's lots of resources out there, but the gist of it is:
Put your executable in bin
Put your files in lib/YOUR_APP/whatever.rb where "YOUR_APP" is the name of your app. I'd also recommend namespacing your classes with modules named for your app
In your executable, require the files in lib as if lib were in the load path
In your gemspec, make sure to indicate what your bin files are and what your lib files are (if you generate it with bundle gem and are using git, you should be good to go)
This way, your app will have access to the files in lib at runtime, when installed with RubyGems. In development, you will need to either do bundle exec bin/my_app or RUBYLIB=lib bin/my_app. Point is, RubyGems takes care of the load path at runtime, but not at development time.

Setting up rake-pipeline for use with handlebars alongside Google App Engine

So here's what I'm attempting to do. I'm building an ember.js application, with a java backend running on GAE.
I'm using handlebars, but I want them divided up into separate files, not just all pasted into the index.html.
Via the ember.js irc I was turned on to rake-pipeline along with minispade
Along with the web filters and a custom handlebars filter I started building the assetfile. I don't know Ruby, or gem files, etc.
So I'm trying to figure out the best way to be able to compile my coffeescript/handlebars files on the fly, minispade them, but keep the individual files accessible while in dev mode so I can debug them. What makes that hard is that the rake pipeline is running on a different port than GAE. So I'm not sure exactly how to handle this. Do I make my index file in GAE point to individual files at the 9292 port (rakep) during development, but in production mode point to the fully concatenated version? I'm not sure.
So I was attempting to do that here: https://gist.github.com/1495740 by having only one section that was triggered by the 'build' flag. Not even sure if that works that way.
I know there's a lot of confusion here. Apologies, like I said I'm not even remotely familiar with the Ruby style of doing things.
Since you're not a Ruby person, here are the most reliable steps for getting a stock OSX environment set up with rake pipeline:
Step 1: Install bundler
# on OSX, using built-in Ruby
$ sudo gem install bundler --pre
Step 2: Create a Gemfile
# inside your app directory
$ bundle init
# will create a file named Gemfile in the root
Step 3: Add rake-pipeline to the Gemfile
# inside the Gemfile
gem "rake-pipeline-web-filters"
Step 4: Install your gems
$ bundle install --binstubs
Step 5: Set up Assetfile
However you were already doing it...
Step 6: Run Rake::Pipeline
# to run the preview server
$ bin/rakep
# to build your assets
$ bin/rakep build
Rake::Pipeline.build is the method that evaluates an Assetfile. You can imagine that your entire Assetfile is wrapped inside a Rake::Pipeline.build {} block; you shouldn't ever need to write one inside an Assetfile.
Some of the filters in the docs are hypothetical, most of those docs were written before there were any filters at all. A CoffeeScript compiler has been recently added, though.
As to your main question, I'm not sure there's a clean way to do it with the current rakep implementation. An Assetfile is just Ruby, though, so it's possible to hack something together that should work. Here's how I would write yours:
require "json"
require "rake-pipeline-web-filters"
require "rake-pipeline-web-filters/helpers"
class HandlebarsFilter < Rake::Pipeline::Filter
def initialize(&block)
block ||= proc { |input| input.sub(/\.handlebars$/, '.js') }
super(&block)
end
def generate_output(inputs, output)
inputs.each do |input|
output.write "return Ember.Handlebars.compile(#{input.read.to_json})"
end
end
end
# process all js, css and html files in app/assets
input "assets"
# processed files should be outputted to public
output "public"
# process all coffee files
match "**/*.coffee" do
# compile all CoffeeScript files. the output file
# for the compilation should be the input name
# with the .coffee extension replaced with .js
coffee_script
# The coffee_script helper is exactly equivalent to:
# filter Rake::Pipeline::Web::Filters::CoffeeScriptCompiler
end
match "**/*.js" do
minispade
if ENV['RAKEP_ENV'] == "production"
concat "application.js"
else
concat
end
end
match "**/*.handlebars" do
filter HandlebarsFilter
minispade
concat "templates.js"
end
The if ENV['RAKEP_ENV'] bit reads an environment variable to decide whether to concatenate your JS to a single file.
So now you can run RAKEP_ENV="production" rakep build for a concatenated build, or just rakep build for a development build.

Ruby separate large source files into multiple files

I am writing a Ruby script which was supposed to be a small thing but has grown quite large, way to large to have everything crammed into one source file. So I am trying to separate the project into different files. I have four classes and I want to put each in its own separate source file.
What I did:
I moved all of the classes into their own files so now I have this
proj/GoogleChart.rb
proj/BarChart.rb
proj/PieChart.rb
proj/GroupedBarChart.rb
Now that they are in other files I am getting uninitialized constant GoogleChart (NameError) in all of my subclasses on the line where I inherit from GoogleChart, i.e.
require 'GoogleChart'
BarChart < GoogleChart
Can anyone tell me what is wrong?
Thanks
EDIT
Using ruby version 1.8.4
Also I have tried using the absolute path:
require 'C:/Documents and Settings/proj/GoogleChart.rb' and this is still producing a NameError
In Ruby 1.8.x, the . is part of your load path. So you should at least try to debug that by including something like:
puts $:
require 'GoogleChart'
class BarChart < GoogleChart
end
and load that in an IRB session:
Open the session in your directory proj.
Enter there require 'BarChart'
Look at the result.
For me it is:
c:\apps\ruby\test\proj>irb
irb(main):001:0> require 'BarChart'
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/site_ruby/1.8
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/site_ruby/1.8/i386-msvcrt
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/site_ruby
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/vendor_ruby/1.8
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/vendor_ruby/1.8/i386-msvcrt
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/vendor_ruby
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/1.8
C:/Users/mliebelt/.pik/rubies/Ruby-187-p334/lib/ruby/1.8/i386-mingw32
.
=> true
So the require is successful for me, and the . is part of the path (as it should). As you can see, I am working with Ruby 1.8.7, I don't know if anything has changed since 1.8.4 that is relevant here.
So please describe exactly how you run your file:
Have you opened a shell to run the file?
What is the current working directory of that shell?
Do you run by double-clicking it?
It only works when you are in your proj directory and run there (with ruby in your shell path) ruby BarChart.rb.

Ruby gem testing before deployment

I'm creating a gem which has
several scripts in the bin directory
the utility classes in the lib directory
and several tests in the test directory
supertool
bin
toolA
toolB
lib
supertool
supertool.rb
helper.rb
test
tc_main.rb
tc_etc.rb
Now, to run the tests before I even install the gem, I have the following snippet at the top of my tests:
base = File.basename(Dir.pwd)
if base == 'test' || base =~ /supertool/
Dir.chdir('..') if base == 'test'
$LOAD_PATH.unshift(Dir.pwd + '/lib')
Dir.chdir('test') if base =~ /supertool/
end
This seems tedious though, especially if I have to put these in the scripts in the bin directory too. Is there a better way of setting up the environment so we can test gems before they are installed? I'm sure it's something simple that I just can't find. A simple link to the right place would help a lot :)
I'm not sure what you're trying to achieve with that script. It doesn't seem to have anything to do with gems...
Is it so that you can run ruby tc_main.rb from within the test directory (or ruby test/tc_main.rb from the base dir), and have it set the load path appropriately? If so, here's a much nicer way:
In your test directory, create a test_helper.rb file. In that file, put this
$LOAD_PATH << File.expand_path( File.dirname(__FILE__) + '/../lib' )
And in all your test files, set the first line to
require 'test_helper'
If you have subdirectories inside your test dir, then files in those subdirs can just do
require '../test_helper'
Take a look at hoe gem, it is a helper for other gems.

Resources