I have a Ruby IronWorker which depends on a private gem that isn't published to RubyGems.
Is there a way to merge this local mygemname-0.0.1.gem into my IronWorker in the .worker file?
I'm hoping to be able to specify something the following in myruby.worker:
gem 'mygemname', '>=0.0.1', :path=> 'vendor/bundle'
Currently this give the following error
.rvm/gems/ruby-1.9.3-p0/gems/iron_worker_ng-0.12.2/lib/iron_worker_ng/code/base.rb:79 :in `eval':
wrong number of arguments (3 for 2) (ArgumentError)
Hoping for defaults gives:
gem 'mygemname', '>=0.0.1'
Gives the following error
Could not find gem 'mygemname (>= 0.0.1) ruby' in the gems available on this machine.
Am I on the right track trying to get this to work via the .worker file? Or should I be looking into specifying a custom build step?
If your unpublished gem itself has dependancies, you need to do a little massaging to get things going. Here is a technique that works for me:
mygem.worker
runtime "ruby"
#Merge in an unpublished local gem
dir '../opensource-cli-tools/facebook_exporter', '__gems__/gems'
file '../opensource-cli-tools/facebook_exporter/mygem.gemspec', '__gems__/specifications'
#Merge in a custom build script to fetch the unpublished gem's dependancies
file "Gemfile"
file "install_dependancies.sh"
remote_build_command 'chmod +x install_dependancies.sh && ./install_dependancies.sh'
#Run the puppy!
exec "run.rb"
install_dependancies.sh
echo "Installing dependancies to __gems__/"
gem install bundler --install-dir ./__gems__ --no-ri --no-rdoc
bundle install --standalone --path ./__gems__
cp -R ./__gems__/ruby/*/* ./__gems__
rm -rf ./__gems__/ruby
echo "Fixing install location of mygem"
mv ./__gems__/gems/mygem ./__gems__/gems/mygem-0.0.1
As far as i know, git and local paths unsupported right now.
Here is way to manually include local gem:
Add these lines to .worker file:
dir '../vendor/bundle/mygemname', '__gems__/gems'
file '../vendor/bundle/mygemname/mygemname.gemspec', '__gems__/specifications'
Related
I am building a Ruby 2.7 Lambda application.
My application depends on a gem that exists in github.
gem 'my-gem', git: 'https://github.com/my-org/my-gem', branch: 'main'
I want to write a script that can build a deployment zip file containing this dependency.
When I run bundle install, my gem is installed to vendor/bundle/ruby/2.7.0/bundler/gems/my-gem-GITHASH.
For Lambda packaging, I believe that I need construct the following
vendor/bundle/ruby/2.7.0/gems/my-gem-1.0.0/* (ruby code)
vendor/bundle/ruby/2.7.0/specifications/my-gem-1.0.0.gemspec
The following scripted actions can assemble this structure, but I wish I had a simpler approach.
cd vendor/bundle/ruby/2.7.0/bundler/gems/my-gem-*
# build the git gem (*.gem)
gem build
# copy the gem and the gemspec to the vendor/bundle/ruby/2.7.0 directories
cp *.gem ../../../gems
cp *.gemspec ../../../specifications/my-gem-1.0.0.gemspec
# upack the .gem file in the proper directory
cd ../../../gems
gem unpack *.gem
# return to the working directory
cd ../../../../..
# Zip the dependencies
zip -r deploy.zip \
vendor/bundle/ruby/2.7.0/gems \
vendor/bundle/ruby/2.7.0/specifications \
vendor/bundle/ruby/2.7.0/extensions \
lib
I would be curious to find a simpler approach to this problem.
Running bundle install --deployment should do what you want (documentation page).
Update: It seems that the AWS docs use bundle install --path vendor/bundle.
An alternative solution is to add the bundler/gems path to the $LOAD_PATH, like so:
load_paths = Dir['./vendor/bundle/ruby/2.7.0/bundler/gems/**/lib']
$LOAD_PATH.unshift(*load_paths)
(at the very top of lambda_function.rb)
I'm developing a couple of private gems and I think I don't understand correctly the PATH/GEM_PATH and/or Bundler/RVM installation flow, would love if someone could chip in.
I have a repository with two gems (A & B for simplicity sake). I've developed the gems using the scaffolding + following the guidelines provided by this bundler tutorial.
Thanks to the Bundler project I have a few Rakefile tasks like rake build, rake install, rake install:local and rake release. Because of the private nature of these gems I can't release them to RubyGems (and we haven't looked into hosting our rubygems).
My machines are using RVM to manage ruby versions and Bundler version 1.15.1
What I want to do: Assuming a new machine/developer trying out the project, ideally we would cd into each of the subfolders (currently 2, gem A and gem B), run rake install and after that we should have the gems available system wide for the current user.
What is happening: The gems are built and work properly, but they are only available inside the subfolder of each gem i.e. gem A is only available inside the subfolder A and gem B is only available inside subfolder B.
What I've tried: So, after rake build/install/install:local a new .gem file is generated under pkg. I've tried to manually install the "compiled" file using gem install pkg/A.gem, gem install --local pkg/A.gem and gem install --local --user-install pkg/A.gem without success. (there are plenty of SO questions/answers about this)
I believe this has something to do with the PATH variables, but like I said before I don't fully understand the way they are managed. I get the following results from these commands:
# Our gem
> gem which A
/home/ubuntu/.rvm/gems/ruby-2.4.0/gems/A-0.1.8/lib/A.rb
# Pry, available globally
> gem which pry
/home/ubuntu/.rvm/gems/ruby-2.4.0/gems/pry-0.11.1/lib/pry.rb
I've been lost and frustrated for far too long now, any help is appreciated. Also open to hear suggestions of better private gem installation flows :)
Yes, it has something to do with your PATH variables. Your installation seems to be good.
I advise you to first affirm your gems installation path with:
echo $GEM_HOME
The double check your PATH to ensure its present and also confirm that the GEM home is also where the gem got installed into from the rake install
echo $PATH
If not, put it in your path and you should be fine with something like this:
echo PATH=$PATH:$GEM_HOME >> ~/.bashrc
source ~/.bashrc
Build your gem as per that guide you linked. You should end up with a gem file. Distribute this as you see fit (I use rsync/crontab to download newer gem versions but anything goes). User can install the gem as follows:
gem install --user-install /path/to/your/file.gem
This will install the gem in the user's ~/.gem/ruby/<version>/gems/<your-gem-name> directory.
Tried it with an empty gem (foodie, as in that example guide) and it works fine. But if you don't specify the --user-install parameter it will try to install in the system ruby dir (/usr/lib/ruby/gems...)
Ruby dabbler/newbie here who's not familiar with the ecosystem, so apologies if this is one of those super duh questions.
Is there a way to view all the files and/or source code installed by a gem? That is, I just ran
$ gem install sass
And the sass gem is now a part of my local system
$ gem list --local
...
sass (3.1.16, 3.1.2)
...
I want to know what the gem install command put on my system. Is there a command I can run to see all the files installed by the gem?
After some googling, man gem and gem help commands, I discovered the contents command.
$ gem contents sass
However, when I run this command with the aforementioned sass gem, I get the following results
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/engine_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/functions_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/extend_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/logger_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/css2sass_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/conversion_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/script_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/util/subset_map_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/util/multibyte_string_scanner_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/callbacks_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/importer_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/scss/css_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/scss/scss_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/scss/rx_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/util_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/script_conversion_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/less_conversion_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/cache_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/test/sass/plugin_test.rb
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/bin/sass
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/bin/sass-convert
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.16/bin/scss
However, this list seems incomplete as I know there are files in
.../.rvm/gems/ruby-1.9.2-p180/gems/sass-3.1.2/lib/
Why does contents not show the files from lib?
Is it possible for a gem installer to install files outside of the gems folder?
Is there a command that can show everything installed by a gem?
gem has an unpack command: http://guides.rubygems.org/command-reference/#gem-unpack
gem unpack rake
ls rake-0.4.3/
There are two really good ways to do this. There is another gem which allows you to open the gem and edit. This gem is call gem-open
gem install gem-open
then
gem open sass
Another way is to generate your own rdocs.
gem rdoc sass
You can then look at your rdocs by
gem server
Also if you are using rvm, you can type rvm info and it will show GEM_HOME location.
This will be where all your gems source code is.
cd $GEM_HOME
cd gems/sass-3.1.2/
Update:
This is the way I mostly do this now, when using bundler.
cd $(bundle show sass)
This will be the version of sass in your Gemfile.
I usually open a gem by running this command from the console
EDITOR=<your editor> bundle open <name of gem>
The lib/ directory you mentioned is for version 3.1.2 of the gem; gem contents without --version will just list one version (it appears to pick the newest version, but I'm unable to verify this is always true). What output do you get for gem contents --version 3.1.2 sass?
You can also use just rename the .gem file to .tar and extract as a posix archive. The source code is inside it in the lib folder. See https://blog.srcclr.com/extracting-ruby-source-code-from-gem-packages/ for more details.
In addition to gem contents, another command you might find useful is gem environment. If you have multiple paths for your gem installations, they will all be listed under the "GEM PATHS" label.
I have a Rails app with some non-Rails-dependent files under `lib/services'. One of these files uses the Domainatrix gem.
require "domainatrix"
class SuggestionParser
# various suggestion parsing methods
end
I have an empty spec for this file under spec/lib.
require "services/suggestion_parser"
describe SuggestionParser do
end
Unfortunately, when I try to run that spec without bundle exec I hit an error:
$: rspec spec/lib/services/suggestion_parser_spec.rb
-> /Users/davidtuite/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:55:in `require': cannot load such file -- domainatrix (LoadError)
Every other spec and gem in my project will run without using bundle exec. Why do I need to prefix this one in order to get it to run?
For convenience, here's a link to the Domainatrix gemspec.
My guess would be that domainatrix is declared using the :path or :git options in the Gemfile, neither of which install the gem in a way that makes it accessible to rubygems.
This could be confirmed if you post the line for domainatrix from the Gemfile.
try running the following commands:
$ rvm get head && rvm reload
$ chmod +x $rvm_path/hooks/after_cd_bundler
$ bundle install --without production --binstubs=./bundler_stubs
This won't solve the specific problem with your gem, but it will take away the necessity to type in bundle exec every time you run your tests if you're using rvm.
I'm trying to develop a gem locally, and have installed it with Bundler.
My Gemfile looks like this:
source "http://rubygems.org"
gemspec
And my gemspec is a standard gemspec file.
I can install the gem with 'bundle install' in the directory, and i see the local gem and all it's dependencies install:
bundle install
Using rack (1.3.4)
Using tilt (1.3.3)
Using sinatra (1.3.1)
Using {my gem} (0.0.2) from source at .
Using bundler (1.0.21)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
However, when I do a 'gem list', my gem is not included in the list of gems - which is my guess as to why my bin directory does not appear in the path. Is there a way to test a local gem and include it in the list of installed gems using bundler, so that the bin directory properly works?
Easiest way to get rid of bundler: command not found: {your bin executable}:
git add bin/* # git-ls-files will now list your bin executables.
bundle install
# No git-commit necessary.
bundle exec <MY_BIN_EXECUTABLE>
gem list shows your system installed gems, not the gems in your Bundle (this are often the same but not always--as in this case). When you're using Bundler, you should always execute gem executables with bundle exec so that Bundler can set up the environment for you. So, if you have a binary called, for example, mygem, you should use bundle exec mygem.
See more info at Bundler's site or in the manpage.
[Edit]
Also be sure that your gemspec includes a bin directory! Common convention is to create a directory called bin at the same level as your lib directory, put your binaries in there, and then add this as the directory in your gemspec. If you don't do this, Bundler won't expose your binaries!
I had this problem too.
Make sure the executables and default_executable lines don't contain 'bin/'. Then:
git add add . # You can be more precice if you want.
git commit -m "My lousy commit message."
bundle install
bundle exec <binaryname>