I am working on a Makefile to build an application where frontend and backend are completely separated in two different directories, like:
- myapp
|_be
|_fe
This is my Makefile:
.ONESHELL:
all: frontend backend
frontend:
cd ./fe && \
npm install && \
./node_modules/webpack/bin/webpack.js
backend: clean
cd ./be && \
gem install bundler && \
bundle install
clean:
rm -f ./fe/public/bundle.js && rm -rf ./fe/public/resources
webpack:
cd ./fe && \
.node_modules/webpack/bin/webpack.js
test:
cd ./fe && \
npm test
run:
cd ./be && \
rackup config.ru
The frontend is a React application and it is being built correctly when I type make frontend. No issues here.
But the backend is a Rack application written in Ruby. During the build process I need to enter its directory (be), make sure bundler is installed and run bundle install.
It happens that make creates its own shell to run the Makefile commands. And this shell does NOT inherit the environment variables of the console shell I am working. This turns out to be a problem to me, because I use rvm and rvm uses some environment variables to pick the correct gems. I use the .ruby-gemset and .ruby-version to create a private bundle to my application, not using the default gemset. This is necessary, because I have applications using many different Ruby versions and don't want to care about this.
This particular application uses JRuby 9.1.7.0 and then my Gemfile (inside be directory) has the following line:
ruby '2.3.1', :engine => 'jruby', :engine_version => '9.1.7.0'
I noticed all this issue because I got the following error when I typed make backend:
Your Ruby engine is ruby, but your Gemfile specified jruby
As far as I know rvm (and I know it quite well), this means it is trying to use my default Ruby version (which is set to Ruby 2.5.0) as engine, instead of the correct JRuby version.
Now, here is my question: Is there a way to force the make inner shell to use the same configuration of the console shell I am using?
This would solve the issue, because the console shell is already configured by rvm. I know it is because when I enter the be directory manually and run bundle install inside it, everything goes fine.
I used the .ONESHELL directive to force make to do this, but it seems it is not what I need.
Suggestions?
EDIT: make documentation says the -e flag may be used to force the make grab all console shell variables. I tried it and it didn't work.
EDIT: It all works if I do
rvm use --default jruby-9.1.7.0
before doing my make backend. But for many reasons it is not convenient to do this. Besides, I really believe this is a better way to do this. make is too smart not to allow something as simple as this.
I believe there are 2 problems:
1) when rvm creates its environment variables, it does not export them, so they are not visible to child shells, and
2) because the child shell is not started as a login shell, it does not see and process the rvm function declarations, and can therefore not execute rvm my-ruby-version.
Here are a couple of workarounds:
Before creating the child shell, do something like this:
export RVM_VERSION=`rvm current`
When starting the child shell, do so in login mode:
zsh --login
or
bash --login
Then, rvm should be a function, and this should work:
rvm $RVM_VERSION
First I added two lines to my backend: recipe, which now was reading
backend:
cd ./be && \
source $(HOME)/.rvm/scripts/rvm && \
rvm use jruby-9.1.7.0 && \
gem install bundler && \
bundle install
The idea was setting the correct Ruby (JRuby 9.1.7.0) without having to change it permanently at the console.
It happened this wouldn't work, because I wasn't choosing the shell I was using and it seems for some reason it wasn't /bin/bash and source wouldn't work. As a sequel, rvm wouldn't work either and I would receive that irritating message saying
rvm is not a function
Then I added
SHELL = /bin/bash
right after the .ONESHELL: directive.
And now everything worked fine!
Related
I have a Ruby Script that when I run in the shell it works (as user):
/home/user/wpscan/$ ruby ./wpscan.rb -u www.mysite.com
However, I'd like to automate this with a function that I created with Python. Here is the python script:
#!/usr/bin/python
import os
wpscan_env = "/wpscan/"
os.chdir(os.environ['HOME'] + wpscan_env)
os.system("ruby ./wpscan.rb -u www.mysite.com")
Notice that the Python script is in a different folder, home/user/python/first.py and this is why I do the os.chdir() function. When I go back to the shell and type:
/home/user/python/$ python first.py
This is the output I get:
Could not find addressable-2.4.0 in any of the sources
Run `bundle install` to install missing gems.
I am using Ubuntu 14.04 and in order to get wpscan to work, it asked me to install Ruby 2.3.0. I did this via RVM.
wpscan.rb has a few dependencies and it seems it's not instantiating them. Also if I'm inside the wpscan folder and do ruby ./wpscan.rb ... it will work. However if I try to do this from the home directory: ruby wpscan/wpscan.rb ... it throws an error:
[ERROR] cannot load such file --typhoeus
[TIP] try to run 'gem install typhoeus' or 'gem install --user-install typhoeus'. If you still get an error, Please see README file or https://github.com/wpscanteam/wpscan
I have no knowledge of Ruby, this is my first real Python script, and I just installed wpscan 2 nights ago. I'm way out of my league here and I need help. Any further question can be elaborated per request.
Most probably, ruby in your python script is not the same as ruby in your shell. This can happen if you e.g. have a default system ruby installed (e.g. from the system packages) and you install another ruby version via RVM.
When using shell, the RVM automatically loads it's environment from the shell init scripts (e.g. ~/.bashrc) and knows which ruby to use from its settings. Whereas your python script does not load any rvm environment (it's not a login shell) and calls the default system ruby.
In that case, you need to explicitly call the correct ruby from the RVM in your python script. You can to it by calling the RVM wrapper:
browse directories under ~/.rvm/wrappers/ and find the correct ruby version and gemset that you want to use
in your python script, call the ruby command from this wrapper directory instead of the plain `ruby, something like:
rvm_ruby = os.environ['HOME'] + "/.rvm/wrappers/ruby-2.3.0-p100#myproject/ruby"
os.system(rvm_ruby + " ./wpscan.rb -u www.mysite.com")
This should fix your problem.
Thats so simple to execute ruby scripts from python...
import os
os.system('ruby filename.rb')
#if you want to store the output in seperate file
os.system('ruby filename.rb > outfilename.txt)
I have my bundler configured to install gems into a .bundle/ directory inside each project.
$ cat ~/.bundle/config
---
BUNDLE_PATH: ".bundle"
BUNDLE_BIN: ".bundle/bin"
How can I make RVM respect this setting and not set the $GEM_HOME to another directory (inside ~/.rvm/) whenever I cd into my project?
So far I had a little script that would set $GEM_HOME (and $PATH and $GEM_PATH) whenever I would cd inside my project's directory, but with recent versions of rvm it has stopped working and now RVM basically unsets the Ruby version whenever I manually change $GEM_HOME and I have no clue how to make it work again.
First of all you should specify that you want to use the very separate gemset for your project (consider it’s name is myproject42):
$ cd myproject42 && rvm --rvmrc --create use 2.1.0#myproject42 --ignore-gemsets
rvmrc option will create the configuration file in your folder. You are already half-separated from the universe. Now you want to modify this file a bit, setting whatever you want:
...
unset __hook
# my exports
export GEM_PATH=`pwd`
export GEM_HOME=`pwd`
...
cd out and back in into this directory. This will result in the warning:
You are using '.rvmrc', it requires trusting, it is slower and ...
[LINES SKIPPED]
************************
y[es], n[o], v[iew], c[ancel]> y
Using: /tmp/myproject42
Once you confirm you are aware of this modification, the gemset is set to what you wanted (/tmp/myproject42 in this case.) You may verify that:
$ rvm gemset dir
Warning! PATH is not properly set up, '/tmp/myproject42/bin' is not available,
[LINES SKIPPED]
/tmp/myproject42
You might want to make some cleanup, like suppressing those warnings. The “howtos” are printed out inside this warnings, they are pretty straightfoward so I left the description of initial process “dirty” to show it exactly how it would be run on your machine.
I hope this helps.
I'm trying to understand how bundle exec works and what it does. I've installed the gems using bundle install like so:
bundle install --binstubs ./bundle/bin --path ./bundle/lib'
This creates a script ./bundle/bin/thin that I can use to start my Rails application using thin like so:
./bundle/bin/thin start -p 8080
However I see most articles on the internet recommend starting thin using bundle exec like this:
bundle exec thin start -p 8080
What is the difference between the two? My tests show that bundle exec doesn't call the ./bundle/bin/thin script, so how does bundle exec differ from the script?
There's no significant difference: they're two ways to accomplish the same thing, which is to run the correct version of the command for your bundle, with the environment set up to ensure that other bundled gens can be loaded by the command. The choice comes down to a matter of convenience.
The benefit of bundle exec is that you don't need to generate binstubs to use it: it just works with the existing Gemfile. This explains why you don't see it invoking the binstub you do have.
Some people don't like having to type bundle exec before every command, so the goal with binstubs is that you can add the directory to the front of your PATH and call the command normally. The drawback is that there is a potential security or usability concern if a bundled gem contains a command that shadows an important system command (e.g., ls).
If you don't put it in your PATH and always call it as bundle/bin/thin, you don't have the security concern, but it also gives you no particular benefit over using bundle exec thin.
In that specific case, there is no difference. bundle exec thin start -p 8080 will end up calling .bundle/bin/thin, but what if you you installed binstubs in a different path? bundle exec thin start will read your .bundle/config to find where your binstubs folder is. If you don't have binstubs installed and say you have 3 versions of thin installed, bundle exec will execute the one that's defined in your Gemfile.
Edit: #tadman also made a good point that I initially missed. When you use bundle exec, the gem environment from your Gemfile will be used, without it, it'll load the latest version of each gem currently installed.
The bundle exec method is the "official" way to do it. If you want to make use of the --binstubs feature, which is unusual, you can.
Remember that ./bundle/bin/thin is not the script, but a wrapper or "stub" that calls the script with the proper bundle exec environment loaded.
I am writing a Ruby script designed to run from the command line. The script has a corresponding RSpec file that verifies its functionality. The folder structure is:
./main_script.rb
./spec/main_script_spec.rb
Running rspec spec in the top level directory works as expected. Test results from the ./spec/main_script_spec.rb file are shown. I'd like to avoid running this manually every time I change either the main script file or the spec file. All my search results turn up things like guard which (as far as I can tell) are all designed for Rails apps.
How do I setup RSpec to watch for script or spec changes and run automatically with non-Rails Ruby code?
Like David said, Guard can be used to watch a wide variety of files and perform actions when those files are modified. It does not have to be used with a Rails app. I have set up something similar in the past using guard. Here is what I did:
Place the following in your Gemfile:
source 'https://rubygems.org'
gem 'guard'
gem 'guard-shell'
gem 'rspec'
gem 'rb-fsevent', '~> 0.9'
Then run:
$ bundle install
Create a Guardfile in your home directory with:
$ guard init
In the Guardfile, comment out the examples and add this:
guard :shell do
watch(%r{^*\.rb}) { `bundle exec rspec spec/` }
end
This tells guard to watch for modifications to any ruby files in the directory and execute the command bundle exec rspec spec/ when they change (the backticks are used to execute the command in the shell).
Then open up a new terminal window in your current directory and start a guard server to start watching the files:
$ bundle exec guard
Now your Rspec test suite should automatically run when you modify ruby files in the directory.
I used guard at the past, but now I'm using a combination of rspec focus feature and watch command.
It's very simple, just add an f before a describe of it block you want to run the test. So it would becomes fdescribe or fit block. This is the same as adding a tag :focus => true to your block.
We can then filter specs with the focus tag: rspec -t focus
Now, to keeping running theses specs (every 0.5 seconds) with focus tag we call it with watch command:
watch -n 0.5 rspec -t focus
But with that the output won't show colors. So, we need to use with unbuffer.
sudo apt-get install expect
With a little customization:
watch -n 0.5 --color 'unbuffer bundle exec rspec -t focus'
Since it's annoying to type this all, I made two alias at my ~/.bash_aliases file (your can use .bashrc as well):
alias focus="watch -n 0.5 --color 'unbuffer bundle exec rspec -t focus'"
alias focuss="bundle exec rspec -t focus"
Now I can type focus to keep running it, or for a single focus execution I type focuss
Guard can be used for plain old ruby. I generally have trouble with guard so I like to use watchr, another gem. With a few lines of code you can tell watchr to watch for changes to your files and run a command when they change.
For an example of guard with plain ruby, see the shuhari gem.
update on watchr gem: There appears to be an issue with this gem, perhaps with versions of ruby >= 2.0. The observr gem addresses this issue and works as expected in ruby 2.3.
I have used guard and the guard-rspec addition with great results, and I don't believe it to be Rails-specific. Other Ruby/RSpec projects should work equally well.
The guard documentation recommends the use of Bundler and to "always run Guard through Bundler to avoid errors". I.e. you install it through your Gemfile and always run it with bundle exec guard (or use rubygems-bundler to avoid the bundle exec part).
I would like to have ctags generate a TAGS file of all my bundled gems or all the gems under the rvm gemset directory bundler installs its gems. Ideally, a bundle install or bundle update should generate a TAGS file at the last step using a ruby script I'll provide. Afterthat emacs joy.
Is there any kind of a bundler after hook I can use?
You could look at what Tim Pope does in his Hookup project:
https://github.com/tpope/hookup
I'd imagine it wouldn't be too hard to an an extra step after the bundler run.
Personally I just have a good old Makefile in my Ruby project:
.PHONY: tags
tags:
ETAGS=ctags
rm -rf TAGS
ctags -a -e -f TAGS --tag-relative -R app lib vendor
I have a shell script I run in the morning which sets up my dev environment which also runs make tags.
According to https://github.com/bundler/bundler/blob/dd1e11d8f8e869ffab4fc68d4854b27e1f486de4/lib/bundler/source/path.rb, there is the ability to run 'post_install' hooks. It uses meta-programming to deduce the method name, and the gem is supposed to implement that method. Will try and check if this works
My approach has been two pronged:
1) Put a rake task in place that generates tags for all code in the project as well as all required gems:
desc 'Create ctags'
task :tags do
system "ctags -R --language-force=ruby app config lib `rvm gemdir`/gems"
end
2) Using the excellent "foreman" gem (which I was using anyway) to run inotifywait and fire off the rake task if a file changes:
tags: while inotifywait -q -r -e MODIFY --exclude swp$ app/ config/ lib/ ; do bundle exec rake tags; done
If you are not using foreman you can of course just run that line without the first "tags:" part manually in a shell.