I am trying to split Dangerfile into multiple files and call them from the main Dangerfile.
I tried this way,
In the parent Dangerfile
puts "Branch #{github.branch_for_base} #{git.lines_of_code}"
require_relative "scripts/large_diff.rb"
warn 'Some warning'
And in the scripts/large_diff.rb
puts '---->>>>'
puts github.pr_json.inspect
puts '<<<<---->>>>'
puts github.inspect
puts '<<<<----'
But this gives this error
You used `puts` in your Dangerfile. To print out text to GitHub use `message` instead
Branch release/abc/xyz 304
---->>>>
bundler: failed to load command: danger (/Users/runner/work/my-proj/my-proj/vendor/bundle/ruby/3.0.0/bin/danger)
/Users/runner/work/my-proj/my-proj/scripts/large_diff.rb:2:in `<top (required)>': \e[31m (Danger::DSLError)
[!] Invalid `Dangerfile` file: undefined local variable or method `github' for main:Object. Updating the Danger gem might fix the issue. Your Danger version: 8.5.0, latest Danger version: 8.6.1
\e[0m
# from Dangerfile:3
# -------------------------------------------
# puts "Branch #{github.branch_for_base} #{git.lines_of_code}"
> require_relative "scripts/large_diff.rb"
# warn 'Big PR, split it into smaller ones'
# -------------------------------------------
from Dangerfile:3:in `require_relative'
from Dangerfile:3:in `eval_file'
from /Users/runner/work/bitrise-comment/bitrise-comment/vendor/bundle/ruby/3.0.0/gems/danger-8.5.0/lib/danger/danger_core/dangerfile.rb:311:in `eval'
from /Users/runner/work/bitrise-comment/bitrise-comment/vendor/bundle/ruby/3.0.0/gems/danger-8.5.0/lib/danger/danger_core/dangerfile.rb:311:in `eval_file'
...
...
...
I know this is because the github object is not avaialble in the child danger file. How can I use variables and functions (the object github functions warn, fail etc) available in Parent Dangerfile also in the child danger file?
PS: I am new to ruby :)
What you're trying to do is not so much Ruby-specific but rather a way of how things are done with Danger. One of the ways to split the file into several smaller ones would be to dedicate a folder for these smaller danger files. Here's an example of such a structure
.
├── Dangerfile
└── checks
├── large_diff
│ └── Dangerfile
└── other_check
└── Dangerfile
Then in the main Dangerfile, you can just reference the other ones like so
danger.import_dangerfile(path: 'checks/large_diff')
danger.import_dangerfile(path: 'checks/other_check')
It doesn't really matter how you name the directories, but Danger will expect to find a file named Dangerfile in that directory. Personally, I prefer this approach, but there are other methods like importing a Dangerfile from a gem or creating a Danger plugin and calling it through danger.import_plugin(...) See the reference for more info - https://danger.systems/reference.html
Related
How can I quickly run a custom function on any node with puppet apply ?
Let's say I have this file (test.pp)
file { "/root/files/f":
content => test('test'),
}
And this ruby file (test.rb) which only log and return the first agument.
require 'logger'
module Puppet::Parser::Functions
newfunction(:test, :type => :rvalue
) do |args|
log = Logger.new(STDOUT)
log.level = Logger::INFO
log.info(args[0])
args[0]
end
end
How does one call the ruby function test with a puppet apply test.pp?
My problem is that is take like 5minutes to run my entire puppet agent -t and I need to add a function to puppet so I would like to test it quickly instead of waiting 5minutes each time.
This will certainly work, but it's hard to maintain.
A more maintainable way to do this is to add your function to a module, then install that module on your master.
For example, create a new with the name of your module (eg. test_function):
mkdir -p test_function/lib/puppet/parser/functions
You should have the following tree in your test_function module
├── lib
│ └── puppet
│ ├── parser
│ │ └── functions
│ │ ├── test.rb
Add your test.rb code to the test.rb file
Copy this module to your master's module path (this is probably /etc/puppet/modules depending on your Puppet version, use puppet module list to find out)
After that, the puppet master will read its module list and dynamically add-in all functions it finds.
More documentation about it here:
https://docs.puppetlabs.com/guides/plugins_in_modules.html
https://docs.puppetlabs.com/guides/custom_functions.html
I just found out.
All you need to do is to add the test.rb file directly into /var/lib/puppet/lib/puppet/parser/functions/test.rb and the puppet apply will "see" the function.
I'm trying to setup Guard in my Exercism folder, so that all tests are automatically run (using guard-minitest).
The folder structure looks like this:
.
├── exercism
└── ruby
├── bob
│ ├── bob.rb
│ ├── bob_test.rb
├── etl
│ ├── etl.rb
│ ├── etl_test.rb
...
└── binary_search_tree
├── bst.rb
├── binary_search_tree.rb
└── README.md
Due to the unusual folder structure, I made the following configuration in the Guardfile:
# tell Guard to watch all folders under ruby/
all_dirs = Dir.glob('ruby/*')
directories all_dirs
options = {
test_folders: all_dirs,
test_file_patterns: '*_test.rb',
pride: true
}
guard :minitest, options do
watch(%r{^ruby/([^/]+)/([^/]+).rb$})
end
With this setup I expect that all my tests will be run after any edit to a .rb file.
I run guard in debug mode.
All tests run on startup, as expected.
However, when I edit a .rb file the tests are not run again, although Guard outputs this:
20:43:26 - DEBUG - Interactor was stopped or killed
20:43:26 - DEBUG - Hook :run_on_additions_begin executed for Guard::Minitest
20:43:26 - DEBUG - Hook :run_on_additions_end executed for Guard::Minitest
20:43:26 - DEBUG - Start interactor
I tried many variations in the guard configuration, such as:
watch(%r{^ruby/([^/]+)/([^/]+).rb$}) do |m|
Dir.glob("ruby/#{m[1]}/*_test.rb").first
end
(I expect this to only run the test from the folder in which a file was modified.)
Nothing seems to work. I have to go to guard's interactive console and press ENTER to get tests to run.
What am I doing wrong?
EDIT
After reading Cezary's answer below, I tried some more things, including trying this on other environments.
It turns out it wasn't guard's fault, but probably the environment I was using.
I initially ran everything in a Codio box.
I moved the project to other two machines, a Win8 and an Ubuntu 14.04, and it works fine on both using this Guardfile:
all_dirs = Dir.glob('ruby/*')
directories all_dirs
clearing :on
options = {
test_folders: all_dirs,
test_file_patterns: '*_test.rb',
pride: true
}
guard :minitest, options do
watch(%r{^ruby/([^/]+)/([^/]+).rb$}) do |m|
file = Dir.glob("ruby/#{m[1]}/*_test.rb").first
puts " Should test #{file}"
file
end
end
The output of guard -d is as follows:
On the Codio box (where it doesn't work):
<!-- language: lang-none -->
08:55:09 - DEBUG - Interactor was stopped or killed
Should test ruby/anagram/anagram_test.rb
08:55:09 - DEBUG - Hook :run_on_additions_begin executed for Guard::Minitest
08:55:09 - DEBUG - Hook :run_on_additions_end executed for Guard::Minitest
08:55:09 - DEBUG - Start interactor
[1] guard(main)>
On Win/Ubuntu (where it works fine):
<!-- language: lang-none -->
11:02:10 - DEBUG - Interactor was stopped or killed
Should test ruby/anagram/anagram_test.rb
11:02:10 - DEBUG - Hook :run_on_modifications_begin executed for Guard::Minitest
11:02:10 - INFO - Running: ruby/anagram/anagram_test.rb
Run options: --seed 52507
# Running tests:
..........
Finished tests in 0.001249s, 8006.0205 tests/s, 8006.0205 assertions/s.
10 tests, 10 assertions, 0 failures, 0 errors, 0 skips
11:02:10 - DEBUG - Hook :run_on_modifications_end executed for Guard::Minitest
11:02:10 - DEBUG - Start interactor
[1] guard(main)>
I don't know why on Codio I get run_on_additions and on the other two I get run_on_modifications.
Anyway, I guess that's a Codio issue. I tried with and without Manual Save and it's the same.
Hi I'm one of the guys "reponsible" for Guard, and based on this I wrote a walkthrough: https://github.com/guard/guard/wiki/Understanding-Guard
I highly recommend going through it (feedback appreciated).
Here's a list of things to consider:
structure: if you can, put tests in a separate top folder, ideally 'ruby/test' in your example, and the files in 'ruby/lib'. This is closer to the convention and then things like Guard::Minitest would work out of the box. (Some editors - like Vim and Emacs - allow you to switch between "alternative" files and they will automatically know where to look for tests or implementation files).
Guard watches directories recursively (and this can't even be turned off right now). For small projects, you can just watch everything by not specifying a directories option. But, if you have lots of disk activity in your folder and you have large projects, you'll want to select directories. E.g. in your case, it would be: directories ruby. So `Dir.glob('ruby/*') doesn't make much sense here.
Guard::Minitest options - if you use the structure I described, this shouldn't be necessary. If not, I think having test_folder: %w(ruby) should be enough. Also, test_file_patterns shouldn't be necessary, since your files seem to following the defaults/convention (bob_test.rb and etl_test.rb) - unless you have something really strange as having test suites inside the bob.rb and etl.rb files.
Your watch expression has no block, so it returns the files changed. Unfortunately, when you change an implementation file (like bob.rb), that file is passed to Guard::Minitest, which ignore non-test files (probably using the test_file_patterns option, so since bob.rb won't match bob_test.rb, Guard::Minitest will quietly do ... nothing.
Ideally, rename your Guardfile, update your gems (Guard::Minitest basically) and run bundle exec guard init minitest to so what the "current" recommended template is for Guard::Minitest and try to tweak that instead. You'll see that the default has:
watch(%r{^lib/(.*/)?([^/]+)\.rb$}) { |m| "test/#{m[1]}test_#{m[2]}.rb" }
which shows how to translate changed implementations files into test files (which are the only files Guard::Minitest cares about).
In your case you may want:
watch(%r{^ruby/(.*/)?([^/]+)\.rb$}) { |m| "ruby/#{m[1]}/#{m[2]}_test.rb" }
I'd say the regexen are horrible - and I'm planning to implement glob pattern support - but this will take a while (lots of critical bug fixes in the pipeline for me).
If this doesn't help - definitely go through the Wiki document above, since there are about 20-50 problems you can have with Guard/Listen - and most are completely outside Guard's/Listen's control. (I wish I could change this to make it simpler, so any suggestions would be helpful).
Since bugs are a priority for me, like most maintainers - we rely on issues reported int GitHub. So we don't usually watch stuff in Stack Overflow (we expect tough issues to be reported in Github).
Ok, I've pointed out so many things wrong, time for stuff you did very, very well:
Running Guard in debug mode - you're AWESOME for doing that
Showing output regarding the changes - you're AWESOME for doing that as well (since this shows the Guard::Minitest is getting the changes, but it's ignoring them - likely since you're passing implementation files and not test files).
Submitting such a detailed issue - that's so helpful. Ideally, you shouldn't have gone through all the trouble (or even had the issue in the first place), but by reporting this you're showing us what is wrong with Guard on so many levels. (Often, as developers/maintainers we "get" the internals, so we "know" what's wrong immediately - even though the docs or messages or debug output says nothing to others - so issues like this help us help others very much).
Thanks again for this - if you ever see other people with issues you can resolve - please do so. If you see obvious bugs or issues in Guard - please open an issue in Guard. If it's in another plugin repository (like Guard::Minitest), just mention me so that important issues aren't ignored (mention me as #e2 on GitHub).
I do hope to make Guard better - currently I'm working hard to make it simpler, more intuitive, more flexible and more reliable. The more I can count on people reporting broken stuff, the faster I can move ahead with cool stuff.
Thanks again and have a great day!
I had a similar issue when wanting to use Guard with any new Ruby Exercism exercise I'd pull down: running the full test suite would work, but tests would not run automatically when files changed. What I did in order to get this working was the following:
Create a common Guardfile and place it at ~/exercism/ruby/Guardfile with the following content:
# frozen_string_literal: true
group :red_green_refactor, halt_on_fail: true do
guard :minitest,
all_on_start: false,
test_folders: ["."] do
# Re-test test files when they're edited.
watch(%r{\A.+_test\.rb\z}) { |m| "./#{m[1]}" }
# Run the test file of the (non-test) file that was edited.
watch(%r{\A(.+)(?<!_test)\.rb\z}) { |m| "./#{m[1]}_test.rb" }
end
guard :rubocop,
all_on_start: false,
cli: ["--display-cop-names"] do
# Only run Rubocop over implementation files only
# as test files are not written by me.
watch(%r{\A(.+)(?<!_test)\.rb\z})
watch(%r{(?:.+/)?\.rubocop\.yml\z}) { |m| File.dirname(m[0]) }
end
end
When running Guard within an exercise folder, always specifically reference this Guardfile ie guard --guardfile ~/exercism/ruby/Guardfile.
Some specific gotchas here about the Guardfile configuration:
The test_folders option, the locations where guard-minitest will go looking for test files, defaults to %w[test spec]. This is fine for any standard Ruby project, but not for Exercism, where the test file and the implementation file are in the same directory. So, it needed to be changed to ["."] to specify that.
In the blocks for the Minitest watch methods, the string returned needs to look like a path, otherwise the tests will just not run when a file is changed (with no error output or indication that something went wrong). For example:
watch(%r{\A.+_test\.rb\z}) { |m| "#{m[1]}" } will not work. It has to be watch(%r{\A.+_test\.rb\z}) { |m| "./#{m[1]}" }.
My gem uses a local copy of Tidy HTML Validator. It's available in lib/tidy/tidy. My tests run perfectly locally, but on Travis CI, I'm getting the error that Tidy can't be found.
Errno::ENOENT:
No such file or directory - tidy
The code looks like this:
stdin, stdout, stderr = Open3.popen3('tidy -quiet')
Is there a restriction or something that prevents executing a local executable? I also call a local copy of the Jigsaw CSS Validator which is a Java jar file and is executed like this:
`java -jar css-validator.jar --output=soap12 file:#{uri}`
This works perfectly on Travis CI. Could there be a problem with Open3?
Here's the link to the GitHub repo: https://github.com/jmuheim/headhunter
Here's the link to the failing Travis CI build: https://travis-ci.org/jmuheim/headhunter
Update
I have rewritten the code that executes the external java executable to also use Open3, and it's working. So the problem has to do with tidy.
For further debugging I have renamed tidy into tidy2 (to make sure that really the wanted executable is called), and I added a manual File.exist? 'tidy' before calling it, so this would raise an error if it couldn't be found.
Dir.chdir(VALIDATOR_DIR) do
raise "Could not find tidy in #{Dir.pwd}" unless File.exists? EXECUTABLE
# Docs for Tidy: http://tidy.sourceforge.net/docs/quickref.html
stdin, stdout, stderr = Open3.popen3("#{EXECUTABLE} -quiet")
# ...
end
The file is definitely found, but executing it still results in this error.
1) Headhunter::HtmlValidator#validate returns a local response when calling the validator succeeds
Failure/Error: subject.validate('invalid.html', read_file('html_validator/invalid.html'))
Errno::ENOENT:
No such file or directory - tidy2
# ./lib/headhunter/html_validator.rb:20:in `block in validate'
# ./lib/headhunter/html_validator.rb:16:in `chdir'
# ./lib/headhunter/html_validator.rb:16:in `validate'
# ./spec/lib/headhunter/html_validator_spec.rb:9:in `block (4 levels) in <top (required)>'
# ./spec/lib/headhunter/html_validator_spec.rb:8:in `block (3 levels) in <top (required)>'
So the only thing I can guess now is that be there is a permissions problem? Is there a way to set permissions to local executables with Travis CI?
I also tried putting tidy into bin like somebody suggested in the comments, but this didn't seem to change a thing, too.
It looks like your tidy2 executable is a Mach-O binary for OS X, but the Travis worker runs Linux. Try to include a suitable ELF binary from http://tidy.sourceforge.net/#binaries.
A couple things:
Typically the environment is different than your current environment when spawning programs so you have to check what the environment is. This is the case in cron, monit, and for some ruby method calls.
In your case, for Open, it uses Process.spawn so you're not guaranteed to be in your own directory. Open.popen3 takes a :chdir option, so set that to your Dir.pwd and see what happens.
You can also try just using absolute paths.
Sources
http://ruby-doc.org/stdlib-1.9.3/libdoc/open3/rdoc/Open3.html#method-c-popen3
https://stackoverflow.com/a/5917927/463225
http://ruby-doc.org/core-1.9.3/Process.html#method-c-spawn
I am trying to use the main gem for making command line utilities. This was presented in a recent Ruby Rogues podcast.
If I put all the code in one file and require that file, then rspec gives me an error, as the main dsl regards rpsec as a command line invocation of the main utility.
I can break out a method into a new file and have rspec require that file. Suppose you have this program, but want to put the do_something method in a separate file to test with rspec:
require 'main'
def do_something(foo)
puts "foo is #{foo}"
end
Main {
argument('foo'){
required # this is the default
cast :int # value cast to Fixnum
validate{|foo| foo == 42} # raises error in failure case
description 'the foo param' # shown in --help
}
do_something(arguments['foo'].value)
}
What is the convenient way to distribute/deploy a ruby command line program with multiple files? Maybe create a gem?
You are on the right track for testing - basically you want your "logic" in separate files so you can unit test them. You can then use something like Aruba to do an integration test.
With multiple files, your best bet is to distribute it as a RubyGem. There's lots of resources out there, but the gist of it is:
Put your executable in bin
Put your files in lib/YOUR_APP/whatever.rb where "YOUR_APP" is the name of your app. I'd also recommend namespacing your classes with modules named for your app
In your executable, require the files in lib as if lib were in the load path
In your gemspec, make sure to indicate what your bin files are and what your lib files are (if you generate it with bundle gem and are using git, you should be good to go)
This way, your app will have access to the files in lib at runtime, when installed with RubyGems. In development, you will need to either do bundle exec bin/my_app or RUBYLIB=lib bin/my_app. Point is, RubyGems takes care of the load path at runtime, but not at development time.
So here's what I'm attempting to do. I'm building an ember.js application, with a java backend running on GAE.
I'm using handlebars, but I want them divided up into separate files, not just all pasted into the index.html.
Via the ember.js irc I was turned on to rake-pipeline along with minispade
Along with the web filters and a custom handlebars filter I started building the assetfile. I don't know Ruby, or gem files, etc.
So I'm trying to figure out the best way to be able to compile my coffeescript/handlebars files on the fly, minispade them, but keep the individual files accessible while in dev mode so I can debug them. What makes that hard is that the rake pipeline is running on a different port than GAE. So I'm not sure exactly how to handle this. Do I make my index file in GAE point to individual files at the 9292 port (rakep) during development, but in production mode point to the fully concatenated version? I'm not sure.
So I was attempting to do that here: https://gist.github.com/1495740 by having only one section that was triggered by the 'build' flag. Not even sure if that works that way.
I know there's a lot of confusion here. Apologies, like I said I'm not even remotely familiar with the Ruby style of doing things.
Since you're not a Ruby person, here are the most reliable steps for getting a stock OSX environment set up with rake pipeline:
Step 1: Install bundler
# on OSX, using built-in Ruby
$ sudo gem install bundler --pre
Step 2: Create a Gemfile
# inside your app directory
$ bundle init
# will create a file named Gemfile in the root
Step 3: Add rake-pipeline to the Gemfile
# inside the Gemfile
gem "rake-pipeline-web-filters"
Step 4: Install your gems
$ bundle install --binstubs
Step 5: Set up Assetfile
However you were already doing it...
Step 6: Run Rake::Pipeline
# to run the preview server
$ bin/rakep
# to build your assets
$ bin/rakep build
Rake::Pipeline.build is the method that evaluates an Assetfile. You can imagine that your entire Assetfile is wrapped inside a Rake::Pipeline.build {} block; you shouldn't ever need to write one inside an Assetfile.
Some of the filters in the docs are hypothetical, most of those docs were written before there were any filters at all. A CoffeeScript compiler has been recently added, though.
As to your main question, I'm not sure there's a clean way to do it with the current rakep implementation. An Assetfile is just Ruby, though, so it's possible to hack something together that should work. Here's how I would write yours:
require "json"
require "rake-pipeline-web-filters"
require "rake-pipeline-web-filters/helpers"
class HandlebarsFilter < Rake::Pipeline::Filter
def initialize(&block)
block ||= proc { |input| input.sub(/\.handlebars$/, '.js') }
super(&block)
end
def generate_output(inputs, output)
inputs.each do |input|
output.write "return Ember.Handlebars.compile(#{input.read.to_json})"
end
end
end
# process all js, css and html files in app/assets
input "assets"
# processed files should be outputted to public
output "public"
# process all coffee files
match "**/*.coffee" do
# compile all CoffeeScript files. the output file
# for the compilation should be the input name
# with the .coffee extension replaced with .js
coffee_script
# The coffee_script helper is exactly equivalent to:
# filter Rake::Pipeline::Web::Filters::CoffeeScriptCompiler
end
match "**/*.js" do
minispade
if ENV['RAKEP_ENV'] == "production"
concat "application.js"
else
concat
end
end
match "**/*.handlebars" do
filter HandlebarsFilter
minispade
concat "templates.js"
end
The if ENV['RAKEP_ENV'] bit reads an environment variable to decide whether to concatenate your JS to a single file.
So now you can run RAKEP_ENV="production" rakep build for a concatenated build, or just rakep build for a development build.