Given this code:
expect(exit_code).to eq(0), result
I understand what expect(exit_code).to eq(0) is supposed to do, but I don't understand what the comma or result variable are doing. Could someone shed some light on that for me?
It's not so common, but to method can accept an additional argument, which is a message displayed, when the example is failed. Have a look at the source code.
Thus, a custom message can be provided with extra information, how to handle the failed spec:
expect(exit_code).to(eq(0), "spec failed, because exit_code is 0, please handle it")
The second argument, can be also a proc:
expect(exit_code).to(eq(0), -> { "expected zero, got: #{exit_code}" })
Following #Stefan's comment, here's the documentation about customizing failure messages.
Related
Jasmine expect statements can produce worthless error messages like:
Expected true to be false.
To address this, matchers allow you to add a clarifying message as a second argument, expectationFailedOutput:
toBe(expected: any, expectationFailOutput?: any): Promise<void>;
This allows you to write:
expect(await password.isDisplayed).toBe(true, "Password field should be visible");
expect(await password.isDisplayed).toBe(true, "Password field was not visible");
These will produce the following error messages, respectively:
Expected false to be true, 'Password field should be visible'.
Expected false to be true, 'Password field was not visible'.
Note, that these lines are the same except that in the first case, I described what the expect was testing for, and in the second, I describe what actually happened.
Obviously, I should choose one these conventions and use it consistently in my code base, but I can't find anything in the documentation about what the typical convention is. Should the message describe what we expected to happen, or should it describe what did happen?
If the Jasmine team doesn't have a convention for this, perhaps somebody who's worked on a lot of Jasmine projects knows what the typical convention is.
I don't see why it should be consistent and why it obviously. Some checks are easy to understand, some hard. When you feel that you need message - add it. Don't make it hard when it could be simple.
When creating a Test Double in RSpec, I have the option to pass in a string which seems to "name" the double. I'm finding it hard to find documentation about this argument, the top result on google doesn't seem to mention it.
I made a test to play around with it and as far as I can tell it just used to for presentation/printing purposes. Does it have any use outside of this?
It sets a title for the double to be used in error messages.
#<Double "A Double"> received unexpected message :foo with (no args)
If you have multiple doubles, it will help you to figure out which one received an unexpected message.
In addition to the title, you can make your double respond in a predefined way to a method without using something like allow(double_x).to receive(:foo).and_return('bar') to set it up:
let(:double_x) { double("Double X", foo: 'bar') }
# double_x.foo
# => "bar"
The title is optional and can be left out.
I had a test that did this:
expect(#parser.parse('adsadasdas')).to raise_error(Errno::ENOENT)
and it didn't work. I changed to:
expect { #parser.parse('adsadasdas') }.to raise_error(Errno::ENOENT)
And it worked.
When do we use curly braces and when do we use parentheses with expect?
In response to OP's comment, I've edited and completely rewritten my answer. I realize that my original answer was oversimplified, so much so that it could be considered incorrect.
Your question was actually addressed somewhat by this other StackOverflow question.
One poster, Peter Alfvin, makes a good point when he says:
As for rules, you pass a block or a Proc if you're trying to test
behavior (e.g. raising errors, changing some value). Otherwise, you
pass a "conventional" argument, in which case the value of that
argument is what is tested.
The reason you're encountering the phenomenon you're seeing has to do with the raising of errors. When you pass #parser.parse('adsadasdas') as an argument (use parentheses) to expect, you are essentially telling ruby:
Evaluate #parser.parse('adsadasdas') first.
Take the result and pass this to expect.
expect should see if this result matches my expectation (that is, that Errno:ENOENT will be raised).
But, what happens is: when ruby evaluates #parser.parse('adsadasdas'), an error is raised right then and there. Ruby doesn't even get a chance to pass the result on to expect. (For all we care, you could have passed #parser.parse('adsadasdas') as an argument to any function... like multiply() or capitalize()) The error is raised, and expect never even gets a chance to do its work.
But when you pass #parser.parse('adsadasdas') as a proc (a code block) to expect using curly braces, what you are telling ruby is this:
expect, get ready to do some work.
expect, I would like you to keep track of what happens as we evaluate #parser.parse('adsadasdas').
Ok, expect, did the code block that was just evaluated raise a Errno:ENOENT error? I was expecting that it would.
When you pass a code block to expect, you are telling expect that you want it to examine the resulting behavior, the changes, made by your code block's execution, and then to let you know if it meets up to the expectations that you provide it.
When you pass an argument to expect, you are telling ruby to evaluate that argument to come to some value before expect even gets involved, and then you are passing that value to expect to see if it meets up to some expectation.
TL;DR: use expect(exp) to specify something about the value of exp and use expect { exp } to specify a side effect that occurs when exp is executed.
Let's unpack this a bit. Most of RSpec's matchers are value matchers. They match (or not) against any ruby object. In contrast, a handful of RSpec's matchers can only be matched against a block, because they have to observe the block while it's running in order to operate properly. These matchers concern side effects that take place (or not) while the block executes. The matcher would have no way to tell if the named side effect had occurred unless it is passed a block to execute. Let's consider the built-in block matchers (as of RSpec 3.1) one-by-one:
raise_error
Consider that one can return an exception from a method, and that is different than raising the exception. Raising an exception is a side effect, and can only be observed by the matcher by it executing the block with an appropriate rescue clause. Thus, this matcher must receive a block to work properly.
throw_symbol
Throwing symbols is similar to raising errors -- it causes a stack jump and is a side effect that can only be observed by running a block inside an appropriate catch block.
change
Mutation to state is a side effect. The matcher can only tell if there was a change to some state by checking the state before hand, running the block, then checking the state after.
output
I/O is a side effect. For the output matcher to work, it has to replace the appropriate stream ($stdout or $stderr) with a new StringIO, execute the block, restore the stream to its original value, and then check the contents of theStringIO`.
yield_control/yield_with_args/yield_with_no_args/yield_with_successive_args
These matchers are a bit different. Yielding isn't really a side effect (it's really just syntactic sugar for calling another function provided by the caller), but yielding can't be observed by looking at the return value of the expression. For the yield matchers to work, they provide a probe object that you pass on to the method-under-test as a block using the &probe syntax:
expect { |probe| [1, 2, 3].each(&probe) }.to yield_with_successive_args(1, 2, 3)
What do all these matchers have in common? None of them can work on simple ruby values. Instead, they all have to wrap a block in an appropriate context (i.e. rescuing, catching or checking before/after values).
Note that in RSpec 3, we added some logic to provide users clear errors when they use the wrong expect form with a given matcher. However, in the specific case of expect(do_something).to raise_error, there's nothing we can do to provide you a clear explanation there -- if do_something raises an error (as you expect it to...), then the error is raised before ruby evaluates the to argument (the raise_error matcher) so RSpec has no way to check with the matcher to see if supports value or block expectations.
in short:
use curly-brace (a block): when you want to test the behavior
use parenthesis when you want to test the returned value
worth reading: As for rules, you pass a block or a Proc if you're trying to test behavior (e.g. raising errors, changing some value). Otherwise, you pass a "conventional" argument, in which case the value of that argument is what is tested. - from this answer
In the test written with parentheses, the code is executed normally, including all normal error handling. The curly-brace syntax defines a block object upon which you can place the expectation. It encapsulates the code you expect to be broken and allows rspec to catch the error and provide its own handling (in this case, a successful test).
You can think of it this way as well: with the parentheses, the code is executed before being passed to the expect method, but with the block, expect will run the code itself.
Since there is no type in ruby, how do Ruby programmers make sure a function receives correct arguments? Right now, I am repeating if object.kind_of/instance_of statements to check and raise runtime errors everywhere, which is ugly. There must be a better way of doing this.
My personal way, which I am not sure if it a recommended way in general, is to type-check and do other validations once an error occurs. I put the type check routine in a rescue block. This way, I can avoid performance loss when correct arguments are given, but still give back the correct error message when an error occurs.
def foo arg1, arg2, arg3
...
main_routine
...
rescue
## check for type and other validations
raise "Expecting an array: #{arg1.inspect}" unless arg1.kind_of?(Array)
raise "The first argument must be of length 2: #{arg1.inspect}" unless arg1.length == 2
raise "Expecting a string: #{arg2.inspect}" unless arg2.kind_of?(String)
raise "The second argument must not be empty" if arg2.empty?
...
raise "This is `foo''s bug. Something unexpected happened: #{$!.message}"
end
Suppose in the main_routine, you use the method each on arg1 assuming that arg1 is an array. If it turns out that it is something else, to which each is not defined, then the bare error message will be something like method each not defined on ..., which, from the perspective of the user of the method foo, might be not helpful. In that case, the original error message will be replaced by the message Expecting an array: ..., which is much more helpful.
Ruby is, of course, dynamically typed.
Thus the method documentation determines the type contract; the type-information is moved from the formal type-system to the [informal type specification in the] method documentation. I mix generalities like "acts like an array" and specifics such as "is a string". The caller should only expect to work with the stated types.
If the caller violates this contract then anything can happen. The method need not worry: it was used incorrectly.
In light of the above, I avoid checking for a specific type and avoid trying to create overloads with such behavior.
Unit-tests can help ensure that the contract works for expected data.
If a method has a reason to exist, it will be called.
If reasonable tests are written, everything will be called.
And if every method is called, then every method will be type-checked.
Don't waste time putting in type checks that may unnecessarily constrain callers and will just duplicate the run-time check anyway. Spend that time writing tests instead.
I recommend to use raise at the beginning of the method to add manual type checking, simple and effective:
def foo(bar)
raise TypeError, "You called foo without the bar:String needed" unless bar.is_a? String
bar.upcase
end
Best way when you don't have much parameters, also a recommendation is to use keyword arguments available on ruby 2+ if you have multiple parameters and watch for its current/future implementation details, they are improving the situation, giving the programmer a way to see if the value is nil.
plus: you can use a custom exception
class NotStringError < TypeError
def message
"be creative, use metaprogramming ;)"
#...
raise NotStringError
You can use a Design by Contract approach, with the contracts ruby gem. I find it quite nice.
In Ruby's test/unit, and other such nunit style frameworks, what makes a good failure message?
Should the failure message merely describe how the expected value does not match the expected value?
assert_match("hey", "hey this is a test", "The word does not exist in the string")
Should it describe what you expected to happen?
assert_match("hey", "hey this is a test", "I expected hey to be in the string")
Should it describe why you wanted the behavior to happen?
assert_match("hey", "hey this is a test", "Program should provide a greeting")
Should it describe why you thought the test may fail?
assert_match("konnichiwa", "konnichiwa this is a test",
"Program failed to use supplied i18n configuration")
Should information about tests also exist in the name of the test method, and in the name of the test case?
This is based on Ruby "test/unit" , how do I display the messages in asserts
the failure message is supposed to add context to the failure message. So anything that saves you having to drill into the test code to know what failed.
So if the [method name, expected, actual] set is adequate for the above purpose, you can skip the failure message. If you need more information, then you add the optional failure message.
e.g.
Expected true but was false, doesn't tell me anything.
You can use a failure message so that
Return value should contain only multiples of 10. Expected true but was false
You can first try to use more descriptive matchers.
So that failures read Expected all items to be divisible by 10 but was [10,20,35,40] does.
Personally I prefer matchers... use failure messages as the last resort. (because like comments, it decays. You need discipline to ensure that the failure message is updated if you change the check.)