If a test wants to assert the file permission error, for example, writing to the root of file system, "go test" returns an syscall.EACCES error, while "bazel test" returns an syscall.EPERM. How to make the test passes in both "bazel test" and "go test"?
An example can be found here.
You can disable the sandbox by using bazel --spawn_strategy=standalone test //.... I suspect this will work around the problem.
However, you may want to consider whether writing to / is the behavior that you want to test. If you need to run code on a different operating system or inside a Docker container, you'll get different behavior in this case, so you could think about testing a more predictable code path, or mocking out the file access layer to isolate your tests from it.
Related
I have a file with N>10 tests defined in BDD in a feature file.
For debugging purposes, I need to execute only one of the tests many times, but I am not finding any other way than commenting out the other tests.
I am trying now to use the #focus tag in front of the test, in a way that cypress IDE actually marks the rest of the tests as sync skip; aborting execution, but the test that I need to run actually says No commands were issued in this test..
This is the example .feature file I'm playing with:
#e2eTests
#cases
Feature: Test feature
Scenario: Test 1
Given foo
When bar
Then foobar
#focus
Scenario: Test 2
Given bar
When foo
Then barfoo!
And when running npm run cypress and selecting this feature, I get:
What am I doing wrong?
Thanks
I would recommend using cypress-grep if you are looking for a flaky test, that needs repeating n times.
After installation and configuration (see here: Burning Tests with cypress-grep) in terminal indicate which test needs to be repeated and how many times (cypress 7.6) ->
npx cypress run --spec cypress/folder/with/test/testFile.spec.ts, --env grep="name/description of specific test", burn= >>numberOfIterations<<
Example:
npx cypress run --spec cypress/integration/testExample.spec.ts, --env grep="A popup should appear after login",burn=250
So check this out: it appears as though, upon running bundle exec rspec, there's a race between jekyll serve and puma/rspec's boot up. Sometimes I run the command, and my tests run fine. Other times, I get the error for each of my spec files: cannot load such file -- /path/to/project/sitename_jekyll/_layouts/spec/form_spec.rb which is interesting cause that's not where my spec files are located. They're in /path/to/project/sitename_jekyll/spec/form_spec.rb.
What's crazy is that I can literally just re-run the command over and over and over again and sometimes it'll go through and run the spec tests in the correct location, and sometimes it'll look for them in _layouts and error out. It probably runs correctly maybe once out of ever three or five attempts. All the other times I get the following errors:
Here's what my spec_helper.rb looks like: https://gist.github.com/johnhutch/2cddfafcde0485ff021501d5696c0c2d
And here's an example test file:
https://gist.github.com/johnhutch/a35d15c170f5fd9ca07998bf035d111d
My .rspec only contains two lines:
--color
--require spec_helper
And here's the output, both successful and unsuccesful, back to back:
https://gist.github.com/johnhutch/7927d609170ef5c70a595735502b128d
HEEELLLLLP!
This sounds like jekyll is changing the current directory while building the site, which since it is being run in a thread also affects the tests RSpec is trying to run (See https://bugs.ruby-lang.org/issues/9785 for why Dir.chdir is not threadsafe) - leading to attempts to load things from incorrect locations.
A potential solution to this would be to wait for the Jekyll site to be built before actually running your tests. A comment in your spec_helper seems to state that someone thought passing force_build: true would do this but from a quick perusal of the jekyll-rack code I don't think that's true and you actually need to wait for compiling? to return false (v 0.5) (complete? to return true in the current master branch) to ensure building has finished (as well as passing force_build). This could either be done in a loop sleeping and checking (simpler)
sleep 0.1 while <jekyll app>.compiling?
or (if using the master branch) via the mutex/conditional Rack::Jekyll exposes like in its test suite - https://github.com/adaoraul/rack-jekyll/blob/master/test/helper.rb#L49
Note: Also check my comment about your tests that aren't actually testing anything.
As per Thomas Walpole's super helpful responses this ended up working:
sleep 0.1 while Capybara.app.compiling?
inserted right after:
51 Capybara.app = Rack::Jekyll.new(force_build: true)
in my spec_helper.rb
Thanks again, Thomas!
Currently I am using
parallel tests
rspec
allure 0.8.0
After I run the tests I get the following error:
RSpec::Core::MultipleExceptionError
I need the whole backtrace of the error. Is it some parameter that I need to pass to the command that I use to run and is there a permanent way so that it always prints the whole error
You have two options, to define a helper with that attribute or in command line
## spec/spec_helper.rb
config.full_backtrace = true # false to dismiss
or in the command line
$ rspec spec/folder/your_spec.rb --backtrace
If you get this error you need to find the specific test in the rspec log (scroll up). There you will see details about the issue.
In my case error was shown only on CI and I needed some time in order to figure out that everything I need is "hidden" somewhere in the super long output.
UPDATE: when I run rake test from root, I get:
C:\workspace\faker>rake test
DL is deprecated, please use Fiddle
C:/RailsInstaller/Ruby2.1.0/bin/ruby.exe -w -I"lib;test;." -I"C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib" "C:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/rake_test_loader.r
b" "test/test_array_sample_method_compat.rb" "test/test_avatar.rb" "test/test_en_au_ocker_locale.rb" "test/test_en_ca_locale.rb" "test/test_en_locale.rb" "test/test_en_ug_locale.rb" "test/test_en_us_locale.rb" "test/test_es_l
ocale.rb" "test/test_faker.rb" "test/test_faker_app.rb" "test/test_faker_bitcoin.rb" "test/test_faker_book.rb" "test/test_faker_business.rb" "test/test_faker_city.rb" "test/test_faker_code.rb" "test/test_faker_color.rb" "test
/test_faker_commerce.rb" "test/test_faker_company.rb" "test/test_faker_date.rb" "test/test_faker_hacker_talk.rb" "test/test_faker_internet.rb" "test/test_faker_lorem.rb" "test/test_faker_name.rb" "test/test_faker_number.rb" "
test/test_faker_shakespear.rb" "test/test_faker_shakespeare.rb" "test/test_faker_slack_emoji.rb" "test/test_faker_street.rb" "test/test_faker_team.rb" "test/test_faker_time.rb" "test/test_faker_university.rb" "test/test_flexi
ble.rb" "test/test_helper.rb" "test/test_locale.rb" "test/test_pl_locale.rb" "test/test_uk_locale.rb"`
I don't fully understand what rake is doing at this point, but the system hangs infinitely. What am I missing?
So I have built some added functionality for the faker gem, and want to run the tests I wrote before submitting a pull request. However, all my attempts to run the existing tests, and my new tests, have failed. I am worried that it is a Windows machine related issue.
I have tried running rake test:units but it gives me an error and then hangs, while also ringing the Bell on my system 5 times. Strange right?
I have also tried running bundle exec rspec spec, but no tests are found becasue they are not RSpec unit tests, or at least not in the right location.
To see how the gem is formatted, check it out at the repo
All the test files, including my new one, live in the test directory. But for the life of me, I can't run them.
I tried this and works fine:
bundle install
bundle exec ruby test
I edited my target scheme to run a script action after testing as below
Target Scheme -> Test (Debug) -> Post Actions
The script hw.sh had a simple command line call:
open /Applications/Safari.app/
It worked fine for the above script. When I changed it to the following
groovy http://frankencover.it/with -s /Users/sasokan/Downloads/MyProject
Nothing happened. How can I call this groovy application using a script.
I am also trying to run frankencover.it and had the same problem you did. I eventually found this answer on SO that lead me to a solution. I added the following before calling frankencover.it and it fixed the issue.
PATH=${PATH}:/usr/local/bin
I will further note that even if you use the full path to groovy in the command frankencover.it will fail internally because it cannot find 'lcov' for the same reason.