Ruby Mongo Driver question:
How do I output 5_000 document batches from the collection at a time until I read the last document in the collection without dumping the entire database into memory first?
This is really bad method for me:
mongo = MongoClient.new('localhost', 27017)['sampledb']['samplecoll']
#whois.find.to_a....
Mongo::Collection#find returns a Mongo::Cursor that is Enumerable. For batch processing Enumerable#each_slice is your friend and well worth adding to your toolkit.
Hope that you like this.
find_each_slice_test.rb
require 'mongo'
require 'test/unit'
class FindEachSliceTest < Test::Unit::TestCase
def setup
#samplecoll = Mongo::MongoClient.new('localhost', 27017)['sampledb']['samplecoll']
#samplecoll.remove
end
def test_find_each_slice
12345.times{|i| #samplecoll.insert( { i: i } ) }
slice__max_size = 5000
#samplecoll.find.each_slice(slice__max_size) do |slice|
puts "slice.size: #{slice.size}"
assert(slice__max_size >= slice.size)
end
end
end
ruby find_each_slice_test.rb
Run options:
# Running tests:
slice.size: 5000
slice.size: 5000
slice.size: 2345
.
Finished tests in 6.979301s, 0.1433 tests/s, 0.4298 assertions/s.
1 tests, 3 assertions, 0 failures, 0 errors, 0 skips
Related
I have a test file, ipca_test.rb:
require "test_helper"
require "matrix" # Does needing to include this here mean I'm doing something wrong?
class IpcaTest < Minitest::Test
def test_that_it_has_a_version_number
refute_nil ::Ipca::VERSION
end
def test_it_does_something_useful
refute false
end
def test_on_a_random_matrix
p = rand(3..10)
n = rand(20..50)
m = Matrix.build(n, p) {|_, _| rand(-10.0..10.0)}
pca = Ipca::Pca.new(m)
eigenvalue, r = pca.first_principal_component
puts "eigenvalue: #{eigenvalue}, r: #{r}"
assert eigenvalue.kind_of? Numeric
assert_equal Vector, r.class
end
end
The program I'm trying to test is ipca.rb:
require "ipca/version"
module Ipca
class Error < StandardError; end
class Pca
def initialize data
#data = data.class == Matrix ? data : Matrix.rows(data)
end
# see https://en.wikipedia.org/wiki/Principal_component_analysis#Iterative_computation
def first_principal_component(c = 100, tolerance = 0.001) # not sure whether defaults are apropos
p = #data.column_vectors.count
r = Vector.elements(Array.new(p) {|_| rand}).normalize
eigenvalue = nil
c.times do
s = Vector.zero(p)
#data.row_vectors.each do |x|
s += x.dot(r)*x
end
eigenvalue = r.dot(s) # ?
error = (eigenvalue*r-s).norm
r = s.normalize
exit if error < tolerance
end
return [eigenvalue, r]
end
end
end
(Git repo of entire tree)
Sometimes the test is successful, but much more often the test never gets to "Finished". In these cases there are zero or more dots representing (I assume) successful assertions. I'm guessing that the tests runs are being halted by some kind of timeout somewhere in my bundle, and that this times out intermittently because the input data for the test is of varying size. But if it's a timeout issue why is there not a message to that effect? Here is a series of test runs:
josie#josie-Inspiron-580:/var/www/html/ruby/ipca$ ruby -Itest
test/ipca_test.rb Run options: --seed 44059
# Running:
..josie#josie-Inspiron-580:/var/www/html/ruby/ipca$ ruby -Itest
test/ipca_test.rb Run options: --seed 57681
# Running:
.josie#josie-Inspiron-580:/var/www/html/ruby/ipca$ ruby -Itest
test/ipca_test.rb Run options: --seed 57222
# Running:
..josie#josie-Inspiron-580:/var/www/html/ruby/ipca$ ruby -Itest
test/ipca_test.rb Run options: --seed 7474
# Running:
..josie#josie-Inspiron-580:/var/www/html/ruby/ipca$ ruby -Itest
test/ipca_test.rb Run options: --seed 1938
# Running:
..josie#josie-Inspiron-580:/var/www/html/ruby/ipca$ ruby -Itest
test/ipca_test.rb Run options: --seed 61325
# Running:
..eigenvalue: 2027.687580111128, r: Vector[0.03288542301229099,
-0.09533529249551115, 0.3033273986606458, 0.07951734565050736, 0.3575555246291426, 0.41614419068773545, 0.4928822662304588, 0.28785088479078025, 0.5144766379975693] .
Finished in 0.037173s, 80.7047 runs/s, 107.6063 assertions/s.
3 runs, 4 assertions, 0 failures, 0 errors, 0 skips
josie#josie-Inspiron-580:/var/www/html/ruby/ipca$
Use break instead of exit:
break if error < tolerance
break exits just the do loop.
exit exits the program itself not giving a chance for minitest to fail the test
Hi I'm completely new to Ruby. I'm trying to run Minitests, it use to work fine until I added a constructor to my CountDown class.
Here is the code:
require 'benchmark'
require 'minitest/autorun'
#! /usr/bin/env ruby
#This is our countdown class
# :reek:DuplicateMethodCall
# :reek:TooManyStatements
class CountDown
def initialize(time)
#time = time
end
def count()
wait_time = #time.to_i
print wait_time.to_s + " seconds left\n"
sleep 1
wait_time = wait_time - 1
wait_time.downto(1).each do |time_left|
sleep 1
print time_left.to_s + " seconds left\n" if (time_left % 60) == 0
end
print "\a"
end
end
#This class is responsible for our test cases
class CountDownTest < Minitest::Test
def setup
#count_down = CountDown.new(10)
end
def testing_lowerbound
time = Benchmark.measure{
#count_down.count
}
assert time.real.to_i == 10
end
end
This my my output:
teamcity[enteredTheMatrix timestamp = '2017-09-28T15:10:11.470-0700']
teamcity[testCount count = '0' timestamp = '2017-09-28T15:10:11.471-0700'] Finished in 0.00038s 0 tests, 0
assertions, 0 failures, 0 errors, 0 skips
Process finished with exit code 0
Any idea what's wrong? It looks fine to me.
Tests should start with the prefix test_ not testing_. Using the wrong prefix makes MiniTest assume they're doing something other than running a test, so it ignores them.
I have a mongodb collection. Using the ruby driver, the following works:
search = 'LONDON'
result = posts.find(:district => search).to_a.to_json
and produces the following in the firebug console:
[{"_id":"{AB46B6E4-46F7-44F6-8D88-0002C05947BB}","price":"450000","date_sold":"2013-10-23 00:00","post_code":"NW6 2DT","house_type":"F","condition":"N","freehold":"L","house_number":"72","flat_number":"FLAT 3","street":"LOVERIDGE ROAD","town":null,"district":"LONDON","region":"CAMDEN","county":"GREATER LONDON"}]
There are 30 records in the collection, when I change to an aggregate function as follows:
result = posts.find( { :price => { $gt => 100000 } } ).to_a.to_json
I get an empty [ ] in the console. Is this because the data type, in the collection, is not set to integer? If so, how can I change it programatically (i.e. not in the shell)?
Or is the query wrong? I am using the mongodb ruby driver.
All help gratefully received, thank you.
You are so close - your example needs quotes around $gt,
and the value of the price field needs to be integer and not string so that $gt will work as you desire.
Here's a test that verifies this. Hope that this helps.
test.rb
require 'mongo'
require 'json'
require 'test/unit'
class MyTest < Test::Unit::TestCase
def setup
#posts = Mongo::MongoClient.new['test']['posts']
#docs = JSON.parse <<-EOT
[{"_id":"{AB46B6E4-46F7-44F6-8D88-0002C05947BB}","price":"450000","date_sold":"2013-10-23 00:00","post_code":"NW6 2DT","house_type":"F","condition":"N","freehold":"L","house_number":"72","flat_number":"FLAT 3","street":"LOVERIDGE ROAD","town":null,"district":"LONDON","region":"CAMDEN","county":"GREATER LONDON"}]
EOT
#posts.remove
#posts.insert(#docs)
end
test "find examples" do
result = #posts.find( { :price => { '$gt' => 100000 } } ).to_a
assert(result.count == 0)
puts "result from post: #{result.to_json}"
#docs.each{|doc| doc.delete("_id"); doc["price"] = doc["price"].to_i}
#posts.insert(#docs)
result = #posts.find( { :price => { '$gt' => 100000 } } ).to_a
assert(result.count > 0)
puts "result after fixes: #{result.to_json}"
end
end
ruby test.rb
Loaded suite test
Started
result from post: []
result after fixes: [{"_id":{"$oid": "5355e4c3a3f57661f3000001"},"price":450000,"date_sold":"2013-10-23 00:00","post_code":"NW6 2DT","house_type":"F","condition":"N","freehold":"L","house_number":"72","flat_number":"FLAT 3","street":"LOVERIDGE ROAD","town":null,"district":"LONDON","region":"CAMDEN","county":"GREATER LONDON"}]
.
Finished in 0.00482 seconds.
1 tests, 2 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
207.47 tests/s, 414.94 assertions/s
I have a Webdriver test case written in Ruby as below, and I want to make it run for example 100 times:
require 'rubygems'
require 'selenium-webdriver'
$i = 0
$num = 100
while $i<$num do
driver = Selenium::WebDriver.for :firefox
driver.get "https://dev08-olb.nz.thenational.com/ib4b/app/login"
# some other test which require $i to be incremental as unique ID
driver.close
i++
end
It does like it.
Can you show me how to execute it as many time as I want?
Thanks.
Not sure what you are trying to do here but try this
num = 100
num.times do |i|
#INSERT YOUR BLOCK TO RUN HERE i will increment from 0..num
end
if you want to specify a different number of times each time it is called I would create a method
def run_times(n)
n.times do |i|
#INSERT YOUR BLOCK TO RUN HERE
end
end
Then you can call it with run_times 100
This question sounds stupid, but I never found an answer online to do this.
Assume you have a test suite like this page:
http://en.wikibooks.org/wiki/Ruby_Programming/Unit_testing
or code:
require "simpleNumber"
require "test/unit"
class TestSimpleNumber < Test::Unit::TestCase
def test_simple
assert_equal(4, SimpleNumber.new(2).add(2) )
assert_equal(4, SimpleNumber.new(2).multiply(2) )
end
def test_typecheck
assert_raise( RuntimeError ) { SimpleNumber.new('a') }
end
def test_failure
assert_equal(3, SimpleNumber.new(2).add(2), "Adding doesn't work" )
end
end
Running the code:
>> ruby tc_simpleNumber2.rb
Loaded suite tc_simpleNumber2
Started
F..
Finished in 0.038617 seconds.
1) Failure:
test_failure(TestSimpleNumber) [tc_simpleNumber2.rb:16]:
Adding doesn't work.
<3> expected but was
<4>.
3 tests, 4 assertions, 1 failures, 0 errors
Now, how to use a variable (what kind?) to save the testing results?
e.g., an array like this:
[{:name => 'test_simple', :status => :pass},
{:name => 'test_typecheck', :status => :pass},
{:name => 'test_failure', :status => :fail},]
I am new to testing, but desperate to know the answer...
you need to execute your test script file, that's it, the result will display pass or fails.
Suppose you save file test_unit_to_rspec.rb, after that execute below command
ruby test_unit_to_rspec.rb
Solved the problem with setting a high verbose level, in a test runner call.
http://ruby-doc.org/stdlib-1.8.7/libdoc/test/unit/rdoc/Test/Unit/UI/Console/TestRunner.html
require 'test/unit'
require 'test/unit/ui/console/testrunner'
class MySuperSuite < Test::Unit::TestSuite
def self.suite
suites = self.new("My Super Test Suite")
suites << TestSimpleNumber1
suites << TestSimpleNumber2
return suites
end
end
#run the suite
# Pass an io object
#new(suite, output_level=NORMAL, io=STDOUT)
runner = Test::Unit::UI::Console::TestRunner.new(MySuperSuite, 3, io)
results will be saved in the io stream in a nice format fo each test case.
What about using '-v' (verbose):
ruby test_unit_to_rspec.rb -v
This should show you a lot more information
You can check out another of Nat's posts for a way to capture the results. The short answer to your question is that there is no variable for capturing the results. All you get is:
Loaded suite My Special Tests
Started
..
Finished in 1.000935 seconds.
2 tests, 2 assertions, 0 failures, 0 errors
Which is not very helpful if you want to report to someone else what happened. Nat's other post shows how to wrap the Test::Unit in rspec to get a better result and more flexibility.
class Test::Unit::TestCase
def setup
#id = self.class.to_s()
end
def teardown
#test_result = "pass"
if(#_result.failure_count > 0 || #_result.error_count > 0)
#test_result = "fail"
# making sure no errors/failures exist before the next test case runs.
i = 0
while(i < #_result.failures.length) do
#_result.failures.delete_at(i)
i = i + 1
end
while(i < #_result.errors.length) do
#_result.errors.delete_at(i)
i = i + 1
end
#test_result = "fail"
end # if block ended
puts"#{#id}: #{#test_result}"
end # teardown definition ended
end # class Test::Unit::TestCase ended
Example Output :
test1: Pass
test2: fail
so on....