I've got an assignment where I have had to write a function, which takes two integers and returns it's bitwise product. Bitwise product equals to bitwise sum (&) of all numbers between these two.
For example: bitwise product of 251 and 253 is:
irb(main):164:0> 251 & 252
=> 248
irb(main):165:0> 252 & 253
=> 252
irb(main):166:0> 248 & 252
=> 248 # this a bitwise & of these two between 251 & 253
My function:
def solution(m,n)
(m..n).to_a.inject{|sum, x| sum &= x}
end
Test:
irb(main):160:0> (251..253).to_a.inject{|sum, x| sum &= x}
=> 248 #same result
The program's evaluation:
correctness 100%
performance 0%
Can anyone, please explain what is with performance of this function? Thanks in advance!
EDIT
Since performance is undefined here, can you provide an analysis of the function for, for example, really big inputs and criticise/suggest better/more efficient solution?
p.s. thx for comments guys, i will follow the recommendations!
Following up on my comment, you can experiment using Benchmark:
def solution(m,n)
(m..n).to_a.inject{|sum, x| sum &= x}
end
def solution2(m,n)
(m..n).reduce(:&)
end
n = 50000
Benchmark.bm(7) do |b|
b.report("orig:") { n.times do; solution(128,253); end }
b.report("new: ") { n.times do; solution2(128,253); end }
end
Here's what I get:
user system total real
orig: 1.560000 0.000000 1.560000 ( 1.557156)
new: 0.640000 0.000000 0.640000 ( 0.634063)
You appear not to have any choice but to do the above, experiment to find a faster algorithm, and run the test until you register on the performance meter.
The most obvious performance problem here is a conversion to array (to_a), it's not necassary. You can call reduce and inject (appear to be the same) an any Enumerable, Range includes it as well. Enumerable is basically everything that can produce a finite (it's arguable, but mostly true) sequence of elements. Range seems to fit, if treated as a set of integers.
In order to iterate over an array with elements in this range, you first create an array and fill it with elements (by iterating over Range). Then you iterate on the resulting array applying your operations. So you create an array for the purpose of iterating on it, fill it, use it, toss it away. And it takes quite a memory allocation to do that if the range is large.
The next one, which is not critical, but reduces the amount of code that you have, requires some knowledge on the internal implementation of operators in Ruby. When you write a&b you are actually doing a.&(b): calling a & method on a number.
The sum you get in the block is not really some sort of accumulator, you don't actually need to assign anything there, sum is an intermediate value, the result of adding a new element should actually be a return value. Since an assignment returns the assigned value, it works even this way. Proof of that is this:
(251..253).inject{|sum, x| sum & x } # => 248, as expected
...and it turns out, this block is trivial: take one value, call a method on it with another value. Since inject takes a pair of values on each iteration, you can just give it a method name and let it handle the situation. Method names in Ruby are typically referenced with symbols like this:
(251..253).inject(:&) # => 248, as expected
All right, about twice as less code, less actions done and less objects made, nevertheless same result.
def solution(m,n)
(m..n).inject(:&)
end
We haven't looked closely at inject. Can you beat inject performance-wise? It's unlikely, it's a C method and is used here exactly for its purpose, you can look it up easily using gems pry and pry-doc (install, launch pry and type in show-source Range#inject):
[11] pry(main)> show-source Range#inject
From: enum.c (C Method):
Owner: Enumerable
Visibility: public
Number of lines: 34
...a bunch of C code here
Related
Substrings work where "hello"[0..2] returns "hel"
Is there an integer equivalent, that returns the sub-bits, as in 9[1..3] returns 4 or "100"?
Context:
I'm trying to do a genetic algorithm, and there is a cross over stage where you split two chromosomes in two, then swap halves, e.g. "10101010" crossed with "00000000" at i = 4 would be "00001010", "10100000"
However, I'm doing this a few million times, so using strings is inefficient, so is there a bitwise way to do this?
You might consider refining the class Fixnum to permit Fixnum#[]'s argument to be either a bit index or a range of bit indices.
module StabBitRange
def [](arg)
case arg
when Range
mask = arg.reduce(0) { |n, idx| n |= (1 << idx) }
(self & mask) >> arg.first
else
super
end
end
end
module StabBits
refine Fixnum do
prepend StabBitRange
end
end
See Refinements and Module#prepend. Prepend StabBitRange moves the methods contained therein (here just one) before the methods with the same names in Fixnum in the ancestor chain. The alternative (commonly used before Prepend was introduced in Ruby v2.0) is to alias Fixnum#[] and then redefine Fixnum[] to be the new method that takes either a bit index or a range of bit indices. (See the edit history of this answer to see how that is done.) I expect readers will agree that Prepend is a more sensible way of doing that.
To make this code available for use we need only invoke the keyword using.
using StabBits
n = 682
n.to_s(2) #=> "1010101010"
n[0] #=> 0
n[1] #=> 1
n[2] #=> 0
n[0..7] #=> 170 170.to_s(2) => "10101010"
n[1..7] #=> 85 85.to_s(2) => "1010101"
n[2..6] #=> 10 10.to_s(2) => "1010"
When StabBitRange#\[\] is called and the argument is not a range, super invokes Fixnum#[] and passes the argument. That method handles argument errors as well as returning the desired bit when there are no errors.
When this code is run in IRB, the following exception is raised: RuntimeError: main.using is permitted only at top level. That's because IRB is running at the top level, but code run within IRB is running at a higher level from the perspective of the Ruby parser. To run this in IRB it is necessary to enclose using StabBits and the code following in a module.
Why is each loop preferred over for loop in Ruby? Is there a difference in time complexity or are they just syntactically different?
Yes, these are two different ways of iterating over, But hope this calculation helps.
require 'benchmark'
a = Array( 1..100000000 )
sum = 0
Benchmark.realtime {
a.each { |x| sum += x }
}
This takes 5.866932 sec
a = Array( 1..100000000 )
sum = 0
Benchmark.realtime {
for x in a
sum += x
end
}
This takes 6.146521 sec.
Though its not a right way to do the benchmarking, there are some other constraints too. But on a single machine, each seems to be a bit faster than for.
The variable referencing an item in iteration is temporary and does not have significance outside of the iteration. It is better if it is hidden from outside of the iteration. With external iterators, such variable is located outside of the iteration block. In the following, e is useful only within do ... end, but is separated from the block, and written outside of it; it does not look easy to a programmer:
for e in [:foo, :bar] do
...
end
With internal iterators, the block variable is defined right inside the block, where it is used. It is easier to read:
[:foo, :bar].each do |e|
...
end
This visibility issue is not just for a programmer. With respect to visibility in the sense of scope, the variable for an external iterator is accessible outside of the iteration:
for e in [:foo] do; end
e # => :foo
whereas in internal iterator, a block variable is invisible from outside:
[:foo].each do |e|; end
e # => undefined local variable or method `e'
The latter is better from the point of view of encapsulation.
When you want to nest the loops, the order of variables would be somewhat backwards with external iterators:
for a in [[:foo, :bar]] do
for e in a do
...
end
end
but with internal iterators, the order is more straightforward:
[[:foo, :bar]].each do |a|
a.each do |e|
...
end
end
With external iterators, you can only use hard-coded Ruby syntax, and you also have to remember the matching between the keyword and the method that is internally called (for calls each), but for internal iterators, you can define your own, which gives flexibility.
each is the Ruby Way. Implements the Iterator Pattern that has decoupling benefits.
Check also this: "for" vs "each" in Ruby
An interesting question. There are several ways of looping in Ruby. I have noted that there is a design principle in Ruby, that when there are multiple ways of doing the same, there are usually subtle differences between them, and each case has its own unique use, its own problem that it solves. So in the end you end up needing to be able to write (and not just to read) all of them.
As for the question about for loop, this is similar to my earlier question whethe for loop is a trap.
Basically there are 2 main explicit ways of looping, one is by iterators (or, more generally, blocks), such as
[1, 2, 3].each { |e| puts e * 10 }
[1, 2, 3].map { |e| e * 10 )
# etc., see Array and Enumerable documentation for more iterator methods.
Connected to this way of iterating is the class Enumerator, which you should strive to understand.
The other way is Pascal-ish looping by while, until and for loops.
for y in [1, 2, 3]
puts y
end
x = 0
while x < 3
puts x; x += 1
end
# same for until loop
Like if and unless, while and until have their tail form, such as
a = 'alligator'
a.chop! until a.chars.last == 'g'
#=> 'allig'
The third very important way of looping is implicit looping, or looping by recursion. Ruby is extremely malleable, all classes are modifiable, hooks can be set up for various events, and this can be exploited to produce most unusual ways of looping. The possibilities are so endless that I don't even know where to start talking about them. Perhaps a good place is the blog by Yusuke Endoh, a well known artist working with Ruby code as his artistic material of choice.
To demonstrate what I mean, consider this loop
class Object
def method_missing sym
s = sym.to_s
if s.chars.last == 'g' then s else eval s.chop end
end
end
alligator
#=> "allig"
Aside of readability issues, the for loop iterates in the Ruby land whereas each does it from native code, so in principle each should be more efficient when iterating all elements in an array.
Loop with each:
arr.each {|x| puts x}
Loop with for:
for i in 0..arr.length
puts arr[i]
end
In the each case we are just passing a code block to a method implemented in the machine's native code (fast code), whereas in the for case, all code must be interpreted and run taking into account all the complexity of the Ruby language.
However for is more flexible and lets you iterate in more complex ways than each does, for example, iterating with a given step.
EDIT
I didn't come across that you can step over a range by using the step() method before calling each(), so the flexibility I claimed for the for loop is actually unjustified.
I have a hash:
students = {
class1: 11,
class2: 24,
class3: 38,
class4: 62
}
I want there to be four lines of output:
1) 11
2) 35 #11 + 24
3) 73 #35 + 38
4) 135 #73 + 62
It goes through each element, and adds a value to a counter, printing each iteration as it goes. I need something like:
students.each do |key, value|
value + counter = total
puts total
end
but I have no idea how to do it. Please advise.
There are many ways to do this, but I will suggest a way that will teach you a few different things about Ruby. This is also a very Ruby-like way to address this problem.
Code
students = {
class1: 11,
class2: 24,
class3: 38,
class4: 62 }
students.reduce(0) do |tot, (k,v)|
tot += v
puts k[/\d+/] + ") #{tot}"
tot
end
1) 11
2) 35
3) 73
4) 135
Explanation
I've used Enumerable#reduce (a.k.a. inject) because that method is convenient for totaling a collection of numbers, using a variable (here tot) that maintains the running total within the block. That's just what you need.
Aside: you will learn a lot be reading the documentation for Ruby methods. Methods are referenced like this: SomeClass#method or SomeModule#method. Here, reduce is an instance method of the module Enumerable. students is an instance of the class Hash, but that class "mixes-in" (includes) the instance methods of the module Enumerable.
The object tot is a Fixnum that is initialized to reduce's argument, which here is zero. (If no initial value were given, the initial value from student--11--would be assigned to tot). Each time the code in the block is executed the value at the end of the block is returned to the enumerator (which is why tot is there). After all the elements of the receiver students have been enumerated, the value of tot is returned by reduce (though you will not be making use of that).
The first time the block is called the block variables are as follows:
tot => 0
k => :class1
v => 11
To print
1) 11
I presume you want the label 1) to be the right end of :class1. To exact 1 from the symbol k => :class1, you can use the method Symbol#[] with the regex /\d+/, which extracts a string of one or more digits 0-9 (as many as there are).
In reading the documentation for the method Symbol#[], you will see that it converts the symbol :class1 to the string "class1" and then invokes the method String[] on that string.
Since Ruby 1.9+, many prefer to use Enumerable#each_with_object rather than reduce. That method would be used like this:
students.each_with_object(0) do |(k,v),tot|
tot += v
puts k[/\d+/] + ") #{tot}"
end
Notice that with this method it is not necessary to return the value of the object (tot) to the enumerator, and that tot is at the end of the list of block variables, whereas it is at the beginning for reduce.
You were actually fairly close. You need to define the total variable outside of your block, just assign 0 to it using
total=0
this will make it stay through all the iterations. Then you just need a small change in
students.each do |key, value|
total=total+value
puts total
end
and it will do what you want.
Switching the order of value + counter = total is important, as assignments (through =) always assign to the variable on the left.
Using your code, made some small change:-- Try this:--
total = 0
students.each do |key, value|
total += value
puts total
end
[1,2,3].partition.inject(0) do |acc, x|
x>2 # this line is intended to be used by `partition`
acc+=x # this line is intended to be used by `inject`
end
I know that I can write above stanza using different methods but this is not important here.
What I want to ask why somebody want to use partition (or other methods like keep_if, delete_if) at the beginning of the "chain"?
In my example, after I chained inject I couldn't use partition. I can write above stanza using each:
[1,2,3].each.inject(0) do |acc, x|
x>2 # this line is intended to be used by `partition`
acc+=x # this line is intended to be used by `inject`
end
and it will be the same, right?
I know that x>2 will be discarded (and not used) by partition. Only acc+=x will do the job (sum all elements in this case).
I only wrote that to show my "intention": I want to use partition in the chain like this [].partition.inject(0).
I know that above code won't work as I intended and I know that I can chain after block( }.map as mentioned by Neil Slater).
I wanted to know why, and when partition (and other methods like keep_if, delete_if etc) becomes each (just return elements of the array as partition do in the above cases).
In my example, partition.inject, partition became each because partition cannot take condition (x>2).
However partition.with_index (as mentioned by Boris Stitnicky) works (I can partition array and use index for whatever I want):
shuffled_array
.partition
.with_index { |element, index|
element > index
}
ps. This is not question about how to get sum of elements that are bigger than 2.
This is an interesting situation. Looking at your code examples, you are obviously new to Ruby and perhaps also to programming. Yet you managed to ask a very difficult question that basically concerns the Enumerator class, one of the least publicly understood classes, especially since Enumerator::Lazy was introduced. To me, your question is difficult enough that I am not able to provide a comprehensive answer. Yet the remarks about your code would not fit into a comment under the OP. That's why I'm adding this non-answer.
First of all, let us notice a few awful things in your code:
Useless lines. In both blocks, x>2 line is useless, because its return value is discarded.
[1,2,3].partition.inject(0) do |x, acc|
x>2 # <---- return value of this line is never used
acc+=x
end
[1,2,3].each.inject(0) do |x, acc|
x>2 # <---- return value of this line is never used
acc+=x
end
I will ignore this useless line when discussing your code examples further.
Useless #each method. It is useless to write
[1,2,3].each.inject(0) do |x, acc|
acc+=x
end
This is enough:
[1,2,3].inject(0) do |x, acc|
acc+=x
end
Useless use of #partition method. Instead of:
[1,2,3].partition.inject(0) do |x, acc|
acc+=x
end
You can just write this:
[1,2,3].inject(0) do |x, acc|
acc+=x
end
Or, as I would write it, this:
[ 1, 2, 3 ].inject :+
But then, you ask a deep question about using #partition method in the enumerator mode. Having discussed the trivial newbie problems of your code, we are left with the question how exactly the enumerator-returning versions of the #partition, #keep_if etc. should be used, or rather, what are the interesting way of using them, because everyone knows that we can use them for chaining:
array = [ *1..6 ]
shuffled_arrray = array.shuffle # randomly shuffles the array elements
shuffled_array
.partition # partition enumerator comes into play
.with_index { |element, index| # method Enumerator#with_index comes into play
element > index # and partitions elements into those greater
} # than their index, and those smaller
And also like this:
e = partition_enumerator_of_array = array.partition
# And then, we can partition the array in many ways:
e.each &:even? # partitions into odd / even numbers
e.each { rand() > 0.5 } # partitions the array randomly
# etc.
An easily understood advantage is that instead of writing longer:
array.partition &:even?
You can write shorter:
e.each &:even?
But I am basically sure that enumerators provide more power to the programmer than just chaining collection methods and shortening code a little bit. Because different enumerators do very different things. Some, such as #map! or #reject!, can even modify the collection on which they operate. In this case, it is imaginable that one could combine different enumerators with the same block to do different things. This ability to vary not just the blocks, but also the enumerators to which they are passed, gives combinatorial power, which can very likely be used to make some otherwise lengthy code very concise. But I am unable to provide a very useful concrete example of this.
In sum, Enumerator class is here mainly for chaining, and to use chaining, programmers do not really need to undestand Enumerator in detail. But I suspect that the correct habits regarding the use of Enumerator might be as difficult to learn as, for instance, correct habits of parametrized subclassing. I suspect I have not grasped the most powerful ways to use enumerators yet.
I think that the result [3, 3] is what you are looking for here - partitioning the array into smaller and larger numbers then summing each group. You seem to be confused about how you give the block "rules" to the two different methods, and have merged what should be two blocks into one.
If you need the net effects of many methods that each take a block, then you can chain after any block, by adding the .method after the close of the block like this: }.each or end.each
Also note that if you create partitions, you are probably wanting to sum over each partition separately. To do that you will need an extra link in the chain (in this case a map):
[1,2,3].partition {|x| x > 2}.map do |part|
part.inject(0) do |acc, x|
x + acc
end
end
# => [3, 3]
(You also got the accumulator and current value wrong way around in the inject, and there is no need to assign to the accumulator, Ruby does that for you).
The .inject is no longer in a method chain, instead it is inside a block. There is no problem with blocks inside other blocks, in fact you will see this very often in Ruby code.
I have chained .partition and .map in the above example. You could also write the above like this:
[1,2,3].partition do
|x| x > 2
end.map do |part|
part.inject(0) do |acc, x|
x + acc
end
end
. . . although when chaining with short blocks, I personally find it easier to use the { } syntax instead of do end, especially at the start of a chain.
If it all starts to look complex, there is not usually a high cost to assigning the results of the first part of a chain to a local variable, in which case there is no chain at all.
parts = [1,2,3].partition {|x| x > 2}
parts.map do |part|
part.inject(0) do |acc, x|
x + acc
end
end
So I need to get all possible permutations of a string.
What I have now is this:
def uniq_permutations string
string.split(//).permutation.map(&:join).uniq
end
Ok, now what is my problem: This method works fine for small strings but I want to be able to use it with strings with something like size of 15 or maybe even 20. And with this method it uses a lot of memory (>1gb) and my question is what could I change not to use that much memory?
Is there a better way to generate permutation? Should I persist them at the filesystem and retrieve when I need them (I hope not because this might make my method slow)?
What can I do?
Update:
I actually don't need to save the result anywhere I just need to lookup for each in a table to see if it exists.
Just to reiterate what Sawa said. You do understand the scope? The number of permutations for any n elements is n!. It's about the most aggressive mathematical progression operation you can get. The results for n between 1-20 are:
[1, 2, 6, 24, 120, 720, 5040, 40320, 362880, 3628800, 39916800, 479001600,
6227020800, 87178291200, 1307674368000, 20922789888000, 355687428096000,
6402373705728000, 121645100408832000, 2432902008176640000]
Where the last number is approximately 2 quintillion, which is 2 billion billion.
That is 2265820000 gigabytes.
You can save the results to disk all day long - unless you own all the Google datacenters in the world you're going to be pretty much out of luck here :)
Your call to map(&:join) is what is creating the array in memory, as map in effect turns an Enumerator into an array. Depending on what you want to do, you could avoid creating the array with something like this:
def each_permutation(string)
string.split(//).permutation do |permutaion|
yield permutation.join
end
end
Then use this method like this:
each_permutation(my_string) do |s|
lookup_string(s) #or whatever you need to do for each string here
end
This doesn’t check for duplicates (no call to uniq), but avoids creating the array. This will still likely take quite a long time for large strings.
However I suspect in your case there is a better way of solving your problem.
I actually don't need to save the result anywhere I just need to lookup for each in a table to see if it exists.
It looks like you’re looking for possible anagrams of a string in an existing word list. If you take any two anagrams and sort the characters in them, the resulting two strings will be the same. Could you perhaps change your data structures so that you have a hash, with keys being the sorted string and the values being a list of words that are anagrams of that string. Then instead of checking all permutations of a new string against a list, you just need to sort the characters in the string, and use that as the key to look up the list of all strings that are permutations of that string.
Perhaps you don't need to generate all elements of the set, but rather only a random or constrained subset. I have written an algorithm to generate the m-th permutation in O(n) time.
First convert the key to a list representation of itself in the factorial number system. Then iteratively pull out the item at each index specified by the new list and of the old.
module Factorial
def factorial num; (2..num).inject(:*) || 1; end
def factorial_floor num
tmp_1 = 0
1.upto(1.0/0.0) do |counter|
break [tmp_1, counter - 1] if (tmp_2 = factorial counter) > num
tmp_1 = tmp_2 #####
end # #
end # #
end # returns [factorial, integer that generates it]
# for the factorial closest to without going over num
class Array; include Factorial
def generate_swap_list key
swap_list = []
key -= (swap_list << (factorial_floor key)).last[0] while key > 0
swap_list
end
def reduce_swap_list swap_list
swap_list = swap_list.map { |x| x[1] }
((length - 1).downto 0).map { |element| swap_list.count element }
end
def keyed_permute key
apply_swaps reduce_swap_list generate_swap_list key
end
def apply_swaps swap_list
swap_list.map { |index| delete_at index }
end
end
Now, if you want to randomly sample some permutations, ruby comes with Array.shuffle!, but this will let you copy and save permutations or to iterate through the permutohedral space. Or maybe there's a way to constrain the permutation space for your purposes.
constrained_generator_thing do |val|
Array.new(sample_size) {array_to_permute.keyed_permute val}
end
Perhaps I am missing the obvious, but why not do
['a','a','b'].permutation.to_a.uniq!