I have a method that returns an array of arrays.
For convenience I use collect on a collection to gather them together.
arr = collection.collect {|item| item.get_array_of_arrays}
Now I would like to have a single array that contains all the arrays.
Of course I can loop over the array and use the + operator to do that.
newarr = []
arr.each {|item| newarr += item}
But this is kind of ugly, is there a better way?
There is a method for flattening an array in Ruby: Array#flatten:
newarr = arr.flatten(1)
From your description it actually looks like you don't care about arr anymore, so there is no need to keep the old value of arr around, we can just modify it:
arr.flatten!(1)
(There is a rule in Ruby that says that if you have two methods that do basically the same thing, but one does it in a somewhat surprising way, you name that method the same as the other method but with an exlamation point at the end. In this case, both methods flatten an array, but the version with the exclamation point does it by destroying the original array.)
However, while in this particular case there actually is a method which does exactly what you want, there is a more general principle at work in your code: you have a sequence of things and you iterate over it and try to "reduce" it down into a single thing. In this case, it is hard to see, because you start out with an array and you end up with an array. But by changing just a couple of small details in your code, it all of the sudden becomes blindingly obvious:
sum = 0
arr.each {|item| sum += item } # assume arr is an array of numbers
This is exactly the same pattern.
What you are trying to do is known as a catamorphism in category theory, a fold in mathematics, a reduce in functional programming, inject:into: in Smalltalk and is implemented by Enumerable#inject and its alias Enumerable#reduce (or in this case actually Array#inject and Array#reduce) in Ruby.
It is very easy to spot: whenever you initialize an accumulator variable outside of a loop and then assign to it or modify the object it references during every iteration of the loop, then you have a case for reduce.
In this particular case, your accumulator is newarr and the operation is adding an array to it.
So, your loop could be more idiomatically rewritten like this:
newarr = arr.reduce(:+)
An experienced Rubyist would of course see this right away. However, even a newbie would eventually get there, by following some simple refactoring steps, probably similar to this:
First, you realize that it actually is a fold:
newarr = arr.reduce([]) {|acc, el| acc += el }
Next, you realize that assigning to acc is completely unnecessary, because reduce overwrites the contents of acc anyway with the result value of each iteration:
newarr = arr.reduce([]) {|acc, el| acc + el }
Thirdly, there is no need to inject an empty array as the starting value for the first iteration, since all the elements of arr are already arrays anyway:
newarr = arr.reduce {|acc, el| acc + el }
This can, of course, be further simplified by using Symbol#to_proc:
newarr = arr.reduce(&:+)
And actually, we don't need Symbol#to_proc here, because reduce and inject already accept a symbol parameter for the operation:
newarr = arr.reduce(:+)
This really is a general pattern. If you remember the sum example above, it would look like this:
sum = arr.reduce(:+)
There is no change in the code, except for the variable name.
arr.inject([]) { |main, item| main += item }
I don't seem to understand the question fully... Is Array#flatten what you are looking for?
[[:a,:b], [1,2,3], 'foobar'].flatten
# => [:a, :b, 1, 2, 3, 'foobar']
Related
Why is each loop preferred over for loop in Ruby? Is there a difference in time complexity or are they just syntactically different?
Yes, these are two different ways of iterating over, But hope this calculation helps.
require 'benchmark'
a = Array( 1..100000000 )
sum = 0
Benchmark.realtime {
a.each { |x| sum += x }
}
This takes 5.866932 sec
a = Array( 1..100000000 )
sum = 0
Benchmark.realtime {
for x in a
sum += x
end
}
This takes 6.146521 sec.
Though its not a right way to do the benchmarking, there are some other constraints too. But on a single machine, each seems to be a bit faster than for.
The variable referencing an item in iteration is temporary and does not have significance outside of the iteration. It is better if it is hidden from outside of the iteration. With external iterators, such variable is located outside of the iteration block. In the following, e is useful only within do ... end, but is separated from the block, and written outside of it; it does not look easy to a programmer:
for e in [:foo, :bar] do
...
end
With internal iterators, the block variable is defined right inside the block, where it is used. It is easier to read:
[:foo, :bar].each do |e|
...
end
This visibility issue is not just for a programmer. With respect to visibility in the sense of scope, the variable for an external iterator is accessible outside of the iteration:
for e in [:foo] do; end
e # => :foo
whereas in internal iterator, a block variable is invisible from outside:
[:foo].each do |e|; end
e # => undefined local variable or method `e'
The latter is better from the point of view of encapsulation.
When you want to nest the loops, the order of variables would be somewhat backwards with external iterators:
for a in [[:foo, :bar]] do
for e in a do
...
end
end
but with internal iterators, the order is more straightforward:
[[:foo, :bar]].each do |a|
a.each do |e|
...
end
end
With external iterators, you can only use hard-coded Ruby syntax, and you also have to remember the matching between the keyword and the method that is internally called (for calls each), but for internal iterators, you can define your own, which gives flexibility.
each is the Ruby Way. Implements the Iterator Pattern that has decoupling benefits.
Check also this: "for" vs "each" in Ruby
An interesting question. There are several ways of looping in Ruby. I have noted that there is a design principle in Ruby, that when there are multiple ways of doing the same, there are usually subtle differences between them, and each case has its own unique use, its own problem that it solves. So in the end you end up needing to be able to write (and not just to read) all of them.
As for the question about for loop, this is similar to my earlier question whethe for loop is a trap.
Basically there are 2 main explicit ways of looping, one is by iterators (or, more generally, blocks), such as
[1, 2, 3].each { |e| puts e * 10 }
[1, 2, 3].map { |e| e * 10 )
# etc., see Array and Enumerable documentation for more iterator methods.
Connected to this way of iterating is the class Enumerator, which you should strive to understand.
The other way is Pascal-ish looping by while, until and for loops.
for y in [1, 2, 3]
puts y
end
x = 0
while x < 3
puts x; x += 1
end
# same for until loop
Like if and unless, while and until have their tail form, such as
a = 'alligator'
a.chop! until a.chars.last == 'g'
#=> 'allig'
The third very important way of looping is implicit looping, or looping by recursion. Ruby is extremely malleable, all classes are modifiable, hooks can be set up for various events, and this can be exploited to produce most unusual ways of looping. The possibilities are so endless that I don't even know where to start talking about them. Perhaps a good place is the blog by Yusuke Endoh, a well known artist working with Ruby code as his artistic material of choice.
To demonstrate what I mean, consider this loop
class Object
def method_missing sym
s = sym.to_s
if s.chars.last == 'g' then s else eval s.chop end
end
end
alligator
#=> "allig"
Aside of readability issues, the for loop iterates in the Ruby land whereas each does it from native code, so in principle each should be more efficient when iterating all elements in an array.
Loop with each:
arr.each {|x| puts x}
Loop with for:
for i in 0..arr.length
puts arr[i]
end
In the each case we are just passing a code block to a method implemented in the machine's native code (fast code), whereas in the for case, all code must be interpreted and run taking into account all the complexity of the Ruby language.
However for is more flexible and lets you iterate in more complex ways than each does, for example, iterating with a given step.
EDIT
I didn't come across that you can step over a range by using the step() method before calling each(), so the flexibility I claimed for the for loop is actually unjustified.
[1,2,3].partition.inject(0) do |acc, x|
x>2 # this line is intended to be used by `partition`
acc+=x # this line is intended to be used by `inject`
end
I know that I can write above stanza using different methods but this is not important here.
What I want to ask why somebody want to use partition (or other methods like keep_if, delete_if) at the beginning of the "chain"?
In my example, after I chained inject I couldn't use partition. I can write above stanza using each:
[1,2,3].each.inject(0) do |acc, x|
x>2 # this line is intended to be used by `partition`
acc+=x # this line is intended to be used by `inject`
end
and it will be the same, right?
I know that x>2 will be discarded (and not used) by partition. Only acc+=x will do the job (sum all elements in this case).
I only wrote that to show my "intention": I want to use partition in the chain like this [].partition.inject(0).
I know that above code won't work as I intended and I know that I can chain after block( }.map as mentioned by Neil Slater).
I wanted to know why, and when partition (and other methods like keep_if, delete_if etc) becomes each (just return elements of the array as partition do in the above cases).
In my example, partition.inject, partition became each because partition cannot take condition (x>2).
However partition.with_index (as mentioned by Boris Stitnicky) works (I can partition array and use index for whatever I want):
shuffled_array
.partition
.with_index { |element, index|
element > index
}
ps. This is not question about how to get sum of elements that are bigger than 2.
This is an interesting situation. Looking at your code examples, you are obviously new to Ruby and perhaps also to programming. Yet you managed to ask a very difficult question that basically concerns the Enumerator class, one of the least publicly understood classes, especially since Enumerator::Lazy was introduced. To me, your question is difficult enough that I am not able to provide a comprehensive answer. Yet the remarks about your code would not fit into a comment under the OP. That's why I'm adding this non-answer.
First of all, let us notice a few awful things in your code:
Useless lines. In both blocks, x>2 line is useless, because its return value is discarded.
[1,2,3].partition.inject(0) do |x, acc|
x>2 # <---- return value of this line is never used
acc+=x
end
[1,2,3].each.inject(0) do |x, acc|
x>2 # <---- return value of this line is never used
acc+=x
end
I will ignore this useless line when discussing your code examples further.
Useless #each method. It is useless to write
[1,2,3].each.inject(0) do |x, acc|
acc+=x
end
This is enough:
[1,2,3].inject(0) do |x, acc|
acc+=x
end
Useless use of #partition method. Instead of:
[1,2,3].partition.inject(0) do |x, acc|
acc+=x
end
You can just write this:
[1,2,3].inject(0) do |x, acc|
acc+=x
end
Or, as I would write it, this:
[ 1, 2, 3 ].inject :+
But then, you ask a deep question about using #partition method in the enumerator mode. Having discussed the trivial newbie problems of your code, we are left with the question how exactly the enumerator-returning versions of the #partition, #keep_if etc. should be used, or rather, what are the interesting way of using them, because everyone knows that we can use them for chaining:
array = [ *1..6 ]
shuffled_arrray = array.shuffle # randomly shuffles the array elements
shuffled_array
.partition # partition enumerator comes into play
.with_index { |element, index| # method Enumerator#with_index comes into play
element > index # and partitions elements into those greater
} # than their index, and those smaller
And also like this:
e = partition_enumerator_of_array = array.partition
# And then, we can partition the array in many ways:
e.each &:even? # partitions into odd / even numbers
e.each { rand() > 0.5 } # partitions the array randomly
# etc.
An easily understood advantage is that instead of writing longer:
array.partition &:even?
You can write shorter:
e.each &:even?
But I am basically sure that enumerators provide more power to the programmer than just chaining collection methods and shortening code a little bit. Because different enumerators do very different things. Some, such as #map! or #reject!, can even modify the collection on which they operate. In this case, it is imaginable that one could combine different enumerators with the same block to do different things. This ability to vary not just the blocks, but also the enumerators to which they are passed, gives combinatorial power, which can very likely be used to make some otherwise lengthy code very concise. But I am unable to provide a very useful concrete example of this.
In sum, Enumerator class is here mainly for chaining, and to use chaining, programmers do not really need to undestand Enumerator in detail. But I suspect that the correct habits regarding the use of Enumerator might be as difficult to learn as, for instance, correct habits of parametrized subclassing. I suspect I have not grasped the most powerful ways to use enumerators yet.
I think that the result [3, 3] is what you are looking for here - partitioning the array into smaller and larger numbers then summing each group. You seem to be confused about how you give the block "rules" to the two different methods, and have merged what should be two blocks into one.
If you need the net effects of many methods that each take a block, then you can chain after any block, by adding the .method after the close of the block like this: }.each or end.each
Also note that if you create partitions, you are probably wanting to sum over each partition separately. To do that you will need an extra link in the chain (in this case a map):
[1,2,3].partition {|x| x > 2}.map do |part|
part.inject(0) do |acc, x|
x + acc
end
end
# => [3, 3]
(You also got the accumulator and current value wrong way around in the inject, and there is no need to assign to the accumulator, Ruby does that for you).
The .inject is no longer in a method chain, instead it is inside a block. There is no problem with blocks inside other blocks, in fact you will see this very often in Ruby code.
I have chained .partition and .map in the above example. You could also write the above like this:
[1,2,3].partition do
|x| x > 2
end.map do |part|
part.inject(0) do |acc, x|
x + acc
end
end
. . . although when chaining with short blocks, I personally find it easier to use the { } syntax instead of do end, especially at the start of a chain.
If it all starts to look complex, there is not usually a high cost to assigning the results of the first part of a chain to a local variable, in which case there is no chain at all.
parts = [1,2,3].partition {|x| x > 2}
parts.map do |part|
part.inject(0) do |acc, x|
x + acc
end
end
Let's say I have an array
arry = ["a","b","c","d","e","f","g"]
and I want to get the fourth, second, and seventh element of that array.
Let's say some other function determined which are the needed values and passed me backan array of indices like [3, 1, 6]. In most languages I could write this
array[3,1,6] #=> ["d","b","g"] (or "dbg"?)
or
array[3 1 6] #=> ["d","b","g"]
But that won't work in Ruby, of course. Is there a simple way to do this in Ruby? The cleanest I can find is something like:
[3,1,6].map { |i| array[i] }
I really tried to find a duplicate to this question since it seems so basic, but I just couldn't.
This is remarkably easy to do in most languages, so I'm almost assuming I'm just overlooking the remarkably obvious.
array.values_at(3,1,6) #=> ["d","b","g"]
If your other function returns an array then you must flatten it before you can pass it as parameters to values_at. Like this
arry = ["a","b","c","d","e","f","g"]
def indices
[1,3,6]
end
puts arry.values_at(*indices)
output
b
d
g
I was reading a Ruby question about the .each iterator, and someone stated that using .each can be a code smell if higher order iterators are better suited for the task. What are higher order iterators in Ruby?
edit: Jörg W Mittag, the author of the StackOverflow answer that I was referring to mentioned that he meant to write higher level iterators, but he also explained what they are very well below.
Oops. I meant higher-level iterators, not higher-order. Every iterator is of course by definition higher-order.
Basically, iteration is a very low-level concept. The purpose of programming is to communicate intent to the other stakeholders on the team. "Initializing an empty array, then iterating over another array and adding the current element of this array to the first array if it is divisible by two without a remainder" is not communicating intent. "Selecting all even numbers" is.
In general, you almost never iterate over a collection just for iteration's sake. You either want to
transform each element in some way (that's usually called map, in Ruby and Smalltalk it's collect and in .NET and SQL it's Select),
reduce the whole collection down to some single value, e.g. computing the sum or the average or the standard deviation of a list of football scores (in category theory, that's called a catamorphism, in functional programming it is fold or reduce, in Smalltalk it's inject:into:, in Ruby it's inject and in .NET Aggregate),
filter out all elements that satisfy a certain condition (filter in most functional languages, select in Smalltalk and Ruby, also find_all in Ruby, Where in .NET and SQL),
filter out all elements that do not satisfy a condition (reject in Smalltalk and Ruby)
find the first element that satisfies a condition (find in Ruby)
count the elements thats satisfy a condition (count in Ruby)
check if all elements (all?), at least one element (any?) or no elements (none?) satisfy a condition
group the elements into buckets based on some discriminator (group_by in Ruby, .NET and SQL)
partition the collection into two collections based on some predicate (partition)
sort the collection (sort, sort_by)
combine multiple collections into one (zip)
and so on and so forth …
Almost never is your goal to just iterate over a collection.
In particular, reduce aka. inject aka. fold aka. inject:into: aka. Aggregate aka. catamorphism is your friend. There's a reason why it has such a fancy-sounding mathematical name: it is extremely powerful. In fact, most of what I mentioned above, can be implemented in terms of reduce.
Basically, what reduce does, is it "reduces" the entire collection down to a single value, using some function. You have some sort of accumulator value, and then you take the accumulator value and the first element and feed it into the function. The result of that function then becomes the new accumulator, which you pair up with the second element and feed to the function and so on.
The most obvious example of this is summing a list of numbers:
[4, 8, 15, 16, 23, 42].reduce(0) {|acc, elem|
acc + elem
}
So, the accumulator starts out as 0, and we pass the first element 4 into the + function. The result is 4, which becomes the new accumulator. Now we pass the next element 8 in and the result is 12. And this continues till the last element and the result is that they were dead the whole time. No, wait, the result is 108.
Ruby actually allows us to take a couple of shortcuts: If the element type is the same as the accumulator type, you can leave out the accumulator and Ruby will simply pass the first element as the first value for the accumulator:
[4, 8, 15, 16, 23, 42].reduce {|acc, elem|
acc + elem
}
Also, we can use Symbol#to_proc here:
[4, 8, 15, 16, 23, 42].reduce(&:+)
And actually, if you pass reduce a Symbol argument it will treat as the name of the function to use for the reduction operation:
[4, 8, 15, 16, 23, 42].reduce(:+)
However, summing is not all that reduce can do. In fact, I find this example a little dangerous. Everybody I showed this to, immediately understood, "Aah, so that's what a reduce is", but unortunately some also thought that summing numbers is all reduce is, and that's definitely not the case. In fact, reduce is a general method of iteration, by which I mean that reduce can do anything that each can do. In particular, you can store arbitrary state in the accumulator.
For example, I wrote above that reduce reduces the collection down to a single value. But of course that "single value" can be arbitrarily complex. It could, for example, be itself a collection. Or a string:
class Array
def mystery_method(foo)
drop(1).reduce("#{first}") {|s, el| s << foo.to_str << el.to_s }
end
end
This is an example how far you can go with playing tricks with the accumulator. If you try it out, you'll of course recognize it as Array#join:
class Array
def join(sep=$,)
drop(1).reduce("#{first}") {|s, el| s << sep.to_str << el.to_s }
end
end
Note that nowhere in this "loop" do I have to keep track of whether I'm at the last or second-to-last element. Nor is there any conditional in the code. There is no potential for fencepost errors here. If you think about how to implement this with each, you would have to somehow keep track of the index and check whether you are at the last element and then have an if in there, to prevent emitting the separator at the end.
Since I wrote above that all iteration can be done with reduce, I might just as well prove it. Here's Ruby's Enumerable methods, implemented in terms of reduce instead of each as they normally are. (Note that I only just started and have only arrived at g yet.)
module Enumerable
def all?
reduce(true) {|res, el| res && yield(el) }
end
def any?
reduce(false) {|res, el| res || yield(el) }
end
alias_method :map, def collect
reduce([]) {|res, el| res << yield(el) }
end
def count
reduce(0) {|res, el| res + 1 if yield el }
end
alias_method :find, def detect
reduce(nil) {|res, el| if yield el then el end unless res }
end
def drop(n=1)
reduce([]) {|res, el| res.tap {|res| res << el unless n -= 1 >= 0 }}
end
def drop_while
reduce([]) {|res, el| res.tap {|res| res << el unless yield el }}
end
def each
reduce(nil) {|_, el| yield el }
end
def each_with_index
tap { reduce(-1) {|i, el| (i+1).tap {|i| yield el, i }}}
end
alias_method :select, def find_all
reduce([]) {|res, el| res.tap {|res| res << el if yield el }}
end
def grep(pattern)
reduce([]) {|res, el| res.tap {|res| res << yield(el) if pattern === el }}
end
def group_by
reduce(Hash.new {|hsh, key| hsh[key] = [] }) {|res, el| res.tap {|res|
res[yield el] = el
}}
end
def include?(obj)
reduce(false) {|res, el| break true if res || el == obj }
end
def reject
reduce([]) {|res, el| res.tap {|res| res << el unless yield el }}
end
end
[Note: I made some simplifications for the purpose of this post. For example, according to the standard Ruby Enumerable protocol, each is supposed to return self, so you'd have to slap an extra line in there; other methods behave slightly differently, depending on what kind and how many arguments you pass in and so on. I left those out because they distract from the point I am trying to make.]
They're talking about more specialized methods such as map, filter or inject. For example, instead of this:
even_numbers = []
numbers.each {|num| even_numbers << num if num.even?}
You should do this:
even_numbers = numbers.select {|num| num.even?}
It says what you want to do but encapsulates all the irrelevant technical details in the select method. (And incidentally, in Ruby 1.8.7 or later, you can just write even_numbers = numbers.select(&:even?), so even more concise if slightly Perl-like.)
These aren't normally called "higher-order iterators," but whoever wrote that probably just had a minor mental mixup. It's a good principle whatever terminology you use.
From the usual definition of "higer-order" I would say a higher-order iterator is an iterator which takes an iterator as an argument or returns an iterator. So something like enum_for maybe. However I don't think this is what the person meant.
I think the person meant iterators like map or select which are higher-order functions, but did not realize that each is of course also a higher-order function. So basically this is just a case of terminological confusion.
The point of the poster presumably was that you should not use each in cases where map, select or inject could naturally be used instead. And to make that point he used a term, that didn't really make sense in that context.
I get this question a fair bit, so I blogged about the most commonly used iterators: select and reject. In the post there are examples of where 'each' is used incorrectly and how to correct the code to use either 'select' or 'reject'. Anyway, I hope it helps.
http://www.natontesting.com/2011/01/01/rubys-each-select-and-reject-methods/
I just wrote a blog that is very relavent to this question - The reason you want to use higher order functions is that doing so elevates the programmer to a higher level of abstraction, to a point that a problem can be expressed declaratively, pushing the implementation down either to the Ruby standard library or lower level code.
http://www.railstutors.com/blog/declarative-thinking-with-higher-order-functions-and-blocks#.UG5x6fl26jJ
I'm just learning Ruby so apologies if this is too newbie for around here, but I can't work this out from the pickaxe book (probably just not reading carefully enough).
Anyway, if I have an array like so:
arr = [1,2,3,4,5]
...and I want to, say, multiply each value in the array by 3, I have worked out that doing the following:
arr.each {|item| item *= 3}
...will not get me what I want (and I understand why, I'm not modifying the array itself).
What I don't get is how to modify the original array from inside the code block after the iterator. I'm sure this is very easy.
Use map to create a new array from the old one:
arr2 = arr.map {|item| item * 3}
Use map! to modify the array in place:
arr.map! {|item| item * 3}
See it working online: ideone
To directly modify the array, use arr.map! {|item| item*3}. To create a new array based on the original (which is often preferable), use arr.map {|item| item*3}. In fact, I always think twice before using each, because usually there's a higher-order function like map, select or inject that does what I want.
arr.collect! {|item| item * 3}
Others have already mentioned that array.map is the more elegant solution here, but you can simply add a "!" to the end of array.each and you can still modify the array. Adding "!" to the end of #map, #each, #collect, etc. will modify the existing array.