I have a mongodb collection like this
{"assigneeId" => 1000, "status" => 3, "starttime" => "2014 Feb 25", "numofdays => 6}
{"assigneeId" => 1000, "status" => 2, "starttime" => "2014 Jan 10", "numofdays => 6}
{"assigneeId" => 1000, "status" => 3, "starttime" => "2014 Jan 1", "numofdays => 20}
I wrote a MongoDB query to group the above collection with assigneeId whose status is 3 and add the value of numofdays like this
db.events.group ( { key: {assigneeId:1}, cond: {status: {$gte: 3}}, reduce: function (curr, result) {result.total++; result.numofdays += curr.numofdays;}, initial: {total:0, numofdays:0} })
This gives the output as expected from the Mongodb cli.
When I write the ruby code for executing the same query, I could not make it work. Am getting lot of different hits for similar problem but none of the format seems to work. I could not go past the syntax error in ruby for grouping function. Please note I am not using mongoid. If I could get this work with Aggregation framework, that is also fine with me.
This is the ruby query I wrote and the error for the same.
#events_h.group (
:cond => {:status => { '$in' => ['3']}},
:key => 'assigneeId',
:initial => {count:0},
:reduce => "function(x, y) {y.count += x.count;}"
)
Error I am getting
analyze.rb:43: syntax error, unexpected ',', expecting ')'
out = #events_h.group (["assigneeId"], { }, { }, "function() { }")
^
analyze.rb:43: syntax error, unexpected ')', expecting keyword_end
analyze.rb:83: syntax error, unexpected end-of-input, expecting keyword_end
Any help is much appreciated.
Thanks
The following test includes solutions for using both methods, group and aggregate.
I recommend that you use aggregate, it is faster and avoids JavaScript and consuming a V8 engine.
Hope that this helps.
test.rb
require 'mongo'
require 'test/unit'
class MyTest < Test::Unit::TestCase
def setup
#events_h = Mongo::MongoClient.new['test']['events_h']
#docs = [
{"assigneeId" => 1000, "status" => 3, "starttime" => "2014 Feb 25", "numofdays" => 6},
{"assigneeId" => 1000, "status" => 2, "starttime" => "2014 Jan 10", "numofdays" => 6},
{"assigneeId" => 1000, "status" => 3, "starttime" => "2014 Jan 1", "numofdays" => 20}]
#events_h.remove
#events_h.insert(#docs)
end
test "group" do
result = #events_h.group(
:cond => {:status => {'$in' => [3]}},
:key => 'assigneeId',
:initial => {numofdays: 0},
:reduce => "function(x, y) {y.numofdays += x.numofdays;}"
)
assert_equal(26, result.first['numofdays'])
puts "group: #{result.inspect}"
end
test "aggregate" do
result = #events_h.aggregate([
{'$match' => {:status => {'$in' => [3]}}},
{'$group' => {'_id' => '$status', 'numofdays' => {'$sum' => '$numofdays'}}}
])
assert_equal(26, result.first['numofdays'])
puts "aggregate: #{result.inspect}"
end
end
$ ruby test.rb
Loaded suite test
Started
aggregate: [{"_id"=>3, "numofdays"=>26}]
.group: [{"assigneeId"=>1000.0, "numofdays"=>26.0}]
.
Finished in 0.010575 seconds.
2 tests, 2 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
189.13 tests/s, 189.13 assertions/s
Related
Although there are similar questions at SO (example), for some reason the error is formatted in another way in my case (and also I understand the general meaning of FunctionClauseError), so let me ask "once more".
Here's the question I'm getting:
{:error, reason} ->
Logger.error(fn -> ["failed to fetch internal transactions for >blocks: ", inspect(reason)] end,
error_count: unique_numbers_count
)
gives (I've added line breaks)
2022-04-13T18:44:19.749 application=indexer fetcher=internal_transaction count=10 error_count=10 [error]
failed to fetch internal transactions for blocks: %FunctionClauseError{args: nil, arity: 1,
clauses: nil, function: :finalize, kind: nil, module: EthereumJSONRPC.Geth.Tracer}
so obviously reason is
%FunctionClauseError{args: nil, arity: 1, clauses: nil, function: :finalize, kind: nil,
module: EthereumJSONRPC.Geth.Tracer}
I wonder what do these bits mean and can they help with debugging?
In module EthereumJSONRPC.Geth.Tracer function finalize is defined as
defp finalize(%{stack: [top], calls: [calls]}) do
and an example of the argument passed to it (causing the error) is:
%{
calls: [[], [], [], []],
depth: 4,
stack: [
%{
"callType" => "staticcall",
"from" => nil,
"gas" => 432013,
"gasUsed" => 21661,
"input" => "0xefc4cfa.....",
"output" => "0x",
"outputLength" => 64,
"outputOffset" => 4346,
"to" => "0x0000000000000000000000000000000000000001",
"traceAddress" => [0, 0, 0],
"type" => "call",
"value" => "0x0"
},
%{
"callType" => "staticcall",
"from" => nil,
"gas" => 438139,
"gasUsed" => 1726,
"input" => "0xefc4c.......",
"output" => "0x",
"outputLength" => 64,
"outputOffset" => 4026,
"to" => "0x0000000000000000000000000000000000000001",
"traceAddress" => [0, 0],
"type" => "call",
"value" => "0x0"
},
%{
"callType" => "staticcall",
"from" => nil
"gas" => 445060,
"gasUsed" => 2521,
"input" => "0xefc4......",
"output" => "0x",
"outputLength" => 64,
"outputOffset" => 3706,
"to" => "0x0000000000000000000000000000000000000001",
"traceAddress" => [0],
"type" => "call",
"value" => "0x0"
},
%{
"callType" => "call",
"from" => "0x9a66644084108a1bc23a9ccd50d6d63e53098db6",
"gas" => 460960,
"gasUsed" => 11500,
"input" => "0xba2c.........",
"output" => "0x",
"to" => "0x841ce48f9446c8e281d3f1444cb859b4a6d0738c",
"traceAddress" => [],
"type" => "call",
"value" => "0x0"
}
],
trace_address: [0, 0, 0, 0]
}
which looks ok for me (it has both stack and calls props). Can I extract something more from the args: nil, arity: 1, clauses: nil, function: :finalize, kind: nil bit for debugging? If you point the cause, it will be helpful too, but I also want to understand how to debug this. Can extra props (depth, trace_address) be the source of the problem? Or should I look for the error cause inside the finalize function body? (as far as I understand, no)
PS A note for others seeking help with debugging: the bit before the error text
application=indexer fetcher=internal_transaction count=10 error_count=10
may contain useful info: if you look for Logger.metadata, you may find something like
Logger.metadata(fetcher: :internal_transaction)
which can give a hint about the source of the error.
By default, if you don't use try / rescue, a FunctionClauseError will be logged with all the information (values, patterns, stacktrace) needed to debug it:
> String.downcase(1)
** (FunctionClauseError) no function clause matching in String.downcase/2
The following arguments were given to String.downcase/2:
# 1
1
# 2
:default
...
But if you rescure the error and return it as a tuple, you will lose most of this information:
try do
String.downcase(1)
rescue
err -> {:error, err}
end
The result will only contain a very barebones struct:
{:error,
%FunctionClauseError{
args: nil,
arity: 2,
clauses: nil,
function: :downcase,
kind: nil,
module: String
}}
You can log directly in the rescue clause, where __STACKTRACE__ is available, using the approach explained in this official guide:
try do
String.downcase(1)
rescue
err ->
# format the exception in a readable way
Exception.format(:error, err, __STACKTRACE__)
|> Logger.error()
# return value
:error
end
That being said, this is not so common in Elixir to rely on try/rescue as explained in the guide above. Most of the time you just let unexpected things fail (the famous "Let it crash" philosophy), and use :error tuples for cases you actually expect and want to handle.
Sidenote: the answer provided by #sabiwara has a ton of helpful info and should be marked as correct.
Here is the cause of the error you got:
defp finalize(%{stack: [top], calls: [calls]})
This defines a function, accepting a map with keys, including but not limited to :stack and :calls, and their values being lists containing exactly one element.
You are passing longer lists there, hence the error. This is what you likely wanted to do:
defp finalize(%{stack: top, calls: calls})
when is_list(top) and is_list(calls)
I quite like this indentation style:
hash =
[ "bla" => :bla
, "bli" => :bli
, "blo" => :blo
]
but Ruby does not like that so much.
This is OK:
{ 'a' => 1, 'b' => 2 }
#=> {"a"=>1, "b"=>2}
But this:
{ 'a' => 1
, 'b' => 2 }
yields
-:2: syntax error, unexpected ',', expecting '}'
, 'b' => 2 }
^
-:2: syntax error, unexpected =>, expecting end-of-input
, 'b' => 2 }
^
Is intended by the author or is it a weird bug?
You could add a \ to each line:
hash =
[ "bla" => :bla \
, "bli" => :bli \
, "blo" => :blo \
]
#=> [{"bla"=>:bla, "bli"=>:bli, "blo"=>:blo}]
Just move the commas before the line breaks, and ruby is happy again:
hash =
[ "bla" => :bla,
"bli" => :bli,
"blo" => :blo
]
#=> [{"bla"=>:bla, "bli"=>:bli, "blo"=>:blo}]
1. { :a => 10 } #=> no error
2. { a: 10 } #=> no error
3. { :"str" => 10 } #=> no error
4. { "str": 10 } #=> syntax error, unexpected ':', expecting =>
Isn't 4. same as 2? Why 2 is working and 4 throws syntax error?
My understanding is that {"key": value} is not a valid syntax as it is not clear whether it means {:"key" => value} or {"key" => value}
There is a discussion on this here. Quote from Matz in the discussion
| Iff {'key': 'value'} means {:key => 'value'} I have no objection.
| Won't that be misleading? I think the OP wants {'key': 'value'} to mean {'key' => 'value}
But considering the fact that {key: "value"}
is a shorthand for {:key => "value"}, {"key": "value"} should be a
shorthand for {:"key" => "value"}. Besides that, since it reminds me
JSON so much, making a: and "a": different could cause more confusion
than the above misleading.
matz.
Hash: Hashes allow an alternate syntax form when your keys are always symbols.
options = { :font_size => 10, :font_family => "Arial" }
You could write it as:
options = { font_size: 10, font_family: "Arial" }
In your first 3 cases all are symbols in key position,but the fourth is a string instance,not the symbol instance as key.That's the reason 4th case is invalid Ruby syntax.
{ :a => 10 }.keys[0].class # => Symbol
{ a: 10 }.keys[0].class # => Symbol
{ :"str" => 10 }.keys[0].class # => Symbol
No. (1) is standard symbol, (2) is shorthand 1.9 syntax for symbol-key hashes, (3) is shorthand for "str".to_sym, (4) does not exist and you should use the hashrocket.
I need to find a certain hash element where one of the keys is equal to a certain value. I've tried many ways and can't seem to figure it out with jsonpath gem.
Need to get tire tag where grip == 'bad'
require "jsonpath"
hash = {
:id => 1,
:cars => [
{:id => 1, :tire => {:grip => "good", :color => "black"}},
{:id => 2, :tire => {:grip => "bad", :color => "red"}},
{:id => 3, :tire => {:grip => "good", :color => "green"}}
]
}
puts JsonPath.on(hash, "$..tire[?(#['grip'] == 'bad')]").inspect
No results.
The [?()] filter only works for arrays (or at least for either arrays or hashes, not both at the same time). In order for it to work, I had to enclose the :tire hash in an array.
Original:
:tire => {:grip => "good", :color => "black"}
New:
:tire => [{:grip => "good", :color => "black"}]
That's a "fix" that works for me. It would be better if someone fixed the jsonpath gem to make it work for both arrays and hashes (of the same type and at the same time).
In JsonPath.on first argument must be json, not hash.
I can't do it over ruby, it is not fully correct solution. But may be this help you.
require 'jsonpath'
require 'json'
hash = {
:id => 1,
:cars => [
{:id => 1, :tire => {:grip => "good", :color => "black"}},
{:id => 2, :tire => {:grip => "bad", :color => "red"}},
{:id => 3, :tire => {:grip => "good", :color => "green"}}
]
}
json = hash.to_json
obj = JsonPath.new( "$..tire")[json]
result = obj.inject(Array.new){|res, x| res << x if x["grip"]=='bad'; res }
p result # [{"grip"=>"bad", "color"=>"red"}]
Say for example I've got a collection like this:
[{"name" => "Ganesh", "magic_number" => 7}, {"name" => "Comrade", "magic_number" => 2}...]
How can I change the value of ALL the magic_numbers in the collection to be the same value (e.g. 8)?
I'm sure it's using something like map or collect but I can't seem to do it at the moment and return me the whole collection with the changes, just one or the other...
Just use .each:
a = [{"name" => "Ganesh", "magic_number" => 7}, {"name" => "Comrade", "magic_number" => 2} ]
a.each { |x| x['magic_number'] = 8 }
# a is now [{"magic_number"=>8, "name"=>"Ganesh"}, {"magic_number"=>8, "name"=>"Comrade"}]
The argument to the block is a reference to the original elements so you can change them as desired. Note that this changes a in-place which I think is what you're after.
This works:
x = [{"name" => "Ganesh", "magic_number" => 7}, {"name" => "Comrade", "magic_number" => 2}]
x.map{|i| i["magic_number"] = 0; i }
=> [{"magic_number"=>0, "name"=>"Ganesh"}, {"magic_number"=>0, "name"=>"Comrade"}]