Sorting a reference to an array of rows where each row is stored as a hash - sorting

I'm trying to sort the following data structure in Perl, by location_id.
my $employees = $dbh->selectall_arrayref(qq[
SELECT name, type, code, emp_cat_id,
percentage, location_id
FROM table_1
],{ Slice => {} });
for my $row (#$employees) {
push #{
$args->{employees}{ $row->{emp_cat_id} }
}, $row;
}
Example:
123 => [
{
percentage => 0.25,
code => "XYZ",
name => "John Doe",
type => "pt",
location_id => 001,
emp_cat_id => 123
}
],
555 => [
{
percentage => 0.50,
code => "ZZZ"
name => "Chris Cringle",
type => "ft",
location_id => 007,
emp_cat_id => 555
},
{
percentage => 0.25,
code => "XXX"
name => "Tom Thompson",
type => "pt",
location_id => 002,
emp_cat_id => 555
}
]
For every emp_cat_id, I need the structure to have the location_ids in asc order.
I've tried the following, but I get "useless use of sort in void context at line #" or "useless use of sort in scalar context at line #" errors.
$args->{employees} = sort {
$a->{location_id} <=> $b->{location_id}
} $args->{employees};
Any help understanding the sort is appreciated!

The problem is that you are sorting the array(ref) at emp_cat_id of 555, then of 123, and so need to dereference for sorting those arrayrefs. So
foreach my $id (keys $args->{employees}) {
#{ $args->{employees}{$id} } = sort {
$a->{location_id} <=> $b->{location_id}
}
#{ $args->{employees}{$id} }
}
(tested with the structure shown in the question, omitted here)†
Doing anything like this loses 007 into 7. This is of course possible to fix, let me know if it matters.
If you really have only the key employees then consider extracting the $args->{employees} hashref and working with that. It'll be way easier
use Storable qw(dclone);
my $employees = dclone $args->{employees}; # need deep copy
† Oh well, here's the whole thing
use warnings;
use strict;
use feature 'say';
use Data::Dump qw(dd);
my $args = {
employees => {
123 => [
{
percentage => 0.25,
code => "XYZ",
name => "John Doe",
type => "pt",
location_id => 001,
emp_cat_id => 123
}
],
555 => [
{
percentage => 0.50,
code => "ZZZ",
name => "Chris Cringle",
type => "ft",
location_id => 007,
emp_cat_id => 555
},
{
percentage => 0.25,
code => "XXX",
name => "Tom Thompson",
type => "pt",
location_id => 002,
emp_cat_id => 555
}
]
}
};
foreach my $id (keys $args->{employees}) {
#{ $args->{employees}{$id} } = sort {
$a->{location_id} <=> $b->{location_id}
}
#{ $args->{employees}{$id} }
}
dd $args;

So, you have a hashref where each element is an arrayref of hashrefs that should be sorted based on a key of that inside hashref?
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
my $hashref = {
123 => [
{
percentage => 0.25,
code => "XYZ",
name => "John Doe",
type => "pt",
location_id => 001,
emp_cat_id => 123
}
],
555 => [
{
percentage => 0.50,
code => "ZZZ",
name => "Chris Cringle",
type => "ft",
location_id => 007,
emp_cat_id => 555
},
{
percentage => 0.25,
code => "XXX",
name => "Tom Thompson",
type => "pt",
location_id => 002,
emp_cat_id => 555
}
]
};
foreach my $arrayref (values %$hashref) {
#$arrayref = sort { $a->{location_id} <=> $b->{location_id} } #$arrayref;
}
print Dumper($hashref);
The important part you're missing is in dereferencing the arrayrefs. #$arrayref instead of just $arrayref.

Related

Recursive not working in kv filter in logstash

I want to know about the use of recursive function in kv filter. I am using a csv file. I uploaded the file to ES using logstash. After reading the guide from this link https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html#plugins-filters-kv-recursive
I came to know that it duplicates the key/values pair and store it in a separate key. But i can't get additional info or examples about the filter. I added a recursive line in logstash config file. No changes.
Is it duplicates the fields with values(key-value pairs) or else what this function doing???
Here's my sample csv file data passing through logstash:
"host" => "smackcoders",
"Driveline" => "Four-wheel drive",
"Make" => "Jeep",
"Width" => "79",
"Torque" => "260",
"Year" => "2012",
"Horsepower" => "285",
"City_mpg" => "17",
"Height" => "34",
"Classification" => "Manual,Transmission",
"Model_Year" => "2012 Jeep Wrangler",
"Number_of_Forward_Gears" => "6",
"Length" => "41",
"Highway_mpg" => "21",
"#version" => "1",
"message" => "17,\"Manual,Transmission\",Four-wheel drive,Jeep 3.6L 6 Cylinder 280 hp 260 lb-ft,Gasoline,34,21,285,False,2012 Jeep Wrangler Arctic,41,Jeep,2012 Jeep Wrangler,6,260,6 Speed Manual,79,2012",
"Fuel_Type" => "Gasoline",
"Engine_Type" => "Jeep 3.6L 6 Cylinder 280 hp 260 lb-ft",
"path" => "/home/paulsteven/log_cars/cars.csv",
"Hybrid" => "False",
"ID" => "2012 Jeep Wrangler Arctic",
"#timestamp" => 2019-04-20T07:58:26.552Z,
"Transmission" => "6 Speed Manual"
}
Here's the config file:
input {
file {
path => "/home/paulsteven/log_cars/cars.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["City_mpg","Classification","Driveline","Engine_Type","Fuel_Type","Height","Highway_mpg","Horsepower","Hybrid","ID","Length","Make","Model_Year","Number_of_Forward_Gears","Torque","Transmission","Width","Year"]
}
kv {
recursive => "true"
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "kvfilter1"
document_type => "details"
}
stdout{}
}
Found some examples for recursive in kv filter:
input { generator { count => 1 message => 'foo=1,bar="foor=10,barr=11"' } }
filter {
kv { field_split => "," value_split => "=" recursive => false }
}
will produce
"foo" => "1",
"bar" => "foor=10,barr=11",
whereas
input { generator { count => 1 message => 'foo=1,bar="foor=10,barr=11"' } }
filter {
kv { field_split => "," value_split => "=" recursive => true }
}
will produce
"foo" => "1",
"bar" => {
"foor" => "10",
"barr" => "11"
},

Display contents of a hash if value exists

I have a hash:
req = {
"count" => 50100,
"results" => [
{"listing_id" => 615929315, "state" => "active", "user_id" => 140604756, "category_id" => 69150367},
{"listing_id" => 615929311, "state" => "active", "user_id" => 152528025, "category_id" => 69150367}
]
}
I want to find and display the entire internal hash if a particular user_id exists. I can find it:
req["results"][0].select{|key, value| value == 152528025}
# => {"user_id" => 152528025}
How do I then display this entire (nested) hash?
{"listing_id" => 615929311, "state" => "active", "user_id" => 152528025, "category_id" => 69150367}
req["results"].select{|x| x["user_id"] == 152528025}

How can i update the ids field with this rethinkdb document structure?

Having trouble trying to update the ids field in the document structure:
[
[0] {
"rank" => nil,
"profile_id" => 3,
"daily_providers" => [
[0] {
"relationships" => [
[0] {
"relationship_type" => "friend",
"count" => 0
},
[1] {
"relationship_type" => "acquaintance",
"ids" => [],
"count" => 0
}
],
"countries" => [
[0] {
"country_name" => "United States",
"count" => 0
},
[1] {
"country_name" => "Great Britain",
"count" => 0
}
],
"provider_name" => "foo",
"date" => 20130912
},
[1] {
"provider_name" => "bar"
}
]
}
]
In JavaScript, you can do
r.db('test').table('test').get(3).update(function(doc) {
return {daily_providers: doc("daily_providers").changeAt(
0,
doc("daily_providers").nth(0).merge({
relationships: doc("daily_providers").nth(0)("relationships").changeAt(
1,
doc("daily_providers").nth(0)("relationships").nth(1).merge({
ids: [1]
})
)
})
)}
})
Which becomes in Ruby
r.db('test').table('test').get(3).update{ |doc|
{"daily_providers" => doc["daily_providers"].changeAt(
0,
doc["daily_providers"][0].merge({
"relationships" => doc["daily_providers"][0]["relationships"].changeAt(
1,
doc["daily_providers"][0]["relationships"][1].merge({
ids => [1]
})
)
})
)}
}
You should probably have another table for the daily providers and do joins.
That would make things way more simpler.

ruby one-liner from two hashes

a = {"rows" => [{"id" => "231-z", "name" => 'jon', "age"=> 27, "state" => 'AL'},
{"id" => "4121-x", "name" => 'ton', "age"=> 37, "state" => 'VA'}
]
}
b = {"rows" => [{"key" => ["xyz","4121-x"], "value" =>{"sum" => 12312, "realage" => 29}},
{"key" => ["xyz","231-z"], "value" =>{"sum" => 1212, "realage" => 33}}
]
}
In hash a, age is incorrect
In hash b, realage is correct. Also in hash b id is the second value in the first array that maps to id of hash a . Those are 4121-x, 231-z correspond to hash a
I want to correct the age in hash a and swap it with the realage of hash b
I can do it in multiple steps, but is it possible to do it in one liner or very short? So finally correct hash a should look like
a = {"rows" => [{"id" => "231-z", "name" => 'jon', "age"=> 33, "state" => 'AL'},
{"id" => "4121-x", "name" => 'ton', "age"=> 29, "state" => 'VA'}
]
}
does this look reasonable?
a['rows'].each_with_index do |ah, i|
(bh = b['rows'].select {|h| h['key'].last == ah['id'] }.first) &&
a['rows'][i] = ah.update('age' => bh['value']['realage'])
end
p a
{
"rows" => [
[0] {
"id" => "231-z",
"name" => "jon",
"age" => 33,
"state" => "AL"
},
[1] {
"id" => "4121-x",
"name" => "ton",
"age" => 29,
"state" => "VA"
}
]
}
Please note it will update a only if corresponding id found in b.
Also, the rows order does not matter, nor matter the rows number, it is only important b to have a row with same id as processed row in a
Here is a Working Demo

Ruby nested hash syntax & structure

I'm building a tree of html elements, class names and their counts.
How would I structure this code with the proper syntax?
$html = {
:p => [
{ 'quote' => 10 },
{ 'important' => 4 }
],
:h2 => [
{ 'title' => 33 },
{ 'subtitle' => 15 }
]
}
I'm confused by the nested hash syntax. Thanks for the help setting me straight.
A simple way to structure a HTML tree could be:
html = [
{ _tag: :p, quote: 10, important: 4 },
{ _tag: :h2, title: 33, subtitle: 15 },
]
Where html[0][:_tag] is the tag name, and other attributes are accessible through html[0][attr]. The root element is an array since multiple elements of the same type (multiple paragraphs) could exist in the same namespace and a hash would only store the last added one.
A more advanced example which would allow nested contents:
tree = { _tag: :html, _contents: [
{ _tag: :head, _contents: [
{ _tag: :title, _contents: "The page title" },
]},
{ _tag: :body, id: 'body-id', _contents: [
{ _tag: :a, href: 'http://google.com', id: 'google-link', _contents: "A link" },
]},
]}
After defining the HTML element you don't assign another hash, but a list and from your question title I guess you want to nest another hash directly. Thus you do not start with a square bracket, but with another curly brace:
$html = {
:p => { 'quote' => 10, 'important' => 4 },
:h2 => { 'title' => 33, 'subtitle' => 15 }
}
#Example
puts $html[:p]['quote']
Which will print:
10
Take a look at the constructor documentation of Hash, there are different ways to initialize hashes, maybe you find a more intuitive one.

Resources