I have an Elasticsearch (8.1) index on which I run a simple match or multi_match query (I tried both, both show the same behavior, even the simplest ones).
In the result it is always the case that max_score is higher than the first hit's _score.
If I add a terms aggregation (on a keyword field) with a top_hits sub-aggregation (with sorting on _score) then the first hit from the first bucket actually has _score == max_score (but it is obviously also a different hit compared to the "main" hits). So, the top_hits aggregation actually does what I want ("fetch all matching documents and sort by _score"). The "main" hits seem to miss some results, however.
How can I make sure that the "main" hits do not "drop" documents? What is the internal mechanics behind this?
I added my PHP array that gets JSON encoded and produces the Elasticsearch query:
[
'size' => 10,
'query' => [
// the result of this does not have all documents
// that appear in the aggregation
// and the highest ranked doc has lower score than max_score
'bool' => [
'must' => [
[
'match' => [
'my_text_field' => [
'query' => 'searchword'
]
]
],
['term' => ['my_other_field' => ['value' => 3]]],
// plus some more other simple term conditions
// on other simple integral fields, but no scripts ore similar
// simple "WHERE a = 5" conditions
]
]
],
// this aggregation has other/more hits than the directly retrieved docs, matching the max_score
// If I remove the aggregation nothing changes for the actual result
'aggs' => [
'my_agg' => [
'terms' => ['field' => 'my_agg_field', 'order' => ['score' => 'desc']],
'aggs' => [
'score' => ['max' => ['script' => '_score']],
'filteredHits' => [
'top_hits' => [
'size' => 10
]
]
]
]
]
]
we got an ES/Kibana [7.17] running, everything works fine so far but we want to autotranslate a field based on a static table how is this possible? I remember it was possible over custom formats in older Kibana versions but I cannot find how to do it in this one.
e.g.
1 => HR Department
2 => IT Department
3 => Production
etc.
Data is:
Max Muster 3
Data should be
Max Muster 3 Production
P.S. I tried adding a runtime field to the template but it always complains that the syntax is wrong
filter {
translate {
source => "[dep]
target => "[department]"
dictionary => {
"1" => "HR Dep"
"2" => "IT Dep"
"3" => "Production"
}
}
}
}
I have the following object returned from an InfluxDB query, and I want to be able to check if any of the derivatives are equal or greater than say 100, if so then do stuff.
I've been trying to use select to check that field, but I really don't actually understand how to work with a data structure like this. How would I go about iterating through every derivative value in my returned object?
I'm not really seeing an example that's similar to my case in the enumerable documentation.
https://ruby-doc.org/core-2.4.0/Enumerable.html
[{
"name" => "powerdns_value",
"tags" => nil,
"values" => [
{ "time" => "2017-03-21T14:20:00Z", "derivative" => 1},
{ "time" => "2017-03-21T14:30:00Z", "derivative" => 900},
{ "time" => "2017-03-21T14:40:00Z", "derivative" => 0},
{ "time" => "2017-03-21T15:20:00Z", "derivative" => 0}
]
}]
If you just want to know if one of the hashes in your array meet the condition
arr.first['values'].any? { |hash| hash['derivative'] >= 100 }
I'm working on a script that will return value in a specific field and exclude other fields i tried these codes:
name = 'bierc'
puts collection.find({"name"=> name},{:fields => { "_id" => 0}}).to_a
and
name = 'bierc'
collection.find("name" => name,"_id" => 0).each{|row| puts row.inspect}
These two returns
{"_id"=>BSON::ObjectId('55f0d965fcd4fe1c659cf472'), "name"=>"bierc", "song"=>"testsong"}
I want to select name only and exclude the song and especially the _id then will work on to pass the value of name field to a variable.
The option is not fields but projection. You do need _id => 0 but then 1 for any field or fields you do want to select should exclude the others.
collection.find("name" => name, projection: => { "_id" => 0, "name" => 1}})
Have an array of hashes,
my #arr = get_from_somewhere();
the #arr contents (for example) is:
#arr = (
{ id => "id2", requires => 'someid', text => "another text2" },
{ id => "xid4", requires => 'id2', text => "text44" },
{ id => "someid", requires => undef, text => "some text" },
{ id => "id2", requires => 'someid', text => "another text2" },
{ id => "aid", requires => undef, text => "alone text" },
{ id => "id2", requires => 'someid', text => "another text2" },
{ id => "xid3", requires => 'id2', text => "text33" },
);
need something like:
my $texts = join("\n", get_ordered_texts(#arr) );
soo need write a sub what return the array of texts from the hashes, - in the dependent order, so from the above example need to get:
"some text", #someid the id2 depends on it - so need be before id2
"another text2", #id2 the xid3 and xid4 depends on it - and it is depends on someid
"text44", #xid4 the xid4 and xid3 can be in any order, because nothing depend on them
"text33", #xid3 but need be bellow id2
"alone text", #aid nothing depends on aid and hasn't any dependencies, so this line can be anywhere
as you can see, in the #arr can be some duplicated "lines", ("id2" in the above example), need output only once any id.
Not providing any code example yet, because havent any idea how to start. ;(
Exists some CPAN module what can be used to the solution?
Can anybody points me to the right direction?
Using Graph:
use Graph qw( );
my #recs = (
{ id => "id2", requires => 'someid', text => "another text2" },
{ id => "xid4", requires => 'id2', text => "text44" },
{ id => "someid", requires => undef, text => "some text" },
{ id => "id2", requires => 'someid', text => "another text2" },
{ id => "aid", requires => undef, text => "alone text" },
{ id => "id2", requires => 'someid', text => "another text2" },
{ id => "xid3", requires => 'id2', text => "text33" },
);
sub get_ordered_recs {
my %recs;
my $graph = Graph->new();
for my $rec (#_) {
my ($id, $requires) = #{$rec}{qw( id requires )};
$graph->add_vertex($id);
$graph->add_edge($requires, $id) if $requires;
$recs{$id} = $rec;
}
return map $recs{$_}, $graph->topological_sort();
}
my #texts = map $_->{text}, get_ordered_recs(#recs);
An interesting problem.
Here's my first round solution:
sub get_ordered_texts {
my %dep_found; # track the set of known dependencies
my #sorted_arr; # output
my $last_count = scalar #_; # infinite loop protection
while (#_ > 0) {
for my $value (#_) {
# next unless we are ready for this text
next if defined $value->{requires}
and not $dep_found{ $value->{requires} };
# Add to the sorted list
push #sorted_arr, $value->{text};
# Remember that we found it
$dep_found{ $value->{id} }++;
}
if (scalar #_ == $last_count) die "some requirements don't exist or there is a dependency loop";
$last_count = scalar #_;
}
return \#sorted_arr;
}
This is not terribly efficient and probably runs in O(n log n) time or something, but if you don't have a huge dataset, it's probably OK.
I would use a directed graph to represent the dependency tree and then walk the graph. I've done something very similiar using Graph.pm
Each of your hashes would be a graph vertex and the edge would represent the dependency.This has the added benefit of supporting more complex dependencies in the future as well as providing shortcut functions for working with the graph.
you didn't say what to do of the dependencies are "independent" of each other.
E.g. id1 requires id2; id3 requires id4; id3 requires id5. What should the order be? (other than 1 before 2 and 3 before both 4/5)
What you want is basically a BFS (Breadth First Search) of a tree (directed graph) of dependencies (or a forest depending on answers to #1 - the forest being a set of non-connected trees).
To do that:
Find all of the root nodes (ids that don't have a requirement themselves)
You can easily do that by making a hash of ALL the IDs using grep on your data structure
Put all those root modes into a starting array.
Then implement BFS. If you need help implementing basic BFS using an array and a loop in Perl, ask a separate question. There may be a CPAN module but the algorithm/code is rather trivial (at least once you wrote it once :)