Hi i have collection which contains around 200 documents looks like
e.g
"_id": 0,
"name": "Demarcus Audette",
"scores": [
{
"type": "exam",
"score": 30.61740640636871
},
{
"type": "quiz",
"score": 14.23233821353732
},
{
"type": "homework",
"score": 31.41421298576332
},
{
"type": "homework",
"score": 30.09304792394713
}
]
now i wrote code like
DBCursor cursor = collection.find().sort(new BasicDBObject("scores.score":1));
while( cursor.hasNext() )
{
DBobject obj=cursor.next();
BasicDBList list=(BasicDBList) Bobj.get("scores");
// Now here i am getting list of documents which consists of an scores array and i need to remove 3rd elements of it and save collection.... but how to do?
if i use for loop like
for(int i=0;i<list.size();i++)
{
list.remove(2);------ it gives an error here
collection.save(obj);
}
}
Are you sure that you can sort it as 'scores.score'. As 'scores' is a list of objects you cant reference objects inside list using dot notation. Error should be on that line.
Edit:
DBCursor cursor = collection.find();
while ( cursor.hasNext()) {
DBObject dbObject = cursor.next();
BasicDBList dbList = (BasicDBList) (dbObject.get("score"));
//then use java Collections.sort() to sort the List (refer to two methods at the bottom)
dbList.remove(3);
collection.save(dbObject);
}
Use one of the following to sort the DBList
1 . Using Lambda expression
Collections.sort(dbList, (o1, o2) -> Double.compare(((BasicDBObject) o1).getDouble("score"), ((BasicDBObject) o2).getDouble("score")));
or java 7 <
Collections.sort(dbList, new Comparator<Object>() {
public int compare(Object o, Object o2) {
if (((BasicDBObject) o).getDouble("score") >= ((BasicDBObject) o2).getDouble("score")) {
return 1;
}
return -1;
}
});
See this link https://gist.github.com/machadolucas/5188447/raw/e3f5c44c2be756c6087809df63f7ea8f4682ace9/Mongo3_1.java
I`m has been
[some symbol for post this message)))]
Related
I have an old version of Elasticsearch (5.6.16) on a production environment that I can't upgrade.
I'm trying to use a loop in a painless script_score script but I always face a runtime error.
All my documents can have one or several "badges", here the mapping:
"myDocument":{
"properties":{
"badges":{
"type":"nested",
"properties":{
"name":{
"type":"keyword"
}
}
},
}
},
My goal is to do a custom script that will provide a better score for documents with a specific type of badge
So I made this script
for (item in doc['badges']) {
if (item['name'] == "myCustomBadge") {
return _score * 10000;
}
}
return _score;
But unfortunately, I'm getting errors while I try to use it
{
"query":{
"function_score":{
"query":{
"match_all": {}
},
"functions":[
{
"script_score":{
"script":{
"inline":"for (item in doc['badges']) { if (item['name'] == \"myCustomBadge\") { return _score * 10000; }}return _score;",
"lang":"painless"
}
}
},
]
}
}
}
"error":{
"root_cause":[
{
"type":"script_exception",
"reason":"runtime error",
"script_stack":[
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:77)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:36)",
"for (item in doc['badges']) { ",
" ^---- HERE"
],
"script":"for (item in doc['badges']) { if (item['name'] == \"myCustomBadge\") { return _score * 10000; }}return _score;",
"lang":"painless"
}
],
"type":"search_phase_execution_exception",
"reason":"all shards failed"
}
I tried to change the for with an other variant, but same error.
for(int i = 0; i < doc['badges']; i++) {
if (doc['badges'][i]['name'] == "uaWorker") {
return _score * 10000;
}
}
return _score;
Could you help me find what I did wrong?
Thanks you all
The problem is not the loop but the fact that badges is nested and you're trying to access it from doc values. In this case, you need to access the array of badges from the _source document directly, like this:
for (item in params._source['badges'])
I have data
{
"id": 1000,
"price": "99,01USA",
},
{
"id": 1001,
"price": "100USA",
},
{
"id": 1002,
"price": "780USA",
},
{
"id": 1003,
"price": "20USA",
},
How I sort order by price (ASC , DESC)
You can alter it a little to parse price to integer and then sort it
You can create a dynamic sort function that sorts objects by their value that you pass:
function dynamicSort(property) {
var sortOrder = 1;
if(property[0] === "-") {
sortOrder = -1;
property = property.substr(1);
}
return function (a,b) {
/* next line works with strings and numbers,
* and you may want to customize it to your needs
*/
var result = (a[property] < b[property]) ? -1 : (a[property] > b[property]) ? 1 : 0;
return result * sortOrder;
}
}
So you can have an array of objects like this:
var People = [
{Name: "Name", Surname: "Surname"},
{Name:"AAA", Surname:"ZZZ"},
{Name: "Name", Surname: "AAA"}
];
...and it will work when you do:
People.sort(dynamicSort("Name"));
People.sort(dynamicSort("Surname"));
People.sort(dynamicSort("-Surname"));
Actually this already answers the question. Below part is written because many people contacted me, complaining that it doesn't work with multiple parameters.
Multiple Parameters
You can use the function below to generate sort functions with multiple sort parameters.
function dynamicSortMultiple() {
/*
* save the arguments object as it will be overwritten
* note that arguments object is an array-like object
* consisting of the names of the properties to sort by
*/
var props = arguments;
return function (obj1, obj2) {
var i = 0, result = 0, numberOfProperties = props.length;
/* try getting a different result from 0 (equal)
* as long as we have extra properties to compare
*/
while(result === 0 && i < numberOfProperties) {
result = dynamicSort(props[i])(obj1, obj2);
i++;
}
return result;
}
}
Which would enable you to do something like this:
People.sort(dynamicSortMultiple("Name", "-Surname"));
Subclassing Array
For the lucky among us who can use ES6, which allows extending the native objects:
class MyArray extends Array {
sortBy(...args) {
return this.sort(dynamicSortMultiple(...args));
}
}
That would enable this:
MyArray.from(People).sortBy("Name", "-Surname");
My requirement is to search documents from elasticsearch based on fuzzy matching and then 'rescore' the documents by comparing the value of the document and an input string for e.g. If the query is returning 3 documents (doc:1,2,3), then for comparing the constant value 'Star Wars', the comparison should be as:
doc:1, MovieName:"Star Wars" (compare ('Star Wars','Star Wars'))
doc:2, MovieName:"Starr Warz" (compare ('Star Wars','Starr Warz'))
doc:3, MovieName:"The Star Wars" (compare ('Star Wars','The Star Wars'))
I found the following elasticsearch rescore plugin example and implemented it to achieve the above.
https://github.com/elastic/elasticsearch/blob/6.2/plugins/examples/rescore/src/main/java/org/elasticsearch/example/rescore/ExampleRescoreBuilder.java
I am able to pass and access the input 'Star Wars' in the plugin, however I am facing trouble getting the value of the MovieName field of the documents returned in the results (topdocs).
My Query:
GET movie-idx/_search?
{
"query": {
"bool": {
"must": [
{
"query_string": {
"fields": [
"MovieName"
],
"query": "Star Wars",
"minimum_should_match": "61%",
"fuzziness": 1,
"_name": "fuzzy"
}
}
]
}
},
"rescore": {
"calculateMovieScore": {
"MovieName": "Star Wars"
}
}
}
And my rescorer class looks like:
private static class DocsRescorer implements Rescorer {
private static final DocsRescorer INSTANCE = new DocsRescorer();
#Override
public TopDocs rescore(TopDocs topDocs, IndexSearcher searcher, RescoreContext rescoreContext) throws IOException {
DocRescoreContext context = (DocRescoreContext) rescoreContext;
int end = Math.min(topDocs.scoreDocs.length, rescoreContext.getWindowSize());
MovieScorer MovieScorer = new MovieScorerBuilder()
.withInputName(context.MovieName)
.build();
for (int i = 0; i < end; i++) {
String name = <get MovieName values from actual document returned by topdocs>
float score = MovieScorer.calculateScore(name);
topDocs.scoreDocs[i].score = score;
}
List<ScoreDoc> scoreDocList = Stream.of(topDocs.scoreDocs).filter((a) -> a.score >= context.threshold).sorted(
(a, b) -> {
if (a.score > b.score) {
return -1;
}
if (a.score < b.score) {
return 1;
}
// Safe because doc ids >= 0
return a.doc - b.doc;
}
).collect(Collectors.toList());
ScoreDoc[] scoreDocs = scoreDocList.toArray(new ScoreDoc[scoreDocList.size()]);
topDocs.scoreDocs = scoreDocs;
return topDocs;
}
#Override
public Explanation explain(int topLevelDocId, IndexSearcher searcher, RescoreContext rescoreContext,
Explanation sourceExplanation) throws IOException {
DocRescoreContext context = (DocRescoreContext) rescoreContext;
// Note that this is inaccurate because it ignores factor field
return Explanation.match(context.factor, "test", singletonList(sourceExplanation));
}
#Override
public void extractTerms(IndexSearcher searcher, RescoreContext rescoreContext, Set<Term> termsSet) {
// Since we don't use queries there are no terms to extract.
}
}
My understanding is that the plugin code will execute once, it will get topdocs as results from the initial query (the fuzzy search in this case) and for (int i = 0; i < end; i++) will loop through each document returned in the result. The place where I need help is:
String name = <get MovieName value from actual document returned by topdocs>
I know it's been over 2 years, but i've ran into the same problem and found a solution so i'm posting it here. this was done for a Rescorer plugin in ES 7.8.0. the base example i used is the Grouping plugin Link.
It's a bunch of code that i don't fully understand, but the main principle is that you need an IFD (IndexFieldData<?>) instance of the field you want to get. in my example, i just needed _id of the hits. it looks like this:
prepare the IFD in advance and pass it to the RescoreContext: add a member to the class extending RescoreContext to keep this IFD on the context, lets call it "idField" (later used in section 3).
#Override
public RescoreContext innerBuildContext(int windowSize, QueryShardContext queryShardContext) throws IOException {
return new MyRescoreContext(windowSize, queryShardContext.getForField(queryShardContext.fieldMapper("_id")));
}
next, in the Rescorer itself: (method rescore(...) )
2.1) first sort by scoreDoc.doc
ScoreDoc[] hits = topDocs.scoreDocs;
Arrays.sort(hits, Comparator.comparingInt((d) -> d.doc));
2.2) PERFORM BLACK MAGIC (code i don't understand)
List<LeafReaderContext> readerContexts = searcher.getIndexReader().leaves();
int currentReaderIx = -1;
int currentReaderEndDoc = 0;
LeafReaderContext currentReaderContext = null;
for (int i = 0; i < end; i++) {
ScoreDoc hit = hits[i];
// find segment that contains current document
while (hit.doc >= currentReaderEndDoc) {
currentReaderIx++;
currentReaderContext = readerContexts.get(currentReaderIx);
currentReaderEndDoc = currentReaderContext.docBase + currentReaderContext.reader().maxDoc();
}
int docId = hit.doc - currentReaderContext.docBase;
// code from section 3 goes here //
}
And now, with this magical "docId" in hand, you can fetch from the IFD inside the For loop:
SortedBinaryDocValues values = rescoreContext.idField.load(currentReaderContext).getBytesValues();
values.advanceExact(docId);
String id = values.nextValue().utf8ToString();
in your case, instead of the _id field, get the IFD for the field you want, and create a Hashmap from docId -> string value inside the For loop.
then use this map in the same For loop where you apply the score.
hope this helps everybody! this technique is not documented at all and there are no explanations anywhere!
I have a JSON that have a JSONArray as a value in one of the json inside it. here is the example of it.
[
{
"id": 1,
"symptoms" : [{\"key\":\"sample1\",\"value\":5},{\"key\":\"sample2\",\"value\":5}]
},
{
"id": 2,
"symptoms" : [{\"key\":\"sample3\",\"value\":1}]
},
{ "id": 3,
"symptoms" : []
},
{
"id": 4,
"symptoms": [{\"key\":\"sample4\",\"value\":1}]
}
]
So what I am doing is that I am parsing the inner JSON and place it in a String Array. But whenever I look up to symptoms it skips the empty JSONArray. So whenever i print the String Array it goes like this (with the given sample on top) ["sample1", "sample2", "sample3", "sample4"]. But i want to do is to append an "" to the String Array whenever the JSONArray is empty so it should be like this ["sample1", "sample2", "sample3", "", "sample4"]. Anyone can help me with this? Here is my code
var arrayHolder: [String] = []
var idHolder: [Int] = []
for item in swiftyJSON.arrayValue {
idHolder.append(item["id"].intValue)
//for the inner JSON
let innerJSON = JSON(data: item["symptoms"].dataUsingEncoding(NSUTF8StringEncoding)!)
for symptoms in innerJSON.arrayValue {
arrayHolder.append(symptoms["key"].stringValue)
}
}
print(idHolder) // [1,2,3,4]
print(arrayHolder) // ["sample1","sample2","sample3","sample4"]
Just check if innerJSON is empty:
for item in swiftyJSON.arrayValue {
idHolder.append(item["id"].intValue)
//for the inner JSON
let innerJSON = item["symptoms"].arrayValue // non need to create a new JSON object
if innerJSON.isEmpty {
arrayHolder.append("")
} else {
for symptoms in innerJSON {
arrayHolder.append(symptoms["key"].stringValue)
}
}
}
I get the above exception at this line of code of jqGrid.src.js
fmt = $.jgrid.formatter.integer || {};
I can't for the life of me tell you what that means or why I can't get the damn thing to work. I suspect it has to do with how I am building my json object.
for(int j = rowstart; (j <= rowend && variable.Template.Count > j); j++){
PatientACOModel patMod = variable.Template[j];
var rowData = new{
id = patMod.EncounterId,
cell = new {
MRN = patMod.MRN,
Hospital_Fin = patMod.HospitalFinNumber,
First_Name = patMod.FirstName,
Last_Name = patMod.LastName,
Date_of_birth = patMod.DateOfBirth
}
};
al.Add(rowData);
}
var griddata = new {
total = variable.Template.Count % rows > 0 ? (variable.Template.Count / rows) + 1 : (variable.Template.Count / rows),
page = page,
records = al.Count,
rows = al.ToArray()
};
I have no clue what I am doing wrong? It looks like everything is set up correctly?
UPDATE
This would be the json data that is being sent back to my jqGRid... These are just a small selection of rows, and columns. Am I missing something here?
{"total": 2,
"page": 1,
"records": 15,
"rows": [{
"id": 2148,
"cell": {
"MRN": "840134833",
"Hospital_Fin": "987141516",
"First_Name": "YELLOW",
"Last_Name": "CRAYON",
"Date_of_birth": "\/Date(1253160000000)\/"
}
},
{
"id": 1898,
"cell": {
"MRN": "785528039",
"Hospital_Fin": "6669511596226",
"First_Name": "RAYFIELD",
"Last_Name": "BOYD",
"Date_of_birth": "\/Date(-720298800000)\/"
}
}]}
To eliminate this error, you need to make sure that you are including the localization file
<script src="~/Scripts/i18n/grid.locale-en.js"></script>
before the jqGrid file. That eliminated the problem for me.