Currently I am using elasticsearch helper scan api, but it is not able to fetch data.
command :
helpers.scan(
client=client,
query={"query":{"match_all":{}}},
scroll='10m',
index="debug",
doc_type = "tool",
_source=True
)
output :
......
generator object scan at 0x1556640
generator object scan at 0x1556640
generator object scan at 0x1556640
generator object scan at 0x1556640
generator object scan at 0x1556640
.......
when I am doing
curl -XGET"http://localhost:9200/debug/tool/_search?pretty=true&q=*:*"
(only 10 by default)
it is able to fetch the data.
After digging the elastic when I check the indices using this command :
http://127.0.0.1:9200/_cat/indices
I found this : No handler found for uri [/_cat/indices] and method [GET]
But when I am using http://localhost:9200/_aliases, I can see my indexing. Why indexes is not coming when I run _cat/indices command?
_cat/indices is available in ES versions after 1.0.x.
Related
I need examples for DeleteRequest with respect to ES 8.3.0 Java Api client.
I am looking for code reference where I want to delete one particular document by passing index name and the condition to delete the document.
I have found only Java High Level Rest Client(Deprecated in 7.15.0), and
Transport Client(Deprecated in 7.0.0).
You can use below code for delete specific document using id:
DeleteRequest request = DeleteRequest.of(d -> d.index("index_name").id("doc_id"));
DeleteResponse response = esClient.delete(request);
If you want to do DeleteByQuery then you can use below code (it will delete document where country is india):
DeleteByQueryRequest dbyquery = DeleteByQueryRequest
.of(fn -> fn.query(TermQuery.of(tq -> tq.field("country").value("india"))._toQuery()).index("index_name"));
DeleteByQueryResponse dqr = esClient.deleteByQuery(dbyquery);
There is no details document available for above. You can see open github issue for same here
I have an application which is logging level as integers. I am using filebeat to send the logs to ES. I have set level as string in the ES index, which is working for most of the applications. But when filebeat is receiving an integer, the indexation is failing of course with:
"type":"illegal_argument_exception","reason":"field [level] of type [java.lang.Integer] cannot be cast to [java.lang.String]"
In my document: "level":30
I added a step Script in my ingestion pipeline. But I can't manage to make it work: either I get a compilation error or the script is somehow failing and nothing at all get indexed.
Some very basic script I tried:
if (doc['level'].value == 30) {
doc['level'].value = 'info';
}
Any idea on how to handle this in ES pipelines?
Regards
The best way is to transform data before sending to ES.
You can usse processsor in filebeat to filter you data.
https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html
I am trying to run a query using com.parse:parse-android:1.13.0 version of the parse android SDK. I am creating a query on the local storage and using ParseQuery$whereMatchesQuery() method to match a column storing pointer to another class in my database. The code that I have is the following:
ParseQuery<PracticeSessionDetails> query = ParseQuery.getQuery(PracticeSessionDetails.class);
query.fromLocalDatastore();
query.ignoreACLs();
query.whereEqualTo("user", ParseUser.getCurrentUser());
ParseQuery courseQuery = new ParseQuery("Course");
courseQuery.whereEqualTo("objectId",courseId);
query.whereMatchesQuery("course", courseQuery);
When I run the query using query.getFirst(), I do not get anything retrieved from the local storage. I have already checked running the courseQuery separately and it fetches me Course object that I need. Is this a known issue? I proceeded with this way by getting help from this post.
I think you are mixing query while storing pointer to another class in your database. Use following code to resolve your problem:
ParseQuery<PracticeSessionDetails> query = ParseQuery.getQuery(PracticeSessionDetails.class);
query.fromLocalDatastore();
query.ignoreACLs();
query.whereEqualTo("user", ParseUser.getCurrentUser());
ParseQuery<ParseObject> courseQuery = ParseQuery.getQuery("Course");
courseQuery.whereMatchesQuery("course", query);
I'm currently writing an aggregation query for MongoDB in my Spring project in which I'm using $project operator. Within this operator I would like to compare two fields in order to return the result as projected "matches" key value. Here's the mongoDB shell equivalent (which works):
{$project:
{matches:
{$eq: ["$lastDate", "$meta.date"]}
}
}
I've read Spring Data MongoDB documentation and found some useful info about ProjectionOperator's 'andExpression' method which uses SpEL. The result Java code of my investigation was:
new ProjectionOperation().andExpression("lastDate == meta.date").as("matches")
Unfortunately I'm receiving exception:
java.lang.IllegalArgumentException: Unsupported Element:
org.springframework.data.mongodb.core.spel.OperatorNode#70c1152a Type: class org.springframework.data.mongodb.core.spel.OperatorNode You probably have a syntax error in your SpEL expression!
As far as I've checked, Spring Data MongoDB handles all Arithmetic operators correctly but cannot handle the comparison ones. Therefore I want to ask is there any other way to create such query with Spring Data MongoDB? Or maybe I don't know something crucial about SpEL?
I resolved this issue by passing JSON aggregate command (created with DBObjects in order to preserve flexibility of the query) to MongoDB, i.e.:
MongoOperations#executeCommand(DBObject command)
I'm using tire and elasticsearch. The service has started using port 9200. However, it was returning 2 errors:
"org.elasticsearch.search.SearchParseException: [countries][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"query_string":{"query":"name:"}}}]]"
and
"Caused by: org.apache.lucene.queryParser.ParseException: Cannot parse 'name:': Encountered "<EOF>" at line 1, column 5."
So, I reinstalled elasticsearch and the service container. Service starts fine.
Now, when I search using tire I get no results when results should appear and I don't receive any error messages.
Does anybody have any idea how I might find out what is wrong, let alone fix it?
first of all, you don't need to reindex anything, in the usual cases. It depends how you installed and configured elasticsearch, but when you install and upgrade eg. with Homebrew, the data are persisted safely.
Second, no need to reinstall anything. The error you're seeing means just what it says on the tin: SearchParseException, ie. your query is invalid:
{"query":{"query_string":{"query":"name:"}}}
Notice that you didn't pass any query string for the name qualifier. You have to pass something, eg:
{"query":{"query_string":{"query":"name:foo"}}}
or, in Ruby terms:
Tire.index('test') { query { string "name:hey" } }
See this update to the Railscasts episode on Tire for an example how to catch errors due to incorrect Lucene queries.