Is there a way to rename the output fields from an elasticsearch query (like the "select .. AS .." in SQL)? I want to keep the original names of the fields in the index, and only modify their names in the result of the query.
I've read this but it says to re-index the data.
This isn't currently possible in Elasticsearch. If you need this kind of transformation, you'll need to do it application-side.
Related
I have already ingested data to the existing Elasticsearch index with _id as one of the column name "mainid" value in database. Now I have another table in that I have two columns "mainid" and path to the files. I want to ingest these files using fSCrawler into the existing Elasticsearch index and files should get ingested into corresponding _id.
The _id is generated from the filename by FSCrawler. If you want to provide your own _id, for the time being, you need to use the REST service instead.
It allows you to provide the _id you want. Like:
echo "This is my text" > test.txt
curl -F "file=#test.txt" -F "id=my-test" "http://127.0.0.1:8080/fscrawler/_upload"
I have a use case where I have to store documents with field names after some processing. But for search purposes I want that document to return with an alias of mine.
This is specific to removing Dots "." from the input field names, but keeping the search results oblivious of the change.
Example:
Fieldname recieved: My.Field.Name
Processed name in ES: My<Separator>Field<Separator>Name
Expected Search Result: My.Field.Name
I am assuming that Field Aliases are not supported by ElasticSearch right now. But is there any work around for this.
I have a few records in my elasticsearch collection and i want to use a GroupBy aggregation in elasticsearch querystring.
I want to know if it is possible, because i tried to google it always give result about this
i want to use this something like this in the query string , which can
give me records in the group.
For i.e.
http://localhost:9200/_all/tweets/_count?q=user:Pu*+user:Kim*
This will give me count of all the records which has name starts from Pu and Kim,
But i want to know that how many records are there has name starting with Pu
and Kim,
aggregations need to be specified in addition in the search request, you cannot specify them as part of a query string query.
You could also just execute two queries to find out this particular requirement...
For example I have next documents.
{sourceIP:1.1.1.1, destIP:2.2.2.2}
{sourceIP:1.1.1.1, destIP:3.3.3.3}
{sourceIP:1.1.1.1, destIP:4.4.4.4}
Is there anyway to automatically aggregate them into one document which will contain next data?
{sourceIP:1.1.1.1, destIP:{2.2.2.2,3.3.3.3,4.4.4.4}}
So it looks like group by in SQL, but generate new documents in elasticsearch instead of old one.
I dont think there is anyway to do indexing time auto-merging of documents.
However , it should be possible to acheive whatever result you are planning to query should be possible by using one of querying options offered by Elasticsearch - while indexing one document for ,
Like ..
You can index seperate documents, query by sourceIP and use aggregations to give dest_ip
Take count of documents if its just to find dest_ips for a source_ip
Also if you want to avoid duplicate source_id + dest_id combinations , you can concat and use it as _id of document
Hope this helps.
I have a use case which is a bit similar to the ES example of dynamic_template where I want certain strings to be analyzed and certain not.
My document fields don't have such a convention and the decision is made based on an external schema. So currently my flow is:
I grab the inputs document from the DB
I grab the approrpiate schema (same database, currently using logstash for import)
I adjust the name in the document accordingly (using logstash's ruby mutator):
if not analyzed I don't change the name
if analyzed I change it to ORIGINALNAME_analyzed
This will handle the analyzed/not_analyzed problem thanks to dynamic_template I set but now the user doesn't know which fields are analyzed so there's no easy way for him to write queries because he doesn't know what's the name of the field.
I wanted to use field name aliases but apparently ES doesn't support them. Are there any other mechanisms I'm missing I could use here like field rename after indexation or something else?
For example this ancient thread mentions that field.sub.name can be queried as just name but I'm guessing this has changed when they disallowed . in the name some time ago since I cannot get it to work?
Let the user only create queries with the original name. I believe you have some code that converts this user query to Elasticsearch query. When converting to Elasticsearch query, instead of using the field name provided by the user alone use both the field names ORIGINALNAME as well as ORIGINALNAME_analyzed. If you are using a match query, convert it to multi_match. If you are using a term query, convert it to a bool should query. I guess you get where I am going with this.
Elasticsearch won't mind if a field does not exists. This can be a problem if there is already a field with _analyzed appended in its original name. But with some tricks that can be fixed too.