I need to use Watson Discovery Services to search patterns of text from unstructured documents. It gives me the whole document as JSON. How can I apply a filter to train/find certain patterns during indexing or querying?
Related
Elasticsearch versions 7 and 8.
When Elasticsearch ingests data, it generates certain phonetic keys for the tokens (and other types depending on the analyzer you specify). Is there a way to retrieve and view these for a given document via query?
You can use the analyze API on specific index and provide the text of your field in your documents to see the tokens generated by Elasticsearch.
Please refer to the examples given in the documentation.
Hi I'm trying out Elastic Enterprise Search with Elasticsearch. I have a couple of questions on data indexing.
When referring to Elasticsearch documentation, I read that there is a limit to the number of fields that an Elasticsearch index could have. Since Elasticsearch is used with Elastic Enterprise Search I believe there is no arguing that the same applies here. In that case lets say I have multiple document types with various fields. For an example Person.json and Dog.json, they both have different properties. So when indexing I use one search engine in Elastic Enterprise Search to index both Person and Dog so that when I query using the Elastic Enterprise Search API I'll get results which are both Person and Dog depending on the search term.
Is this the way to go,or should I specify a seperate search engine for each schema type?
I am assuming that your person.json and dog.json contains different fields as your heading suggest and weather to create a separate index for these entities or have them in a single index, depends on the various use-cases you have in your application and you will not find elasticsearch marking one approach better than other and mainly will explain the pros/cons based on a particular context(like relevance, performance, management etc).
Please refer to my this SO answer, where I talked about various pros/cons of both the approach and discussion in chat to get more context why OP chose an approach based on his use-case, after knowing the pros/cons.
I have been using ES to handle regular text/keyword search, is there a way to use elastic search to handle context based search i.e when user have given a search text "articles between 10 august and 24 September" and such similar scenarios, ES should be able identify what user is asking and present results. I suppose we are supposed to involve ML to handle such scenarios, If any NLP or ML integrations need to be done where should i start to up the search experience.
Any insight over this is much appreciated
This is called semantic parsing. What you need to do is to map the sentence to a logical form. This is a challenging task, since the computer needs to understand your sentence. You may create your own Semantic Parser(e.g., SEMPRE) to do the translation, or use existing methods to do such translations (translate human language to elastic search queries).
I am evaluating search technologies and one of my requirements is the ability to hit translated text also.
For example, there are text documents written in English and French. And lucene will index them.
If I am searching for the string "apple", it should search for both "apple" and "pomme" and show documents with either.
Will any technologies provide automatic translation of token words ?
Or only way to do that is to translate it using Google API and then feed it to lucene for indexing?
There are no automatic translations in Lucene/Solr/Elasticsearch, but they have a similar feature, called Synonyms. You can create a list of synonyms with Google Api to translate the terms in the search time, not the index time.
With this approach, you can search for "apple" and the search engine will see "apple" and "pomme" as synonyms, and you will get the result as expected.
Couchbase FTS is now an official feature in version 5. Why would one still use ElasticSearch along with Couchbase?
Quoting from the documentation:
Couchbase FTS is similar in purpose to other search software such as
ElasticSearch or Solr. Couchbase FTS is not intended as a replacement
for third party search software if search is at the core of your
application. It is a simple and lightweight way to add search to your
Couchbase data without deploying additional software and servers. If
you have many queries which look like SELECT ... field1 LIKE %pattern% OR field2 LIKE %pattern, then full-text search may be right for you.
It will depend on your specific use case, but there is a reason why search is a complicated problem and some products spent years and years on working on that (and continue).
Full text search NOT EQUAL Search engine. Full Text Search does support a lot of functions that ElasticSearch provides. For example in ElasticSearch you can set weight of fields in result set, do geo search etc. Couchbase full text search is just full text search implementation, i.e. basic string matching function in specially indexed field only.
So, if your task is to do basic search on sub string as a part of a query, then you don't need ElasticSearch anymore. It make development quicker and infrastructure cheaper. However, if you are building system that need proper search engine, then you need ElasticSearch as much as before.