I am trying to call the below query using NEST
GET 123_original/_doc/_mtermvectors
{
"ids": [
"9a271078-086f-4f4b-8ca0-16376c2f49a7",
"481ce3db-69bf-4886-9c38-fcb878d44925"
],
"parameters": {
"fields": ["*"],
"positions": false,
"offsets": false,
"payloads": false,
"term_statistics": false,
"field_statistics": false
}
}
The NEST API (I think) would look something like this
var term = await elasticClient.MultiTermVectorsAsync(x =>
{
return x.Index(indexOriginal) // 123_original
.Type(typeName) // _doc
.GetMany<ElasticDataSet>(ids.Keys) // list of strings
.Fields("*")
.FieldStatistics(false)
.Positions(false)
.Offsets(false)
.TermStatistics(false)
.Payloads(false);
});
The problem is that the above API is returning the following error
Index name is null for the given type and no default index is set. Map an index name using ConnectionSettings.DefaultMappingFor<TDocument>() or set a default index using ConnectionSettings.DefaultIndex().
And this is the query that its trying to execute which has the index in it and is missing the ids, but works in Kibana when the ids are set.
123_original/_doc/_mtermvectors?fields=%2A&field_statistics=false&positions=false&offsets=false&term_statistics=false&payloads=false
I cannot find a documentation on how to use the Multi Term Vector using NEST.
The Multi Term Vectors API within NEST does not expose the ability to set only Ids, it always assumes that you are passing "docs".
Even when passing
client.MultiTermVectors(mt => mt
.Index("123_original")
.Type("_doc")
.GetMany<object>(ids)
.Fields("*")
.Positions(false)
.Offsets(false)
.Payloads(false)
.TermStatistics(false)
.FieldStatistics(false)
);
The _index and _type for each id is inferred from object in GetMany<T>
POST http://localhost:9200/123_original/_doc/_mtermvectors?pretty=true&fields=*&positions=false&offsets=false&payloads=false&term_statistics=false&field_statistics=false
{
"docs": [
{
"_index": "users",
"_type": "object",
"_id": "9a271078-086f-4f4b-8ca0-16376c2f49a7"
},
{
"_index": "users",
"_type": "object",
"_id": "481ce3db-69bf-4886-9c38-fcb878d44925"
}
]
}
I think this could be exposed in a more consumable way within the client in the future.
The good news is that you can submit the exact query that you would like with the low level client exposed on IElasticClient, and still get back a high level response
MultiTermVectorsResponse response =
client.LowLevel.Mtermvectors<MultiTermVectorsResponse>("123_original", "_doc", PostData.Serializable(new
{
ids = ids,
parameters = new
{
fields = new[] { "*" },
positions = false,
offsets = false,
payloads = false,
term_statistics = false,
field_statistics = false
}
}));
which will send the following request:
POST http://localhost:9200/123_original/_doc/_mtermvectors?pretty=true
{
"ids": [
"9a271078-086f-4f4b-8ca0-16376c2f49a7",
"481ce3db-69bf-4886-9c38-fcb878d44925"
],
"parameters": {
"fields": [
"*"
],
"positions": false,
"offsets": false,
"payloads": false,
"term_statistics": false,
"field_statistics": false
}
}
Related
I am trying to check whether client_id is already exists in the index or not. but problem is ES still retrieving full ID, even though I am giving half of the id. here is the mapping.
'mappings': {
'properties': {
'client_id': {'index': 'true','type': 'keyword'},
'client_name': {'index': 'true', 'type': 'keyword'},
'data_index_server': {'type': 'ip'},
'data_file_node_path': {'index': 'true', 'type': 'keyword'},
}
If I have the record like this
{
"_index": "client_index",
"_type": "_doc",
"_id": "wYlkrYMB_q_jkYaCv6pU",
"_version": 1,
"_score": 1,
"_source": {
"doc": {
"client_id": "0935be6b-61fe-4ec4-80c8-5c5ee8384378",
"client_name": "citi",
"data_file_node_path": " ",
"data_index_server": " "
}
},
"fields": {
"doc.client_id": [
"0935be6b-61fe-4ec4-80c8-5c5ee8384378"
],
"doc.client_name": [
"sample_name"
],
"doc.data_index_server": [
" "
],
"doc.client_name.keyword": [
"citi"
],
"doc.data_file_node_path.keyword": [
" "
],
"doc.client_id.keyword": [
"0935be6b-61fe-4ec4-80c8-5c5ee8384378"
],
"doc.data_index_server.keyword": [
" "
],
"doc.data_file_node_path": [
" "
]
}
}
my request for search is this. and if I am taking some part of the ID and search against it. I am expecting to be hits will be zero
POST /client_index/_search
{
"query": {
"match": {
"doc.client_id":"0935be6b-61fe-4ec4-80c8" ,
}
}
}
I have follwed this url: How to make elastic search only match full field. change the fields type to keywords and also toggled index between true and false, but no result
I think you have mixed up few things, in the mapping you showed just four properties which are simple fields, while in the search response, it has additional fields under fields which are of type object.
You read it correct, keyword fields are not analyzed and on these fields you will get results only on full field value.
You should just try to define your client-id as a keyword field in new index with your sample field and search on that index to see it in action.
I am new to elasticsearch and I am having troubles with the Nest/Elastic.Net library.
I would like to retrieve not the entire document but just part of it. I am able to do it in Postman but I cannot do it via Elastic.Net library or Nest library.
Document structure looks like following
{
“Doc_id”: “id_for_cross_refference_with_othersystem”
“Ocr”:[
{
“word”: “example_word1”,
“box”: [],
“cord”: “some_number”,
},
{
“word”: “example_word2”,
“box”: [],
“cord”: “some_number2”,
}
]
}
The document has a huge amount of properties but I am interested only in Doc_id , ocr.word, ocr.box and ocr.cord.
The following postman request fully satisfies my needs :
{
"query": {
"bool": {
"must": [
{
"match": {
"doc_id": "2a558865-7dc2-4e4d-ad02-3f683159984e"
}
},
{
"nested": {
"path": "ocr",
"query": {
"match": {
"ocr.word": "signing"
}
},
"inner_hits": {
"_source": {
"includes":[
"ocr.word",
"ocr.box",
"ocr.conf"
]
}
}
}
}
]
}
},
"_source":"false"
}
Result of that request is following :
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 18.99095,
"hits": [
{
"_index": "irrelevant",
"_type": "irrelevant",
"_id": "irrelevant",
"_score": 18.99095,
"_source": {},
"inner_hits": {
"ocr": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 7.9260864,
"hits": [
{
"_index": "irrelevant",
"_type": "irrelevant",
"_id": "irrelevant",
"_nested": {
"field": "ocr",
"offset": 11
},
"_score": 7.9260864,
"_source": {
"box": [
],
"conf": "96.452858",
"word": "signing"
}
}
]
}
}
}
},
{
"_index": "there_rest _of_object_is_ommited",
},
{
"_index": "there_rest _of_object_is_ommited",
}
]
}
}
However when I try to convert that request to Nest Query DSL I am not able to achieve the same result.
When I try to use the NEST library I don’t see any way to provide output result model/type. It looks like the Type of Document should match the output type which is not my case.
Query that I am using :
var searchResponse = client2.Search<Model>(s => s
.Query(q1 => q1.Bool(b1 => b1.Must(s1 =>
s1.Match(m => m.Field(f => f.doc_id).Query("2a558865-7dc2-4e4d-ad02-3f683159984e")),
s2 => s2.Nested(n => n.Path("ocr").Query(q2 => q2.Bool(b => b.Must(m => m.Match(m => m.Field(f => f.ocr.First().word).Query("signing")))))
.InnerHits(ih => ih.Source(s => s.Includes(i => i.Field(f => f.ocr.First().word).Field(f => f.ocr.First().conf))))
)
)))
.Source(false)
);
Due to the fact that the Model type is created for a document and it doesn’t match the output type I am getting [null, null, null] as the output .
There is property such properties as Hits in ISearchResponse? But when I look into it I cannot see values of fields.
I tried using a low level client (Elastic.Net) and providing json request as a string. But It looks like there is not way of specifying the output type either. When I ran my code with the low level library it returns me 3 object of class Model with empty fields.
My questions are :
Is it possible to specify output type different from document type for Nest query DSL or Elatic.Net library ?
Is it possible to get values of the fields that I specified in request for inner hits with help of Nest or Elastic.Net libraries?
How would you solve such problem ? I mean we have huge documents and we don’t want to pass unnecessary information back and forth. The inner hits approach looks like a neat solution for us but it doesn’t look like it works with the recommended libraries Unless I am doing some silly mistake.
NOTE: I can achieve desired result with simple use of HTTPClient and manually doing what I need , but I hope to leverage library that is written for this purpose(Nest or Elastic.Net).
I'm working on a call to readQuery. I'm getting an error message:
modules.js?hash=2d0033b4773d9cb6f118946043f7a3d4385825fe:25847
Error: Can't find field resolutions({"id":"Resolution:DHSzPa8bvPCDjuAac"})
on object (ROOT_QUERY) {
"resolutions": [
{
"type": "id",
"id": "Resolution:AepgCCio9KWGkwyMC",
"generated": false
},
{
"type": "id",
"id": "Resolution:DHSzPa8bvPCDjuAac", // <==ID I'M SEEKING
"generated": false
}
],
"user": {
"type": "id",
"id": "User:WWv57KsvqWeAoBNHY",
"generated": false
}
}.
The object with that id appears to be plainly visible as the second entry in the list of resolutions.
Here's my query:
const GET_CURRENT_RESOLUTION_AND_GOALS = gql`
query Resolutions($id: String!) {
resolutions(id: $id) {
_id
name
completed
goals {
_id
name
completed
}
}
}
`;
...and here's how I'm calling it:
<Mutation
mutation={CREATE_GOAL}
update={(cache, {data: {createGoal}}) => {
let id = 'Resolution:' + resolutionId;
const {resolutions} = cache.readQuery({
query: GET_CURRENT_RESOLUTION_AND_GOALS,
variables: {
id
},
});
}}
>
What am I missing?
Update
Per the GraphQL Dev Tools extension for Chrome, here's the whole GraphQL data store:
{
"data": {
"resolutions": [
{
"_id": "AepgCCio9KWGkwyMC",
"name": "testing 123",
"completed": false,
"goals": [
{
"_id": "TXq4nvukpLcqQhMRL",
"name": "test goal abc",
"completed": false,
"__typename": "Goal"
},
],
"__typename": "Resolution"
},
{
"_id": "DHSzPa8bvPCDjuAac",
"name": "testing 345",
"completed": false,
"goals": [
{
"_id": "PEkg5oEEi2tJ6i8LH",
"name": "goal abc",
"completed": false,
"__typename": "Goal"
},
{
"_id": "X4H4dFzGm5gkq5bPE",
"name": "goal bcd",
"completed": false,
"__typename": "Goal"
},
{
"_id": "hYunrXsMq7Gme7Xck",
"name": "goal cde",
"completed": false,
"__typename": "Goal"
}
"__typename": "Resolution"
}
],
"user": {
"_id": "WWv57KsvqWeAoBNHY",
"__typename": "User"
}
}
}
Posted as answer for fellow apollo users with similar problems:
Remove the prefix of Resolution:, the query should only take the id.
Then the question arises how is your datastore filled?
To read a query from cache, the query needs to have been called with exactly the same arguments on the remote API before. This way apollo knows what the result for a field is with specific arguments. If you never called the remote endpoint with the arguments you want to use but know what the result would be, you can circumvent that and resolve the query locally by implementing a cache resolver. Have a look at the example in the documentation. Here the store contains a list of books (in your case resultions) and the query for a single book by id can be resolved with a simple cache lookup.
I'm trying to use NEST for searching through elastic search indexex that were created with logstash (basically logstash-*).
I have setup NEST with following code:
Node = new Uri("http://localhost:9200");
Settings = new ConnectionSettings(Node);
Settings.DefaultIndex("logstash-*");
Client = new ElasticClient(Settings);
this is how I try to get results:
var result = Client.Search<Logstash>(s => s
.Query(p => p.Term("Message", "*")));
and I get 0 hits:
http://screencast.com/t/d2FB9I4imE
Here is an example of entry I would like to find:
{
"_index": "logstash-2016.06.20",
"_type": "logs",
"_id": "AVVtswJxpdkh1tFPP9S5",
"_score": null,
"_source": {
"timestamp": "2016-06-20 14:04:55.6650",
"logger": "xyz",
"level": "debug",
"message": "Processed command service method SearchService.SearchBy in 65 ms",
"exception": "",
"url": "",
"ip": "",
"username": "",
"user_id": "",
"role": "",
"authentication_provider": "",
"application_id": "",
"application_name": "",
"application": "ZBD",
"#version": "1",
"#timestamp": "2016-06-20T12:04:55.666Z",
"host": "0:0:0:0:0:0:0:1"
},
"fields": {
"#timestamp": [
1466424295666
]
},
"sort": [
1466424295666
]
}
I'm using 5.0.0-alpha3 version, and NEST client is alpha2 version atm.
This is because of
...
"_type": "logs",
...
When you are doing query like yours it will hit logstash not logs type, because NEST infers type name from generic parameter. You have two options to solve this problem.
Tell NEST to map Logstash type to logs type whenever making
request to elasticsearch, by setting this mapping in client's
settings:
var settings = new ConnectionSettings()
.MapDefaultTypeNames(m => m.Add(typeof(Logstash), "logs");
var client = new ElasticClient(settings);
Override default behaviour by setting type explicitly in request
parameters:
var result = Client.Search<Logstash>(s => s
.Type("logs")
.Query(p => p.Term("message", "*")));
Also notice you sould use message not Message in term descriptor
as you don't have such field in index. Second this is as far as I
know wildcards are not supported in term query. You may want to use
query string instead.
Hope it helps.
How do I get the timestamp value? I already set enable and store as true, I can see it on Sense, but was not able to get it.
{
"cubx": {
"mappings": {
"organization": {
"_timestamp": {
"enabled": true,
"store": true
},
"properties": {
"address": {
.
.
.
I can see it...
GET /abc/organization/1234?fields=_timestamp
{
"_index": "abc",
"_type": "organization",
"_id": "1234",
"_version": 1,
"found": true,
"fields": {
"_timestamp": 1430535032967
}
}
But I can't retrieve it...
public GetField getTimestamp(Long companyId) {
GetResponse response = client
.prepareGet(index, type, companyId.toString()).execute()
.actionGet();
return response.getField("_timestamp");
It returns null. I already read a lot of posts here but didn't find an example to get the value to a object. I also tried to use script_value as suggested in this post but without success.
Can someone help me to figure out what I'm doing wrong?
You would need to use it like this GetResponse response = client .prepareGet(index, type, companyId.toString()).setFields("_timestamp").execute() .actionGet();.