I have the weird effect that a search on an index returns a document I have just deleted before. A "get" works correctly. Am I doing something wrong? The search has no restrictions(client.prepareSearch("test").execute(...))
I'm running an "ESIntegTestCase" with Elastic Search 5.0
#Test
public void testES() throws Exception {
String index = "test";
String type = "event";
String doc = "{\"Key0\":\"Val0\"}";
createIndex(index);
Semaphore sem = new Semaphore(0);
client().prepareIndex(index, type).setSource(doc).execute(handleOrError(postResp -> {
client().prepareGet(postResp.getIndex(), postResp.getType(), postResp.getId()).execute(handleOrError(getResp -> {
printGR(getResp);
client().prepareSearch(postResp.getIndex()).execute(handleOrError(searchResponse -> {
printSR(searchResponse);
client().prepareDelete(postResp.getIndex(), postResp.getType(), postResp.getId()).execute(handleOrError(resp -> {
printDR(resp);
client().prepareGet(postResp.getIndex(), postResp.getType(), postResp.getId()).execute(handleOrError(getResp2 -> {
printGR(getResp2);
client().prepareSearch(postResp.getIndex()).execute(handleOrError(searchResponse2 -> {
printSR(searchResponse2);
sem.release();
}));
}));
}));
}));
}));
}));
sem.acquire();
}
Prints:
1) GetResponse: {"_index":"test","_type":"events","_id":"AVgv1NHPHZ0vJaA-eRhJ","_version":1,"found":true,"_source":{"Key0":"Val0"}}
2) SearchResponse:{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"test","_type":"events","_id":"AVgv1NHPHZ0vJaA-eRhJ","_score":1.0,"_source":{"Key0":"Val0"}}]}}
3) DeleteResponse: DeleteResponse[index=test,type=events,id=AVgv1NHPHZ0vJaA-eRhJ,version=2,result=deleted,shards="_shards"{"total":2,"successful":2,"failed":0}]
4) GetResponse: {"_index":"test","_type":"events","_id":"AVgv1NHPHZ0vJaA-eRhJ","found":false}
5) SearchResponse:{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":"test","_type":"events","_id":"AVgv1NHPHZ0vJaA-eRhJ","_score":1.0,"_source":{"Key0":"Val0"}}]}}
You found the difference between the search index and doing a get request. A get request also makes use of the transaction log. If you want the delete to have an effect on search, you need to execute a refresh. With elastic 5 there now is an option to wait for a refresh after an insert or a delete. Using that functionality should give you what you want. More information can be found here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-refresh.html
Related
I am currently trying to build a request to ElasticSearch using the elasticsearch dependency.
Below, find the simplified version of the code I have written:
fn test<'a>(client: &'a Elasticsearch) -> BoxFuture<'a, std::result::Result<Bytes, elasticsearch::Error>> {
let index_parts = ["foo", "bar"]; // Imagine this list being computed and not literal
let search_response = client
.search(SearchParts::Index(&index_parts))
.from(0)
.size(1000)
.body(json!(
{ "query": { "match_all": { } } }
))
.send();
search_response
.and_then(|resp| resp.bytes())
.boxed()
}
The error I get:
cannot return value referencing local variable index_parts
returns a value referencing data owned by the current function
I totally understand why I get this error - I create a new array inside of test, but SearchParts::Index expects only a &'b [&'b str], so I have no way of giving ownership to it. So I am stuck with it.
Of course there are a couple of simple solutions to this, first and foremost simply inlining test instead of creating a separate function, or somehow returning index_parts with the Future, but those solutions leak implementation details and we all know that this is bad.
So, how do I fix this error without breaking encapsulation?
A colleague of mine suggested the following solution:
async fn test<'a>(client: &'a Elasticsearch) -> std::result::Result<Bytes, elasticsearch::Error> {
let index_parts = vec!["foo", "bar"]; // Imagine this list being computed and not literal
let search_response = client
.search(SearchParts::Index(&index_parts))
.from(0)
.size(1000)
.body(json!(
{ "query": { "match_all": { } } }
))
.send()
.await?;
search_response.bytes().await
}
Although I understand that with async, everything inside of the function body is effectively a closure, I feel like there should be a way of doing this without using async as well, but I am not sure.
There is a further problem with this solution though - the returned Future has a lifetime bound to the lifetime of the Elasticsearch reference which leads to issues down the line.
To fix this:
async fn test(client: & Elasticsearch) -> std::result::Result<Bytes, elasticsearch::Error> {
let client = client.clone();
async move {
let index_parts = ["foo", "bar"];
let search_response = client
.search(SearchParts::Index(&index_parts))
.from(0)
.size(1000)
.body(json!(
{ "query": { "match_all": { } } }
))
.send()
.await?;
Ok(search_response.bytes().await?)
}.await
}
I'm struggling to figure out how to include multiple indexes in a search using the Elastic low level client.
My understanding (right or wrong) is that I should be able to include multiple indexes by separation of commas, this doesn't work for me though. In the code example below, I find that the first index specified is still working and returning results, but the second one is ignored. Any ideas?
Appsettings.json file:
// System settings configured here for the WebApp. Applicable to all users.
"SystemSettings": {
// Sets the maximum number of distinct values returned by Elastic for a log property
"_distinctPropertyValuesLimit": 1000, // See LogPropertiesController.cs
// String for the list of Elastic Search indexes that are searched by default.
"indexesToSearch": "webapp-razor-*, systemconfig-api-*"
}
Query class:
_indexesToSearch = configuration.GetSection("SystemSettings").GetSection("indexesToSearch").Value;
var searchResponse = await _elasticLowLevelClient.SearchAsync<StringResponse>(_indexesToSearch, #"
{
""from"": """ + fromParameter + #""",
""size"": """ + rowsPerPage + #""",
""query"": {
""match"": {
""" + searchColumn + #""": {
""query"": """ + searchString + #"""
}
}
},
""sort"": [
{
""#timestamp"": {
""order"": ""desc""
}
}
]
}
");
Turns out that there must not be any spaces between the index names when multiple values are provided, see below:
After doing a simple natural language query in the build query page, set the options for "include relevant passages to Yes. I get back 5 passages and results. All good. When I try from npm ibm-watson 6 nodejs sdk. I get the results, but an empty passages array with the same natural langauge text.
Here is the the url from the query page showing the options, which I tried all of them
https://api.us-east.discovery.watson.cloud.ibm.com/instances/d45df72b-93e4-4897-b9ac-98dc1e088afc/v1/environments/xx/collections/xx/query?version=2018-12-03&deduplicate=false&highlight=true&passages=true&passages.count=5&natural_language_query=what%20is%20autism
Here is the answer from the query builder page
Here is code example,
var discovery = new watson_discovery_v1({
authenticator : new IamAuthenticator({apikey: msg.startup.discovery_password}),
serviceUrl : msg.startup.discovery_endpoint,
version: '2020-09-22'
});
msg.WDSParams = {
environmentId: "x",
collectionId: "x",
passages: true,
count:5,
natural_language_query: msg.params.input.text
}
discovery.query(msg.WDSParams)
.then(results => {
msg.WDSResults = results; //your query results
node.send(msg);
})
.catch(err => {
console.log('error:', err);
});
Here is the json that came back from the discovery call
I have tried all of the passage options, duplicated the exact options that the query builder used. The same results come back, but no passages. Anyone have an idea? BTW using the Lite plan until I can prove passages works.
The problem was related to the way I called the query method. Below code resolved the issue. This code is for a nodeRed function node.
const watson_discovery_v1 = global.get('watson_discovery_v1');
const { IamAuthenticator } = global.get('ibm_auth');
const discovery = new watson_discovery_v1({
authenticator : new IamAuthenticator({apikey:
msg.startup.discovery_password}),
serviceUrl : msg.startup.discovery_endpoint,
version: '2019-04-30'
});
async function run() {
try {
const result = await discovery.query({
environmentId: 'x',
collectionId: 'x',
passages: true,
passagesCount: 2,
count: 5,
naturalLanguageQuery: msg.params.input.text
})
msg.WDSResults = result
clearTimeout(myTM)
}
catch(e){
node.error(e)
}
node.send(msg);
}
run()
This is a pretty simple question.
How to implement subscriptions in graphql?
I'm asking specifically for when using graphql.js constructors like below ?
I could not find a clean/simple implementation.
There is another question here, but it deals with relay.js - i don't want to unnecessarily increase the nr of external dependencies in my app.
What i have:
module.exports = function (database){
return new GraphQLSchema(
{ query: RootQuery(database)
, mutation: RootMutation(database)
, subscription: RootSubscription(database) -- i can see this in graphiql - see below
}
);
}
function RootSubscription(database){
return new GraphQLObjectType(
{ name: "RootSubscriptionType"
, fields:
{ getCounterEvery2Seconds:
{ type: new GraphQLNonNull(GraphQLInt)
, args :
{ id: { type: GraphQLString }
}
, subscribe(parent, args, context){
// this subscribe function is never called .. why?
const iterator = simpleIterator()
return iterator
}
}
}
}
)
}
I learned that i need a subscribe() which must return an iterator from this github issue.
And here is a simple async iterator. All this iterator does - is to increase and return the counter every 2 seconds. When it reaches 10 it stops.
function simpleIterator(){
return {
[ Symbol.asyncIterator ]: () => {
let i = 0
return {
next: async function(){
i++
await delay(2000)
if(i > 10){
return { done: true }
}
return {
value: i,
done: false
}
}
}
}
}
}
When i run the graphiql subscription, it returns null for some reason:
I'm piecing together code from multiple sources - wasting time and hacking it basically. Can you help me figure this one out?
Subscriptions are such a big feature, where are they properly documented? Where is that snippet of code which you just copy paste - like queries are for example - look here.
Also, i can't use an example where the schema is separate - as a string/from a file. I already created my schema as javascript constructors. Now since im trying to add subscriptions i can't just move back to using a schema as a string. Requires rewriting the entire project. Or can i actually have both? Thanks :)
I am doing a chat app using parse server, everything is great but i tried make to list just last message for every remote peer. i didn't find any query limitation how to get just one message from every remote peer how can i make this ?
Query limitation with Parse SDK
To limit the number of object that you get from a query you use limit
Here is a little example:
const Messages = Parse.Object.extend("Messages");
const query = new Parse.Query(Messages);
query.descending("createdAt");
query.limit(1); // Get only one result
Get the first object of a query with Parse SDK
In you case as you really want only one result you can use Query.first.
Like Query.find the method Query.first make a query and will return only the first result of the Query
Here is an example:
const Messages = Parse.Object.extend("Messages");
const query = new Parse.Query(Messages);
query.descending("createdAt");
const message = await query.first();
I hope my answer help you 😊
If you want to do this using a single query, you will have to use aggregate:
https://docs.parseplatform.org/js/guide/#aggregate
Try something like this:
var query = new Parse.Query("Messages");
var pipeline = [
{ match: { local: '_User$' + userID } },
{ sort: { createdAt: 1 } },
{ group: { remote: '$remote', lastMessage: { $last: '$body' } } },
];
query.aggregate(pipeline)
.then(function(results) {
// results contains unique score values
})
.catch(function(error) {
// There was an error.
});