I am using a Reactive client for Elasticsearch. Both of them (ReactiveElasticsearchTemplate or ReactiveCrudRepository) have support for Flux/Mono and this is working fine for me.
My current problem is that using ReactiveElasticsearchTemplate:
return reactiveElasticsearchTemplate.search(
new NativeSearchQueryBuilder()
.withPageable(PageRequest.of(0, 10))
.build(),
UserDocument.class
)
It is returning slice, but I don't have any information about total elements, as method signature returns Flux<SearchHit<UserDocument>>. Ideally, I would like to get SearchHits or something similar, but I don't know how to achieve it.
Currently this information is not available in the reactive implementation there is an open issue for that. The only possibility at the moment is sending a count request first.
Related
I've read the docs and watched the Laracast. I'm still left wondering why you would use them?
I get that you can map different data to different names if your field names were to change but you want to keep a consistent public API. But surly you can just do the same on the model with the toArray() method and change the mappings there?
If I were to do:
return User::find(1);
I get a response like:
{"id":1,"name":"Ova Parker"}
If I do:
return new UserResource(User::find(1));
I get a response like:
{"data":{"id":1,"name":"Ova Parker"}}
Is there a significance in wrapping it with a data tag? I am just guessing but is this a standard JSON format for API's? Why would you not just do return User::find(1); instead of using an API resource, if this is under API routes then it'll return it as JSON anyway automatically.
You kind of answer the question by yourself. The idea behind API Resources or Transformers (like the ones from Fractal) is to hide the db field names from the client. With return User::find(1) you expose your whole db structure which might not a good idea security-wise and also bad for the release process. If you need to change a db field name, you have to change the API too. With Resources you have a mapping between your application and the consumer of you API.
It seems more work in the beginning, but once you started it, you won't wanna miss it anymore.
There is no toArray() method in PHP, which gets magically called like __toString(). You have to write you own and call it by yourself. Resources are built-in by Laravel and will be resolved automatically.
I have Django app which use elasticsearch database, I use elasticsearch-dsl and all my filters and queries works. But I have a problem with one parameter, it's UUID. I always got 0 results from my request in shell:
s = Search(index='my_index_name').filter('term', UUID='0deaa49b-15b6-4c10-acb7-d98df800e0df')
response=s.execute()
response
I use django-rest-elasticsearch and I have the same issue, I got correct REST result with all my filters, but not with UUID request. Something like this works, but I need to use filtering.
q = Q("multi_match", query="0deaa49b-15b6-4c10-acb7-d98df800e0df", fields=["UUID",])
response=s.execute()
response
Maybe someone know hot to use UUID in my REST, because UUID=0deaa49b-15b6-4c10-acb7-d98df800e0df don't work.
I want to consume data from a GraphQL API.
how can I achieve that? I am new to webapi and any help is appreciated
Assuming the API you want to consume uses HTTP, you should be able to use cUrl, wget, Charles, Postman or even just the URL bar in a browser to make a request.
To write your first query, you can start with the following:
query theNameOfMyQuery {
}
Now that you have a named query, you can start populating it with whatever fields your GraphQL server is exposing. For a blog example, you might have something like this:
query theNameOfMyQuery {
posts {
title
author
}
}
Now, to turn that into something you can request, all you need to do is URL encode it and add it to your URL. A typical URL looks like this:
https://www.someserver.com/?query=...&variables=...
So for the above example, you would have the above query
https://www.someserver.com/?query=query%20theNameOfMyQuery%20%7B%0D%0A%20%20posts%20%7B%0D%0A%20%20%20%20title%0D%0A%20%20%20%20author%0D%0A%20%20%7D%0D%0A%7D
Some Resources:
Evolution of API Design - this video explains some of the concepts of GraphQL and why it exists.
howtographql.com - This is an amazing set of tutorials for every implementation you could imagine
I need to have the sort configuration to be part of the response, so lets assume i have this query
{ sort: [ {"name":"asc"},{"age:"descr"}]}
I would need to have this as part of the response to synchronise my facets / ui state with that sorting. I see there is a "sort" response field, but it basically lists the values which have been picked for the sort, but not which field and which sort type.
Reading the docs i am not sure it should be the case https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_sort_values
Not able to find anything about this in the web, a lot of how to sort examples, also on stack, but nothing about how to reflect the sort in the response.
If it matters, i am currently using Elasticsearch 2.4
This is in fact not possible out of the box.
I solved this using my middleware. So when a client wants to search on ES, it happens this way
client -> middleware -> ES
To include sorting in the response, the middleware does something like this
result = es.search(query)
result['sort'] = query['sort'] if query.key?('sort')
return result
So i copy the sort field from the request into the response and this is in fact very useful for the client, when dealing with aggregations / facetted searches
I'm trying to debug an ElasticSearch query. I've enabled explain for the problematic query, and that is showing that the query is doing a product of intermediate scores where it should be doing a sum. (I'm creating the query request using elastic4s.)
The problem is I cannot see what the generated query actually is. I want to determine whether the bug is in elastic4s (generating the query request incorrectly), in my code, or in elasticsearch. So I've enabled logging for the embedded elasticsearch instance used in the tests using the following code:
ESLoggerFactory.setDefaultFactory(new Slf4jESLoggerFactory())
val settings = Settings.settingsBuilder
.put("path.data", dataDirPath)
.put("path.home", "/var/elastic/")
.put("cluster.name", clusterName)
.put("http.enabled", httpEnabled)
.put("index.number_of_shards", 1)
.put("index.number_of_replicas", 0)
.put("discovery.zen.ping.multicast.enabled", false)
.put("index.refresh_interval", "10ms")
.put("script.engine.groovy.inline.search", true)
.put("script.engine.groovy.inline.update", true)
.put("script.engine.groovy.inline.mapping", true)
.put("index.search.slowlog.threshold.query.debug", "0s")
.put("index.search.slowlog.threshold.fetch.debug", "0s")
.build
but I can't find any queries being logged in the log file configured in my logback.xml. Other log messages from elasticsearch are appearing there, just not the actual queries.
You can't, at least not directly, at least not in ES versions currently available. It's something that has been discussed at some length (eg https://github.com/elastic/elasticsearch/issues/9172 and https://github.com/elastic/elasticsearch/issues/12187) it seems like this may change soon, with the rewrite of the tasks API. In the meantime, you can use things like ES Restlog (https://github.com/etsy/es-restlog) and/or put nginx in front of ES and capture the queries in the nginx logs. You can also use tcpdump (eg tcpdump -vvv -x -X -i any port 9200) and capture the query as it's running on the server. One last option is to modify your application and echo the query instead of executing it (and/or inserting the query into ES itself before you execute it, since the query itself is JSON).
In the specific case of elastic4s, it offers the ability to call .show on the elastic4s query object to generate what the JSON body part of the request would have been if the JSON-over-HTTP protocol had been used to send the request, for most types of request. This can then be logged at a convenient point in your code, e.g. if you have one method that generates all ES search queries. The code in Elasticsearch that generates the fake JSON could still have bugs of course, so it should not entirely be trusted. However, it's worth trying to reproduce the issue with the output of .show using Sense against a real Elasticsearch cluster over HTTP - if you can, you (a) know that it's not an elastic4s bug, and (b) can easily manipulate the JSON to try to figure out what's causing the problem.
show calls toString in some cases, so with the plain Elasticsearch API or another JVM-based wrapper on top of it, you can call that to get the JSON string to log.
With embedded Elasticsearch, this is as good as you're going to get in terms of logging - short of putting a breakpoint on the builder invocations and observing the actual Java Elasticsearch request objects that are created (which is the most accurate approach).