GraphQL related resources explain how query analysis can be done to protect the GraphQL endpoint. Some of the approaches which are being used are query depth analysis, query complexity analysis, etc. The question that I have is, does Query Analysis only refer to Queries? What about Mutations and Subscriptions? Or are all 3 (Query, Mutation, Subscription) included when we talk about query analysis?
Apollo Tracing is one of the Query Analysis Tools and based on what I have used it in graphql-java , it can be used on all Query , Mutation and Subscription. So , I believe the term Query Analysis can apply to all of them.
After all , all of them are handled in a pretty much the same way internally inside a GraphQL engine as defined by the spec. In the Execution Operation section, you can see both Query and Mutation has the same execution logic.The only differences are that Query is allow to execute in parallel while mutation can only execute in serial.
Then in the subscription response stream section ,it mentions :
The ExecuteSubscriptionEvent() algorithm is intentionally similar to
ExecuteQuery() since this is how each event result is produced.
which mean at the end, its execution logic is the same as Query.
Related
I have a GraphQL api implemented with Nexus-graphql and Prisma + Postgres.
Prisma has a nice solution for N+1 issue with query batching when it comes to GraphQL but seems to be working on 2 levels of nesting, e.g. User -> Posts, but when we need multiple levels of GraphQL nesting (User -> Posts -> Comments -> Likes -> Users) it seems to execute hundreds of queries.
What are the general best practices for doing query optimization in such cases?
I have 2 ideas but maybe there are more:
Using prisma's "include" option. Construct a map of prisma relations based on the received graphql query (from "info" field) and query all the relations at once in a single query.
Using a dataloader, query all necessary data and save to dataloader and within the resolvers load models from dataloder.
1. I want to create a GraphQL query that can filter by multiple records.
It is about filtering details of statistics. For example, the statistic contains the fields "Number of deaths", "Number of cases", "Number of recovered".
I have already written queries that can filter by the individual fields. Now I want to program a query that uses multiple filters, or a query in which multiple queries are nested.
I have already tried to define the individual steps of each query in a common query. You can see this in the attached images. The program compiles first. However, when I execute the query in the GraphQL UI, I get error messages
2. Unfortunately, I have not yet received any helpful tips regarding my query or my error.
Screenshot
At the top left you can see the individual queries, at the top right the merged query and at the bottom the errors as soon as I try to execute the query.
In Elasticsearch, can I associate each document with a (different) function that must be satisfied by parameters I supply on a search, in order to be returned on that search?
The particular functions I would particularly like to use involve a loop, some kind of simple branching (if-statement of switch-statement), an array-like data structure, strings comparisons, and simple boolean operators.
couple of keynotes here:
At query time:
- If your looking to shape the relevancy function, meaning the actual relevancy score of each document, you could use a script score query.
- If you're only looking to filter out unwanted documents, you could use a script query that allows you to do just that.
Both of those solutions enables you to compute a score comparing incoming query parameters against existing previously indexed values.
Take note that usage of scripts at query time can lead to increased memory usage and performance issues.
Elastic can also handle a second batch of filtering rules that are applied to the actual query result in the form of a post filter. Can come in handy sometime if you're not in a position of stream processing the output at API view level.
At index time:
There is such a thing called script fields that allows you to store a function that computes a result based on other fields value and incoming query parameters. they can be really powerful given the fact that they are assigned at index time. I think they might be what you are looking for.
I would not be using those if i weren't to have those field values compared against query params. Reason is that I like my index process to be lean and fast so I tend to compute those kinds of values at stream level, in upstream from the actual bulk indexing query.
Although convenient, those custom scripts results are likely to be achievable with a combination of regular queries and filters. In each release, the elasticsearch teams is adding new query and field types that let you do what you use to do via scripted queries whiteout the risk of blowing out you memory. a good example of this is the rank feature datatype recently introduced in the 7.x release.
A piece of advice for you. think of your elasticsearch service as a regular API in your datalayer. As such you can do query processing before the actual call to elastic and you can do data processing from the actual elastic results. If you really can't fit your business rules in there, that would be your last resort.
Fell free to contact me if you still have any questions. All the best.
I have a Spring Boot project with org.springframework.boot:spring-boot-starter-data-elasticsearch:jar:2.0.0.RELEASE connecting to a elasticsearch-6.3.1 server.
I have the following scenario: for some elasticsearch query (which involves a should bool), I get different scores from when I run the query manually, using curl.
Steps I have tried: Extract query with debugger from SearchQuery before calling the repo, extract query from elasticsearch logs (using "index.search.slowlog.threshold.fetch.debug" : "0s", "index.search.slowlog.threshold.query.debug" : "0s"); in both cases, running the queries manually, with curl, gives a set of scores that are different from the ones given by Java api.
I mention that I couldn't find a pattern by looking at the diff between the two score sets. The scores returned by the manual query seem to be the correct ones, because I expect some of them to have the same value, which does not happen for the ones returned by the api.
If you have any ideas on what might cause this or how to continue the investigation it is much appreciated.
I have managed to make the api return the same scores as the manual run by wrapping the inner query with a constantScoreQuery, it seems that the TF/IDF criteria was the 'culprit'.
It is still curious, though, why the manual query behaved as ignoring TF/IDF in the first place ..
Horsing around the CosmosDB .NET SDK, I found an intriguing issue.
I wanted to use the SDK LINQ support for the following query:
SELECT VALUE COUNT(1) FROM c WHERE <predicate>
Well, immediately after writing the LINQ query, I realized there might be no way to handle such queries as document queries. However, since document queries allow you to capture query metrics and unblock a thread between pages of results, I strongly prefer it. Here's the code:
client.CreateDocumentQuery<TEntity>.Where(<predicate>).Count()
Even though I understand that the result type of Count() isn't IQueryable, is there a way to handle "count" queries as document queries?
Of course there is a way to do that.
There is a CountAsync extension built into the SDK.
Simply construct your IQueryable and use .CountAsync() when you are ready to get the count.
Keep in mind however that there will be no metric collection as the result is aggregated in the SDK.
If you really need metrics then you can use the usual DocumentQuery created from SQL rather than LINQ and the while.HasMoreResults, ExecuteNextAsync logic and capture the metrics from the paginated result, per iteration.