Sometimes I'm facing too big Elasticsearch queries with duplicated parts with applying the same filtering structure into aggregations (for every aggregation field). Such queries are too massive for inspecting them. Is there any way to decrease request body size? A kind of aliases maybe, I need something like variables in YAML. Or maybe you could suggest something else. Thanks!
Please have a look on search templates. You'll be able to store query templates in the cluster, use variables and even build dynamic queries:
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template.html
Using this feature will reduce your request body dramatically as you'll just refer a pre registered template, providing some parameters if needed.
Repeating blocks and conditional sections are possible using mustache templating language http://mustache.github.io/mustache.5.html
Have fun!
Related
I'm fairly new to GraphQL, so please correct me if I'm wrong where need be. I'm wondering what would be the best way, or the most appropriate way, to execute batch queries in a single request. I've seen examples that do this:
[
{
query: "query(param1)"
},
{
query: "query(param2)"
}
]
Or examples that use aliases to accomplish the same thing:
query {
alias1: resolver1(param1)
alias2: resolver2(param2)
}
In my opinion, I think the aliases is the more appropriate option as it's a single query and within that query I'm requesting data with different parameters. So, between the two options above, what would be the more appropriate option for executing batch queries in GraphQL, and why? Thanks!
Batching operations by sending an array of them to the server is something that's not supported by all libraries. This is not a feature of GraphQL as much as just a convenient feature supported by some server libraries. On the other hand, including multiple root-level fields in a single operation is supported by all GraphQL servers. So, if you're anticipating the backend you are querying to possibly change in the future, you may want to use Option #2 over Option #1 to avoid having to potentially refactor your code in the future.
On the other hand, batching is often desirable because it allows you to mix operation types. While you can include any number of root-level fields in a single operation, only one operation may be executed at a time -- that means you can't mix queries and mutations with Option #2. If this is something you need to do, batching is the only way to achieve it in a single request (assuming your server supports it).
Did the Explain endpoint ever support search_type: dfs_query_then_fetch? If it does now (I'm on 7.1), how do I specify it?
I was thrown for a loop when using the Explain API on two identical documents, but seeing different score calculations. Learning the documents lived in different shards, and that the TF/IDF inputs were calculated per-shard explained the difference. Using dfs_query_then_fetch on the Search API normalized the scores, but the ElasticSearch .net client (both LowLevel and NEST) don't appear to expose a way to specify it for calls to the Explain API.
I also tried to form a request manually, passing it as a querystring or request body parameter. Both fail saying the argument is invalid. I thought perhaps the Explain endpoint didn't offer a way to specify dfs_query_then_fetch, but digging through some old issues it appears that it at least did at some point:
https://github.com/elastic/elasticsearch/issues/2612
Search type is not supported on the explain API. An approach that might work would be to use the Search API with dfs_query_then_fetch and explain, with a compound query that filters only to the document you're interested in (using IdsQuery), along with the query you want the explanation for.
I am getting some numerical data with API from URL and I am looking for a way to make some mathematical operations in apache nifi before putting data to file directory. Thanks already now.
By the way, I am using InvokeHTTP processor to get data and to put file in somewhere I am using PutFile processor. I searched some related websites but I could not find out a working way.
Try using QueryRecord processor and Define Record Reader/Writer controller services to read/write the flowfile.
Add new property to the QueryRecord processor by using Apache calcite SQL query with your mathematical operations on flowfile.
Results of the SQL query will be added to the outgoing flowfile in your desired format.
Ultimately the answer depends on whether the data you're working with is in the content of the FlowFile or in the attributes. If the data is small enough and it's only a couple operations, the suggested approach would be to work with the data as attributes and use NiFi's expression language to do the transformations.
There is a section of mathematical operations[1] in the Apache documentation[2]. The operations range from simple operand like plus/minus to exposing the java.lang.Math static methods.
[1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#numbers
[2] https://nifi.apache.org/docs.html
You can try ExecuteStreamCommand if you want to intake the whole file and then run operations. Alternatively, you can fiddle around with the variables on the flowfile - depending on how large your operation is.
For example if you have some initial variables you can include them in the name of your file and then extract them, run the operations within the variables of the flowfile, then add to the bottom of the original file
Is it possible with a FHIR search procedure to search for TWO resource types and sort them according to the date? I'd like a list of Observation and QuestionnareResponses, in a single response, returning the newest 10 regardless of resource type.
Searching for one would be:
http://apps.ehelselab.com/baseDstu2/Observation?_sort:desc=date
Any query using the standard "search" capability is always against exactly one resource type. You can include referencing and referenced resources, but filtering and sorting are always done against the "base" resource for the search. To do what you're interested in doing, you have a few options:
define a custom query using the OperationDefinition mechanism (only works if you've got a direct relationship between client and server systems so you can ensure all participants support the operation
Use a "Batch" to execute queries against both, then interpolate the results as you page through both result sets
You can do a query just on the "base", however there isn't presently a way to constrain the types of resources returned - you'd need a custom search criteria
You might be able to use the _filter mechanism - I haven't dived into it very deeply. But I suspect that it also uses the "single target resource type" approach.
The best bet is probably #3. If you submit a request to add a search criteria to "Resource" allowing constraining the resource type, that would probably let you do what you wanted.
I want to change the snippet(<S> in results xml) length , is it possible using front-end or some other way. Currently its returning long snippet, i want to reduce it as per new requirement. But this should not affect other collections and frontend in gsa, because GSA box is serving lots of websites.
Have you tried changing the res_snippet_size variable in the XSLT for your specific front-end?