ES Version : 7.10.2
I have a requirement to show index statistics, I have come across the index stats API which does fulfill my requirement.
But the issue is I don't necessarily need all the fields for a particular metric.
Ex: curl -XGET "http://localhost:9200/order/_stats/docs"
It shows response as below (omitted for brevity)
"docs" : {
"count" : 7,
"deleted" : 0
}
But I only want "count" not "deleted" field, from this.
So, in Index Stats API documentation, i came across a query param as :
fields:
(Optional, string) Comma-separated list or wildcard expressions of fields to include in the statistics.
Used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters
As per above when I perform curl -XGET "http://localhost:9200/order/_stats/docs?fields=count"
It throws an exception
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "request [/order/_stats/docs] contains unrecognized parameter: [fields]"
}
],
"type" : "illegal_argument_exception",
"reason" : "request [/order/_stats/docs] contains unrecognized parameter: [fields]"
},
"status" : 400
}
Am I understanding the usage of fields correctly ?
If yes/no, how can I achieve the above requirement ?
Any help is much appreciated :)
You can use the filter_path argument, like:
curl -XGET "http://localhost:9200/order/_stats?filter_path=_all.primaries.docs.count
This will return you only one field like:
{
"_all" : {
"primaries" : {
"docs" : {
"count" : 10
}
}
}
}
I created a simple project in Drools work bench. The project has 1 data object and 2 DRL files. I build and deployed the project to the KIE server. And I have created a simple spring boot application which loads data into the data object using a rest service and fires the rules. Below is the code:
public class Main {
private static String containerId = "ImportProducts_1.0.1-LATEST";
private static String user = "kieserver";
private static String password = "kieserver1!";
private static String url = "http://localhost:8180/kieserver/services/rest/server";
private static final Logger LOGGER = LoggerFactory.getLogger(Main.class);
private static final MarshallingFormat FORMAT = MarshallingFormat.JSON;
private static String CLASS_NAME = "ImportProduct";
public static void main(String[] args) {
List<ImportProduct> prods = new ArrayList<>();
prods.add(new ImportProduct("1", "Grocery - Milk", "OK", 25.0));
prods.add(new ImportProduct("2", "Fashion - Trouser", "NOT_OK", 1300.0));
prods.add(new ImportProduct("3", "Grocery - Wheat", "OK", 425.0));
prods.add(new ImportProduct("4", "Grocery - Dairy Milk Chocolate", "OK", 100.0));
KieServicesConfiguration config = KieServicesFactory.newRestConfiguration(url, user, password, 60000);
config.setMarshallingFormat(MarshallingFormat.JSON);
RuleServicesClient client = KieServicesFactory.newKieServicesClient(config).getServicesClient(RuleServicesClient.class);
List<Command<?>> cmds = new ArrayList<>();
KieCommands commands = KieServices.Factory.get().getCommands();
cmds.add(commands.newInsert(prods, CLASS_NAME));
cmds.add(commands.newFireAllRules());
BatchExecutionCommand myCommands = CommandFactory.newBatchExecution(cmds);
ServiceResponse<ExecutionResults> response = client.executeCommandsWithResults(containerId, myCommands);
if (response.getType() == ServiceResponse.ResponseType.SUCCESS) {
LOGGER.info("Commands executed with success! Response: ");
LOGGER.info("{}", response.getResult());
List<ImportProduct> prodUpdated = (List<ImportProduct>) response.getResult().getValue(CLASS_NAME);
//sale.setDiscount(saleUpdated.getDiscount());
LOGGER.info("Response is: {}", response.getMsg());
LOGGER.info("Output is: {}", prodUpdated.toString());
} else {
LOGGER.error("Error executing rules. Message: {}", response.getMsg());
}
//KieServices kieServices = KieServices.Factory.get();
//ReleaseId releaseId = (ReleaseId) kieServices.newReleaseId( "com.test", "ImportProducts", "1.0.1-LATEST" );
}
}
The application runs without errors but the there is no impact of the rules on the data fed in the KIE container. The rule deletes the data object if the status != 'OK'. I am getting all the data back I sent in the request body of the POST request service. I think DRL capability is enabled as shown in the stack trace below:
22:11:40.139 [main] DEBUG org.kie.server.client.impl.AbstractKieServicesClientImpl - About to deserialize content:
'{
"type" : "SUCCESS",
"msg" : "Kie Server info",
"result" : {
"kie-server-info" : {
"id" : "kie-server-a017ffcb29dc",
"version" : "7.55.0.Final",
"name" : "kie-server-a017ffcb29dc",
"location" : "http://172.17.0.3:8080/kie-server/services/rest/server",
**"capabilities" : [ "KieServer", "BRM", "BPM", "CaseMgmt", "BPM-UI", "BRP", "DMN", "Swagger" ],**
"messages" : [ {
"severity" : "INFO",
"timestamp" : {
"java.util.Date" : 1623926563857
},
"content" : [ "Server KieServerInfo{serverId='kie-server-a017ffcb29dc', version='7.55.0.Final', name='kie-server-a017ffcb29dc', location='http://172.17.0.3:8080/kie-server/services/rest/server', capabilities=[KieServer, BRM, BPM, CaseMgmt, BPM-UI, BRP, DMN, Swagger]', messages=null', mode=DEVELOPMENT}started successfully at Thu Jun 17 10:42:43 UTC 2021" ]
} ],
"mode" : "DEVELOPMENT"
}
}
}'
Below is the response I got back:
22:11:41.515 [main] DEBUG org.kie.server.client.impl.AbstractKieServicesClientImpl - About to deserialize content:
'{
"type" : "SUCCESS",
"msg" : "Container ImportProducts_1.0.1-LATEST successfully called.",
"result" : {
"execution-results" : {
"results" : [ {
"value" : [{"com.test.importproducts.ImportProduct":{
"id" : "1",
"category" : "Grocery - Milk",
"status" : "OK",
"price" : 25.0
}},{"com.test.importproducts.ImportProduct":{
"id" : "2",
"category" : "Fashion - Trouser",
"status" : "NOT_OK",
"price" : 1300.0
}},{"com.test.importproducts.ImportProduct":{
"id" : "3",
"category" : "Grocery - Wheat",
"status" : "OK",
"price" : 425.0
}},{"com.test.importproducts.ImportProduct":{
"id" : "4",
"category" : "Grocery - Dairy Milk Chocolate",
"status" : "OK",
"price" : 100.0
}}],
"key" : "ImportProduct"
} ],
"facts" : [ {
"value" : {"org.drools.core.common.DefaultFactHandle":{
"external-form" : "0:3:459769708:-992461270:3:DEFAULT:NON_TRAIT:java.util.ArrayList"
}},
"key" : "ImportProduct"
} ]
}
}
}'
Please help.
I fixed the issue. Below is the working code:
private Command<?> prepareCommands(List<ImportProduct> facts, String sessionName, String outIdentifier) {
KieCommands commandsFactory = KieServices.Factory.get().getCommands();
List<Command> commands = facts.stream().map(commandsFactory::newInsert).collect(Collectors.toList());
commands.add(commandsFactory.newFireAllRules());
ObjectFilter factsFilter = new ClassObjectFilter(ImportProduct.class);
//System.out.println("Check1: "+commandsFactory.newGetObjects(factsFilter, CLASS_NAME));
//System.out.println("Check2: "+commands.toString());
commands.add(commandsFactory.newGetObjects(factsFilter, CLASS_NAME));
return commandsFactory.newBatchExecution(commands, sessionName);
}
I'm trying to understand why I get 401.
Steps taken:
created a elasticsearch.keystore
add the bootstrap.password successfully, with cat ~/.elk.secret | /opt/elasticsearch/bin/elasticsearch-keystore add -x 'bootstrap.password'
Since I've multiple instances running, I move it in the new conf directory
but now I get 401:
curl -X GET 'https://elastic:myFancyPass#myServer:9200/_cluster/health?pretty' -k
{
"error" : {
"root_cause" : [
{
"type" : "security_exception",
"reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]",
"header" : {
"WWW-Authenticate" : [
"Bearer realm=\"security\"",
"ApiKey",
"Basic realm=\"security\" charset=\"UTF-8\""
]
}
}
],
"type" : "security_exception",
"reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]",
"header" : {
"WWW-Authenticate" : [
"Bearer realm=\"security\"",
"ApiKey",
"Basic realm=\"security\" charset=\"UTF-8\""
]
}
},
"status" : 401
}
My understanding is that this error can be caused by two things:
wrong password (which I doubt it's my case)
wrong elasticsearch.file
How do I find which elasticsearch.keystore it is loading?
Running rootLogger.level = debug didn't help. I feel it would be great getting a confirmation such as /opt/elasticsearch/instance2/conf/elasticsearch.keystore
Dynamic value or a variable doesn't work inside a elasticsearch "range" query.
For explaining more this is a elasticsearch range query which find productId from 1000 to 11100, which is working perfectly ---
$json = '{
"query" : {
"range" : {
"productId" : {
"from" : '1000',
"to" : '11100'
}
}
}
}';
On the other hand using the same query with a variable with the same value it returns me error like ---
{"error":"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures
$a =1000;
$b = 11100;
$json = '{
"query" : {
"range" : {
"productId" : {
"from" : '$a',
"to" : '$b'
}
}
}
}';
Do anyone knows where i am making the mistake.
Any suggestion will be great help. Thanks in advanced.
If this is PHP, there's a problem with string concatenation:
$a = 1000;
$b = 11100;
$json = '{
"query" : {
"range" : {
"productId" : {
"from" : '.$a.',
"to" : '.$b.'
}
}
}
}';
see the dots around the variables.
If you run the original piece of code by itself, the PHP parser should give you a parse error.
Is it possible to use a manually created histogram facet (or the results of its curl request) like this one in a Kibana dashboard:
{
"query" : {
"match_all" : {}
},
"facets" : {
"histo1" : {
"histogram" : {
"key_script" : "doc['date'].date.minuteOfHour * factor1",
"value_script" : "doc['num1'].value + factor2",
"params" : {
"factor1" : 2,
"factor2" : 3
}
}
}
}
}
Thanks
It looks like it will be supported in Kibana4, but there doesn't seem to be much more info out there than that.
For reference: https://github.com/elasticsearch/kibana/issues/1249