I am trying to migrate my es 2.2.1 with spring data elastic 2 to ES 5.6.1 with spring Data elastic 3 but when i am trying to use scan scroll method for large dataaset i am not able to access them, its look like ElasticsearchTemaplate class do not have these function any more in the newer version Can you please let me know what is the substitute of scan and scroll in SDA 3.x:-
ElasticsearchTemplate rep=null;
String scrollId = rep.scan(searchQuery, (long)25000, false);
//List<SampleEntity> sampleEntities = new ArrayList<SampleEntity>();
boolean hasRecords = true;
You now can use startScroll method instead of scan() and scroll().
It's not presented in current docs.
Here are example ElasticsearchTemplate retrieve big data sets
Related
I am trying to fetch data from influxdb(version 2) bucket into my Spring Boot application. I have not found any references on google to implement this feature.
Can anyone help me in this my providing any references or documentation so I can implement this feature.
String token = "tokenDetails";
String bucket = "test123";
String org = "test";
InfluxDBClient client = InfluxDBClientFactory.create("https:localhost:8086/",
token.toCharArray());
String query = String.format("from(bucket: \"%s\") |> range(start: -15m)", bucket);
List<FluxTable> tables = client.getQueryApi().query(query, org);
Iterate on tables and records to operate on data.
Steps to write elastic search count api using spring boot.?
how to use countRequest to get count of documents...??
need code snippets.
Using new Java API Client:
var countRequestResponse = client.count(CountRequest.of(c -> c.index("idx_name")));
System.out.println(countRequestResponse.count());
Using Java High Client Library
var countRequest = new CountRequest();
countRequest.indices("idx_name");
var response = getClient().count(countRequest, RequestOptions.DEFAULT);
System.out.println(response.getCount());
I'm using the Nest client to programmatically execute requests against an Elasticsearch index. I need to use the UpdateByQuery API to update existing data in my index. To improve performance on large data sets, the recommended approach is to use slicing. In my case I'd like to use the automatic slicing feature documented here.
I've tested this out in the Kibana dev console and it works beautifully. I'm struggling on how to set this property in code through the Nest client interface. here's a code snippet:
var request = new Nest.UpdateByQueryRequest(indexModel.Name);
request.Conflicts = Elasticsearch.Net.Conflicts.Proceed;
request.Query = filterQuery;
// TODO Need to set slices to auto but the current client doesn't allow it and the server
// rejects a value of 0
request.Slices = 0;
var elasticResult = await _elasticClient.UpdateByQueryAsync(request, cancellationToken);
The comments on that property indicate that it can be set to "auto", but it expects a long so that's not possible.
// Summary:
// The number of slices this task should be divided into. Defaults to 1, meaning
// the task isn't sliced into subtasks. Can be set to `auto`.
public long? Slices { get; set; }
Setting to 0 just throws an error on the server. Has anyone else tried doing this? Is there some other way to configure this behavior? Other APIs seem to have the same problem, like ReindexOnServerAsync.
This was a bug in the spec and an unfortunate consequence of generating this part of the client from the spec.
The spec has been fixed and the change will be reflected in a future version of the client. For now though, it can be set with the following
var request = new Nest.UpdateByQueryRequest(indexModel.Name);
request.Conflicts = Elasticsearch.Net.Conflicts.Proceed;
request.Query = filterQuery;
((IRequest)request).RequestParameters.SetQueryString("slices", "auto");
var elasticResult = await _elasticClient.UpdateByQueryAsync(request, cancellationToken);
I have to upgrade our ES from 6.2 (TCP client) to 7.4.2 (REST API)
And I have a little problem with UpdateByQueryRequestBuilder, that looks like need to be change to UpdateByQueryRequest (doc). The old code looks like this:
BoolQueryBuilder dateQueryBuilder = ...
QueryBuilder a = ...
BoolQueryBuilder b = ...
UpdateByQueryRequestBuilder updateByQuery = new UpdateByQueryRequestBuilder(tcpClient, UpdateByQueryAction.INSTANCE);
updateByQuery.filter(dateQueryBuilder.filter(a).filter(b)).script(updateScript);
As I wrote, I'm understanding UpdateByQueryRequestBuilder (that using the oldest client) should be replace with UpdateByQueryRequest but this new API haven't filter method (just setQuery that will replace the current query in chain case...)
UpdateByQueryRequest updateRequest = new UpdateByQueryRequest();
updateRequest.setQuery(dateQueryBuilder)
// .setQuery(a) - will replace dateQueryBuilder instead of chain new filter...
// .filter - not exist in the new API
So the question, how should I replace this code with newest ES REST API (or chain the queries)?
I have configured 2 solr cores and trying to map to 2 different classes thru solrnet. I'm currently using Ninject but willing to change to say Windsor if it's not possible in Ninject. I'm trying to use the AllPropertiesMappingManager for mapping. Since I need to set 2 different unique keys for 2 different cores I don't know how to do the same using AllPropertiesMappingManager.
Currently without using the Mapping manager I'm getting the error: Document is missing mandatory uniqueKey field: TranscriptId
EDIT: Error disappears after using attribute based mapping
var solrServers = new SolrServers {
new SolrServerElement {
Id = "markup",
Url = solrMarkupUrl,
DocumentType = typeof(SolrMarkup).AssemblyQualifiedName,
},
new SolrServerElement {
Id = "transcript",
Url = solrTranscriptUrl,
DocumentType = typeof(SolrTranscript).AssemblyQualifiedName,
}
};
kernel = new StandardKernel();
kernel.Load(new SolrNetModule(solrServers));
SolrMarkupCore = kernel.Get<ISolrOperations<SolrMarkup>>("markup");
SolrTranscriptCore = kernel.Get<ISolrOperations<SolrTranscript>>("transcript");
You can look at the SolrNet unit tests for Ninject with multiple cores - NinjectMultiCoreFixtures.cs for a working example.
Also, if you are not using the mapping manager are you useing Attribute based mapping to get things setup? Because you will still need to setup the mapping between your SolrMarkup and SolrTranscript classes fo things to work properly.