Is it possible to have field type that would look like List>?
So I have Java POJOs:
public class Document {
//omitted other fields
private List<ChildDocument> values;
class ChildDocument {
private List<String> words;
private Timerange timerange;
}
class Timerange {
// time in format "10:15:30.222"
// LocalTime from Java-8 time api
private LocalTime start;
private LocalTime end;
}
}
Example data in such field:
index -> {time-range -> list of strings}
(0) -> "08:00:00.000 - 09:00:00.000" -> {"1", "2", "3"}
(1) -> "11:00:00.000 - 13:00:00.000" -> {"4", "5", "6"}
And I would query "find 'time-range' that has '5' in it" and I would get "08:00:00.000 - 09:00:00.00" as property defined as in my Java ChildDocument. Is it possible to store in ES just time-range without dates? And do this kind of nested objects? I know I cound store that TimeRange field as String and parse it back in my Java app but trying to optimize it so ES does most of the work.
Still new to ES (came from Solr and that's impossible there).
Related
I need to fetch Employees who joined between 2021-12-01 to 2021-12-31. I am using ElasticsearchRepository to fetch data from ElasticSearch index.
How can we fetch range criteria using repository.
public interface EmployeeRepository extends ElasticsearchRepository<Employee, String>,EmployeeRepositoryCustom {
List<Employee> findByJoinedDate(String joinedDate);
}
I have tried Between option like below: But it is returning no results
List<Employee> findByJoinedDateBetween(String fromJoinedDate, String toJoinedDate);
My Index configuration
#Document(indexName="employee", createIndex=true,type="_doc", shards = 4)
public class Employee {
#Field(type=FieldType.Text)
private String joinedDate;
Note: You seem to be using an outdated version of Spring Data Elasticsearch. The type parameter of the #Document
annotation was deprecated in 4.0 and removed in 4.1, as Elasticsearch itself does not support typed indices since
version 7.
To your question:
In order to be able to have a range query for dates in Elasticsearch the field in question must be of type date (the
Elasticsearch type). For your entity this would mean (I refer to the attributes from the current version 4.3):
#Nullable
#Field(type = FieldType.Date, pattern = "uuuu-MM-dd", format = {})
private LocalDate joinedDate;
This defines the joinedDate to have a date type and sets the string representation to the given pattern. The
empty format argument makes sure that the additional default values (DateFormat.date_optional_time and DateFormat. epoch_millis) are not set here. This results in the
following mapping in the index:
{
"properties": {
"joinedDate": {
"type": "date",
"format": "uuuu-MM-dd"
}
}
}
If you check the mapping in your index (GET localhost:9200/employee/_mapping) you will see that in your case the
joinedDate is of type text. You will either need to delete the index and have it recreated by your application or
create it with a new name and then, after the application has written the mapping, reindex the data from the old
index into the new one (https://www.elastic.co/guide/en/elasticsearch/reference/7.16/docs-reindex.html).
Once you have the index with the correct mapping in place, you can define the method in your repository like this:
List<Employee> findByJoinedDateBetween(LocalDate fromJoinedDate, LocalDate toJoinedDate);
and call it:
repository.findByJoinedDateBetween(LocalDate.of(2021, 1, 1), LocalDate.of(2021, 12, 31));
I want to get one Embedded Document with a specific field (version) from an array with mongodb and spring boot.
This is the data structure:
{
"_id": 5f25882d28e40663719d0b52,
"versions": [
{
"versionNr": 1
"content": "This is the first Version of some Text"
},
{
"versionNr": 2
"content": "This is the second Version of some Text"
},
...
]
...
}
Here are my entities:
#Data
#Document(collection = "letters")
public class Letter {
#Id
#Field("_id")
private ObjectId _id;
#Field("versions")
private List<Version> versions;
}
//There is no id for embedded documents
#Data
#Document(collection = "Version")
public class Version{
#Field("content")
private String content;
#Field("version")
private Long version;
}
And this is the query that doesn't work. I think the "join" isn't correct. But can't figure out the right way.
public Optional<Version> findByIdAndVersion(ObjectId id, Long version) {
Query query = new Query(Criteria.where("_id").is(id).and("versions.version").is(version));
return Optional.ofNullable(mongoTemplate.findOne(query,Version.class,"letters"));
}
}
EDIT: This is a working Aggregation, I'm sure it isn't a pretty solution but it works
#Override
public Optional<Version> findByIdAndVersion(ObjectId id, Long version) {
MatchOperation match = new MatchOperation(Criteria.where("_id").is(id).and("versions.version").is(version));
Aggregation aggregate = Aggregation.newAggregation(
match,
Aggregation.unwind("versions"),
match,
Aggregation.project()
.andInclude("versions.content")
.andInclude("versions.version")
);
AggregationResults<Version> aggregateResult = mongoTemplate.aggregate(aggregate, "letters", Version.class);
Version version = aggregateResult.getUniqueMappedResult();
return Optional.ofNullable(mongoRawPage);
}
Query query = new Query(Criteria.where("_id").is(id).and("versions.version").is(version));
return Optional.ofNullable(mongoTemplate.findOne(query,Version.class,"letters"));
You are querying the Letter document but your entity class is specified as Version.class, since findOne from MongoDB doesn't return the subdocument by itself but rather the whole document, you need to have Letter.class as return type and filter (project) what fields to get back. So you can either project the single version subdocument that you want to receive, like so:
Query query = new Query()
.addCriteria(Criteria.where("_id").is(id).and("versions.version").is(version))
.fields().position("versions", 1);
Optional.ofNullable(mongoTemplate.findOne(query, Letter.class))
.map(Letter::getVersions)
.findFirst()
.orElse(null);
or use aggregation pipeline:
newAggregation(
Letter.class,
match(Criteria.where("_id").is(id)),
unwind("versions"),
replaceRoot("versions"),
match(Criteria.where("version").is(version))),
Version.class)
Note -- I typed this on a fly.
I use Spring boot 2.0.1.RELEASE/ Spring Data Elasticsearch 3.0.6.
I annotate my domain class with #Document annotation and i have a field as below:
#Field(store = true, type = FieldType.?)
private String ipRange;
as you see, I need to set the field type to IP_Range (exists in elastic search engine data types)
but not exists in FieldType enum.
I want to create this document index by ElasticsearchTemplate.createIndex(doc) method. but none of any FieldType enum support ip_range data type.
Spring Data Elasticsearch currently (3.2.0.M2) does not support this. I saw that you already opened an issue, thanks for that. The answer here is just for the completeness and for other users having the same problem
Thanks #P.J.Meisch for your reply, I used #Mapping annotation to specify my mapping directly via json format. Already Spring data supports creating index based on this config. but i am also waiting for Range Data Structure Support to refactor my code.
My Document:
#Document(createIndex = true, indexName = "mydomain", type = "doc-rule"
, refreshInterval = BaseDocument.REFRESH_INTERVAL, replicas = BaseDocument.REPLICA_COUNT, shards = BaseDocument.SHARD_COUNT)
#Mapping(mappingPath = "/elasticsearch/mappings/mydomain-mapping.json")
public class MyDomainDoc {
#Field(store = true, type = FieldType.text)
private List<String> ipRange;
... other fields
}
And My mydomain-mapping.json file:
{
"properties": {
...,
"ipRange": {
"type": "ip_range",
...
},
...
}
}
I have 2 collections (articles & tags). Articles has 'tags' which is array of objectIds. I am trying to join articles and tags so as to get tagName from tags. Below is my mongodb query:
db.articles.aggregate([
{"$unwind": "$tags"},
{"$lookup": {
"localField": "tags",
"from": "tags",
"foreignField": "_id",
"as": "materialTags"
}
}
])
I converted this into spring Data as below
UnwindOperation unwindOperation = Aggregation.unwind("tags");
LookupOperation lookupOperation1 = LookupOperation.newLookup()
.from("tags")
.localField("tags")
.foreignField("_id")
.as("materialTags");
Aggregation aggregation = Aggregation.newAggregation(unwindOperation, lookupOperation1 );
}
AggregationResults<Article> resultList
= mongoTemplate.aggregate(aggregation, "articles", Article.class);
My entity Article.class
....
private List<ObjectId> tags = new ArrayList<>(); //array of tag ids
#Transient
private List<Tag> materialTags = new ArrayList<>(); // array of all tag details
//getters and setters
...
Tag.class
#Id private ObjectId id;
private String tagName;
...
If I use mongodb query, I am getting 'materialTags' array list with tag object filled in. Through Spring Data, I do get the exact same result. After unwind I get multiple rows (1 row per tag array entry) which is correct. Through debug point I can see resultList -> rawResult contain 'materialTags'. But resultList -> mappedResults -> 'materialTags' is empty.
Why Spring Data does not give me proper result inside Mapped Result? What am I doing wrong ? Please help.
Thanks in advance.
I'm using Spring Data MongoDB for my project. I work with a mongo database containing a lot of data, and I want to map this data within my Java application.
The problem I have is that some data back in time had a different structure.
For example sport_name is an array now, while in some old records is a String:
sport_name: "Soccer" // Old data
sport_name: [ // Most recent entries
{
"lang" : "en",
"val" : "Soccer"
},
{
"lang" : "de",
"val" : "Fussball"
}
]
Here is what I have until now:
#Document(collection = "matches")
public class MatchMongo {
#Id
private String id;
private ??? sport_name; // Best way?!
(What's the best way to)/(How would you) handle something like this?
If old data can be considered as "en" language, then separate structure can be used to store localized text:
class LocalName {
private String language;
private String text;
// getters/setters
}
So mapped object will store the collection of localized values:
public class MatchMongo {
// it can also be a map (language -> text),
// in this case we don't need additional structure
private List<LocalName> names;
}
This approach can be combined with custom converter, to use plain string as "en" locale value:
public class MatchReadConverter implements Converter<DBObject, MatchMongo> {
public Person convert(DBObject source) {
// check what kind of data located under "sport_name"
// and define it as "en" language text if it is an old plain text
// if "sport_name" is an array, then simply convert the values
}
}
Custom mapping is described here in details.
Probably you can write a utility class which will fetch all the data where sport_name is not an array and update the element sport_name to array. But this all depends on the amount of data you have.
You can use query {"sport_name":{$type:2}}, here 2 stands for String.
Refer for more details on $type: http://docs.mongodb.org/manual/reference/operator/query/type/