I try to use Mongodb with Spring Data.
I wanted to use UUID instead of ObjectId. I have followed this tutorial: https://www.baeldung.com/java-mongodb-uuid (some differences might exist because I use Kotlin). I took path 2 where I added Entity callback. When I create a new entity it is saved to the database with UUID as I wanted. If I use mongo console I can type:
db.home.find({_id: UUID("18aafcf9-0c5a-46f3-84ff-1c25b00dd1ab")})
And I will find my entity by id.
However, when I try to do it by code it doesn't work as it should. It will always throw here DataOperationException(NOT_FOUND) because findById returns null.
fun findHomeById(id: String): Home {
val home = homeRepository.findById(id)
return home.unwrap() ?: throw DataOperationException(NOT_FOUND)
}
Here is repository
#Repository
interface HomeRepository: MongoRepository<Home, String>
Abstraction with id.
abstract class UuidIdentifiedEntity {
#Id
var id: UUID? = null // I tried use UUID type and String with the same result
}
And my home class
class Home(
var address: String,
var rooms: Int,
): UuidIdentifiedEntity()
Not sure if this will help, but see if changing your #Id annotation to #MongoId fixes this. Under certain circumstances, Mongo needs to know more about a field being used for an id. #MongoId should give you more control on how the field is stored too.
What is use of #MongoId in Spring Data MongoDB over #Id?
I have a JPA Entity (Terminal) which uses an AttributeConverter to convert a Database String into a list of Objects (ProgrmRegistration). The converter just uses a JSON ObjectMapper to turn the JSON String into POJO objects.
Entity Object
#Entity
#Data
public class Terminal {
#Id
private String terminalId;
#NotEmpty
#Convert(converter = ProgramRegistrationConverter.class)
private List<ProgramRegistration> programRegistrations;
#Data
public static class ProgramRegistration {
private String program;
private boolean online;
}
}
The Terminal uses the following JPA AttributeConverter to serialize the Objects from and to JSON
JPA AttributeConverter
public class ProgramRegistrationConverter implements AttributeConverter<List<Terminal.ProgramRegistration>, String> {
private final ObjectMapper objectMapper;
private final CollectionType programRegistrationCollectionType;
public ProgramRegistrationConverter() {
this.objectMapper = new ObjectMapper().setSerializationInclusion(JsonInclude.Include.NON_EMPTY);
this.programRegistrationCollectionType =
objectMapper.getTypeFactory().constructCollectionType(List.class, Terminal.ProgramRegistration.class);
}
#Override
public String convertToDatabaseColumn(List<Terminal.ProgramRegistration> attribute) {
if (attribute == null) {
return null;
}
String json = null;
try {
json = objectMapper.writeValueAsString(attribute);
} catch (final JsonProcessingException e) {
LOG.error("JSON writing error", e);
}
return json;
}
#Override
public List<Terminal.ProgramRegistration> convertToEntityAttribute(String dbData) {
if (dbData == null) {
return Collections.emptyList();
}
List<Terminal.ProgramRegistration> list = null;
try {
list = objectMapper.readValue(dbData, programRegistrationCollectionType);
} catch (final IOException e) {
LOG.error("JSON reading error", e);
}
return list;
}
}
I am using Spring Boot and a JPARepository to fetch a Page of Terminal results from the Database.
To filter the results I am using a BooleanExpression as the Predicate. For all the filter values on the Entity it works well, but the List of objects converted from the JSON string does not allow me to easily write an Expression that will filter the Objects in the list.
REST API that is trying to filter the Entity Objects using QueryDSL
#GetMapping(path = "/filtered/page", produces = MediaType.APPLICATION_JSON_VALUE)
public Page<Terminal> findFilteredWithPage(
#RequestParam(required = false) String terminalId,
#RequestParam(required = false) String programName,
#PageableDefault(size = 20) #SortDefault.SortDefaults({ #SortDefault(sort = "terminalId") }) Pageable p) {
BooleanBuilder builder = new BooleanBuilder();
if (StringUtils.isNotBlank(terminalId))
builder.and(QTerminal.terminal.terminalId.upper()
.contains(StringUtils.upperCase(terminalId)));
// TODO: Figure out how to use QueryDsl to get the converted List as a predicate
// The code below to find the programRegistrations does not allow a call to any(),
// expects a CollectionExpression or a SubqueryExpression for calls to eqAny() or in()
if (StringUtils.isNotBlank(program))
builder.and(QTerminal.terminal.programRegistrations.any().name()
.contains(StringUtils.upperCase(programName)));
return terminalRepository.findAll(builder.getValue(), p);
}
I am wanting to get any Terminals that have a ProgramRegistration object with the program name equal to the parameter passed into the REST service.
I have been trying to get CollectionExpression or SubQueryExpression working without success since they all seem to be wanting to perform a join between two Entity objects. I do not know how to create the path and query so that it can iterate over the programRegistrations checking the "program" field for a match. I do not have a QProgamRegistration object to join with, since it is just a list of POJOs.
How can I get the predicate to match only the Terminals that have programs with the name I am searching for?
This is the line that is not working:
builder.and(QTerminal.terminal.programRegistrations.any().name()
.contains(StringUtils.upperCase(programName)));
AttributeConverters have issues in Querydsl, because they have issues in JPQL - the query language of JPA - itself. It is unclear what actually the underlying query type of the attribute is, and whether the parameter should be a basic type of that query type, or should be converted using the conversion. Such conversion, whilst it appears logical, is not defined in the JPA specification. Thus a basic type of the query type needs to be used instead, which leads to new difficulties, because Querydsl can't know the type it needs to be. It only knows the Java type of the attribute.
A workaround can be to force the field to result into a StringPath by annotating the field with #QueryType(PropertyType.STRING). Whilst this fixes the issue for some queries, you will run into different issues in other scenarios. For more information, see this thread.
Although the following desired QueryDsl looks like it should work
QTerminal.terminal.programRegistrations.any().name().contains(programName);
In reality JPA would never be able to convert it into something that would make sense in terms of SQL. The only SQL that JPA could convert it into could be as follows:
SELECT t.terminal_id FROM terminal t where t.terminal_id LIKE '%00%' and t.program_registrations like '%"program":"MY_PROGRAM_NAME"%';
This would work in this use case, but be semantically wrong, and therefore it is correct that it should not work. Trying to select unstructured data using a structured query language makes no sense
The only solution is to treat the data as characters for the DB search criteria, and to treat it as a list of Objects after the query completes and then perform filtering of the rows in Java. Although This makes the paging feature rather useless.
One possible solution is to have a secondary read only String version of the column that is used for the DB search criteria, that is not converted to JSON by the AttributeConverter.
#JsonIgnore
#Column(name = "programRegistrations", insertable = false, updatable = false)
private String programRegistrationsStr;
The real solution is do not use unstructured data when you want structured queries on that data Therefore convert the data to either a database that supports the JSON natively for queries or model the data correctly in DDL.
To have a short answer: the parameter used in the predicate on attribute with #QueryType must be used in another predicate on attribute of type String.
It's a clearly known issue describe in this thread: https://github.com/querydsl/querydsl/issues/2652
I simply want to share my experience about this bug.
Model
I have an entity like
#Entity
public class JobLog {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private String id;
#QueryType(PropertyType.STRING)
private LocalizedString message;
}
Issue
I want to perform some predicate about message. Unfortunately, with this configuration, I can't do this:
predicates.and(jobLog.message.likeIgnoreCase(escapedTextFilter));
because I have the same issues that all people!
Solution
But I find a way to workaround :)
predicates.and(
(jobLog.id.likeIgnoreCase(escapedTextFilter).and(jobLog.id.isNull()))
.or(jobLog.message.likeIgnoreCase(escapedTextFilter)));
Why it workaround the bug?
It's important that escapedTextFilter is the same in both predicate!
Indeed, in this case, the constant is converter to SQL in the first predicate (which is of String type). And in the second predicate, we use the conterted value
Bad thing?
Add a performance overflow because we have OR in predicate
Hope this can help someone :)
I've found one way to solve this problem, my main idea is to use mysql function cast(xx as char) to cheat hibrenate. Below is my base info. My code is for work , so I've made an example.
// StudentRepo.java
public interface StudentRepo<Student, Long> extends JpaRepository<Student, Long>, QuerydslPredicateExecutor<Student>, JpaSpecificationExecutor<Student> {
}
// Student.java
#Data
#AllArgsConstructor
#NoArgsConstructor
#EqualsAndHashCode(of = "id")
#Entity
#Builder
#Table(name = "student")
public class Student {
#Convert(converter = ClassIdsConvert.class)
private List<String> classIds;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
}
// ClassIdsConvert.java
public class ClassIdsConvert implements AttributeConverter<List<String>, String> {
#Override
public String convertToDatabaseColumn(List<String> ips) {
// classid23,classid24,classid25
return String.join(",", ips);
}
#Override
public List<String> convertToEntityAttribute(String dbData) {
if (StringUtils.isEmpty(dbData)) {
return null;
} else {
return Stream.of(dbData.split(",")).collect(Collectors.toList());
}
}
}
my db is below
id
classIds
name
address
1
2,3,4,11
join
北京市
2
2,31,14,11
hell
福建省
3
2,12,22,33
work
福建省
4
1,4,5,6
ouy
广东省
5
11,31,34,22
yup
上海市
-- ----------------------------
-- Table structure for student
-- ----------------------------
DROP TABLE IF EXISTS `student`;
CREATE TABLE `student` (
`id` int(11) NOT NULL,
`classIds` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`address` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
SET FOREIGN_KEY_CHECKS = 1;
Use JpaSpecificationExecutor solve the problem
Specification<Student> specification = (root, query, criteriaBuilder) -> {
String classId = "classid24"
String classIdStr = StringUtils.wrap(classId, "%");
var predicate = criteriaBuilder.like(root.get("classIds").as(String.class), classIdStr);
return criteriaBuilder.or(predicate);
};
var students = studentRepo.findAll(specification);
log.info(new Gson().toJson(students))
attention the code root.get("classIds").as(String.class)
In my opinion, if I don't add .as(String.class) , hibernate will think the type of student.classIds is list and throw an Exception as below.
SQL will like below which runs correctly in mysql. But hibnerate can't work.
org.springframework.dao.InvalidDataAccessApiUsageException: Parameter value [%classid24%] did not match expected type [java.util.List (n/a)]; nested exception is java.lang.IllegalArgumentException: Parameter value [%classid24%] did not match expected type [java.util.List (n/a)]
SELECT
student0_.id AS id1_0_,
student0_.class_ids AS class_ids2_0_
FROM
student student0_
WHERE
student0_.class_ids LIKE '%classid24%' ESCAPE '!'
if you add .as(String.class) , hibnerate will think the type of student.classIds as string and won't check it at all.
SQL will be like below which can run correct in mysql. Also in JPA.
SELECT
student0_.id AS id1_0_,
student0_.class_ids AS class_ids2_0_
FROM
student student0_
WHERE
cast( student0_.class_ids AS CHAR ) LIKE '%classid24%' ESCAPE '!'
when the problem is solved by JpaSpecificationExecutor, so I think this can be solve also in querydsl. At last I find the template idea in querydsl.
String classId = "classid24";
StringTemplate st = Expressions.stringTemplate("cast({0} as string)", qStudent.classIds);
var students = Lists.newArrayList<studentRepo.findAll(st.like(StringUtils.wrap(classId, "%"))));
log.info(new Gson().toJson(students));
it's sql is like below.
SELECT
student0_.id AS id1_0_,
student0_.class_ids AS class_ids2_0_
FROM
student student0_
WHERE
cast( student0_.class_ids AS CHAR ) LIKE '%classid24%' ESCAPE '!'
I have two collections called persons and addresses. The idea is to have person hold an address in the field address. I use Spring Data MongoDB to persist those mentioned documents.
My usual way of crafting the "relation" between Person > Address was to store the ID of the address and give it to the person object. Later when I find() a person I resolve the address object by it's id and voila I have my person + address.
However I find this somewhat every cumbersome since in my code I just want to add the Address object as whole and not only it's ID so I can work with it while also saving it to the repository at any point of time.
I therefore started a little unit test to see how Spring Data MongoDB saves the Address object if it's just a field of Person and is not saved by it's own Repository.
This is what I came up with:
import org.springframework.data.mongodb.core.mapping.Document
import org.springframework.data.mongodb.repository.MongoRepository
import org.springframework.stereotype.Repository
#Document("person")
data class Person(
val id: String,
val name: String,
val age: Int,
var address: Address
)
#Document("addresses")
data class Address(
val id: String,
val street: String?,
val number: Int?
)
#Repository
interface PersonRepository : MongoRepository<Person, String>
#Repository
interface AddressRepository : MongoRepository<Address, String>
And this is the unit test - that fails with the last steps as I was expecting:
internal class FooTest #Autowired constructor(
private val personRepository: PersonRepository,
private val addressRepository: AddressRepository
) {
#Test
fun `some experiment`() {
val testPerson = Person("001", "Peter", 25, Address("011","Lumberbumber", 12))
personRepository.save(testPerson)
val person = personRepository.findAll()[0]
assertThat(person).isNotNull
assertThat(person.address).isNotNull
assertThat(person.address.street).isEqualTo("Lumberbumber")
assertThat(person.address.number).isEqualTo(12)
// works because address was just copied into the object structure
// of `person` and was not seen as a standalone document
val address = addressRepository.findAll()[0]
assertThat(address.street).isEqualTo("Lumberbumber") // fails
assertThat(address.number).isEqualTo(12) // fails
// As expected `address` was not persisted alongside the `person` document.
}
}
So I thought about using AbstractMongoEventListener<Person> to intercept the saving process and pick the Address object out from Person here and do a addressRepository.save(addressDocument) while putting a lightweight address object (only having the ID) back in the Person document.
The same I'd to in the reverse when doing a find for Person and assembling Person and Address together again.
#Component
class MongoSaveInterceptor(
val addressRepository: AddressRepository
) : AbstractMongoEventListener<Person>() {
override fun onBeforeConvert(event: BeforeConvertEvent<Person>) {
val personToSave = event.source
val extractedAddress = personToSave.address
val idOfAddress = addressRepository.save(extractedAddress).id
personToSave.address = Address(idOfAddress, null, null)
}
override fun onAfterConvert(event: AfterConvertEvent<Person>) {
val person = event.source
val idOfAddress = person.address.id
val foundAddress = addressRepository.findById(idOfAddress)
foundAddress.ifPresent {
person.address = it
}
}
}
It works that way and might be a workaround solution for my requirement.
BUT
I feel that there has to be something like that already working and I might just need to find the proper configuration for that.
That's where I am stuck atm and need some guidance.
Another research showed me it's about #DBRef (https://www.baeldung.com/cascading-with-dbref-and-lifecycle-events-in-spring-data-mongodb) I have to use. This way Spring Data MongoDB stores the embedded document class and resolves it when loading the parent document object from the database.
I'm using spring data mongo. I have a collection within a document that when I add an item to it I would like to assign a new automatically generated unique identifier to it e.g. (someGeneratedId)
#Document(collection = "questionnaire")
public class Questionnaire {
#Id
private String id;
#Field("answers")
private List<Answer> answers;
}
public class Answer {
private String someGeneratedId;
private String text;
}
I am aware I could use UUID.randomUUID() (wrapped in some kind of service) and set the value, I was just wondering if there was anything out of the box that can handle this? From here #Id seems to be specific to _id field in mongo:
The #Id annotation tells the mapper which property you want to use for
the MongoDB _id property
TIA
No there is no out of the box solution for generating ids for properties on embedded documents.
If you want to keep this away from your business-logic you could implement a BeforeConvertCallback which generates the id's for your embedded objects.
#Component
class BeforeConvertQuestionnaireCallback implements BeforeConvertCallback<Questionnaire> {
#Override
public Questionnaire onBeforeConvert(#NonNull Questionnaire entity, #NonNull String collection) {
for (var answer : entity.getAnswers()) {
if (answer.getId() == null) {
answer.setId(new ObjectId().toString());
}
}
return entity;
}
}
You could also implement this in a more generic manner:
Create a new annotation: #AutogeneratedId.
Then listen to all BeforeConvertCallback's of all entities and iterate through the properties with reflection. Each property annotated with the new annotation gets a unique id if null.
I have a collection, with documents having a field named _id of type String, not generated manually.
I have been trying to get a document using its id.
val criteria = Criteria.where("_id").`is`("a2z3e44R")
val document = mongoTemplate.findOne(Query.query(criteria), MyDocument::class.java) // returns null
val criteria = Criteria.where("_id").`is`(ObjectId("a2z3e44R"))
val document = mongoTemplate.findOne(Query.query(criteria), MyDocument::class.java) // returns null
val document = mongoTemplate.findById("a2z3e44R", MyDocument::class.java) // returns null
mongoTemplate.findAll(MyDocument::class.java).first { myDocument ->
myDocument._id == "a2z3e44R"
} // OK...
MyDocument is
data class MyDocument(val _id: String, val name: String)
Trying to find a document by another field works.
An idea of what I could be missing or a workaround?
You should indicate the type of the id like this
public class Article {
#MongoId(value = FieldType.OBJECT_ID)
private String id;
private String title;
private String desc;
}
Try mark _id with annotation #Id. The #Id annotation is used to specify the identifier for Spring.
data class MyDocument(#Id val _id: String, val name: String)
Ypu could define in your repository:
public interface MyDocumentRepository extends MongoRepository<MyDocument, String> {
Pets findBy_id(ObjectId _id);
}
and use it :
myDocumentRepository.findBy_id("a2z3e44R");
for more info see
or
ObjectId objID = new ObjectId("a2z3e44R");
query.addCriteria(Criteria.where("_id").lt(objID));
like this other answer link