I have 3 different entities: Partner is main entity, Offer is partner's offer (many to one) and Location is any location in partner.
#Entity
class ObjectLocation(#ManyToOne var place: Place, var partnerId: String) {
constructor() : this(Place(), "")
#Id
var id: String = IDGenerator.longId()
// other fields omitted
...
}
#Entity
class Offer(var partnerId: String, ...) {
constructor() : this(...)
#Id
var id: String = IDGenerator.longId()
...
}
#Entity
class Partner(...) {
constructor() : this(...)
#Id
var id: String = IDGenerator.longId()
...
}
So, I need to find all the Offers by the Place criteria. I've tried this:
Specification {
root: Root<Offer>, criteriaQuery: CriteriaQuery<*>, criteriaBuilder: CriteriaBuilder ->
val objectLocationRoot = criteriaQuery.from(ObjectLocation::class.java)
val objectCityId: Expression<String> = objectLocationRoot
.get<Place>("place")
.get<City>("parentCity")
.get<String>("id")
val objectPartnerId: Expression<String> = objectLocationRoot.get<String>("partnerId")
val offerPartnerId: Expression<String> = root.get<String>("partnerId")
val goodLocations: Predicate = criteriaBuilder.equal(objectCityId, cityId)
val objQuery: Subquery<String> = criteriaQuery.subquery(String::class.java)
.select(objectPartnerId)
.where(goodLocations)
return#Specification criteriaBuilder.equal(objQuery, offerPartnerId)
}
But this only gave me following exception:
antlr.NoViableAltException: unexpected token: where
at org.hibernate.hql.internal.antlr.HqlBaseParser.fromRange(HqlBaseParser.java:1519) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
at org.hibernate.hql.internal.antlr.HqlBaseParser.fromClause(HqlBaseParser.java:1343) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
at org.hibernate.hql.internal.antlr.HqlBaseParser.selectFrom(HqlBaseParser.java:1063) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
at org.hibernate.hql.internal.antlr.HqlBaseParser.queryRule(HqlBaseParser.java:748) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
at org.hibernate.hql.internal.antlr.HqlBaseParser.subQuery(HqlBaseParser.java:3910) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
at org.hibernate.hql.internal.antlr.HqlBaseParser.primaryExpression(HqlBaseParser.java:967) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
at org.hibernate.hql.internal.antlr.HqlBaseParser.atom(HqlBaseParser.java:3549) [hibernate-core-5.2.17.Final.jar:5.2.17.Final]
....
org.springframework.dao.InvalidDataAccessApiUsageException: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: where near line 1, column 175
[select generatedAlias0 from com.arkell.entity.Offer as generatedAlias0, com.arkell.entity.geo.ObjectLocation as generatedAlias1 where
(select generatedAlias1.partnerId from where generatedAlias1.place.parentCity.id=:param0)=generatedAlias0.partnerId]; nested exception is java.lang.IllegalArgumentException:
org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token: where near line 1, column 175 [select generatedAlias0
from com.arkell.entity.Offer as generatedAlias0, com.arkell.entity.geo.ObjectLocation as generatedAlias1 where
(select generatedAlias1.partnerId from where generatedAlias1.place.parentCity.id=:param0)=generatedAlias0.partnerId]
at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:367)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:227)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:527)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:153)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:135)
How it is possible to find all offers by their partner's location cities?
Oh wow, that was very easy.
Specification {
root: Root<Offer>, criteriaQuery: CriteriaQuery<*>, criteriaBuilder: CriteriaBuilder ->
val objectLocationRoot: Root<ObjectLocation> = criteriaQuery.distinct(true).from(ObjectLocation::class.java)
return#Specification criteriaBuilder.and(
criteriaBuilder.equal(root.get<String>("partnerId"), objectLocationRoot.get<String>("partnerId")),
criteriaBuilder.equal(objectLocationRoot.get<Place>("place")
.get<City>("parentCity")
.get<String>("id")("streetType"), cityId)
)
}
Related
I want to write integration test for my microservice currently using Kotlin, Jooq and R2dbc at repository level.
I want my test to work in R2dbc mode as well, but for some reason getting this exception:
Caused by: org.testcontainers.containers.JdbcDatabaseContainer$NoDriverFoundException: Could not get Driver
at org.testcontainers.containers.JdbcDatabaseContainer.getJdbcDriverInstance(JdbcDatabaseContainer.java:187)
at org.testcontainers.containers.JdbcDatabaseContainer.createConnection(JdbcDatabaseContainer.java:209)
at org.testcontainers.containers.JdbcDatabaseContainer.waitUntilContainerStarted(JdbcDatabaseContainer.java:147)
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:466)
... 10 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
Probably, I have to point somewhere that I want to use r2dbc only, not jdbc? I've seen the specs but not sure whether I applied TC_INITSCRIPT and TC_IMAGE_TAG correctly.
I don't use Spring Data r2dbc (jooq only), that's why ResourceDatabasePopulator is not an option for me.
My test looks like:
#SpringBootTest(classes = [UserServiceApp::class])
#ActiveProfiles(profiles = ["test"])
#AutoConfigureWebTestClient
class UserServiceAppIT(#Autowired val client: WebTestClient) {
#Nested
inner class Find {
#Test
#DisplayName("Find existing user by id")
fun `existing user credentials returns OK`() {
val expectedUser = getCredentialsUser() //this is a class with expected data
val response = client.get()
.uri("/user/2") //this is my endpoint
.accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus().isOk
.expectBody(UserCredentialsModel::class.java)
.returnResult()
.responseBody
assertThat(response)
.isNotNull
.isEqualTo(expectedUser)
}
}
Test config in yaml file:
server.port: 8080
spring:
application:
name: User Service Test
r2dbc:
url: r2dbc:tc:mysql:///pharmacy?TC_IMAGE_TAG=8.0.26&TC_INITSCRIPT=classpath/resources/init.sql
password: root
username: root
pool:
initial-size: 1
max-size: 10
max-idle-time: 30m
Dependencies (gradle):
buildscript {
ext {
springDependencyVersion = '1.0.11.RELEASE'
springBootVersion = '2.5.3'
kotlinVersion = '1.5.0'
jooqPluginVersion = '6.0'
springdocVersion = '1.5.10'
r2dbcMySQLVersion = '0.8.2.RELEASE'
r2dbcPoolVersion = '0.8.7.RELEASE'
mockKVersion = '1.12.0'
kotestVersion = '4.4.3'
kotlinJsonVersion = '1.2.1'
kotlinDateVersion = '0.2.1'
testcontainersVersion = '1.16.0'
}
}
It is easy to integrate Jooq with R2dbc.
#Configuration
class JooqConfig {
#Bean
fun dslContext(connectionFactory: ConnectionFactory) =
using(TransactionAwareConnectionFactoryProxy(connectionFactory), SQLDialect.POSTGRES)
}
NOTE: Do not include Jooq starter if you are using Spring 2.7.x. The Jooq autoconfiguration only supports Jdbc.
An example using Jooq.
class PostRepositoryImpl(private val dslContext: DSLContext) : PostRepositoryCustom {
override fun findByKeyword(title: String): Flow<PostSummary> {
val sql = dslContext
.select(
POSTS.ID,
POSTS.TITLE,
field("count(comments.id)", SQLDataType.BIGINT)
)
.from(
POSTS
.leftJoin(COMMENTS.`as`("comments"))
.on(COMMENTS.POST_ID.eq(POSTS.ID))
)
.where(
POSTS.TITLE.like("%$title%")
.and(POSTS.CONTENT.like("%$title%"))
.and(COMMENTS.CONTENT.like("%$title%"))
)
.groupBy(POSTS.ID)
return Flux.from(sql)
.map { r -> PostSummary(r.value1(), r.value2(), r.value3()) }
.asFlow();
}
override suspend fun countByKeyword(title: String): Long {
val sql = dslContext
.select(
DSL.field("count(distinct(posts.id))", SQLDataType.BIGINT)
)
.from(
POSTS
.leftJoin(COMMENTS.`as`("comments"))
.on(COMMENTS.POST_ID.eq(POSTS.ID))
)
.where(
POSTS.TITLE.like("%$title%")
.and(POSTS.CONTENT.like("%$title%"))
.and(COMMENTS.CONTENT.like("%$title%"))
)
return Mono.from(sql).map { it.value1() ?: 0 }.awaitSingle()
}
}
TestContainers database requires a Jdbc driver, add MySQL Jdbc driver with testcontainter into your test scope.
The following is an example using Postgres and Testcontainers.
#OptIn(ExperimentalCoroutinesApi::class)
#Testcontainers
#DataR2dbcTest()
#Import(JooqConfig::class, R2dbcConfig::class)
class PostRepositoriesTest {
companion object {
private val log = LoggerFactory.getLogger(PostRepositoriesTest::class.java)
#Container
val postgreSQLContainer = PostgreSQLContainer("postgres:12")
.withCopyFileToContainer(
MountableFile.forClasspathResource("/init.sql"),
"/docker-entrypoint-initdb.d/init.sql"
)
#JvmStatic
#DynamicPropertySource
fun registerDynamicProperties(registry: DynamicPropertyRegistry) {
registry.add("spring.r2dbc.url") {
"r2dbc:postgresql://${postgreSQLContainer.host}:${postgreSQLContainer.firstMappedPort}/${postgreSQLContainer.databaseName}"
}
registry.add("spring.r2dbc.username") { postgreSQLContainer.username }
registry.add("spring.r2dbc.password") { postgreSQLContainer.password }
}
}
#Autowired
lateinit var postRepository: PostRepository
#Autowired
lateinit var dslContext: DSLContext
#BeforeEach
fun setup() = runTest {
log.info(" clear sample data ...")
val deletedPostsCount = Mono.from(dslContext.deleteFrom(POSTS)).awaitSingle()
log.debug(" deletedPostsCount: $deletedPostsCount")
}
#Test
fun `query sample data`() = runTest {
log.debug(" add new sample data...")
val insertPostSql = dslContext.insertInto(POSTS)
.columns(POSTS.TITLE, POSTS.CONTENT)
.values("jooq test", "content of Jooq test")
.returningResult(POSTS.ID)
val postId = Mono.from(insertPostSql).awaitSingle()
log.debug(" postId: $postId")
val insertCommentSql = dslContext.insertInto(COMMENTS)
.columns(COMMENTS.POST_ID, COMMENTS.CONTENT)
.values(postId.component1(), "test comments")
.values(postId.component1(), "test comments 2")
val insertedCount = Mono.from(insertCommentSql).awaitSingle()
log.info(" insertedCount: $insertedCount")
val querySQL = dslContext
.select(
POSTS.TITLE,
POSTS.CONTENT,
multiset(
select(COMMENTS.CONTENT)
.from(COMMENTS)
.where(COMMENTS.POST_ID.eq(POSTS.ID))
).`as`("comments")
)
.from(POSTS)
.orderBy(POSTS.CREATED_AT)
Flux.from(querySQL).asFlow()
.onEach { log.info("querySQL result: $it") }
.collect()
val posts = postRepository.findByKeyword("test").toList()
posts shouldNotBe null
posts.size shouldBe 1
posts[0].commentsCount shouldBe 2
postRepository.countByKeyword("test") shouldBe 1
}
// other tests
My example project is based Postgres, R2dbc, And Spring Data R2dbc: https://github.com/hantsy/spring-r2dbc-sample/blob/master/jooq-kotlin-co-gradle
I use Spring Boot and QueryDSL to combine a sql query and predicate. The problem is that I have to create a predicate to fetch data from table based on email BUT:
email can be in Freight.sender.email
OR in Freight.message.senderAddress
where Freight, Message, Sender are of course tables.
In table Freight we can have empty sender_id or message_id and depending on this I have to fetch rows by email from Freight.sender.email OR Freight.message.senderAddress (if Freight.sender is null)
Is it possible to create such a predicate that compares email from request query param with Freight.sender.email and only if Freight.sender.email doesn't exist, then my predicate shold search email in Freight.message.senderAddress
public Predicate build(Map<String, String> filters) {
return new OptionalBooleanBuilder(Expressions.asBoolean(true).isTrue())
.notNullAnd(qFreight.loadingAddress::containsIgnoreCase, filters.get(LOADING_ADDRESS))
.notNullAnd(qFreight.unloadingAddress::containsIgnoreCase, filters.get(UNLOADING_ADDRESS))
.notNullAnd(qFreight.loadingDate.eq(filters.get(LOADING_DATE) != null ? LocalDate.parse(filters.get(LOADING_DATE)) : now()), filters.get(LOADING_DATE))
.notNullAnd(qFreight.unloadingDate.eq(filters.get(UNLOADING_DATE) != null ? LocalDate.parse(filters.get(UNLOADING_DATE)) : now()), filters.get(UNLOADING_DATE))
//MY ATTEMPT - NOT WORKING:
.notNullAnd(qFreight.sender.email.eq(filters.get(SENDER_EMAIL)).or(qFreight.emailMessage.senderAddress.eq(SENDER_EMAIL)), filters.get(SENDER_EMAIL))
.build();
}
public class OptionalBooleanBuilder {
private BooleanExpression predicate;
public OptionalBooleanBuilder(BooleanExpression predicate) {
this.predicate = predicate;
}
public <T> OptionalBooleanBuilder notNullAnd(Function<T, BooleanExpression> expressionFunction, T value) {
if (nonNull(value)) {
return new OptionalBooleanBuilder(predicate.and(expressionFunction.apply(value)));
}
return this;
}
public BooleanExpression build() {
return predicate;
}
public <T>OptionalBooleanBuilder notNullAnd(BooleanExpression expression, T value) {
if(nonNull(value)){
return new OptionalBooleanBuilder(predicate.and(expression));
}
return this;
}
}
UPDATE
After suggestion from:
private Predicate addPredicate(OptionalBooleanBuilder builder, String email) {
if (nonNull(email)) {
return builder.notNullAnd(qFreight.sender.email.coalesce(qFreight.emailMessage.senderAddress.eq(email)).asBoolean(), email).build();
return builder.build();
}
I get error:
antlr.NoViableAltException: unexpected AST node: (
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.logicalExpr(HqlSqlBaseWalker.java:2169) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.logicalExpr(HqlSqlBaseWalker.java:2089) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.whereClause(HqlSqlBaseWalker.java:827) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.query(HqlSqlBaseWalker.java:621) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.selectStatement(HqlSqlBaseWalker.java:325) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.statement(HqlSqlBaseWalker.java:273) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.analyze(QueryTranslatorImpl.java:276) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:192) ~[hibernate-core-5.4.17.Final.jar:5.4.17.Final]
2020-12-24 00:45:35.535 ERROR 16068 --- [nio-9090-exec-8] p.a.m.s.filter.JwtAuthorizationFilter : Request processing failed; nested exception is org.springframework.dao.InvalidDataAccessApiUsageException: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected AST node: ( near line 3, column 52 [select freight
from pl.appwise.mtf.freight.domain.model.Freight freight
where ?1 = ?1 and freight.user.id = ?2 and coalesce(freight.sender.email, freight.emailMessage.senderAddress = ?3)
order by freight.loadingDate desc]; nested exception is java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected AST node: ( near line 3, column 52 [select freight
from pl.appwise.mtf.freight.domain.model.Freight freight
where ?1 = ?1 and freight.user.id = ?2 and coalesce(freight.sender.email, freight.emailMessage.senderAddress = ?3)
order by freight.loadingDate desc]
2020-12-24 00:45:37.906 INFO 16068 --- [ scheduling-1] ilAccoun
Use Freight.sender.email.coalesce(Freight.message.senderAddress). Freight.message.senderAddress will be NULL if Freight.message is NULL. For optional associations left outer joins are used by default, so this should cause no issue. Otherwise, explicitly use a left join yourself.
Goal:
I want to create a parent-child index with 2 entities. A profile and a comment. A profile (for simplicity) has a custom id (UUID converted to a string), age, and location (GeoPoint). A comment (for simplicity) has a custom id (UUID converted to a string). With this information, I want to be able to search for all comments given some filtering data against a profile. For example, I want to find all comments by profiles between the ages of 26 and 36 and is located within 100km of lat: 3.0, long 5.0.
Classes:
// Profile.kt
import org.elasticsearch.common.geo.GeoPoint
import org.springframework.data.annotation.Id
import org.springframework.data.elasticsearch.annotations.Document
import org.springframework.data.elasticsearch.annotations.Field
import org.springframework.data.elasticsearch.annotations.FieldType
import org.springframework.data.elasticsearch.annotations.GeoPointField
#Document(indexName = "message_board", createIndex = false, type = "profile")
data class Profile(
#Id
val profileId: String,
#Field(type = FieldType.Short, store = true)
val age: Short,
#GeoPointField
val location: GeoPoint
)
// Comment.kt
import org.springframework.data.annotation.Id
import org.springframework.data.elasticsearch.annotations.Document
import org.springframework.data.elasticsearch.annotations.Field
import org.springframework.data.elasticsearch.annotations.FieldType
import org.springframework.data.elasticsearch.annotations.Parent
#Document(indexName = "message_board", createIndex = false, type = "comment")
data class Comment(
#Id
val commentId: String,
#Field(type = FieldType.Text, store = true)
#Parent(type = "profile")
val parentId: String
)
// RestClientConfig.kt
import org.elasticsearch.client.RestHighLevelClient
import org.springframework.context.annotation.Configuration
import org.springframework.data.elasticsearch.client.ClientConfiguration
import org.springframework.data.elasticsearch.client.RestClients
import org.springframework.data.elasticsearch.config.AbstractElasticsearchConfiguration
#Configuration
class RestClientConfig(
private val elasticSearchConfig: ElasticSearchConfig
) : AbstractElasticsearchConfiguration() {
override fun elasticsearchClient(): RestHighLevelClient {
val clientConfiguration: ClientConfiguration = ClientConfiguration.builder()
.connectedTo("${elasticSearchConfig.endpoint}:${elasticSearchConfig.port}")
.build()
return RestClients.create(clientConfiguration).rest()
}
}
// Controller.kt
import org.springframework.web.bind.annotation.RestController
import org.springframework.data.elasticsearch.core.ElasticsearchOperations
import org.springframework.data.elasticsearch.core.mapping.IndexCoordinates
import org.springframework.http.MediaType
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestMapping
#RestController
#RequestMapping("/", produces = [MediaType.APPLICATION_JSON_VALUE])
class Controller constructor(
private val elasticsearchOperations: ElasticsearchOperations
) {
init {
elasticsearchOperations.indexOps(IndexCoordinates.of("message_board")).let { indexOp ->
if (!indexOp.exists() && indexOp.create()) {
val profileMapping = indexOp.createMapping(Profile::class.java)
println("Profile Mapping: $profileMapping")
indexOp.putMapping(profileMapping)
val commentMapping = indexOp.createMapping(Comment::class.java)
println("Comment Mapping: $commentMapping")
indexOp.putMapping(commentMapping)
indexOp.refresh()
}
}
}
#GetMapping("comments")
fun getComments(): List<Comment> {
val searchQuery = NativeSearchQueryBuilder()
.withFilter(
HasParentQueryBuilder(
"profile",
QueryBuilders
.boolQuery()
.must(
QueryBuilders
.geoDistanceQuery("location")
.distance(100, DistanceUnit.KILOMETERS)
.point(3.0, 5.0)
)
.must(
QueryBuilders
.rangeQuery("age")
.gte(26)
.lte(36)
),
false
)
)
.build()
return elasticsearchOperations.search(searchQuery, Comment::class.java, IndexCoordinates.of("message_board")).toList().map(SearchHit<Comment>::getContent)
}
}
My Setup:
I have elasticsearch running in docker via:
docker run --name es -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -d -v es_data:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:7.4.2
Spring Boot: "2.3.2.RELEASE"
Spring Data Elasticsearch: "4.0.2.RELEASE"
Issues:
I'm failing to get past the init block of my controller with the following exception:
Profile Mapping: MapDocument#?#? {"properties":{"age":{"store":true,"type":"short"},"location":{"type":"geo_point"}}}
Comment Mapping: MapDocument#?#? {"_parent":{"type":"profile"},"properties":{"parentId":{"store":true,"type":"text"}}}
Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://localhost:9200], URI [/message_board/_mapping?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
Caused by: org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=mapper_parsing_exception, reason=Root mapping definition has unsupported parameters: [_parent : {type=profile}]]
I need a solution that doesn't involve making a direct POST request to ES. Ideally, this is solved with the Elasticsearch client API. It feels like there's something missing with my annotations on the data classes however I couldn't find any documentation about this.
This is no problem of using the REST API. These calls are created by the Elasticsearch RestHighlevelClient.
Having multiple types in one index is not supported anymore by Elasticsearch since version 7.0.0. So you cannot model your data in this way.
Elasticsearch supports the join data type for this. We are currently working on a PR that will add the support for this for the next version of Spring Data Elasticsearch (4.1).
I was able to find a short-term solution with the following changes.
// Comment.kt
#Document(indexName = "message_board")
data class Comment(
#Id
val commentId: String,
val relationField: Map<String, String>
)
// Profile.kt
#Document(indexName = "message_board")
data class Profile(
#Id
val profileId: String,
#Field(type = FieldType.Short)
val age: Short,
#GeoPointField
val location: GeoPoint,
#Field(type = FieldType.Text)
val relationField: String = "profile"
)
// Controller.kt
class Controller constructor(
private val elasticsearchOperations: ElasticsearchOperations
) {
init {
elasticsearchOperations.indexOps(IndexCoordinates.of("message_board")).let { indexOp ->
if (!indexOp.exists() && indexOp.create()) {
val relationMap = Document.from(
mapOf(
"properties" to mapOf(
"relationField" to mapOf(
"type" to "join",
"relations" to mapOf(
"profile" to "comment"
)
),
"location" to mapOf(
"type" to "geo_point"
)
)
)
)
indexOp.putMapping(relationMap)
indexOp.refresh()
}
}
}
}
Note the relationField added to both data classes as well as the manually generated mapping document. Now, ES has a proper mapping on initialization:
{
"message_board" : {
"mappings" : {
"properties" : {
"location" : {
"type" : "geo_point"
},
"relationField" : {
"type" : "join",
"eager_global_ordinals" : true,
"relations" : {
"profile" : "comment"
}
}
}
}
}
}
Now, creating a profile is straightforward:
val profile = Profile(
profileId = UUID.randomUUID().toString(),
age = 27,
location = GeoPoint(3.0, 5.0)
)
val indexQuery = IndexQueryBuilder()
.withId(profile.profileId)
.withObject(profile)
.build()
elasticsearchOperations.index(indexQuery, IndexCoordinates.of("message_board"))
However, creating a comment is a little tricker because a routing ID is required:
val comment = Comment(
commentId = UUID.randomUUID().toString(),
relationField = mapOf(
"name" to "comment",
"parent" to profileId
)
)
val bulkOptions = BulkOptions.builder()
.withRoutingId(profileId)
.build()
val indexQuery = IndexQueryBuilder()
.withId(comment.commentId)
.withObject(comment)
.withParentId(profileId)
.build()
elasticsearchOperations.bulkIndex(listOf(indexQuery), bulkOptions, IndexCoordinates.of("message_board"))
This is how I was able to get a parent-child relation with the new JOIN relation type.
I am new to spring boot and spring data jpa. I am trying to use native queries for executing search based on search attributes received from UI.
The records that are obtained based on the searchParam should search if the searchParam is contained in any of the specified columns (as mentioned in the native query)
I have written the following code but I end up receiving the error as mentioned in the title. I have tried looking up for response in stackoverflow. But i believe i have followed the suggestions as mentioned in many of the threads.
Any help in this regard would be highly appreciated.
Code snippet below
EpicController.java
#CrossOrigin
#RequestMapping(value="/search", method = RequestMethod.GET)
public Page<Epic> searchEpicsByProjectIdAndSearchParam(#RequestParam String searchParam, #RequestParam String projectId, Pageable pageable) throws Exception {
logger.info("Inside searchEpicsByAttributes() based on searchQuery API");
Page<Epic> results = null;
try {
results = epicService.searchEpicsByProjectIdAndSearchParam(searchParam, projectId, pageable);
}
catch(Exception ex) {
ex.printStackTrace();
throw new Exception("Exception occurred :: " + ex.getStackTrace());
}
return results;
}
EpicService.java (Interface)
public interface EpicService {
Page<Epic> searchEpicsByProjectIdAndSearchParam(String searchParam, String projectId, Pageable pageable);
}
EpicServiceImpl.java
#Override
public Page<Epic> searchEpicsByProjectIdAndSearchParam(String searchParam, String projectId, Pageable pageable) {
logger.info(" Inside searchEpicsByProjectIdAndSearchParam() API in EpicServiceImpl");
return epicRepository.findBySearchParamsAndProjectId(searchParam,projectId, pageable);
}
EpicRepository.java
#Repository
public interface EpicRepository extends JpaRepository<Issue, String> {
#Query(value =
"select i.* from issue i where ("
+ "upper(i.name) like upper('%?1%'))"
+ "and upper(i.project_id) = upper('%?2%')"
+ "ORDER BY i.name DESC \n-- #pageable\n",
countQuery =
"select count(i.*) from issue i where ("
+ "upper(i.name) like upper('%?1%'))"
+ "and upper(i.project_id) = upper('%?2%')",
nativeQuery = true)
Page<Epic> findBySearchParamsAndProjectId(String name, String projectId, Pageable pageable);
}
Exception:
2019-02-08 23:25:21.199 INFO 12556 --- [nio-8080-exec-1] c.a.m.A.controller.ProjectController : Inside searchEpicsByProjectIdAndSearchParam() API in EpicServiceImpl
org.springframework.dao.InvalidDataAccessApiUsageException: Parameter with that position [1] did not exist; nested exception is java.lang.IllegalArgumentException: Parameter with that position [1] did not exist
at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:384)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:246)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:525)
at org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:59)
at org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:209)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:147)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:133)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
Caused by: java.lang.IllegalArgumentException: Parameter with that position [1] did not exist
at org.hibernate.jpa.spi.BaseQueryImpl.findParameterRegistration(BaseQueryImpl.java:502)
at org.hibernate.jpa.spi.BaseQueryImpl.setParameter(BaseQueryImpl.java:692)
at org.hibernate.jpa.spi.AbstractQueryImpl.setParameter(AbstractQueryImpl.java:181)
at org.hibernate.jpa.spi.AbstractQueryImpl.setParameter(AbstractQueryImpl.java:32)
at org.springframework.data.jpa.repository.query.ParameterBinder.bind(ParameterBinder.java:141)
at org.springframework.data.jpa.repository.query.StringQueryParameterBinder.bind(StringQueryParameterBinder.java:61)
at org.springframework.data.jpa.repository.query.ParameterBinder.bind(ParameterBinder.java:101)
at org.springframework.data.jpa.repository.query.SpelExpressionStringQueryParameterBinder.bind(SpelExpressionStringQueryParameterBinder.java:76)
at org.springframework.data.jpa.repository.query.ParameterBinder.bindAndPrepare(ParameterBinder.java:161)
at org.springframework.data.jpa.repository.query.ParameterBinder.bindAndPrepare(ParameterBinder.java:152)
at org.springframework.data.jpa.repository.query.AbstractStringBasedJpaQuery.doCreateQuery(AbstractStringBasedJpaQuery.java:81)
at org.springframework.data.jpa.repository.query.AbstractJpaQuery.createQuery(AbstractJpaQuery.java:202)
at org.springframework.data.jpa.repository.query.JpaQueryExecution$PagedExecution.doExecute(JpaQueryExecution.java:188)
Skip Single Quotations "'" around params i.e. ?1 and ?2. Working query will be like:
"select i.* from issue i where ("
+ "upper(i.name) like upper(%?1%))"
+ "and upper(i.project_id) = upper(%?2%)"
+ "ORDER BY i.name DESC \n-- #pageable\n",
countQuery =
"select count(i.*) from issue i where ("
+ "upper(i.name) like upper(%?1%))"
+ "and upper(i.project_id) = upper(%?2%)"
I'm attempting to pull data from a SOAP service that I have no control over. The hierarchy contains ProductOrder -> ShipTo -> Item where there are one or more shipToes and one or more Items per shipto.
Their API uses a mock SQL like query language. I'm getting stack traces like the following when trying to pull data including the items. if I exclude item, I'm able to pull the ProductOrders along with ShipTo objects, but items is always an empty list.
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at
org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:779)
~[spring-boot-1.5.0.RELEASE.jar:1.5.0.RELEASE] at
org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:760)
~[spring-boot-1.5.0.RELEASE.jar:1.5.0.RELEASE] at
org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:747)
~[spring-boot-1.5.0.RELEASE.jar:1.5.0.RELEASE] at
org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
~[spring-boot-1.5.0.RELEASE.jar:1.5.0.RELEASE] at
edu.umich.oud.giftformatter.convioexport.Application.main(Application.java:39)
~[classes/:na] Caused by:
org.springframework.oxm.UncategorizedMappingException: Unknown JAXB
exception; nested exception is javax.xml.bind.JAXBException: Field
order for ShipTo.Item.ItemId does not match the schema definition for
record type ProductOrder at
org.springframework.oxm.jaxb.Jaxb2Marshaller.convertJaxbException(Jaxb2Marshaller.java:915)
~[spring-oxm-4.3.6.RELEASE.jar:4.3.6.RELEASE] at
edu.umich.oud.giftformatter.convioexport.CustJaxbUnMarshaller.unmarshal(CustJaxbUnMarshaller.java:37)
~[classes/:na] at
org.springframework.ws.support.MarshallingUtils.unmarshal(MarshallingUtils.java:62)
~[spring-ws-core-2.4.0.RELEASE.jar:2.4.0.RELEASE] at
org.springframework.ws.client.core.WebServiceTemplate$3.extractData(WebServiceTemplate.java:413)
~[spring-ws-core-2.4.0.RELEASE.jar:2.4.0.RELEASE] at
org.springframework.ws.client.core.WebServiceTemplate.doSendAndReceive(WebServiceTemplate.java:619)
~[spring-ws-core-2.4.0.RELEASE.jar:2.4.0.RELEASE] at
org.springframework.ws.client.core.WebServiceTemplate.sendAndReceive(WebServiceTemplate.java:555)
~[spring-ws-core-2.4.0.RELEASE.jar:2.4.0.RELEASE] at
org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:390)
~[spring-ws-core-2.4.0.RELEASE.jar:2.4.0.RELEASE] at
org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:383)
~[spring-ws-core-2.4.0.RELEASE.jar:2.4.0.RELEASE] at
edu.umich.oud.giftformatter.convioexport.services.ConvioClient.queryInternal(ConvioClient.java:159)
~[classes/:na] at
edu.umich.oud.giftformatter.convioexport.services.ConvioClient.query(ConvioClient.java:134)
~[classes/:na] at
edu.umich.oud.giftformatter.convioexport.services.ProductOrderService.getProductOrders(ProductOrderService.java:87)
~[classes/:na] at
edu.umich.oud.giftformatter.convioexport.services.ConvioService.load(ConvioService.java:82)
~[classes/:na] at
edu.umich.oud.giftformatter.convioexport.Application.lambda$runner$0(Application.java:72)
~[classes/:na] at
org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:776)
~[spring-boot-1.5.0.RELEASE.jar:1.5.0.RELEASE] ... 4 common frames
omitted Caused by: javax.xml.bind.JAXBException: Field order for
ShipTo.Item.ItemId does not match the schema definition for record
type ProductOrder ... 17 common frames omitted
The product order service contains a method like so:
public List<ProductOrderObj> getProductOrders(final Date startDate, final Date endDate) {
final String query = String.format("SELECT siteId,orderId,transactionId,purchaseAmount,taxDeductibleValue,\n" +
"shippingCharge,additionalDonation,discountAmount,discountCode,\n" +
"creationDate,createdBy,modifyDate,lastChangeBy,storeId,payment,\n" +
"purchaser,interactionSource,shipTo,\n" +
"receiptNumber,shipTo.item FROM ProductOrder where creationDate > %s and creationDate < %s",
convertDate(startDate), convertDate(endDate));
log.info("query is " + query);
final Session session = convioClient.startSession();
final ArrayList<ProductOrderObj> events = new ArrayList<>();
for (int page = 1; page < 100; page++) {
final List<? extends RecordObj> items = convioClient.query(session, page, ConvioConfiguration.MAX_DOWNLOADS_PER_REQUEST, query);
if (items.size() < ConvioConfiguration.MAX_DOWNLOADS_PER_REQUEST) {
events.addAll((List<ProductOrderObj>) items);
break;
}
events.addAll((List<ProductOrderObj>) items);
}
return events;
}
Which in turn calls the convioService.query method that effectively does this
private List<? extends RecordObj> queryInternal(final Session session, final
int page, final int pageSize, final String q) {
// setup query
final Query query = new Query();
query.setPage(BigInteger.valueOf(page));
query.setPageSize(BigInteger.valueOf(pageSize));
query.setQueryString(q);
log.trace(q);
// perform query
try {
final Object obj = getWebServiceTemplate().marshalSendAndReceive(query,
new SoapActionExecutionIdCallback(session));
final QueryResponse response = (QueryResponse) obj;
if (response != null) {
log.debug("Response was a " + response.getClass().getName());
return response.getRecord();
}
} catch (final Exception e) {
log.error(e.getMessage());
throw e;
}
throw new NullPointerException("response was null");
}
There seemed to be two issues causing this not to work:
Bad field definition for the child object. shipTo.items vs shipTo.Items
Disabling validation of the dtd in the marshaller/unmarshaller