I have a table that contains entities with a String id, String jobId, and String status. Given a jobId and a List of ids, I would like to query that table and return a Flux of ids that are not present in the database.
I can do this successfully if I manually execute the following query in pgadmin:
SELECT a.id FROM (VALUES ('20191001_182447_1038'),('abc'),('fdjk')) AS a(id) LEFT JOIN (SELECT * FROM items WHERE job_id = '10a7a04a-aa67-499a-83eb-0cd3625fe27a') b ON a.id = b.id WHERE b.id IS null
The response comes back with only the ids that are not present, 'abc' and 'fdjk'.
In my spring data repo, I define the following method:
#Query("SELECT a.id FROM (VALUES (:ids)) AS a(id) LEFT JOIN (SELECT * FROM items WHERE job_id = :jobId) b ON a.id = b.id WHERE b.id IS null")
Flux<ItemId> getNotContains(#Param("jobId") String jobId, #Param("ids") Collection<String> ids);
The problem is, when I run the code, the query gets expanded to:
SELECT a.id FROM (VALUES ($1, $2, $3)) AS a(id) LEFT JOIN (SELECT * FROM items WHERE job_id = $251) b ON a.id = b.id WHERE b.id IS null]
This always returns a single value because the values are being grouped into a single set of parenthesis instead of wrapping each element of my collection in parenthesis. Just curious if there is a way to handle this properly.
EDIT
Entity class is:
#Data
#Table("items")
public class Item implements Persistable {
#Id
private String id;
private String jobId;
private String customerId;
private Date queuedDate;
private Date lastUpdated;
private String destination;
private String type;
private Status status;
}
Also, my repo is:
public interface ItemRepository extends R2dbcRepository<Item, String>
R2dbcRepository doesn't currently support the fancy magic of more mature spring data repos, so you can't do things like findByJobId and have it auto-gen the query for you.
You can enforce parenthesis by wrapping your arguments into a Object[] to render parameters as expression list.
interface MyRepo {
#Query(…)
Flux<ItemId> getNotContains(#Param("jobId") String jobId, #Param("ids") Collection<Object[]> ids);
}
MyRepo myRepo = …;
Collection<Object[]> ids = Arrays.asList(new Object[]{"1"}, new Object[]{"2"});
myRepo.getNotContains("foo", ids);
See also:
NamedParameterUtils Javadoc
Related
I have a snowflake database and it doesn't support unique constraint enforcement (https://docs.snowflake.com/en/sql-reference/constraints-overview.html).
I'm planning to have a method on JPA repository with a custom SQL query to check for duplicates before inserting to the table.
Entity
#Entity
#Table(name = "STUDENTS")
public class Students {
#Id
#Column(name = "ID", columnDefinition = "serial")
#GenericGenerator(name = "id_generator", strategy = "increment")
#GeneratedValue(generator = "id_generator")
private Long id;
#Column(name = "NAME")
private String studentName;
}
Snowflake create table query
CREATE table STUDENTS(
id int identity(1,1) primary key,
name VARCHAR NOT NULL,
UNIQUE(name)
);
Repository
public interface StudentRepository extends JpaRepository<Students, Long> {
//
#Query(value = "???", nativeQuery = true)
List<Student> bulkUpsertStudents(List<Student> students);
}
You can use a SELECT query to check for duplicate values in the name column before inserting a new record into the table. For example:
#Query(value = "SELECT * FROM STUDENTS WHERE name = :name", nativeQuery = true)
List<Student> findByName(#Param("name") String name);
This method will return a list of Student records with the specified name value. If the list is empty, it means that there are no records with that name value, and you can safely insert a new record with that name value.
List<Student> studentList = new ArrayList<>();
for (Student student : students) {
List<Student> existingStudents = studentRepository.findByName(student.getName());
if (existingStudents.isEmpty()) {
studentsToInsert.add(student);
}
}
studentRepository.bulkUpsertStudents(studentList)
EDIT
If the above solution doesn't work. You can use the MERGE statement to update existing records in the table if the data has changed. For example, if you want to update the name of a Student if it has changed, you can use the following MERGE statement:
#Query(value = "MERGE INTO students t USING (SELECT :name AS name, :newName AS newName) s
ON t.name = s.name
WHEN MATCHED AND t.name <> s.newName THEN UPDATE SET t.name = s.newName
WHEN NOT MATCHED THEN INSERT (name) VALUES (s.name)", nativeQuery = true)
List<Student> bulkUpsertStudents(List<Student> students);
This query will update the name of each Student in the students list if it has changed, and if a conflict occurs, it will not insert a new record. This will ensure that only unique name values are inserted into the table, without having to perform a separate query for each record.
I was able to overcome this using the below approach but need to verify the performance of the queries.
Repository saveAll() method to save all the entities.
Using the custom nativeQuery as below
INSERT OVERWRITE INTO STUDENTS
WITH CTE AS(SELECT ROW_NUMBER() OVER (PARTITION BY NAME ORDER BY ID) AS RNO, ID, NAME FROM STUDENTS)
SELECT ID, NAME FROM CTE WHERE RNO = 1;
Example code :
import static io.vavr.collection.List.ofAll;
import static io.vavr.control.Option.of;
import static java.util.function.Predicate.not;
public Validation<ValidationError, List<Students>> saveAll(List<String> students) {
return of(students)
.filter(not(List::isEmpty))
.map(this::mapToEntities) // maps the list to list of database entities
.map(repository::saveAll) // save all
.toValidation(ERROR_SAVING_STUDENTS) // vavr validation in case of error
.peek(x -> repository.purgeStudents()) // purging to remove duplicates
.toValidation(ERROR_PURGING_STUDENTS);
}
This issue is only due to snowflake's incapability to check uniqueness.
Consider the entities below -
#Entity
public class Employee {
#Id
#GeneratedValue
private long id;
private String name;
#OneToMany(mappedBy = "employee", fetch = FetchType.LAZY, cascade = CascadeType.ALL)
private List<Phone> phones; //contains both "active" & "inactive" phones
}
#Entity
public class Phone {
#Id
#GeneratedValue
private long id;
private boolean active;
private String number;
#ManyToOne(fetch = FetchType.LAZY)
private Employee employee;
}
I need to pull all the employees and sort them depending on the count of "active" phones they have.
Please note that the employee can have active as well as inactive phones. So the query I am trying to achieve is
ORDER BY (SELECT
COUNT(phone4_.employee_id)
FROM
phone phone4_
WHERE
employee4_.id = phone4_.employee_id
AND phone4_.active = true
) DESC
I am stuck with specification here because of some reason and below is the code I have used -
List<Order> orders = new ArrayList<>();
orders.add(cb.desc(cb.size(employee.get("phones"))));
cq.orderBy(orders);
When I run the code the query that's get generated is
ORDER BY (SELECT
COUNT(phone4_.employee_id)
FROM
phone phone4_
WHERE
employee4_.id = phone4_.employee_id) DESC
I am unable to add an extra AND condition to the logic. Please suggest
As specified in the Persistence API specification:
4.6.16 Subqueries
Subqueries may be used in the WHERE or HAVING clause.
JPA doesn't support subqueries in the order by clause, nor in the select clause.
Hibernate ORM, though, supports them in the SELECT and WHERE clauses.
So you cannot write that query and being JPA compliant.
This HQL should work though and it's covered by Hibernate ORM:
SELECT e1, (SELECT count(p)
FROM Phone p
WHERE p.active = true AND p.employee = e1) as activeCount
FROM Employee e1
ORDER BY activeCount DESC
Surprisingly, writing this query with criteria doesn't work:
CriteriaBuilder builder = ormSession.getCriteriaBuilder();
CriteriaQuery<Object> criteria = builder.createQuery();
Root<Employee> root = criteria.from( Employee.class );
Subquery<Long> activePhonesQuery = criteria.subquery( Long.class );
Root<Phone> phoneRoot = activePhonesQuery.from( Phone.class );
Subquery<Long> phonesCount = activePhonesQuery
.select( builder.count( phoneRoot ) )
.where( builder.and( builder.isTrue( phoneRoot.get( "active" ) ), builder.equal( phoneRoot.get( "employee" ), root ) ) );
criteria.multiselect( root, phonesCount )
.orderBy( builder.desc( phonesCount ) );
The reason is that, Hibernate ORM tries to expand the subquery in the order by clause instead to refer to an alias. And as I mentioned before, this is not supported.
I think the HQL is the easiest option if you don't want to use native queries.
I have problem to select all value from one table and few other columns using Spring Data JPA. I am using PostgreSql database and when I send query through PgAdmin I get values I want, but if I use it in Spring Boot Rest returns only one table values (subquery not working). What I am doing wrong?
#Query(value = "SELECT item.*, MIN(myBid.bid) AS myBid, (SELECT MIN(lowestBid.bid) AS lowestbid FROM bids lowestBid WHERE lowestBid.item_id = item.item_id GROUP BY lowestBid.item_id) FROM item JOIN bids myBid ON item.item_id = myBid.item_id WHERE myBid.user_id = :user_id GROUP BY item.item_id", nativeQuery = true)
public List<Item> findAllWithDescriptionQuery(#Param("user_id") UUID userId);
Added Item class
#Data
#Entity(name = "item")
public class Item {
#Id
#GeneratedValue
private UUID itemId;
#NotNull
#Column(name = "title")
#Size(max = 255)
private String title;
#NotNull
#Column(name = "description")
private String description;
#NotNull
#Column(name = "created_user_id")
private UUID createdUserId;
}
The result from your native query cannot simply be mapped to entities due to the in-database aggregation performed to calculate the MIN of own bids, and the MIN of other bids. In particular, your Item entity doesn't carry any attributes to hold myBid or lowestbid.
What you want to return from the query method is therefore a Projection. A projection is a mere interface with getter methods matching exactly the fields returned by your query:
public interface BidSummary {
UUID getItem_id();
String getTitle();
String getDescription();
double getMyBid();
double getLowestbid();
}
Notice how the query method returns the BidSummary projection:
#Query(value = "SELECT item.*, MIN(myBid.bid) AS myBid, (SELECT MIN(lowestBid.bid) AS lowestbid FROM bids lowestBid WHERE lowestBid.item_id = item.item_id GROUP BY lowestBid.item_id) FROM item JOIN bids myBid ON item.item_id = myBid.item_id WHERE myBid.user_id = :user_id GROUP BY item.item_id", nativeQuery = true)
public List<BidSummary> findOwnBids(#Param("user_id") UUID userId);
Return type is List of Item objects and the query specified is having columns which are not part of return object. I recommend using appropriate Entity which full-fills your response type.
I have two entity classes Request,User:
//Ommiting some annotations for brevity
public class User{
private Long id;
private String name;
private Integer age;
}
public class Request{
private Long id;
private String message;
private Date createTime;
#ManyToOne
#JoinColumn(name="user_id")
private User user;
}
I can sort request list by create time :
Sort = new Sort(Direction.ASC,"createTime");
Is there a possible way to sort request list by User's name? Like:
Sort = new Sort(Direction.ASC,"User.name");
Yes. new Sort(Direction.ASC,"user.name"); should work just fine.
Spring Data JPA will left outer join the User to the Request and order by the joined column's name resulting in SQL like this:
select
id, message, createTime
from
Request r
left outer join User u on u.id = r.user_id
order by
u.name asc
This works great on one-to-one and, like you have here, many-to-one relationships but because a left outer join is employed to implement the order by clause, if the joined entity represents a many relationship (like in a one-to-many), the Sort may result in SQL in which duplicate records are returned. This is the case because the Sort parameter will always result in a new left outer join even if the entity being joined is already joined in the query!
Edit Incidentally, there is an open ticket concerning this issue: https://jira.spring.io/browse/DATAJPA-776
Have a simple DTO Object as follows,
#BatchSize(size=100)
public class ProductDto {
#BatchSize(size=100)
private Product product;
private UUID storeId;
private String storeName;
public ProductDto(Product product, UUID storeId, String storeName) {
super();
this.product = product;
this.storeId = storeId;
this.storeName = storeName;
}
// setters and getters ...
}
Have Spring Data Repository as follows,
public interface ProductRepository extends CrudRepository<Product, java.util.UUID>
{
#BatchSize(size=100)
#Query("SELECT new com.app1.dto.ProductDto(c, r1.storeName, r1.id) "
+ "FROM Product c left outer join c.storeRef r1")
public Page<ProductDto> findProductList_withDTO(Pageable pageable);
}
When the code runs, it executes one SQL for loading Product ID and StoreName and StoreId
SELECT product0_.id AS col_0_0_,store1_.store_name AS col_1_0_,
store1_.id AS col_2_0_ FROM Product product0_
LEFT OUTER JOIN Store store1_ LIMIT 10
But the problem is next line,
For each Product Row that exists in Database, it's executing SQL Query, instead of using IN to batch load all matching Products in 1 SQL.
For each products it executes following,
SELECT product0_.id AS id1_20_0_, product0_.prd_name AS prd_nam15_20_0_ FROM Product product0_ WHERE product0_.id = ?
SELECT product0_.id AS id1_20_0_, product0_.prd_name AS prd_nam15_20_0_ FROM Product product0_ WHERE product0_.id = ?
Question,
How can we inform hibernate to load all Product Rows in Single SQL Query? I have tried adding #BatchSize(size=100) at all places. Still it's executing multiple queries to load Product Data.
Any hints/solution will be appreciated.
Thanks
Mark