I've modified an existing RESTful/JDBC application i have to work with new features in Spring 4... specifically the JpaRepository. It will:
1) Retrieve a list of transactions for a specified date. This works fine
2) Retrieve a count of transactions by type for a specified date. This is not working as expected.
The queries are setup similarly, but the actual return types are very different.
I have POJOs for each query
My transactions JPA respository looks like:
public interface MyTransactionsRepository extends JpaRepository<MyTransactions, Long>
//My query works like a charm.
#Query( value = "SELECT * from ACTIVITI_TMP.BATCH_TABLE WHERE TO_CHAR(last_action, 'YYYY-MM-DD') = ?1", nativeQuery = true )
List< MyTransactions > findAllBy_ToChar_LastAction( String lastActionDateString );
This returns a list of MyTransactions objects as expected. Debugging, i see the returned object as ArrayList. Looking inside the elementData, I see that each object is, as expected, a MyTransactions object.
My second repository/query is where i'm having troubles.
public interface MyCountsRepository extends JpaRepository<MyCounts, Long>
#Query( value = "SELECT send_method, COUNT(*) AS counter FROM ACTIVITI_TMP.BATCH_TABLE WHERE TO_CHAR(last_action, 'YYYY-MM-DD') = ?1 GROUP BY send_method ORDER BY send_method", nativeQuery = true )
List<MyCounts> countBy_ToChar_LastAction( String lastActionDateString );
This DOES NOT return List as expected.
The object that holds the returned data was originally defined as List, but when I inspect this object in Eclipse, I see instead that it is holding an ArrayList. Drilling down to the elementData, each object is actually an Object[2]... NOT a MyCounts object.
I've modified the MyCountsRepository query as follows
ArrayList<Object[]> countBy_ToChar_LastAction( String lastActionDateString );
Then, inside my controller class, I create a MyCounts object for each element in List and then return List
This works, but... I don't understand why i have to go thru all this?
I can query a view as easily as a table.
Why doesn't JPA/Hibernate treat this as a simple 2 column table? send_method varchar(x) and count (int or long)
I know there are issues or nuances for how JPA treats queries with counts in them, but i've not seen anything like this referenced.
Many thanks for any help you can provide in clarifying this issue.
Anthony
That is the expected behaviour when you're doing a "group by". It will not map to a specific entity. Only way this might work is if you had a view in your database that summarized the data by send_method and you could map an entity to it.
Related
I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.
I have entity like:
#Id
#Column_name = "abc"
int pk;
#Column_name = "def"
int id;
And I have Repository as:
interface fetchDataRepository extends jpaRepository<className, int> {
#Query("Select S_Test.nextVal from dual");
Long generateId();
}
In above example S_Test is hardcoded sequence name.
But the problem is that I want to pass sequence name dynamically as follows:
Long generateId(#Param("sequenceName") String sequenceName)
and use inside #Query annotation as:
#Query("Select :sequenceName.nextVal from dual");
Is there anyway to do that? Any suggestion would be appreciated.
Edit: Isn't there possible to use #(#entityName). If yes, then please tell me how?
Unfortunately you can only substitute in things that you could do in JDBC anyway (so, pretty much just values in the INSERT and WHERE clauses). No dynamic table, column, schema names are supported.
There is one exception that may apply, and that is a limited subset of SpEL can be used. There is one variable available - #entityName. So, assuming that the #Entity annotation on your entity class is named identically to the sequence, you could use an #Query like so:
#Query("Select #{#entityName}.nextVal from dual");
Otherwise, since your query is simple and does not involve any object relational mapping, you would probably need to Create a custom repository implementation and inject a JdbcTemplate into it in order to run the query.
Else you could inject an EntityManager and try using the JPA Criteria API - but again you arent actualy trying to map a resultset to an entity so JdbcTemplate will be simpler.
I came to a problem where I need to put ordering by multiplication of two columns of entity, for the sake of imagination entity is:
#Entity
public class Entity {
#Column(name="amount")
private BigDecimal amount;
#Column(name="unitPprice")
private BigDecimal unitPrice;
.
.
.
many more columns
}
My repo interface implements JpaRepository and QuerydslPredicateExecutor,
but I am struggling to find a way to order my data by "amount*unitPrice",
as I can't find a way to put it into
PageRequest (new Sort.Order(ASC, "amount * unitPrice"))
without having PropertyReferenceException: No property amount * unitPrice... thrown.
I can't user named query, as my query takes quite massive filter based on user inputs (can't put where clause into query, because if user hasn't selected any value, where clause can't just be in query).
To make it simple. I need something like findAll(Predicate, Pageable), but I need to force that query to order itself by "amount * unitPrice", but also have my Preditate (filter) and Pageable (offset, limit, other sortings) untouched.
Spring Sort can be used only for sorting by properties, not by expressions.
But you can create a unique sort in a Predicate, so you can add this sort-predicate to your other one before you call the findAll method.
I've got a bit of a bizarre problem that I can't figure out why it's happening. I'm sure I did something wrong, because this is my first time using a data projection and I've never had such problems using DTOs.
Pretty much I have a SELECT statemen that is returning certain columns of various data types. And I have an interface that I'm passing to the JPA Repository so it can do the interface mapping. But instead of mapping the results based on the column name (eg. 'accountnum' -> getAccountnumber()), it's mapping the columns in alphabetical order. So if 'date_of_order' is the first in the SELECT statement, its value will be returned by getAccountnumber().
I have a projected interface that looks something like this:
public interface FlatSearchResult {
String getAccountnumber();
UUID getTrackingId;
Date getDateOfOrder;
}
My model has three tables something like this:
ACCOUNT
- account_id : uuid (pkey)
- accountnumber : string
ORDERS
- order_id : uuid (pkey)
- date_of_order : timestamp
- account_id : uuid (fkey)
TRACKING
- tracking_id : uuid (pkey)
- order_id : uuid (fkey)
There's other columns in each of those tables, but they're not relevant.
I have a repository defined with a simple query:
public interface OrderTrackingRepository extends JpaRepository<Account, UUID> {
#Query( nativeQuery = true,
value = "SELECT o.date_of_order, a.accountnumber, t.tracking_id " +
"FROM account as a " +
"INNER JOIN orders as o USING (account_id) " +
"INNER JOIN tracking as t USING (tracking_id) " +
"WHERE a.accountnumber = :acctnum")
<T> Collection<T> findOrderInfoForAccount(#Param("acctnum") acctNumber, Class<T> type);
}
When I call this method, the correct rows are returned by the query. But instead of mapping using the column name (eg. date_of_order to getDateOfOrder()), it is mapping based on the order of the columns in the SELECT statement to the alphabetically-ordered methods in the interface.
So:
SELECT date_of_order, accountnumber, tracking_id
Results in:
getAccountNumber() -> date_of_order
getDateOfOrder() -> accountnumber
getTrackingId() -> tracking_id
It will consistently return in this fashion, so it's not a transient issue.
As a temporary workaround, I've reordered the columns in my SELECT statement. But I would rather not have to do this since it's like iterating through a result set and relying on column position, which just makes me twitchy....
How can I get Spring JPA to map from the result set to my interface? Do I need to annotate my projection interface's methods with something to tell Spring what column name it's referring to?
My database is Postgres. I'm using Spring 5.0.2.RELEASE and Spring-Boot 2.0.0.M7. I can adjust either of those to newer versions if needed, but nothing older. I'm using C3P0 0.9.5.2 for my connection pooling, and postgres-9.2-1002.jdbc4. All my other dependencies (hibernate, etc) are what is pulled in by this version of Spring-Boot.
Not sure if this is the correct solution because it only fits 80% of the description. But it is too long for a comment. So here we go.
I think you misunderstood #osamayaccoub or the documentation. Your property name is fine. But the columns in your select should match the java convention.
So the first attempt to fix that would be
value = "SELECT o.date_of_order as dateOfOrder, a.accountnumber as accountNumber, t.tracking_id as trackingId "
Note: This might actually work, but might break later, so read on, even if it does work
But Postgres converts everything that isn't double quoted into lower case (Oracle and MySql do similar stuff though details vary, don't know about other DBs yet). So you really should use:
value = "SELECT o.date_of_order as \"dateOfOrder\", a.accountnumber as \"accountNumber\", t.tracking_id as \"trackingId\" "
This probably doesn't work, because the Hibernate version you are using has a bug in that it converted everything to lower case.
So you should upgrade to the latest Hibernate version 5.3.13 which has the issue fixed.
This bug fix interestingly might break the version without the double quotes.
But it should work again with this PR for this Spring Data JPA issue.
The part I don't understand is, why stuff gets assigned using the column order.
I had the same problem and i solved by odering the query columns alphabetically.
In you case:
public interface OrderTrackingRepository extends JpaRepository<Account, UUID> {
#Query( nativeQuery = true,
value = "SELECT a.accountnumber, o.date_of_order, t.tracking_id " +
"FROM account as a " +
"INNER JOIN orders as o USING (account_id) " +
"INNER JOIN tracking as t USING (tracking_id) " +
"WHERE a.accountnumber = :acctnum")
<T> Collection<T> findOrderInfoForAccount(#Param("acctnum") acctNumber, Class<T> type);
}
So you will get:
getAccountNumber() -> accountnumber
getDateOfOrder() -> date_of_order
getTrackingId() -> tracking_id
Hibernate sorts the query in alphabetic order so you have to change the select to: "SELECT a.accountnumber, o.date_of_order, t.tracking_id ..." and the interface's getters should follow the same alphabetic order.
I have the following HQL in Hibernate using Spring MVC.
List<Colour>list=session.createQuery("from Colour order by colourId desc")
.setFirstResult((currentPage-1)*rowsPerPage)
.setMaxResults(rowsPerPage).list();
It works and returns a list of rows from the colour table (actually operates upon the Colour entity (POJO) that I can understand) in Oracle 10g.
What if I need to retrieve a list fields, I'm trying the following.
List<Colour>list=session.createQuery("colourId, colourName, colourHex from Colour order by colourId desc")
.setFirstResult((currentPage-1)*rowsPerPage)
.setMaxResults(rowsPerPage).list();
It ends with an excpetion
java.lang.IllegalArgumentException: node to traverse cannot be null!
In some articles, it was mentioned that the following version of HQL should (or may) work
List<Colour>list=session.createQuery("select colourId, colourName, colourHex from Colour order by colourId desc")
.setFirstResult((currentPage-1)*rowsPerPage)
.setMaxResults(rowsPerPage).list();
but unfortunately, it also didn't work for me. Using the createSQLQuery() method to execute native SQL would work but I want to stick to the createQuery() method with HQL unless it's absolutely necessary. How can I specify a list of fields in HQL?
I agree with yorkw's comment. If you select properties in your query then you cannot ask for a List<Colour> object to be returned from a call to .list().
Instead you should do this
List<Object[]> rows = session.createQuery("select c.colourId, c.colourName, c.colourHex " +
" from Colour c " +
" order by c.colourId desc").list();
Then iterate over the list object and instantiate your objects. Or whatever you need to do.
for ( Object[] row : rows ) {
Long colourId = (Long)row[0];
// ... etc
}
Why don't you try creating a map? Something like this:
SELECT NEW MAP( colour.colourId AS id
, colour.colourName AS name ...)
FROM Colour colour
ORDER BY colour.colourId
I use the alias for Colour "colour" so hibernate knows from which entity is the property I am referencing, I am implying all those properties are from the same entity, if not, then check your referencing!