Any help would be greatly appreciated. I am working on a project using Spring JDBC for data access and am performing a simple query with an order by column expression, I am currently getting inconsistent results meaning the order by doesn't seem to be working. I have tried more than one database still no avail.
String sql = "select * from account where upper(name) like upper(:query) order by name asc";
MapSqlParameterSource params = new MapSqlParameterSource().addValue("query", "%" + query + "%");
List<Account> accountsSearched = namedParameterJdbcTemplate.query(sql, params, new BeanPropertyRowMapper<Account>(Account.class));
Any ideas what could be the issue?
So the problem is not within your SQL code, but problem exist in search method implementation
Existing Code
public List<Account> search(String uncleanedQuery, int offset) {
String query = uncleanedQuery.replaceAll("([-+.^:,])","");
String sql = "select * from account where upper(name) like upper(:query) order by name asc";
MapSqlParameterSource params = new MapSqlParameterSource().addValue("query", "%" + query + "%");
List<Account> accountsSearched = namedParameterJdbcTemplate.query(sql, params, new BeanPropertyRowMapper<Account>(Account.class));
Set<Account> accountSearchSet = new HashSet<Account>(accountsSearched);
List<Account> accounts = new ArrayList<Account>(accountSearchSet);
return accounts;
}
In the above code, we are fetching data correctly but assigning it to HashSet. HashSet does not respect ordering by name and generates random order for Account, due to which you are getting random order every time.
Solution 1:
There is no reason, you actually need Set. Using set just making your program slow. If you want to get DISTINCT data then modify SQL query.
public List<Account> search(String uncleanedQuery, int offset) {
String query = uncleanedQuery.replaceAll("([-+.^:,])","");
String sql = "select * from account where upper(name) like upper(:query) order by name asc";
MapSqlParameterSource params = new MapSqlParameterSource().addValue("query", "%" + query + "%");
List<Account> accountsSearched = namedParameterJdbcTemplate.query(sql, params, new BeanPropertyRowMapper<Account>(Account.class));
return accountsSearched;
}
Solution 2:
Still, you want to go with your approach then change code to use TreeSet and order based on the name
public List<Account> search(String uncleanedQuery, int offset) {
String query = uncleanedQuery.replaceAll("([-+.^:,])", "");
System.out.println("Search Query Called");
String sql = "select * from account where upper(name) like upper(:query) order by name";
MapSqlParameterSource params = new MapSqlParameterSource().addValue("query", "%" + query + "%");
List<Account> accountsSearched = namedParameterJdbcTemplate.query(sql, params,
new BeanPropertyRowMapper<Account>(Account.class));
Comparator<Account> comp = new Comparator<Account>() {
#Override
public int compare(Account a1, Account a2) {
return a1.getName().compareTo(a2.getName());
}
};
SortedSet<Account> accountSearchSet = new TreeSet<Account>(comp);
accountSearchSet.addAll(accountsSearched);
List<Account> accounts = new ArrayList<Account>(accountSearchSet);
return accounts;
}
Related
I am trying to sort my table's content on the backend side, so I am sending org.springframework.data.domain.Pageable object to controller. It arrives correctly, but at the repository I am getting org.hibernate.hql.internal.ast.InvalidPathException. Somehow the field name I would use for sorting gets an org. package name infront of the filed name.
The Pageable object logged in the controller:
Page request [number: 0, size 10, sort: referenzNumber: DESC]
Exception in repository:
Invalid path: 'org.referenzNumber'","logger_name":"org.hibernate.hql.internal.ast.ErrorTracker","thread_name":"http-nio-8080-exec-2","level":"ERROR","level_value":40000,"stack_trace":"org.hibernate.hql.internal.ast.InvalidPathException: Invalid path: 'org.referenzNumber'\n\tat org.hibernate.hql.internal.ast.util.LiteralProcessor.lookupConstant(LiteralProcessor.java:111)
My controller endpoint:
#GetMapping(value = "/get-orders", params = { "page", "size" }, produces = { MediaType.APPLICATION_JSON_VALUE })
public ResponseEntity<PagedModel<KryptoOrder>> getOrders(
#ApiParam(name = "searchrequest", required = true) #Validated final OrderSearchRequest orderSearchRequest,
#PageableDefault(size = 500) final Pageable pageable, final BindingResult bindingResult,
final PagedResourcesAssembler<OrderVo> pagedResourcesAssembler) {
if (bindingResult.hasErrors()) {
return ResponseEntity.badRequest().build();
}
PagedModel<Order> orderPage = PagedModel.empty();
try {
var orderVoPage = orderPort.processOrderSearch(resourceMapper.toOrderSearchRequestVo(orderSearchRequest), pageable);
orderPage = pagedResourcesAssembler.toModel(orderVoPage, orderAssembler);
} catch (MissingRequiredField m) {
log.warn(RESPONSE_MISSING_REQUIRED_FIELD, m);
return ResponseEntity.badRequest().build();
}
return ResponseEntity.ok(orderPage);
}
the repository:
#Repository
public interface OrderRepository extends JpaRepository<Order, UUID> {
static final String SEARCH_ORDER = "SELECT o" //
+ " FROM Order o " //
+ " WHERE (cast(:partnerernumber as org.hibernate.type.IntegerType) is null or o.tradeBasis.account.retailpartner.partnerbank.partnerernumber = :partnerernumber)"
+ " and (cast(:accountnumber as org.hibernate.type.BigDecimalType) is null or o.tradeBasis.account.accountnumber = :accountnumber)"
+ " and (cast(:orderReference as org.hibernate.type.LongType) is null or o.tradeBasis.referenceNumber = :orderReference)"
+ " and (cast(:orderReferenceExtern as org.hibernate.type.StringType) is null or o.tradeBasis.kundenreferenceExternesFrontend = :orderReferenceExtern)"
+ " and (cast(:dateFrom as org.hibernate.type.DateType) is null or o.tradeBasis.timestamp > :dateFrom) "
+ " and (cast(:dateTo as org.hibernate.type.DateType) is null or o.tradeBasis.timestamp < :dateTo) ";
#Query(SEARCH_ORDER)
Page<Order> searchOrder(#Param("partnerernumber") Integer partnerernumber,
#Param("accountnumber") BigDecimal accountnumber, #Param("orderReference") Long orderReference,
#Param("orderReferenceExtern") String orderReferenceExtern, #Param("dateFrom") LocalDateTime dateFrom,
#Param("dateTo") LocalDateTime dateTo, Pageable pageable);
}
Update:
I removed the parameters from the sql query, and put them back one by one to see where it goes sideways. It seems as soon as the dates are involved the wierd "org." appears too.
Update 2:
If I change cast(:dateTo as org.hibernate.type.DateType) to cast(:dateFrom as date) then it appends the filed name with date. instead of org..
Thanks in advance for the help
My guess is, Spring Data is confused by the query you are using and can't properly append the order by clause to it. I would recommend you to use a Specification instead for your various filters. That will not only improve the performance of your queries because the database can better optimize queries, but will also make use of the JPA Criteria API behind the scenes, which requires no work from Spring Data to apply an order by specification.
Since your entity Order is named as the order by clause of HQL/SQL, my guess is that Spring Data tries to do something stupid with the string to determine the alias of the root entity.
I have a method that should output me a JSON from a desired table. To keep the service as flexible as possible, I decided to use native queries, not to use entities and also to work without repositories. But so that I don't just get a list of objects from the database like this statement "final String sql = "SELECT * FROM " + schemaName + "." + tableName;", I extended my SQL statement with array_to_json. This is how I get a JSON from my DB.
Here is my method:
#Override
public List<Map<String, Object>> getList(String tableName, String schemaName) {
// final String sql = "SELECT * FROM " + schemaName + "." + tableName;
final String sql = "SELECT array_to_json(array_agg(" + tableName + ")) FROM " + schemaName + "." + tableName;
final Query query = em.createNativeQuery(sql);
final List<Map<String, Object>> queryResult = query.;
return queryResult;
}
When I execute my sql statement in the PG Admin then I get the following JSON:
[{"id":1,"analytic_id":1,"propertytype":"shape","min_val":null,"max_val":null,"string_val":"102941","case_else":0,"result":"kugel"},{"id":2,"analytic_id":1,"propertytype":"shape","min_val":null,"max_val":null,"string_val":"019283","case_else":0,"result":"rechteck"},{"id":3,"analytic_id":1,"propertytype":"shape","min_val":null,"max_val":null,"string_val":"0122343","case_else":0,"result":"prism"},{"id":6,"analytic_id":1,"propertytype":"color","min_val":0,"max_val":20,"string_val":null,"case_else":0,"result":"grun"},{"id":7,"analytic_id":1,"propertytype":"color","min_val":21,"max_val":50,"string_val":null,"case_else":0,"result":"gelb"},{"id":8,"analytic_id":1,"propertytype":"color","min_val":51,"max_val":80,"string_val":null,"case_else":0,"result":"hellblau"},{"id":9,"analytic_id":1,"propertytype":"color","min_val":81,"max_val":999,"string_val":null,"case_else":0,"result":"rot"},{"id":10,"analytic_id":1,"propertytype":"color","min_val":null,"max_val":null,"string_val":null,"case_else":1,"result":"lila"}]
Now for my question:
So I get a JSON from my query. But how do I rebuild my method so that I can display the result of my query in the GetRequest. It is important to me that I get a JSON with all data back and that the return type is compatible with OPENAPI 3.0.
Thanks in advance
The following ItemReader get a list of thousands accounts that need to be retrieved from MD database.
In this approach I am limited to the number of accounts that I can use:
#StepScope
#Bean
public ItemReader<OmsDto> itemReader(#Value("#{stepExecutionContext[accOms]}") List<String> notLoadedFiles) {
StringBuffer buffer = new StringBuffer();
notLoadedFiles.forEach(accountNumber -> buffer.append("'"+accountNumber+"',"));
buffer.replace(buffer.length()- 1, buffer.length(), "");
DriverManagerDataSource mdDataSource = new DriverManagerDataSource();
mdDataSource.setDriverClassName("prestosql");
mdDataSource.setUrl("jdbc:presto:....");
mdDataSource.setUsername(".....");
mdDataSource.setPassword("....");
String sql ="SELECT DISTINCT "
.....
.....
+ "FROM MD.ONLINE WHERE acct IN ";
JdbcCursorItemReader<OmsDto> reader = new JdbcCursorItemReader<OmsDto>();
reader.setVerifyCursorPosition(false);
reader.setDataSource(mdDataSource);
reader.setSql(sql);
reader.open(new ExecutionContext());
BeanPropertyRowMapper<OmsDto> rowMapper = new BeanPropertyRowMapper<>(OmsDto.class);
rowMapper.setPrimitivesDefaultedForNullValue(true);
reader.setRowMapper(rowMapper);
return reader;
}
What is the correct way to create dynamic IN Query (WHERE A IN (…, .., …)) ?
Thank you
Here is an example to generate IN query dynamically,
Example Query: SELECT * FROM USER WHERE ID IN (?,?,?,?,?)
List ids = List.of(1,2,3,4,5);
String inParams = String.join(",", ids.stream().map(id -> "?").collect(Collectors.toList()));
String query = String.format("SELECT * FROM USER WHERE ID IN (%s)", inParams);
Note that, if your query IN clause parameters limit more than 1000, it's better to use TEMP tables. Here some examples on github
In my JDBC training, I have a question on the use of the where clause.
Suppose that i have a table in my db that i want manage with a spring application using a jdbc template, let's assume "Logbase", with this column: host, user, clientip. Suppose now that i want allow query db based on a single column for all column, that is:
Select * from Logbase where host = x
and
Select * from Logbase where user = y
and
Select * from Logbase where clientip = z
I suppose I must write a separated java method for every of this query, something like this:
public Logbase getLogbaseFromHost(String id)
{
String SQL = "select * from Logbase where host = ?";
Logbase logbase = (Logbase) jdbcTemplate.queryForObject(SQL, new Object[]{id},
(rs, rowNum) -> new Logbase(rs.getString("host"), rs.getString("user"),
rs.getInt("clientip")));
return logbase;
}
public Logbase getLogbaseFromUser(String id)
{
String SQL = "select * from Logbase where user = ?";
Logbase logbase = (Logbase) jdbcTemplate.queryForObject(SQL, new Object[]{id},
(rs, rowNum) -> new Logbase(rs.getString("host"), rs.getString("user"),
rs.getInt("clientip")));
return logbase;
}
public Logbase getLogbaseFromClientIP(String id)
{
String SQL = "select * from Logbase where clientip = ?";
Logbase logbase = (Logbase) jdbcTemplate.queryForObject(SQL, new Object[]{id},
(rs, rowNum) -> new Logbase(rs.getString("host"), rs.getString("user"),
rs.getInt("clientip")));
return logbase;
}
Now, if i want allow query db based on 2 parameters, i suppose i must write a method for the 3 possible pair of parameters (one for clientip-user, another for clientip-host and the last for user-host).
Finally, if i want allow query db selecting all the parameters, i must write another method with the where clause in the query that ask for all variables.
If I did not say heresies and everything is correct, i have 7 method. But, i the number of parameters and combinations grow, this may be a problem. There is a way to get around it?
Note: for work reasons, i cant use Hibernate or other ORM framework. I MUST use jdbc.
Tnx to all for patience and response.
The solution could be based on SQL
Select *
from Logbase
where
(? is null or host = ?)
AND (? is null or user = ?)
AND (? is null or clientip = ?)
jdbcTemplate.queryForObject(SQL, new Object[]{host, host, user, user, clienttip, clienttip}
So e.g. if user is not specified (user is null - true) all the records are included
So, you also can use org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate
** Select *
from Logbase where
(:host is null or host = :host)
AND (:user is null or user = :user)
AND (:clientip is null or clientip = :clientip)**
And java code:
MapSqlParameterSource params = new MapSqlParameterSource();
params.addValue("host", host);
params.addValue("user", user);
params.addValue("clientip", clientip);
namedParameterJdbcTemplate.queryForObject(sqlQuer, params);
NamedParameterJdbcTemplate namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(jdbcTemplate.getDataSource());
MapSqlParameterSource params = new MapSqlParameterSource();
params.addValue("STR_REF",(String)strRefkeys);
String sql = "SELECT TAGDESCRIPTION FROM xx, xx WHERE localeid='en_US' AND " +
"xx.TAGID=xx.RECORDID and TAGDESCRIPTION is not null AND xx.REFERENCEKEY in (:STR_REF)";
List<String> ifCurrent = namedParameterJdbcTemplate.queryForList(sql, params,String.class);
not getting any results in ifCurrent although run same query as SQL query and get results.
Am I passing any wrong params?
This is what is getting passed in strRefkeys
for(String refStr : strRefkeysLst) {
strRefkeysBuf.append("'");
strRefkeysBuf.append(refStr.toUpperCase());
strRefkeysBuf.append("',");
}
strRefkeys = strRefkeysBuf.toString();
if(strRefkeys.trim().length()>1){strRefkeys = strRefkeys.substring(0, strRefkeys.length()-1);}
You're passing a string parameter where a collection of values is expected. strRefKeys should be a set of accepted values for the REFERENCEKEY column.