I have the following code
#Indexed
#PrimaryKeyColumn(name = "x", ordinal = 1, type = PrimaryKeyType.PARTITIONED)
#Column(value="x")
private String x;
#Indexed
#PrimaryKeyColumn(name = "code", ordinal = 2, type = PrimaryKeyType.PARTITIONED)
#Column(value="code")
private String code;
#Query(value = "select * from customers where code = ?0")
Optional<Customer> findByCode(String code);
When this is executed, I get Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING.
Is there a way to avoid this just from spring-data-cassandra? I do not want to add ALLOW FILTERING in my query. I tried creating a separate index on the code column but this haven't solved the issue. I think it stops in the spring data configuration. If I execute the same query in cqlsh, it works.
You must specify partition key on your query, unless you create index or use ALLOW FILTERING
Executing query with allow filtering might not be a good idea as it can use a lot of your computing resources and Might not return any result because of timeout. Don't use allow filtering in production Read the datastax doc about using ALLOW FILTERING
https://docs.datastax.com/en/cql/3.3/cql/cql_reference/select_r.html?hl=allow,filter
When using a no-sql database, you need to properly design your data to avoid filtering. You can add a secondary index to optimize retrieval by a specific field. More details here: https://docs.datastax.com/en/archived/cql/3.3/cql/cql_using/useSecondaryIndex.html
If you are sure that the query is what you need, you can use the allowFiltering parameter on the #Query annotation to explicitly indicate that ALLOW FILTERING be used.
#Query(value = "select * from customers where code = ?0", allowFiltering = true)
Optional<Customer> findOneByCode(String code);
Related
I am facing Second Order SQL Injection in the Spring-Hibernate application after scanning through the Checkmarx tool, I have gone through multiple questions in StackOverflow and in other platforms as well but did not get the right finding.
could you please look into the below code snip,
public String getOrderId(order_name){
String returnId= null;
Query query = entityManager.createNativeQuery("select order_id from order where order_name=?");
List<String> dataset = query.setParameter(1,order_name).getResultList();
if(dataset!=null){
returnId = dataset. Get(0);
}
return returnId;
}
In this above method, while calling getResultList(), getting a high vulnerability issue that, this method returns data flows through the code without being properly sanitized or validated, and eventually used in further database query in the method.
Earlier code was like this,
public String getOrderId(order_name){
String returnId= null;
String q = "select order_id from order where order_name="+order_name;
Query query = entityManager.createNativeQuery(q);
and directly it was used as a string append in query, which I have modified with set parameter,
Query query = entityManager.createNativeQuery("select order_id from order where order_name=?");
List<String> dataset = query.setParameter(1,order_name).getResultList();
but still after getting data from query.getResultSet(), it is asking for sanitizing and validating the data before use in further database query method.
and this return data is being used in further query like select * from return_Data where clause. (properly used in where clause to set parameter to avoid SQL injection).
and in the above query is used in another method where we pass return_Data as input to it.
could you please help here to know what checks and validation can be added to overcome this type of issue. Thanks in advance for prompt response.
How can I achieve this query
select *
from table t
where name like '%Ami%'
order by (name = 'Ami') desc, length(col);
(just the sort part)
Using springframework Sort..
What I tried is
Sort.by(Sort.Direction.DESC, "name") // But I need to sort by name = 'Ami'
Sort.by(Sort.Direction.DESC, "name" = 'Ami'") // throws an error
JpaSort.unsafe(Sort.Direction.DESC, "name" = 'Ami'") // throws an error
Looks like the documentation has an example almost identical to your question:
https://docs.spring.io/spring-data/jpa/docs/2.4.5/reference/html/#jpa.query-methods.sorting
However, using Sort together with #Query lets you sneak in
non-path-checked Order instances containing functions within the ORDER
BY clause. This is possible because the Order is appended to the given
query string. By default, Spring Data JPA rejects any Order instance
containing function calls, but you can use JpaSort.unsafe to add
potentially unsafe ordering.
I have a requirement to create a REST api. Api allows user to provide dynamic search criteria in URL. For example, let say I have NOTES table with column as Note_ID, NOTE_TEXT, STATUS, PERSON_ID. This table is used to keep notes of every person.
Now I want my REST api to be as https://server:host/MyApi/Notes?search=NoteText=='My Java adventure'. API should provide all notes having NOTE_TEXT as 'My Java adventure'. Similarly user can provide status also in url and also he can use operators as LIKE. I was able to do it via rsql parser as mentioned in https://www.baeldung.com/rest-api-search-language-rsql-fiql
Now I have additional requirement that based on user security person_id filter should be applied on query automatically.
I found that we can't have findBy method which can take Specification, Pageable and extra personId. For example I can't have a repository function as
findByPersonId(Specification spec, Pageable page, Long personId);
I thought of using SpEL to use it, but then I found that if we use #Query annotation on findBy method, Specifications are ignored.
Seems like there is no way I can have Specification and #Query both. I need to add more clauses using specification only. In reality my where clause is very complex which I have to append and getting it with Specification seems to be difficult. Its something like
Select * from NOTES where exists (select 'x' from ABC a where n.person_id = a.person_id)
Is there a way I can write #Query and also have Specification working on top of it?
Ideally I have achieve a query like
select * from test.SCH_FORUM_THREAD t
where exists (select 'x' from test.FORUM_THREAD_ACCESS fta, school.SCH_GROUP_PERSON gp
where gp.GROUP_ID = fta.GROUP_ID
and t.THREAD_ID = fta.THREAD_ID
and gp.PERSON_ID = :personId)
or exists (select 'x' from test.FORUM_THREAD_ACCESS fta
where fta.THREAD_ID = t.THREAD_ID
and fta.PERSON_ID = :personId);
So there are two exists clauses with or condition. I was able to make second exists by following How to write query(include subquery and exists) using JPA Criteria Builder
Now struggling with first exists as it has join also. Any idea how to do that with Specification.
Also as there are two exists, does that mean I need two specifications. Can I achieve it in one specification.
I was able to resolve it by creating a complex specification code. Something like
#Override
public Predicate toPredicate(Root<ForumThread> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
Subquery<ForumThread> subQuery = query.subquery(ForumThread.class);
Root<ForumThread> subRoot = subQuery.from(ForumThread.class);
Join<ForumThreadAccess, GroupPerson> fragpjoin = subRoot.join("groupPersons");
Predicate threadPredicate = builder.equal(root.get("threadId"), subRoot.get("threadId"));
Predicate personPredicate = builder.equal(fragpjoin.get("personId"), personId);
subQuery.select(subRoot).where(threadPredicate, personPredicate);
Predicate existsGroupPredicate = builder.exists(subQuery);
Subquery<ForumThreadAccess> subQuery1 = query.subquery(ForumThreadAccess.class);
Root<ForumThreadAccess> subRoot1 = subQuery1.from(ForumThreadAccess.class);
Predicate threadPredicate1 = builder.equal(root.get("threadId"), subRoot1.get("threadId"));
Predicate personPredicate1 = builder.equal(subRoot1.get("personId"), personId);
subQuery1.select(subRoot1).where(threadPredicate1, personPredicate1);
Predicate existsPersonPredicate = builder.exists(subQuery1);
return builder.or(existsGroupPredicate,existsPersonPredicate);
}
To make it work your entities should also have proper #OneToMany and #ManyToMany in place.
Thanks
I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.
I've gone down a bit of a path and hit a wall with how this could be possibly achieved.
Basically, a query is constructed using JPA and passed to an Oracle DB. On the DB there is a Package, used to generate a reference, and this is dynamically named, based on the environment. This value is user-editable, and stored as a DB property within the application. I don't have any control over the architecture of this.
At a pre-JPA stage, a Query String is generated using the reference value for the Package, which is set as a property (again, I can't change the way this has been designed). I set this up using the Query method setParameter(), like so:
(pseudocode replacing the irrelevant parts for focused context)
String referenceRef = [ reference is fetched from DB properties ];
String queryString = "SELECT ?1 FROM sys.dual";
final Query myQuery = getEntityManager().createNativeQuery( queryString );
myQuery.setParameter( 1, referenceRef );
return myQuery.getSingleResult();
I pretty much did this as a reflex, only to realise (in retrospec, quite obviously) that this won't actually work, as it is escaping the element that should not be escaped...
So, where the referenceRef = "DynamicallyNamedPackage.DoThisDynamicallyNamedThing", the above code will just return "DynamicallyNamedPackage.DoThisDynamicallyNamedThing", as it is obviously making it safe, and the point of doing so is, to a certain extent, the antethesis of what I'm trying to do.
Is it possible to achieve this without creating a whole chunk of additional code? All I can currently think of, as an alternative, is to query dba_procedures for all package objects that match, and using the result of that query to construct the queryString (hence circumnavigating using any user-editable values), but it feels like it's going to be convoluted. This is the alternative, which I am using in lieu of an improvement:
final String verifyReference = "SELECT object_name FROM "
+ "dba_procedures WHERE object_type = 'PACKAGE' AND object_name =?1";
final Query refQuery = getEntityManager().createNativeQuery( verifyReference );
refQuery.setParameter( 1, referenceRef );
final String result = refQuery.getSingleResult();
final String queryString = "SELECT " + result + " FROM sys.dual";
final Query myQuery = getEntityManager().createNativeQuery( queryString );
return myQuery.getSingleResult();
It will essentially look up the user-editable property reference against a list of existing packages, then use the result of that query for building the original reference. It has more null checking and so on involved, and does remove the vulnerability, but feels a bit 'unpolished'.
(As has already been mentioned in the comments, this sort of is designed to need a SQL injection, but needs to prevent "SQL Injection" as a definition of not allowing the DB to be manipulated outside of the design by using an unintended value.)
The Oracle dictionary view all_procedures contains a list of all procedures accessible to the current user.
Specifically in the view there are columns OWNER, OBJECT_NAME (=package name), PROCEDURE_NAME.
You may use this view to sanitize the configured input by simple adding an EXISTS subquery such as:
select
?
from dual where exists (
select null from all_procedures
where
OWNER||'.'||OBJECT_NAME||'.'||PROCEDURE_NAME = upper(?) and
object_type = 'PACKAGE');
You will have to bind twice the same input parameter.
The query returns no data if there is not procedure with the given name, so you may raise an exception.
The query above expects a full qualified stored procedure name, i.e. owner.package.procedure, you'll have to adapt it slightly if you allow unqualified names (without the owner).