I have an issue related to the performance of a SQL query using JPA.
Response time:
Using Toad - 200 ms
Inside my project using Glassfish 2.1, Java 1.5, Hibernate 3.4.0.ga - 27 s
Oracle 10g
Glassfish and Toad are hosting in the same machine. I have connected to other ddbb from the same Glassfish, JPA, etc, and performance is good. so I don't know what is happening.
I have two different environments. In one of this (the worst, theoretically) it runs fast. In the other, it's where I have the problem.
The query is executed with a Javax.persistence.Query object and in this object are inserted the parameters with the method setParameter(). After that, I call to getResultList() method and this method returns the registers to me. In this point is where the time is excessive.
But, if I replace the parameters in code and I call to getResultList() method directly, without setting parameters into Query object, the performance is much better.
Anyone could help me with any clue about the problem or how to trace it?
Query
SELECT A, B, ..., DATE_FIELD FROM
(SELECT A, B, C FROM Table1
WHERE REGEXP_LIKE(A, NVL(UPPER(:A),'')) AND DATE_FIELD = :DATE
UNION
SELECT A, B, C FROM Table2
WHERE REGEXP_LIKE(A, NVL(UPPER(:A),'')) AND DATE_FIELD = :DATE)
Java Code
public Query generateQuerySQL(String stringQuery, HashMap<String, Object> hParams) {
Query query = em.createNativeQuery(stringQuery);
if (hParams != null) {
for (Iterator<String> paramNameList = hParams.keySet().iterator(); paramNameList.hasNext() {
String name = paramNameList.next();
Object value = hParams.get(name);
query.setParameter(name, value);
}
}
return query;
}
Query query = em.createNativeQuery(stringQuery);
will elaborate a query plan to execute the query. Unfortunally the metadata that is used to elaborate the query plan do not fit the actual parameters values that will be used when the query will be executed.
If you substitute the parameter before elaborating the plan : the plan is fine and run very fast.
Similar question here
you should change cursor_sharing = FORCE in oracle to enable hibernate support in JPA for oracle.
please refer to following for more details
Related
I am facing Second Order SQL Injection in the Spring-Hibernate application after scanning through the Checkmarx tool, I have gone through multiple questions in StackOverflow and in other platforms as well but did not get the right finding.
could you please look into the below code snip,
public String getOrderId(order_name){
String returnId= null;
Query query = entityManager.createNativeQuery("select order_id from order where order_name=?");
List<String> dataset = query.setParameter(1,order_name).getResultList();
if(dataset!=null){
returnId = dataset. Get(0);
}
return returnId;
}
In this above method, while calling getResultList(), getting a high vulnerability issue that, this method returns data flows through the code without being properly sanitized or validated, and eventually used in further database query in the method.
Earlier code was like this,
public String getOrderId(order_name){
String returnId= null;
String q = "select order_id from order where order_name="+order_name;
Query query = entityManager.createNativeQuery(q);
and directly it was used as a string append in query, which I have modified with set parameter,
Query query = entityManager.createNativeQuery("select order_id from order where order_name=?");
List<String> dataset = query.setParameter(1,order_name).getResultList();
but still after getting data from query.getResultSet(), it is asking for sanitizing and validating the data before use in further database query method.
and this return data is being used in further query like select * from return_Data where clause. (properly used in where clause to set parameter to avoid SQL injection).
and in the above query is used in another method where we pass return_Data as input to it.
could you please help here to know what checks and validation can be added to overcome this type of issue. Thanks in advance for prompt response.
I'm using EF 5 with Oracle database.
I'm doing a select count in a table with a specific parameter. When I'm using EF, the query returns the value 31, as expected, But the result takes about 10 seconds to be returned.
using (var serv = new Aperam.SIP.PXP.Negocio.Modelos.SIP_PA())
{
var teste = (from ens in serv.PA_ENSAIOS_UM
where ens.COD_IDENT_UNMET == "FBLDY3840"
select ens).Count();
}
If I execute the simple query bellow the result is the same (31), but the result is showed in 500 milisecond.
SELECT
count(*)
FROM
PA_ENSAIOS_UM
WHERE
COD_IDENT_UNMET 'FBLDY3840'
There are a way to improve the performance when I'm using EF?
Note: There are 13.000.000 lines in this table.
Here are some things you can try:
Capture the query that is being generated and see if it is the same as the one you are using. Details can be found here, but essentially, you will instantiate your DbContext (let's call it "_context") and then set the Database.Log property to be the logging method. It's fine if this method doesn't actually do anything--you can just set a breakpoint in there and see what's going on.
So, as an example: define a logging function (I have a static class called "Logging" which uses nLog to write to files)
public static void LogQuery(string queryData)
{
if (string.IsNullOrWhiteSpace(queryData))
return;
var message = string.Format("{0}{1}",
queryData.Trim().Contains(Environment.NewLine) ?
Environment.NewLine : "", queryData);
_sqlLogger.Info(message);
_genLogger.Trace($"EntityFW query (len {message.Length} chars)");
}
Then when you create your context point to LogQuery:
_context.Database.Log = Logging.LogQuery;
When you do your tests, remember that often the first run is the slowest because the server has to actually do the work, but on the subsequent runs, it often uses cached data. Try running your tests 2-3 times back to back and see if they don't start to run in the same time.
I don't know if it generates the same query or not, but try this other form (which should be functionally equivalent, but may provide better time)
var teste = serv.PA_ENSAIOS_UM.Count(ens=>ens.COD_IDENT_UNMET == "FBLDY3840");
I'm wondering if the version you have pulls data from the DB and THEN counts it. If so, this other syntax may leave all the work to be done at the server, where it belongs. Not sure, though, esp. since I haven't ever used EF with Oracle and I don't know if it behaves the same as SQL or not.
Admittedly I don't perform lots of LINQ queries. Therefore I'm uncertain whether the problem I see is due to an obvious LINQ blunder or a legitimate Mongo driver problem (I use 10Gen 1.9.2 C# driver). In the below code I get an error indicating invalid where clause for .where(ques => unAnswered...). Code compiles fine but generates runtime error stating "unsupported where clause". Am I up against a driver limitation or is my LINQ bad?
public IEnumerable<QuestionDataModel> getUnanswered(String username, Category cat)
{
IQueryable<QuestionDataModel> questions =
from e in this.questionCollection.AsQueryable<QuestionDataModel>()
where (e.questionCategory == cat)
select e;
IQueryable<AnswerDataModel> answers =
from e in this.answerCollection.AsQueryable<AnswerDataModel>()
where (e.questionCategory == cat && e.username == username)
select e;
IEnumerable<QuestionDataModel> filteredquestionslist = null;
if (answers.Count()==0) // it's possible the user has not answered anything
filteredquestionslist = questions.ToList();
else
filteredquestionslist = questions.Where(ques => unAnswered(ques, ref answers)).ToList();
return filteredquestionslist;
}
private bool unAnswered(QuestionDataModel qdm, ref IQueryable<AnswerDataModel> answer_queryable)
{
bool retval;
retval = answer_queryable.Any(ans => ans.questionID == qdm.questionID) ? false:true;
return retval;
}
You can't combine two collections in a single query like this with MongoDB - there are no join operations in the database. (You also generally can't use your own method like that in LINQ since they don't translate into SQL (or any other database) but that's a separate issue and even if you fixed that it still wouldn't help here. unAnswered question cannot be translated into Mongo a query).
You must either iterate over one collection, performing the other query and yield return the results you want (i.e. the join happens not in the database but on the computer making the query), or you could denormalize the data in some way such that you can query a single collection to get the results. Of if the number of questions is really small you could possibly load them into a list using .ToList() and then operating on that list in memory.
I have been trying for hours to get a Distinct to work for my code.
I am using EF 4.3, MVC3, Razor and trying to get a list downto product id and name.
When I run the Sql query against the DB, it's fine.
Sql Query is
SELECT DISTINCT [ProductId]
,[Product_Name]
FROM [dbo].[PRODUCT]
The only other column in that table is a country code so that's why a standard distinct() isn't working.
I have gone as far as creating an IEqualityComparer
Here is code:
public class DistinctProduct : IEqualityComparer<PRODUCT>
{
public bool Equals(PRODUCT x, PRODUCT y)
{
return x.ProductId.Equals(y.ProductId);
}
public int GetHashCode(PRODUCT obj)
{
return obj.ProductId.GetHashCode();
}
}
here is where I called it.
IEqualityComparer<PRODUCT> customComparer = new DistinctProduct();
IEnumerable<PRODUCT> y = db.PRODUCTs.Distinct(customComparer);
But when it hit's that Last line I get an error out of it stating...
LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[MyService.Models.PRODUCT] Distinct[PRODUCT](System.Linq.IQueryable`1[MyService.Models.PRODUCT], System.Collections.Generic.IEqualityComparer`1[MyService.Models.PRODUCT])' method, and this method cannot be translated into a store expression.
Can anyone tell me what I'm doing wrong?
Thanks,
David
Is there any reason you could just not use a distinct like the following?
var distinctProdcts = (from p in db.PRODUCTs
select new {
ProductId = p.ProductId,
Product_Name = p.ProductName
}).Distinct();
This would remove the country code from the query before you do the distinct.
Entity Framework is trying to translate your query to a SQL query. Obviously it does not know how to translate the IEqualityComparerer. I think the question is whether you want to do the Distinct in the datbase (in which case your client gets only filtered results) or you are OK with bringing all the data to the client and select distinct on the client. If you want the filtering to happen on the database side (which will make your app perform much better) and you want to be able to use different strategies for comparing you can come up with a code that builds distinct criteria on top of your query. If you are fine with bringing your data to the client (note that it can be a lot of data) you should be able just to do (.ToList() will trigger querying the database and materializing results):
IEnumerable<PRODUCT> y = db.PRODUCTs.ToList().Distinct(customComparer);
I've been developing a webapp using Linq to NHibernate for the past few months, but haven't profiled the SQL it generates until now. Using NH Profiler, it now seems that the following chunk of code hits the DB more than 3,000 times when the Linq expression is executed.
var activeCaseList = from c in UserRepository.GetCasesByProjectManagerID(consultantId)
where c.CompletionDate == null
select new { c.PropertyID, c.Reference, c.Property.Address, DaysOld = DateTime.Now.Subtract(c.CreationDate).Days, JobValue = String.Format("£{0:0,0}", c.JobValue), c.CurrentStatus };
Where the Repository method looks like:
public IEnumerable<Case> GetCasesByProjectManagerID(int projectManagerId)
{
return from c in Session.Linq<Case>()
where c.ProjectManagerID == projectManagerId
select c;
}
It appears to run the initial Repository query first, then iterates through all of the results checking to see if the CompletionDate is null, but issuing a query to get c.Property.Address first.
So if the initial query returns 2,000 records, even if only five of them have no CompletionDate, it still fires off an SQL query to bring back the address details for the 2,000 records.
The way I had imagined this would work, is that it would evaluate all of the WHERE and SELECT clauses and simply amalgamate them, so the inital query would be like:
SELECT ... WHERE ProjectManager = #p1 AND CompleteDate IS NOT NULL
Which would yield 5 records, and then it could fire the further 5 queries to obtain the addresses. Am I expecting too much here, or am I simply doing something wrong?
Anthony
Change the declaration of GetCasesByProjectManagerID:
public IQueryable<Case> GetCasesByProjectManagerID(int projectManagerId)
You can't compose queries with IEnumerable<T> - they're just sequences. IQueryable<T> is specifically designed for composition like this.
Since I can't add a comment yet. Jon Skeet is right you'll want to use IQueryable, this is allows the Linq provider to Lazily construct the SQL. IEnumerable is the eager version.