I'm working with a framework that uses EntityManager's transactions to give a transaction context to batch treatments. (not #Transactional)
For each item treated, I use a SQL native query (JPA - ORACLE) that contains a UNION ALL to process a transitive closure on a certain relation.
Something like this :
public String getMostRecent(String var) {
String sql = """"
WITH testo(a,b) AS
(SELECT a,b
FROM foo
WHERE a = :var
UNION ALL
SELECT t2.a,
t2.b
FROM foo t2,
testo
WHERE t2.a = testo.b)
SELECT a FROM testo WHERE b IS NULL;
""";
Query q = broker.getContext().createNativeQuery(sql).setParameter("var", var);
try {
List<Object> result = q.getResultList();
// if var is not part of the transitive closure, it's already the most recent record
if (result == null || result.isEmpty()) {
return var;
} else {
return result.get(0).toString();
}
} catch (Exception e) {
// we couldn't find the most recent record corresponding to var because of a cyclic reference.
// we thus return the one given as parameter" + var+ ".);
return var;
}
}
When the said relation present a cycle I get the following exception :
2022-03-16 16:28:17,645 WARN o.h.e.j.s.SqlExceptionHelper - SQL Error: 32044, SQLState: 99999
2022-03-16 16:28:17,650 ERROR o.h.e.j.s.SqlExceptionHelper - ORA-32044: cycle detected while executing recursive WITH query
2022-03-16 16:28:17,650 ERROR o.h.e.j.s.SqlExceptionHelper - ORA-32044: cycle detected while executing recursive WITH query
This exception is caught in the try catch, but that doesn't matter to us. If the query fails, "var" is a "good enough" return value.
The problem however is that this exception seem to marks the transaction as rollback only ... which thus rollback the treatment of all the items in the transaction...
I found a way to get over the problem by using a nested transaction just to execute this SQL statement as follows:
public String getMostRecent(String var) {
String sql = """"
WITH testo(a,b) AS
(SELECT a,b
FROM foo
WHERE a = :var
UNION ALL
SELECT t2.a,
t2.b
FROM foo t2,
testo
WHERE t2.a = testo.b)
SELECT a FROM testo WHERE b IS NULL;
""";
Query q = broker.getContext().createNativeQuery(sql).setParameter("var", var);
try {
getTransactionManager().beginTransaction(TransactionMode.REQUIRES_NEW);
List<Object> result = q.getResultList();
// if var is not part of the transitive closure, it's already the most recent record
if (result == null || result.isEmpty()) {
return var;
} else {
return result.get(0).toString();
}
} catch (Exception e) {
// we couldn't find the most recent record corresponding to var because of a cyclic reference.
// we thus return the one given as parameter" + var+ ".);
return var;
} finally {
getTransactionManager().endTransaction(false);
}
}
( it works for me as I do not care if the nested transaction rollbacks from the moment that the main transaction gets committed).
But I don't find the solution sexy.
Also, I'm wondering in somewhere else in our project, we have rollback that happens without us noticing because of this phenomenon (project is pretty old ...)
Is there a way to prevent the automatic marking of transactions to Rollback only in case of a cached ORA error ?
Is there a way to specify something similar to the "noRollbackFor" of the #Transactional attribute ?
Related
i have create a request in SQL and put them in dataset. apparently it hang when the data very huge. so i use an Entity.
my original sql is like this:
SELECT NO_ORDRE,ORDRE.CODE_DEST as CODE_DEST,REF_EXPED,ORDRE.MODAL_MODE,RS_NOM,ADRESSE,TEL,VILLE,
ORDRE.NBR_COLIS,ORDRE.POID,DATE_CREE,DATE_CLOTUR,STATUT_ORDRE,ORDRE.TRANSPORTEUR,ORDRE.LIB_TOURNE,
ORDRE.DATE_CLOTUR_REEL,ORDRE.OBS,AUTRE_REF,
ORDRE.CODE_CLIENT+'_'+CAST(NOID as VARCHAR(50))+'_'+SUBSTRING(NO_ORDRE_CUMMUL, 0, CHARINDEX('_', NO_ORDRE_CUMMUL + '_')) as NOLV
FROM ORDRE
LEFT OUTER JOIN LETTRE_VOIT_FINAL
ON charindex('_'+cast(ORDRE.NO_ORDRE as varchar(255))+'_', '_'+LETTRE_VOIT_FINAL.NO_ORDRE_CUMMUL+'_') > 0
WHERE DATE_CREE BETWEEN #DATE_CREE_DEB AND #DATE_CREE_FIN
ORDER BY NO_ORDRE DESC
and i try my linq like this:
public IQueryable<ORDRE> Get_OrdreEntity(DateTime datedeb, DateTime datefin)
{
try
{
IQueryable<ORDRE> LesListe;
Soft8Exp_ClientEntities oEntite_T = new Soft8Exp_ClientEntities();
var query = from o in oEntite_T.ORDRE
where o.DATE_CREE >= datedeb && o.DATE_CREE <= datefin
select o;
LesListe = query;
return LesListe;
}
catch (Exception excThrown)
{
throw new Exception("Err_02", excThrown);
}
}
it works well but i don't know how to make a join from this sql:
LEFT OUTER JOIN LETTRE_VOIT_FINAL
ON charindex('_'+cast(ORDRE.NO_ORDRE as varchar(255))+'_', '_'+LETTRE_VOIT_FINAL.NO_ORDRE_CUMMUL+'_') > 0
and how can i translate it to linq from this sql:
ORDRE.CODE_CLIENT+'_'+CAST(NOID as VARCHAR(50))+'_'+SUBSTRING(NO_ORDRE_CUMMUL, 0, CHARINDEX('_', NO_ORDRE_CUMMUL + '_')) as NOLV
I can't see any reason to have exception handling in the Get_OrdreEntity function. It should be coded in way it just work. Debug it. In any way you do nothing in catch.
I you query and filter data in this function and want to get results it is a good idea to return collection instead of query in the result of this function to eliminate performance and side effect isssues. I.e. return IEnumerable, ICollection, IList wherether you want.
It is easy to find a ton of Linq join examples, just use Google. Here is all you need.
I have a SQL statement similar to:
SELECT DATEDIFF(Day, startDate, endDate) FROM Data WHERE ProjectId=#id
In the case where Data doesn't have any records for ProjectId, SQL Server returns null.
In Dapper, I execute this via:
value = conn.Query<int>("...").SingleOrDefault()
In this case, I expect the semantics of SingleOrDefault to mean "if this is null, return zero." In fact, my code is even more zero-friendly:
int toReturn = 0;
using (var conn = ...) {
toReturn = conn.Query<int>("SELECT DATEDIFF(...))");
}
return toReturn;
When I debug and step into this code, I find that the line yield return (T)func(reader) is throwing a null pointer exception.
Am I doing something wrong here, or is this by design?
(FYI, the work-around is to wrap my select in an ISNULL(..., 0))
In the case where Data doesn't have any records for ProjectId, SQL Server returns null.
In the case where Data doesn't have any matching records, SQL server does not really return null - it returns no rows. This scenario works fine:
var result = connection.Query<int>( // case with rows
"select DATEDIFF(day, GETUTCDATE(), #date)", new { date = DateTime.UtcNow.AddDays(20) })
.SingleOrDefault();
result.IsEqualTo(20);
result = connection.Query<int>( // case without rows
"select DATEDIFF(day, GETUTCDATE(), #date) where 1 = 0", new { date = DateTime.UtcNow.AddDays(20) })
.SingleOrDefault();
result.IsEqualTo(0); // zero rows; default of int over zero rows is zero
both of which work fine. The fact that you say ISNULL "fixes" it means that you are talking about a different scenario - the "I returned rows" scenario. Since that is the case, what your code is saying is "take this 1-or-more integers which contains a null, and map it as a non-nullable int" - that isn't possible, and the mapper is correct to throw an exception. Instead, what you want is:
int? result = connection.Query<int?>(...).SingleOrDefault();
Now, if there are rows, it is mapping the value to int?, and then applying the single-or-default. It you want that as an int, then maybe:
int result = connection.Query<int?>(...).SingleOrDefault() ?? 0;
If you need to be able to tell the difference between "zero rows" and "null result", then I would suggest:
class NameMe {
public int? Value {get;set;}
}
var row = connection.Query<NameMe>("select ... as [Value] ...", ...)
.SingleOrDefault();
if(row == null) {
// no rows
} else if(row.Value == null) {
// one row, null value
} else {
// one row, non-null value
}
or something similar
I have method where I need to retrieve using EF the last ten entries in the database and check to see if there is a match between the value and the current term. Here is what I have thus far
public static int ValidatePassword(string username, string password, int securityUserId)
{
int validResult = 0;
/*Need to pass to client a value based upon success or failure of validation
* 0 - success
* 1 - password has already been used in the last 10 entries
* 2 - password does not meet CJIS requirements
*/
IEnumerable<string> oldpassword = null;
// Create a Regular Expression to determine whether or not special characters are present.
Regex regularExpression = new Regex("[^a-z0-9]");
//if id exists pull last ten passwords
if (securityUserId > 0)
{
long id = Convert.ToInt64(securityUserId);
using (var context = new SecurityEntities(string.Empty))
{
try
{
oldpassword = (from p in context.SecurityAudits
where p.SecurityUserId == id &&
p.OldPassword == password
orderby p.ActionDate descending
select p.OldPassword.Take(10).ToString()).ToList();
}
catch (Exception ex)
{
string err = string.Format("ValidateCJISPassword() was unable to return an object msg:{0}", ex.Message);
throw new Exception(err, ex.InnerException);
}
finally
{
context.Dispose();
}
}
}
else if (oldpassword == null)
{
//no matching record found now check other requirements
if ((password.Length >= DEFAULT_CJIS_PASSWORD_MIN_LENGTH) && regularExpression.IsMatch(password) && (password != username))
{
//success
validResult = 0;
}
else
{
//password does not meet standard CJIS requirements
validResult = 2;
}
}
else
{
//matching record was found
validResult = 1;
}
return validResult;
}
}
Where I am currently hung up is the query throws an exception on the ToString() method
LINQ to Entities does not recognize the method 'System.String ToString()' method, and this method cannot be translated into a store expression.
I'm still learning EF and how linq works so I'm not sure what the best approach here is. Should I try to set the result to something other than IEnumerable like an array or List or is there another approach I should consider?
Thanks in advance,
Cheers,
Change this
oldpassword = (from p in context.SecurityAudits
where p.SecurityUserId == id &&
p.OldPassword == password
orderby p.ActionDate descending
select p.OldPassword.Take(10).ToString()).ToList();
To this
oldpassword = (from p in context.SecurityAudits
where p.SecurityUserId == id &&
p.OldPassword == password
orderby p.ActionDate descending
select p.OldPassword).Take(10).ToList();
The problem was that your Take(10) clause was not part of the whole result but inside the actual linq statement.. it goes on the outside of it to take the top 10 of the entire resultset.. then you do the ToList() which turns the whole thing into an array
The next problem is that you just created an array and assigned it to oldpassword
I don't see anything here that does anything with the array...
You need to do something like:
declare your array of strings
assign the array to the return of the linq query
evaluate the return for > 0 results
if > 0 then the password has been used in the last 10
if = 0 then new password should be ok, correct?
Now that I have an understanding of what I needed in the query I was able to also update the linq statement as follows:
var lastTenPassword = (from p in context.SecurityAudits.Take(10)
orderby p.ActionDate descending
where p.SecurityUserId == id
select p.OldPassword).ToList();
string oldpassword = lastTenPassword.Where(a => a == password).FirstOrDefault();
Testing is further down the line but now by moving the .Take() method inside the query I am explicitly grabbing the top ten where as my first attempt would have retrieved all the records and then grabbed the top ten.
For testing you can also see where I broke out the initial where() to first grab all records by id and then perform a filter on that set by looking for a matching password within that set.
Thanks again for your help
With Salesforce's Apex, is there any way to sort list items, for Date field value. Please refer the TODO section of the following code, thanks.
/**
* Select list of jobOccurrences belongs to particular list of jobs
*/
private List<Job_Occurrence__c> getJobsByJobCode(List<Job__c> jobList) {
// Select relevant Job Occurrence objects
List<Job_Occurrence__c> jobOccuList = new List<Job_Occurrence__c>();
for (Job__c job : jobList) {
Job_Occurrence__c jobOccurrence = [SELECT Id, Job__c,
Schedule_Start_Date__c
FROM Job_Occurrence__c
WHERE Job__c =: job.Id];
if((jobOccurrence != null) && (jobOccurrence.Id != null)) {
jobOccuList.add(jobOccurrence);
}
}
if((jobOccuList != null) && (jobOccuList.size() > 0)) {
// TODO
// How I sort the 'jobOccuList' with Date field 'Schedule_Start_Date__c',
// for select the items according to sequence of latest jobOccurrence
return jobOccuList;
} else {
throw new RecordNotFoundException ('Could not found any jobOccurrence for given list of jobs');
}
}
You really should bulkify this code, i.e. not run a query inside a loop which could cause potentially cause issues with the governor limits, and since you just want the combined list for all Job__c records this makes your ordering easy too — you can do it in the query!
The code you want to change is this:
// Select relevant Job Occurrence objects
List<Job_Occurrence__c> jobOccuList = new List<Job_Occurrence__c>();
for (Job__c job : jobList) {
Job_Occurrence__c jobOccurrence = [SELECT Id, Job__c,
Schedule_Start_Date__c
FROM Job_Occurrence__c
WHERE Job__c =: job.Id];
if((jobOccurrence != null) && (jobOccurrence.Id != null)) {
jobOccuList.add(jobOccurrence);
}
}
Essentially we can optimise this to not only use one query instead of N (where N is jobList.size()) and get them ordered at the same time. First we need to gather the list of Job__c IDs, and then we can use the IN statement in the WHERE clause of the SOQL:
// Select relevant Job Occurrence objects
List<Job_Occurrence__c> jobOccuList;
Set<Id> setJobIds = new Set<Id>();
setJobIds.addAll(jobList);
// get the job occurances starting with the latest, use ASC for reverse!
jobOccuList = [SELECT Id, Job__c, Schedule_Start_Date__c
FROM Job_Occurrence__c
WHERE Job__c IN : setJobIds
ORDER BY Schedule_Start_Date__c DESC];
Finally, if you need to be able to easily map back from the Job_Occurrence_c records to Job_c records, you could replace the set with a map as below, though given that you just want this list I don't think it's needed here (just providing it for completeness).
Map<Id, Job__c> mapJobs = new Map<Id, Job__c>();
for (Job__c job : jobList) {
mapJobs.put(job.Id, job);
}
** snip **
for (Job_Occurrence__c jobOccu : jobOccuList) {
Job__c theJob = mapJobs.get(jobOccu.Job__c);
// do stuff with the job
}
All of this code has been written in browser, so there may be some syntax errors but it should be good!
You can try :
liDates.sort();
It's working for me.
Cheers
This is a bit of subjective question about a specific situation. Main goal for this question for me is to remind my self to code up the solution. However if there is already a solution, or an alternate approach, I would like to know it.
I'm working on a project and I'm using Entity Framework 4 for database access. The database design is something that I don't have control over. The database was designed many years ago, and in my opinion the database design does not fit for the current database purposes. This results in very complicated queries.
This is the first time I'm using Entity Framework in a project, but I have extensive experience in development against MS SQL Server.
What I found myself doing again and again is this:
I write a complex L2E query. The query either slow or returns wrong results
I'm looking at my L2E query and I have absolutely no idea how to improve it
I fire up SQL Profiler and capture the SQL that EF generated from my query
I want to execute part of that sql to identify the part of the query that is giving problems
The query comes through as sp_executesql with a dozen of parameters, because if a parameter is used 3 times in a query, L2E creates 3 parameters and passes to all of them the same value. Same deal with every parameter.
Now I have to extract the SQL from sp_executesql, unescape all escaped apostrophes, and substitute every parameter in the query with its value
After this is done I finally can run parts of the query and pin-point the problem.
I go back to my L2E code, change it to fix the problem I found and the cycle repeats.
To be honest, I'm starting thinking that one should not use an ORM if you don't own database design.
This aside, the process of unescaping the sql and substituting the parameters is the one that I want to automate. The goal is to get 'naked', de-parametrized sql, that I can run in SSMS.
This a very simple example of what I see in the profile and what I want to get in result. My real cases are many times more complex.
The capture:
exec sp_executesql N'SELECT
[Extent1].[ProductName] AS [ProductName]
FROM [dbo].[Products] AS [Extent1]
INNER JOIN [dbo].[Categories] AS [Extent2] ON [Extent1].[CategoryID] = [Extent2].[CategoryID]
WHERE ([Extent1].[UnitPrice] > #p__linq__0) AND ([Extent2].[CategoryName] = #p__linq__1) AND (N''Chang'' <> [Extent1].[ProductName])',N'#p__linq__0 decimal(1,0),#p__linq__1 nvarchar(4000)',#p__linq__0=1,#p__linq__1=N'Beverages'
Desired result:
SELECT
[Extent1].[ProductName] AS [ProductName]
FROM [dbo].[Products] AS [Extent1]
INNER JOIN [dbo].[Categories] AS [Extent2] ON [Extent1].[CategoryID] = [Extent2].[CategoryID]
WHERE ([Extent1].[UnitPrice] > 1) AND ([Extent2].[CategoryName] = N'Beverages') AND (N'Chang' <> [Extent1].[ProductName])
I'm just going to write code to convert the likes of first to the likes of second if there is nothing better, I'll post the solution here. But maybe it's already done by someone? Or maybe there is a profiler or something, that can give me sql code I can execute partially in SSMS?
So here is what I ended up with. A couple of notes:
This won't work in 100% of cases, but this is good enough for me
There is a lot to improve in terms of usability. Currently I put a shortcut to the compiled binary on the desktop, cut the text to convert to clipboard, double-click the shortcut and paste the result.
using System;
using System.Text.RegularExpressions;
using System.Windows.Forms;
namespace EFC
{
static class Program
{
[STAThread]
static void Main()
{
try
{
string input = Clipboard.GetText();
const string header = "exec sp_executesql N'";
CheckValidInput(input.StartsWith(header), "Input does not start with {0}", header);
// Find part of the statement that constitutes whatever sp_executesql has to execute
int bodyStartIndex = header.Length;
int bodyEndIndex = FindClosingApostroph(input, bodyStartIndex);
CheckValidInput(bodyEndIndex > 0, "Unable to find closing \"'\" in the body");
string body = input.Substring(bodyStartIndex, bodyEndIndex - bodyStartIndex);
// Unescape 's
body = body.Replace("''", "'");
// Work out where the paramters are
int blobEndIndex = FindClosingApostroph(input, bodyEndIndex + 4);
CheckValidInput(bodyEndIndex > 0, "Unable to find closing \"'\" in the params");
string ps = input.Substring(blobEndIndex);
// Reverse, so that P__linq_2 does not get substituted in p__linq_20
Regex regexEf = new Regex(#"(?<name>#p__linq__(?:\d+))=(?<value>(?:.+?)((?=,#p)|($)))", RegexOptions.RightToLeft);
Regex regexLinqToSql = new Regex(#"(?<name>#p(?:\d+))=(?<value>(?:.+?)((?=,#p)|($)))", RegexOptions.RightToLeft);
MatchCollection mcEf = regexEf.Matches(ps);
MatchCollection mcLinqToSql = regexLinqToSql.Matches(ps);
MatchCollection mc = mcEf.Count > 0 ? mcEf : mcLinqToSql;
// substitutes parameters in the statement with their values
foreach (Match m in mc)
{
string name = m.Groups["name"].Value;
string value = m.Groups["value"].Value;
body = body.Replace(name, value);
}
Clipboard.SetText(body);
MessageBox.Show("Done!", "CEF");
}
catch (ApplicationException ex)
{
MessageBox.Show(ex.Message, "Error");
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Error");
MessageBox.Show(ex.StackTrace, "Error");
}
}
static int FindClosingApostroph(string input, int bodyStartIndex)
{
for (int i = bodyStartIndex; i < input.Length; i++)
{
if (input[i] == '\'' && i + 1 < input.Length)
{
if (input[i + 1] != '\'')
{
return i;
}
i++;
}
}
return -1;
}
static void CheckValidInput(bool isValid, string message, params object[] args)
{
if (!isValid)
{
throw new ApplicationException(string.Format(message, args));
}
}
}
}
Well,may be this will be helpfull. MSVS 2010 has IntelliTrace. Every time when EF make a query there is an ADO.Net Event with a query
Execute Reader "SELECT TOP (1)
[Extent1].[id] AS [id],
[Extent1].[Sid] AS [Sid],
[Extent1].[Queue] AS [Queue],
[Extent1].[Extension] AS [Extension]
FROM [dbo].[Operators] AS [Extent1]
WHERE [Extent1].[Sid] = #p__linq__0" Command Text = "SELECT TOP (1) \r\n[Extent1].[id] AS [id], \r\n[Extent1].[Sid] AS [Sid], \r\n[Extent1].[Queue] AS [Queue], \r\n[Extent1].[Extension] AS [Extension]\r\nFROM [dbo].[Operators] AS [Extent1]\r\nWHERE [Extent1].[Sid] = #p__linq__0",
Connection String = "Data Source=paris;Initial Catalog=telephony;Integrated Security=True;MultipleActiveResultSets=True"