Hi I have some problem with group by a result of a custom function in Hibernate.
The query looks like:
#Query("SELECT NEW ReportDay(" +
"FUNCTION('atTimezone', p.startDate, :timezone), "
"SUM(p.energy)) " +
"FROM Process p GROUP BY FUNCTION('atTimezone', p.startDate, :timezone)")
The error thrown is that p.startDate must appear in groupBy clause or in an aggregate function. But the custom function is in the group by clause.
Any ideas? The custom function is this:
registerFunction("atTimezone", new SQLFunctionTemplate(StandardBasicTypes.TIMESTAMP,"date_trunc('day', (cast(?1 as timestamp) AT TIME ZONE cast(?2 as varchar)))"));
and is trying to get the date trunc to day in a specific timezone.
many thanks!
The problem here is that in SQL the two timezone parameters will be separate parameters and your database (at least PostgreSQL) won't be able to recognize that these are actually the same value.
In order to work around this issue, you can give your function an alias which can be referenced in the GROUP BY clause. Select aliases are not allowed in the GROUP BY clause according to ANSI SQL and JPQL. Some database dialects such as PostgreSQL, MySQL and H2 however do support it and since Hibernate 5.4.10 aliases in the group by clause are propagated to the SQL query for these supporting dialects (HHH-9301).
So instead of:
SELECT FUNCTION('atTimezone', p.startDate, :timezone), SUM(p.energy))
FROM Process p GROUP BY FUNCTION('atTimezone', p.startDate, :timezone)
You could do:
SELECT FUNCTION('atTimezone', p.startDate, :timezone) AS d, SUM(p.energy))
FROM Process p GROUP BY d
How this works together with object projection (NEW ReportDay(...)) is not known to me, but this should at least get you started.
Related
I am trying to convert a "raw" PostGIS SQL query into a Rails ActiveRecord query. My goal is to convert two sequential ActiveRecord queries (each taking ~1ms) into a single ActiveRecord query taking (~1ms). Using the SQL below with ActiveRecord::Base.connection.execute I was able to validate the reduction in time.
Thus, my direct request is to help me to convert this query into an ActiveRecord query (and the best way to execute it).
SELECT COUNT(*)
FROM "users"
INNER JOIN (
SELECT "centroid"
FROM "zip_caches"
WHERE "zip_caches"."postalcode" = '<postalcode>'
) AS "sub" ON ST_Intersects("users"."vendor_coverage", "sub"."centroid")
WHERE "users"."active" = 1;
NOTE that the value <postalcode> is the only variable data in this query. Obviously, there are two models here User and ZipCache. User has no direct relation to ZipCache.
The current two step ActiveRecord query looks like this.
zip = ZipCache.select(:centroid).where(postalcode: '<postalcode>').limit(1).first
User.where{st_intersects(vendor_coverage, zip.centroid)}.count
Disclamer: I've never used PostGIS
First in your final request, it seems like you've missed the WHERE "users"."active" = 1; part.
Here is what I'd do:
First add a active scope on user (for reusability)
scope :active, -> { User.where(active: 1) }
Then for the actual query, You can have the sub query without executing it and use it in a joins on the User model, such as:
subquery = ZipCache.select(:centroid).where(postalcode: '<postalcode>')
User.active
.joins("INNER JOIN (#{subquery.to_sql}) sub ON ST_Intersects(users.vendor_coverage, sub.centroid)")
.count
This allow minimal raw SQL, while keeping only one query.
In any case, check the actual sql request in your console/log by setting the logger level to debug.
The amazing tool scuttle.io is perfect for converting these sorts of queries:
User.select(Arel.star.count).where(User.arel_table[:active].eq(1)).joins(
User.arel_table.join(ZipCach.arel_table).on(
Arel::Nodes::NamedFunction.new(
'ST_Intersects', [
User.arel_table[:vendor_coverage], Sub.arel_table[:centroid]
]
)
).join_sources
)
I have a simple table (SQL server and EF6) Myvalues, with columns Id & Value (double)
I'm trying to get the sum of the natural log of all values in this table. My LINQ statement is:
var sum = db.Myvalues.Select(x => Math.Log(x.Value)).Sum();
It compiles fine, but I'm getting a RTE:
LINQ to Entities does not recognize the method 'Double Log(Double)' method, and this method cannot be translated into a store expression.
What am I doing wrong/how can I fix this?
FWIW, I can execute the following SQL query directly against the database which gives me the correct answer:
select exp(sum(LogCol)) from
(select log(Myvalues.Value) as LogCol From Myvalues
) results
LINQ tries to translate Math.Log into a SQL command so it is executed against the DB.
This is not supported.
The first solution (for SQL Server) is to use one of the existing SqlFunctions. More specifically, SqlFunctions.Log.
The other solution is to retrieve all your items from your DB using .ToList(), and execute Math.Log with LINQ to Objects (not LINQ to Entities).
As EF cannot translate Math.Log() you could get your data in memory and execute the function form your client:
var sum = db.Myvalues.ToList().Select(x => Math.Log(x.Value)).Sum();
Ok, I feel really stupid for asking this, but it's driving me nuts and I can't figure it out. The docs say I should be able to use select AS in a Rails/ActiveRecord query. So:
d = Dvd.where(id: 1).select("title AS my_title")
Is a valid query and if I do a to_sql on it, it produces the expected SQL:
SELECT title AS my_title FROM `dvd` WHERE `dvd`.`id` = 1
However, d.my_title will give an error:
NoMethodError: undefined method `my_title' for #<ActiveRecord::Relation
I need to be able to use AS since the columns I want to retrieve from different joins have the same name so I can't access them the "regular" way and have to resort to using AS.
I also don't want to resort to using find_by_sql for future compatibility and a possible switch form Mysql to PostGresql.
Just to clarify, what I'm really trying to do is write this SQL in a Railsy way:
SELECT tracks.name AS track_name, artists.name AS artist_name, composers.name AS composer_name, duration
FROM `tracks_cds`
INNER JOIN `tracks` ON `tracks`.`id` = `tracks_cds`.`track_id`
INNER JOIN `artists` ON `artists`.`id` = `tracks_cds`.`artist_id`
INNER JOIN `composers` ON `composers`.`id` = `tracks_cds`.`composer_id`
WHERE cd_id = cd.id
The top example was just a simplification of the fact that SELECT AS will not give you an easy way to refer to custom fields which I find hard to believe.
ActiveRecord automatically creates getter and setter methods for attributes based on the column names in the database, so there will be none defined for my_title.
Regarding the same common names, why not just do this:
d = Dvd.where(id: 1).select("dvds.title")
You can write your sql query and then just pass into ActiveRecord's execute method
query = "SELECT title AS my_title FROM `dvd` WHERE `dvd`.`id` = 1"
result = ActiveRecord::Base.connection.execute(query)
Can anyone explain to me why I get a 00904 error when I run the following
SELECT "OASM"."DT_GROUPEPG".GROUPEPGID,
"OASM"."DT_GROUPEPG".GROUPID,
"OASM"."DT_GROUPEPG".EPGID,
"OASM"."DT_GROUPEPG".ZAPID,
"OASM"."LU_EPG".LASTREADTIME,
"OASM"."LU_EPG".SERVICE_NAME,
"OASM"."LU_EPG".SOURCE_ID,
"OASM"."LU_EPG".ONID,
"OASM"."LU_EPG".TSID,
"OASM"."LU_EPG".SID,
"OASM"."LU_EPG".TYPE_ID,
"OASM"."LU_EPG".OPERATOR_ID,
"OASM"."LU_EPG".URL
FROM "OASM"."DT_GROUPEPG"
INNER JOIN "OASM"."LU_EPG"
ON "OASM"."DT_GROUPEPG".EPGID = "OASM"."LU_EPG".EPGID
ORDER BY LastReadTime;
I'm still new to Oracle, and was of the impression that, because Oracle executes blocks of statements, and not line by line, that doing this kind of query would be valid? The error currently fires at the OPERATOR_ID line, but removing/commenting it out just moves the erro up a line, until all the LU_EPG table references are removed
You can't (and don't need to) specify the schema name when referring to the columns. Also, I recommend you use table aliases (e.g. a and b in the example below):
SELECT a.GROUPEPGID,
a.GROUPID,
a.EPGID,
a.ZAPID,
b.LASTREADTIME,
b.SERVICE_NAME,
b.SOURCE_ID,
b.ONID,
b.TSID,
b.SID,
b.TYPE_ID,
b.OPERATOR_ID,
b.URL
FROM "OASM"."DT_GROUPEPG" a
INNER JOIN "OASM"."LU_EPG" b
ON a.EPGID = b.EPGID
ORDER BY b.LastReadTime;
I have a LINQ query which is attempting to get all of the distinct months of all of the dates in a table.
I had this working using the Distinct() extension method. I then made it more readable by using an extension method to extract the month. And then it stopped returning Distinct results.
Can anyone help me work out what happened here?
As an aside, if someone can tell me the best way to get the distinct months, that would be nice too. But it's more important that I understand why this is failing.
Here's the code.
static class DcUtils
{
public static DateTime GetMonth(this Timesheet_Entry entry)
{
DateTime dt = new DateTime(
entry.Entry_Start_DateTime.Year,
entry.Entry_Start_DateTime.Month,
1
);
return dt;
}
}
public class Demo
{
public DemonstrateBug()
{
TimesheetDataClassesDataContext dc = new TimesheetDataClassesDataContext();
/////////////////////////////////
//// Here are the queries and their behaviours
var q1 = (
from ts
in dc.Timesheet_Entries
select new DateTime(ts.Entry_Start_DateTime.Year, ts.Entry_Start_DateTime.Month, 1)
).Distinct();
// This returns 3 (which is what I want)
int lengthQuery1 = q1.Count();
// And now for the bug!
var q2 = (
from ts
in dc.Timesheet_Entries
select ts.GetMonth()
).Distinct();
// This returns 236 (WTF?)
int lengthQuery2 = q2.Count();
}
}
LINQ to SQL is smart enough to convert the new DateTime() expression from your initial lambda expression into a SQL statements that can be executed at the server. If you replace this expression with an (extension) method, LINQ to SQL will only see a call to an opaque method it knows nothing about, hence it cannot generate any SQL for the method call and the part of the SQL query messing with the dates disappears.
But this shouldn't break anything - what cannot be transformed into SQL must be executed at the client. So what happens? The date you want to perform the distinct operation on cannot be calculated at the server because of the opaque method call, hence the distinct operation cannot be performed at the server, too. But the query you recorded from the broken version contains a DISTINCT statement.
I don't use the LINQ query syntax, but I assume you have written something you don't actually mean or the compiler or LINQ to SQL inferred something you didn't mean.
context
.Timesheet_Entries
.Select(tse => tse.GetMonth())
.Distinct()
versus
context
.Timesheet_Entries
.Distinct()
.Select(tse => tse.GetMonth())
So I guess you got the second one for what ever reason - the distinct operation seems to get propagated over the select. Maybe it's the combination of Distinct() with the query syntax and the contained opaque method call. Try both versions without the query syntax and see what you get back and what queries are send to the server. You can also try to insert ToList() calls to force the transition from LINQ to SQL to LINQ to Objects - this might help to cast some light onto the situation, too.
It occurred to me to run this through the SQL Server Profiler.
This query:
var q1 = (
from ts
in dc.Timesheet_Entries
select new DateTime(ts.Entry_Start_DateTime.Year,
ts.Entry_Start_DateTime.Month,
1)
).Distinct();
generates the following SQL. As you can see, it converted the System.DateTime calls into Transact SQL.
SELECT DISTINCT
[t1].[value]
FROM (
SELECT
CONVERT(
DATETIME,
CONVERT(
NCHAR(2),
DATEPART(
Month,
[t0].[Entry_Start_DateTime]
)
)
+ (''/'' + (CONVERT(NCHAR(2), #p0)
+ (''/'' + CONVERT(NCHAR(4),
DATEPART(
Year,
[t0].[Entry_Start_DateTime]
)
)
))), 101
) AS [value]
FROM [dbo].[Timesheet_Entry] AS [t0]
) AS [t1]
But if I put the month extraction logic in the extension method:
var q2 = (
from ts
in dc.Timesheet_Entries
select ts.GetMonth()
).Distinct();
It generats the following SQL.
SELECT DISTINCT
[t0].[Timesheet_Entry_ID],
[t0].[Entry_Start_DateTime],
[t0].[Entry_End_DateTime],
[t0].[Task_Description],
[t0].[Customer_ID]
FROM [dbo].[Timesheet_Entry] AS [t0]
So it's moved the DISTINCT function to the server, but kept the date extraction code until after the DISTINCT operation, which is not what I want, and is not what happens in the first example.
I don't know if I should call this a bug or a leaky abstraction.