I am trying to implement the same in redshift and i am finding it little difficult to do that. Since redshift is in top of postgresql engine, if any one can do it in postgresql it would be really helpfull. Basically the code gets the count for previous two month at column level. If there is no count for exact previous month then it gives 0.
This is my code:
with abc(dateval,cnt) as(
select 201908, 100 from dual union
select 201907, 200 from dual union
select 201906, 300 from dual union
select 201904, 600 from dual)
select dateval, cnt,
last_value(cnt) over (order by dateval
range between interval '1' month preceding
and interval '1' month preceding ) m1,
last_value(cnt) over (order by dateval
range between interval '2' month preceding
and interval '2' month preceding ) m2
from (select to_date(dateval, 'yyyymm') dateval, cnt from abc)
I get error in over by clause. I tried to give cast('1 month' as interval) but still its failing. Can someone please help me with this windows function.
expected output:
Regards
This is how I would do it. In Redshift there's no easy way to generate sequences, do I select row_number() from an arbitrary table to create a sequence:
with abc(dateval,cnt) as(
select 201908, 100 union
select 201907, 200 union
select 201906, 300 union
select 201904, 600),
cal(date) as (
select
add_months(
'20190101'::date,
row_number() over () - 1
) as date
from <an arbitrary table to generate a sequence of rows> limit 10
),
with_lag as (
select
dateval,
cnt,
lag(cnt, 1) over (order by date) as m1,
lag(cnt, 2) over (order by date) as m2
from abc right join cal on to_date(dateval, 'YYYYMM') = date
)
select * from with_lag
where dateval is not null
order by dateval
Related
I have the following recursive CTE which splits each element coming from base per month:
with
base (id, start_date, end_date) as (
select 1, date '2022-01-15', date '2022-03-15' from dual
union
select 2, date '2022-09-15', date '2022-12-31' from dual
union
select 3, date '2023-09-15', date '2023-09-25' from dual
),
split (id, start_date, end_date) as (
select base.id, base.start_date, least(last_day(base.start_date), base.end_date) from base
union all
select base.id, split.end_date + 1, least(last_day(split.end_date + 1), base.end_date) from base join split on base.id = split.id and split.end_date < base.end_date
)
select * from split order by id, start_date, end_date;
It works on Oracle and gives the following result:
id
start_date
end_date
1
2022-01-15
2022-01-31
1
2022-02-01
2022-02-28
1
2022-03-01
2022-03-15
2
2022-09-15
2022-09-30
2
2022-10-01
2022-10-31
2
2022-11-01
2022-11-30
2
2022-12-01
2022-12-31
3
2023-09-15
2023-09-25
The two following stop conditions work correctly:
... from base join split on base.id = split.id and split.end_date < base.end_date
... from base, split where base.id = split.id and split.end_date < base.end_date
The following one fails with the message ORA-32044: cycle detected while executing recursive WITH query:
... from base join split on base.id = split.id where split.end_date < base.end_date
I fail to understand how the last one is different from the two others.
It looks like a bug as all your queries should result in identical explain plans.
However, you can rewrite the recursive sub-query without the join (and using a SEARCH clause so you may not have to re-order the query later):
WITH split (id, start_date, month_end, end_date) AS (
SELECT id,
start_date,
LEAST(
ADD_MONTHS(TRUNC(start_date, 'MM'), 1) - INTERVAL '1' SECOND,
end_date
),
end_date
FROM base
UNION ALL
SELECT id,
month_end + INTERVAL '1' SECOND,
LEAST(
ADD_MONTHS(month_end, 1),
end_date
),
end_date
FROM split
WHERE month_end < end_date
) SEARCH DEPTH FIRST BY id, start_date SET order_id
SELECT id,
start_date,
month_end AS end_date
FROM split;
Note: if you want to just use values at midnight rather than the entire month then use INTERVAL '1' DAY rather than 1 second.
Which, for the sample data:
CREATE TABLE base (id, start_date, end_date) as
select 1, date '2022-01-15', date '2022-04-15' from dual union all
select 2, date '2022-09-15', date '2022-12-31' from dual union all
select 3, date '2023-09-15', date '2023-09-25' from dual;
Outputs:
ID
START_DATE
END_DATE
1
2022-01-15T00:00:00Z
2022-01-31T23:59:59Z
1
2022-02-01T00:00:00Z
2022-02-28T23:59:59Z
1
2022-03-01T00:00:00Z
2022-03-31T23:59:59Z
1
2022-04-01T00:00:00Z
2022-04-15T00:00:00Z
2
2022-09-15T00:00:00Z
2022-09-30T23:59:59Z
2
2022-10-01T00:00:00Z
2022-10-31T23:59:59Z
2
2022-11-01T00:00:00Z
2022-11-30T23:59:59Z
2
2022-12-01T00:00:00Z
2022-12-31T00:00:00Z
3
2023-09-15T00:00:00Z
2023-09-25T00:00:00Z
fiddle
It's because WHERE and ON conditions are not evaluated at the same level:
when the condition is in the ON clause it's limiting the rows concerned by the JOIN, where it's in the WHERE it's filtering the results after the JOIN has been applied, and since a recursive CTE see all rows selected up to now...
Table looks like below:
Expected answer below:
Write an oracle query to fix space and return hh:mm format. Consider same time in a same group and return total count
The WITH clause in the query below is just for testing; remove it, and use the actual table and column names in the main query. Note: count is a reserved keyword, so it can't be a column name. I changed it to count_ (with an underscore).
with
test_data (srt_tm, count_) as (
select '1:00' , 125 from dual union all
select '01:00' , 19000 from dual union all
select ' 01:00', 27 from dual union all
select '4:00' , 22000 from dual union all
select '04:00' , 1800 from dual union all
select ' 04:00', 15000 from dual
)
-- END OF TEST DATA; ACTUAL QUERY BEGINS **BELOW THIS LINE**
select lpad(trim(srt_tm), 5, '0') as srt_tm, sum(count_) as count_
from test_data
group by lpad(trim(srt_tm), 5, '0')
order by srt_tm
;
SRT_TM COUNT_
------ --------
01:00 19152
04:00 38800
I have data for a member like below
EFF_DT-Term_dt
1/1/13-7/31/14
1/1/15-3/31/15
5/1/15-5/31/15
6/1/15-12/31/15
1/1/16-12/31/16
Here there are 2 gaps - after 7/31/14 and 3/31/15. I want to select the row 5/1/15-5/31/15 as it is the minimum date after maximum gap. I tried using
select ( FIRST_VALUE(EFF_DT) OVER (PARTITION BY MemberID ORDER BY FLAG DESC) AS CUR_EFF_DT)
from
(
select EFF_DT,
CASE WHEN LAG(TERM_DT, 1) OVER (PARTITION BY MemberID ORDER BY TERM_DT) = EFF_DT - 1 THEN 0
ELSE sequence.nextval
END AS FLAG
from effective_dates_table).
This is giving correct result, but i don't want to use sequence Is there any other easiest way to do this?
Here's one way... compute the differences using the analytic LAG() function, then group by member_id and use the aggregate LAST() function.
NOTE: there may be more than one pair of rows with the same, greatest gap between term_dt and the following eff_dt. You must clarify which row should be selected if that happens. The solution below picks the earliest occurrence (if this happens). If you want the latest occurrence, change MIN to MAX. If you want something else, just say what the requirement is.
with
inputs ( member_id, eff_dt, term_dt ) as (
select 101, to_date('1/1/13', 'mm/dd/yy'), to_date('7/31/14' , 'mm/dd/yy') from dual union all
select 101, to_date('1/1/15', 'mm/dd/yy'), to_date('3/31/15' , 'mm/dd/yy') from dual union all
select 101, to_date('5/1/15', 'mm/dd/yy'), to_date('5/31/15' , 'mm/dd/yy') from dual union all
select 101, to_date('6/1/15', 'mm/dd/yy'), to_date('12/31/15', 'mm/dd/yy') from dual union all
select 101, to_date('1/1/16', 'mm/dd/yy'), to_date('12/31/16', 'mm/dd/yy') from dual
)
-- End of simulated inputs (for testing only, not part of the solution).
-- Use your actual table and column names in the SQL query below.
select member_id,
min(eff_dt) keep (dense_rank last order by diff nulls first) as eff_dt,
min(term_dt) keep (dense_rank last order by diff nulls first) as term_dt
from (
select member_id, eff_dt, term_dt,
eff_dt - lag(term_dt) over (partition by member_id order by eff_dt) as diff
from inputs
)
group by member_id
;
MEMBER_ID EFF_DT TERM_DT
--------- ------------------- -------------------
101 2015-01-01 00:00:00 2015-03-31 00:00:00
Using Oracle 12c, I need to run a script on an existing summary table of projects. The summary table has a project, a start date, and an end date. I need to break this data out into the number of days per month for each project.
An example is Project A has a start date of 2/10/2016 and an end date of 3/10/2016. My ending result for this example should be:
Project A, February, 19
Project A, March, 10
This was an easier one as some dates may span 2 or 3 months. This doesn't seem too difficult but for some reason I'm having trouble wrapping my head around it and overthinking it. Does someone have an quick and easy solution to this? I would like to run this as a SQL statement but a PL/SQL script would also work.
In this solution we don't assume any prior knowledge of the time period covered. Also, this solution does not use joins (which may be important for performance).
with
-- begin test data (this section can be deleted)
inputs ( project, start_date, end_date ) as (
select 'A', date '2014-10-03', date '2014-12-15' from dual union all
select 'B', date '2015-03-01', date '2015-03-31' from dual union all
select 'C', date '2015-11-30', date '2016-03-01' from dual
),
-- end test data; solution begins here (it includes the word "with" from the first line)
prep ( project, end_date, dt ) as (
select project, end_date, start_date from inputs union all
select project, end_date, end_date + 1 from inputs union all
select project, end_date, add_months( trunc(start_date, 'mm'), level )
from inputs
connect by add_months (trunc(start_date, 'mm'), level) <= end_date
and prior project = project
and prior sys_guid() is not null
),
computations ( project, dt, month, day_count ) as (
select project, dt, to_char(dt, 'Mon-yyyy'),
lead(dt) over (partition by project order by dt) - dt
from prep
where dt <= end_date + 1
)
select project, month, day_count
from computations
where day_count > 0
order by project, dt
;
OUTPUT:
PROJECT MONTH DAY_COUNT
------- -------- ---------
A Oct-2014 29
A Nov-2014 30
A Dec-2014 15
B Mar-2015 31
C Nov-2015 1
C Dec-2015 31
C Jan-2016 31
C Feb-2016 29
C Mar-2016 1
9 rows selected
If you can do an assumption on the maximum number of days for a project (1000 in my example), you can use the following:
with yourTable(project, startDate, endDate) as
(
select 'Project a' as project,
date '2016-02-10' as startDate,
date '2016-03-10' as endDate
from dual
UNION ALL
select 'Project XX',
date '2016-01-01',
date '2016-01-10'
from dual
)
select project, to_char(startDate + n, 'MONTH'), count(1)
from yourTable
inner join (
select level n
from dual
connect by level <= 1000
)
on (startDate + n <= endDate)
group by project, to_char(startDate + n, 'MONTH')
The part with the CONNECT BY is used as a date generator, assuming that every project is at maximum 1000 days long; the external query uses the date generator to split the row of a project in many rows, one for each day between start and end date, and then aggregates by month and project to build the output.
A slightly different way, based on months and not days, could be:
select project, to_char(add_months(startDate, n ), 'MONTH'),
case
when trunc(add_months(startDate, n ), 'MONTH') = trunc(add_months(endDate, n ), 'MONTH')
then endDate - startDate +1
when trunc(add_months(startDate, n ), 'MONTH') <= startDate
then last_day(add_months(startDate, n)) - startDate
when last_day(add_months(startDate, n )) >= endDate
then endDate - trunc(add_months(startDate, n ), 'MONTH') +1
else
last_day(add_months(startDate, n )) - trunc(last_day(add_months(startDate, n )), 'MONTH')
end as numOfDays
from yourTable
inner join (
select level -1 n
from dual
connect by level <= 1000
)
on trunc(add_months(startDate, n ), 'MONTH') <= trunc(endDate, 'MONTH')
This is a bit more complicated, to handle the different cases, but more efficient, given that it works at month level, not day level
I think you're after something like:
WITH sample_data AS (SELECT 'A' PROJECT, to_date('10/02/2016', 'dd/mm/yyyy') start_date, to_date('10/03/2016', 'dd/mm/yyyy') end_date FROM dual UNION ALL
SELECT 'B' PROJECT, to_date('10/02/2016', 'dd/mm/yyyy') start_date, to_date('10/06/2016', 'dd/mm/yyyy') end_date FROM dual UNION ALL
SELECT 'C' PROJECT, to_date('10/02/2016', 'dd/mm/yyyy') start_date, to_date('18/02/2016', 'dd/mm/yyyy') end_date FROM dual)
SELECT PROJECT,
to_char(add_months(trunc(start_date, 'mm'), LEVEL -1), 'fmMonth yyyy', 'nls_date_language=english') mnth,
CASE WHEN trunc(end_date, 'mm') = add_months(trunc(start_date, 'mm'), LEVEL -1)
THEN end_date
ELSE add_months(trunc(start_date, 'mm'), LEVEL) -1
END - CASE WHEN trunc(start_date, 'mm') = add_months(trunc(start_date, 'mm'), LEVEL -1)
THEN start_date + 1
ELSE add_months(trunc(start_date, 'mm'), LEVEL -1)
END + 1 num_days
FROM sample_data
CONNECT BY PRIOR PROJECT = PROJECT
AND PRIOR sys_guid() IS NOT NULL
AND add_months(trunc(start_date, 'mm'), LEVEL -1) <= TRUNC(end_date, 'mm');
PROJECT MNTH NUM_DAYS
------- -------------- ----------
A February 2016 19
A March 2016 10
B February 2016 19
B March 2016 31
B April 2016 30
B May 2016 31
B June 2016 10
C February 2016 8
This uses the multi-row connect-by-level technique (the presence of the and prior sys_guid() is not null enables the connect by to loop through each row separately) to loop through each project row in the sample_data table (you presumably have the project information in a table already, so you wouldn't need to have the sample_data subquery at all; you could just reference your table directly in the main SQL).
We then compare the month of the start date with the month of the row being generated by the connect by, and if it's the same month, then we know we need to use the start date, otherwise we use the first of the month of the generated row; we do similarly for the end date.
That way, we can now subtract one from the other and make adjustments to make the calculation correct. You may need to tweak this yourself if you need a start and end date of the same day to count as 1 day, rather than 0 - it'll probably need an extra case statement to take account of when the start and end date are in the same month.
Using this approach won't limit your project length; it could be as long as you liked.
ETA: Looks like Mathguy posted an answer whilst I was typing out my answer, and whilst our basic methods are the same, mine doesn't use an analytic function to determine the difference in the number of days. You may or may not find their answer more performant than mine - you should test both to see which one works best with your data.
The question I need to answer is this "What is the maximum number of page requests we have ever received in a 60 minute period?"
I have a table that looks similar to this:
date_page_requested date;
page varchar(80);
I'm looking for the MAX count of rows in any 60 minute timeslice.
I thought analytic functions might get me there but so far I'm drawing a blank.
I would love a pointer in the right direction.
You have some options in the answer that will work, here is one that uses Oracle's "Windowing Functions with Logical Offset" feature instead of joins or correlated subqueries.
First the test table:
Wrote file afiedt.buf
1 create table t pctfree 0 nologging as
2 select date '2011-09-15' + level / (24 * 4) as date_page_requested
3 from dual
4* connect by level <= (24 * 4)
SQL> /
Table created.
SQL> insert into t values (to_date('2011-09-15 11:11:11', 'YYYY-MM-DD HH24:Mi:SS'));
1 row created.
SQL> commit;
Commit complete.
T now contains a row every quarter hour for a day with one additional row at 11:11:11 AM. The query preceeds in three steps. Step 1 is to, for every row, get the number of rows that come within the next hour after the time of the row:
1 with x as (select date_page_requested
2 , count(*) over (order by date_page_requested
3 range between current row
4 and interval '1' hour following) as hour_count
5 from t)
Then assign the ordering by hour_count:
6 , y as (select date_page_requested
7 , hour_count
8 , row_number() over (order by hour_count desc, date_page_requested asc) as rn
9 from x)
And finally select the earliest row that has the greatest number of following rows.
10 select to_char(date_page_requested, 'YYYY-MM-DD HH24:Mi:SS')
11 , hour_count
12 from y
13* where rn = 1
If multiple 60 minute windows tie in hour count, the above will only give you the first window.
This should give you what you need, the first row returned should have
the hour with the highest number of pages.
select number_of_pages
,hour_requested
from (select to_char(date_page_requested,'dd/mm/yyyy hh') hour_requested
,count(*) number_of_pages
from pages
group by to_char(date_page_requested,'dd/mm/yyyy hh')) p
order by number_of_pages
How about something like this?
SELECT TOP 1
ranges.date_start,
COUNT(data.page) AS Tally
FROM (SELECT DISTINCT
date_page_requested AS date_start,
DATEADD(HOUR,1,date_page_requested) AS date_end
FROM #Table) ranges
JOIN #Table data
ON data.date_page_requested >= ranges.date_start
AND data.date_page_requested < ranges.date_end
GROUP BY ranges.date_start
ORDER BY Tally DESC
For PostgreSQL, I'd first probably write something like this for a "window" aligned on the minute. You don't need OLAP windowing functions for this.
select w.ts,
date_trunc('minute', w.ts) as hour_start,
date_trunc('minute', w.ts) + interval '1' hour as hour_end,
(select count(*)
from weblog
where ts between date_trunc('minute', w.ts) and
(date_trunc('minute', w.ts) + interval '1' hour) ) as num_pages
from weblog w
group by ts, hour_start, hour_end
order by num_pages desc
Oracle also has a trunc() function, but I'm not sure of the format. I'll either look it up in a minute, or leave to see a friend's burlesque show.
WITH ranges AS
( SELECT
date_page_requested AS StartDate,
date_page_requested + (1/24) AS EndDate,
ROWNUMBER() OVER(ORDER BY date_page_requested) AS RowNo
FROM
#Table
)
SELECT
a.StartDate AS StartDate,
MAX(b.RowNo) - a.RowNo + 1 AS Tally
FROM
ranges a
JOIN
ranges b
ON a.StartDate <= b.StartDate
AND b.StartDate < a.EndDate
GROUP BY a.StartDate
, a.RowNo
ORDER BY Tally DESC
or:
WITH ranges AS
( SELECT
date_page_requested AS StartDate,
date_page_requested + (1/24) AS EndDate,
ROWNUMBER() OVER(ORDER BY date_page_requested) AS RowNo
FROM
#Table
)
SELECT
a.StartDate AS StartDate,
( SELECT MIN(b.RowNo) - a.RowNo
FROM ranges b
WHERE b.StartDate > a.EndDate
) AS Tally
FROM
ranges a
ORDER BY Tally DESC