i have a table in hive with two columns: session_id and duration_time like this:
|| session_id || duration||
1 14
1 10
1 20
1 10
1 12
1 16
1 8
2 9
2 6
2 30
2 22
i want to add a new column with unique id when:
the session_id is changing or the duration_time > 15
i want the output to be like this:
session_id duration unique_id
1 14 1
1 10 1
1 20 2
1 10 2
1 12 2
1 16 3
1 8 3
2 9 4
2 6 4
2 30 5
2 22 6
any ideas how to do that in hive QL?
thanks!
SQL tables represent unordered sets. You need a column specifying the ordering of the values, because you seem to care about the ordering. This could be an id column or a created-at column, for instance.
You can do this using a cumulative sum:
select t.*,
sum(case when duration > 15 or seqnum = 1 then 1 else 0 end) over
(order by ??) as unique_id
from (select t.*,
row_number() over (partition by session_id order by ??) as seqnum
from t
) t;
Related
SQL> select ID,rank() over(order by ID) RANK , dense_rank() over(order by ID) DENSE_RANK from a;
ID RANK DENSE_RANK
1 1 1
1 1 1
2 3 2
3 4 3
3 4 3
4 6 4
5 7 5
5 7 5
6 9 6
7 10 7
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0
I expect I'm just missing something, but if I run this query without the "connect by", I get 2 rows. When I add "connect by level <= 4", I would expect to get each of those 2 rows 4 times. The actual result is different.
Can anyone help me understand what's happening here? I'm not looking for a solution that only repeats each row 4 times - I've already got that. I'm just looking to understand what's happening and why.
with alpha as (
select 1 as id
from dual
),
beta as (
select 1 as alpha_id,
1 as beta_no
from dual
union all
select 1 as alpha_id,
2 as beta_no
from dual
)
select a.id,
b.beta_no,
level as the_level
from alpha a
inner join beta b
on b.alpha_id = a.id
connect by level <= 4
order by a.id,
b.beta_no,
level
;
ID BETA_NO THE_LEVEL
1 1 1
1 1 2
1 1 2
1 1 3
1 1 3
1 1 3
1 1 3
1 1 4
1 1 4
1 1 4
1 1 4
1 1 4
1 1 4
1 1 4
1 1 4
1 2 1
1 2 2
1 2 2
1 2 3
1 2 3
1 2 3
1 2 3
1 2 4
1 2 4
1 2 4
1 2 4
1 2 4
1 2 4
1 2 4
1 2 4
30 rows selected
Many thanks to mathguy. The second link he provided in the answer below had exactly what I was looking for. Specifically:
1 with t as (select 1 as id from dual union all
2 select 2 from dual)
3 --
4 select id, level
5 ,prior id
6 ,sys_connect_by_path(id,'=>') as cpath
7 from t
8* connect by level <= 3
SQL> /
ID LEVEL PRIORID CPATH
---------- ---------- ---------- --------------------------------------------------
1 1 =>1
1 2 1 =>1=>1
1 3 1 =>1=>1=>1
2 3 1 =>1=>1=>2
2 2 1 =>1=>2
1 3 2 =>1=>2=>1
2 3 2 =>1=>2=>2
2 1 =>2
1 2 2 =>2=>1
1 3 1 =>2=>1=>1
2 3 1 =>2=>1=>2
2 2 2 =>2=>2
1 3 2 =>2=>2=>1
2 3 2 =>2=>2=>2
14 rows selected.
It's clear to me from that example, but I'd be hard-pressed to succinctly put it into words.
With no condition other than "level <= 4", every row from the original table, view etc. (from the join, in this case) will produce two rows at level 2, then four more rows at level 3, and 8 more at level 4. "Connect by" is essentially a succession of joins, and you are doing cross joins if you have no condition with the PRIOR operator.
You probably want to add "and prior a.id = a.id". This will lead to Oracle complaining about cycles (because Oracle decides a cycle is reached when it sees the same values in the columns subject to PRIOR). That, in turn, is solved by adding a third condition, usually "and prior sys_guid() is not null".
(Edited; the original answer made reference to NOCYCLE, which is not needed when using the "prior sys_guid() is not null" approach.)
This has been discussed recently on OTN: https://community.oracle.com/thread/3999985
Same question discussed here: https://community.oracle.com/thread/2526535
To illustrate Mathguy's answer, you are missing some predicates out of your CONNECT BY clause:
with alpha as (
select 1 as id
from dual
),
beta as (
select 1 as alpha_id,
1 as beta_no
from dual
union all
select 1 as alpha_id,
2 as beta_no
from dual
)
select a.id,
b.beta_no,
level as the_level
from alpha a
inner join beta b
on b.alpha_id = a.id
connect by level <= 4
AND PRIOR a.id = a.id
AND PRIOR b.beta_no = b.beta_no
AND PRIOR sys_guid() IS NOT NULL
order by a.id,
b.beta_no,
LEVEL;
ID BETA_NO THE_LEVEL
---------- ---------- ----------
1 1 1
1 1 2
1 1 3
1 1 4
1 2 1
1 2 2
1 2 3
1 2 4
An alternative would be to use the recursive with clause:
with alpha as (
select 1 as id
from dual
),
beta as (
select 1 as alpha_id,
1 as beta_no
from dual
union all
select 1 as alpha_id,
2 as beta_no
from dual
),
multiply (id, beta_no, rn) AS (SELECT a.id,
b.beta_no,
1 rn
FROM alpha a
INNER JOIN beta b
ON a.id = b.alpha_id
UNION ALL
SELECT ID,
beta_no,
rn + 1
FROM multiply
WHERE rn + 1 <= 4)
SELECT ID,
beta_no,
rn AS the_level
FROM multiply
order by id,
beta_no,
rn;
ID BETA_NO THE_LEVEL
---------- ---------- ----------
1 1 1
1 1 2
1 1 3
1 1 4
1 2 1
1 2 2
1 2 3
1 2 4
I am having trouble solve. I am suppose to be getting a record every time there is a change to an account in our data warehouse, but I am only receiving one. The table below is a sample of what I am working with.
Row Acct1 Acct2 Date Total_Reissued Reissue_Per_Day
1 A 1 1/1/2016 2 2
2 A 1 1/2/2016 3 1
3 A 1 1/3/2016 5 2
4 A 1 1/4/2016 6 1
1 B 3 1/1/2016 1 1
2 B 3 1/2/2016 2 1
1 B 4 1/1/2016 1 1
2 B 4 1/2/2016 2 1
The Reissued Column is a running total. For Acct A on 1/1/2016 there were 2 reissues, then On 1/2/2016 there was 1 more making a total of 3. My problem is calculating the actual number of reissues per day.
You can use the lag() function to peek back at the previous row; assuming that 'previous' is the last date you saw for the acct1/acct2 combination you can do:
select row_number() over (partition by acct1, acct2 order by dt) as row_num,
acct1, acct2, dt, total_reissued,
total_reissued - nvl(lag(total_reissued)
over (partition by acct1, acct2 order by dt), 0) as reissue_per_day
from your_table;
ROW_NUM A ACCT2 DT TOTAL_REISSUED REISSUE_PER_DAY
---------- - ---------- ---------- -------------- ---------------
1 A 1 2016-01-01 2 2
2 A 1 2016-01-02 3 1
3 A 1 2016-01-03 5 2
4 A 1 2016-01-04 6 1
1 B 3 2016-01-01 1 1
2 B 3 2016-01-02 2 1
1 B 4 2016-01-01 1 1
2 B 4 2016-01-02 2 1
I'm not sure if your 'row' column actually exists, or is required, or was just to illustrate your data. I've generated it anyway, in case you need it.
The main bit of interest is:
lag(total_reissued) over (partition by acct1, acct2 order by dt)
which finds the previous date's value (using dt as a column name, since date isn't a valid name). That then has an nvl() wrapper so the first row sees a dummy value of zero instead of null. And then that is subtracted from the current row's value to get the difference.
ORACLE select
calculate Sales، returns and the rest for a customer in the same table for the same product according to trans type
i need to calculate total sales and total returns and the rest for the customer and items.
and group by customer
Trans_Type:
1= Sales
2= Return
ID Trans_Type DATE Items_ID Quantity Clint_ID
--- ---------- -------- ---------- ---------- ----------
1 1 16-OCT-09 701555 3 1
2 2 12-DEC-09 701555 1 1
3 1 30-JUL-10 701511 63 2
4 2 30-JUL-10 701555 1 1
5 1 30-JUL-10 701234 2 3
6 1 30-JUL-10 701234 5 3
7 2 30-JUL-10 701511 1 2
8 1 30-JUL-10 701522 3 2
9 1 30-JUL-10 701555 2 3
10 1 30-JUL-10 701555 4 2
11 2 30-JUL-10 701555 2 2
If I understood everything correct you need to use case when ... and group by ... clauses, like here:
select clint_id, items_id, qty, ret, nvl(qty,0) - nvl(ret,0) rest
from (
select clint_id, items_id,
sum(case when trans_type = 1 then quantity end) qty,
sum(case when trans_type = 2 then quantity end) ret
from data group by clint_id, items_id )
order by clint_id, items_id
SQLFiddle demo
I need an o/p as below.
1,1
2,1
2,2
3,1
3,2
3,3
4,1
4,2
4,3
4,4
... and so on.
I tried to write the query as below. But throwing error. SIngle row subquery returns more than one row.
with test1 as(
SELECT LEVEL n
FROM DUAL
CONNECT BY LEVEL <59)
select n,(
SELECT LEVEL n
FROM DUAL
CONNECT BY LEVEL <n) from test1
Appreciate your help in solving the same.
Here is one of the methods how you could get the desired result:
SQL> with t1(col) as(
2 select level
3 from dual
4 connect by level <= 5
5 )
6 select a.col
7 , b.col
8 from t1 a
9 join t1 b
10 on a.col >= b.col
11 ;
COL COL
---------- ----------
1 1
2 1
2 2
3 1
3 2
3 3
4 1
4 2
4 3
4 4
5 1
5 2
5 3
5 4
5 5
15 rows selected