Earlier I asked about manipulating a data structure in Hive or Pig. I was able to get an answer in SQL, and from there figured out the answer for Hive. I'm still looking for a solution in Pig.
I want to change myTable:
Into myTable2:
I tried:
myTable2 = FOREACH myTable GENERATE item, year,
'jan' AS month, jan AS value,
'feb' AS month, feb AS value,
'mar' AS month, mar AS value;
Which more or less is what worked in Hive, but Pig gives me:
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1108:
<line 2, column 35> Duplicate schema alias: month
I figured it out, although I would love to see a more concise version:
JAN = FOREACH myTable GENERATE item, year, 'jan' AS month, jan AS value;
FEB = FOREACH myTable GENERATE item, year, 'feb' AS month, feb AS value;
MAR = FOREACH myTable GENERATE item, year, 'mar' AS month, mar AS value;
myTable2 = union JAN, FEB, MAR;
Pig Script:
data = LOAD '/pigsamples/sampledata' USING PigStorage(',')
AS (item:CHARARRAY, year:INT, jan:DOUBLE, feb:DOUBLE, mar:DOUBLE);
--concatenating month name to its value so that they won't get separated when i perform a flatten on the tuple.
concat_data = FOREACH data GENERATE item, year, CONCAT('jan:', (CHARARRAY)jan) AS jan,
CONCAT('feb:', (CHARARRAY)feb) AS feb, CONCAT('mar:', (CHARARRAY)mar) AS mar;
--convert the month (name,value) pairs to a bag and flatten them
flatten_values = FOREACH concat_data GENERATE item, year, FLATTEN (TOBAG (jan, feb, mar)) AS month_values;
--split the string based on the delimiter that we used above to concat
split_flatten_values = FOREACH flatten_values GENERATE item, year, FLATTEN (STRSPLIT (month_values, ':')) AS (month:CHARARRAY, value:CHARARRAY);
Related
How can we group reocrds weekly/monthly in oracle between from date and to date so that one record can be returned for one week/month.
e.g If we have 5 records for Ist weeek and 3 records for second week between a from date and to date then it should return total 2 records(one for Ist week and one for second week).
Thanks in advance.
You can group the results using GROUP BY: http://www.techonthenet.com/oracle/group_by.php
To select only those between from date and to date use WHERE
EDIT
For selecting the begining of the week or month, you can use TRUNC(the_date_field, ): http://www.techonthenet.com/oracle/functions/trunc_date.php
For example this groups by week:
SELECT TRUNC(datecol_name, 'WW') FROM table_name GROUP BY TRUNC(datecol_name, 'WW');
I have an Item model.
There are many records in the database with column created_at filled in.
I want to generate a view with such a hierarchy:
2014
December
31
items here
30
items here
29
items here
...
November
31
...
...
2013
...
What's the most elegant way to do that?
EDIT: Thank you so much for queries. How do I get that worked in Ruby on Rails?
To achieve this, we will order the records by the parts of date. Sample query below
SELECT
ItemDescription,
Year(DateField) AS Year,
Datename(mm, DateField) AS Month,
Day(DateField) AS Day
FROM tblName
ORDER BY
Year(DateField) DESC,
Month(DateField) DESC,
Day(DateField) DESC
This will provide you the data in the order expected. Now you can either create a stored procedure to modify the output to the format you need.
SELECT DATEPART(Year, PaymentDate) Year, DATEPART(Month, PaymentDate) Month, DATEPART(day, PaymentDate) Day,item_name
FROM Payments
GROUP BY DATEPART(Year, PaymentDate), DATEPART(Month, PaymentDate),DATEPART(day, PaymentDate) desc
ORDER BY Year, Month,day
I am trying to set between a date range for 6 months in the past for two different fields that will group the data by month. How do I set such a between clause to achieve this?
SELECT TO_CHAR(mopend, 'MM-yyyy') AS month, MOPSTATUS, COUNT(*) MTS_COMPLETE_CNT
FROM MOPUSER.MOPACTIVITY
WHERE UPPER(MOPSTATUS) = 'COMPLETE'
AND TO_CHAR(MOPACTIVITY.MOPSTART, 'yyyy-mm-dd hh24:mi') BETWEEN TO_CHAR(sysdate,'YYYY-MM-DD')||' 06:02:00' AND TO_CHAR(sysdate,'YYYY-MM-DD')||' 22:59:59'
OR TO_CHAR(MOPACTIVITY.MOPEND, 'yyyy-mm-dd hh24:mi') BETWEEN TO_CHAR(SYSDATE,'YYYY-MM-DD')||' 06:02:00' AND TO_CHAR(SYSDATE,'YYYY-MM-DD')||' 22:59:59'
GROUP BY TO_CHAR(mopend, 'MM-yyyy'), MOPSTATUS
ORDER BY TO_CHAR(mopend, 'MM-yyyy'), MOPSTATUS
I will answer one part of your question first, and then based on your comments, I can give you the full query.
The following query returns the end points between which you want to search. T1 is 06:02 in the morning on the date that is six months back in time. T2 is the last second of today.
select sysdate
,add_months( trunc(sysdate) + interval '06:02' hour to minute, -6) as t1
, trunc(sysdate) + interval '23:59:59' hour to second as t2
from dual;
The above query returns the following (using yyyy-mm-dd hh24:mi:ss):
sydate: 2014-04-11 13:54:28
t1: 2013-10-11 06:02:00
t2: 2014-04-11 23:59:59
If I interpret you correctly, this is the time period you want to search?
For the second part of the answer, I'd need to know the following:
Can any of MOPSTART or MOPEND be null? If so, how do you want to treat those rows?
Do you want to include the end points, i.e. rows where MOPSTART >= t1? Or only where MOTSTART > t1?
Same as (2) but for MOPEND
What month do you want to group by (see below)?
For example, row (a), do you want count it once for each month, or only in JAN (started) or only in JUN(ended)?
JAN FEB MAR APR MAY JUN
a: |-------------------|
b: |---|---|
c: |---|
d: |-----------|
e: |--------|
How to write a following SQL query in doctrine2 as DQL .
SELECT COUNT(id)
FROM stats
WHERE YEAR(record_date) = 2009
GROUP BY YEAR(record_date), MONTH(record_date)
i.e i would like to group by results based on month,year of datetime field stored in MySQL table.
In DQL you could also group by month, year, day etc with SUBSTRING.
For example - group by month (datetime format Y-m-d H:i:s):
SELECT p, SUBSTRING(p.date, 6, 2) as month
FROM Entity p
GROUP BY month
USING VB 6.0
How to select previous row value?
Am Selecting a Date between fromdate and todate using date picker
Code
Dim stdate, endate as string
stdate = Fromdate
endate = todate
Example:
fromdate: 01-01-2009
todate: 01-06-2009
Data’s will display fromdate to todate.
I want to select previous date means previous row value.
How to select previous row value?
We cannot give
stdate = - fromdate
endate = - todate
It will display a data’s between 31-01-2008 to 31-05-2009
We cannot give
Stdate = <fromdate
endate = < todate
It will display a data’s before 01-01-2009 and also we cannot use > in between condition
Stdate = fromdate (Here how can I give “from previous row value of the fromdate”)
endate = todate (here how can I give “to previous row value of the todate”)
Example:
id, date, name
01, 02-01-2009, raja
01, 04-01-2009, raja
02, 04-01-2009, ravi
so on.....
01, 28-05-2009, raja
01, 31-05-2009, raja
so on...
am selecting a date stdate = 04-01-2009, endate= 31-05-2009
Output shoud display like this -
01, 02-01-2009, raja
01, 04-01-2009, raja
02, 04-01-2009, ravi
.......
01, 28-05-2009, raja
It should dipslay one record Before ssdate and endate.
How to select previous row value?
Need Help in VB 6 code.
I do not completely understand what you are doing, but if you have a recordset object that is sorted on date you need to need something like
If (recordset.BOF = False) Then
recordset.MovePrevious
End If
If you are asking how to get a recordset that contains one record before the start date you are going to have to expand your query to try to include at least on record before the fromdate, find the first record with the fromdate, then use the MovePrevious method above.
If you are just asking how to get a date prior to a given date then use the DateAdd method.
example - the below code will subtract 1 day from 2/1/2009 and put 1/31/2009 into the valriable.
dtPreviousDate = DateAdd("D", CDate("02-01-2009"), -1)