I have a table that contains the following datetime and customer
example :
data1 :datetime : 2011-7-25 10:23:30 AM
customer:A
data2 :datetime : 2011-7-25 10:40:30 AM
customer:B
I need query to know time duration between Customer A and B,
I am using Firebird database. thank you before
What about DATEDIFF?
SELECT DATEDIFF(second, -- or the unit you want
a.datetime, b.datetime) AS result
FROM ExampleTable a, ExampleTable b
WHERE a.customer = 'A' and b.customer = 'B'
Related
I have a pig relation like below:
FINAL= {input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray,test_1:: type: chararray,test_2::name:chararray}
I am trying to store all columns for input_md5 relation to a hive table.
like all input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray not taking test_1:: type: chararray,test_2::name:chararray
is there any command in pig which filters only columns of input_md5.Something like below:
STORE= FOREACH FINAL GENERATE all input_md5::type .
I know that pig have :
FOREACH FINAL GENERATE all input_md5::type as type syntax, but i have many columns so I cannot use as in my code.
Because when i try:
STORE= FOREACH FINAL GENERATE input_md5::type .. bus_input_md5::name;
Pig throws an error:
org.apache.hive.hcatalog.common.HCatException : 2007 : Invalid column position in partition schema : Expected column <type> at position 1, found column <input_md5::type>
Thanks in advance,
Resolved this issue , below is the fix:
Create a relation with some filter condition as below:
DUMMY_RELATION= FILTER SOURCE_TABLE BY type== ''; (I took a column named type ,this can be filtered by any column in the table , all that matters is we need its schema)
FINAL_DATASET= UNION DUMMY_RELATION,SCHEMA_1,SCHEMA_2;
(this new DUMMY_RELATIONn should be placed 1st in the union)
Now you no more have :: operator And your column names would match hive table's column names, provided your source table (to DUMMY_RELATION) and target table have same column order.
Thanks to myself :)
I implemented Neethu's example this way. May have typos, but it shows how to implement this idea.
tableA = LOAD 'default.tableA' USING org.apache.hive.hcatalog.pig.HCatLoader();
tableB = LOAD 'default.tableB' USING org.apache.hive.hcatalog.pig.HCatLoader();
--load empty table
finalTable = LOAD 'default.finalTable' USING org.apache.hive.hcatalog.pig.HCatLoader();
--example operations that end up with '::' in column names
g = group tableB by (id);
j = JOIN tableA by id LEFT, g by group;
result = foreach j generate tableA::id, tableA::col2, g::tableB;
--union empty finalTable and result
result2 = union finalTable, result;
--bob's your uncle
STORE result2 INTO 'finalTable' USING org.apache.hive.hcatalog.pig.HCatStorer();
Thanks to Neethu!
I am looking to achieve the below functionality in Pig. I have a set of sample records like this.
Note that the EffectiveDate column is sometimes blank and also different for the same CustomerID.
Now, as output, I want one record per CustomerID where the EffectiveDate is the MAX. So, for the above example, i want the records highlighted as shown below.
The way I am doing it currently using PIG is this:
customerdata = LOAD 'customerdata' AS (CustomerID:chararray, CustomerName:chararray, Age:int, Gender:chararray, EffectiveDate:chararray);
--Group customer data by CustomerID
customerdata_grpd = GROUP customerdata BY CustomerID;
--From the grouped data, generate one record per CustomerID that has the maximum EffectiveDate.
customerdata_maxdate = FOREACH customerdata_grpd GENERATE group as CustID, MAX(customerdata.EffectiveDate) as MaxDate;
--Join the above with the original data so that we get the other details like CustomerName, Age etc.
joinwithoriginal = JOIN customerdata by (CustomerID, EffectiveDate), customerdata_maxdate by (CustID, MaxDate);
finaloutput = FOREACH joinwithoriginal GENERATE customerdata::CustomerID as CustomerID, CustomerName as CustomerName, Age as Age, Gender as gender, EffectiveDate as EffectiveDate;
I am basically grouping the original data to find the record with the maximum EffectiveDate. Then I join these 'grouped' records with the Original dataset again to get that same record with Max Effective date, but this time I will also get additional data like CustomerName, Age and Gender. This dataset is huge, so this approach is taking a lot of time. Is there a better approach?
Input :
1,John,28,M,1-Jan-15
1,John,28,M,1-Feb-15
1,John,28,M,
1,John,28,M,1-Mar-14
2,Jane,25,F,5-Mar-14
2,Jane,25,F,5-Jun-15
2,Jane,25,F,3-Feb-14
Pig Script :
customer_data = LOAD 'customer_data.csv' USING PigStorage(',') AS (id:int,name:chararray,age:int,gender:chararray,effective_date:chararray);
customer_data_fmt = FOREACH customer_data GENERATE id..gender,ToDate(effective_date,'dd-MMM-yy') AS date, effective_date;
customer_data_grp_id = GROUP customer_data_fmt BY id;
req_data = FOREACH customer_data_grp_id {
customer_data_ordered = ORDER customer_data_fmt BY date DESC;
req_customer_data = LIMIT customer_data_ordered 1;
GENERATE FLATTEN(req_customer_data.id) AS id,
FLATTEN(req_customer_data.name) AS name,
FLATTEN(req_customer_data.gender) AS gender,
FLATTEN(req_customer_data.effective_date) AS effective_date;
};
Output :
(1,John,M,1-Feb-15)
(2,Jane,F,5-Jun-15)
I am working in Asp.Net 4.0 C#-- MVC-3. I have one problem and don't know how to solve. I have 4 tables and want to fetch data from that tables with LINQ.
TABLES
1) Project_Master
Field Names :
project_id (pk)
project_name
company_id (FK with company_master)
company_category_id (FK with Company_Category_master)
project_status_id (FK with Project_Status_Master)
2)Company_Master
Field Names :
company_id
company_name
company_category_id (FK with Company_Category_Master)
3) Company_Category_Master
Field Names :
Company_Category_Id
Company_Category_Name
4) Project_Status_Master
Field Name :
Project_Status_Id
Project_Status_Name
Below are the fields I need to fetch..(using LINQ Query)
Company_Name
Total completed project using status id(1)=complete (where staus 1 means completed)
Total Project
Company_category_name
So, how can I fetch data with linq query??
Thanks in advance...
Try the below example:
(From lse In Me.Leases, nty In Me.Entities, psg In Me.ProductionStages, lsg In LeaseStages _
Where lse.LeaseName = leaseName _
Select lse, lsg, nty, psg).Single
or you can use below example too:
var employeesQuery = from populationTable in db.Populations
join personTable in db.Persons on populationTable.Guid equals personTable.PopulationGuid
join employeeTable in db.Employees on personTable.Guid equals employeeTable.PersonGuid
select new { populationTable, personTable, employeeTable};
I have one table, 'a', with id and timestamp. Another table, 'b', has N multiple rows referring to id, and each row has 'type', and "some other data".
I want a LINQ query to produce a single row with id, timestamp, and "some other data" x N. Like this:
1 | 4671 | 46.5 | 56.5
where 46.5 is from one row of 'b', and 56.5 is from another row; both with the same id.
I have a working query in SQLite, but I am new to LINQ. I dont know where to start - I don't think this is a JOIN at all.
SELECT
a.id as id,
a.seconds,
COALESCE(
(SELECT b.some_data FROM
b WHERE
b.id=a.id AND b.type=1), '') AS 'data_one',
COALESCE(
(SELECT b.some_data FROM
b WHERE
b.id=a.id AND b.type=2), '') AS 'data_two'
FROM a first
WHERE first.id=1
GROUP BY first.ID
you didn't mention if you are using Linq to sql or linq to entities. However following query should get u there
(from x in a
join y in b on x.id equals y.id
select new{x.id, x.seconds, y.some_data, y.type}).GroupBy(x=>new{x.id,x.seconds}).
Select(x=>new{
id = x.key.id,
seconds = x.Key.seconds,
data_one = x.Where(z=>z.type == 1).Select(g=>g.some_data).FirstOrDefault(),
data_two = x.Where(z=>z.type == 2).Select(g=>g.some_data).FirstOrDefault()
});
Obviously, you have to prefix your table names with datacontext or Objectcontext depending upon the underlying provider.
What you want to do is similar to pivoting, see Is it possible to Pivot data using LINQ?. The difference here is that you don't really need to aggregate (like a standard pivot), so you'll need to use Max or some similar method that can simulate selecting a single varchar field.
I have a database which consists of the following:
** Table 1 **
Id (PK)
Field1
** Table 2 **
Id (PK)
Field2
** Link Table **
Table1Id (FK)
Table2Id (FK)
The problem is, I cannot access Table 2 from Table 1, even though the relationship exists in the database.
For example, the following should be possible:
var Results = from c in DataContext.Table1
where c.Table2.Field2 == "Test"
select c;
However, "c.Table2.Field2" is not available for some reason - all I get for "c.Table2." is the following (among the standard any<>, where<> et al):
RelationshipName
RelationshipSet
SourceRoleName
TargetRoleName
So obviously something is screwy somewhere, but I cannot work out what!
Both tables exist in the Entity Schema, and have a valid relationship between them.
The reason that c.Table2.Field2 is not available is that c.Table2 doesn't have a property called Field2. c.Table2 is a list of entities which have that property, not an instance of that entity. It's not clear what your intention is here, but I think you want:
var Results = from c in DataContext.Table1
where c.Table2.Any(t2 => t2.Field2 == "Test")
select c;