I am trying to write a query in kusto (kql), where I would check which users how many times accessed what projects.
pageViews
| extend projectId = extract(#"/projects/([0-9a-f]{32}|[0-9a-zA-Z]{21,22})", 1, url)
| where url matches regex #'/projects/([0-9a-f]{32}|[0-9a-zA-Z]{21,22})'
and timestamp > ago(90d)
and isnotempty(user_Id)
| project timestamp, user_AccountId, user_Id, url, projectId
| summarize projectRequests = count() by user_Id, projectId
I need to extend this functionality by when was the last access
I tried:
pageViews
| extend projectId = extract(#"/projects/([0-9a-f]{32}|[0-9a-zA-Z]{21,22})", 1, url)
| where url matches regex #'/projects/([0-9a-f]{32}|[0-9a-zA-Z]{21,22})'
and timestamp > ago(90d)
and isnotempty(user_Id)
| project timestamp, user_AccountId, user_Id, url, projectId
| summarize projectRequests = count() by user_Id, projectId, LastEdit = max(timestamp)
but it gave me
Function 'max' cannot be invoked in current context
How could I extend this query to include when was the project accessed last?
You should replace:
| summarize projectRequests = count() by user_Id, projectId, LastEdit = max(timestamp)
with
| summarize projectRequests = count(), LastEdit = max(timestamp) by user_Id, projectId
I had to flip around LastEdit = max(timestamp) and ProjectRequests = count() by user_Id, projectId.
like:
pageViews
| extend projectId = extract(#"/projects/([0-9a-f]{32}|[0-9a-zA-Z]{21,22})", 1, url)
| where url matches regex #'/projects/([0-9a-f]{32}|[0-9a-zA-Z]{21,22})'
and timestamp > ago(90d)
and isnotempty(user_Id)
| project timestamp, user_AccountId, user_Id, url, projectId
| summarize LastEdit = max(timestamp), ProjectRequests = count() by user_Id, projectId
Related
I have a table of users
+-------+------+-----+
| User | Sale | Sex |
+-------+------+-----+
| user1 | 120 | M |
| user2 | 98 | M |
| user3 | 5 | F |
+-------+------+-----+
in laravel, we query this by the following statement:
$allMale = UserTable::where('Sex', 'M')->get();
What will happen in I try to query again on the same table
$allFemales = UserTable::where('Sex', 'F')->get();
$user1 = UserTable::where('User', 'user1')->get();
will it query 3 times? is it possible to query once and then Parse it multiple times?
Yes, I'm not sure if UserTable here is Builder or Eloquent but Eloquet under the hood is using MySQL so it will run 3 different queries:
SELECT * FROM users WHERE Sex = 'M';
SELECT * FROM users WHERE Sex = 'F';
SELECT * FROM users WHERE User = 'user1';
Of course you can do it like this:
$users = UserTable::where(function($q) {
$q->whereIn('Sex', ['M', 'F'])->orWhere('User', 'user1')
})->get();
This will generate query:
SELECT * FROM users WHERE ((Sex IN ('M', 'F') OR USER = 'user1'))
and now you can get those users from variable like this using Laravel collection methods:
$allMale = $users->where('Sex', 'M')->get();
$allFemales = $users->where('Sex', 'F')->get();
$user1 = $users->where('User', 'user1')->get();
Now, assuming that user1 has sex set, you could in fact use:
$users = UserTable::whereIn('Sex', ['M', 'F'])->get();
In addition, assuming there is only single user1 instead of:
$user1 = $users->where('User', 'user1')->get();
probably better solution would be:
$user1 = $users->where('User', 'user1')->first();
I created a table in impala ,the have two columns ,
+-----------+---------------------+---------+
| name | type | comment |
+-----------+---------------------+---------+
| unique_id | string | |
| cmap | array<struct< | |
| | fieldname:string, | |
| | fieldid:string, | |
| | fielddata:string | |
| | >> | |
+-----------+---------------------+---------+
i need set the conditions for cmap to query unique_id, such as
(fieldname="ip"and fielddata="192.168.1.145") and(fieldname="wid" and fielddata="15")
i wrote this sql but can't query it, but i inserted the data in the table
sql：
select unique_id from s_click_parquet,s_click_parquet.cmap as lst where ( fieldname="ip" and fieldData="192.168.1.145") and(fieldname="wid" and fielddata="15");
You typically access arrays with square brackets, but you can have an array of structs, not just an array.
You can INLINE() an array of struct to make it multiple rows, as well as splitting the fields to queryable columns.
Then you can query it as you would a normal table.
Play around with
select *
from table1 t
lateral view inline(cmap) a;
From my work collogue:
so rule of thumb is structs can get accessed via strucName., while arrays have to be joined*
So if we have the following types from an avro schema:
case class Visit (var Patient: PatientType = null)
case class PatientType(
var Name: String = null,
var Address: List[AddressType] = null
)
case class AddressType(
var Line1: String = null,
var Line2: String = null,
var City: String = null,
var State: String = null,
var ZipCode: String = null
)
You would query for patient name and address like:
select vis.patient.name, addr.item.*
from visit vis
left join vis.patient.address addr
I have a Hive table named 'Login'. It contains the following columns :-
UserID | UserName | UserIP | UserCountry | Date
On a particular day (all that logins of that day), I want to find out the UserIDs, which has been accessed from a country (UserCountry) from where the user has never accessed his account from OR the IPs (UserIP) from which the account has never been accessed before.
I would start with an except where I remove prior countries and IPs
select userid, usercountry, userip
from table
where date=xx
except
select userid, usercountry, userip
from table
where date<xx
I think that the best way is the GROUP clause!
You say "has never been accessed before", means COUNT = 1.
To find the IP use only once :
select UserId, UserIP, COUNT(UserIP) FROM Login WHERE Date = yourdate GROUP BY UserIP, UserId HAVING COUNT(UserIP) = 1
To find the country use only once :
select UserId, UserCountry, COUNT(UserCountry) FROM Login WHERE Date = yourdate GROUP BY UserCountry, UserId HAVING COUNT(UserCountry) = 1
Left Outer Join will be able to satisfy your requirement in HIVE.
select t1.userid, t1.usercountry, t1.userip
from table t1
LEFT OUTER JOIN
from table t2
ON (t1.userid=t2.userid)
WHERE t1.date=xx and
t2.data < xx and
(t2.usercountry IS NULL or
t2.userip IS NULL);
Hope this helps...
I have a list of integers: ids. There is also collection, IdNames which comes from an sql table. For each integer in ids I want to find the matching id in, IdNames. Then for each record in IdNames that has a matching id I'd like to select the value in the Name and DisplayName columns and the id.
So here is the table IdNames
Id | Name | DisplayName
--------------------------------
1 | fistName | firstDisplayName
2 | secondName | secondDisplayName
3 | thirdName | thirdDisplayName
If ids contained the integers 2 and 3, I'd want this collection to be returned
Id | Name | DisplayName
--------------------------------
2 | secondName | secondDisplayName
3 | thirdName | thirdDisplayName
How would I write this as a linq query?
I stared writing it like this: IdNames.Select(x => x.Id == ids.Any()), but obviously it's not right.
var idNames = from idName in DataContext.IdNames
where ids.Contains(idName.Id)
select idName;
Good enough?
Use Join in Linq-To-Objects("I have a list of integers: ids. There is also collection, IdNames"):
var query = from id in ids
join idName in IdNames
on id equals idName.Id
select idName;
Why is LINQ JOIN so much faster than linking with WHERE?
I need the corresponding query in Criteria language to this one (to retrieve all categories from my table but to distinct them):
SELECT DISTINCT categoryName
FROM Category
WHERE CategoryID IN (
SELECT CategoryID
FROM FoodCategory
)
ORDER BY categoryName
I have table FoodCategory table
id | FoodID | CategoryID
--------|---------------|------------
| |
| |
| |
Actually CategoryID is a foreign key that is pointing to this table here. This is table for Category:
CategoryID | categoryName | otherField
---------------|------------------|------------
| |
| |
| |
And this is table for Food:
FoodID | FoodName | otherField
---------------|------------------|------------
| |
| |
| |
Something like that should do the trick :
public List<String> retrieveFoodCategoryNames() {
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery<String> criteriaQuery = builder.createQuery(String.class);
Root<FoodCategory> root = criteriaQuery.from(FoodCategory.class);
// by default an inner join so it will only get the categories which have their id in the FoodCategory table
Join<FoodCategory, Category> joinCategory = root.join(FoodCategory_.category);
Fetch<FoodCategory, Category> fetchCategory = root.fetch(FoodCategory_.category);
Path<String> categoryNamePath = fetchCategory.get(Category_.categoryName);
criteriaQuery.select(categoryNamePath).distinct(true);
criteriaQuery.orderBy(builder.asc(categoryNamePath));
return entityManager.createQuery(criteriaQuery).getResultList();
}
This is not the exact same SQL request because you used a subquery where I'm using a join but it seemed more suited to this particular case. The subquery syntax is a bit more complex and I will not try to write it without compiling! ^^
If something is unclear let me know :-)