I have a trouble with counting percents for unique values. Theres is no problem to calculate percents with exact values (total, sum etc). But with uniq function result always are different which is understandable. The main problem is that percents don't add up. For example, total unique is 5000, but sum of pieces could be 4999 or 5001. For example:
WITH (
SELECT uniq(t.id)
FROM test.table t
) AS total
SELECT t.name as gender,
t.age as age,
uniq(t.id) as uniques,
COALESCE((( uniques / total ) * 100), 0) as uniquesPercent
FROM test.table t
GROUP BY gender, age
So, is there any method to handle this problem. I can't use uniqExact due to performance issues. Thank you in advance.
Let's assume that the same user cannot have several genders or ages (other words the same user cannot appear in more than one group) then total can be calculated as the sum of unique counts each group:
SELECT result.1 gender, result.2 age, result.3 uniquesPercent
FROM (
SELECT
groupArray((gender, age, uniques)) groups,
arraySum(x -> x.3, groups) total,
arrayJoin(arrayMap(x -> (x.1, x.2, x.3 / total), groups)) result
FROM (
SELECT
t.name AS gender,
t.age AS age,
uniq(t.id) AS uniques
FROM test.table AS t
GROUP BY
gender,
age
)
)
Related
Overview of the table in question
I need to get a distinct count of the column Fkey_Dim_Resource_ID that has holiday to spare.
My Table consists of five columns:
Resource_Allocated_Holiday_ID (Primary Key)
Fkey_Dim_Resource_ID
Fkey_Dim_HolidayYear_ID
Fkey_Dim_Company_ID
Allocated_Holiday_Hrs_Qty
Measure:
Allocated Holiday (Hrs):= Var X= SUM([Allocated_Holiday_Hrs_Qty])
Return if(X =0; BLANK();X)
This measure below then uses the above, and the holiday spent from another metric:
Remaining Holiday (Hrs):= Var X = 'HolidayEntry Numbers'[Allocated Holiday (Hrs)] - [#Holiday Hours]
Return if(X=0;BLANK();X)
And now, I would like a metric that gives me the distinct count of Fkey_Dim_ResourceID where 'Remaining Holiday (hrs)' >0.
I have tried a lot of different stuff, but cannot seem to get it right.
test:=
ADDCOLUMNS(
SUMMARIZE('HolidayEntry Numbers'
;'HolidayEntry Numbers'[Fkey_Dim_Company_ID]
;'HolidayEntry Numbers'[Fkey_Dim_Resource_ID];
'HolidayEntry Numbers'[Fkey_Dim_HolidayYear_Id]
)
;"RemainingHoliday"; sum( [Remaining Holiday (Hrs)])
)
I would like for a distinct count of Fkey_Dim_Resource_ID that has holiday left, that takes into account the context.
Thanks in advance.
With this measure:
test4 virker når ressourcen er med:=COUNTROWS (
FILTER (
ADDCOLUMNS (
VALUES ( 'HolidayEntry
Numbers'[Fkey_Dim_Resource_ID]);
"remholiday"; CALCULATE ( [Remaining Holiday
(Hrs)] )
);
[remholiday] > 0
)
)
I get the following result:
Result of the advice1
So the metric works, when in the context of a Resource, but not when in the context of a Fkey_dim_holiday_Year_ID.
Thanks ion advance.
Resources with remaining holiday hours =
COUNTROWS ( // counts rows in a table
FILTER ( // returns a table, filtering based on predicate
// below is unique values of the column in context, as a
// one-column table
VALUES ( 'HolidayEntry Numbers'[Fkey_Dim_Resource_ID] ),
[Remaining Holiday (hrs)] > 0 // keep rows meeting this criterion
)
)
As a matter of style, you should fully qualify column names as 'Table'[Column], and never fully qualify measure references, i.e. don't prefix with table name. This conforms with all style guides I know, and helps to ensure your code is unambiguous (since both columns and measures are referenced in square brackets).
I'm trying to capture the average of FIRST_CONTACT_CAL_DAYS but what I would like to do is create an indicator for the top and bottom 10% of values so I can exclude those (outliers) from my average calculation.
Not sure how to go about do this, any thoughts?
SELECT DISTINCT
TO_CHAR(A.FIRST_ASSGN_DT,'DAY') AS DAY_NUMBER,
A.FIRST_ASSGN_DT,
A.FIRST_CONTACT_DT,
TO_CHAR(A.FIRST_CONTACT_DT,'DAY') AS DAY_NUMBER2,
A.FIRST_CONTACT_DT AS FIRST_PHONE_CONTACT,
A.ID,
ABS(TO_DATE(A.FIRST_CONTACT_DT, 'DD/MM/YYYY') - TO_DATE(A.FIRST_ASSGN_DT, 'DD/MM/YYYY')) AS FIRST_CONTACT_CAL_DAYS,
FROM HIST A
LEFT JOIN CONTACTS D ON A.ID = D.ID
WHERE 1=1
You may be looking for something like this. Please adapt to your situation.
I assume you may have more than one "group" or "partition" and you need to compute the average for each group separately, after throwing out the outliers in each partition. (An alternative, which can be easily accommodated by adapting the query below, is to throw out the outliers at the global level, and only then to group and take the average for each group.)
If you don't have any groups, and everything is one big pile of data, it's even easier - you don't need GROUP BY and PARTITION BY.
Then: the function NTILE assigns a bucket number, in this example between 1 and 10, to each row, based on where they fall (first decile, i.e. first 10%, next decile, ... all the way to the last decile). I do this in a subquery. Then in the outer query just filter out the first and last bucket before you group by and you compute the average.
For testing purposes I create three groups with 10,000 random numbers each in a WITH clause - no need to spend any time on that portion of the code, since it is not part of the solution (the SQL code to solve your problem) - it's just a dirty trick to create test data on the fly.
with
inputs ( grp, val ) as (
select ceil(level/10000), dbms_random.value(0, 150)
from dual
connect by level <= 30000
)
select grp, avg(val) as avg_val
from (
select grp, val, ntile(10) over (partition by grp order by val) as bkt
from inputs
)
where bkt between 2 and 9
group by grp
;
GRP AVG_VAL
--- -----------------------
1 75.021614866547043734458
2 74.286117923344418598032
3 75.437412573353736953791
I have a table called loan with loan amount,annual income, year (MMM-YY format) and member id. I am trying to find the highest loan amount in a year along wit annual income and member id details.
I tried to group the highest loan amount by year using the code
select max(cast(loan_amt as int)),issue_d from loan group by issue_d;
then I wanted also to fetch the member id and annual income information so I wrote the following code
but it is giving me error message for using alias for a column which is cast.
Code:
select a.loan_amt,a.member_id,a.annual_inc,a.issue_d
from
(select loan_amt,member_id,annual_inc,issue_d from loan) a
join
(select max(cast(loan_amt as int)) as ml,issue_d from loan group by issue_d) c
where ((a.issue_d=c.issue_d) and (a.loan_amt=a.ml));
What you want to do is rank the records based on the Amount, per Period, then keep only the top 1 record for each Period.
Use one of the analytic functions that are designed exactly for that purpose -- Hive has a pretty good support of the SQL standard on that topic.
Since you don't say what to do about ties (i.e. what if several loans have the same Amount???) I assume you want just one record chosen randomly...
select X, Y, Z, Period, Amount as TopAmount
from
(select X, Y, Z, Period, cast(StrAmt as double) as Amount,
row_number() over (partition by Period order by cast(StrAmt as double) desc) as TmpRank
from WTF
) TMPWTF
where TmpRank =1
If you want all the records with top Amount then replace row_number with rank or dense_rank (the "dense" stuff would make a difference for the top 2, but not for the top 1)
I can't find out a way, how to sort my query, this is the simple query:
SELECT {[Measures].[IB]}
ON COLUMNS,
{[Dim_Product_Models_new].[PLA].members } *
{[Dim Dates_new].[Date Full].&[2013-02-01]:[Dim Dates_new].[Date Full].&[2014-01-01]}
ON ROWS
FROM [cub_dashboard_spares]
The think is, I would get a result for 6 PLAs combined across 12 months (72 rows in total), however it is sorted alphabetically upon PLA.
What i need, is to sort the PLAs based on a measure in last month (2014-01-01 in this case).
Is there any way to perform this task so that the groupping (PLAs, Dates from 2013-02 to 2013-12) is perserved, but only the order of my PLAs is different. (PLA with highest measure in last month would be first, and so on)
Thank you very much for any kind of help
Just put the sorted set on the rows, using the Order function. The third parameter of this function is DESC if you want to sort within each hierarchy level, but still want to get parents before children (like ALL before the single attribute members), or BDESC if you want to sort across all levels.
SELECT {[Measures].[IB]}
ON COLUMNS,
Order({[Dim_Product_Models_new].[PLA].members },
([Measures].[IB], [Dim Dates_new].[Date Full].&[2014-01-01]),
DESC)
*
{[Dim Dates_new].[Date Full].&[2013-02-01]:[Dim Dates_new].[Date Full].&[2014-01-01]}
ON ROWS
FROM [cub_dashboard_spares]
The order function over a crossjoin should preserve the initial order of the first set so reversing the order of the tuple will do the job:
SELECT
{
[Measures].[IB]
} ON COLUMNS,
order(
{[Dim Dates_new].[Date Full].&[2013-02-01]:[Dim Dates_new].[Date Full].&[2014-01-01]} *
{[Dim_Product_Models_new].[PLA].members } ,
[Measures].[IB],
desc
) ON ROWS
FROM [cub_dashboard_spares]
If you want to preserve the oder of appearance of the column labels, you can use the generate function like in the following example from the AW cube:
SELECT
{[Measures].[Internet Sales Amount]} ON 0
,Generate
(
{[Customer].[Country].&[Australia]:[Customer].[Country].&[United Kingdom]}
,(
Order
(
[Date].[Calendar Year].[Calendar Year].MEMBERS
,(
[Customer].[Country].CurrentMember
,[Measures].[Internet Sales Amount]
)
,DESC
)
,[Customer].[Country].CurrentMember
)
) ON 1
FROM [Adventure Works];
Philip,
Is it possible to use windowing with any of the percentile functions? Or do you know a work around to get a rolling percentile value?
It is easy with a moving average:
select avg(foo) over (order by foo_date rows
between 20 preceding and 1 preceding) foo_avg_ma
from foo_tab
But I can't figure out how to get the median (50% percentile) over the same window.
You can use PERCENTILE_CONT or PERCENTILE_DISC function to find the median.
PERCENTILE_CONT is an inverse distribution function that assumes a
continuous distribution model. It takes a percentile value and a sort
specification, and returns an interpolated value that would fall into
that percentile value with respect to the sort specification. Nulls
are ignored in the calculation.
...
PERCENTILE_DISC is an inverse distribution function that assumes a
discrete distribution model. It takes a percentile value and a sort
specification and returns an element from the set. Nulls are ignored
in the calculation.
...
The following example computes the median salary in each department:
SELECT department_id,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY salary DESC) "Median cont",
PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY salary DESC) "Median disc"
FROM employees
GROUP BY department_id
ORDER BY department_id;
...
PERCENTILE_CONT and PERCENTILE_DISC may return different results.
PERCENTILE_CONT returns a computed result after doing linear
interpolation. PERCENTILE_DISC simply returns a value from the set of
values that are aggregated over. When the percentile value is 0.5, as
in this example, PERCENTILE_CONT returns the average of the two middle
values for groups with even number of elements, whereas
PERCENTILE_DISC returns the value of the first one among the two
middle values. For aggregate groups with an odd number of elements,
both functions return the value of the middle element.
a SAMPLE with windowing simulation trough range self-join
with sample_data as (
select /*+materialize*/ora_hash(owner) as table_key,object_name,
row_number() over (partition by owner order by object_name) as median_order,
row_number() over (partition by owner order by dbms_random.value) as any_window_sort_criteria
from dba_objects
)
select table_key,x.any_window_sort_criteria,x.median_order,
PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY y.median_order DESC) as rolling_median,
listagg(to_char(y.median_order), ',' )WITHIN GROUP (ORDER BY y.median_order) as elements
from sample_data x
join sample_data y using (table_key)
where y.any_window_sort_criteria between x.any_window_sort_criteria-3 and x.any_window_sort_criteria+3
group by table_key,x.any_window_sort_criteria,x.median_order
order by table_key, any_window_sort_criteria
/