Power Query - Can a range be queried using SQL? - powerquery

I have a range named tDetails. How can I query this using SQL?
I click in my range, and on the Power Query tab, I click "From Table/Range". This opens the Power Query editor.
In the advanced editor, I see something like:
let
Source = Excel.CurrentWorkbook(){[Name="tDetails"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"User ID", type text},
{"Company", type text},
{"State", type text},
{"Billed", type Int65.Type}})
in
#"Changed Type"
How do I go about querying this data? I'm needing to do things like:
SELECT 'UserID' = "User ID"
, 'Discount' = "Billed" * .51
, 'Note' = CASE WHEN State = 'NY' THEN 'Standard'
WHEN State = 'OH' THEN 'Second'
WHEN State = 'CA' THEN 'Third'
ELSE 'Fourth' END
FROM Source
WHERE "State" = "NY"
Also, can temp tables be used here?
SELECT 'UserID' = "User ID"
, 'Discount' = "Billed" * .51
INTO #UserDiscount
FROM Source
WHERE "State" = "NY"
SELECT * FROM #UserDiscount ORDER BY Discount DESC
These are super basic examples. What I really need to do is far more complex, but this would give me a place to start.
Thanks!

I think the equivalent Power Query would be
let
Source = Excel.CurrentWorkbook(){[Name="tDetails"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"User ID", type text},
{"Company", type text},
{"State", type text},
{"Billed", type Int65.Type}})
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([State] = "NY")),
#"Added Custom" = Table.AddColumn(#"Filtered Rows", "Discount", each [Billed]*0.51),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "UserID", each [UserID]),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"UserID", "Discount", "Billed"})
in
#"Removed Other Columns"
Also, the equivalent of SQL #temp table in Power Query would be use of table variable. In power query each step is table variable, e.g. Source, #"Changed Type", #"Filtered Rows", #"Added Custom".... which contains the table that was evaluated in that step. You can keep on utilizing the tables contained in each table variable in succeeding steps.
e.g.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUcrPS1WK1YlWMgKyS8rzwWxjEDujKBUoEwsA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}, {"Column2", type text}}),
Custom0 = Table.SelectRows(#"Changed Type", each ([Column2] = "one")),
Custom1 = Table.SelectRows(#"Changed Type", each ([Column2] = "two")),
Custom2 = Table.SelectRows(#"Changed Type", each ([Column2] = "three")),
Custom3 = Custom0&Custom1&Custom2
in
Custom3

Yes, but you have to use a reserved function SqlExpression.ToExpression, and it doesn't look like your SQL is "valid" according to Power Query which likely needs in T-SQL format, so you'll have to experiment with how quotes are handled and columns renamed.
See this page for details: https://bengribaudo.com/blog/2021/07/13/5868/m-mysteries-sqlexpression-toexpression
Queries:
// tDetails
// Input from Excel range. This example uses embedded data so you can replicate my result.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WcspPUtJRcs7PK8kvzgey/CKBhKGRsYlSrE60kldqWhqKtL8HSNoADJRiYwE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"User ID" = _t, Company = _t, State = _t, Billed = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"User ID", type text}, {"Company", type text}, {"State", type text}, {"Billed", Int64.Type}})
in
#"Changed Type"
// Result
// Executes translated M code
let
ReferenceTables = [Source = tDetails],
Source = Expression.Evaluate(SQL, #shared)(ReferenceTables)
in
Source
// SQL
// Translates SQL to M
let
ReferenceTables = [Source = tDetails],
SqlQuery = "SELECT [User ID] as UserID, Billed * 0.51 as Discount, 'Standard' as [Note] FROM Source WHERE State = 'NY'",
TranslatedToM = SqlExpression.ToExpression(SqlQuery, ReferenceTables)
in
TranslatedToM
tDetails:
My SQL that works: SELECT [User ID] as UserID, Billed * 0.51 as Discount, 'Standard' as [Note] FROM Source WHERE State = 'NY'
Result:

Related

Max Group Filter Power Query

I'm trying to use the following code to get a new added column with information about a max value from a given field.
But what I would want is to have the possiblity to group a table where I filter it by a condition where the Custom column value would be 1.
I need to transform somehow the following parte of the code:
[
filter = [ID] /*should I add here another filter?/,
max2=Table.Group(
Source, {"ID"},
{{"MaxFiltered2", each List.Max([CODE])}}
){[ID=filter]}[MaxFiltered2]
][max2]
)
Here follows the code sequence:
#"Added Custom" = Table.AddColumn(
#"Changed Type",
"maxInt",
each
[
filter = [ID],
max=Table.Group(
Source, {"ID"},
{{"MaxFiltered", each List.Max([TAX])}}
){[ID=filter]}[MaxFiltered]
][max]
),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom", each if [maxInt]=[TAX] then 1 else 0),
#"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([Custom] = 1)),
#"Added Custom2" = Table.AddColumn(
#"Changed Type",
"maxInt2",
each
[
filter = [ID],
max2=Table.Group(
Source, {"ID"},
{{"MaxFiltered2", each List.Max([CODE])}}
){[ID=filter]}[MaxFiltered2]
][max2]
)
in
#"Added Custom2"
Inputput desired:
ID TAX CODE
A 4 921
A 6 500
A 6 200
B 2 700
B 2 500
B 1,5 100
Output desired:
ID TAX CODE
A 6 500
B 2 700
(to get the max for A and B IDs both on TAX and CODE variables)
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTIBYksjQ6VYHQjfDIhNDQxQ+EZQvhOIDcTmaHxTJL6hnimIBInEAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, TAX = _t, CODE = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"TAX", type number}, {"CODE", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {{"All", each _, type table [ID=nullable text, TAX=nullable number, CODE=nullable number]}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom",
each Table.First(
Table.Sort([All],{{"TAX", Order.Descending},{"CODE", Order.Descending}})
)
),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"ID", "All"}),
#"Expanded Custom" = Table.ExpandRecordColumn(#"Removed Columns", "Custom", {"ID", "TAX", "CODE"}, {"ID", "TAX", "CODE"})
in
#"Expanded Custom"
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTIBYksjQ6VYHQjfDIhNDQxQ+EZQvhOIDcTmaHxTJL6hnimIBInEAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, TAX = _t, CODE = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"TAX", type number}, {"CODE", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {{"TAX", each List.Max([TAX]), type nullable number}, {"CODE", each List.Max([CODE]), type nullable number}})
in
#"Grouped Rows"

How to make a pivot table with power query

I have a table with number like below ,
Phone Number
123, 456, 890
123453
902, 423
so i would like to do the pivot table with can show all the phone number (delimiter is ",") and count how many time it appear in the list ? can someone assist for that?
I just have a initial step with the code below
let
Source = Excel.CurrentWorkbook(){[Name="Phone_Number"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Phone Number", type text}})
in
#"Changed Type"
updated: question solved.
In powerquery,
right click the column,
home .. split column by delimiter ... delimiter:comma, Advanced Options:rows
then right click column and group by...
use default options and hit ok
let Source = Excel.CurrentWorkbook(){[Name="Phone_Number"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Phone Number", type text}}),
#"Split Column by Delimiter" = Table.ExpandListColumn(Table.TransformColumns(Table.TransformColumnTypes(#"Changed Type", {{"Phone Number", type text}}, "en-US"), {{"Phone Number", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Phone Number"),
#"Grouped Rows" = Table.Group(#"Split Column by Delimiter", {"Phone Number"}, {{"Count", each Table.RowCount(_), Int64.Type}})
in #"Grouped Rows"

Powerquery - rows to columns

I have sample data like below and I am trying to use PowerQuery to transpose it into different shape.
Here is my data:
Identifier Id
Account Type 1
Account Type 2
Account Type 3
Here is what I need:
Identifier Column.1 Column.2 Column.3
Account Type 1 2 3
I tried all combinations of Transpose + Unpivot but nothing worked.
You can Group by Identifier; then do a custom Text Aggregation which you can split into columns:
let
Source = Excel.CurrentWorkbook(){[Name="Table25"]}[Content],
//type the ID column as text for later purposes
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Identifier", type text}, {"Id", type text}}),
//group by Identifier, then custom Text aggregation with delimiter
#"Grouped Rows" = Table.Group(#"Changed Type", {"Identifier"}, {
{"Id",each Text.Combine([Id],";")}}),
//split the column by the delimiter, and set the data types
#"Split Column by Delimiter" = Table.SplitColumn(#"Grouped Rows", "Id", Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), {"Id.1", "Id.2", "Id.3"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Id.1", Int64.Type}, {"Id.2", Int64.Type}, {"Id.3", Int64.Type}})
in
#"Changed Type1"
How about this?
Method1: If there will only be one identifier type, then Add column... index column .... Then click select the index column and use transform...pivot column... and select ID as the values column, advanced options Don't Aggregate
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Added Index", {{"Index", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Added Index", {{"Index", type text}}, "en-US")[Index]), "Index", "Id")
in #"Pivoted Column"
Method2: If you plan to have different identifiers, then you need something a bit more complex. This adds the index within each group. Then you can pivot
let Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Grouped Rows" = Table.Group(Source, {"Identifier"}, {{"data", each Table.AddIndexColumn(_, "Index", 1, 1), type table}}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"Id", "Index"}, {"Id", "Index"}),
#"Pivoted Column" = Table.Pivot(Table.TransformColumnTypes(#"Expanded data", {{"Index", type text}}, "en-US"), List.Distinct(Table.TransformColumnTypes(#"Expanded data", {{"Index", type text}}, "en-US")[Index]), "Index", "Id", List.Sum)
in #"Pivoted Column"

Filter based on max value of other query

Lets assume we're having these two tables:
table_a
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxmDSBEyagkkzMGmuFBsLAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}})
in
#"Changed Type"
table_b
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxmDSBEyagkkzMGkOJi3ApCWYNDSAUBDdhkDtsQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column9 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column9", Int64.Type}})
in
#"Changed Type"
Goal:
I want to filter table_b like all values equal or less than max value from table_a.
In R or Python I'd do it like:
# R (data.table)
table_b[Column9 <= max(table_a$Column1)]
# Python (pandas)
table_b[table_b["Column9"] <= table_a["Column1"].max()]
But I have no idea how to solve this in M.
In table_b, add a Table.SelectRows that references max value of Column1 in table_a which is List.Max(table_a[Column1])
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxmDSBEyagkkzMGkOJi3ApCWYNDSAUBDdhkDtsQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column9 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column9", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Column9] <= List.Max(table_a[Column1]))
in #"Filtered Rows"

Power Query M - Custom Column for Rolling 28 Days Sales

I'm looking for some Power Query help. I have a huge set of sales data for 40k products over one year. For each product on each day I need to add a 28 day sales column.
I essentially want to do a sumifs like the below but in M.
=SUMIFS([SALES],[Product Code],[This Product Code],[Date],<=[This Date],[Date],>=[This Date]-28))
Try this then, it should work but would likely do so at a crawl
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Sales", Int64.Type}, {"Product Code", type text}, {"Date", type date}}),
TotalAmountAdded = Table.AddColumn(Source, "Total Amount", (i) => List.Sum(Table.SelectRows(Source, each ([Product Code] = i[Product Code] and [Date]<=i[Date] and [Date]>=Date.AddDays(i[Date],-28)))[Sales]), type number )
in TotalAmountAdded
Add a custom column with date logic (based on your sample sumif formula), filter the new column to get the relevant rows, then group by product code and sum Sales. Assuming source data is in Table1 with three columns (Sales,Product Code, Date) the code would be
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Sales", Int64.Type}, {"Product Code", type text}, {"Date", type date}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "AddMe", each if [Date]<=DateTime.Date(DateTime.LocalNow()) and [Date]>=Date.AddDays(DateTime.Date(DateTime.LocalNow()),-28) then 1 else 0),
#"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([AddMe] = 1)),
#"Grouped Rows" = Table.Group(#"Filtered Rows", {"Product Code"}, {{"ProductSales", each List.Sum([Sales]), type number}})
in #"Grouped Rows"

Resources