I'm trying to use the following code to get a new added column with information about a max value from a given field.
But what I would want is to have the possiblity to group a table where I filter it by a condition where the Custom column value would be 1.
I need to transform somehow the following parte of the code:
[
filter = [ID] /*should I add here another filter?/,
max2=Table.Group(
Source, {"ID"},
{{"MaxFiltered2", each List.Max([CODE])}}
){[ID=filter]}[MaxFiltered2]
][max2]
)
Here follows the code sequence:
#"Added Custom" = Table.AddColumn(
#"Changed Type",
"maxInt",
each
[
filter = [ID],
max=Table.Group(
Source, {"ID"},
{{"MaxFiltered", each List.Max([TAX])}}
){[ID=filter]}[MaxFiltered]
][max]
),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom", each if [maxInt]=[TAX] then 1 else 0),
#"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([Custom] = 1)),
#"Added Custom2" = Table.AddColumn(
#"Changed Type",
"maxInt2",
each
[
filter = [ID],
max2=Table.Group(
Source, {"ID"},
{{"MaxFiltered2", each List.Max([CODE])}}
){[ID=filter]}[MaxFiltered2]
][max2]
)
in
#"Added Custom2"
Inputput desired:
ID TAX CODE
A 4 921
A 6 500
A 6 200
B 2 700
B 2 500
B 1,5 100
Output desired:
ID TAX CODE
A 6 500
B 2 700
(to get the max for A and B IDs both on TAX and CODE variables)
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTIBYksjQ6VYHQjfDIhNDQxQ+EZQvhOIDcTmaHxTJL6hnimIBInEAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, TAX = _t, CODE = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"TAX", type number}, {"CODE", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {{"All", each _, type table [ID=nullable text, TAX=nullable number, CODE=nullable number]}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom",
each Table.First(
Table.Sort([All],{{"TAX", Order.Descending},{"CODE", Order.Descending}})
)
),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"ID", "All"}),
#"Expanded Custom" = Table.ExpandRecordColumn(#"Removed Columns", "Custom", {"ID", "TAX", "CODE"}, {"ID", "TAX", "CODE"})
in
#"Expanded Custom"
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WclTSUTIBYksjQ6VYHQjfDIhNDQxQ+EZQvhOIDcTmaHxTJL6hnimIBInEAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, TAX = _t, CODE = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", type text}, {"TAX", type number}, {"CODE", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID"}, {{"TAX", each List.Max([TAX]), type nullable number}, {"CODE", each List.Max([CODE]), type nullable number}})
in
#"Grouped Rows"
Related
Starting with the table above, the headers and their respective values are in alternating rows. For example for Nike, the Serial for the boots is 123 and Part No. is ABC, and it is sold on 12 Apr 22 for $23.03 with 20 left in stock. What I am trying to achieve by using power query is the following table:
I have tried adding an index and divide-integer 2 as there are 2 rows (1 header, 1 value) for each item sold and grouping using the resultant index. Then unpivot all except the index.
Then split the Attribute and Value columns using #(lf)
But I'm stuck here and running out of ideas. Any advice will be greatly appreciated. Thanks.
Try
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"AlternateRows"=Table.AlternateRows(Source,0,1,1),
#"Added Index" = Table.AddIndexColumn(AlternateRows, "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn( #"Added Index", "Custom", each Text.Split([Column2],"#(lf)")),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each Text.Split([Column4],"#(lf)")),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "Custom.2", each
Table.AddColumn(
Table.UnpivotOtherColumns(
Table.AddIndexColumn(
Table.FromColumns({[Custom],[Custom.1]})
, "Index", 0, 1, Int64.Type)
, {"Index"}, "Attribute", "Value")
,"Key", each Text.From([Index]) & Text.End([Attribute],1))
),
#"Expanded Custom.2" = Table.ExpandTableColumn(#"Added Custom2", "Custom.2", {"Value", "Key"}, {"Value", "Key"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Custom.2",{"Column2", "Column4", "Custom", "Custom.1"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns", List.Distinct(#"Removed Columns"[Key]), "Key", "Value"),
#"Renamed Columns" = Table.RenameColumns(#"Pivoted Column",{{"01", "Serial"}, {"02", "Date"}, {"11", "PartNo"}, {"12", "Price"}, {"22", "Item"}, {"32", "Stocks Left"}, {"Column1", "Currency"}, {"Column3", "Brand"}}),
#"Removed Columns1" = Table.RemoveColumns(#"Renamed Columns",{"Index"}),
#"Changed Type" = Table.TransformColumnTypes(#"Removed Columns1",{{"Date", type date}, {"Price", type number}, {"Stocks Left", type number}})
in #"Changed Type"
I'm using this code for creating a custom column!
max per group
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(
Source,
{{"Column1", type text}, {"Column2", Int64.Type}}
),
#"Added Custom" = Table.AddColumn(
#"Changed Type",
"maxInt",
each
[
filter = [Column1],
max=Table.Group(
Source, {"Column1"},
{{"MaxFiltered", each List.Max([Column2])}}
){[Column1=filter]}[MaxFiltered]
][max]
) in
#"Added Custom"
It works fine for getting the maxInt new column with the expected values!
But if I go on and try to reapeat the same code for adding another column to look for another maxInt but for another field, it gives me the output for this last column, and the other dissapears.
Can someone give a feedback on this matter?
Here follows my code:
#"Added Custom" = Table.AddColumn(
#"Changed Type",
"maxInt",
each
[
filter = [ID],
max=Table.Group(
Source, {"ID"},
{{"MaxFiltered", each List.Max([TAX])}}
){[BOOKING ID=filter]}[MaxFiltered]
][max]
),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom", each if [maxInt]=[TAX] then 1 else 0),
#"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([Custom] = 1)),
#"Added Custom2" = Table.AddColumn(
#"Changed Type",
"maxInt2",
each
[
filter = [ID],
max2=Table.Group(
Source, {"ID"},
{{"MaxFiltered2", each List.Max([CODE])}}
){[BOOKING ID=filter]}[MaxFiltered2]
][max2]
)
in
#"Added Custom2"
The Added Custom Column dissapears and it only gives me the Added Custom2.
Lets assume we're having these two tables:
table_a
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxmDSBEyagkkzMGmuFBsLAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}})
in
#"Changed Type"
table_b
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxmDSBEyagkkzMGkOJi3ApCWYNDSAUBDdhkDtsQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column9 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column9", Int64.Type}})
in
#"Changed Type"
Goal:
I want to filter table_b like all values equal or less than max value from table_a.
In R or Python I'd do it like:
# R (data.table)
table_b[Column9 <= max(table_a$Column1)]
# Python (pandas)
table_b[table_b["Column9"] <= table_a["Column1"].max()]
But I have no idea how to solve this in M.
In table_b, add a Table.SelectRows that references max value of Column1 in table_a which is List.Max(table_a[Column1])
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlSK1YlWMgKTxmDSBEyagkkzMGkOJi3ApCWYNDSAUBDdhkDtsQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column9 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column9", Int64.Type}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [Column9] <= List.Max(table_a[Column1]))
in #"Filtered Rows"
I have a range named tDetails. How can I query this using SQL?
I click in my range, and on the Power Query tab, I click "From Table/Range". This opens the Power Query editor.
In the advanced editor, I see something like:
let
Source = Excel.CurrentWorkbook(){[Name="tDetails"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"User ID", type text},
{"Company", type text},
{"State", type text},
{"Billed", type Int65.Type}})
in
#"Changed Type"
How do I go about querying this data? I'm needing to do things like:
SELECT 'UserID' = "User ID"
, 'Discount' = "Billed" * .51
, 'Note' = CASE WHEN State = 'NY' THEN 'Standard'
WHEN State = 'OH' THEN 'Second'
WHEN State = 'CA' THEN 'Third'
ELSE 'Fourth' END
FROM Source
WHERE "State" = "NY"
Also, can temp tables be used here?
SELECT 'UserID' = "User ID"
, 'Discount' = "Billed" * .51
INTO #UserDiscount
FROM Source
WHERE "State" = "NY"
SELECT * FROM #UserDiscount ORDER BY Discount DESC
These are super basic examples. What I really need to do is far more complex, but this would give me a place to start.
Thanks!
I think the equivalent Power Query would be
let
Source = Excel.CurrentWorkbook(){[Name="tDetails"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"User ID", type text},
{"Company", type text},
{"State", type text},
{"Billed", type Int65.Type}})
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([State] = "NY")),
#"Added Custom" = Table.AddColumn(#"Filtered Rows", "Discount", each [Billed]*0.51),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "UserID", each [UserID]),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"UserID", "Discount", "Billed"})
in
#"Removed Other Columns"
Also, the equivalent of SQL #temp table in Power Query would be use of table variable. In power query each step is table variable, e.g. Source, #"Changed Type", #"Filtered Rows", #"Added Custom".... which contains the table that was evaluated in that step. You can keep on utilizing the tables contained in each table variable in succeeding steps.
e.g.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUcrPS1WK1YlWMgKyS8rzwWxjEDujKBUoEwsA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", Int64.Type}, {"Column2", type text}}),
Custom0 = Table.SelectRows(#"Changed Type", each ([Column2] = "one")),
Custom1 = Table.SelectRows(#"Changed Type", each ([Column2] = "two")),
Custom2 = Table.SelectRows(#"Changed Type", each ([Column2] = "three")),
Custom3 = Custom0&Custom1&Custom2
in
Custom3
Yes, but you have to use a reserved function SqlExpression.ToExpression, and it doesn't look like your SQL is "valid" according to Power Query which likely needs in T-SQL format, so you'll have to experiment with how quotes are handled and columns renamed.
See this page for details: https://bengribaudo.com/blog/2021/07/13/5868/m-mysteries-sqlexpression-toexpression
Queries:
// tDetails
// Input from Excel range. This example uses embedded data so you can replicate my result.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WcspPUtJRcs7PK8kvzgey/CKBhKGRsYlSrE60kldqWhqKtL8HSNoADJRiYwE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"User ID" = _t, Company = _t, State = _t, Billed = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"User ID", type text}, {"Company", type text}, {"State", type text}, {"Billed", Int64.Type}})
in
#"Changed Type"
// Result
// Executes translated M code
let
ReferenceTables = [Source = tDetails],
Source = Expression.Evaluate(SQL, #shared)(ReferenceTables)
in
Source
// SQL
// Translates SQL to M
let
ReferenceTables = [Source = tDetails],
SqlQuery = "SELECT [User ID] as UserID, Billed * 0.51 as Discount, 'Standard' as [Note] FROM Source WHERE State = 'NY'",
TranslatedToM = SqlExpression.ToExpression(SqlQuery, ReferenceTables)
in
TranslatedToM
tDetails:
My SQL that works: SELECT [User ID] as UserID, Billed * 0.51 as Discount, 'Standard' as [Note] FROM Source WHERE State = 'NY'
Result:
I'm new to Power Query, so I'm hoping someone can help me with this problem.
I have a dataset with ID, Status, Start Date and End Date. There are multiple rows for each ID with different start and end dates. Here's a sample of my dataset.
ID
Status
Start Date
End Date
1
A
01/04/2015
28/05/2015
1
A
28/05/2015
15/06/2016
1
B
15/06/2016
19/06/2016
1
B
19/06/2016
31/07/2016
1
B
31/07/2016
2
B
01/03/2017
03/06/2018
2
A
03/06/2018
07/08/2018
2
A
07/08/2018
31/12/2018
2
C
31/12/2018
01/09/2019
2
C
01/09/2019
03/05/2020
2
A
03/05/2020
I want to group consecutive rows (End Date same as the Start Date of next row) with same status for each ID and take the oldest Start Date and the most recent End Date. End Date will be blank for records that are still active. Here is the the output I'm looking for.
ID
Status
Start Date
End Date
1
A
01/04/2015
15/06/2016
1
B
15/06/2016
2
B
01/03/2017
03/06/2018
2
A
03/06/2018
31/12/2018
2
C
31/12/2018
03/05/2020
2
A
03/05/2020
Is this possible?
Many Thanks for your help.
This M code seems to work for what you described. It starts with your sample data in a table named Table1 in an Excel spreadsheet.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1, Int64.Type),
#"Added Custom" = Table.AddColumn(#"Added Index", "New Start Date",
each if [Index] = 0 or ([Index] > 0 and [ID] = #"Added Index"[ID]{[Index]-1} and [Status] <> #"Added Index"[Status]{[Index]-1}) then [Start Date]
else if [ID] <> #"Added Index"[ID]{[Index]-1}
then [Start Date]
else null),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "New Finish Date",
each if [Index] = List.Last(#"Added Index"[Index]) or ([ID] <> #"Added Index"[ID]{[Index]+1} or [Status] <> #"Added Index"[Status]{[Index]+1}) then [End Date]
else null),
#"Filled Down" = Table.FillDown(#"Added Custom1",{"New Start Date"}),
#"Added Custom2" = Table.AddColumn(#"Filled Down", "Date Flag",
each if [Index] = 0
then if ([ID] <> #"Filled Down"[ID]{[Index]+1} or [Status] <> #"Filled Down"[Status]{[Index]+1}) or ([ID] = #"Filled Down"[ID]{[Index]+1} and [Status] = #"Filled Down"[Status]{[Index]+1} and [New Start Date] <> #"Filled Down"[New Start Date]{[Index]+1})
then 1
else 0
else if [Index] = List.Last(#"Filled Down"[Index])
then 1
else if ([ID] = #"Filled Down"[ID]{[Index]-1} and [Status] = #"Filled Down"[Status]{[Index]-1} and [New Start Date] <> #"Filled Down"[New Start Date]{[Index]+1}) or ([ID] = #"Filled Down"[ID]{[Index]+1} and [Status] <> #"Filled Down"[Status]{[Index]+1} and [New Start Date] <> #"Filled Down"[New Start Date]{[Index]+1})
then 1
else 0),
#"Filtered Rows" = Table.SelectRows(#"Added Custom2", each ([Date Flag] = 1)),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"ID", "Status", "New Start Date", "New Finish Date"})
in
#"Removed Other Columns"
First. You must change null value in End Data with Today Data
Second. Group by advanced... ID and Status with 2 New column name: min for Start data and max for End data
Below you have the code:
P.S. #Promoted Headers is necessary only in my sample.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type any}, {"Column2", type text}, {"Column3", type any}, {"Column4", type any}}),
#"Promoted Headers" = Table.PromoteHeaders(#"Changed Type", [PromoteAllScalars=true]),
#"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"Status", type text}, {"Start Date", type datetime}, {"End Date", type datetime}}),
#"Replaced null withToday" = Table.ReplaceValue(#"Changed Type1",null,DateTime.LocalNow(),Replacer.ReplaceValue,{"End Date"}),
#"Grouped Rows" = Table.Group(#"Replaced null withToday", {"ID", "Status"}, {{"Minim Data (1)", each List.Min([Start Date]), type nullable datetime}, {"Maxim Data (1)", each List.Max([End Date]), type nullable datetime}})
in
#"Grouped Rows"
Thanks #Marc Pincince and #Marius Durlea for your response. As #Storax' comments, I have got the solution now and it is here. Adding the code below too.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bZBLDoUgDEW3Yjom6YeHwFDfMoz734YtEKXRAYN7esItHAcwBNj0ECP9UIiTBilIqYczPM6EA3BCWi2st7N7rKF+O3VyohbntzPhAEsbyRjZqtFG2UIcd5Xb2TzWkJHKh/Pg1sfinb/HvbhaqM6ZcC+2PxJ67TNwe855AQ==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, Status = _t, #"Start Date" = _t, #"End Date" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"Status", type text}, {"Start Date", type date}, {"End Date", type date}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"ID", "Status"}, {{"Start Date", each List.First([Start Date]), type nullable date}, {"End Date", each List.Last([End Date]), type nullable date}},GroupKind.Local
)
in
#"Grouped Rows"