Is there an an equivalent to EARLIER in M/Power Query?
Say, I have a table with lots of different dates in column DATE and a smaller number of letters in column LETTER. I now want the maximum date for each letter.
In DAX, I would use something like CALCULATE(MAX([Date]),FILTER(ALL(Table),[Letter]=EARLIER([Letter])).
How would I achieve the same in M?
Thanks
2 Solutions in the code below. Notice that each uses "PreviousStep" as basis, so these are separate solutions.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
PreviousStep = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Letter", type text}}),
// 1. Add a column to the original table with the MaxDate for each letter
// "earlier" is just the name of a function parameter; it could as well have been "x" or "MarcelBeug"
AddedMaxDate = Table.AddColumn(PreviousStep, "MaxDate", (earlier) => List.Max(Table.SelectRows(PreviousStep, each [Letter] = earlier[Letter])[Date])),
// 2. Group by letter and get the MaxDate for each letter
GroupedOnLetter = Table.Group(PreviousStep, {"Letter"}, {{"MaxDate", each List.Max([Date]), type date}})
in
GroupedOnLetter
In short, there is no exact match for this function. Still, you can use other ways that can produce same results.
To reproduce example offered by Microsoft in help for EARLIER function, you can use following code (table1 equals table given in the example before ranking):
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("TVNNaxtBDP0rxuSoivn+uMYlLSUFE4f2YHIYd4d48Xq3rO1C/n01o1mc4+i9kZ6epP1+LcMa1o/94cvuOM3XCz0epHUgnccQ1m+wXytXGae8ekl/TpWhlACvBHrBDL8wdtc0dpWiLTgV0EVm1CrT9Trky4ooq016z5VnI2ij0OjKs402nVePM1XLrMgEcEaj8ZVU9czpxAmcAik1SlcxGSm2SX/5m4eoDVpToSJyc0z9WLEAwXgUrcX6a8hpzDNb4CAEhU5VuIjfzGk8XZoeGSVYpVBwd+X31zynfhjyjRM4A9FZ1NyWFhR7ymPX0hsJ0RuUbJ+s6DSzt96QtR4d96MK9m2Y/uVmfABtNVrWbSj2newc8iEtwjUoS401O2Rh5NQtyq0HZyNGFq4ZHs6Lz1aCjAopXmFV4I9uTtd+GlfbZfyR3IkafTOvJPlBneUPbj1GMCouMFkA6+f+/VhLcKjofp5aNmlBkKQ23JLs53QbrzSoVdkp3iYDWlgIzqBi6VJ9Jj7N6cxMA1ZSE16ga/XLTm3TOPZsPv8uora5SwNLMIIkK1Q8EF02bHs78xZJBS5alK1bCr1Mqbtro7+WfHPRoeZNk2Yh3XVpcNqBjgE9myuLrl3qaHg8GUUr5RYbVKlzP0kdLHhBJ9kOrsjfLQaWndCEWcZK8dfF7wcZIrkRUXNe7Ss6tzN8vR2WxTIQtMLQJl9Y023ux/d7o1JTHVOH0MyQ7hPv3isdh7F01gYFH5Aqvf7KF5akyLEYBYrmVpH0+5jz0C4nADEq+vYf", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ProductSubcategoryKey = _t, EnglishProductSubcategoryName = _t, TotalSubcategorySales = _t]),
table1 = Table.TransformColumnTypes(Source,{{"ProductSubcategoryKey", Int64.Type}, {"EnglishProductSubcategoryName", type text}, {"TotalSubcategorySales", Currency.Type}}, "en-US"),
AddCount = Table.AddColumn(
table1,
"SubcategoryRanking", //(a) is a parameter for function, which equals current record, and function should return value for new cell of "SubcategoryRanking"
(a)=> Table.RowCount(
Table.SelectRows(
table1, //(b) equals whole table1. This function returns table filtered by given criteria
(b) => b[TotalSubcategorySales] < a[TotalSubcategorySales])
) + 1,
Int64.Type)
in
AddCount
I think you can use the GroupBy function to group the data by Letter and find the Max of the date column. So your code should look like.
= Table.Group(#"Previous step", {"Letter"}, {{"Max Date", each List.Max([Date]), type date}})
Related
I am trying to convert columns to numeric. If TransformColumnTypes causes an error, I want to keep it text. Something like this:
#"Changed Type" = try Table.TransformColumnTypes(CombineTables,List.Transform(sTranCol, each {_, type number})), otherwise Table.TransformColumnTypes(CombineTables,List.Transform(sTranCol, each {_, type number})),
Obviously this doesn't work. sTranCol is the list of columns to covert to numeric. It is dynamically created and isn't static. I don't care if it puts error in the cell but transposing with errors in the cells is causing query to abort.
The M Code methods I've seen to detect data type of a column consist of sampling the data and determining the type. This seems messy.
But perhaps an alternative might be type the columns as numeric, and then replace the error values with something that won't cause a problem when transposing.
Here is some sample code to replace errors with null, but you could replace with anything null or numeric:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUTJUitWJBpI6SsZglhGQZQ5mVQBZiWCWKZCVBGaZA1kVEB0ghYYmSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,
List.Transform(Table.ColumnNames(Source), each {_,type nullable number})),
nullList = List.Transform(Table.ColumnNames(#"Changed Type"), each {_, null}),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Changed Type", nullList)
in
#"Replaced Errors"
Source
Changed Type
Replaced Errors
Edit: Add M Code to set column types depending on if all numeric
let
Source = Excel.CurrentWorkbook(){[Name="Table37"]}[Content],
//check data type
//if all numbers set to number, else any
colTypes = List.Accumulate(Table.ColumnNames(Source),
{},
(state,current)=> List.Combine({state,
if List.IsEmpty(
List.RemoveMatchingItems(
List.Transform(Table.Column(Source,current), each Value.Type(_)),
{type number}))
then {{current, type number}}
else {{current, type any}}})),
#"Changed Type" = Table.TransformColumnTypes(Source,colTypes)
in
#"Changed Type"
Source
Changed Type
This is the corrected code with the guidance from Alexis. My PDF returns two tables (and 1 page Table) per output page. Table001 is a throw away. I only need even numbered Tables so I use the List.Select to remove the Page Table and List.Alternate to skip odd numbered tables.
let
Source = Pdf.Tables(File.Contents("State_Fico.pdf"), [Implementation="1.3"]),
TableNames = List.Alternate(List.Select(Table.Column(Source, "Id"),each Text.Contains(_,"Table")),1,1),
TableList = List.Transform(TableNames, each Source{[Id=_]}[Data]),
CombineTables = Table.Combine(TableList)
in
CombineTables
This allows me to generate 1 table no matter how many pages the pdf is.
While this doesn't really answer the question in your title, I think the best way to do what you're ultimately after is to not use Expression.Evaluate at all and, instead, use list transformation(s).
For example, if you want to append Table002 and Table004, you can use Table.Combine on a list of tables, {Table002, Table004}.
Here's what the code might look like:
let
Source = Pdf.Tables(File.Contents("State_Fico.pdf"), [Implementation="1.3"]),
#"Transposed Table" = Table.Transpose(Source),
#"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Page001", type any}, {"Table001", type any}, {"Page002", type any}, {"Table002", type any}, {"Table003", type any}, {"Table004", type any}}),
ColumnNames = Table.ColumnNames(#"Changed Type"),
TableNames = List.Alternate(List.Select(ColumnNames, each Text.Contains(_ ,"Table")),1,1),
//--New Steps Below--//
TableList = List.Transform(TableNames, each Source{[ID=_]}[Data]),
CombineTables = Table.Combine(TableList)
in
CombineTables
I am having issues with my calculated column and the multiple tables I am joining. It is not filtering my visuals correctly. After researching it was recommended to use a custom column in the query instead but I do not know where to start to convert the following DAX to M query.
overall =
VAR skills =
CALCULATETABLE (
VALUES ( tsr_skill[ts_skill] ),
ALLEXCEPT ( tsr_skill, tsr_skill[ts_tsr] )
)
RETURN
SWITCH (
TRUE (),
"JMSR" IN skills, "Senior",
"JMOV" IN skills, "Over",
"JMUN" IN skills, "Under",
"JMRH" IN skills, "RHT",
"MNT"
)
Data structure in Query:
How I would like the data to show in the Query instead of showing as a calculated column.
Preferred Output:
Based on your explanation, and the levels assigned in your DAX formula, it would seem that all should be assigned as "under".
In your "Preferred Output" you do show JMXX being assigned as "Over", but that tsr does not include the JMOV skill
If your written explanation is correct, and your Preferred Output screenshot incorrect based on the posted data, then, in PQ you can
Group by tsr
Create a custom aggregation returning the "overall" based on containing one of the skills listed in your DAX formula.
If that is not the case, please clarify how you are assigning "Over" to JMXX.
Edit: M Code simplified
M Code
let
//Source = the data structure you show
Source = Excel.CurrentWorkbook(){[Name="Table13"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ts_tsr", type text}, {"ts_skill", type text}}),
//Group rows by tsr, then check if it has one of the defined skills
//If so, return the appropriate ranking.
#"Grouped Rows" = Table.Group(#"Changed Type", {"ts_tsr"}, {
{"ALL", each _, type table [ts_tsr=nullable text, ts_skill=nullable text]},
{"overall", each if List.Contains([ts_skill],"JMSR") then "Senior"
else if List.Contains([ts_skill],"JMOV") then "Over"
else if List.Contains([ts_skill],"JMUN") then "Under"
else if List.Contains([ts_skill],"JMRH") >=0 then "RHT"
else "MNT"}
}),
//Then re-expand the table
#"Expanded ALL" = Table.ExpandTableColumn(#"Grouped Rows", "ALL", {"ts_skill"}, {"ts_skill"})
in
#"Expanded ALL"
Data
Output
How can I turn this
Staff
Hours
Amy
5,10,20
Ben
6
Charles
10,1
into this using POWER QUERY?
Staff
Hours
Amy
35
Ben
6
Charles
11
Steps:
Convert the source table to an Excel Table
Get data from that table
Divide the column by delimiter (comma) into rows
Change the column type to number
Group rows and sum the Hours column
M code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
DivideByDelimiter = Table.ExpandListColumn(Table.TransformColumns(Table.TransformColumnTypes(Source, {{"Hours", type text}}, "es-CO"), {{"Hours", Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv), let itemType = (type nullable text) meta [Serialized.Text = true] in type {itemType}}}), "Hours"),
ChangeType = Table.TransformColumnTypes(DivideByDelimiter,{{"Hours", type number}}),
GroupRows = Table.Group(ChangeType, {"Staff"}, {{"Sum", each List.Sum([Hours]), type nullable number}})
in
GroupRows
Let me know if it works
Another way to do this
right click the Hours column and replace values, replacing the comma with a +
right click Hours column and transform lowercase
In formula bar, replace the resulting formula so that it ends with
, each Expression.Evaluate(_), type number}})
instead of
, Text.Lower, type text}})
Full sample code
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Staff", type text}, {"Hours", type text}}),
#"Replaced Value" = Table.ReplaceValue(#"Changed Type",",","+",Replacer.ReplaceText,{"Hours"}),
#"Lowercased Text" = Table.TransformColumns(#"Replaced Value",{{"Hours", each Expression.Evaluate(_), type number}})
in #"Lowercased Text"
I would like to choose a certain columns in power query, but not using their names. Ex. I can do this in R, by command: select. I'm wondering how i can do it in power query. I found some information here, but not all that I need.
Any idea, if I want to refer to more than one column?
It doesn't work if I write the code as below:
#"Filtered Part Desc" = Table.SelectRows (
#"Removed Columns3",
each List.Contains(
{ "ENG", "TRANS" },
Record.Field(_, Table.ColumnNames(#"Removed Columns3") { 5, 6, 7 })
)
)
Let's say I have this table and want to do a couple of things to it.
First, I want to change the column type of the second and last columns. We can use Table.ColumnNames to do this using simple indexing (which starts at zero) as follows:
Table.TransformColumnTypes(
Source,
{
{Table.ColumnNames(Source){1}, Int64.Type},
{Table.ColumnNames(Source){3}, Int64.Type}
}
)
That works but requires specifying each index separately. If we want to unpivot these columns like this
Table.Unpivot(#"Changed Type", {"Col2", "Col4"}, "Attribute", "Value")
but using the index values instead we can use the same method as above
Table.Unpivot(
#"Changed Type",
{
Table.ColumnNames(Source){1},
Table.ColumnNames(Source){3}
}, "Attribute", "Value"
)
But is there a way to do this where we can use a single list of positional index values and use Table.ColumnNames only once? I found a relatively simple though unintuitive method on this blog. For this case, it works as follows:
Table.Unpivot(
#"Changed Type",
List.Transform({1,3}, each Table.ColumnNames(Source){_}),
"Attribute", "Value"
)
This method starts with the list of positional index values and then transforms them into column names by looking up the names of the columns corresponding to those positions.
Here's the full M code for the query I was playing with:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUTIE4nIgtlSK1YlWSgKyjIC4AogtwCLJQJYxEFcCsTlYJAXIMgHiKiA2U4qNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Col1 = _t, Col2 = _t, Col3 = _t, Col4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{Table.ColumnNames(Source){1}, Int64.Type},{Table.ColumnNames(Source){3}, Int64.Type}}),
#"Unpivoted Columns" = Table.Unpivot(#"Changed Type", List.Transform({1,3}, each Table.ColumnNames(Source){_}), "Attribute", "Value")
in
#"Unpivoted Columns"