I would like to choose a certain columns in power query, but not using their names. Ex. I can do this in R, by command: select. I'm wondering how i can do it in power query. I found some information here, but not all that I need.
Any idea, if I want to refer to more than one column?
It doesn't work if I write the code as below:
#"Filtered Part Desc" = Table.SelectRows (
#"Removed Columns3",
each List.Contains(
{ "ENG", "TRANS" },
Record.Field(_, Table.ColumnNames(#"Removed Columns3") { 5, 6, 7 })
)
)
Let's say I have this table and want to do a couple of things to it.
First, I want to change the column type of the second and last columns. We can use Table.ColumnNames to do this using simple indexing (which starts at zero) as follows:
Table.TransformColumnTypes(
Source,
{
{Table.ColumnNames(Source){1}, Int64.Type},
{Table.ColumnNames(Source){3}, Int64.Type}
}
)
That works but requires specifying each index separately. If we want to unpivot these columns like this
Table.Unpivot(#"Changed Type", {"Col2", "Col4"}, "Attribute", "Value")
but using the index values instead we can use the same method as above
Table.Unpivot(
#"Changed Type",
{
Table.ColumnNames(Source){1},
Table.ColumnNames(Source){3}
}, "Attribute", "Value"
)
But is there a way to do this where we can use a single list of positional index values and use Table.ColumnNames only once? I found a relatively simple though unintuitive method on this blog. For this case, it works as follows:
Table.Unpivot(
#"Changed Type",
List.Transform({1,3}, each Table.ColumnNames(Source){_}),
"Attribute", "Value"
)
This method starts with the list of positional index values and then transforms them into column names by looking up the names of the columns corresponding to those positions.
Here's the full M code for the query I was playing with:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUTIE4nIgtlSK1YlWSgKyjIC4AogtwCLJQJYxEFcCsTlYJAXIMgHiKiA2U4qNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Col1 = _t, Col2 = _t, Col3 = _t, Col4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{Table.ColumnNames(Source){1}, Int64.Type},{Table.ColumnNames(Source){3}, Int64.Type}}),
#"Unpivoted Columns" = Table.Unpivot(#"Changed Type", List.Transform({1,3}, each Table.ColumnNames(Source){_}), "Attribute", "Value")
in
#"Unpivoted Columns"
Related
I am trying to convert columns to numeric. If TransformColumnTypes causes an error, I want to keep it text. Something like this:
#"Changed Type" = try Table.TransformColumnTypes(CombineTables,List.Transform(sTranCol, each {_, type number})), otherwise Table.TransformColumnTypes(CombineTables,List.Transform(sTranCol, each {_, type number})),
Obviously this doesn't work. sTranCol is the list of columns to covert to numeric. It is dynamically created and isn't static. I don't care if it puts error in the cell but transposing with errors in the cells is causing query to abort.
The M Code methods I've seen to detect data type of a column consist of sampling the data and determining the type. This seems messy.
But perhaps an alternative might be type the columns as numeric, and then replace the error values with something that won't cause a problem when transposing.
Here is some sample code to replace errors with null, but you could replace with anything null or numeric:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUTJUitWJBpI6SsZglhGQZQ5mVQBZiWCWKZCVBGaZA1kVEB0ghYYmSrGxAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t, Column2 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,
List.Transform(Table.ColumnNames(Source), each {_,type nullable number})),
nullList = List.Transform(Table.ColumnNames(#"Changed Type"), each {_, null}),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Changed Type", nullList)
in
#"Replaced Errors"
Source
Changed Type
Replaced Errors
Edit: Add M Code to set column types depending on if all numeric
let
Source = Excel.CurrentWorkbook(){[Name="Table37"]}[Content],
//check data type
//if all numbers set to number, else any
colTypes = List.Accumulate(Table.ColumnNames(Source),
{},
(state,current)=> List.Combine({state,
if List.IsEmpty(
List.RemoveMatchingItems(
List.Transform(Table.Column(Source,current), each Value.Type(_)),
{type number}))
then {{current, type number}}
else {{current, type any}}})),
#"Changed Type" = Table.TransformColumnTypes(Source,colTypes)
in
#"Changed Type"
Source
Changed Type
I had a formula in a table in excel
=IF([#STATUS]="",[KEY]&"_"&COUNTIF(INDEX([KEY],1):[#KEY],[#KEY]),"")
which showed me how often a value showed in the data. But the same is not working in Power Query
with the formula I use to get if the same value's position in a long data list, and then I use the same in index match formula to find and locate other relevant data
I am trying to achieve:
Date Name Frequency
1/10/2019 Adrian Bartholomeusz 1
1/10/2019 Aditya Tipnis 1
2/10/2019 Abdul Atef 1
2/10/2019 Aditya Tipnis 2
3/10/2019 Abdul Atef 2
In excel I used the formula "=COUNTIF(INDEX([Name],1):[#Name],[#Name])" but when I use the same in Power Query I am getting error
The key steps are:
Add Index
Group Rows
Transform Columns to add a sub-index.
Expand the data back.
The rest are cosmetics.
let
Source = Excel.CurrentWorkbook(),
Table1 = Source{[Name="Table1"]}[Content],
#"Added Index" = Table.AddIndexColumn(Table1, "Index", 0, 1),
#"Grouped Rows" = Table.Group(#"Added Index", {"key"}, {{"Data", each _, type table [key=number, f=text, Index=number]}}),
#"TransformColumns" = Table.TransformColumns(#"Grouped Rows",{"Data", (x) => Table.AddIndexColumn(x, "Index2", 1, 1)}),
#"Expanded Data" = Table.ExpandTableColumn(#"TransformColumns", "Data", {"excel formula", "Index", "Index2"}, {"excel formula", "Index", "Index2"}),
#"Added Custom" = Table.AddColumn(#"Expanded Data", "PQ method", each Text.From([key]) & "_" & Text.From([Index2])),
#"Sorted Rows" = Table.Sort(#"Added Custom",{{"Index", Order.Ascending}}),
#"Removed Columns" = Table.RemoveColumns(#"Sorted Rows",{"Index", "Index2"})
in
#"Removed Columns"
Suppose I have a column representing object type and another column representing object color. I want to remove blue and red fruits (example of object type) but keep all other red and blue objects.
How can I acheive this in Power Query ?
Thanks,
Just (un)select (not) matching rows
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Filtered = Table.SelectRows(Source, each not ([ObjectType] = "Fruit" and ([ObjectColor]="Red" or [ObjectColor]="Blue")))
in
Filtered
Here's one way:
If you start with this:
You can merge the two columns together like this:
Then filter out the "Fruit,Blue" and "Fruit,Red":
Which yields this:
And you can then delete the "Merged" column to get this:
Here's the M code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ObjectType", type text}, {"ObjectColor", type text}}),
#"Inserted Merged Column" = Table.AddColumn(#"Changed Type", "Merged", each Text.Combine({[ObjectType], [ObjectColor]}, ","), type text),
#"Filtered Rows" = Table.SelectRows(#"Inserted Merged Column", each ([Merged] <> "Fruit,Blue" and [Merged] <> "Fruit,Red")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Merged"})
in
#"Removed Columns"
Is there an an equivalent to EARLIER in M/Power Query?
Say, I have a table with lots of different dates in column DATE and a smaller number of letters in column LETTER. I now want the maximum date for each letter.
In DAX, I would use something like CALCULATE(MAX([Date]),FILTER(ALL(Table),[Letter]=EARLIER([Letter])).
How would I achieve the same in M?
Thanks
2 Solutions in the code below. Notice that each uses "PreviousStep" as basis, so these are separate solutions.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
PreviousStep = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Letter", type text}}),
// 1. Add a column to the original table with the MaxDate for each letter
// "earlier" is just the name of a function parameter; it could as well have been "x" or "MarcelBeug"
AddedMaxDate = Table.AddColumn(PreviousStep, "MaxDate", (earlier) => List.Max(Table.SelectRows(PreviousStep, each [Letter] = earlier[Letter])[Date])),
// 2. Group by letter and get the MaxDate for each letter
GroupedOnLetter = Table.Group(PreviousStep, {"Letter"}, {{"MaxDate", each List.Max([Date]), type date}})
in
GroupedOnLetter
In short, there is no exact match for this function. Still, you can use other ways that can produce same results.
To reproduce example offered by Microsoft in help for EARLIER function, you can use following code (table1 equals table given in the example before ranking):
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("TVNNaxtBDP0rxuSoivn+uMYlLSUFE4f2YHIYd4d48Xq3rO1C/n01o1mc4+i9kZ6epP1+LcMa1o/94cvuOM3XCz0epHUgnccQ1m+wXytXGae8ekl/TpWhlACvBHrBDL8wdtc0dpWiLTgV0EVm1CrT9Trky4ooq016z5VnI2ij0OjKs402nVePM1XLrMgEcEaj8ZVU9czpxAmcAik1SlcxGSm2SX/5m4eoDVpToSJyc0z9WLEAwXgUrcX6a8hpzDNb4CAEhU5VuIjfzGk8XZoeGSVYpVBwd+X31zynfhjyjRM4A9FZ1NyWFhR7ymPX0hsJ0RuUbJ+s6DSzt96QtR4d96MK9m2Y/uVmfABtNVrWbSj2newc8iEtwjUoS401O2Rh5NQtyq0HZyNGFq4ZHs6Lz1aCjAopXmFV4I9uTtd+GlfbZfyR3IkafTOvJPlBneUPbj1GMCouMFkA6+f+/VhLcKjofp5aNmlBkKQ23JLs53QbrzSoVdkp3iYDWlgIzqBi6VJ9Jj7N6cxMA1ZSE16ga/XLTm3TOPZsPv8uora5SwNLMIIkK1Q8EF02bHs78xZJBS5alK1bCr1Mqbtro7+WfHPRoeZNk2Yh3XVpcNqBjgE9myuLrl3qaHg8GUUr5RYbVKlzP0kdLHhBJ9kOrsjfLQaWndCEWcZK8dfF7wcZIrkRUXNe7Ss6tzN8vR2WxTIQtMLQJl9Y023ux/d7o1JTHVOH0MyQ7hPv3isdh7F01gYFH5Aqvf7KF5akyLEYBYrmVpH0+5jz0C4nADEq+vYf", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ProductSubcategoryKey = _t, EnglishProductSubcategoryName = _t, TotalSubcategorySales = _t]),
table1 = Table.TransformColumnTypes(Source,{{"ProductSubcategoryKey", Int64.Type}, {"EnglishProductSubcategoryName", type text}, {"TotalSubcategorySales", Currency.Type}}, "en-US"),
AddCount = Table.AddColumn(
table1,
"SubcategoryRanking", //(a) is a parameter for function, which equals current record, and function should return value for new cell of "SubcategoryRanking"
(a)=> Table.RowCount(
Table.SelectRows(
table1, //(b) equals whole table1. This function returns table filtered by given criteria
(b) => b[TotalSubcategorySales] < a[TotalSubcategorySales])
) + 1,
Int64.Type)
in
AddCount
I think you can use the GroupBy function to group the data by Letter and find the Max of the date column. So your code should look like.
= Table.Group(#"Previous step", {"Letter"}, {{"Max Date", each List.Max([Date]), type date}})
So I have two tables (power query), and want to combine them into one. The second table just looks at the first table (power query) and applies a parameter filter to it. When i try to combine the parameter code into the original query the filter doesn't work. I have enabled fast combine to made all queries public to get rid of any firewall issues.
So as not to break the original working set of pq, i duplicated the first pq and modified using advanced by coping the needed code to apply the parameter (third pq)
Second power query code (this looks at first pq an applies a parameter filter) and it works
let
Date_Parameter = Excel.CurrentWorkbook(){[Name="Parameter"]}[Content],
Date_Value = Date_Parameter{0}[Value],
Source = Excel.CurrentWorkbook(){[Name="Timesheet1"]}[Content],
#"Filtered Rows" = Table.SelectRows(Source, each ([Date] = Date_Value))
in
#"Filtered Rows"
Third power query code (this is the one where i duplicated the first pq and added parameter code from second pq) this doesn't work
let
Date_Parameter = Excel.CurrentWorkbook(){[Name="Parameter"]}[Content],
Date_Value = Date_Parameter{0}[Value],
Source = Excel.Workbook(File.Contents("\\192.168.12.31\Project Files\Daily Truck Sheet\TimeTrack\TimeTrack.xlsm")),
Timesheet_Table = Source{[Item="Timesheet",Kind="Table"]}[Data],
Merge = Table.NestedJoin(Timesheet_Table,{"Ref"},Project,{"Ref"},"NewColumn"),
#"Expand NewColumn" = Table.ExpandTableColumn(Merge, "NewColumn", {"Crew"}, {"NewColumn.Crew"}),
#"Renamed Columns" = Table.RenameColumns(#"Expand NewColumn",{{"NewColumn.Crew", "Crew"}}),
#"Removed Duplicates" = Table.Distinct(#"Renamed Columns", {"Ref"}),
#"Removed Columns" = Table.RemoveColumns(#"Removed Duplicates",{"Ref", "Employee Name", "Truck #", "Hours", "Per Diem", "Piecework", "Travel Day", "Timecard Filename", "Paid DT Hrs.", "hours check", "project hours", "Paid Regular Hours", "Paid OT Hrs.", "PayPeriod", "Employee Number", "Lead Hand Employee Number", "Crew Count", "Employee Revenue"}),
#"Reordered Columns"= Table.ReorderColumns(#"Removed Columns",{"Date", "Date Received", "Lead Hand", "Crew", "Project#", "Comments", "Work Performed", "time card hours", "Revenue per hour", "Total Reveneu"}),
Rounding = Table.TransformColumns(#"Reordered Columns",{{"Revenue per hour", each Number.Round(_, 2)}, {"Total Reveneu", each Number.Round(_, 2)}}),
#"Filtered Rows" = Table.SelectRows(Rounding, each ([Date] = Date_Value))
in
#"Filtered Rows"
so i had to insert a transform for pq to treat as a date. Even though in the Parameter pq (that loads the value from the parameter table) it is already transformed. replaced the first three lines before the source line with the following and it worked
Date_Parameter = Excel.CurrentWorkbook(){[Name="Parameter"]}[Content],
#"Changed Type1" = Table.TransformColumnTypes(Date_Parameter,{{"Value", type date}}),
Date_Value = #"Changed Type1"{0}[Value],
So maybe now i can get rid of the parameter pq as it is all built into the final pq but haven't tried yet