I have a table formatted as follows:
BOM
Imported
COM123
True
COM123
True
COM123
False
COM999
True
COM999
True
COM999
True
I'd like to filter the table to show only rows where all matching BOM rows are True in the Imported column. I.e., in this case, COM999 rows should show, and COM123 rows are filtered out because one entry is False.
FYI this is to produce a list of Production BOMs where all the components already exist in Business Central. The Imported column is the result from a merge of the query with an extract from BC, and sets the value to true where the BOM components exist.
Can anyone please give me a steer?
I've been farting around with this half the day, but I can't find the equivalent to a EXISTS query in SQL...
Algorithm should be clear in the code comments
Group by BOM
Test each BOM subgroup for ALLTRUE
Filter the Grouped table
Re-expand the original
let
//change next line to reflect actual data source
Source = Excel.CurrentWorkbook(){[Name="Table21"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"BOM", type text}, {"Imported", type logical}}),
//Group rows by BOM
//Then determine where all "Imported" = True
#"Grouped Rows" = Table.Group(#"Changed Type", {"BOM"}, {
{"all true", each List.AllTrue([Imported]), type logical},
{"all", each _, type table [BOM=nullable text, Imported=nullable logical]}}),
//Remove the false rows
//then delete the "all true" column
#"Filtered Rows" = Table.SelectRows(#"Grouped Rows", each ([all true] = true)),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"all true"}),
//re-expand
#"Expanded all" = Table.ExpandTableColumn(#"Removed Columns", "all", {"Imported"}, {"Imported"})
in
#"Expanded all"
Related
I can do things in PowerQuery but I can't find how to achieve the following result:
Before:
After:
The goal is to duplicate the last row (filtered with Project Code=null) for each item in Project Code column. I think duplicating the row as is is important to keep the Metadata Table and expand it later.
Thank you very much for your help.
Try this
Grab all rows except null row
Get unique values of Project column in a list
Grab null row
Create row with the list and expand it
Put the two tables back together.
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
AllButNull=Table.SelectRows( Source, each ([Project Code] <> null)),
UniqueProjects=List.Distinct(AllButNull[Project Code]),
OnlyNull = Table.SelectRows(#"Added Custom", each ([Project Code] = null)),
#"Replaced Value" = Table.ReplaceValue(OnlyNull,null, UniqueProjects,Replacer.ReplaceValue,{"Project Code"}),
#"Expanded Project Code" = Table.ExpandListColumn(#"Replaced Value" , "Project Code"),
combined = AllButNull & #"Expanded Project Code"
in combined
Alternately, grab the last row instead of the null row:
Grab all rows except last
Get unique values of Project column in a list
Grab last row
Create row with the list and expand it
Put the two tables back together.
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
AllButLast=Table.RemoveLastN(Source ,1),
UniqueProjects=List.Distinct(AllButLast[Project Code]),
OnlyBottomRow = Table.LastN(#"Added Custom", 1),
#"Replaced Value" = Table.ReplaceValue(OnlyBottomRow,null, UniqueProjects,Replacer.ReplaceValue,{"Project Code"}),
#"Expanded Project Code" = Table.ExpandListColumn(#"Replaced Value", "Project Code"),
combined = AllButLast & #"Expanded Project Code"
in combined
I am having issues with my calculated column and the multiple tables I am joining. It is not filtering my visuals correctly. After researching it was recommended to use a custom column in the query instead but I do not know where to start to convert the following DAX to M query.
overall =
VAR skills =
CALCULATETABLE (
VALUES ( tsr_skill[ts_skill] ),
ALLEXCEPT ( tsr_skill, tsr_skill[ts_tsr] )
)
RETURN
SWITCH (
TRUE (),
"JMSR" IN skills, "Senior",
"JMOV" IN skills, "Over",
"JMUN" IN skills, "Under",
"JMRH" IN skills, "RHT",
"MNT"
)
Data structure in Query:
How I would like the data to show in the Query instead of showing as a calculated column.
Preferred Output:
Based on your explanation, and the levels assigned in your DAX formula, it would seem that all should be assigned as "under".
In your "Preferred Output" you do show JMXX being assigned as "Over", but that tsr does not include the JMOV skill
If your written explanation is correct, and your Preferred Output screenshot incorrect based on the posted data, then, in PQ you can
Group by tsr
Create a custom aggregation returning the "overall" based on containing one of the skills listed in your DAX formula.
If that is not the case, please clarify how you are assigning "Over" to JMXX.
Edit: M Code simplified
M Code
let
//Source = the data structure you show
Source = Excel.CurrentWorkbook(){[Name="Table13"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ts_tsr", type text}, {"ts_skill", type text}}),
//Group rows by tsr, then check if it has one of the defined skills
//If so, return the appropriate ranking.
#"Grouped Rows" = Table.Group(#"Changed Type", {"ts_tsr"}, {
{"ALL", each _, type table [ts_tsr=nullable text, ts_skill=nullable text]},
{"overall", each if List.Contains([ts_skill],"JMSR") then "Senior"
else if List.Contains([ts_skill],"JMOV") then "Over"
else if List.Contains([ts_skill],"JMUN") then "Under"
else if List.Contains([ts_skill],"JMRH") >=0 then "RHT"
else "MNT"}
}),
//Then re-expand the table
#"Expanded ALL" = Table.ExpandTableColumn(#"Grouped Rows", "ALL", {"ts_skill"}, {"ts_skill"})
in
#"Expanded ALL"
Data
Output
I would like to choose a certain columns in power query, but not using their names. Ex. I can do this in R, by command: select. I'm wondering how i can do it in power query. I found some information here, but not all that I need.
Any idea, if I want to refer to more than one column?
It doesn't work if I write the code as below:
#"Filtered Part Desc" = Table.SelectRows (
#"Removed Columns3",
each List.Contains(
{ "ENG", "TRANS" },
Record.Field(_, Table.ColumnNames(#"Removed Columns3") { 5, 6, 7 })
)
)
Let's say I have this table and want to do a couple of things to it.
First, I want to change the column type of the second and last columns. We can use Table.ColumnNames to do this using simple indexing (which starts at zero) as follows:
Table.TransformColumnTypes(
Source,
{
{Table.ColumnNames(Source){1}, Int64.Type},
{Table.ColumnNames(Source){3}, Int64.Type}
}
)
That works but requires specifying each index separately. If we want to unpivot these columns like this
Table.Unpivot(#"Changed Type", {"Col2", "Col4"}, "Attribute", "Value")
but using the index values instead we can use the same method as above
Table.Unpivot(
#"Changed Type",
{
Table.ColumnNames(Source){1},
Table.ColumnNames(Source){3}
}, "Attribute", "Value"
)
But is there a way to do this where we can use a single list of positional index values and use Table.ColumnNames only once? I found a relatively simple though unintuitive method on this blog. For this case, it works as follows:
Table.Unpivot(
#"Changed Type",
List.Transform({1,3}, each Table.ColumnNames(Source){_}),
"Attribute", "Value"
)
This method starts with the list of positional index values and then transforms them into column names by looking up the names of the columns corresponding to those positions.
Here's the full M code for the query I was playing with:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUTIE4nIgtlSK1YlWSgKyjIC4AogtwCLJQJYxEFcCsTlYJAXIMgHiKiA2U4qNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Col1 = _t, Col2 = _t, Col3 = _t, Col4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{Table.ColumnNames(Source){1}, Int64.Type},{Table.ColumnNames(Source){3}, Int64.Type}}),
#"Unpivoted Columns" = Table.Unpivot(#"Changed Type", List.Transform({1,3}, each Table.ColumnNames(Source){_}), "Attribute", "Value")
in
#"Unpivoted Columns"
Suppose I have a column representing object type and another column representing object color. I want to remove blue and red fruits (example of object type) but keep all other red and blue objects.
How can I acheive this in Power Query ?
Thanks,
Just (un)select (not) matching rows
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
Filtered = Table.SelectRows(Source, each not ([ObjectType] = "Fruit" and ([ObjectColor]="Red" or [ObjectColor]="Blue")))
in
Filtered
Here's one way:
If you start with this:
You can merge the two columns together like this:
Then filter out the "Fruit,Blue" and "Fruit,Red":
Which yields this:
And you can then delete the "Merged" column to get this:
Here's the M code:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ObjectType", type text}, {"ObjectColor", type text}}),
#"Inserted Merged Column" = Table.AddColumn(#"Changed Type", "Merged", each Text.Combine({[ObjectType], [ObjectColor]}, ","), type text),
#"Filtered Rows" = Table.SelectRows(#"Inserted Merged Column", each ([Merged] <> "Fruit,Blue" and [Merged] <> "Fruit,Red")),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Merged"})
in
#"Removed Columns"
So I have two tables (power query), and want to combine them into one. The second table just looks at the first table (power query) and applies a parameter filter to it. When i try to combine the parameter code into the original query the filter doesn't work. I have enabled fast combine to made all queries public to get rid of any firewall issues.
So as not to break the original working set of pq, i duplicated the first pq and modified using advanced by coping the needed code to apply the parameter (third pq)
Second power query code (this looks at first pq an applies a parameter filter) and it works
let
Date_Parameter = Excel.CurrentWorkbook(){[Name="Parameter"]}[Content],
Date_Value = Date_Parameter{0}[Value],
Source = Excel.CurrentWorkbook(){[Name="Timesheet1"]}[Content],
#"Filtered Rows" = Table.SelectRows(Source, each ([Date] = Date_Value))
in
#"Filtered Rows"
Third power query code (this is the one where i duplicated the first pq and added parameter code from second pq) this doesn't work
let
Date_Parameter = Excel.CurrentWorkbook(){[Name="Parameter"]}[Content],
Date_Value = Date_Parameter{0}[Value],
Source = Excel.Workbook(File.Contents("\\192.168.12.31\Project Files\Daily Truck Sheet\TimeTrack\TimeTrack.xlsm")),
Timesheet_Table = Source{[Item="Timesheet",Kind="Table"]}[Data],
Merge = Table.NestedJoin(Timesheet_Table,{"Ref"},Project,{"Ref"},"NewColumn"),
#"Expand NewColumn" = Table.ExpandTableColumn(Merge, "NewColumn", {"Crew"}, {"NewColumn.Crew"}),
#"Renamed Columns" = Table.RenameColumns(#"Expand NewColumn",{{"NewColumn.Crew", "Crew"}}),
#"Removed Duplicates" = Table.Distinct(#"Renamed Columns", {"Ref"}),
#"Removed Columns" = Table.RemoveColumns(#"Removed Duplicates",{"Ref", "Employee Name", "Truck #", "Hours", "Per Diem", "Piecework", "Travel Day", "Timecard Filename", "Paid DT Hrs.", "hours check", "project hours", "Paid Regular Hours", "Paid OT Hrs.", "PayPeriod", "Employee Number", "Lead Hand Employee Number", "Crew Count", "Employee Revenue"}),
#"Reordered Columns"= Table.ReorderColumns(#"Removed Columns",{"Date", "Date Received", "Lead Hand", "Crew", "Project#", "Comments", "Work Performed", "time card hours", "Revenue per hour", "Total Reveneu"}),
Rounding = Table.TransformColumns(#"Reordered Columns",{{"Revenue per hour", each Number.Round(_, 2)}, {"Total Reveneu", each Number.Round(_, 2)}}),
#"Filtered Rows" = Table.SelectRows(Rounding, each ([Date] = Date_Value))
in
#"Filtered Rows"
so i had to insert a transform for pq to treat as a date. Even though in the Parameter pq (that loads the value from the parameter table) it is already transformed. replaced the first three lines before the source line with the following and it worked
Date_Parameter = Excel.CurrentWorkbook(){[Name="Parameter"]}[Content],
#"Changed Type1" = Table.TransformColumnTypes(Date_Parameter,{{"Value", type date}}),
Date_Value = #"Changed Type1"{0}[Value],
So maybe now i can get rid of the parameter pq as it is all built into the final pq but haven't tried yet