Related
I have been searching for a week now and cannot find a resolution to my problem. I have a table which lists the "event" and individual is in during a certain week. I want to add a column - via PowerQuery - that will count the number of weeks a person has been in that event and then resets if the event changes in the following week. For example...
Pers1
Date
Event
Weeks in Event
Pers1
12/22/2022
Consideration
1
Pers1
12/26/2022
Consideration
2
Pers1
1/05/2022
Interview
1
Pers1
1/12/2022
Consideration
1
Pers1
1/19/2022
Consideration
2
Pers1
1/26/2022
Awaiting Hire
1
Pers2
1/19/2022
Awaiting Hire
1
Pers2
1/26/2022
Awaiting Hire
2
Note how the count resets back to starting at 1 when Pers1 has their second stint of Consideration during weeks 1/12 and 1/19. Additionally, I need the solution to be smart enough to distinguish between two different individuals and uniquely count their time in an event.
This community has always come through for me. Please help!
EDIT 1: I incorporated the code provided by Ron and am receiving the following error: Expression.Error: 5 arguments were passed to function which expects between 2 and 4.
Details:
Pattern=
Arguments=List
PQ Advanced Editor Code is below:
let
Source = Excel.Workbook(File.Contents("C:\Location"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Pers1", type text}, {"Date", type date}, {"Event", type text}}),
//add an offset column for Pers and Event to do easy comparison with previous row
offsetPersonEvent=Table.FromColumns(
Table.ToColumns(#"Changed Type")
& {List.RemoveFirstN(#"Changed Type"[Pers1]) & {null}}
& {List.RemoveFirstN(#"Changed Type"[Event]) & {null}},
type table[Pers=text, Date=date,Event=text,offsetPers=text, offsetEvent=text]
),
//create "grouper" column by checking where both Pers and Event change
#"Added Index" = Table.AddIndexColumn(offsetPersonEvent, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "groups",
each if [Pers]=[offsetPers] and [Event]=[offsetEvent] then null else [Index]),
//remove unneeded columns, fillUp the grouper, and group by "grouper"
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"offsetPers", "offsetEvent", "Index"}),
#"Filled Up" = Table.FillUp(#"Removed Columns",{"groups"}),
//Add Index column to each subtable
#"Grouped Rows" = Table.Group(#"Filled Up", {"groups"}, {
{"addedIndex", each Table.AddIndexColumn(_,"Weeks in Event",1,1,Int64.Type)
, type table}}),
//Remove unneccessary columns
// Expand the grouped tables
// reset the data types
#"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"groups"}),
#"Expanded addedIndex" = Table.ExpandTableColumn(#"Removed Columns1", "addedIndex", {"Pers", "Date", "Event", "Weeks in Event"}, {"Pers", "Date", "Event", "Weeks in Event"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded addedIndex",{{"Pers", type text}, {"Date", type date}, {"Event", type text}, {"Weeks in Event", Int64.Type}})
in
#"Changed Type1"
Assuming your data is sorted by Person and then by Date, as you show, you can use the following M-Code.
(If your data is not so sorted, then you could merely add steps initially to sort it appropriately, and then continue with the code shown)
Please read the code comments and examine the Applied steps to understand the algorithm
Open the Advanced Editor and paste in the code below
let
//change table name in next line to your actual table name
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Pers1", type text}, {"Date", type date}, {"Event", type text}}),
//add an offset column for Pers and Event to do easy comparison with previous row
offsetPersonEvent=Table.FromColumns(
Table.ToColumns(#"Changed Type")
& {List.RemoveFirstN(#"Changed Type"[Pers1]) & {null}}
& {List.RemoveFirstN(#"Changed Type"[Event]) & {null}},
type table[Pers=text, Date=date,Event=text,offsetPers=text, offsetEvent=text]
),
//create "grouper" column by checking where both Pers and Event change
#"Added Index" = Table.AddIndexColumn(offsetPersonEvent, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "groups",
each if [Pers]=[offsetPers] and [Event]=[offsetEvent] then null else [Index]),
//remove unneeded columns, fillUp the grouper, and group by "grouper"
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"offsetPers", "offsetEvent", "Index"}),
#"Filled Up" = Table.FillUp(#"Removed Columns",{"groups"}),
//Add Index column to each subtable
#"Grouped Rows" = Table.Group(#"Filled Up", {"groups"}, {
{"addedIndex", each Table.AddIndexColumn(_,"Weeks in Event",1,1,Int64.Type)
, type table}}),
//Remove unneccessary columns
// Expand the grouped tables
// reset the data types
#"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"groups"}),
#"Expanded addedIndex" = Table.ExpandTableColumn(#"Removed Columns1", "addedIndex", {"Pers", "Date", "Event", "Weeks in Event"}, {"Pers", "Date", "Event", "Weeks in Event"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded addedIndex",{{"Pers", type text}, {"Date", type date}, {"Event", type text}, {"Weeks in Event", Int64.Type}})
in
#"Changed Type1"
Ron's code worked with one minor tweak. For me, the following section of code was causing an error
//Add Index column to each subtable
#"Grouped Rows" = Table.Group(#"Filled Up", {"groups"}, {
{"addedIndex", each Table.AddIndexColumn(_,"Weeks in Event",1,1,Int64.Type)
, type table}}),
I removed the Int64.Type parameter from the Table.AddIndexColumn and everything functioned. I've included the updated code snip-it below:
let
Source = Excel.Workbook(File.Contents("C:\Location"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Pers1", type text}, {"Date", type date}, {"Event", type text}}),
//add an offset column for Pers and Event to do easy comparison with previous row
offsetPersonEvent=Table.FromColumns(
Table.ToColumns(#"Changed Type")
& {List.RemoveFirstN(#"Changed Type"[Pers1]) & {null}}
& {List.RemoveFirstN(#"Changed Type"[Event]) & {null}},
type table[Pers=text, Date=date,Event=text,offsetPers=text, offsetEvent=text]
),
//create "grouper" column by checking where both Pers and Event change
#"Added Index" = Table.AddIndexColumn(offsetPersonEvent, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index", "groups",
each if [Pers]=[offsetPers] and [Event]=[offsetEvent] then null else [Index]),
//remove unneeded columns, fillUp the grouper, and group by "grouper"
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"offsetPers", "offsetEvent", "Index"}),
#"Filled Up" = Table.FillUp(#"Removed Columns",{"groups"}),
//Add Index column to each subtable
#"Grouped Rows" = Table.Group(#"Filled Up", "groups", {
{"addedIndex", each Table.AddIndexColumn(_,"Weeks in Event",1,1)
, type table}}),
//Remove unneccessary columns
// Expand the grouped tables
// reset the data types
#"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"groups"}),
#"Expanded addedIndex" = Table.ExpandTableColumn(#"Removed Columns1", "addedIndex", {"Pers", "Date", "Event", "Weeks in Event"}, {"Pers", "Date", "Event", "Weeks in Event"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded addedIndex",{{"Pers", type text}, {"Date", type date}, {"Event", type text}, {"Weeks in Event", Int64.Type}})
in
#"Changed Type1"
Situation: I am working in Power Query with data that was imported from a pdf file, and it came in a bit messy. I have a column that contains numbers, as well as text strings. Some of the text strings are mixed-case, containing both upper and lower case characters, while others contain only upper case characters.
Goal: I want to remove all numbers and all mixed-case text strings. The end result should show only the text strings that are completely upper case.
For example, I would want my end result to include things like IRA, IRREVOCABLE TRUST, CHARITABLE TRUST, but replace things like Number of Accounts, Totals, 14 with null.
What I have tried so far:
The following gets rid of numbers and lower case characters, but it doesn't quite work since it keeps upper case characters included in mixed-case character strings.
Table.AddColumn(#"Added Custom2", "Account Type" each Text.Select([AccountType], {"A".."Z"," "}), type text)
The following code gets rid of the mixed-case text strings, but it doesn't quite work because it doesn't remove the numbers. Also, it is too specific, requiring me to remove strings that contain specific words. I'd prefer to remove all strings that contain lower case characters.
Table.AddColumn(#"Added Custom2", "Account Type", each if [AccountType]= null or Text.Contains([AccountType],"Totals") or Text.Contains([AccountType],"of") or Text.Contains([AccountType],"report") then null else [AccountType])
Your insights would be appreciated. I'm a new PowerQuery user, so please be specific and detailed with your responses.
Hard to know exactly what you want since you provide no examples of your data.
If you have multiple (space-separated) strings in a cell, then you can use:
#"Added Custom" = Table.AddColumn(#"Changed Type", "allCaps", each
Text.Combine(
List.Accumulate(Text.Split([Column1]," "),
{},
(state, current)=>
if List.ContainsAny(
Text.ToList(current),
{"0".."9","a".."z",",",":","?","/","\"," "})
then state
else state & {current}),", "))
If you just have a single string in a cell, then you can use
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each
if List.ContainsAny(
Text.ToList([Column1]),
{"0".."9","a".."z",",",":","?","/","\"," "})
then null
else [Column1])
Then you can filter by deselecting the null in the added column
In each case, #"Changed Type" is the previous step. If that is not the case in your code, replace with the actual name of your previous step
First formula just looks for all capitals that dont contain a number
= if [Column1] = Text.Remove ([Column1],{"0".."9","a".."z"}) then [Column1] else null
Second formula removes all numbers, THEN looks for all capitals that dont contain a number
= if Text.Remove ([Column1],{"0".."9"}) = Text.Remove ([Column1],{"a".."z","0".."9"}) then Text.Remove ([Column1],{"0".."9"}) else null
let Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each if [Column1] = Text.Remove ([Column1],{"0".."9","a".."z"}) then [Column1] else null),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom2", each if Text.Remove ([Column1],{"0".."9"}) = Text.Remove ([Column1],{"a".."z","0".."9"}) then Text.Remove ([Column1],{"0".."9"}) else null)
in #"Added Custom1"
~ ~ ~
If you are looking to parse words out of a list of words.......
This retains words that (a) have no numbers before after or within, and (b) are all in capitals
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each
Text.Combine(
List.RemoveNulls(
List.Transform(Text.Split([Column1]," "), each
if _ = Text.Remove (_,{"a".."z","0".."9"}) then _ else null
))," "))
in #"Added Custom"
This removes all numbers, and after that retains words that are all in capitals
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom", each
Text.Combine(
List.RemoveNulls(
List.Transform(Text.Split(Text.Remove ([Column1],{"0".."9"})," "), each
if _ = Text.Remove (_,{"a".."z"}) then _ else null
))," "))
in #"Added Custom"
I had a formula in a table in excel
=IF([#STATUS]="",[KEY]&"_"&COUNTIF(INDEX([KEY],1):[#KEY],[#KEY]),"")
which showed me how often a value showed in the data. But the same is not working in Power Query
with the formula I use to get if the same value's position in a long data list, and then I use the same in index match formula to find and locate other relevant data
I am trying to achieve:
Date Name Frequency
1/10/2019 Adrian Bartholomeusz 1
1/10/2019 Aditya Tipnis 1
2/10/2019 Abdul Atef 1
2/10/2019 Aditya Tipnis 2
3/10/2019 Abdul Atef 2
In excel I used the formula "=COUNTIF(INDEX([Name],1):[#Name],[#Name])" but when I use the same in Power Query I am getting error
The key steps are:
Add Index
Group Rows
Transform Columns to add a sub-index.
Expand the data back.
The rest are cosmetics.
let
Source = Excel.CurrentWorkbook(),
Table1 = Source{[Name="Table1"]}[Content],
#"Added Index" = Table.AddIndexColumn(Table1, "Index", 0, 1),
#"Grouped Rows" = Table.Group(#"Added Index", {"key"}, {{"Data", each _, type table [key=number, f=text, Index=number]}}),
#"TransformColumns" = Table.TransformColumns(#"Grouped Rows",{"Data", (x) => Table.AddIndexColumn(x, "Index2", 1, 1)}),
#"Expanded Data" = Table.ExpandTableColumn(#"TransformColumns", "Data", {"excel formula", "Index", "Index2"}, {"excel formula", "Index", "Index2"}),
#"Added Custom" = Table.AddColumn(#"Expanded Data", "PQ method", each Text.From([key]) & "_" & Text.From([Index2])),
#"Sorted Rows" = Table.Sort(#"Added Custom",{{"Index", Order.Ascending}}),
#"Removed Columns" = Table.RemoveColumns(#"Sorted Rows",{"Index", "Index2"})
in
#"Removed Columns"
I have a power query, with some results maybe a null value. How ever, when I import the data into excel, the null value changed to blank. How can I keep the null value?
Why am I doing so? There are other cells calculating based on this cells. In case there is an error, it means 'this data is not available'. But if it is blank, it will be dealt as 0, which totally has the different meanings.
How about adding a conditional column with null replaced as "null" ?
Here is a screenshot:
Here is the M code:
#"Promoted Headers" = Table.PromoteHeaders(Tabelle1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Test", type any}}),
#"Added Conditional Column" = Table.AddColumn(#"Changed Type", "Custom", each if [Test] = null then "null" else [Test])
in
#"Added Conditional Colum
I hope this helps
I'm trying to figure out a simple way (using PowerQuery) to convert this:
into this:
I've spent days trying to figure out a simple way to do it.
Everything I've tried has failed.
You can use Group By on the Transform tab, group by project and define a aggregation for each of the segment columns (e.g. Sum).
Then adjust the created code from List.Sum to List.RemoveNulls.
Then add a column with nested tables from the segment columns, using Table.FromColumns.
Remove the original segment columns and expand the nested tables.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Project", type text}, {"Segment1", type text}, {"Segment2", type text}, {"Segment3", type text}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Project"}, {{"Segment1", each List.RemoveNulls([Segment1]), type text}, {"Segment2", each List.RemoveNulls([Segment2]), type text}, {"Segment3", each List.RemoveNulls([Segment3]), type text}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Tabled", each Table.FromColumns({[Segment1],[Segment2],[Segment3]},{"Segment1","Segment2","Segment3"})),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Segment1", "Segment2", "Segment3"}),
#"Expanded Tabled" = Table.ExpandTableColumn(#"Removed Columns", "Tabled", {"Segment1", "Segment2", "Segment3"}, {"Segment1", "Segment2", "Segment3"})
in
#"Expanded Tabled"