Power Query Function - get column name - powerquery

I have different files containing various attributes (one file will have Job Name, another one will have Location, another one will have Grade, etc.).
Sample Data:
Job Name
Job Location
Job Grade
Controller
Some values
High
Director
No Values
Low
Analyst
Some Other values
VP Of Finance
Blank
Medium
Director
I want to have a function that will add a column in which each cell will have combined: the Column Name (Job Name in case of the above sample) AND length of the Column Name (Length of each Job Name in case of the above sample).
Desired Output:
Job Name
Job Location
Job Grade
Desired Output
Controller
Some values
High
Job Name 10
Director
No Values
Low
Job Name 8
Analyst
Some Other values
Job Name 7
VP Of Finance
Blank
Medium
Job Name 13
Director
Executive
Job Name 8
What I have is:
(MyColumn) as text =>
let
Source = Text.Length(MyColumn),
NewSource = Number.ToText(Source),
FinalText = Text.Combine({MyColumn,NewSource}," ")
in FinalText
I need to find a way to reference the column name to be added to the FinalText, because for now the output is following:
Job Name
Current Output (incorrect!)
Engineer
Engineer 1
Director
Director 2
Analyst
Analyst 3
Engineer
Engineer 1
Btw I tried using Record.Field(_, MyColumn) but it didn't work with below error:
An error occurred in the ‘’ query. Expression.Error: The name '_' wasn't recognized. Make sure it's spelled correctly.
Thank you in advance for help.
Edit: I have edited the post, I hope it's more clear now.

Is this what you want?
zzz = Table.AddColumn(#"PriorStepNameHere", "Custom", each Table.ColumnNames(#"PriorStepNameHere"){0} &" "& Text.From(Text.Length(Record.Field(_,Table.ColumnNames(#"PriorStepNameHere"){0}))))

Here is a similar answer.
Table1 looks like this.
Job Name
Controller
Director
Analyst
VP Of Finance
Director
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Output", each Text.Combine({
Table.ColumnNames(Source){0}," ",
Text.From(Text.Length(Record.Field(_,Table.ColumnNames(Source){0})))
}))
in
#"Added Custom"
Edit 1 from comment:
Based on what you want I recommend that you don't use "Add Column>Invoke Custom Function". Instead in the "Applied Steps" right click and choose "Insert step after" you would then type in the =function_name(prior_step_table,number_of_column) the function is below.
(Table as table, column_n as number) =>
let
Source = Table,
#"Added Custom" = Table.AddColumn(Source, "Output", each Text.Combine({Table.ColumnNames(Source){column_n}," ",Text.From(Text.Length(Record.Field(_,Table.ColumnNames(Source){column_n})))}))
in
#"Added Custom"

Related

DAX or Power Query | Correct value of record based on related records within same table

The problem I'm trying to solve is on a multi-doctor planner database. Due to human error & bad habits, 3 different appointment status indicate to the user that the appointment actually took place. Ufortunately, there is an exception, evident only to the user, that takes place whenever a patient has more than one appointment on the same day, with the same doctor.
Case in point, Fulano de Tal had a multi stage consult with Dr. Smith on the 9th, starting at 13:30 hrs. The first 2 sessions (APP_IDs 2 and 3) are easily indentified as completed, but the one at 14:30 hrs had to have taken place, or would have been outright marked as cancelled. The reason it is known that APP_ID 4 took place is that 2 and 3 were completed. Fulano did not arrive to app_id 5, because it was on the next day, and there was no previous engagement on that day that could be used as a reference.
On the other hand, Pedrito was supposed to have a 3 stage consult with Dr. Doe. Pedrito did not arrive to APP_ID 6, but he did arrive for 7 and 8. APP_ID 7 completion is evident, but we only know 8 did so, because it was scheduled on the same day, at a later hour, whereas APP_ID 6 was scheduled before the one we know for certain took place.
APP_ID
Planner ID
Patient
Date
Date_Time
System Status
Completed?
1
Dr. Smith
Juan Perez
09-dec-2022
09-dec-2022 12:00
Completed
YES
2
Dr. Smith
Fulano de Tal
09-dec-2022
09-dec-2022 13:00
In Consult
YES
3
Dr. Smith
Fulano de Tal
09-dec-2022
09-dec-2022 13:30
Waiting
YES
4
Dr. Smith
Fulano de Tal
09-dec-2022
09-dec-2022 14:00
Called Upon
should be YES
5
Dr. Smith
Fulano de Tal
10-dec-2022
10-dec-2022 14:30
Called Upon
NO
6
Dr. Doe
Pedrito
09-dec-2022
09-dec-2022 09:00
Called Upon
NO
7
Dr. Doe
Pedrito
09-dec-2022
09-dec-2022 09:30
Completed
YES
8
Dr. Doe
Pedrito
09-dec-2022
09-dec-2022 10:00
Called Upon
should be YES
What I need is a calculated column that returns YES whenever:
The status is either Completed, In Consult or Waiting (this is the easy part)
The status is Called Upon AND the patient already had an appointment whose status is one of the above AND it took place on the same day AND it took place at a later time.
I already tried it on Dax, using a calculated countrows, like in this post, and adding additional conditions within the filter. But I guess because powerbi sorts the table so as to optimize storage, the earlier() function does can't properly do a sweep based on dates and time. Therefore, the solution might lie at powerquery, where I can use table.buffer to forcefully sort the table, but what I outright don't know how to do is add the calculated column that makes the full sweep to check for the easy condition and the four less than easy ones.
A solution in either powerquery or with dax work for me.
Please, help me out.
M / Powerquery method
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Date_Time", type datetime}}),
// x[] is current row, [] is entire table
#"Added Custom" = Table.AddColumn(#"Changed Type", "Custom",
(x)=>Table.RowCount(Table.SelectRows( #"Changed Type", each
[Patient]=x[Patient]
and ([System Status]="Completed" or [System Status]="In Consult" or [System Status]="Waiting")
and x[System Status]="Called Upon"
and [Date]=x[Date]
and [Date_Time]<x[Date_Time]))
),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "Completed", each if [System Status]="Completed" or [System Status]="In Consult" or [System Status]="Waiting" or [Custom]>0 then "YES" else "NO"),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom1",{"Custom"})
in #"Removed Columns"
EDITED VERISON
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
process=(z as table)=> let
#"ProcessTable" = Table.AddColumn(z, "Custom",
(x)=>Table.RowCount(Table.SelectRows( z, each
([System Status]="Completed" or [System Status]="In Consult" or [System Status]="Waiting")
and x[System Status]="Called Upon"
and [Date_Time]<x[Date_Time]))
) in #"ProcessTable",
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Date_Time", type datetime}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"Patient", "Date"}, {{"data", each process(_), type table }}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"APP_ID", "Planner ID", "Date_Time", "System Status", "Custom"}, {"APP_ID", "Planner ID", "Date_Time", "System Status", "Custom"}),
#"Added Custom1" = Table.AddColumn(#"Expanded data", "Completed", each if [System Status]="Completed" or [System Status]="In Consult" or [System Status]="Waiting" or [Custom]>0 then "YES" else "NO")
in #"Added Custom1"
#horseyride
In the end, I went for an auxiliary table. I made a reference query in powerquery, selecting only the states that guarantee the completion of each appointment, and the columns needed to relate the ambiguous consults in the main table.
Although the appointment IDs are not unique, they almost are. So, even though a calculated ditinct count proved to be too much for my machine, a countrows of a filtered table (by the relatable traits) proved quite fast.
Thank you #horseyride.

How to aggregate/group the data from one table and output the result as another table in PowerBI?

I have a Raw Data Table as shown in the screenshot below:
I want to group the data in the raw data table into the Output Table as shown in the screenshot below:
Basically the output table is counting the number of student for each understanding level in different intake. May I know how should I get the output table from the raw data table? I'm still new in Power Query, any help will be greatly appreciated!
This is what I have tried:
Code:
= Table.Group(Source, {"Intake"}, {
{"Count_Little_Understand", each Table.RowCount(Table.SelectRows(_, each ([Topic 1] = "Little Understanding"))), Int64.Type},
{"Count_General_Understanding", each Table.RowCount(Table.SelectRows(_, each ([Topic 1] = "General Understanding"))), Int64.Type},
{"Count_Good_Understand", each Table.RowCount(Table.SelectRows(_, each ([Topic 1] = "Good Understanding"))), Int64.Type},
{"Count_Fully_Understand", each Table.RowCount(Table.SelectRows(_, each ([Topic 1] = "Fully Understand"))), Int64.Type}
})
I only able to get the table by individual Topic, not sure how to include other Topic appended below and also add another extra column to label the Topic as shown in my second screenshot. Hope to get some advice/help on how should I modified the code. Thank you/1
I've rebuilt a similar but shorter table:
Now we first go into Transform (1), mark the Topic Cols (2) and Unpivot Columns (3).
Your table now looks like the following screenshot. Finally, we select the Value column (1), click on Pivot Column (2), select Employee Name (3).
Result:
You can Unpivot the Topic columns, then Pivot the Understanding column, using Count of Employee Name as the aggregate value.
Then simply reorder columns and sort rows, to suit the output you need:
#"Unpivoted Topic" = Table.UnpivotOtherColumns(#"Raw Data Table", {"Employee Name", "Intake"}, "Topic", "Understanding"),
#"Pivoted Understanding" = Table.Pivot(#"Unpivoted Topic", List.Distinct(#"Unpivoted Topic"[Understanding]), "Understanding", "Employee Name", List.NonNullCount),
#"Reordered Columns" = Table.ReorderColumns(#"Pivoted Understanding",{"Intake", "Topic", "Little Understanding", "General Understanding", "Good Understanding", "Fully Understand"}),
#"Sorted Rows" = Table.Sort(#"Reordered Columns",{{"Topic", Order.Ascending}, {"Intake", Order.Ascending}})
Output:

Can I add a new column with Linear Interpolation in Power Query M?

I am working on extracting an Interest Rate curve from futures market prices and create a table (Table 1) inside power query with the following columns:
- BusinessDays: Represents the nr o business days from today to the expiry of each future contract
- InterestRate: Represents the rate from today until the expiry of the futures contract
The second table (table 2) refers to the ID of internal financial products that expire in different business days.
- InstrumentID: Unique internal ID a financial product selled by a financial institution
- BusinessDays: Represents the nr o business days from today to the expiry of each financial product
I am having some trouble with M language, and unfortunately this specific calculation must be executed in Excel, so i am restricted to Power Query M.
The specific step i am not able to do is:
Creating a function in power query that adds a new column do table 2 containing the interpolated interest rate os each financial product.
The end result i am looking for would look like this
There are several ways to approach this but one way or another, you'll need to do some kind of lookup to determine which bracket to match your BusinessDays value with, so you can calculate the interpolated value.
I think it's simpler to just generate an all inclusive list of days vs interest rates, and then do a Join to pull out the matches.
I Name'd this first query intRates and expanded the Interest Rate table:
let
//Get the interest rate/business day table
Source = Excel.CurrentWorkbook(){[Name="intRates"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"BusinessDays", Int64.Type}, {"InterestRate", Percentage.Type}}),
//Add two columns which are the interest rate and business day columns offset by one
//It is faster to subtract this way than by adding an Index column
offset=
Table.FromColumns(
Table.ToColumns(#"Changed Type")
& {List.RemoveFirstN(#"Changed Type"[BusinessDays]) & {null}}
& {(List.RemoveFirstN(#"Changed Type"[InterestRate])) & {null}},
type table[BusinessDays=Int64.Type, InterestRate=Percentage.Type, shifted BusDays=Int64.Type, shifted IntRate=Percentage.Type]),
//Add a column with a list of the interest rates for each data interpolated between the segments
#"Added Custom" = Table.AddColumn(offset, "IntList", each let
sbd=[shifted BusDays],
intRateIncrement = ([shifted IntRate]-[InterestRate])/([shifted BusDays]-[BusinessDays]),
Lists= List.Generate(
()=>[d=[BusinessDays],i=[InterestRate]],
each [d]< sbd,
each [d=[d]+1, i = [i]+intRateIncrement],
each [i])
in Lists),
//add another column with a list of days corresponding to the interest rates
#"Added Custom1" = Table.AddColumn(#"Added Custom", "dayList", each {[BusinessDays]..[shifted BusDays]-1}),
//remove the last row as it will have an error
remErrRow = Table.RemoveLastN(#"Added Custom1",1),
//create the new table which has the rates for every duration
intRateTable = Table.FromColumns(
{List.Combine(remErrRow[dayList]),List.Combine(remErrRow[IntList])},
type table[Days=Int64.Type, Interest=Percentage.Type])
in
intRateTable
This results in a table that has every day (from 39 to , with its corresponding interest rate.
Then read in the "Instruments" table and Join it with the intRates, using a JoinKind.LeftOuter
let
Source = Excel.CurrentWorkbook(){[Name="Instruments"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"InstrumentID", type text}, {"BusinessDays", Int64.Type}}),
//add the rate column
#"Merged Queries" = Table.NestedJoin(#"Changed Type", {"BusinessDays"}, intRates, {"Days"}, "intRates", JoinKind.LeftOuter),
#"Expanded intRates" = Table.ExpandTableColumn(#"Merged Queries", "intRates", {"Interest"}, {"Interest"})
in
#"Expanded intRates"
Some of the results in the middle part of the table differ from what you've posted, but seem to be consistent with the linear interpolation formula for between two values, so I'm not sure how the discrepancy arises

In Power Query populate a column based on whether a value exists in another column

I am working in Excel Power Query. In the data an opportunity number (column A) will be listed several times; one row person assigned to helping on the opportunity. Another column (Column B) will indicate the name of each person and another (Column C) for the role of each assigned person. I want to create another column (Column D: All Roles) based on the following rule (Note this is a small sample but there are several other roles)I:
If the role PA is assigned to an opportunity then populate PA for every line of that opportunity.
If the role AA is assigned to an opportunity then populate AA for every line of that opportunity.
If the role PA and the role AA are assigned to an opportunity then populate PA, AA for every line of that opportunity.
Example
Probably 500 ways to do this
Remove everything except the role and opportunity columns. Remove duplicates. Group on Opportunity, and edit the resulting group code in formula window so that it merges the roles into a single cell, so that it ends with:
each Text.Combine([Role],",")
Merge that back into original data
full sample code:
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
// get list of unique roles for each Opportunity
#"Removed Columns" = Table.SelectColumns(Source,{"Opportunity", "Role"}),
#"Removed Duplicates" = Table.Distinct(#"Removed Columns"),
GroupEm = Table.Group(#"Removed Duplicates", {"Opportunity"}, {{"Concat", each Text.Combine(List.Sort([Role]),","), type text}}),
// merge into original data
#"Merged Queries" = Table.NestedJoin(Source,{"Opportunity"},GroupEm ,{"Opportunity"},"Table1",JoinKind.LeftOuter),
#"Expanded Table1" = Table.ExpandTableColumn(#"Merged Queries", "Table1", {"Concat"}, {"All Roles"})
in #"Expanded Table1"

In power query, how to turn a number into a duration in seconds?

I am trying, in power BI, to create change the type of one of my columns. Said column contain Numbers and I am trying to turn that number into a duration in seconds. But whenever I use the default type change, it turn the duration into days.
= Table.TransformColumnTypes(#"Changed Type",{{"Duration", type duration}})
is the default, I've tried puttin duration(seconds) or duration.seconds, but it didn't work.
I've looked around for a solution, but all I get are DAX solutions. I couldn't find much about power query in general.
Thanks for the help
I believe this does what you want.
If you start with a column called Seconds:
You can add a column with = Duration.From([Seconds]/86400) to get:
Alternatively, you could use:
= Table.ReplaceValue(Source, each [Seconds],each Duration.From([Seconds]/86400),Replacer.ReplaceValue,{"Seconds"})
to change...
directly to...
Here's the M code for the two different options:
Adding a column:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Duration", each Duration.From([Seconds]/86400))
in
#"Added Custom"
Directly changing:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Replaced Value" = Table.ReplaceValue(Source, each [Seconds],each Duration.From([Seconds]/86400),Replacer.ReplaceValue,{"Seconds"})
in
#"Replaced Value"

Resources