I am having issues with my calculated column and the multiple tables I am joining. It is not filtering my visuals correctly. After researching it was recommended to use a custom column in the query instead but I do not know where to start to convert the following DAX to M query.
overall =
VAR skills =
CALCULATETABLE (
VALUES ( tsr_skill[ts_skill] ),
ALLEXCEPT ( tsr_skill, tsr_skill[ts_tsr] )
)
RETURN
SWITCH (
TRUE (),
"JMSR" IN skills, "Senior",
"JMOV" IN skills, "Over",
"JMUN" IN skills, "Under",
"JMRH" IN skills, "RHT",
"MNT"
)
Data structure in Query:
How I would like the data to show in the Query instead of showing as a calculated column.
Preferred Output:
Based on your explanation, and the levels assigned in your DAX formula, it would seem that all should be assigned as "under".
In your "Preferred Output" you do show JMXX being assigned as "Over", but that tsr does not include the JMOV skill
If your written explanation is correct, and your Preferred Output screenshot incorrect based on the posted data, then, in PQ you can
Group by tsr
Create a custom aggregation returning the "overall" based on containing one of the skills listed in your DAX formula.
If that is not the case, please clarify how you are assigning "Over" to JMXX.
Edit: M Code simplified
M Code
let
//Source = the data structure you show
Source = Excel.CurrentWorkbook(){[Name="Table13"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ts_tsr", type text}, {"ts_skill", type text}}),
//Group rows by tsr, then check if it has one of the defined skills
//If so, return the appropriate ranking.
#"Grouped Rows" = Table.Group(#"Changed Type", {"ts_tsr"}, {
{"ALL", each _, type table [ts_tsr=nullable text, ts_skill=nullable text]},
{"overall", each if List.Contains([ts_skill],"JMSR") then "Senior"
else if List.Contains([ts_skill],"JMOV") then "Over"
else if List.Contains([ts_skill],"JMUN") then "Under"
else if List.Contains([ts_skill],"JMRH") >=0 then "RHT"
else "MNT"}
}),
//Then re-expand the table
#"Expanded ALL" = Table.ExpandTableColumn(#"Grouped Rows", "ALL", {"ts_skill"}, {"ts_skill"})
in
#"Expanded ALL"
Data
Output
Related
Using Power Query in Excel. I am trying to implement a custom column that would iteratively calculate the row based on the previous row's value of the same column.
I have a 3 column table and the 4th column will be the calculation column that I am failing to implement.
The calculation is very easy to apply in Excel which goes as follows:
Formula in cell D3 --> = =IF(A3=1,C3+6.4,IF(C3+D2>=12.8,12.8,IF(C3+D2<=1.28,1.28,C3+D2)))
The same formula is applied to the whole column by dragging.
The idea behind it:
For each category, I have an index column starting from 1,
If Index = 1, then Calculation is Value + 6.4,
else if Value + Value(previous row Custom cumulative) >= 12.8 then 12.8
else if Value + Value(previous row Custom cumulative) <= 1.28 then 1.28
else Value + Value(previous row Custom cumulative)
So, the calculation is a cumulative sum with an upper and lower cap built into it.
How can I implement this in Power Query and M-Language?
I really appreciate your help!
I have tried to use List.Generate and List.Accumulate features, however, I was stuck with creating records that has values from multiple columns in it.
Try this
(edited to make more efficient with single pass process)
let Source = Excel.CurrentWorkbook(){[Name="Table15"]}[Content],
process = (zzz as list) => let x= List.Accumulate( zzz,{0},( state, current ) =>
if List.Last(state) =0 then List.Combine ({state,{6.4+current}}) else
if List.Last(state)+current >=12.8 then List.Combine ({state,{12.8}}) else
if List.Last(state)+current <=1.28 then List.Combine ({state,{1.28}}) else
List.Combine ({state,{List.Last(state)+current}})
) in x,
#"Grouped Rows" = Table.Group(Source, {"Category"}, {{"data", each
let a=process(_[Values])
in Table.AddColumn(_, "Custom Cumulative", each a{[Index]}), type table }}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"Index", "Values", "Custom Cumulative"}, {"Index", "Values", "Custom Cumulative"})
in #"Expanded data"
This is the corrected code with the guidance from Alexis. My PDF returns two tables (and 1 page Table) per output page. Table001 is a throw away. I only need even numbered Tables so I use the List.Select to remove the Page Table and List.Alternate to skip odd numbered tables.
let
Source = Pdf.Tables(File.Contents("State_Fico.pdf"), [Implementation="1.3"]),
TableNames = List.Alternate(List.Select(Table.Column(Source, "Id"),each Text.Contains(_,"Table")),1,1),
TableList = List.Transform(TableNames, each Source{[Id=_]}[Data]),
CombineTables = Table.Combine(TableList)
in
CombineTables
This allows me to generate 1 table no matter how many pages the pdf is.
While this doesn't really answer the question in your title, I think the best way to do what you're ultimately after is to not use Expression.Evaluate at all and, instead, use list transformation(s).
For example, if you want to append Table002 and Table004, you can use Table.Combine on a list of tables, {Table002, Table004}.
Here's what the code might look like:
let
Source = Pdf.Tables(File.Contents("State_Fico.pdf"), [Implementation="1.3"]),
#"Transposed Table" = Table.Transpose(Source),
#"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Page001", type any}, {"Table001", type any}, {"Page002", type any}, {"Table002", type any}, {"Table003", type any}, {"Table004", type any}}),
ColumnNames = Table.ColumnNames(#"Changed Type"),
TableNames = List.Alternate(List.Select(ColumnNames, each Text.Contains(_ ,"Table")),1,1),
//--New Steps Below--//
TableList = List.Transform(TableNames, each Source{[ID=_]}[Data]),
CombineTables = Table.Combine(TableList)
in
CombineTables
I would like to choose a certain columns in power query, but not using their names. Ex. I can do this in R, by command: select. I'm wondering how i can do it in power query. I found some information here, but not all that I need.
Any idea, if I want to refer to more than one column?
It doesn't work if I write the code as below:
#"Filtered Part Desc" = Table.SelectRows (
#"Removed Columns3",
each List.Contains(
{ "ENG", "TRANS" },
Record.Field(_, Table.ColumnNames(#"Removed Columns3") { 5, 6, 7 })
)
)
Let's say I have this table and want to do a couple of things to it.
First, I want to change the column type of the second and last columns. We can use Table.ColumnNames to do this using simple indexing (which starts at zero) as follows:
Table.TransformColumnTypes(
Source,
{
{Table.ColumnNames(Source){1}, Int64.Type},
{Table.ColumnNames(Source){3}, Int64.Type}
}
)
That works but requires specifying each index separately. If we want to unpivot these columns like this
Table.Unpivot(#"Changed Type", {"Col2", "Col4"}, "Attribute", "Value")
but using the index values instead we can use the same method as above
Table.Unpivot(
#"Changed Type",
{
Table.ColumnNames(Source){1},
Table.ColumnNames(Source){3}
}, "Attribute", "Value"
)
But is there a way to do this where we can use a single list of positional index values and use Table.ColumnNames only once? I found a relatively simple though unintuitive method on this blog. For this case, it works as follows:
Table.Unpivot(
#"Changed Type",
List.Transform({1,3}, each Table.ColumnNames(Source){_}),
"Attribute", "Value"
)
This method starts with the list of positional index values and then transforms them into column names by looking up the names of the columns corresponding to those positions.
Here's the full M code for the query I was playing with:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUTIE4nIgtlSK1YlWSgKyjIC4AogtwCLJQJYxEFcCsTlYJAXIMgHiKiA2U4qNBQA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Col1 = _t, Col2 = _t, Col3 = _t, Col4 = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{Table.ColumnNames(Source){1}, Int64.Type},{Table.ColumnNames(Source){3}, Int64.Type}}),
#"Unpivoted Columns" = Table.Unpivot(#"Changed Type", List.Transform({1,3}, each Table.ColumnNames(Source){_}), "Attribute", "Value")
in
#"Unpivoted Columns"
I'm having trouble finding things in the database of Dynamics 365 with Power BI, when you connect to the OrganizationData.svc, there is a LOT of table, and sub table.
For exemple, I'm working with the table "OpportunitySet", someone add a custom combobox, I'm able to get the value "176000004", but I can't find a way to get the text value from this.
I search in the "PickListMappingSet", but there is too much stuff.
Also, all my Opportunities have a Team and people in those teams, I think they are link with "Connections", but I have no idea how to obtain them in Power BI, I need they find some opportunities who have missing people in their team.
Is there any way to search everywhere in the DataBase, or to find where each value is store in it.
Thanks
stringmaps is not listed as an entity in the metadata for the api. However https://<your_dynamics_url>/api/data/v9.1/stringmaps works in Chrome for me although you receive a paginated response, which means you can build a reference query to use when looking up optionsets or even state and status codes:
let
DataList = List.Generate(
() => [
SourceURI="https://<your_dynamics_url>/api/data/v9.1/stringmaps"
,Pagecount=0
,Stringmaps = {}
,Source = []
,ErrorTest = try Source = []
]
,each if [ErrorTest][HasError] then false else true
,each [
ErrorTest = try Source = Json.Document(Web.Contents([SourceURI]))
,Source = Json.Document(Web.Contents([SourceURI]))
,SourceURI = Record.Field(Source,"#odata.nextLink")
,Stringmaps = Source[value]
,Pagecount = [Pagecount] + 1
]
),
#"Converted to Table" = Table.FromList(DataList, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"Stringmaps"}, {"Column1.Stringmaps"}),
#"Removed Errors" = Table.RemoveRowsWithErrors(#"Expanded Column1", {"Column1.Stringmaps"}),
#"Expanded Column1.Stringmaps" = Table.ExpandListColumn(#"Removed Errors", "Column1.Stringmaps"),
#"Removed Blank Rows" = Table.SelectRows(#"Expanded Column1.Stringmaps", each not List.IsEmpty(List.RemoveMatchingItems(Record.FieldValues(_), {"", null}))),
#"Expanded Column1.Stringmaps1" = Table.ExpandRecordColumn(#"Removed Blank Rows", "Column1.Stringmaps", {"value", "attributename", "objecttypecode", "attributevalue"}, {"value", "attributename", "objecttypecode", "attributevalue"}),
#"Sorted Rows" = Table.Sort(#"Expanded Column1.Stringmaps1",{{"objecttypecode", Order.Ascending},{"attributename", Order.Ascending}}),
#"Grouped Rows" = Table.Group(#"Sorted Rows", {"attributename", "objecttypecode"}, {{"Count", each _, type table [value=text, attributename=text, objecttypecode=text, attributevalue=number]}}),
#"Grouped Rows1" = Table.Group(#"Grouped Rows", {"objecttypecode"}, {{"Count", each _, type table [attributename=text, objecttypecode=text, Count=table]}})
in
#"Grouped Rows1"
Basically for onpremise, you can pull from StringMap entity & see all the optionset (=picklist=combobox) values in Database. For online it's a problem.
This is a known issue with PowerBI query as well as oData endpoint, you cannot get the custom or user created picklist values.
You can ask some CRM developers to get all the picklist values/text from CRM customizations & store as a Source in PowerBI datasets to get the desired results after merge with main dataset.
Edit: Xrmtoolbox PowerBI optionset assistant will be helpful.
Is there an an equivalent to EARLIER in M/Power Query?
Say, I have a table with lots of different dates in column DATE and a smaller number of letters in column LETTER. I now want the maximum date for each letter.
In DAX, I would use something like CALCULATE(MAX([Date]),FILTER(ALL(Table),[Letter]=EARLIER([Letter])).
How would I achieve the same in M?
Thanks
2 Solutions in the code below. Notice that each uses "PreviousStep" as basis, so these are separate solutions.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
PreviousStep = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Letter", type text}}),
// 1. Add a column to the original table with the MaxDate for each letter
// "earlier" is just the name of a function parameter; it could as well have been "x" or "MarcelBeug"
AddedMaxDate = Table.AddColumn(PreviousStep, "MaxDate", (earlier) => List.Max(Table.SelectRows(PreviousStep, each [Letter] = earlier[Letter])[Date])),
// 2. Group by letter and get the MaxDate for each letter
GroupedOnLetter = Table.Group(PreviousStep, {"Letter"}, {{"MaxDate", each List.Max([Date]), type date}})
in
GroupedOnLetter
In short, there is no exact match for this function. Still, you can use other ways that can produce same results.
To reproduce example offered by Microsoft in help for EARLIER function, you can use following code (table1 equals table given in the example before ranking):
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("TVNNaxtBDP0rxuSoivn+uMYlLSUFE4f2YHIYd4d48Xq3rO1C/n01o1mc4+i9kZ6epP1+LcMa1o/94cvuOM3XCz0epHUgnccQ1m+wXytXGae8ekl/TpWhlACvBHrBDL8wdtc0dpWiLTgV0EVm1CrT9Trky4ooq016z5VnI2ij0OjKs402nVePM1XLrMgEcEaj8ZVU9czpxAmcAik1SlcxGSm2SX/5m4eoDVpToSJyc0z9WLEAwXgUrcX6a8hpzDNb4CAEhU5VuIjfzGk8XZoeGSVYpVBwd+X31zynfhjyjRM4A9FZ1NyWFhR7ymPX0hsJ0RuUbJ+s6DSzt96QtR4d96MK9m2Y/uVmfABtNVrWbSj2newc8iEtwjUoS401O2Rh5NQtyq0HZyNGFq4ZHs6Lz1aCjAopXmFV4I9uTtd+GlfbZfyR3IkafTOvJPlBneUPbj1GMCouMFkA6+f+/VhLcKjofp5aNmlBkKQ23JLs53QbrzSoVdkp3iYDWlgIzqBi6VJ9Jj7N6cxMA1ZSE16ga/XLTm3TOPZsPv8uora5SwNLMIIkK1Q8EF02bHs78xZJBS5alK1bCr1Mqbtro7+WfHPRoeZNk2Yh3XVpcNqBjgE9myuLrl3qaHg8GUUr5RYbVKlzP0kdLHhBJ9kOrsjfLQaWndCEWcZK8dfF7wcZIrkRUXNe7Ss6tzN8vR2WxTIQtMLQJl9Y023ux/d7o1JTHVOH0MyQ7hPv3isdh7F01gYFH5Aqvf7KF5akyLEYBYrmVpH0+5jz0C4nADEq+vYf", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ProductSubcategoryKey = _t, EnglishProductSubcategoryName = _t, TotalSubcategorySales = _t]),
table1 = Table.TransformColumnTypes(Source,{{"ProductSubcategoryKey", Int64.Type}, {"EnglishProductSubcategoryName", type text}, {"TotalSubcategorySales", Currency.Type}}, "en-US"),
AddCount = Table.AddColumn(
table1,
"SubcategoryRanking", //(a) is a parameter for function, which equals current record, and function should return value for new cell of "SubcategoryRanking"
(a)=> Table.RowCount(
Table.SelectRows(
table1, //(b) equals whole table1. This function returns table filtered by given criteria
(b) => b[TotalSubcategorySales] < a[TotalSubcategorySales])
) + 1,
Int64.Type)
in
AddCount
I think you can use the GroupBy function to group the data by Letter and find the Max of the date column. So your code should look like.
= Table.Group(#"Previous step", {"Letter"}, {{"Max Date", each List.Max([Date]), type date}})