Powerquery - Algorithm required for this task - algorithm

Long time lurker, first time poster. Apologies in advance if my question appears a bit noobish.
I am new to powerquery/powerbi and trying to develop a report to solve a small logistics issue we are facing.
I have a list of cables that are given to our technicians to use of different pre-terminated lengths which they will use for their application. Ideally the closest length cable to the length of the route should be utilized, however that is not always the case. I want to match the cables issued to the routes (Column "Actual Cables Utilised"). Example below:
Cables issued:
Breakdown:
Highlighted in green are the accepted substitutions. Red is the wrong substitution (There isn't a cable issued that is longer than the route).
I cannot find a way to run a loop in powerquery that would populate the breakdown table based on the issued table. Please note that unfortunately, my company does not allow python or r-script on our computers, so I am limited to basic power query.
Any assistance would be greatly appreciated!

This might work
Group the Breakdown table by technician
For each technician, generate a list of Issued cables with an entry for each cable (i.e. if there are 2 - 40m cables, 40 will appear twice in the list.
The Issued cables will be assigned sequentially in the order presented by the Breakdown sheet.
Use a recursive function that, as it generates the Utilized cable list, removes that entry from the Issued cables.
Main Code
let
//Read in the Issued and Breakdown tables
//Change the Source lines to reflect your own data sources for these tables
Source1 = Excel.CurrentWorkbook(){[Name="Issued"]}[Content],
Issued = Table.TransformColumnTypes(Source1, List.Transform(Table.ColumnNames(Source1), each {_, Int64.Type})),
Source = Excel.CurrentWorkbook(){[Name="Breakdown"]}[Content],
Breakdown = Table.TransformColumnTypes(Source,{
{"Technician", type text}, {"Route Length (m)", Int64.Type}, {"Ideal Cable Length (m)", Int64.Type}}),
//Group by Technician
// Generate a list of cables where there is one entry for every issued cable
// Then call the custom (recursive) function to generate an appropriate list for each tech
#"Grouped Rows" = Table.Group(Breakdown, {"Technician"}, {
{"Cables", (t)=>
let
cables = List.Combine(
List.Generate(
()=> [c=List.Repeat({Issued[Cable Lengths]{0}}, Table.Column(Issued, t[Technician]{0}){0}), idx=0],
each [idx] < Table.RowCount(Issued),
each [c=List.Repeat({Issued[Cable Lengths]{[idx]+1}}, Table.Column(Issued, t[Technician]{0}){[idx]+1}), idx = [idx]+1],
each [c])),
x = fnBreakdown(t, cables)
in
x, type list}}),
#"Expanded Cables" = Table.ExpandListColumn(#"Grouped Rows", "Cables"),
//add the utilized cable column back to the Breakdown table
#"Breakdown Table" = Table.FromColumns(
Table.ToColumns(Breakdown) & {#"Expanded Cables"[Cables]},
type table[Technician=text, #"Route Length (m)"=Int64.Type,
#"Ideal Cable Length (m)"=Int64.Type,Actual Cable Utilized=Int64.Type]
)
in
#"Breakdown Table"
Custom Function rename fnBreakdown
//Rename this query "fnBreakdown"
//tech = the subtable all with the same technician
//cables = a list of cables with a single entry for each size
// generated by List.Repeat in the original call from the main code
//cableList = a list of the utilized cable
//Index is the index into the Tech table
(tech as table, cables as list, optional cableList as list, optional Index as number)=>
let
//set intial values for optional arguments
idx = if Index = null then 0 else Index + 1,
cableL = if cableList = null then {} else cableList,
//determine cable matching for "idx"th entry
//if there is no match then write a null
#"New Cab" = List.Select(cables, each _ >= tech[#"Route Length (m)"]{idx}),
#"New Cable" = if #"New Cab" = {} then null else #"New Cab"{0},
//Remove the used cable from the list of "cables"
#"New Cables" = if List.PositionOf(cables,#"New Cable",Occurrence.First) = -1
then cables
else List.RemoveRange(cables,List.PositionOf(cables,#"New Cable",Occurrence.First),1),
//new cable list for this tech
#"New Cable List" = cableL & {#"New Cable"},
//if we haven't checked all the routes then recursively call the function with the updated arguments
result = if idx < Table.RowCount(tech)-1
then fnBreakdown(tech,#"New Cables", #"New Cable List", idx)
else #"New Cable List"
in
result
Results from your data above

Related

Power query, iterate over the column records to apply a custom cumulative calculation

Using Power Query in Excel. I am trying to implement a custom column that would iteratively calculate the row based on the previous row's value of the same column.
I have a 3 column table and the 4th column will be the calculation column that I am failing to implement.
The calculation is very easy to apply in Excel which goes as follows:
Formula in cell D3 --> = =IF(A3=1,C3+6.4,IF(C3+D2>=12.8,12.8,IF(C3+D2<=1.28,1.28,C3+D2)))
The same formula is applied to the whole column by dragging.
The idea behind it:
For each category, I have an index column starting from 1,
If Index = 1, then Calculation is Value + 6.4,
else if Value + Value(previous row Custom cumulative) >= 12.8 then 12.8
else if Value + Value(previous row Custom cumulative) <= 1.28 then 1.28
else Value + Value(previous row Custom cumulative)
So, the calculation is a cumulative sum with an upper and lower cap built into it.
How can I implement this in Power Query and M-Language?
I really appreciate your help!
I have tried to use List.Generate and List.Accumulate features, however, I was stuck with creating records that has values from multiple columns in it.
Try this
(edited to make more efficient with single pass process)
let Source = Excel.CurrentWorkbook(){[Name="Table15"]}[Content],
process = (zzz as list) => let x= List.Accumulate( zzz,{0},( state, current ) =>
if List.Last(state) =0 then List.Combine ({state,{6.4+current}}) else
if List.Last(state)+current >=12.8 then List.Combine ({state,{12.8}}) else
if List.Last(state)+current <=1.28 then List.Combine ({state,{1.28}}) else
List.Combine ({state,{List.Last(state)+current}})
) in x,
#"Grouped Rows" = Table.Group(Source, {"Category"}, {{"data", each
let a=process(_[Values])
in Table.AddColumn(_, "Custom Cumulative", each a{[Index]}), type table }}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"Index", "Values", "Custom Cumulative"}, {"Index", "Values", "Custom Cumulative"})
in #"Expanded data"

Convert Switch True() Dax calculated column to M query custom colum

I am having issues with my calculated column and the multiple tables I am joining. It is not filtering my visuals correctly. After researching it was recommended to use a custom column in the query instead but I do not know where to start to convert the following DAX to M query.
overall =
VAR skills =
CALCULATETABLE (
VALUES ( tsr_skill[ts_skill] ),
ALLEXCEPT ( tsr_skill, tsr_skill[ts_tsr] )
)
RETURN
SWITCH (
TRUE (),
"JMSR" IN skills, "Senior",
"JMOV" IN skills, "Over",
"JMUN" IN skills, "Under",
"JMRH" IN skills, "RHT",
"MNT"
)
Data structure in Query:
How I would like the data to show in the Query instead of showing as a calculated column.
Preferred Output:
Based on your explanation, and the levels assigned in your DAX formula, it would seem that all should be assigned as "under".
In your "Preferred Output" you do show JMXX being assigned as "Over", but that tsr does not include the JMOV skill
If your written explanation is correct, and your Preferred Output screenshot incorrect based on the posted data, then, in PQ you can
Group by tsr
Create a custom aggregation returning the "overall" based on containing one of the skills listed in your DAX formula.
If that is not the case, please clarify how you are assigning "Over" to JMXX.
Edit: M Code simplified
M Code
let
//Source = the data structure you show
Source = Excel.CurrentWorkbook(){[Name="Table13"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ts_tsr", type text}, {"ts_skill", type text}}),
//Group rows by tsr, then check if it has one of the defined skills
//If so, return the appropriate ranking.
#"Grouped Rows" = Table.Group(#"Changed Type", {"ts_tsr"}, {
{"ALL", each _, type table [ts_tsr=nullable text, ts_skill=nullable text]},
{"overall", each if List.Contains([ts_skill],"JMSR") then "Senior"
else if List.Contains([ts_skill],"JMOV") then "Over"
else if List.Contains([ts_skill],"JMUN") then "Under"
else if List.Contains([ts_skill],"JMRH") >=0 then "RHT"
else "MNT"}
}),
//Then re-expand the table
#"Expanded ALL" = Table.ExpandTableColumn(#"Grouped Rows", "ALL", {"ts_skill"}, {"ts_skill"})
in
#"Expanded ALL"
Data
Output

IBeacon. Custom filter regions

Good day. Sorry my English is bad. Whether prompt there is an opportunity to define region of bicons not on UUID and on it is one part. For example, we have 2 regions:
1 - UUID = 57547265-7261-772e-636f-6c0054000000
2- there is only the full part (most likely this is the fourth last group) -
6c001137e000
Can I just specify in my method a filter only for the last group of UUID?
For example:
List<string> listUuids =new List<string>()
{ "57547265-7261-772e-636f-6c0054000000" , "6c001137e000" };
for (int i = 0; i <listUuids.Count; i ++)
{
var uuid = Identifier.Parse (listUuids [i]);
var region = new Region ("R" + i, uuid, null, null);
beaconManager.StartRangingBeaconsInRegion (region);
}
I would really appreciate any answers!
Best Regards!
You can do this with a custom beacon parser. Like this:
BeaconManager.GetBeaconParsers().clear();
BeaconMannager.GetBeaconParsers().add(new Beacon parser().setBeaconLayout("m:2-3=0215,i:14-19,i:4-13,i:20-21,i:22-23,p:24-24");
var region = new Region("partial UUID region", Identifier.parse("0x6c001137e000"), null, null)
The above custom parser defines four identifiers for a iBeacon packet. The first identifier is the last 6 bytes of the UUID. The second identifier is the first 10 bytes of the UUID. The third and fourth identifiers are the major and minor, respectively.
Using the region defined above will match any beacon matching the specified last 6 bytes of the UUID.

DAX EARLIER() function in Power Query

Is there an an equivalent to EARLIER in M/Power Query?
Say, I have a table with lots of different dates in column DATE and a smaller number of letters in column LETTER. I now want the maximum date for each letter.
In DAX, I would use something like CALCULATE(MAX([Date]),FILTER(ALL(Table),[Letter]=EARLIER([Letter])).
How would I achieve the same in M?
Thanks
2 Solutions in the code below. Notice that each uses "PreviousStep" as basis, so these are separate solutions.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
PreviousStep = Table.TransformColumnTypes(Source,{{"Date", type date}, {"Letter", type text}}),
// 1. Add a column to the original table with the MaxDate for each letter
// "earlier" is just the name of a function parameter; it could as well have been "x" or "MarcelBeug"
AddedMaxDate = Table.AddColumn(PreviousStep, "MaxDate", (earlier) => List.Max(Table.SelectRows(PreviousStep, each [Letter] = earlier[Letter])[Date])),
// 2. Group by letter and get the MaxDate for each letter
GroupedOnLetter = Table.Group(PreviousStep, {"Letter"}, {{"MaxDate", each List.Max([Date]), type date}})
in
GroupedOnLetter
In short, there is no exact match for this function. Still, you can use other ways that can produce same results.
To reproduce example offered by Microsoft in help for EARLIER function, you can use following code (table1 equals table given in the example before ranking):
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("TVNNaxtBDP0rxuSoivn+uMYlLSUFE4f2YHIYd4d48Xq3rO1C/n01o1mc4+i9kZ6epP1+LcMa1o/94cvuOM3XCz0epHUgnccQ1m+wXytXGae8ekl/TpWhlACvBHrBDL8wdtc0dpWiLTgV0EVm1CrT9Trky4ooq016z5VnI2ij0OjKs402nVePM1XLrMgEcEaj8ZVU9czpxAmcAik1SlcxGSm2SX/5m4eoDVpToSJyc0z9WLEAwXgUrcX6a8hpzDNb4CAEhU5VuIjfzGk8XZoeGSVYpVBwd+X31zynfhjyjRM4A9FZ1NyWFhR7ymPX0hsJ0RuUbJ+s6DSzt96QtR4d96MK9m2Y/uVmfABtNVrWbSj2newc8iEtwjUoS401O2Rh5NQtyq0HZyNGFq4ZHs6Lz1aCjAopXmFV4I9uTtd+GlfbZfyR3IkafTOvJPlBneUPbj1GMCouMFkA6+f+/VhLcKjofp5aNmlBkKQ23JLs53QbrzSoVdkp3iYDWlgIzqBi6VJ9Jj7N6cxMA1ZSE16ga/XLTm3TOPZsPv8uora5SwNLMIIkK1Q8EF02bHs78xZJBS5alK1bCr1Mqbtro7+WfHPRoeZNk2Yh3XVpcNqBjgE9myuLrl3qaHg8GUUr5RYbVKlzP0kdLHhBJ9kOrsjfLQaWndCEWcZK8dfF7wcZIrkRUXNe7Ss6tzN8vR2WxTIQtMLQJl9Y023ux/d7o1JTHVOH0MyQ7hPv3isdh7F01gYFH5Aqvf7KF5akyLEYBYrmVpH0+5jz0C4nADEq+vYf", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ProductSubcategoryKey = _t, EnglishProductSubcategoryName = _t, TotalSubcategorySales = _t]),
table1 = Table.TransformColumnTypes(Source,{{"ProductSubcategoryKey", Int64.Type}, {"EnglishProductSubcategoryName", type text}, {"TotalSubcategorySales", Currency.Type}}, "en-US"),
AddCount = Table.AddColumn(
table1,
"SubcategoryRanking", //(a) is a parameter for function, which equals current record, and function should return value for new cell of "SubcategoryRanking"
(a)=> Table.RowCount(
Table.SelectRows(
table1, //(b) equals whole table1. This function returns table filtered by given criteria
(b) => b[TotalSubcategorySales] < a[TotalSubcategorySales])
) + 1,
Int64.Type)
in
AddCount
I think you can use the GroupBy function to group the data by Letter and find the Max of the date column. So your code should look like.
= Table.Group(#"Previous step", {"Letter"}, {{"Max Date", each List.Max([Date]), type date}})

How to order integers according to size and track their positions by variable name

I have a program with multiple int variables where individual counts are added to the specific variable each time a set fail condition is encountered. I want the user to be able to track how many failures of each category they have encountered by a button click. I want to display the range on a datagridview in order from highest value integer down to lowest. I also need to display in the adjacent column the name of the test step that relates to the value. My plan was to use Array.sort to order the integers but i then lose track of their names so cant assign the adjacent string column. I tried using a hashtable but if i use the string as a key it sorts alphabetically not numerically and if i use the integer as a key i get duplicate entries which dont get added to the hash table. here is some of the examples i tried but they have the aforementioned problems. essentially i want to end with two arrays where the order matches the naming and value convention. FYI the variables were declared before this section of code, variables ending in x are the string name for the (non x) value of the integer.
Hashtable sorter = new Hashtable();
sorter[download] = downloadx;
sorter[power] = powerx;
sorter[phase] = phasex;
sorter[eeprom] = eepromx;
sorter[upulse] = upulsex;
sorter[vpulse] = vpulsex;
sorter[wpulse] = wpulsex;
sorter[volts] = voltsx;
sorter[current] = currentx;
sorter[ad] = adx;
sorter[comms] = commsx;
sorter[ntc] = ntcx;
sorter[prt] = prtx;
string list = "";
string[] names = new string[13];
foreach (DictionaryEntry child in sorter)
{
list += child.Value.ToString() + "z";
}
int[] ordered = new int[] { download, power, phase, eeprom, upulse, vpulse, wpulse, volts, current, ad, comms, ntc, prt };
Array.Sort(ordered);
Array.Reverse(ordered);
for (int i = 0; i < sorter.Count; i++)
{
int pos = list.IndexOf("z");
names[i] = list.Substring(0, pos);
list = list.Substring(pos + 1);
}
First question here so hope its not too longwinded.
Thanks
Use a Dictionary. And you can order it by the value : myDico.OrderBy(x => x.Value).Reverse(), the sort will be numerical descending. You just have to enumerate the result.
I hope I understand your need. Otherwise ignore me.
You want to be using a
Dictionary <string, int>
to store your numbers.I'm not clear on how you're displaying results at the end - do you have a grid or a list control?
You ask about usings. Which ones do you already have?
EDIT for .NET 2.0
There might be a more elegant solution, but you could implement the logic by putting your rows in a DataTable. Then you can make a DataView of that table and sort by whichever column you like, ascending or descending.
See http://msdn.microsoft.com/en-us/library/system.data.dataview(v=VS.80).aspx for example.
EDIT for .NET 3.5 and higher
As far as sorting a Dictionary by its values:
var sortedEntries = myDictionary.OrderBy(pair => pair.Value);
If you need the results to be a Dictionary, you can call .ToDictionary() on that. For reverse order, use .OrderByDescending(pair => pair.Value).

Resources