How to convert hours, minutes, second in PowerBI to minutes? - time

Hi everyone,
I have a table named Table1 with a column showing Time Taken for student to finish the exam (as shown in the screenshot above). May I know how should I convert the column to minutes? Basically need to convert hrs (if exist) to minutes and convert sec to minute as well.
To simplify the output, if the sec more than or equal to 30sec, then counted as 1 min, otherwise just ignore the sec. I'm still new to PowerBI, any help will be greatly appreciated!
Output from suggested answer

You can perform these below steps in your table to get your desired output-
Advance Query Editor code
let
//... your previous steps,
// Change the PREVIOUS STEP NAME accordingly in the NEXT line
#"Replaced Value1" = Table.ReplaceValue(#"PREVIOUS STEP NAME"," hrs","hrs",Replacer.ReplaceText,{"Time Taken"}),
#"Replaced Value" = Table.ReplaceValue(#"Replaced Value1"," min","min",Replacer.ReplaceText,{"Time Taken"}),
#"Replaced Value2" = Table.ReplaceValue(#"Replaced Value"," sec","sec",Replacer.ReplaceText,{"Time Taken"}),
#"Duplicated Column" = Table.DuplicateColumn(#"Replaced Value2", "Time Taken", "Time Taken - Copy"),
#"Split Column by Delimiter" = Table.SplitColumn(#"Duplicated Column", "Time Taken - Copy", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), {"Time Taken - Copy.1", "Time Taken - Copy.2", "Time Taken - Copy.3"}),
#"Changed Type1" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Time Taken - Copy.1", type text}, {"Time Taken - Copy.2", type text}, {"Time Taken - Copy.3", type text}}),
#"Replaced Value3" = Table.ReplaceValue(#"Changed Type1",null,"",Replacer.ReplaceValue,{"Time Taken - Copy.1", "Time Taken - Copy.2", "Time Taken - Copy.3"}),
#"Added Custom" = Table.AddColumn(#"Replaced Value3", "hrs", each if Text.PositionOf([#"Time Taken - Copy.1"],"hrs") > 0 then Text.Replace([#"Time Taken - Copy.1"],"hrs","")
else if Text.PositionOf([#"Time Taken - Copy.2"],"hrs") > 0 then Text.Replace([#"Time Taken - Copy.2"],"hrs","")
else if Text.PositionOf([#"Time Taken - Copy.3"],"hrs") > 0 then Text.Replace([#"Time Taken - Copy.3"],"hrs","")
else "0"),
#"Added Custom1" = Table.AddColumn(#"Added Custom", "min", each if Text.PositionOf([#"Time Taken - Copy.1"],"min") > 0 then Text.Replace([#"Time Taken - Copy.1"],"min","")
else if Text.PositionOf([#"Time Taken - Copy.2"],"min") > 0 then Text.Replace([#"Time Taken - Copy.2"],"min","")
else if Text.PositionOf([#"Time Taken - Copy.3"],"min") > 0 then Text.Replace([#"Time Taken - Copy.3"],"min","")
else "0"),
#"Added Custom2" = Table.AddColumn(#"Added Custom1", "sec", each if Text.PositionOf([#"Time Taken - Copy.1"],"sec") > 0 then Text.Replace([#"Time Taken - Copy.1"],"sec","")
else if Text.PositionOf([#"Time Taken - Copy.2"],"sec") > 0 then Text.Replace([#"Time Taken - Copy.2"],"sec","")
else if Text.PositionOf([#"Time Taken - Copy.3"],"sec") > 0 then Text.Replace([#"Time Taken - Copy.3"],"sec","")
else "0"),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom2",{"Time Taken - Copy.1", "Time Taken - Copy.2", "Time Taken - Copy.3"}),
#"Changed Type2" = Table.TransformColumnTypes(#"Removed Columns",{{"hrs", Int64.Type}, {"min", Int64.Type}, {"sec", Int64.Type}}),
#"Added Custom3" = Table.AddColumn(#"Changed Type2", "final_minutes", each ([hrs]*60) + [min] + (if [sec]>=30 then 1 else 0))
in
#"Added Custom3"
Sample output-

Related

Convert numbers to Excel Column Letters in Power Query

I would like to transform Numbers to Excel Column Letters.
There are several threads on the topic but it seems that it has not been answered yet for M language.
I managed to address it with the following code but I am sure someone will have a much more efficient code to propose:
= Table.AddColumn(Source, "Column Letter", each Text.Combine({ if (Number.IntegerDivide(
Number.IntegerDivide(
Number.IntegerDivide([Column1]-1,1),26)-1,26))<1 then "" else Character.FromNumber( 64+Number.IntegerDivide(Number.IntegerDivide(Number.IntegerDivide([Column1]-1,1),26)-1,26)),
(if Number.IntegerDivide([Column1]-1,26)-
Number.Abs(
Number.IntegerDivide(
Number.IntegerDivide([Column1]-1,26)
-1,26)*26)<1 then "" else
Character.FromNumber(64+
Number.IntegerDivide([Column1]-1,26)-Number.Abs(
Number.IntegerDivide(
Number.IntegerDivide([Column1]-1,26)-1,26)*26))),
(if [Column1]-Number.Abs(
Number.IntegerDivide([Column1]-1,26)*26)<1 then "" else Character.FromNumber(64+[Column1]-Number.Abs(
Number.IntegerDivide([Column1]-1,26)*26)))}))
Thank you!
I think this custom function will work also:
Edit*Formula for b corrected as per #VassilisAgapitos note below
(n as number)=>
let
//make sure it is an integer
a = Number.IntegerDivide(n,1),
b = Number.IntegerDivide(Number.IntegerDivide(a-1,26)-1,26),
L1 = if b = 0 then null else Character.FromNumber(b+64),
c = Number.IntegerDivide(a-b*26*26-1,26),
L2 = if c = 0 then null else Character.FromNumber(c+64),
d = a-b*26*26-c*26,
L3 = Character.FromNumber(d+64)
in
Text.Combine({L1,L2,L3})
You can use it as a Table.AddColumn argument:
#"Added Custom" = Table.AddColumn(#"Changed Type", "Column Letter", each fnNumberToExcelColumn([Column Number]), type text)
An uglier version than Ron's, just for fun
=(if [Column1] <703 then "" else Character.FromNumber(64+Number.IntegerDivide([Column1]-27,26*26)))
&( if [Column1] >26 then Replacer.ReplaceText(Character.FromNumber(64+ Number.Mod( Number.IntegerDivide([Column1]-1,26) ,26) ), "#", "Z") else "" )
&(Character.FromNumber(65+Number.Mod([Column1]-1,26)))

Power Query M - Way to dynamically use Table.ColumnAdd using value literals to substitute for column names

Here is the script I have based on the steps generated:
let
Source = Sql.Database("BITESTDBSVR1", "DW_FINANCE"),
CORPDB_BTT_SCENARIO_DETAILS_WITH_BACKLOG = Source{[Schema="CORPDB",Item="BTT_SCENARIO_DETAILS_WITH_BACKLOG"]}[Data],
#"Filtered Rows" = Table.SelectRows(CORPDB_BTT_SCENARIO_DETAILS_WITH_BACKLOG, each [ScenarioHeaderID] = FromScenario or [ScenarioHeaderID] = ToScenario),
#"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each [ValueTypeName] <> "Backlog"),
#"Removed Columns" = Table.RemoveColumns(#"Filtered Rows1",{"ID", "ContractID", "LineNumber", "Ophours", "Prime", "ScenarioHeaderID", "Division", "ORN", "BidNumber", "CustomerContractNumber", "ProjectID", "CustomerOrderID", "CurrentStatus", "CustomerBuyer", "ProgramName", "ProgramNotes", "Notes", "MajorProgram", "ProductFamily", "ProductSubFamily", "ProductGroup", "SeriesGroup", "Turret", "ProgramSeriesID", "OrderQuantity", "ContractValueSource", "SourceCurrencyID", "BookingFXRate", "ContractValueUSDollars", "PODateIn", "AcceptanceDate", "ContractAwardDate", "ContractAwardYear", "Probability", "FactoredValue", "ProductAccountManager", "ProgramManager", "InterIntraDivision", "InterIntraExternal", "Territory", "IDIQ", "Platform", "CustomerBuyerCountry", "EndUser", "LHXRegion", "LHXName", "ProcurementLocation", "CustomerType1", "CustomerType2", "ExternalCustomer", "SalesType", "BidType", "PricingType", "ExportLicenceNumber", "USExportStatus", "CANExportStatus", "ExportComments", "RevenueRecognitionCriteria", "IncoTerms", "ShipToCountry", "PaymentTerms", "EOIRMaster", "EOIRChildID", "BidDivision", "SubProduct", "ProductLine", "ServiceAccountManager", "BusinessUnit", "OffsetCapture", "EndUserBusinessPartner", "GeographicRegion", "GeographicSubRegion", "ContractDuration", "RevRecTiming", "ForecastRevenueMonths", "InvoiceNumber", "WorkOrdernumber", "SerialNumber", "OrderLine", "PKLineIdentifier", "Year1", "Year2", "Year3", "BI1", "BI2", "BI3", "RI1", "RI2", "RI3", "VMProject", "ServiceContractNo", "MainProject", "LNProject", "ProjectContract", "SalesForceRef", "SalesRegion", "FiscalDate", "ScenarioCreated", "ScenarioUpdated", "ScenarioWeek", "Scenario", "ValueType"}),
#"Pivoted Column" = Table.Pivot(#"Removed Columns", List.Distinct(#"Removed Columns"[ScenarioName]), "ScenarioName", "Value", List.Sum),
#"Replaced Value" = Table.ReplaceValue(#"Pivoted Column",null,0,Replacer.ReplaceValue,{ScenarioFrom, ScenarioTo}),
#"Inserted Subtraction" = Table.AddColumn(#"Replaced Value", "Subtraction", each [#"Jan 15, 2021"] - [#"Mar 5, 2021"], type number),
#"Changed Type" = Table.TransformColumnTypes(#"Inserted Subtraction",{{"Subtraction", type number}}),
#"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Subtraction", "Variance"}}),
#"Merged Queries" = Table.NestedJoin(#"Renamed Columns", {"InterIntra"}, #"CORPDB INTER_INTRA_MAPPING", {"INTER DIVISION"}, "CORPDB INTER_INTRA_MAPPING", JoinKind.LeftOuter),
#"Expanded CORPDB INTER_INTRA_MAPPING" = Table.ExpandTableColumn(#"Merged Queries", "CORPDB INTER_INTRA_MAPPING", {"INTER DIVISION NAME"}, {"INTER DIVISION NAME"}),
#"Reordered Columns" = Table.ReorderColumns(#"Expanded CORPDB INTER_INTRA_MAPPING",{"Forecast", "ProgramGroupID", "InterIntra", "INTER DIVISION NAME", "EndUserCountry", "ValueTypeName", "Month", "Year", "Quarter", "MonthNumber", "FiscalYearMonth", "QuarterStart", "QuarterEnd", ScenarioFrom, ScenarioTo , "Variance"}),
#"Renamed Columns1" = Table.RenameColumns(#"Reordered Columns",{{"INTER DIVISION NAME", "InterIntraDivision"}}),
#"Removed Columns1" = Table.RemoveColumns(#"Renamed Columns1",{"InterIntra"})
in
#"Removed Columns1"
The part of the code I want to update is
#"Inserted Subtraction" = Table.AddColumn(#"Replaced Value", "Subtraction", each [#"Jan 15, 2021"] - [#"Mar 5, 2021"], type number),
Where I have 2 text variables (ScenarioFrom, ScenarioTo) that I want to dynamically substitute in the definition to say
#"Inserted Subtraction" = Table.AddColumn(#"Replaced Value", "Subtraction", each [Scenariofrom] - [Scenarioto] , type number),
I get that I'm trying to force a literal into a table column which is causing the problem, but wondering if there is a function / easy work-around without transforming the data entirely.
When you are defining a custom column, writing each [ColName] is short for each _[ColName] where _ represents the current row, which is a Record type.
With this in mind, we can define the code like this
each Record.Field(_, ScenarioFrom) - Record.Field(_, ScenarioTo)
instead of
each [#"Jan 15, 2021"] - [#"Mar 5, 2021"]
This question is similar but uses the value in a different column rather than parameters/variables:
PowerQuery choose values based on a key column

Pause every 25th URL being scraped using Power Query

I'm scraping a website with different URLs (about 1000) and only able to get 25 query due to website limitation (of course!). I have used Function.InvokeAfter with Number.RandomBetween but no luck.
However, I found that if I wait/pause about an hour or so after the 25th URL, I could query again. Now, I would like to automate that part.
I created a column called Wait with a value on every 25th row on the URL_List table with a number between 60-90 so that every 25th URL, I can have it wait 60 to 90 mins but I don't know how to implement. If I could replace Number.RandomBetween(10,20) with Number.RandomBetween(10,[Wait]). Or also open to other ideas.
Here's what I have done so far:
Function for fGetResults:
let Scrape=(URL) =>
let
Source = Web.Page(Web.Contents(URL)),
Data0 = ()=>Source{0}[Data],
#"Transposed Table" = Table.Transpose(Function.InvokeAfter(Data0, #duration(0,0,0,Number.RandomBetween(10,20))))
in
#"Transposed Table"
in Scrape
Query:
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each fGetResults([URL_List]))
in
#"Added Custom"
Merci beaucoup...
#############
UPDATE
#############
I did some more digging and found this and this. Instead of a Wait column with a number from 60-90mins after 25th URL or so, I just place a "yes".
WaitMinutes function:
let
Wait = (minutes as number, action as function) =>
if (List.Count(List.Generate(() => DateTimeZone.LocalNow() + #duration(0,0,minutes,0), (x) => DateTimeZone.LocalNow() < x, (x) => x)) = 0) then null else action()
in
Wait
Now, I tried it but is now giving me an error Token Comma expected at the then statement on the New Query, any thoughts?
New Query:
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each [Wait] = "yes" then WaitMinutes(Number.RandomBetween(60,90), () => fGetResults([URL_List])) else fGetResults([URL_List]))
in
#"Added Custom"
#############
UPDATE2
#############
I think I found the issue. Simply forgot the "if" on the if-then-else statement. The only thing is, there maybe a more elegant solution for this but this works for now. Let me know if you have a more efficient solution. Thank you.
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each if [Wait] = "yes" then WaitMinutes(Number.RandomBetween(60,90), () => fGetResults([URL_List])) else fGetResults([URL_List]))
in
#"Added Custom"
#############
UPDATE3: SOLUTION
#############
More problems, I think my usage for minutes is not correct but followed exactly what the website said and used seconds and removed the random function on the Query.
WaitSeconds function:
let
Wait = (seconds as number, action as function) =>
if (List.Count(List.Generate(() => DateTimeZone.LocalNow() + #duration(0,0,0,seconds), (x) => DateTimeZone.LocalNow() < x, (x) => x)) = 0) then null else action()
in
Wait
Query (15mins seems to work which is same as 900sec):
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each if [Wait] = "yes" then WaitSeconds(900, () => fGetResults([URL_List])) else fGetResults([URL_List]))
in
#"Added Custom"

DateKey formatted as YYYYMMDD

I have the following calendar table script:
let
StartDate = #date(2016, 1, 1),
EndDate = #date(2018, 12, 31),
CurrentDate = DateTime.Date(DateTime.FixedLocalNow()),
FiscalYearEndMonth = 6,
ListDates = List.Dates(StartDate, Number.From(EndDate - StartDate)+1, #duration(1,0,0,0)),
#"Converted to Table" = Table.FromList(ListDates, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Renamed Columns as Date" = Table.RenameColumns(#"Converted to Table",{{"Column1", "Date"}}),
#"Changed Type to Date" = Table.TransformColumnTypes(#"Renamed Columns as Date",{{"Date", type date}}),
#"Added Calendar MonthNum" = Table.AddColumn(#"Changed Type to Date", "MonthNum", each Date.Month([Date]), Int64.Type),
#"Added Calendar Year" = Table.AddColumn(#"Added Calendar MonthNum", "Year", each Date.Year([Date]), Int64.Type),
#"Added MonthYearNum" = Table.AddColumn(#"Added Calendar Year", "MonthYearNum", each [Year]*100 + [MonthNum] /*e.g. Sep-2016 would become 201609*/, Int64.Type),
#"Added Day Num" = Table.AddColumn(#"Added MonthYearNum", "DayNum", each Date.Day([Date])),
#"Changed Type1" = Table.TransformColumnTypes(#"Added Datekey",{{"DayNum", Int64.Type}}),
#"Added Datekey" = Table.AddColumn(#"Added Day Num", "DateKey", each [Year]*100 + [MonthNum] + [DayNum]),
#"Changed Type" = Table.TransformColumnTypes(#"Changed Type1",{{"DayNum", Int64.Type}, {"DateKey", Int64.Type}})
in
#"Changed Type"
It results in the following:
So the last column is causing me the headache - I think the code for that column is:
each [Year]*100 + [MonthNum] + [DayNum]
It seems like the first two elements of this expression get concatenated as expected but then DayNum is added e.g. for 1st of Jan we get 2016 concatenated with 1 to give 201601 but then the DayNum gets added mathematically to it, to give 201602 but I want it to give 20160101 (format YYYYMMDD)
Why is this happening and what is the correct M/PowerQuery to avoid this behavior?
I believe you are correct as far as what it's doing. How about this instead?
each [Year]*10000 + [MonthNum]*100 + [DayNum]
or you could do
each [MonthYearNum]*100 + [DayNum]

text contains another field (power query)

i need to filter a field which contains another field in the same query.
#"Filtered Rows" = Table.SelectRows(#"Filtered Rows1", each Text.Contains([ACIKLAMA], [SANTIYE]))
the error i got is
Expression.Error: The field 'SANTIYE' of the record wasn't found.
full code :
let
Source = Table.NestedJoin(Query1,{"Sicil No", "TARIH"},#"IK Bordro",{"Personel Kodu", "Bordro Tarihi"},"NewColumn",JoinKind.LeftOuter),
#"Expanded NewColumn" = Table.ExpandTableColumn(Source, "NewColumn", {"Santiye", "Taseron", "Turk/Yerel"}, {"Santiye", "Taseron", "Turk/Yerel"}),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded NewColumn",{{"TARIH", type date}}),
#"Added Conditional Column" = Table.AddColumn(#"Changed Type", "Bordro", each if [#"Turk/Yerel"] = "YEREL" then "YEREL BORDRO" else if Text.Contains([ACIKLAMA], "RUBLE") then "TURK RUBLE" else if Text.Contains([ACIKLAMA], "USD") then "TURK USD" else if Text.Contains([ACIKLAMA], "RUB") then "TURK RUBLE" else if Text.Contains([ACIKLAMA], "IZIN") then "TURK IZIN HAKKI" else if Text.Contains([ACIKLAMA], "IHBAR") then "TURK IHBAR HAKKI" else if Text.Contains([ACIKLAMA], "KIDEM") then "TURK KIDEM HAKKI" else "DIGER" ),
#"Reordered Columns" = Table.ReorderColumns(#"Added Conditional Column",{"Sicil No", "HESAP ADI", "TARIH", "ACIKLAMA", "Santiye", "Taseron", "Turk/Yerel", "Bordro", "Ruble Tahakkuk", "USD Tahakkuk"}),
#"Filtered Rows1" = Table.SelectRows(#"Reordered Columns", each [Santiye] <> null),
#"Filtered Rows" = Table.SelectRows(#"Filtered Rows1", each Text.Contains([ACIKLAMA], [Santiye] ))
in
#"Filtered Rows"
any ideas, workarounds ?
Maybe you are mixing up step names and field names?
It works fine with me as you can see here.
Code generated:
let
Source = ..... (table created, code not relevant),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"ACIKLAMA", type text}, {"SANTIYE", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each Text.Contains([ACIKLAMA],[SANTIYE]))
in
#"Filtered Rows"

Resources