I have a Date/Time column in format "13.10.2020 06:08:43" and I need to calculate the "age" from current time. Output should be like from given example: "4h" since its 10:08 atm - Date/Time column was 4h ago. I need this to be in M language to use in visual studio 2019.
I used:
let
Source = Sql.Database("xxxx", "xxxxx", [Query="SELECT#(lf)#(tab)storeid as 'Shop'#(lf)#(tab), concat('SCO', posnr) as POS#(lf)#(tab), concat(datediff(hour,[LastTransactionDate]#(lf)#(tab), CURRENT_TIMESTAMP),' h') as 'Age'#(lf)#(tab), POSIPaddress as 'IP'#(lf)#(tab),[PosNr]#(lf) ,[AdapterPosOrGroupId]#(lf) ,[UpdatedOn]#(lf) ,[LastTransactionDate]#(lf) ,[LastStartDate]#(lf) ,[Open]#(lf) ,[Locked]#(lf) ,[CashDisabled]#(lf) ,[IsCashEnabled]#(lf) ,[CashDevicesTraficLight]#(lf) ,[POSIPAddress]#(lf) ,[ScoPosServiceVersion]#(lf) ,[WinScoVersion]#(lf) ,[StoreType]#(lf)FROM [LaneEventDatabase ].[dbo].[POS.LaneCurrentState]"]),
#"Added Conditional Column" = Table.AddColumn(Source, "Custom", each if [CashDevicesTraficLight] = "green" then 1 else if [CashDevicesTraficLight] = "yellow" then 2 else if [CashDevicesTraficLight] = "red" then 3 else 4),
#"Changed Type" = Table.TransformColumnTypes(#"Added Conditional Column",{{"Custom", type number}})
in
#"Changed Type"
which works just great but visual studio doesnt like the way source is handled and while deploying gives me error:
Failed to save modifications to the server. Error returned: 'An M partition uses a data function which results in access to a data source different from those defined in the model.
'.
Just create a new Custom Column using this below code-
= Text.From(
(Duration.Days(DateTime.LocalNow()-[date]) * 24) +
Duration.Hours(DateTime.LocalNow()-[date])
) & "h"
Here is sample output-
Related
I am trying to run the query in power query editor and it failed with below error
where as the same query works fine in winserver 2016 and it fails with above error in win server 2019. Am I missing any thing ?
I did compared the settings and everything looks good.
let
//Get data Report H1
Source1 = Excel.Workbook(File.Contents("\\path\filename.xlsx"), null, true),
#"classic_Sheet1" = Source1{[Item = "classic", Kind = "Sheet"]}[Data],
#"Trimmed Text1" = Table.TransformColumns(#"classic_Sheet1", Text.Trim),
#"Third Row as Header1" = Table.PromoteHeaders(Table.Skip(#"Trimmed Text1", 2)),
#"Selected Columns1" = Table.SelectColumns(
#"Third Row as Header1",
{" ID", " Status", "Customer Id ", "Agent", "Leg"}
),
//Get Report H2
Source2 = Excel.Workbook(File.Contents("\\path\filename.xlsx"), null, true),
#"classic_Sheet2" = Source2{[Item = "classic", Kind = "Sheet"]}[Data],
#"Trimmed Text2" = Table.TransformColumns(#"classic_Sheet2", Text.Trim),
#"Third Row as Header2" = Table.PromoteHeaders(Table.Skip(#"Trimmed Text2", 2)),
#"Selected Columns2" = Table.SelectColumns(
#"Third Row as Header2",
{" ID", "Status", "Customer Id ", "Agent", "Leg"}
)
in
#"Excluded IDs"
The error message is pointing you to the problem:
Table.TransformColumns expects a list as the second parameter,
while you are providing a function:
Table.TransformColumns(
table as table,
transformOperations as list,
optional defaultTransformation as nullable function,
optional missingField as nullable number
) as table
Please read the official documentation here:
https://learn.microsoft.com/en-us/powerquery-m/table-transformcolumns
The issue has nothing to do with winserver 2016 or winserver 2019.
You want something along these lines
= Table.TransformColumns(#"classic_Sheet1",{{"ColumnNameHere", Text.Trim, type text}})
= Table.TransformColumns(#"classic_Sheet1"",{{"ColumnNameHere", Text.Trim, type text}, {"DifferentColumnNameHere", Text.Trim, type text}})
I'm scraping a website with different URLs (about 1000) and only able to get 25 query due to website limitation (of course!). I have used Function.InvokeAfter with Number.RandomBetween but no luck.
However, I found that if I wait/pause about an hour or so after the 25th URL, I could query again. Now, I would like to automate that part.
I created a column called Wait with a value on every 25th row on the URL_List table with a number between 60-90 so that every 25th URL, I can have it wait 60 to 90 mins but I don't know how to implement. If I could replace Number.RandomBetween(10,20) with Number.RandomBetween(10,[Wait]). Or also open to other ideas.
Here's what I have done so far:
Function for fGetResults:
let Scrape=(URL) =>
let
Source = Web.Page(Web.Contents(URL)),
Data0 = ()=>Source{0}[Data],
#"Transposed Table" = Table.Transpose(Function.InvokeAfter(Data0, #duration(0,0,0,Number.RandomBetween(10,20))))
in
#"Transposed Table"
in Scrape
Query:
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each fGetResults([URL_List]))
in
#"Added Custom"
Merci beaucoup...
#############
UPDATE
#############
I did some more digging and found this and this. Instead of a Wait column with a number from 60-90mins after 25th URL or so, I just place a "yes".
WaitMinutes function:
let
Wait = (minutes as number, action as function) =>
if (List.Count(List.Generate(() => DateTimeZone.LocalNow() + #duration(0,0,minutes,0), (x) => DateTimeZone.LocalNow() < x, (x) => x)) = 0) then null else action()
in
Wait
Now, I tried it but is now giving me an error Token Comma expected at the then statement on the New Query, any thoughts?
New Query:
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each [Wait] = "yes" then WaitMinutes(Number.RandomBetween(60,90), () => fGetResults([URL_List])) else fGetResults([URL_List]))
in
#"Added Custom"
#############
UPDATE2
#############
I think I found the issue. Simply forgot the "if" on the if-then-else statement. The only thing is, there maybe a more elegant solution for this but this works for now. Let me know if you have a more efficient solution. Thank you.
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each if [Wait] = "yes" then WaitMinutes(Number.RandomBetween(60,90), () => fGetResults([URL_List])) else fGetResults([URL_List]))
in
#"Added Custom"
#############
UPDATE3: SOLUTION
#############
More problems, I think my usage for minutes is not correct but followed exactly what the website said and used seconds and removed the random function on the Query.
WaitSeconds function:
let
Wait = (seconds as number, action as function) =>
if (List.Count(List.Generate(() => DateTimeZone.LocalNow() + #duration(0,0,0,seconds), (x) => DateTimeZone.LocalNow() < x, (x) => x)) = 0) then null else action()
in
Wait
Query (15mins seems to work which is same as 900sec):
let
Source = Excel.CurrentWorkbook(){[Name="URL_Table"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "Custom", each if [Wait] = "yes" then WaitSeconds(900, () => fGetResults([URL_List])) else fGetResults([URL_List]))
in
#"Added Custom"
I am building an image viewing app in Node.js. I noticed that in Windows, the pictures in a folder can be sorted by name, size, status, type, date and tags etc, and grouped after sorting by the same list and more.
Is there a way of getting the sort parameters or maybe just retrieving the sorted list of files, matching the regular expression /\.(jpg|jpg_large|jpeg|jpe|jfif|jif|jfi|jpe|gif|png|ico|bmp|webp|svg)$/i, as an array (ex: ['c:\man.jpg', 'c:\woman.jpg'] using Powershell?
EDIT:
This article got me closer to a solution. https://cyberforensicator.com/2019/02/03/shellbags-forensics-directory-viewing-preferences/
Unfortunately it doesn't explain how to get the nodelist value for a given folder so I used an app called shellbagsview from nirsoft to get this value. In any case, if the value is found the rest is easy. I have included a sample python script which explains how this is done here.
from winreg import *
# Registry is of the form:
# HKEY_CURRENT_USER\Software\Classes\Local Settings\Software\Microsoft\Windows\Shell\Bags\1375\Shell\{5C4F28B5-F869-4E84-8E60-F11DB97C5CC7}
# where 1375 is a value called the NodeList, and {5C4F28B5-F869-4E84-8E60-F11DB97C5CC7} is a value under Shell chosen based on creation date. It is a good idea to look at the registry after getting the nodelist from shellbagsview
folder_reg_path = "Software\\Classes\\Local Settings\\Software\\Microsoft\\Windows\Shell\\Bags\\1375\\Shell\\{5C4F28B5-F869-4E84-8E60-F11DB97C5CC7}"
# the size of icons used by the folder
def get_folder_icon_size(reg_key):
with OpenKey(HKEY_CURRENT_USER, reg_key) as key:
value = QueryValueEx(key, 'IconSize')
return '%d pixels' % (value[0])
# the folder view. details, list, tiles e.t.c
def get_logical_view_mode(reg_key):
with OpenKey(HKEY_CURRENT_USER, reg_key) as key:
value = QueryValueEx(key, 'LogicalViewMode')
logical_view_mode_dict = {1 : "Details view", 2 : "Tiles view", 3 : "Icons view", 4 : "List view", 5 : "Content view"}
return logical_view_mode_dict[value[0]]
# folder view is based on view mode. so you can have a logical view mode of icons view with a view mode of large icons for instance
def get_folder_view_mode(reg_key):
with OpenKey(HKEY_CURRENT_USER, reg_key) as key:
value = QueryValueEx(key, 'Mode')
# view_mode 7 is only available on xp. A dead os
view_mode_dict = {1 : "Medium icons", 2 : "Small icons", 3 : "List", 4 : "Details", 5 : "Thumbnail icons", 6 : "Large icons", 8 : "Content"}
return view_mode_dict[value[0]]
# how is the folder being sorted
def get_folder_sort_by(reg_key):
with OpenKey(HKEY_CURRENT_USER, reg_key) as key:
value = QueryValueEx(key, 'Sort')
folder_sort_dict = {"0E000000" : "Date Modified", "10000000" : "Date Accessed", "0F000000" : "Date Created", "0B000000" : "Type", "0C000000" : "Size", "0A000000" : "Name", "02000000" : "Title", "05000000" : "Tags"}
# we get a byte value which we will hexify and get a rather long string
# similar to : 000000000000000000000000000000000100000030f125b7ef471a10a5f102608c9eebac0c000000ffffffff
reg_value = value[0].hex()
# now for this string, we need to get the last 16 strings. then we now get the first 8 out of it. so we will have
folder_sort_dict_key = (reg_value[-16:][:8]).upper()
return folder_sort_dict[folder_sort_dict_key]
# in what order is the folder being sorted. ascending or descending???
def get_folder_sort_by_order(reg_key):
with OpenKey(HKEY_CURRENT_USER, reg_key) as key:
value = QueryValueEx(key, 'Sort')
folder_sort_dict = {"01000000" : "Ascending", "FFFFFFFF" : "Descending"}
# we get a byte value which we will hexify and get a rather long string
# similar to : 000000000000000000000000000000000100000030f125b7ef471a10a5f102608c9eebac0c000000ffffffff
reg_value = value[0].hex()
# now for this string, we need to get the last 16 strings. then we now get the last 8 out of it. so we will have
folder_sort_dict_key = (reg_value[-16:][-8:]).upper()
return folder_sort_dict[folder_sort_dict_key]
icon_size = get_folder_icon_size(folder_reg_path)
logical_view_mode = get_logical_view_mode(folder_reg_path)
view_mode = get_folder_view_mode(folder_reg_path)
sorted_by = get_folder_sort_by(folder_reg_path)
sorted_by_order = get_folder_sort_by_order(folder_reg_path)
print ('The folder icon size is %s' % icon_size)
print('The folder logical view mode is %s' % logical_view_mode)
print('The folder view mode is %s' % view_mode)
print('The folder is sorted by %s in %s order' % (sorted_by, sorted_by_order))
The question itself and
the environment to run this in is unclear.
As you reference PowerShell and a RegEx to limit to specific extensions,
With this sample tree:
> tree /f a:\
A:\
└───Test
boy.bmp
child.ico
girl.gif
man.jpg
woman.jpg
this script:
Get-ChildItem -Path A:\Test -File |
Where-Object Extension -match '\.(jpg|jpg_large|jpeg|jpe|jfif|jif|jfi|jpe|gif|png|ico|bmp|webp|svg)$' |
Sort-Object Name |
Select-Object -ExpandProperty FullName |
ConvertTo-Json -Compress
yields:
["A:\\Test\\boy.bmp","A:\\Test\\child.ico","A:\\Test\\girl.gif","A:\\Test\\man.jpg","A:\\Test\\woman.jpg"]
The IShellView implementation (the file list part of Explorer) asks its IShellBrowser for a stream when it needs to load/save its state. My suggestion would be to host a IExplorerBrowser instance "browsed to the folder" and ask the view for its items. I don't know if you can ask it about which column it has sorted by but just getting the items in sorted order should be enough for your needs.
I don't know how to this in a scripting language but I assume PS supports enough COM for it to be possible.
I have Report where the dates are as column headers.
The report is set as "show last x weeks" so each week the date should shift by 7 days, but it causes an issue down the line as PowerQuery fixes the header name as a constant into the code and does not update accordingly.
So this week report should start with 10/12/18 date range and end with 11/02/19.
= Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"Project
Location Name", "Role Name", "03/12/2018", "10/12/2018",
"17/12/2018", "24/12/2018", "31/12/2018", "07/01/2019", "14/01/2019",
"21/01/2019", "28/01/2019", "04/02/2019", "Total"}, {"Project Location
Name", "Role Name", "03/12/2018", "10/12/2018", "17/12/2018",
"24/12/2018", "31/12/2018", "07/01/2019", "14/01/2019", "21/01/2019",
"28/01/2019", "04/02/2019", "Total"})
One way to do this might be by changing this line:
Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"Project Location Name", "Role Name", "03/12/2018", "10/12/2018", "17/12/2018", "24/12/2018", "31/12/2018", "07/01/2019", "14/01/2019", "21/01/2019", "28/01/2019", "04/02/2019", "Total"}, {"Project Location Name", "Role Name", "03/12/2018", "10/12/2018", "17/12/2018", "24/12/2018", "31/12/2018", "07/01/2019", "14/01/2019", "21/01/2019", "28/01/2019", "04/02/2019", "Total"})
to:
let
headersToExpand = List.Distinct(List.Combine(List.Transform(#"Converted to Table"[Column1], Record.FieldNames))),
expandDynamically = Table.ExpandRecordColumn(#"Converted to Table", "Column1", headersToExpand)
in
expandDynamically
Assuming:
every row in "Column1" contains records,
your previous "step" is #"Converted to Table",
the column of records (that you want to expand) is called "Column1",
then the headersToExpand bit should give you a list of unique record field names (rather than the hard coded list you had) -- which can then be used in the expandDynamically step.
I'm still learning PowerBuilder and trying to get familiar with it. I'm receiving the following error when I try to run a program against a specific document in my database:
ct_fetch(): user api layer: internal common library error: The bind of result set item 4 resulted in an overflow. ErrCode: 2.
What does this error mean? What is item 4? This is only when I run this program against a specific document in my database, any other document works fine. Please see code below:
string s_doc_nmbr, s_doc_type, s_pvds_doc_status, s_sql
long l_rtn, l_current_fl, l_apld_fl, l_obj_id
integer l_pvds_obj_id, i_count
IF cbx_1.checked = True THEN
SELECT dsk_obj.obj_usr_num,
dsk_obj.obj_type,
preaward_validation_doc_status.doc_status,
preaward_validation_doc_status.obj_id
INTO :s_doc_nmbr, :s_doc_type, :s_pvds_doc_status, :l_pvds_obj_id
FROM dbo.dsk_obj dsk_obj,
preaward_validation_doc_status
WHERE dsk_obj.obj_id = :gx_l_doc_obj_id
AND preaward_validation_doc_status.obj_id = dsk_obj.obj_id
using SQLCA;
l_rtn = sqlca.uf_sqlerrcheck("w_pdutl095_main", "ue_run_script", TRUE)
IF l_rtn = -1 THEN
RETURN -1
END IF
//check to see if document (via obj_id) exists in the preaward_validation_doc_status table.
SELECT count(*)
into :i_count
FROM preaward_validation_doc_status
where obj_id = :l_pvds_obj_id
USING SQLCA;
IF i_count = 0 THEN
//document doesn't exist
// messagebox("Update Preaward Validation Doc Status", + gx_s_doc_nmbr + ' does not exist in the Preaward Validation Document Status table.', Stopsign!)
//MC - 070815-0030-MC Updating code to insert row into preaward_validation_doc_status if row doesn't already exist
// s_sql = "insert into preaward_validation_doc_status(obj_id, doc_status) values (:gx_l_doc_obj_id, 'SUCCESS') "
INSERT INTO preaward_validation_doc_status(obj_id, doc_status)
VALUES (:gx_l_doc_obj_id, 'SUCCESS')
USING SQLCA;
IF sqlca.sqldbcode <> 0 then
messagebox('SQL ERROR Message',string(sqlca.sqldbcode)+'-'+sqlca.sqlerrtext)
return -1
end if
MessageBox("PreAward Validation ", 'Document number ' + gx_s_doc_nmbr + ' has been inserted and marked as SUCCESS for PreAward Validation.')
return 1
Else
//Update document status in the preaward_validation_doc_status table to SUCCESS
Update preaward_validation_doc_status
Set doc_status = 'SUCCESS'
where obj_id = :l_pvds_obj_id
USING SQLCA;
IF sqlca.sqldbcode <> 0 then
messagebox('SQL ERROR Message',string(sqlca.sqldbcode)+'-'+sqlca.sqlerrtext)
return -1
end if
MessageBox("PreAward Validation ", 'Document number '+ gx_s_doc_nmbr + ' has been marked as SUCCESS for PreAward Validation.')
End IF
update crt_script
set alt_1 = 'Acknowledged' where
ticket_nmbr = :gx_s_ticket_nmbr and
alt_2 = 'Running' and
doc_nmbr = :gx_s_doc_nmbr
USING SQLCA;
Return 1
ElseIF cbx_1.checked = False THEN
messagebox("Update Preaward Validation Doc Status", 'The acknowledgment checkbox must be selected for the script to run successfully. The script will now exit. Please relaunch the script and try again . ', Stopsign!)
Return -1
End IF
Save yourself a ton of headaches and use datawindows... You'd reduce that entire script to about 10 lines of code.
Paul Horan gave you good advice. This would be simple using DataWindows or DataStores. Terry Voth is on the right track for your problem.
In your code, Variable l_pvds_obj_id needs to be the same type as gx_l_doc_obj_id because if you get a result, it will always be equal to it. From the apparent naming scheme it was intended to be long. This is the kind of stuff we look for in peer reviews.
A few other things:
Most of the time you want SQLCode not SQLDbCode but you didn't say what database you're using.
After you UPDATE crt_script you need to check the result.
I don't see COMMIT or ROLLBACK. Autocommit isn't suitable when you need to update multiple tables.
You aren't using most of the values from the first SELECT. Perhaps you've simplified your code for posting or troubleshooting.