I have multiple folders of which I want a filelist. The folders are not wihin each other.
= let
Source = Table.Combine({Folder.Files("c:\folder1"),Folder.Files("c:\folder2"}),
#"Renamed Columns3" = Table.RenameColumns(#"Inserted First Characters1",{{"First Characters", "photo Type"}})
in
#"Renamed Columns3"
but then I get this error :
Expression.SyntaxError: Token Comma expected.
Remove the initial =
After that, assuming these exist (#"Inserted First Characters1", [First Characters]) it will work
let Source = Table.Combine({Folder.Files("c:\folder1"),Folder.Files("c:\folder2")}),
#"Renamed Columns3" = Table.RenameColumns(#"Inserted First Characters1",{{"First Characters", "photo Type"}})
in #"Renamed Columns3"
Related
Hello hope someone can assist me in a power query I'm having trouble with. I'm brand new to Power Query and the M language and while I do have some coding background coding is not my day job. I'm pulling data from web page and the data in one column that is a list nested in a list.
This is a clip of what I see at from query initially:
I then drill down on the list and see this for all of the rows:
I then drill down again in that list and get a this:
At this level I get at least one row but there could be many rows.
What I want is to take all of the values and combine them into one cell as a bulleted list like this:
Any assistance on how to do this would be appreciated
I've tried looking at some of the examples in other threads and only get errors when I do this.
You didn't really provide enough detail here, but it looks like a bunch of lists within lists
you can run them through something like this to expand them all. If the results dont look like what you want, provide more information and sample data we can reproduce
let Source = <<copy whatever your source is here>>,
//Marcel Beug 2017
TableSchema = Table.Schema(Source),
ColumnNames = Table.SelectColumns(TableSchema,{"Name"}),
IsListColumn = Table.AddColumn(ColumnNames, "IsListColumn?", each List.AllTrue(List.Transform(Table.Column(Source,[Name]), each _ is list))),
NonListColumns = Table.SelectRows(IsListColumn, each ([#"IsListColumn?"] = false)),
NonListColumnNames = Table.RemoveColumns(NonListColumns,{"IsListColumn?"})[Name],
SelectNonListColumns = Table.SelectColumns(Source,NonListColumnNames),
ListColumns = Table.SelectRows(IsListColumn, each ([#"IsListColumn?"] = true)),
ListColumnNames = Table.RemoveColumns(ListColumns,{"IsListColumn?"})[Name],
SelectListColumns = Table.SelectColumns(Source,ListColumnNames),
TableFromLists = Table.AddColumn(SelectListColumns, "TableFromLists", each Table.FromColumns(Record.FieldValues(_))),
ListTables = Table.SelectColumns(TableFromLists,{"TableFromLists"}),
Custom1 = Table.FromColumns({Table.ToRecords(SelectNonListColumns),Table.ToRecords(ListTables)}),
#"Expanded Column1" = Table.ExpandRecordColumn(Custom1, "Column1", Table.ColumnNames(#table(List.Min({1,List.Count(NonListColumnNames)}),{})), NonListColumnNames),
#"Expanded Column2" = Table.ExpandRecordColumn(#"Expanded Column1", "Column2", {"TableFromLists"}, {"TableFromLists"}),
#"Expanded TableFromLists" = Table.ExpandTableColumn(#"Expanded Column2", "TableFromLists", Table.ColumnNames(#table(List.Count(ListColumnNames),{})), ListColumnNames),
#"Reordered Columns" = Table.ReorderColumns(#"Expanded TableFromLists",ColumnNames[Name])
in #"Reordered Columns"
EDIT for specific website clarification
let Source = Web.Page(Web.Contents("https://ised-isde.canada.ca/site/high-speed-internet-canada/en/universal-broadband-fund/selected-universal-broadband-fund-projects")),
#"Expanded Data" = Table.ExpandTableColumn(Source, "Data", {"Location of project", "Number of Households to be served / Number of kilometers to be covered (mobile projects)", "Funding recipient", "Funding amountFootnote *"}, {"Location of project", "Number of Households to be served / Number of kilometers to be covered (mobile p", "Funding recipient", "Funding amountFootnote *"}),
#"Added Custom" = Table.AddColumn(#"Expanded Data", "Location of Project2", each Text.Combine([Location of project]{1},"#(lf)")),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Source", "ClassName", "Id", "Location of project"}),
#"Filtered Rows" = Table.SelectRows(#"Removed Columns", each ([Caption] <> "Document")),
#"FundingToAmount" = Table.TransformColumns(#"Filtered Rows",{{"Funding amountFootnote *", each Number.From(Text.Select(_,{"0".."9",".","$"})), type number}})
in #"FundingToAmount"
then in excel format that column as text control [x] wrap text
I have just imported some data into Power Query. The headers are in camel case. I.e.:
headerOne
headerTwo
headerThree
Etc.
I would like them to be in snake case. I.e.:
header_one
header_two
header_three
Etc.
I am not sure, though, how to do this. Any ideas?
Thanks.
Examine the applied steps to understand the algorithm.
let
//change next line to reflect actual data source
Source = Excel.CurrentWorkbook(){[Name="Table4"]}[Content],
//change column headers
colNames = Table.ColumnNames(Source),
#"Split at UpperCase" = List.Transform(colNames, each Splitter.SplitTextByCharacterTransition({"a".."z"},{"A".."Z"})(_)),
#"Snake Case" = List.Transform(#"Split at UpperCase", each Text.Lower(Text.Combine(_,"_"))),
rename = Table.RenameColumns(Source, List.Zip({colNames,#"Snake Case"}))
in
rename
Another method is to replace "X" with "_x" for any upper-case letter.
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlTSUUoC4mSl2FgA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [headerOne = _t, headerTwo = _t, headerThree = _t]),
Headers = Table.ColumnNames(Source),
NewHeaders = List.Transform(Headers, each Text.Combine(List.Transform(Text.ToList(_), each if List.Contains({"A".."Z"}, _) then "_" & Text.Lower(_) else _))),
Result = Table.RenameColumns(Source, List.Zip({Headers, NewHeaders}))
in
Result
This splits each header into a list of characters (using Text.ToList), replaces any capital letter in "A" to "Z" with the "_" prepended to the lower-case version (using List.Transform), and then combines the list back into a string (using Text.Combine).
I can do things in PowerQuery but I can't find how to achieve the following result:
Before:
After:
The goal is to duplicate the last row (filtered with Project Code=null) for each item in Project Code column. I think duplicating the row as is is important to keep the Metadata Table and expand it later.
Thank you very much for your help.
Try this
Grab all rows except null row
Get unique values of Project column in a list
Grab null row
Create row with the list and expand it
Put the two tables back together.
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
AllButNull=Table.SelectRows( Source, each ([Project Code] <> null)),
UniqueProjects=List.Distinct(AllButNull[Project Code]),
OnlyNull = Table.SelectRows(#"Added Custom", each ([Project Code] = null)),
#"Replaced Value" = Table.ReplaceValue(OnlyNull,null, UniqueProjects,Replacer.ReplaceValue,{"Project Code"}),
#"Expanded Project Code" = Table.ExpandListColumn(#"Replaced Value" , "Project Code"),
combined = AllButNull & #"Expanded Project Code"
in combined
Alternately, grab the last row instead of the null row:
Grab all rows except last
Get unique values of Project column in a list
Grab last row
Create row with the list and expand it
Put the two tables back together.
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
AllButLast=Table.RemoveLastN(Source ,1),
UniqueProjects=List.Distinct(AllButLast[Project Code]),
OnlyBottomRow = Table.LastN(#"Added Custom", 1),
#"Replaced Value" = Table.ReplaceValue(OnlyBottomRow,null, UniqueProjects,Replacer.ReplaceValue,{"Project Code"}),
#"Expanded Project Code" = Table.ExpandListColumn(#"Replaced Value", "Project Code"),
combined = AllButLast & #"Expanded Project Code"
in combined
I need a query (query1) from a file in a folder. This file is daily updated and I need to connect to the newest one. In order to do that I created a query (query2) which returns the newest filename in his unique record.
Now, how to set the source of query1 as a dinamic value extracted from query 2.
In below example I want, instead of pointing to staticfilename.xlsx, to point to a dinamic filename, which value in calculated with query2
let
Source = Excel.Workbook(File.Contents("Q:\....\staticfilename.XLSX"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(...)
in
#"Changed Type"
An alternative to this is to load from a folder, sort by date created (or modified), and pick the top row instead of needing a separate query.
More details in this article and this one too.
If query2 returns a 1x1 table with column name Column1, and contents Q:\path\subpath\filename.xlsx then this should work to pull the path from query2 into your next query
Source = Excel.Workbook(File.Contents(query2{0}[Column1]), null, true),
Another option is to use a function to return the latest file:
//fnLatestFile (excluding temp files)
(
FileFolder as text,
optional FileNameContains as text,
optional FileExtension as text,
optional IncludeSubfolders as logical,
optional OutputType as text
) =>
let
fSwitch = (Expression as any, Values as list, Results as list, optional Else as any) =>
try Results{List.PositionOf(Values, Expression)} otherwise if Else = null then "Value not found" else Else,
Source = Table.SelectRows(Folder.Files(FileFolder), each not Text.Contains([Name], "~")),
#"Filtered Name" = if FileNameContains = null then Source else Table.SelectRows(Source, each (Text.Contains([Name], FileNameContains) = true)),
#"Filtered Extension" = if FileExtension = null then #"Filtered Name" else Table.SelectRows(#"Filtered Name", each ([Extension] = FileExtension)),
#"Filtered Subfolder" = if IncludeSubfolders = true then #"Filtered Extension" else Table.SelectRows(#"Filtered Extension", each ([Folder Path] = Text.Combine({FileFolder, if Text.End(FileFolder,1) = "\" then "" else "\"}))),
#"Sorted by Modified Date" = Table.Sort(#"Filtered Subfolder",{{"Date modified", Order.Descending}}),
FileData = #"Sorted by Modified Date"{0},
Output = fSwitch(
Text.Lower(OutputType),
{"name","fullname","date"},
{FileData[Name], FileData[Folder Path] & FileData[Name], FileData[Date modified]},
FileData[Content]
)
in
Output
Applying to your query, your first line then becomes:
Source = Excel.Workbook(fnLatestFile("Q:\....\", "staticfilename", ".xlsx", false), null, true),
In response to this question regarding how to expand all lists in a row of lists at the same time, #Carl Walsh kindly provided this succinct and helpful code:
let
Source = #table({"A", "B"}, {{ {1,2}, {3,4}} }),
Expanded = List.Accumulate(
Table.ColumnNames(Source),
Source,
(state, column) => Table.ExpandListColumn(state, column))
in
Expanded
Which yields this:
I would like to get this result instead:
I don't want values repeated in previously processed columns as each follow-on column is processed.
Is there a simple modification to Carl's code that will get me there?
Maybe not really simple, but effective: the code below combines the columns with adjusted code so the lists are zipped and the inner lists are transformed into records. Next the list column is expanded, resulting in a column with nested records which are subsequently expanded.
Unfortunately, you can't use combine columns with nested lists, so I created some dummy text columns first that I combined in order to generate the base code, which I subsequently adjusted and I removed the steps with the dummy columns.
let
Source = #table({"A", "B"}, {{ {1,2}, {3,4}} }),
#"Merged Columns" = Table.CombineColumns(Source,{"A", "B"}, each List.Transform(List.Zip(_), each Record.FromList(_,{"A","B"})),"Merged"),
#"Expanded Merged" = Table.ExpandListColumn(#"Merged Columns", "Merged"),
#"Expanded Merged1" = Table.ExpandRecordColumn(#"Expanded Merged", "Merged", {"A", "B"}, {"A", "B"})
in
#"Expanded Merged1"