I have a use case where I would like to pass two arguments to a function generated by Function.ScalarVector (see https://learn.microsoft.com/en-us/powerquery-m/function-scalarvector).
I would like the second argument to, optionally, capture two or more columns of a table, e.g. ScalarFun([Col1], {[Col2], [Col3]}). I would expect this to pass a list of lists to the function itself. Alas, it does not.
Consider this example, if I define a function to simply capture the generated input list (which we'd normally pass on to a function doing something useful with the list) we can see that passing a list of two values and a concatenation of two values generate very different behaviour:
let
ScalarFun =
Function.ScalarVector(
type function (col as any) as any,
(t) =>
let
buf = Table.Buffer(t)
in
List.Transform(buf[col], each List.Distinct(buf[col]))
),
TestTable = Table.FromColumns(
{{"a","b","c"}, {"x","y","z"}},
{"Col2", "Col3"}
),
#"List syntax" = Table.AddColumn(
TestTable,
"List1",
each Text.Combine(List.First(ScalarFun({[Col2],[Col3]})), ";")
),
#"Concactenation syntax" = Table.AddColumn(
#"List syntax",
"List2",
each Text.Combine(ScalarFun([Col2] & [Col3]), ";")
)
in
#"Concactenation syntax"
Given this output I can see that the syntax where we combine two values into a list ScalarFun([Col1], {[Col2], [Col3]}) actually operate row by row. Concatenating two text values however works just fine (the function gets passed a list containing the entire column).
A list is just a value like any other, so I don't understand why my preferred syntax doesn't work?
Update:
It's clear to me that the function does not behave in the way I anticipated due to lazy evaluation. I see the same issue if I try to use a record. Passing a list in this rather convoluted way 'works' however:
Text.Split(Text.Combine({[Col2],[Col3]}, ";"),";")
I'm not going to post this as an answer because I don't fully understand why this (silently, without an error) breaks the intended behaviour of Function.ScalarVector.
A list of lists is not the same as concatenated lists so I don't see any reason to expect them to behave the same way.
{TestTable[Col2], TestTable[Col3]} = {{a,b,c},{x,y,z}}
TestTable[Col2] & TestTable[Col3] = {a,b,c,x,y,z}
Related
I have to be honest that I don't quite understand Lua that well yet. I am trying to overwrite a local numeric value assigned to a set table address (is this the right term?).
The addresses are of the type:
project.models.stor1.inputs.T_in.default, project.models.stor2.inputs.T_in.default and so on with the stor number increasing.
I would like to do this in a for loop but cannot find the right expression to make the entire string be accepted by Lua as a table address (again, I hope this is the right term).
So far, I tried the following to concatenate the strings but without success in calling and then overwriting the value:
for k = 1,10,1 do
project.models.["stor"..k].inputs.T_in.default = 25
end
for k = 1,10,1 do
"project.models.stor"..j..".T_in.default" = 25
end
EDIT:
I think I found the solution as per https://www.lua.org/pil/2.5.html:
A common mistake for beginners is to confuse a.x with a[x]. The first form represents a["x"], that is, a table indexed by the string "x". The second form is a table indexed by the value of the variable x. See the difference:
for k = 1,10,1 do
project["models"]["stor"..k]["inputs"]["T_in"]["default"] = 25
end
You were almost close.
Lua supports this representation by providing a.name as syntactic sugar for a["name"].
Read more: https://www.lua.org/pil/2.5.html
You can use only one syntax in time.
Either tbl.key or tbl["key"].
The limitation of . is that you can only use constant strings in it (which are also valid variable names).
In square brackets [] you can evaluate runtime expressions.
Correct way to do it:
project.models["stor"..k].inputs.T_in.default = 25
The . in models.["stor"..k] is unnecessary and causes an error. The correct syntax is just models["stor"..k].
I have a list of values that range anywhere from 500-1000. I have a second list of values that denote relevant breakpoints in the 500-1000 range (500, 520, 540, 600, etc). I need to return the highest value in the second list that is less than the value in a given number from the first list. I noticed the "N" functions let you set a conditional on them, so for example if I do:
List.Max(List.FirstN(SomeTable[Breakpoints], each _ < 530))
It correctly returns 520 to me. However if I put this inside an AddColumn function and change the 530 to a local field reference:
Table.AddColumn(MyTable, "MinValue", each List.Max(List.FirstN(SomeTable[Breakpoints], each _ < [SomeNumbers])))
Then I get a "We cannot apply field access to the type Number" error. Is what I'm trying to do possible and I'm just formatting it wrong? I always get confused with scope and references in PQ, so it may just be that.
After each, [SomeNumbers] by itself is short for _[SomeNumbers] (which is what you see when filtering a column). In the List.FirstN call, _ refers to a number in the list instead of a row in a table: the value of _ is tied to the closest each, where closeness is measured by the number of layers of nesting between _ and the appearance of each . Therefore, in your code [SomeNumbers] is trying to find the column SomeNumbers on a number, which doesn't exist.
There are a couple ways to fix this:
You can use a let...in statement to store the current value of the SomeNumbers column to use it for later, like so:
each
let
currentNumber = [SomeNumbers],
result = List.Max(List.FirstN(SomeTable[Breakpoints], each _ < currentNumber))
in
result
You can explicitly define a function with the (x) => ... syntax instead of using each twice, like so:
each List.Max(List.FirstN(SomeTable[Breakpoints], (point) => point < [SomeNumbers]))
When there is a collection and you must perform two or more operations on all of its elements, what is faster?:
val f1: String => String = _.reverse
val f2: String => String = _.toUpperCase
val elements: Seq[String] = List("a", "b", "c")
iterate multiple times and perform one operation on one loop
val result = elements.map(f1).map(f2)
This approach does have the advantage, that the result after application of the first function could be reused.
iterate one time and perform all operation on each element together
val result = elements.map(element => f2(f1(element)))
or
val result = elements.map(element => f1.compose(f2)
Is there any difference in performance between these two approaches? And if yes, which is faster?
Here's the thing, transformation of a collection is more or less of runtime O(N) , * runtime cost of all the functions applied. So I doubt the 2nd set of choices you present above would make even the slightest difference in runtime. The first option you list, is a different story. New collection creation can be avoided, because that could result in overhead. That's where "view" collections come in (see this good example I spotted)
In Scala, what does "view" do?
If you had the apply several mapping operations you might do this:
val result = elements.view.map(f1).map(f2).force
(force at the end, causes all functions to evaluate)
The 2nd set of examples above would maybe be a tiny bit faster, but the "view" option could make your code more readable if you had a lot of these or complex anonymous functions used in the mapping.
Composing functions to produce a single pass transformation will probably gain you some performance, but will quickly become unreadable. Consider using views as an alernative. While this will create intermediate collections:
val result = elements.map(f1).map(f2)
This will perform lazy evaluation and will perform functional composition the same way you do:
val result = elements.view.map(f1).map(f2)
Notice that result type will be SeqView so you might want to convert it to list later with toList.
results.Where(x=>x.Members.Any(y=>members.Contains(y.Name.ToLower())
I happened to see this query in internet. Can anyone explain this query please.
suggest me a good LINQ tutorial for this newbie.
thank you all.
Edited:
what is this x and y stands for?
x is a single result, of the type of the elements in the results sequence.
y is a single member, of the type of the elements in the x.Members sequence.
These are lambda expressions (x => x.whatever) that were introduced into the language with C# 3, where x is the input, and the right side (x.whatever) is the output (in this particular usage scenario).
An easier example
var list = new List<int> { 1, 2, 3 };
var oddNumbers = list.Where(i => i % 2 != 0);
Here, i is a single int item that is an input into the expression. i % 2 != 0 is a boolean expression evaluating whether the input is even or odd. The entire expression (i => i % 2 != 0) is a predicate, a Func<int, bool>, where the input is an integer and the output is a boolean. Follow? As you iterate over the query oddNumbers, each element in the list sequence is evaluated against the predicate. Those that pass then become part of your output.
foreach (var item in oddNumbers)
Console.WriteLine(item);
// writes 1, 3
Its a lambda expression. Here is a great LINQ tutorial
Interesting query, but I don't like it.
I'll answer your second question first. x and y are parameters to the lambda methods that are defined in the calls to Where() and Any(). You could easy change the names to be more meaningful:
results.Where(result =>
result.Members.Any(member => members.Contains(member.Name.ToLower());
And to answer your first question, this query will return each item in results where the Members collection has at least one item that is also contained in the Members collection as a lower case string.
The logic there doesn't make a whole lot of sense to me with knowing what the Members collection is or what it holds.
x will be every instance of the results collection. The query uses lambda syntax, so x=>x.somemember means "invoke somemember on each x passed in. Where is an extension method for IEnumerables that expects a function that will take an argument and return a boolean. Lambda syntax creates delegates under the covers, but is far more expressive for carrying out certain types of operation (and saves a lot of typing).
Without knowing the type of objects held in the results collection (results will be something that implements IEnumerable), it is hard to know exactly what the code above will do. But an educated guess is that it will check all the members of all the x's in the above collection, and return you an IEnumerable of only those that have members with all lower-case names.
If curly brackets ('{' and '}') are used in Lua, what are they used for?
Table literals.
The table is the central type in Lua, and can be treated as either an associative array (hash table or dictionary) or as an ordinary array. The keys can be values of any Lua type except nil, and the elements of a table can hold any value except nil.
Array member access is made more efficient than hash key access behind the scenes, but the details don't usually matter. That actually makes handling sparse arrays handy since storage only need be allocated for those cells that contain a value at all.
This does lead to a universal 1-based array idiom that feels a little strange to a C programmer.
For example
a = { 1, 2, 3 }
creates an array stored in the variable a with three elements that (coincidentally) have the same values as their indices. Because the elements are stored at sequential indices beginning with 1, the length of a (given by #a or table.getn(a)) is 3.
Initializing a table with non-integer keys can be done like this:
b = { one=1, pi=3.14, ["half pi"]=1.57, [function() return 17 end]=42 }
where b will have entries named "one", "pi", "half pi", and an anonymous function. Of course, looking up that last element without iterating the table might be tricky unless a copy of that very function is stored in some other variable.
Another place that curly braces appear is really the same semantic meaning, but it is concealed (for a new user of Lua) behind some syntactic sugar. It is common to write functions that take a single argument that should be a table. In that case, calling the function does not require use of parenthesis. This results in code that seems to contain a mix of () and {} both apparently used as a function call operator.
btn = iup.button{title="ok"}
is equivalent to
btn = iup.button({title="ok"})
but is also less hard on the eyes. Incidentally, calling a single-argument function with a literal value also works for string literals.
list/ditionary constructor (i.e. table type constructor).
They are not used for code blocks if that's what you mean. For that Lua just uses the end keyword to end the block.
See here
They're used for table literals as you would use in C :
t = {'a', 'b', 'c'}
That's the only common case. They're not used for block delimiters. In a lua table, you can put values of different types :
t={"foo", 'b', 3}
You can also use them as dictionnaries, à la Python :
t={name="foo", age=32}