In powershell
$a = 4;
$a
will write 4 to the output stream.
Write-Ouput $a
writes 4 to output stream
Can you explain which operation is better performance wise if all I want to do is write variables to output stream?
If you just want to see what's in your variable, you should use a dollar sign with the name of your variable. This is most likely the fastest method to display the value of the variable.
UPDATE
Short answer: You should use just $a in order to write data in the output stream. It's the fastest way to do that.
Long answer with explanation:
If you write something like $a, then it goes directly to the output stream of the current PowerShell process, in other words, it goes to the output pipeline. This is because in PowerShell you have in general only three types of possible statements:
Definition statement: They are everything that you can define in your code for further use like functions or classes.
Assignment statement: You assign a value to a variable.
Putting-in-the-pipeline statement: everything else except 1 and 2.
Example for definition statements:
Function foo($bar)
{
#do stuff here
}
Example for assignment statements:
$foo = 'bar'
Example for putting-in-the-pipeline statement:
It is exactly what you asked! If you write $a, then it will be written to the output pipeline. That’s it.
Now about Write-Output. According to this code from the official PowerShell repository, it's just a wrapper for another PowerShell Cmdlet, namely for WriteObject. Therefore, the invocation of Write-Output costs more time then the invocation of WriteObject.
As Lee_Dailey mentioned in the first comment to your question, there is a wonderful article Let’s Kill Write-Output written by Mark Kraus in his blog. At first sight it may seem that it doesn't have much to do with your question, but this article explains what happens with objects that you wrote in your script and when they are not surrounded with different "printing functions" like Write-Out. There you can also find examples of how to "bypass" the use of the Write-Out in different situations that it brings advantages in performance.
I also recommend to read these two articles from Microsoft's DevBlogs:
Understanding Streams, Redirection, and Write-Host in PowerShell
Weekend Scripter: Welcome to the PowerShell Information Stream
They will definitely help to understand how streams are implemented and how they work in PowerShell.
I am currently learning about Powershell and how to write a script to launch an application. The following snippet of code I borrowed and have modified to learn how to launch notepad. The question I have is what does $args.Clone() do or derive from? (It is from the original code which had a different path and executable program being defined/called.) I realize that the variable $myArgs is being initialized to the left of the equal sign by the function on the right. However, I have not been successful finding resources about what can you can do with .Clone() so I thought I would try and ask here.
BTW, the script works as it launchs notepad.exe and names the text file 'pp'. If the file has not previously been created, it asks me if I want to name the text file 'pp'.
$exePath = $env:NOTEPAD_HOME + '/Windows/notepad.exe'
$myArgs = $args.Clone()
$myArgs += '-pp'
$myArgs += $env:NOTEPAD_HOME
& $exePath $myArgs
tl;dr
Your modification of $myArgs can be simplified as follows:
$myArgs = $args + '-pp' + $env:NOTEPAD_HOME
With an array as the LHS, operator + performs concatenation, i.e., it appends to the array, though note that appending means that a new array is created in the process, given that arrays are fixed-size data structures in .NET.
The fact that a new array is implicitly created anyway makes the call to .Clone() in your code unnecessary.
Background Information
Typically - and in the case of the automatic $args variable - the .Clone() method is an implementation of the System.ICloneable interface.
The purpose of .Clone() is to create a copy (clone) of an object, but - as the answer linked to by BACON explains - the semantics of this operation - shallow (see below) vs. deep copying - aren't prescribed, which is why use of System.ICloneable is generally discouraged.
$args is an array (System.Array), and for arrays, .Clone() creates a so-called shallow copy. That is, how the elements of the array are copied depends on whether they are instances of value types or reference types:
for value-type elements, the new array will have an independent copy of the data of the original array's elements.
for reference-type elements, the new array will have a copy of the reference of the original array's elements, which means that elements of both arrays point to same data (objects).
For more information about value types vs. reference types, see this answer.
PowerShell implicitly clones arrays in the way described when you use += and +.
I'm trying to run a FOR loop on robot framework depending of the status of another variable.
${STATUS1}= Run Keyword And Return Status Should Be Equal As Strings ${CELLVALUE} ${EXPECTEDVALUE}
\ ${COUNT}= Set Variable If '${STATUS1}' == 'True' ${COUNT}+1
\ ... '${STATUS1}' == 'False' ${COUNT}+0
But all I get is '''0'+0'+0'+1 or similar, even if I use Run keyword If and Evaluate instead of set var, I tried to convert to integer but nothing happens and I cannot convert it to integer or number. Any suggestions? thanks in advance!
It looks like you're simply wanting to increment ${COUNT} if ${CELLVALUE} equals ${EXPECTEDVALUE}. That can be done pretty easily with Set Variable if
If you know that ${CELLVALUE} and ${EXPECTEDVALUE} are of the same internal type (eg: strings or ints), and you're using robot framework 2.9 or greater, you can write it like this:
${COUNT}= Set variable if $CELLVALUE == $EXPECTEDVALUE
... ${COUNT+1} ${COUNT}
This assumes that ${COUNT} is initialized to an integer value, which you can do by assigning it the value ${0}
If you don't know the type, can't guarantee the type, or are using an older version of robot, you can use triple-quoted strings to coerce the values to strings:
${COUNT}= Set variable if '''${CELLVALUE}''' == '''${EXPECTEDVALUE}'''
... ${COUNT+1} ${COUNT}
Of course, you could use Run Keyword and Return Status like in your example, and then compare the status. That seems like an unnecessary extra step, but it might make sense in your actual test.
The point being, you can use Set variable if and extended variable syntax to solve this problem.
Note 1: With Set variable if, two values are provided. The first value is assigned if the expression is true, the second one is assigned if the value is false. The second value is the original variable, meaning it won't be changed. If you don't provide the second value, the variable will be set to None.
Note 2: Putting an expression inside curly braces (eg: ${COUNT+1} is documented in rule 4 of extended variable syntax.
Note 3: Starting with robot framework 2.9, variables are available in the evaluation namespace with the simplified syntax $varname. So, the robot variable ${CELLVALUE} can be used in python expressions as $CELLVALUE. This is documented in the section Evaluating Expressions in the BuiltIn library documentation.
I am trying to use LINQ in PowerShell. It seems like this should be entirely possible since PowerShell is built on top of the .NET Framework, but I cannot get it to work. For example, when I try the following (contrived) code:
$data = 0..10
[System.Linq.Enumerable]::Where($data, { param($x) $x -gt 5 })
I get the following error:
Cannot find an overload for "Where" and the argument count: "2".
Never mind the fact that this could be accomplished with Where-Object. The point of this question is not to find an idiomatic way of doing this one operation in PowerShell. Some tasks would be light-years easier to do in PowerShell if I could use LINQ.
The problem with your code is that PowerShell cannot decide to which specific delegate type the ScriptBlock instance ({ ... }) should be cast.
So it isn't able to choose a type-concrete delegate instantiation for the generic 2nd parameter of the Where method. And it also does't have syntax to specify a generic parameter explicitly. To resolve this problem, you need to cast the ScriptBlock instance to the right delegate type yourself:
$data = 0..10
[System.Linq.Enumerable]::Where($data, [Func[object,bool]]{ param($x) $x -gt 5 })
Why does [Func[object, bool]] work, but [Func[int, bool]] does not?
Because your $data is [object[]], not [int[]], given that PowerShell creates [object[]] arrays by default; you can, however, construct [int[]] instances explicitly:
$intdata = [int[]]$data
[System.Linq.Enumerable]::Where($intdata, [Func[int,bool]]{ param($x) $x -gt 5 })
To complement PetSerAl's helpful answer with a broader answer to match the question's generic title:
Note: The following applies up to at least PowerShell 7.2. Direct support for LINQ - with syntax comparable to the one in C# - is being discussed for a future version of PowerShell Core in GitHub issue #2226.
Using LINQ in PowerShell:
You need PowerShell v3 or higher.
You cannot call the LINQ extension methods directly on collection instances and instead must invoke the LINQ methods as static methods of the [System.Linq.Enumerable] type to which you pass the input collection as the first argument.
Having to do so takes away the fluidity of the LINQ API, because method chaining is no longer an option. Instead, you must nest static calls, in reverse order.
E.g., instead of $inputCollection.Where(...).OrderBy(...) you must write [Linq.Enumerable]::OrderBy([Linq.Enumerable]::Where($inputCollection, ...), ...)
Helper functions and classes:
Some methods, such as .Select(), have parameters that accept generic Func<> delegates (e.g, Func<T,TResult> can be created using PowerShell code, via a cast applied to a script block; e.g.:
[Func[object, bool]] { $Args[0].ToString() -eq 'foo' }
The first generic type parameter of Func<> delegates must match the type of the elements of the input collection; keep in mind that PowerShell creates [object[]] arrays by default.
Some methods, such as .Contains() and .OrderBy have parameters that accept objects that implement specific interfaces, such as IEqualityComparer<T> and IComparer<T>; additionally, input types may need to implement IEquatable<T> in order for comparisons to work as intended, such as with .Distinct(); all these require compiled classes written, typically, in C# (though you can create them from PowerShell by passing a string with embedded C# code to the Add-Type cmdlet); in PSv5+, however, you may also use custom PowerShell classes, with some limitations.
Generic methods:
Some LINQ methods themselves are generic and therefore require one or more type arguments.
In PowerShell (Core) 7.2- and Windows PowerShell, PowerShell cannot directly call such methods and must use reflection instead, because it only supports inferring type arguments, which cannot be done in this case; e.g.:
# Obtain a [string]-instantiated method of OfType<T>.
$ofTypeString = [Linq.Enumerable].GetMethod("OfType").MakeGenericMethod([string])
# Output only [string] elements in the collection.
# Note how the array must be nested for the method signature to be recognized.
PS> $ofTypeString.Invoke($null, (, ('abc', 12, 'def')))
abc
def
For a more elaborate example, see this answer.
In PowerShell (Core) 7.3+, you now have the option of specifying type arguments explicitly (see the conceptual about_Calling_Generic_Methods help topic); e.g.:
# Output only [string] elements in the collection.
# Note the need to enclose the input array in (...)
# -> 'abc', 'def'
[Linq.Enumerable]::OfType[string](('abc', 12, 'def'))
The LINQ methods return a lazy enumerable rather than an actual collection; that is, what is returned isn't the actual data yet, but something that will produce the data when enumerated.
In contexts where enumeration is automatically performed, notably in the pipeline, you'll be able to use the enumerable as if it were a collection.
However, since the enumerable isn't itself a collection, you cannot get the result count by invoking .Count nor can you index into the iterator; however, you can use member-access enumeration (extracting the values of a property of the objects being enumerated).
If you do need the results as a static array to get the usual collection behavior, wrap the invocation in [Linq.Enumerable]::ToArray(...).
Similar methods that return different data structures exist, such as ::ToList().
For an advanced example, see this answer.
For an overview of all LINQ methods including examples, see this great article.
In short: using LINQ from PowerShell is cumbersome and is only worth the effort if any of the following apply:
you need advanced query features that PowerShell's cmdlets cannot provide.
performance is paramount - see this article.
If you want to achieve LINQ like functionality then PowerShell has some cmdlets and functions, for instance: Select-Object, Where-Object, Sort-Object, Group-Object. It has cmdlets for most of LINQ features like Projection, Restriction, Ordering, Grouping, Partitioning, etc.
See Powershell One-Liners: Collections and LINQ.
For more details on using Linq and possibly how to make it easier, the article LINQ Through Powershell may be helpful.
I ran accross LINQ, when wanting to have a stable sort in PowerShell (stable: if property to sort by has the same value on two (or more) elements: preserve their order). Sort-Object has a -Stable-Switch, but only in PS 6.1+. Also, the Sort()-Implementations in the Generic Collections in .NET are not stable, so I came accross LINQ, where documentation says it's stable.
Here's my (Test-)Code:
# Getting a stable sort in PowerShell, using LINQs OrderBy
# Testdata
# Generate List to Order and insert Data there. o will be sequential Number (original Index), i will be Property to sort for (with duplicates)
$list = [System.Collections.Generic.List[object]]::new()
foreach($i in 1..10000){
$list.Add([PSCustomObject]#{o=$i;i=$i % 50})
}
# Sort Data
# Order Object by using LINQ. Note that OrderBy does not sort. It's using Delayed Evaluation, so it will sort only when GetEnumerator is called.
$propertyToSortBy = "i" # if wanting to sort by another property, set its name here
$scriptBlock = [Scriptblock]::Create("param(`$x) `$x.$propertyToSortBy")
$resInter = [System.Linq.Enumerable]::OrderBy($list, [Func[object,object]]$scriptBlock )
# $resInter.GetEnumerator() | Out-Null
# $resInter is of Type System.Linq.OrderedEnumerable<...>. We'll copy results to a new Generic List
$res = [System.Collections.Generic.List[object]]::new()
foreach($elem in $resInter.GetEnumerator()){
$res.Add($elem)
}
# Validation
# Check Results. If PropertyToSort is the same as in previous record, but previous sequence-number is higher, than the Sort has not been stable
$propertyToSortBy = "i" ; $originalOrderProp = "o"
for($i = 1; $i -lt $res.Count ; $i++){
if(($res[$i-1].$propertyToSortBy -eq $res[$i].$propertyToSortBy) -and ($res[$i-1].$originalOrderProp -gt $res[$i].$originalOrderProp)){
Write-host "Error on line $i - Sort is not Stable! $($res[$i]), Previous: $($res[$i-1])"
}
}
There is a simple way to make Linq chaining fluent, by setting a using statement to the Linq namespace, Then you can call the where function directly, no need to call the static Where function.
using namespace System.Linq
$b.Where({$_ -gt 0})
$b is an array of bytes, and I want to get all bytes that are greater than 0.
Works perfect.
Is there some reason why bash 'variables' are different from variables in other 'normal' programming languages?
Is it due to the fact that they are set by the output of previous programs or have to be set by some kind of literal text, ie they have to be set by the output of some program or something outputting text through standard input/output or the console or such like?
I am at a loss to use the right vocabulary, but can anyone who can understands what I trying to say and perhaps use the right words or point me some docs where I can understand bash variable concepts better.
In most languages, variables can contain different kinds of values. For example, in Python a variable can be a number that you can do arithmetics on (a-1), an array or string you can split (a[3:]), or a custom, nested object (person.name.first_name).
In bash, you can't do any of this directly. If I understood you right, you asked why this is.
There are two reasons why you can't really do the same in bash.
One: environment variables are (conventionally) simple key=value strings, and the original sh was a pretty thin wrapper on top of the Unix process model. Bash works the same, for technical and compatibility reasons. Since all variables are (based on) strings, you can't really have rich, nested types.
This also means that you can't set a variable in a subshell/subscript you call. The variable won't be set in the parent script, because that's not how environment variables work.
Two: Original sh didn't separate code and data, since this makes it easier to work with interactively. Sh treated all non-special characters as literal. I.e. find / -name foo was considered four literal strings: a command and three arguments.
Bash can't just decide that find / -name now means "the value of the variable find divided by the negated value of variable name", since that would mean everyone's find commands would start breaking. This is why you can't have the simple dereferencing syntax other languages do.
Even $name-1 can't be used to substract, because it could just as easily be intended as part of $name-1-12-2012.tar.gz, a filename with a timestamp.
I would say it has to do with Bash functions. Bash functions cannot return a value, only a status code.
So with Bash you can have a function
foo ()
{
grep bar baz
}
But if you try to "save" the return value of the function
quux=$?
It is merely saving the exit status, not any value. Contrast this with a language such as Javascript, functions can actually return values.
foo ()
{
return document.getElementById("dog").getAttribute("cat");
}
and save like this
quux = foo();