Globbing patterns in windows command prompt/ powershell - windows

I would like to know if there is any way to achieve this behavior on windows, for example:
/b?n/ca? /etc/pa??wd -> executes 'cat /etc/passwd'

With limited exceptions in PowerShell, on Windows there is no support for shell-level globbing - target commands themselves must perform resolution of wildcard patterns to matching filenames; if they don't, globbing must be performed manually, up front, and the results passed as literal paths; see the bottom section for background information.
PowerShell:
Perhaps surprisingly, you can invoke an executable by wildcard pattern, as zett42 points out, though that behavior is problematic (see bottom section):
# Surprisingly DOES find C:\Windows\System32\attrib.exe
# and invokes it.
C:\Windows\System3?\attr*.exe /?
Generally, you can discover commands, including external programs, via the Get-Command cmdlet.
Many file-processing cmdlets in PowerShell do perform their own globbing (e.g., Get-ChildItem, Remove-Item); if you're calling commands that do not, notably external programs that don't, you must perform globbing manually, up front, except on Unix-like platforms when calling _external programs, where PowerShell does perform automatic globbbing (see bottom section):
Use Convert-Path to get the full, file-system-native paths of matching files or directories.
While Resolve-Path may work too, it returns objects whose .ProviderPath property you need to access to get the same information (stringifying these objects, as happens implicitly when you pass them to external programs, yields their .Path property, which may be based on PowerShell-only drives that external programs and .NET APIs know nothing about.)
For more control over what is matched, use Get-ChildItem and access the result objects' .Name or .FullName property, as needed; for instance, Get-ChildItem allows you to limit matching to files (-File) or directories (-Directory) only.
PowerShell makes it easy to use the results of manually performed globbing programmatically; the following example passes the full paths of all *.txt files in the current directory to the cmd.exe's echo command as individual arguments; PowerShell automatically encloses paths with spaces in "...", if needed:
cmd /c echo (Get-ChildItem -Filter *.txt).FullName
Generally, note that PowerShell's wildcard patterns are more powerful than those of the host platform's file-system APIs, and notably include support for character sets (e.g. [ab]) and ranges (e.g. [0-9]); another important difference is that ? matches exactly one character, whereas the native file-system APIs on Windows match none or one.
However, when using the -Filter parameter of file-processing cmdlets such as Get-ChildItem, the host platform's patterns are used, which - while limiting features - improves performance; a caveat is that on Unix-like platforms ? then seemingly acts like on Windows, i.e causing it to match none or one character.
cmd.exe (Command Prompt, the legacy shell):
cmd.exe does not support calling executables by wildcard pattern; some of cmd.exe's internal commands (e.g., dir and del) and some standard external programs (e.g., attrib.exe) do perform their own globbing; otherwise you must perform globbing manually, up front:
where.exe, the external program for discovering external programs fundamentally only supports wildcard patterns in executable names (e.g. where find*.exe), not in paths, which limits wildcard-based lookups to executables located in directories listed in the PATH environment variable.
:: OK - "*" is part of a *name* only
where.exe find*.exe
:: !! FAILS: "*" or "?" must not be part of a *path*
:: !! -> "ERROR: Invalid pattern is specified in "path:pattern"."
where.exe C:\Windows\System32\find*.exe
Globbing via dir appears to be limited to wildcard characters in the last path component:
:: OK - "*" is only in the *last* path component.
dir C:\Windows\System32\attri*
:: !! FAILS: "*" or "?" must not occur in *non-terminal* components.
:: !! -> "The filename, directory name, or volume label syntax is incorrect."
dir C:\Windows\System3?\attri*
Using manual globbing results programmatically is quite cumbersome in cmd.exe and requires use of for statements (whose wildcard matching has the same limitations as the dir command); for example, using the syntax for batch files (.cmd or .bat files):
To use the resolved executable file path for invocation (assuming only one file matches):
#echo off
setlocal
:: Use a `for` loop over a wildcard pattern to enumerate
:: the matching filenames - assumed to be just *one* in this case,
:: namely attrib.exe, and save it in a variable.
for %%f in (C:\Windows\System32\attr*.exe) do set "Exe=%%f"
:: Execute the resolved filename to show its command-line help.
"%Exe%" /?
To pass matching filenames as multiple arguments to a single command:
#echo off
setlocal enableDelayedExpansion
:: Use a `for` loop over a wildcard pattern to enumerate
:: matching filenames and collect them in a single variable.
set files=
for %%f in (*.txt) do set files=!files! "%%f"
:: Pass all matching filenames to `echo` in this example.
echo %files%
Background information:
On Unix-like platforms, POSIX-compatible shells such as Bash themselves perform globbing (resolving filename wildcard patterns to matching filenames), before the target command sees the resulting filenames, as part of a feature set called shell expansions (link is to the Bash manual).
On Windows, cmd.exe (the legacy shell also known as Command Prompt) does NOT perform such expansions and PowerShell mostly does NOT.
That is, it is generally up to each target command to interpret wildcard patterns as such and resolve them to matching filenames.
That said, in PowerShell, many built-in commands, known as cmdlets, do support PowerShell's wildcard patterns, notably via the -Path parameter of provider cmdlets, such as Get-ChildItem.
Additionally and more generally, cmdlet parameters that represent names often support wildcards too; e.g., Get-Process exp* lists all processes whose image name start with exp, such as explorer.
Note that the absence of Unix-style shell expansions on Windows also implies that no semantic distinction is made between unquoted and quoted arguments (e.g., *.txt vs. "*.txt"): a target command generally sees both as verbatim *.txt.
In PowerShell, automatic globbing DOES occur in these limited cases:
Perhaps surprisingly, an executable file path can be invoked via a wildcard pattern:
as-is, if the pattern isn't enclosed in '...' or "..." and/or contains no variable references or expressions; e.g.:
C:\Windows\System3?\attri?.exe
via &, the call operator, otherwise; e.g.:
& $env:SystemRoot\System32\attri?.exe
However, this feature is of questionable utility - When would you not want to know up front what specific executable you're invoking? - and it is unclear whether it was implemented by design, given that inappropriate wildcard processing surfaces in other contexts too - see GitHub issue #4726.
Additionally, up to at least PowerShell 7.2.4, if two or more executables match the wildcard pattern, a misleading error occurs, suggesting that no matching executable was found - see GitHub issue #17468; a variation of the problem also affects passing a wildcard-based path (as opposed to a mere name) that matches multiple executables to Get-Command.
In POSIX-compatible shells, the multi-match scenario is handled differently, but is equally useless: the first matching executable is invoked, and all others are passed as its arguments.
On Unix-like platforms only, PowerShell emulates the globbing feature of POSIX-compatible shells when calling external programs, in an effort to behave more like the platform-native shells; if PowerShell didn't do that, something as simple as ls *.txt would fail, given that the external /bin/ls utility would then receive verbatim *.txt as its argument.
However, this emulation has limitations, as of PowerShell 7.2.4:
The inability to use wildcard patterns that contain spaces - see GitHub issue #10683.
The inability to include hidden files - see GitHub issue #4683.
A still experimental feature, available in preview versions of 7.3, PSNativePSPathResolution, automatically translates wildcard patterns based on PowerShell-only drives to their underlying native file-system paths; however, this feature is currently overzealous - see GitHub issue #13640 - and inherently bears the risk of false positives - see GitHub issue #13644

In PowerShell you can use Resolve-Path which Resolves the wildcard characters in a path, and displays the path contents.
Example: I want to locate signtool.exe from the Windows SDK which typically resides in "c:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\signtool.exe" where there could be any other version(s) installed.
So I could use: Resolve-Path 'c:\program*\Windows Kits\10\bin\*\x64\signtool.exe'
EDIT:
If you want to execute it directly you can use the & invocation operator e.g.
&(Resolve-Path 'c:\wind?ws\n?tepad.exe')

Related

Batch script: Pass returned GUID as URL parameter for desktop shortcut

I'm trying to create desktop shortcuts to a private page we work with that will open in Edge, direct to a specific URL, and pass the GUID as a URL parameter.
I've tried the following but as you can expect, only the string "powershell" is passed on to the URL, not the returned GUID.
SET a=powershell -Command "[guid]::NewGuid().ToString()"
C:\Windows\System32\cmd.exe /c start msedge "https://www.website.com/page?user="%a% --no-first-run
How can I replace the %a% portion of the URL with the returned contents of the system GUID?
powershell -Command "[guid]::NewGuid().ToString()"
Note:
This answer addresses the question as asked: it captures the output from a PowerShell command in a batch-file variable for later use in the same batch file.
Alternatively, the specific task at hand can also be performed in full by a single PowerShell command, as shown in zett42's helpful answer.
Batch files (executed by cmd.exe) have no concept of a what is known as command substitution in POSIX-compatible shells (a feature that PowerShell itself provides too, though it has no official name there): the ability to assign a command's output to a variable.[1]
Instead, you must use a for /f loop to capture command output in a variable (which generally loops over each output line, but in your case there is only one output line):
#echo off
setlocal
:: Capture the output from a PowerShell command in variable %guid%, via
:: a for /f loop:
for /f "usebackq delims=" %%a in (`powershell -Command "[guid]::NewGuid().ToString()"`) do set "guid=%%a"
:: Note: No need for `cmd /c` from a batch file to use `start`
start "" msedge "https://www.website.com/page?user=%guid%" --no-first-run
Note: setlocal, while not strictly necessary, localizes any variable definitions to the batch file at hand.
Run for /? in a cmd.exe session for help.
This answer discusses using for /f to capture command output in more detail; notably:
usebackq isn't strictly necessary here, but is generally advisable to give you the freedom to use both ' and " quoting in the command line being invoked.
Similarly, delims= isn't strictly necessary here, since the output by definition contains no spaces, but it is generally advisable if the intent is to capture an output line in full.
The "" as the first start argument isn't strictly necessary here, but in general it is useful when invoking applications whose paths must be double-quoted. Without "" as the first argument, a double-quoted application path would be interpreted as starts window-title argument (which only meaningfully applies to console applications).
[1] In POSIX-compatible shells, use $(...) (e.g, captured="$(whoami)"). In PowerShell, you can use the command -as-is as the RHS of the assignment (e.g., $captured = whoami)
It is possible to do all of this directly using a PowerShell one-liner:
powershell -noprofile -command start msedge \"https://www.website.com/page?user=$(New-Guid) --no-first-run\"
Passing -noprofile to powershell.exe is most of the time a good idea to reduce startup time and provide a more predictable environment as no user profile will be loaded.
start is an alias for the Start-Process command.
Here start gets passed two positional arguments, the name of the process to start (-FilePath parameter) and the process's arguments as a single string (-ArgumentList parameter). Therefore, the 2nd argument must be quoted. To pass the quotes from the command processor cmd.exe through to PowerShell, they must be backslash-escaped.
Within the process's parameter string, the subexpression operator $(…) is used to call the New-Guid command inline and convert it to a string (by implicitly calling the .ToString() method of the Guid object it returns).
If you actually need to use the GUID as a variable in other parts of your batch script (which is not clear from the question), then this helpful answer provides a solution.

Creating a GitHub Action which runs a Powershell script on a Windows host

My GitHub Actions workflow "runs-on" windows-latest. I want a custom Action which executes a PowerShell (core or legacy is fine) script. I have a parallel action that runs on Linux and MacOS. So, my .github/actions/install-tf-win directory contains this action.yml
name: install_tf_win
description: installs Terraform/TerraGrunt for Windows
runs:
using: "composite"
steps:
- run: install-tf-win\install-tf-win.ps1
shell: pwsh
The directory also contains install-tf-win.ps1. I have tried all sorts of variations on that run statement. Starting with "&" and without, variations in paths used with forwards and backwards slashes. I originally started with $GITHUB_ACTION_PATH/install-tf-win.ps1 (works for Linux/MacOS), however it seemed that GITHUB_ACTION_PATH was getting evaluated to be an empty string and then there were complaints about /install-tf-win.ps1 not being found. I tried both pwsh and powershell for the shell key value.
The form shown above results in this error:
Run ./.github/actions/install-tf-win
install-tf-win\install-tf-win.ps1: D:\a\_temp\c1d3d7fa-074b-4f90-ade0-799dcebd84ec.ps1:2
Line |
2 | install-tf-win\install-tf-win.ps1
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| The module 'install-tf-win' could not be loaded. For more information, run 'Import-Module
| install-tf-win'.
I can obviously code my way around this by just putting the PowerShell statements in a step. But, the documentation suggests this should work. https://docs.github.com/en/free-pro-team#latest/actions/creating-actions/metadata-syntax-for-github-actions#runsstepsshell .
So, how do I get this to work, please?
The error message implies that you cannot refer to your script with a relative path:
The command is passed to pwsh, PowerShell [Core]'s CLI, via its -Command (-c) parameter, which interprets the install-tf-win part of
install-tf-win\install-tf-win.ps1 as a module name rather than as a subdirectory in the absence of an actual subdirectory by that name on Windows[1], so the implication is that such a path doesn't exist.
The linked documentation, suggests that you need an absolute path, based on the GITHUB_ACTION_PATH environment variable, which in PowerShell must be referenced as $env:GITHUB_ACTION_PATH (untested):
# ...
- run: '& $env:GITHUB_ACTION_PATH/install-tf-win/install-tf-win.ps1'
shell: pwsh
Note:
The need to use &, the call operator, which is a syntactic necessity due to the script path containing a(n environment-)variable reference; the same would apply if the path were quoted - see this answer for background information.
Since & is a metacharacter in YAML when used unquoted at the start of a value, the entire string is quoted. Single-quoting ('...') is employed in this case, so that YAML doesn't interpret the contents of the string up front.
As an aside: The implication of '...'-quoting working (as confirmed by Kevin, the OP) is that when pwsh -c is ultimately called on Windows, the string's content is (properly) double-quoted ("..."), because PowerShell's CLI only recognizes " as having syntactic function for command-line argument parsing. By contrast, a '...'-quoted -c argument would be interpreted as a verbatim string rather than as a command, causing its content to be simply echoed.
[1] How PowerShell interprets a path such as foo\bar.ps1 when executed as a command, as of PowerShell 7.1
Interpreted as a command - both inside a PowerShell session and via the -Command / -c parameter of the PowerShell CLI, as used by GitHub Actions - the form foo\bar.ps1 is ambiguous:
It could refer to a module named foo, and its bar.ps1 command (even though module commands don't have .ps1 name extensions).
It could refer to a subdirectory foo and file bar.ps1 in it.
This applies to Windows only:
PowerShell first interprets the path as a file-system path.
If the path doesn't exist, it falls back to interpreting it as a module-command reference.
On Unix-like platforms - even though PowerShell generally allows you to use \ and / interchangeably as the path separator on all supported platforms - this particular path form is always interpreted as a module reference; in other words: you cannot use this syntax form to invoke a script this way (see below).
This surprising inconsistency is discussed in GitHub issue #14307.
There are two ways to avoid this ambiguity, which work on all supported platforms; the following cause PowerShell to unequivocally treat the path as file-system script path:
Use / instead of \: foo/bar.ps1
Prefix the path with .\ (or ./): .\foo\bar.ps1
(Of course, you could also use a full path.)
Note that when the -File parameter of PowerShell's is used (rather than -Command / -c), this ambiguity doesn't arise, as the argument is then always considered a (potentially relative) file-system path, even on Unix-like platforms, irrespective of whether you use \ or /.

What is the bash equivalent of `modpath` in csh?

This is the error that I got.
modpath: Command not found.
I'm suspecting that it's because modpath is a csh command but not the right syntax for bash. So, I want to know what's its equivalent in bash. Thanks!
modpath - change global search path for dynamically loadable kernel modules
modpath allows users with appropriate privilege to modify the global search path used to locate object files for dynamically loadable kernel modules. The search path modifications take effect immediately and affect all subsequent loads for all users on the system.
pathname may be either a colon-separated list of absolute pathnames or NULL. If the former, these path names represent directories which should be searched for all autoloads of loadable kernel modules and for demand loads (see modload(2)) where the module is given by a simple file name. This list of directories will be prepended to the existing list of directories and so will be searched before any directories given in previous calls to modpath and before the default location which is always searched last. The directories do not have to exist on the system at the time modpath is called, or when a load actually takes place. If pathname is equal to NULL, the global search path is set back to its initial default value, /stand/dlkm/mod.d.
Notes: modpath is currently implemented as a macro.
[source: http://modman.unixdev.net/?sektion=2&page=modpath&manpath=HP-UX-11.11]
Based on the documentation now included in the question, modpath has nothing to do with $PATH, and it's not specific to csh, bash, or any other shell.
On my system (Ubuntu 16.10), there is no command, system call, or library function by that name, and there appears to be no installable package that provides it. The documentation you quoted is for HP-UX, which is a different flavor of UNIX. I suspect it's a system call that exists only on HP-UX.
Since you've been able to invoke modpath from csh, there's probably a command that's some kind of wrapper around the system call, though the documentation is for the system call itself, not for the command.
If such a command exists, you should be able to find it by typing
which modpath
from csh. If that gives you the full path to the command, then from bash you can either use that full path to invoke the command, or you can add the appropriate directory to your $PATH. (You very likely have a different setting for $PATH in csh vs. bash.)

Does a double-asterisk wildcard mean anything apart from `globstar`?

I have an Ant build.xml script that includes the following snippet:
<fileset dir="${project.home}/${project.lib}">
<include name="**/*.jar"/>
</fileset>
According to the answers to this question and the Bash documentation, the double-asterisk is indicative of globstar pattern-matching:
globstar
If set, the pattern ‘**’ used in a filename expansion context will match all files and zero or more directories and subdirectories. If
the pattern is followed by a ‘/’, only directories and subdirectories
match.
This seems to be the sense in which whoever wrote the code meant it to work: locate all .jar files within the project library directory, no matter how many directories deep.
However, the code is routinely executed in a Bash shell for which the globstar setting is turned off. In this case, it seems like the double asterisk should be interpreted as a single asterisk, which would break the build. Nevertheless, the build executes perfectly well.
Is there any scenario outside of globstar for which the Bash shell will interpret ** in any way differently than *? For example, does the extglob setting alone differentiate the two? Documentation on this seems to be sparse.
You present part of an Ant build file. What makes you think that bash syntax or the settings of any particular bash shell has anything to do with the interpretation of the contents of that file? Ant implements its own pattern-matching; details are documented in the Ant User Manual, and in particular, here: https://ant.apache.org/manual/dirtasks.html#patterns.
As for bash and the actual questions you posed,
Is there any scenario outside of globstar for which the Bash shell will interpret ** in any way differently than *?
In the context of arithmetic evaluation, * means multiplication, whereas ** means exponentiation.
Bash's globstar option affects how ** is interpreted in a bash pathname expansion context, and nothing else. If globstar is enabled then ** has different effect in that context than does * alone; if it is not enabled then ** is just two * wildcards, one after the other, which does not change which file names match.
Other than in those two contexts I don't think ** has any special meaning to bash at all, but there are contexts where * by itself is meaningful. For example, $* is a special variable that represents the list of positional parameters, and if foo is an array-valued variable then ${foo[*]} represents all the elements of the array. There are a few others. Substituting ** for * in those places changes the meaning; in most of them it creates a syntax error.
For example, does the extglob setting alone differentiate the two?
The bash manual has a fairly lengthy discussion of pathname expansion (== filename expansion). There are several options that affect various aspects of it, but the only one that modulates the meaning of ** is globstar.
The noglob option disables pathname expansion altogether, however. If noglob is enabled then * and ** each represents itself in contexts where pathname expansion otherwise would have been performed.
Ant does not use bash to create the fileset, that is all Java code.
The meaning of the double star is indeed as you describe, to dive down into all folders and find all *.jar in any subfolder. But works even on Windows, where there is typically no bash to be seen anywhere.

Blacklist program from bash completion

Fedora comes with "gstack" and a bunch of "gst-" programs which keep appearing in my bash completions when I'm trying to quickly type my git aliases. They're of course installed under /usr/bin along with a thousand other programs, so I can't just remove their directory from my PATH. Is there any way in Linux to blacklist these specific programs from appearing for completion?
I've tried the FIGNORE and GLOBIGNORE environment variables but they don't work, it looks like they're only for file completion after you've entered a command.
In 2016 Bash introduced an option for that. I'm reproducing the text from this newer answer by zuazo:
This is rather new, but in Bash 4.4 you can set the EXECIGNORE variable:
aa. New variable: EXECIGNORE; a colon-separate list of patterns that
will cause matching filenames to be ignored when searching for commands.
From the official documentation:
EXECIGNORE
A colon-separated list of shell patterns (see Pattern Matching) defining the list of filenames to be ignored by command search using
PATH. Files whose full pathnames match one of these patterns are not
considered executable files for the purposes of completion and command
execution via PATH lookup. This does not affect the behavior of the [,
test, and [[ commands. Full pathnames in the command hash table are
not subject to EXECIGNORE. Use this variable to ignore shared library
files that have the executable bit set, but are not executable files.
The pattern matching honors the setting of the extglob shell option.
For Example:
$ EXECIGNORE=$(which pytest)
Or using Pattern Matching:
$ EXECIGNORE=*/pytest
I don't know if you can blacklist specific files, but it is possible to complete from your command history instead of the path. To do that add the following line to ~/.inputrc:
TAB dynamic-complete-history
FIGNORE is for SUFFIXES only. It presumes for whatever reason that you want to blacklist an entire class of files. So you need to knock off the first letter.
E.g. To eliminate gstack from autocompletion:
FIGNORE=stack
Will rid gstack but also rid anything else ending in stack.

Resources