Translating OS X Bash Script for Windows - bash

I use Hedge to transfer Magic Lantern video files shot on my Canon 5D Mark III.
On OS X, I'm able to use Automator to set up a bash script, to execute an mlv_dump to transfer the files from MLV into cDNG sequences.
The script I use currently is:
cd "$(dirname "$1")"
for f in "$#"; do
if [[ -f $f ]]; then
filename=${f##*/};
folder=${filename%.*}
mkdir "${folder}";
~/mlv_dump --dng $f -o ${folder}/${folder}_;
fi
done
Can this easily translate into a Windows equivalent?
Thanks,Thomas

As with any translation between programming languages, there's (a) an as-literal-as-possible approach, which contrasts with (b), an not-immediately-obvious-but-in-the-spirit-of-the-target-language approach.
(b) is always preferable in the long run.
Use PowerShell, because it is the - far superior - successor to the "Command Prompt" (cmd.exe) and its batch files.
The code below is an attempt at (b), in PowerShell (v3+ syntax).
I encourage you to study the code and post an explanation of it in an answer of your own, so that others may benefit too.
To help with the analysis, consider the following resources:
The official Windows PowerShell get-started page and the equivalent for PowerShell Core.
This series of articles is a great, recipe-oriented introduction to PowerShell.
http://hyperpolyglot.org/shell juxtaposes the syntax of POSIX-like shells such as bash with that of cmd.exe and PowerShell in concise, tabular form.
PowerShell-idiomatic translation of your code:
param(
[Parameter(Mandatory, ValueFromRemainingArguments)]
[System.IO.FileInfo[]] $LiteralPath
)
$outputBaseFolder = Split-Path -Parent $LiteralPath[0].FullName
foreach ($f in $LiteralPath) {
if ($f.exists) {
$outputFolder = Join-Path $outputBaseFolder $f.BaseName
New-Item -ItemType Directory $outputFolder
& "$HOME/mlv_dump" --dng $f.FullName -o "$outputFolder/$($f.BaseName)_"
} else {
Write-Warning "Item doesn't exist or is not a file: $($f.FullName)"
}
}

Easy, is a relative answer specific to the skills of the individual, As the others have commented, there is no out of the box thing / tool you can use or buy to do this. It's all manual work, using the pointers and others who have tried to see what you can glean to accomplish you use case.
For example, this, in your block...
filename=${f##*/};
folder=${filename%.*}
mkdir "${folder}";
... is easily translated into...
$filename='PathToFileName'
$foldername='PathToFolder'
mkdir 'FolderName'
Now, that translation is really simplistic, and obviously not complete. That is something you'll have to figure out, using the resources pointed to and the PowerShell built-in help file and those examples.
There are several posts on this very forum on this conversion topic and how others have come to a consensus oh what / if anything can be done.
For example the below will highlight the efforts you'll need to traverse to accomplish X or Y.
Convert simple Bash script to PowerShell?
I am looking to port this bash code to PowerShell. Can anyone shed
some PowerShell light on this one?
Convert simple Bash script to PowerShell?
Convert bash script into Windows script
I have the following Unix shell script. I would like to convert this
into a Windows .bat file (I know I can use Cygwin instead of adapting
it to a Windows environment. But Cygwin is not an option for me).
Convert bash script into Windows script
Convert xargs Bash command to PowerShell?
I've got a simple Bash command to resize some images automatically on
a low-traffic website using ImageMagick - I'd like to convert this to
a PowerShell command so I don't have to install Cygwin on my
webserver.
Convert xargs Bash command to PowerShell?
As noted, there are paid for avenues / even online ones who can potentially be an avenue.
But, you should really read up a bit more on how to do specific things, like..
Creating files and folders
Accessing/Reading files, importing files, filtering files
Loops
Operators (comparison and assignment)
Navigating the file system/PSDrive

Related

How can I run a Bash shell script as a Build Event in Visual Studio?

I would like to run a Bash shell script (.sh) using the Windows Subsystem for Linux as part of a Build Event in Visual Studio, in a C++ project.
But there are lots of errors when I do this, and I can't seem to find the right combination of quotation marks and apostrophes and backslashes to either make Bash run in the first place, or to properly pass the path to the script.
How do I make Visual Studio run my Bash shell script as a build event?
(Feel free to skip to the bottom of this answer if you don't care about how to solve the problem and just want a command you can copy and paste!)
Overview
I run a number of Bash shell scripts as part of Build events in Visual Studio, and I used to use Cygwin to run them. Now that the Windows Subsystem for Linux is available, I spent some time switching my builds over to use WSL, and it wasn't as easy as I'd hoped, but it can work, with a little time and energy.
There are several issues you'll run into if you're going to do this:
The path to bash.exe may not be what you think it is, because under the hood, Visual Studio uses a 32-bit build process, so if you're on a 64-bit machine, you can't simply run the 64-bit bash.exe without getting the dreaded 'C:\Windows\System32\bash.exe' is not recognized error.
The path to your solution or project is a Windows path that uses backslashes (\), and those don't play nice in Unix, which prefers forward slashes (/) as a path delimiter.
The root drive of the solution, typically something like C:\, is meaningless gibberish in Unix; to reach the root drive in WSL, you'll need to use a mounted drive under /mnt.
The casing of the drive letter is different between Windows and WSL: In Windows, it's uppercase C:\, and in WSL, it's lowercase /mnt/c.
And to make it a little harder, we don't want to hard-code any of the paths: It should all Just Work, no matter where the solution is found.
The good news is that they're all solvable issues! Let's tackle them one at a time.
Fixing the Issues
1. The proper path to Bash
Per the answer given here, you'll need to use a magic path to reach Bash when running it from a Visual Studio build. The correct path is not C:\Windows\System32\bash.exe, but is actually
%windir%\sysnative\bash.exe
The magic sysnative folder avoids the invisible filesystem redirection performed by the WOW64 layer, and points to the native bash.exe file.
2. Fixing the backslashes
The next problem you're likely to run into is the backslashes. Ideally, you'd like to run a project script like $(ProjectDir)myscript.sh, but that expands to something like C:\Code\MySolution\MyProject\myscript.sh. At a minimum, you'd like that to be at least C:/Code/MySolution/MyProject/myscript.sh, which isn't exactly right, but which is a lot closer to correct.
So sed to the rescue! sed is a Unix tool that mutates text in files: It searches for text using regular expressions, and, among other things, can replace that text with a modified version. Here, we're going to pipe the path we have into sed, and then use some regex magic to swap the path separators, like this (with lines wrapped here for readability):
%windir%\sysnative\bash.exe -c "echo '$(ProjectDir)myscript.sh'
| sed -e 's/\\\\/\//g;'"
If you include this as your build event, you'll see that it now doesn't run the script, but it at least prints something like C:/Code/MySolution/MyProject/myscript.sh to the output console, which is a step in the right direction.
And yes, that's a lot of backslashes and quotes and apostrophes to get the escaping right, because Nmake.exe and bash and sed are all going to consume some of those special symbols while processing their respective command-lines.
3. Fixing the C:\ root path
We want to mutate the sed script so that it turns the C:\ into /mnt/C. A little more regex substitution magic can do that. (And we have to turn on the -r flag in sed so that we can easily use capture groups.)
%windir%\sysnative\bash.exe -c "echo '$(ProjectDir)myscript.sh'
| sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\1/i;'"
If you run this, you'll now see the output path as something like /mnt/C/Code/MySolution/MyProject/myscript.sh, which is almost but not quite correct.
4. Fixing the case-change in the root path
WSL mounts your disks in lowercase, and Windows mounts them in uppercase. Consistency! How do we fix this? Yet more sed magic!
The \L command can be used to tell sed to transform succeeding characters to lowercase (and there's an equivalent \U for uppercase). The \E command will switch output back to "normal" mode, where characters are left untouched.
Adding these in finally results in the correct path being output:
%windir%\sysnative\bash.exe -c "echo '$(ProjectDir)myscript.sh'
| sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\L\1\E/i;'"
5. Running it
This whole time, Bash has just been printing out the path to the script. How do we run it instead, now that it's the correct path?
The answer is to add `backticks`! Backticks cause Bash to execute the command contained within them, and to then use that command's output as the arguments to the next command. In this case, we're not going to output anything: We just want to run the output of sed as a command.
So including the backticks, here's the result:
%windir%\sysnative\bash.exe -c "`echo '$(ProjectDir)myscript.sh'
| sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\L\1\E/i;'`"
The Complete Solution
Here's what the whole solution looks like, for running a script named myscript.sh as a Build Event, in the current Project directory of the current Solution:
%windir%\sysnative\bash.exe -c "`echo '$(ProjectDir)myscript.sh' | sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\L\1\E/i;'`"
Here's a screen-shot showing the same thing in Visual Studio 2017, for a real C++ project:
It's not easy, but it's not impossible.
If you have Git for Windows installed, try this. I found it simpler than installing WSL. The basic idea is to create an intermediate batch script to call your bash script, using Git bash's in-built bash or sh command from the batch script.
With Git for Windows, you'll have a Git\bin folder e.g. at:
C:\Program Files\Git\bin
Inside that directory you should see the bash.exe and sh.exe programs. So if you add that directory to your Windows Path environment variable then you'll be able to use sh and bash from the Windows command line. These commands will allow you to run your bash scripts "inline" within a CMD console window. That is, they won't spawn a new bash window; meaning the console output will be visible in your VS build.
From there, just create a .bat file which calls your .sh file using either the sh command or the bash command. Not sure the difference; we just use the sh command. So if your bash script is pre.sh, then your batch file would be just a single line calling the bash script:
sh %~dp0\pre.sh
if errorlevel 1 (
exit /b %errorlevel%
)
The %~dp0 assumes the batch and bash scripts are in the same directory. You then point your VS build event to the .bat file. The check for error level is necessary so that any failures from the bash script are forwarded up to the batch script. See: How do I get the application exit code from a Windows command line?.
To hook this in as a build event in VS2019 then, just follow the standard instructions for hooking in a .bat file: https://learn.microsoft.com/en-us/visualstudio/ide/specifying-custom-build-events-in-visual-studio?view=vs-2019.
Update: Beware Visual Studio's (VS's) Path Variable Behaviour
One thing we found quite frustrating with this solution was the tendency of VS to not load in the path variable correctly. It seems to prefer the user variable over the system variable. But even after we deleted the user variable, sometimes the path didn't seem to be getting picked up by VS, and we kept getting "sh is not recognised..." messages on our build console. Whenever that happened, restarting VS seemed to do the trick. Not very satisfying, but it gets us by.
Update: This is not a Full Unix Solution
Git for Windows does have a lot of Unix commands available, but not all of them. So in general, this won't work. For the general case, WSL is more robust. However, if it's just pretty lightweight Unix you need, this will suffice, and will likely be an easier approach for Windows users who would rather avoid the steeper setup cost of installing the full WSL.
Original idea to use Git bash came from here: https://superuser.com/questions/1218943/windows-command-prompt-capture-output-of-bash-script-in-one-step
Instead of backticks, you can wrap command with $( and )

How to pass an asterisk(*) to a command inside a Bash script?

I have done quite a bit of looking on Google, stackexchange, stackoverflow, etc. So far none of the examples satisfies a quite simple use case for using an asterisk character (not as a wildcard) within a bash script.
The following command examples work exactly as needed in the (interactive) terminal window. Both commands give the same results.
find /home/will/ref/ -path "*Java Develop*.pdf"
#
target="Java Develop"
find /home/will/ref/ -path "*$target*.pdf"
The question, and objective, is how to successfully put that command into a script with the find target
"Java Develop"
Passed as an argument? For a script named, get-ref as follows.
get-ref "Java Develop"`
None of the following (not the exhaustive) list of attempts:
find /home/will/ref/ -path \"\*$target\*.pdf\"
find /home/will/ref/ -path \"*$target*.pdf\"
find /home/will/ref/ -path "*$target*.pdf"
Plus several variations around those examples including putting the -path argument string in a variable -- All so far manage to split the target using the space or accept the argument and return nothing. When used on the command-line, about 3 x files are returned.
To be clear though, I'm not concerned with the space inside $target because that side seems to work OK. Ultimately the $target will be a script parameter anyway.
-- update 2015-12-22 --
It appears that the Bash version is important. This version is:
$ bash -version
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
A friend ran more or less the same options as above and they work. When I ran his script on my PC, same result, no output.
To answer the questions:
Everything fails the either by not 'finding' something; or by giving an error about the string "Devel" because of mis-balance quotes we presume.
I used set -x and the results have found an answer that work on this bash (see posted answer).
I'll just post my version as an answer; perhaps someone will have a better method or at least an explanation that lets Bash off the hook.
The concern is that the asterisks(*) are somehow Not being seen or processed as expected when the same command is used inside a Bash script.
With a little help, we have found a working version for this script. It is quite short so I'll post it here and give an example.
First the script, get-ref:
#!/bin/bash
#
# file: get-ref
# usage: get-ref <pattern>
#
# <pattern> ...... target pattern to find
# _____________________________________
#
target=$1
searchPath="$HOME/ref/"
#
# _____________________________________
#
#
find $searchPath -path "*$target*.pdf"
#
echo " -------------------------------"
echo
exit
(Debugging code omitted). This script looks like pattern #3 from the question. Which may mean that I had the answer once but it didn't work for some other reason (I hate it when that happens).
This method worked with my friend's Bhodi Linux distro although <shrug> his set -x didn't show the single-quoting(') below. However ...
The way -path is handled was revealed when we used, set -x which showed the find command looks like this to Bash:
+ find /home/william/_ref/ -path '"*Java Dev*.pdf"'
It seems the script (and not, the command terminal) injects single-quotes(') around the -path argument. This was the missing factor.
praise the set -x built-in
Output example:
$ get-ref "Java Dev"
+ find /home/william/_ref/ -path '*Java Dev*.pdf'
/home/will/ref/technique/games/The Well-Grounded Java Developer.pdf
/home/will/ref/technique/bi/JasperReports 3.5 for Java Developers.pdf
/home/will/ref/lang/ada/Ada for the Cpp or Java Developer.pdf
Finale ...
I'm happy that I can (now) find the document or eBook I want when I need to. Thank you for the comments they were definitely on-track.

Cross-platform command line script (e.g. .bat and .sh)

I noticed that Windows 7 enables to execute .sh files as if they were .bat files. That got me wondering whether it is possible to write a .sh file such that it can be executed in Windows and Linux (let's say bash).
The first thing that comes to my mind is to fabricate an if-statement such that Windows and Ubuntu can deal with it and jump into the according block to execute plattform-specific commands. How could this be done?
Note: I know this is not good practice. I also know that scripting languages like Python are far better suited to solve this problem than a mixed-syntax command line script would be. I'm just curious...
You could use this:
rem(){ :;};rem '
#goto b
';echo sh;exit
:b
#echo batch
It's valid shell script and batch, and will execute different blocks depending on how it's run.
Modify the echo and #echo lines to do what you want.
AFAIK, you can't directly run .sh files from Windows' cmd.exe. For that you'll need a *nix emulation layer to run the shell. Check out Cygwin or Msys/MinGW

How can I call winrar in perl on windows

Is it possible to call winrar through perl on a windows system, such as
perl -e "rar a -rr10 -s c:\backups\backup.rar #backup.lst"
If so, is there a more efficient way to do this?
I've looked up "perl -e" +winrar on google, however none of the results gave me any answer that was remotely close to what i was looking for. The system Im running this on is a Windows XP system. Im open to doing this in another language like python if its easier, however I am more comfertable with perl.
You can access the RAR facilities in Windows using the CPAN module Archive::Rar:
use Archive::Rar;
my $rar = Archive::Rar->new(-archive => $archive_filename);
$rar->Extract();
One way to execute external commands from a Perl script is to use system:
my $cmd = 'rar a -rr10 -s c:\backups\backup.rar #backup.lst';
if (system $cmd) {
print "Error: $? for command $cmd"
}
To use external applications from your Perl program, use the system builtin.
If you need the output from the command, you can use the backtick (``) or qx operator as discussed in perlop. You can also use pipes as discussed in perlipc.

Preprocessor to add functionality to Windows's CMD?

I need to do a fair bit of scripting in my job as a SQL Server DBA. Sometimes, I need to deploy a fix script to a very restricted environment, where the only option for scripting may be DOS Batch. In one such environment, even VBScript/WSH isn't a possibility, let alone PowerShell. Anyone who has written enough batch files on DOS and Windows knows that it's very limited and a huge PIA when you need to do anything too complicated. This is especially true for folks who have worked with Unix shell scripting, Perl, Tcl, Python, Ruby, etc.
A possible solution to this would be a CMD preprocessor that would add some of the useful functionality from more capable scripting languages. I've tried to find such a utility, but so far I've had no luck.
Which finally leads to my question: is anyone aware of a such a CMD preprocessor? If not, what functionality would you like to see in one?
Addendum:
If you're unfamiliar with the idea of a preprocessor see this Wikipedia entry.
To clarify, I'm thinking of a tool that would add features like:
Functions
Backtick (`) ala Unix shell
...and possibly others. Those are two features I've wished CMD had and can think of a way to implement them with a CMD preprocessor. Functions could be implemented with env vars and GOTO/labels; backticks by piping to a temp file and using set /p =< to read in the result to an env var.
You can already achieve these same ends, but it gets to be very tedious and verbose- which is how I came to the idea of having a preprocessor handle the boilerplate for features like those.
Example
Using the example of backticks, here is an example of unprocessed code from my hypothetical Batch++ and processed vanilla batch script, ready to be run by CMD.exe:
Batch++ Source (test.batpp)
copy `dir /b /s c:\ | find "CADR-README.htm"` \\srv01\users
Run it through the preprocessor
bpp test.batpp > post_test.bat
Resulting CMD/BAT code (post_test.bat)
dir /b /s c:\ | find "CADR-README.htm" > _bt001.tmp
set /p _BT001 =< _bt001.tmp
copy %_BT001% \\srv01\users
set _BT001=
del _bt001.tmp
I am not sure to interpret correctly your question. If you run in a controlled environment that doesn't allow you to run any scripting extension, how are you going to access such a preprocessor?
However, and in regard of the two features you request, you are OK with .BATs. Both features are supported by BAT processing in current Windows versions.
Functions: you have the extended CALL syntax, that supports parameter passing thru argument references %1 .. %9, and enhanced with expansion substitution using %~ syntax. Read HELP CALL.
Backtick: not sure what you want but, in the FOR /F command you may pass a backticked string to be run and its output captured. Read HELP FOR.

Resources