Similar question to How to keep from duplicating path variable in csh and How to keep from duplicating path variable in ksh. But this time using Windows command line (cmd.exe).
PATH=C:\path\to\bin;%PATH%
How do I remove duplicates from PATH? It would like to do this in an automated way: as part of our build process a specific path is added to the environment variable PATH; when done 20+ times that path is present 20+ times. I want to avoid that.
My solution is similar to https://stackoverflow.com/a/586748/911550:
echo %PATH% | tr ; \n | awk "!($0 in a) { a[$0]; print }" | paste -sd; - > TEMPORARY.FILE
set /P PATH=< TEMPORARY.FILE
del TEMPORARY.FILE
tr, awk and paste can be downloaded from http://gnuwin32.sourceforge.net. (Actually, the first I do on any new Windows workstation is using the Automated gnuwin32 download tool)
Related
Note: I am particularly looking for a coding hack, not for an alternative solution. I am aware that awk, sed etc. can do this inline edit just fine.
$ echo '1' > test
$ cat test > test
$ cat test
$
Is there a way, to somehow make the second command output the original contents (1 in this case)? Again, I am looking for a hack which will work without visibly having to use a secondary file (using a secondary file in the background is fine). Another question on this forum solely focused on alternative solutions which is not what I am looking for.
You can store the content in a shell variable rather than a file.
var=$(<test)
printf "%s\n" "$var" > test
Note that this might only work for text files, not binary files. If you need to deal with them you can use encoding/decoding commands in the pipeline.
You can't do it without storing the data somewhere. When you redirect output to a file, the shell truncates the file immediately. If you use a pipeline, the commands in the pipeline run concurrently with the shell, and it's unpredictable which will run first -- the shell truncating the file or the command that tries to read from it.
With thanks to the comment made by #Cyrus to the original question
$ sudo apt install moreutils
$ echo '1' > test
$ cat test | sponge test
$ cat test
1
It does require installing an extra package and pre-checking for the binary using something like where sponge to check if it is installed.
if you happen to use macos, if the file isn't too gargantuan, you can always follow these steps :
perform the edits
pipe it to the clipboard (or "pasteboard" in mac lingo)
paste it back to original file name
|
{... edits to file1 ...} | pbcopy; pbpaste > file1
Context:
The problem comes from the work folder location: if I should work in a subfolder of a subfolder of a subfolder et cetera... The command line of the shell bask in Linux is so long that can use two lines to be correctly printed.
Question:
Is there a way of showing only the last (or the few last) working subfolder?
Example:
What is actually printed:
user#user-pc:~/Documents/robotic_arm/Monitoring/difference/develop/component/example/subfolder/subexample/module$
What I would love to see:
user#user-pc:~/.../subexample/module$
More Info:
Xubuntu 16.04
Terminator is used instead of the default "Linux Terminal Emulator"
I had a look on this Stackoverflow's question but it is for the input and not for the default line printed by the shell
Have a look at the PROMPTING section of man bash. You configure the prompt by setting PS1, and I suspect your current setting is something like this:
$ echo $PS1
\u#\h:\w\$
If you change it to
$ PS1='\u#\h:\W\$ '
it will only print the basename of the current working directory.
I do not know why but the previous answer does not work on my machine. However, an alternative solution that works fine is:
PROMPT_DIRTRIM=N
where N is the number of subfolders you want to see.
Example:
user#user-pc:~/Documents/robotic_arm/difference/develop/component/ $ PROMPT_DIRTRIM=2
user#user-pc:~/.../develop/component/ $
The solution was suggested by one of the answers above this question.
I would like to run a Bash shell script (.sh) using the Windows Subsystem for Linux as part of a Build Event in Visual Studio, in a C++ project.
But there are lots of errors when I do this, and I can't seem to find the right combination of quotation marks and apostrophes and backslashes to either make Bash run in the first place, or to properly pass the path to the script.
How do I make Visual Studio run my Bash shell script as a build event?
(Feel free to skip to the bottom of this answer if you don't care about how to solve the problem and just want a command you can copy and paste!)
Overview
I run a number of Bash shell scripts as part of Build events in Visual Studio, and I used to use Cygwin to run them. Now that the Windows Subsystem for Linux is available, I spent some time switching my builds over to use WSL, and it wasn't as easy as I'd hoped, but it can work, with a little time and energy.
There are several issues you'll run into if you're going to do this:
The path to bash.exe may not be what you think it is, because under the hood, Visual Studio uses a 32-bit build process, so if you're on a 64-bit machine, you can't simply run the 64-bit bash.exe without getting the dreaded 'C:\Windows\System32\bash.exe' is not recognized error.
The path to your solution or project is a Windows path that uses backslashes (\), and those don't play nice in Unix, which prefers forward slashes (/) as a path delimiter.
The root drive of the solution, typically something like C:\, is meaningless gibberish in Unix; to reach the root drive in WSL, you'll need to use a mounted drive under /mnt.
The casing of the drive letter is different between Windows and WSL: In Windows, it's uppercase C:\, and in WSL, it's lowercase /mnt/c.
And to make it a little harder, we don't want to hard-code any of the paths: It should all Just Work, no matter where the solution is found.
The good news is that they're all solvable issues! Let's tackle them one at a time.
Fixing the Issues
1. The proper path to Bash
Per the answer given here, you'll need to use a magic path to reach Bash when running it from a Visual Studio build. The correct path is not C:\Windows\System32\bash.exe, but is actually
%windir%\sysnative\bash.exe
The magic sysnative folder avoids the invisible filesystem redirection performed by the WOW64 layer, and points to the native bash.exe file.
2. Fixing the backslashes
The next problem you're likely to run into is the backslashes. Ideally, you'd like to run a project script like $(ProjectDir)myscript.sh, but that expands to something like C:\Code\MySolution\MyProject\myscript.sh. At a minimum, you'd like that to be at least C:/Code/MySolution/MyProject/myscript.sh, which isn't exactly right, but which is a lot closer to correct.
So sed to the rescue! sed is a Unix tool that mutates text in files: It searches for text using regular expressions, and, among other things, can replace that text with a modified version. Here, we're going to pipe the path we have into sed, and then use some regex magic to swap the path separators, like this (with lines wrapped here for readability):
%windir%\sysnative\bash.exe -c "echo '$(ProjectDir)myscript.sh'
| sed -e 's/\\\\/\//g;'"
If you include this as your build event, you'll see that it now doesn't run the script, but it at least prints something like C:/Code/MySolution/MyProject/myscript.sh to the output console, which is a step in the right direction.
And yes, that's a lot of backslashes and quotes and apostrophes to get the escaping right, because Nmake.exe and bash and sed are all going to consume some of those special symbols while processing their respective command-lines.
3. Fixing the C:\ root path
We want to mutate the sed script so that it turns the C:\ into /mnt/C. A little more regex substitution magic can do that. (And we have to turn on the -r flag in sed so that we can easily use capture groups.)
%windir%\sysnative\bash.exe -c "echo '$(ProjectDir)myscript.sh'
| sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\1/i;'"
If you run this, you'll now see the output path as something like /mnt/C/Code/MySolution/MyProject/myscript.sh, which is almost but not quite correct.
4. Fixing the case-change in the root path
WSL mounts your disks in lowercase, and Windows mounts them in uppercase. Consistency! How do we fix this? Yet more sed magic!
The \L command can be used to tell sed to transform succeeding characters to lowercase (and there's an equivalent \U for uppercase). The \E command will switch output back to "normal" mode, where characters are left untouched.
Adding these in finally results in the correct path being output:
%windir%\sysnative\bash.exe -c "echo '$(ProjectDir)myscript.sh'
| sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\L\1\E/i;'"
5. Running it
This whole time, Bash has just been printing out the path to the script. How do we run it instead, now that it's the correct path?
The answer is to add `backticks`! Backticks cause Bash to execute the command contained within them, and to then use that command's output as the arguments to the next command. In this case, we're not going to output anything: We just want to run the output of sed as a command.
So including the backticks, here's the result:
%windir%\sysnative\bash.exe -c "`echo '$(ProjectDir)myscript.sh'
| sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\L\1\E/i;'`"
The Complete Solution
Here's what the whole solution looks like, for running a script named myscript.sh as a Build Event, in the current Project directory of the current Solution:
%windir%\sysnative\bash.exe -c "`echo '$(ProjectDir)myscript.sh' | sed -re 's/\\\\/\//g; s/([A-Z]):/\/mnt\/\L\1\E/i;'`"
Here's a screen-shot showing the same thing in Visual Studio 2017, for a real C++ project:
It's not easy, but it's not impossible.
If you have Git for Windows installed, try this. I found it simpler than installing WSL. The basic idea is to create an intermediate batch script to call your bash script, using Git bash's in-built bash or sh command from the batch script.
With Git for Windows, you'll have a Git\bin folder e.g. at:
C:\Program Files\Git\bin
Inside that directory you should see the bash.exe and sh.exe programs. So if you add that directory to your Windows Path environment variable then you'll be able to use sh and bash from the Windows command line. These commands will allow you to run your bash scripts "inline" within a CMD console window. That is, they won't spawn a new bash window; meaning the console output will be visible in your VS build.
From there, just create a .bat file which calls your .sh file using either the sh command or the bash command. Not sure the difference; we just use the sh command. So if your bash script is pre.sh, then your batch file would be just a single line calling the bash script:
sh %~dp0\pre.sh
if errorlevel 1 (
exit /b %errorlevel%
)
The %~dp0 assumes the batch and bash scripts are in the same directory. You then point your VS build event to the .bat file. The check for error level is necessary so that any failures from the bash script are forwarded up to the batch script. See: How do I get the application exit code from a Windows command line?.
To hook this in as a build event in VS2019 then, just follow the standard instructions for hooking in a .bat file: https://learn.microsoft.com/en-us/visualstudio/ide/specifying-custom-build-events-in-visual-studio?view=vs-2019.
Update: Beware Visual Studio's (VS's) Path Variable Behaviour
One thing we found quite frustrating with this solution was the tendency of VS to not load in the path variable correctly. It seems to prefer the user variable over the system variable. But even after we deleted the user variable, sometimes the path didn't seem to be getting picked up by VS, and we kept getting "sh is not recognised..." messages on our build console. Whenever that happened, restarting VS seemed to do the trick. Not very satisfying, but it gets us by.
Update: This is not a Full Unix Solution
Git for Windows does have a lot of Unix commands available, but not all of them. So in general, this won't work. For the general case, WSL is more robust. However, if it's just pretty lightweight Unix you need, this will suffice, and will likely be an easier approach for Windows users who would rather avoid the steeper setup cost of installing the full WSL.
Original idea to use Git bash came from here: https://superuser.com/questions/1218943/windows-command-prompt-capture-output-of-bash-script-in-one-step
Instead of backticks, you can wrap command with $( and )
I'm trying to write a quick batch file. It will take the result of a command, put some extra text and quotes around it, and put that into a new file.The problem is that the result of the command I'm running includes a new line. Here's the command:
p4 changelists -m 1 -t //depot/...> %FILENAME%
The output of that p4 command has a newline at the end of it. The file I'm putting it into needs to have quotes surrounding the output of that command, but the fact that the command contains a newline in it means that the "closing quote" appears on a new line in the file, which doesn't work for what I'm doing.
I've tried writing the output of that command into a file and reading it back in, and also trying to run FINDSTR on a file containing the output, but I always seem to get back the stupid trailing whitespace. I've even tried inserting backspaces into the file, but that just put a backspace character into the file instead of actually executing a backspace...
Is there anything to be done about this?
I'm no perl wizard, but the following seems to work:
p4 changelists -m 1 -t //depot/...| perl -p -e "s/^/\042/;s/$/\042/"
Check out Strawberry Perl, which provides a Windows version of Perl.
I'm always looking at my Unix tools when solving problems like this, even under Windows. sed and gawk will also get you there, check out msysgit for a nice bundle of Unix tools that will run on Windows.
Given this (among more...):
compile_coffee() {
echo "Compile COFFEESCRIPT files..."
i=0
for folder in ${COFFEE_FOLDER[*]}
do
for file in $folder/*.coffee
do
file_name=$(echo "$file" | awk -F "/" '{print $NF}' | awk -F "." '{print $1}')
file_destination_path=${COFFEE_DESTINATION_FOLDER[${i}]}
file_destination="$file_destination_path/$file_name.js"
if [ -f $file_path ]; then
echo "+ $file -> $file_destination"
$COFFEE_CMD $COFFEE_PARAMS $file > $file_destination #FAIL
#$COFFEE_CMD $COFFEE_PARAMS $file > testfile
fi
done
i=$i+1
done
echo "done!"
compress_javascript
}
And just to clarify, everything except the #FAIL line works flawessly, if I'm doing something wrong just tell me, the problem I have is:
the line executes and does what it have to do, but dont write the file that I put in "file_destination".
if a delete a folder in that route (it's relative to this script, see below), bash throws and error saying that the folder do not exist.
If I make the folder again, no errors, but no file either.
If I change the $file_destination to "testfile", it create the file with correct contents.
The $file_destination path its ok -as you can see, my script echoes it-
if I echo the entire line, copy the exact command with params and execute it onto a shell in the same directory the script is, it
works.
I don't know what is wrong with this, been wondering for two hours...
Script output (real paths):
(alpha)[pyron#vps herobrine]$ ./deploy.sh compile && ls -l database/static/js/
===============================
=== Compile ===
Compile COFFEESCRIPT files...
+ ./database/static/coffee/test.coffee -> ./database/static/js/test.js
done!
Linking static files to django staticfiles folder... done!
total 0
Complete command:
coffee --compile --print ./database/static/coffee/test.coffee > ./database/static/js/test.js
What am I missing?
EDIT I've made some progression through this.
In the shell, If I deactivate the python virtualenv the script works, but If I call deactivate from the script it says command not found.
Assuming destination files have no characters as spaces in their names, directories exist etc. I'd try adding 2>&1 e.g.
$COFFEE_CMD $COFFEE_PARAMS $file > testfile 2>&1
compilers may put desired output and/or compilation messages on stderr instead of stdout. You may also want to put full path to coffee , e.g. /usr/bin/coffee instead of just compiler name.
Found that the problem wasn't the bash script itself. A few lines later the deploy script perform the collectstatic method from django. Noticed that until that line the files were there, I started reading that the collecstatic have a cache system. A very weird one IMO, since I have to delete all the static files and start from scratch to have the script working.
So... the problem wasn't the bash script but the django cache system. Im not givin' reputation to me anyways.
The full deploy script is here: https://github.com/pyronhell/deploy-script-boilerplate and everyone is welcome if you can improve it.
Cheers.