SETUP:
A large repository is cloned via 'git' where line endings are adapted automatically. That is, a bash script checked in under Linux appears under Windows as a file with endings 0D0A instead of only 0A. The policy of checkout cannot be modified.
Any script checked out under Windows appears with 0D0A as line endings. Scripts 'source' other scripts, so converting one script does not suffice. Converting all scripts in the repository is not practical.
PROBLEM:
Window's bash does not run on scripts where the line ending is 0D0A. First idea was bash-0d0a converting before execution:
tmp=$(mktemp)
script=$1
shift
awk 1 RS='\r\n' ORS='\n' $script > $tmp
mv $tmp $script
source $script $#
The adapter 'bash-0d0a', however, does not treat 'source-ed' files from within the file to be treated.
QUESTION:
Would it make sense to do:
export PATH=$(dirname path-to-bash-0d0a):$PATH
mv bash-0d0a bash
Or, is it possible, somehow, to let bash do this directly?
Related
I want to execute bash scripts that happen to use Windows/CRLF line endings.
I know of the tofrodos package, and how to fromdos files, but if possible, I'd like to run them without any modification.
Is there an environment variable that will force bash to handle CRLF?
Perhaps like this?
dos2unix < script.sh|bash -s
EDIT: As pointed out in the comments this is the better option, since it allows the script to read from stdin by running dos2unix and not bash in a subshell:
bash <(dos2unix < script.sh)
Here's a transparent workaround for you:
cat > $'/bin/bash\r' << "EOF"
#!/bin/bash
script=$1
shift
exec bash <(tr -d '\r' < "$script") "$#"
EOF
This gets rid of the problem once and for all by allowing you to execute all your system's Windows CRLF scripts as if they used UNIX eol (with ./yourscript), rather than having to specify it for each particular invocation. (beware though: bash yourscript or source yourscript will still fail).
It works because DOS style files, from a UNIX point of view, specify the interpretter as "/bin/bash^M". We override that file to strip the carriage returns from the script and run actual bash on the result.
You can do the same for different interpretters like /bin/sh if you want.
I get the below error while trying to execute a shell script,
$'\r': command not found: line 2:
Please suggest a solution for the same.
Below are the intial lines used in the script,
#!/bin/sh
if [[ $# -lt 1 ]]; then
echo "ERROR Environment argument missing <dev,test,qa,prod>"
export RC=50
exit $RC
fi
Your problem is that the file has Windows line endings. This can be caused by editing a file in Windows and trying to run it on a non-Windows system.
You can fix this problem using dos2unix to convert the line endings:
dos2unix ConstruedTermsXMLGenerator.sh
The corresponding utility to convert in the other direction is unix2dos.
Some systems have fromdos and todos.
You can use sed -i 's/\r$//' scriptname.sh
Replace the scriptname with actual script name.
I used notepad++ to convert the line endings.
Edit > EOL Conversion > UNIX/OSX Format
I had the same error and what I did was to transform the characters '\r' to '\n'. using this line:
tr '\r' '\n' < oldfile.sh > newfile.sh
mv newfile.sh oldfile.sh
chmod +x oldfile.sh
./oldfile.sh
I think you could also delete the '\r' characters by using:
tr -d '\r' < oldfile.sh > newfile.sh
tr is the command trasnform, and the -d is delete the following character.
I think the shell actually doesn't like '\r' character.
I had this exact issue when creating a .sh file on a Mac (unix) and executing it in Linux.
Turns out that I had to set FileZilla FTP settings to 'Binary' transfer type:
"Settings>Transfers>File Types>Default transfer type" to "Binary" (instead of "Auto")
I got a different error message when running your script under /bin/sh, but when I switched to /bin/bash, it worked fine:
$ cat foo.sh
#!/bin/sh
if [[ $# -lt 1 ]];
then echo "ERROR Environment argument missing"
RC=50
exit $RC
fi
$ sh foo.sh
foo.sh: 6: [[: not found
$ bash foo.sh
ERROR Environment argument missing
You've built in a bashism. This may or may not be a big deal for your organization. If you want to keep using bash-specific features, change the shebang line to #!/bin/bash and see if that helps.
You can just replace '\r' with '' (nothing), this will do the trick.
If you have any GUI editor (Notepad++) recommended directly find/replace.
If you are only doing it on Linux server then use something like:
sed -i 's/old-text/new-text/g' input.txt
But Since '\r' comes only if you have edited it in windows as it's a new line character (\r\n), so first method should work.
I have a shell script that works on Ubuntu and provides me an output as I desire. When I test the same on a slackware linux version, my script fails.
The script fails at:
dialog --title "Test" --gauge "Copying file." 6 100 < <(
rsync -a --progress test.tar.gz /media/sda1 |
unbuffer -p grep -o "[0-9]*%" |
unbuffer -p cut -f1 -d '%'
)
The error is:
Syntax error near unexpected token `<'
What could be different between the two operating systems that the script fails to execute?
The script executes successfully if I get rid of the dialog command and the brackets etc.
Most likely, you are trying to run a bash script with non-bash shell. Or with older bash version.
First, try running it through bash explicitly, i.e.:
bash script.sh
You should also fix your shebang to point at bash:
#!/bin/bash
[Update below]
The < <( ... ) notation is unique for bash and zsh. The syntax error is a clear sign it is not recognised by the slackware shell.
Either slackware does not use bash, or its version of bash is too old for this feature.
Check the value of $BASH_VERSION on both platforms.
A possible alternative for
cat < <(
...
...
)
could be:
cat <<< "$(
...
...
)"
This will work in bash, ksh93, and zsh, and has been around slightly longer.
UPDATE
Based on your feedback, I've looked at the actual pipeline you try to use here.
I believe it's your intention to use column 3 of the --progress output as input for the dialog graphical progress indicator.
I tried this with a directory with lots of small files. Are you aware that this percentage indicator is per file? With my small files, rsync gave only one update per file. As every single file was written in one go, all percentages were equal to 100%.
I wrote a simple bash script because I was using a grep command with the same arguments, repeatedly. I'm running it from ~/bin and it runs just fine.
My problem is: All the coloring is gone when it's run through my bash script. The exact same command put right into the commandline nicely color codes line numbers, filenames, etc.
Here's my bash script
#!/bin/bash
# grep php files inside of myfolder, recursively and with line numbers
grep -rn --include="*.php" "$2" /home/me/myfolder/$1
You've probably defined grep as an alias for grep --color=auto in your .bashrc, but that's not loaded by scripts. Use an explicit grep --color in your script.
When you run a script, a new shell is spawned to do so. This new environment doesn't have the same settings as your default shell. As to how to get the coloring back, I'm not sure. You might try sourcing your profile at the start of the script:
#!/bin/bash
source $HOME/.bash_profile
or whichever file makes sense on your particular unix flavor (.profile, .bash_rc, .bashrc .bash_profile) to name a few.
I get the below error while trying to execute a shell script,
$'\r': command not found: line 2:
Please suggest a solution for the same.
Below are the intial lines used in the script,
#!/bin/sh
if [[ $# -lt 1 ]]; then
echo "ERROR Environment argument missing <dev,test,qa,prod>"
export RC=50
exit $RC
fi
Your problem is that the file has Windows line endings. This can be caused by editing a file in Windows and trying to run it on a non-Windows system.
You can fix this problem using dos2unix to convert the line endings:
dos2unix ConstruedTermsXMLGenerator.sh
The corresponding utility to convert in the other direction is unix2dos.
Some systems have fromdos and todos.
You can use sed -i 's/\r$//' scriptname.sh
Replace the scriptname with actual script name.
I used notepad++ to convert the line endings.
Edit > EOL Conversion > UNIX/OSX Format
I had the same error and what I did was to transform the characters '\r' to '\n'. using this line:
tr '\r' '\n' < oldfile.sh > newfile.sh
mv newfile.sh oldfile.sh
chmod +x oldfile.sh
./oldfile.sh
I think you could also delete the '\r' characters by using:
tr -d '\r' < oldfile.sh > newfile.sh
tr is the command trasnform, and the -d is delete the following character.
I think the shell actually doesn't like '\r' character.
I had this exact issue when creating a .sh file on a Mac (unix) and executing it in Linux.
Turns out that I had to set FileZilla FTP settings to 'Binary' transfer type:
"Settings>Transfers>File Types>Default transfer type" to "Binary" (instead of "Auto")
I got a different error message when running your script under /bin/sh, but when I switched to /bin/bash, it worked fine:
$ cat foo.sh
#!/bin/sh
if [[ $# -lt 1 ]];
then echo "ERROR Environment argument missing"
RC=50
exit $RC
fi
$ sh foo.sh
foo.sh: 6: [[: not found
$ bash foo.sh
ERROR Environment argument missing
You've built in a bashism. This may or may not be a big deal for your organization. If you want to keep using bash-specific features, change the shebang line to #!/bin/bash and see if that helps.
You can just replace '\r' with '' (nothing), this will do the trick.
If you have any GUI editor (Notepad++) recommended directly find/replace.
If you are only doing it on Linux server then use something like:
sed -i 's/old-text/new-text/g' input.txt
But Since '\r' comes only if you have edited it in windows as it's a new line character (\r\n), so first method should work.