Bash - seamlessly run scripts with CRLF line endings - windows

I am using VM (in my case simply boot2docker) to run docker containers on Windows host. For convinience, my source files are mapped from host file system, so text files are by default using Windows-style CRLF line endings instead of Unix-style LF endings.
When I try to run some .sh file from docker container, I'll get an error
bash: ./script.sh: /bin/bash^M: bad interpreter: No such file or directory
Is there a way how could I somehow tell bash/sh interpreter to automatically convert \r\n to \n and run a file?
Sure, I could do some piplelining like this cat script.sh | tr -d "\r" | sh or even create an alias for that, but it would not cover situation where one script includes another.
The only acceptable solution I have found so far, is to set Git to checkout source files in UNIX format.

You can commit a .gitattributes file in your repo in the root folder to automatically tell Git to apply LF line endings to all .sh files when checking out, even on Windows. This way you or other developers don't have to remember to configure Git on each machine where you clone your repo.
# Set line endings to LF, even on Windows. Otherwise, execution within Docker fails.
# See https://help.github.com/articles/dealing-with-line-endings/
*.sh text eol=lf

For some scripts the solution ln -s /bin/bash /bin/bash^M will fail.
So create a script /bin/bash^M that will dos2unix the arguments they get and run them with "$#".
Edit:
I tried it with the following bash^M script:
#!/bin/bash
PROG=$1
shift
dos2unix $PROG > /dev/null 2>&1
$PROG "$#"
This will fail when your script includes other scripts with a dot, in that case you should avoid this work-around and go for the Unix format.
You said you can checkout in Unix format, or you can self change the files.
And commit into git, it is an improvement to have unix files stored in git in unix format. When you really want to edit them under windows, change the settings of your editor or dos2unix afterwards.

I imagine you could write a script such as
#!/bin/bash
#bash_run_unixified
#unixify and run in bash
dos2unix < "$1" | /bin/bash
and assuming you saved it in /urs/local/bin/bash_run_unixified with appropriate permissions (sudo chmod o+r,o+x), then you could prefix your scripts with
#!/usr/local/bin/bash_run_unixified
#This script has CRLFs in it
If you don't want such unusual shebang lines, you could
$ ln -s /usr/local/bin/{bash_run_unixified,bash}
and then use
(As Walter A. notes, it might be a good idea to link it to bash^M too)
#!/usr/bin/env bash
as your shebang line (/usr/local/bin/ is normally closer to the beginning of your $PATH than /bin so the env command should select the bash that links to bash_run_unixified).
(Word of caution: I haven't tested any of this)

The best solution I have is to make sure to use editors (like Notepad++ or Sublime Text) able to save directly with the proper eol style.
That way, I don't have to dos2unix any file in my boot2docker session.

Visual Studio code is also good at changing the line ending to linux or windows file endings

Related

g++ reutrns error, while shell script looks fine [duplicate]

This question already has answers here:
running bash script in cygwin on windows 7 [duplicate]
(2 answers)
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
I have windows, using Cygwin, trying to set JAVA_HOME permanently through my .bashrc file.
.bashrc:
export PATH="$JAVA_HOME/bin:$PATH"
export JAVA_HOME=$JAVA_HOME:"/cygdrive/c/Program Files (x86)/Java/jdk1.7.0_05"
.bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
running cygwin:
-bash: $'\377\376if': command not found
-bash: $'then\r': command not found
: No such file or directorysu//.bashrc
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: syntax error near unexpected token `fi'
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: `fi'
I am not sure if I took the commands from a tutorial that was meant for another system or if I am missing a step. Or whitespace is causing my commands not to run properly.
I've looked at multiple similar questions but I haven't found one where the question has my error exactly.
My home path:
$ echo $HOME
/cygdrive/c/Users/jhsu
$ echo ~
/cygdrive/c/Users/jhsu/
So I believe the files should be placed in the correct spot.
When all else fails in Cygwin...
Try running the dos2unix command on the file in question.
It might help when you see error messages like this:
-bash: '\r': command not found
Windows style newline characters can cause issues in Cygwin.
The dos2unix command modifies newline characters so they are Unix / Cygwin compatible.
CAUTION: the dos2unix command modifies files in place, so take precaution if necessary.
If you need to keep the original file, you should back it up first.
Note for Mac users: The dos2unix command does not exist on Mac OS X.
Check out this answer for a variety of solutions using different tools.
There is also a unix2dos command that does the reverse:
It modifies Unix newline characters so they're compatible with Windows tools.
If you open a file with Notepad and all the lines run together, try unix2dos filename.
For those who don't have dos2unix installed (and don't want to install it):
Remove trailing \r character that causes this error:
sed -i 's/\r$//' filename
Explanation:
Option -i is for in-place editing, we delete the trailing \r directly in the input file. Thus be careful to type the pattern correctly.
For WINDOWS (shell) users with Notepad++ (checked with v6.8.3) you can correct the specific file using the option
Edit
-> EOL conversion
-> Unix/OSX format
And save your file again.
Edit: still works in v7.5.1 (Aug 29 2017)
Edit: Jan 3, 2022. As VSCode is mentioned several times. Go to settings in VSCode and type files.eol in the search field and set to \n (Unix format). Note that this changes this setting for your user or workspace for all files and it may not be what you want. YMMV.
I am using cygwin and Windows7, the trick was NOT to put the set -o igncr into your .bashrc but put the whole SHELLOPTS into you environment variables under Windows. (So nothing with unix / cygwin...) I think it does not work from .bashrc because "the drops is already sucked"
as we would say in german. ;-)
So my SHELLOPTS looks like this
braceexpand:emacs:hashall:histexpand:history:igncr:interactive-comments:monitor
SUBLIME TEXT
With sublime you just go to
View - > Line Endings -> (select)Unix
Then save the file. Will fix this issue.
Easy as that!
If you are using a recent Cygwin (e.g. 1.7), you can also start both your .bashrc and .bash_profile with the following line, on the first non commented line:
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is needed
This will force bash to ignore carriage return (\r) characters used in Windows line separators.
See http://cygwin.com/ml/cygwin-announce/2010-08/msg00015.html.
The error:
'\r': command not found
is caused by shell not able to recognise Windows-like CRLF line endings (0d 0a) as it expects only LF (0a).
Git
If you using Git on Windows, make sure you selected 'Checkout as-is' during setup. Then make sure that you run: git config --global core.autocrlf false, so Git will not perform any conversions when checking out or committing text files.
dos2unix
If you're not using Git, you simply need to convert these affected files/scripts back into Unix-like line endings (LF), either by:
dos2unix ~/.bashrc
Note: The dos2unix command is part of dos2unix package.
Ex/Vim editor + tr
If you've Vim installed, the following command should correct the files:
ex +'bufdo! %! tr -d \\r' -scxa ~/.bash*
Useful alias: alias dos2unix="ex +'bufdo! %! tr -d \\\\r' -scxa".
tr
Here is the method by using tr:
cat ~/.bashrc | tr -d '\r' > ~/.bashrc.fixed && mv -v ~/.bashrc.fixed ~/.bashrc
or:
tr -d '\r' < filename > new_filename
Note: The \r is equivalent to \015.
sed
You can try the following command:
sed -i'.bak' s/\r//g ~/.bash*
recode
The following aliases can be useful (which replaces dos2unix command):
alias unix2dos='recode lat1:ibmpc'
alias dos2unix='recode ibmpc:lat1'
Source: Free Unix Tools (ssh, bash, etc) under Windows.
perl
The following perl command can convert the file from DOS into Unix format:
perl -p -i.bak -e 's/\015//g' ~/.bash*
Source: stripping the ^M.
tofrodos
On Linux, like Ubuntu which doesn’t come standard with either dos2unix or unix2dos, you can install tofrodos package (sudo apt-get install tofrodos), and define the following aliases:
alias dos2unix=’fromdos’
alias unix2dos=’todos’
Then use in the same syntax as above.
Vagrant
If you're using Vagrant VM and this happens for provisioning script, try setting binary option to true:
# Shell provisioner, see: https://www.vagrantup.com/docs/provisioning/shell.html
config.vm.provision "shell" do |s|
s.binary = true # Replace Windows line endings with Unix line endings.
s.path = "script.sh"
end
See: Windows CRLF to Unix LF Issues in Vagrant.
try execution the following command
vim .bashrc
:set ff=unix
:wq!
You can also add the option -o igncr to the bash call, e.g.
bash -x -o igncr script.sh
As per this gist, the solution is to create a ~/.bash_profile (in HOME directory) that contains:
export SHELLOPTS
set -o igncr
May be you used notepad++ for creating/updating this file.
EOL(Edit->EOL Conversion) Conversion by default is Windows.
Change EOL Conversion in Notepad++
Edit -> EOL Conversion -> Unix (LF)
I had the same problem. Solution: I edit the file with pspad editor, and give it a unix format (Menu - Format -> UNIX)
I believe you can set this format to your file with many other editors
For the Emacs users out there:
Open the file
M-x set-buffer-file-coding-system
Select "unix"
This will update the new characters in the file to be unix style. More info on "Newline Representation" in Emacs can be found here:
http://ergoemacs.org/emacs/emacs_line_ending_char.html
Note: The above steps could be made into an Emacs script if one preferred to execute this from the command line.
Issue maybe occured because of the file/script created/downloaded from a windows machine. Please try converting into linux file format.
dos2unix ./script_name.sh
or
dos2unix ~/.bashrc
If you have the vim package installed on your Cygwin install, you can use vim to fix this without find & replace. Start vim as follows: vim filename.sh (often it is aliased to vi also). Then, type :set fileformat=unix, then :wq (write & quit) to save your changes. (The : puts you in vim's edit mode.)
I recommend this over dos2unix since vim is probably more commonly installed.
However, it is probably a best practice to set your text editor to save files that you plan to use in a Unix/Linux environment to have a Unix text format. The answers given above for Notepad++ are a good example.
Additional note: If you are unsure what type a file is (DOS or Unix), you may use the file filename.sh. This can especially help in debugging more obscure issues (such as encoding issues when importing SQL dumps that come from Windows).
For other options on how to modify text file formatting, see this IU knowledge base article
More background information on Bash scripts and line endings is found on this StackOverflow question.
In EditPlus you do this from the
Document → File Format (CR/LF) → Change File Format... menu and then choose the Unix / Mac OS X radio button.
1. Choice
EditorConfig — is my choice.
2. Relevance
This answer is relevant for March 2018. In the future, the data from this answer may be obsolete.
Author of this answer personally used EditorConfig at March 2018.
3. Limitations
You need to use one of supported IDE/editors.
4. Argumentation
Simply usage. I need set my .editorconfig file 1 time, where I create my project, → I can forget some platform-, style- and IDE-specific problems.
Cross-platform, cross-languages, cross-IDE, cross-editors.
5. Example
5.1. Simple
I install EditorConfig plugin for Sublime Text → my text editor. I edit files in Sublime Text.
For example, I have sashacrlf.sh file:
echo "Sasha" &
echo "Goddess!" &
I run this file in Cygwin:
$ bash sashacrlf.sh
Sasha
sashacrlf.sh: line 1: $'\r': command not found
Goddess!
I create a file .editorconfig in same project as sashacrlf.sh.
[*.sh]
end_of_line = lf
It means, that if you save any file with .sh extension in your project in your editor, EditorConfig set UNIX line endings for this file.
I save sashacrlf.sh in my text editor again. I run sashacrlf.sh again:
$ bash sashacrlf.sh
Sasha
Goddess!
I can't get unexpected output in console.
5.2. Multiple file extensions
For example, I want to have UNIX line endings in all files with extensions .sh, .bashrc and .bash_profile.
I add these lines to my .editorconfig file:
[*.{sh,bashrc,bash_profile}]
end_of_line = lf
Now, if I save any file with .sh, .bashrc or .bash_profile extension, EditorConfig automatically set UNIX line ending for this file.
6. Additional links
EditorConfig official site.

Peculiar variations in command results [duplicate]

This question already has answers here:
running bash script in cygwin on windows 7 [duplicate]
(2 answers)
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
I have windows, using Cygwin, trying to set JAVA_HOME permanently through my .bashrc file.
.bashrc:
export PATH="$JAVA_HOME/bin:$PATH"
export JAVA_HOME=$JAVA_HOME:"/cygdrive/c/Program Files (x86)/Java/jdk1.7.0_05"
.bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
running cygwin:
-bash: $'\377\376if': command not found
-bash: $'then\r': command not found
: No such file or directorysu//.bashrc
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: syntax error near unexpected token `fi'
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: `fi'
I am not sure if I took the commands from a tutorial that was meant for another system or if I am missing a step. Or whitespace is causing my commands not to run properly.
I've looked at multiple similar questions but I haven't found one where the question has my error exactly.
My home path:
$ echo $HOME
/cygdrive/c/Users/jhsu
$ echo ~
/cygdrive/c/Users/jhsu/
So I believe the files should be placed in the correct spot.
When all else fails in Cygwin...
Try running the dos2unix command on the file in question.
It might help when you see error messages like this:
-bash: '\r': command not found
Windows style newline characters can cause issues in Cygwin.
The dos2unix command modifies newline characters so they are Unix / Cygwin compatible.
CAUTION: the dos2unix command modifies files in place, so take precaution if necessary.
If you need to keep the original file, you should back it up first.
Note for Mac users: The dos2unix command does not exist on Mac OS X.
Check out this answer for a variety of solutions using different tools.
There is also a unix2dos command that does the reverse:
It modifies Unix newline characters so they're compatible with Windows tools.
If you open a file with Notepad and all the lines run together, try unix2dos filename.
For those who don't have dos2unix installed (and don't want to install it):
Remove trailing \r character that causes this error:
sed -i 's/\r$//' filename
Explanation:
Option -i is for in-place editing, we delete the trailing \r directly in the input file. Thus be careful to type the pattern correctly.
For WINDOWS (shell) users with Notepad++ (checked with v6.8.3) you can correct the specific file using the option
Edit
-> EOL conversion
-> Unix/OSX format
And save your file again.
Edit: still works in v7.5.1 (Aug 29 2017)
Edit: Jan 3, 2022. As VSCode is mentioned several times. Go to settings in VSCode and type files.eol in the search field and set to \n (Unix format). Note that this changes this setting for your user or workspace for all files and it may not be what you want. YMMV.
I am using cygwin and Windows7, the trick was NOT to put the set -o igncr into your .bashrc but put the whole SHELLOPTS into you environment variables under Windows. (So nothing with unix / cygwin...) I think it does not work from .bashrc because "the drops is already sucked"
as we would say in german. ;-)
So my SHELLOPTS looks like this
braceexpand:emacs:hashall:histexpand:history:igncr:interactive-comments:monitor
SUBLIME TEXT
With sublime you just go to
View - > Line Endings -> (select)Unix
Then save the file. Will fix this issue.
Easy as that!
If you are using a recent Cygwin (e.g. 1.7), you can also start both your .bashrc and .bash_profile with the following line, on the first non commented line:
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is needed
This will force bash to ignore carriage return (\r) characters used in Windows line separators.
See http://cygwin.com/ml/cygwin-announce/2010-08/msg00015.html.
The error:
'\r': command not found
is caused by shell not able to recognise Windows-like CRLF line endings (0d 0a) as it expects only LF (0a).
Git
If you using Git on Windows, make sure you selected 'Checkout as-is' during setup. Then make sure that you run: git config --global core.autocrlf false, so Git will not perform any conversions when checking out or committing text files.
dos2unix
If you're not using Git, you simply need to convert these affected files/scripts back into Unix-like line endings (LF), either by:
dos2unix ~/.bashrc
Note: The dos2unix command is part of dos2unix package.
Ex/Vim editor + tr
If you've Vim installed, the following command should correct the files:
ex +'bufdo! %! tr -d \\r' -scxa ~/.bash*
Useful alias: alias dos2unix="ex +'bufdo! %! tr -d \\\\r' -scxa".
tr
Here is the method by using tr:
cat ~/.bashrc | tr -d '\r' > ~/.bashrc.fixed && mv -v ~/.bashrc.fixed ~/.bashrc
or:
tr -d '\r' < filename > new_filename
Note: The \r is equivalent to \015.
sed
You can try the following command:
sed -i'.bak' s/\r//g ~/.bash*
recode
The following aliases can be useful (which replaces dos2unix command):
alias unix2dos='recode lat1:ibmpc'
alias dos2unix='recode ibmpc:lat1'
Source: Free Unix Tools (ssh, bash, etc) under Windows.
perl
The following perl command can convert the file from DOS into Unix format:
perl -p -i.bak -e 's/\015//g' ~/.bash*
Source: stripping the ^M.
tofrodos
On Linux, like Ubuntu which doesn’t come standard with either dos2unix or unix2dos, you can install tofrodos package (sudo apt-get install tofrodos), and define the following aliases:
alias dos2unix=’fromdos’
alias unix2dos=’todos’
Then use in the same syntax as above.
Vagrant
If you're using Vagrant VM and this happens for provisioning script, try setting binary option to true:
# Shell provisioner, see: https://www.vagrantup.com/docs/provisioning/shell.html
config.vm.provision "shell" do |s|
s.binary = true # Replace Windows line endings with Unix line endings.
s.path = "script.sh"
end
See: Windows CRLF to Unix LF Issues in Vagrant.
try execution the following command
vim .bashrc
:set ff=unix
:wq!
You can also add the option -o igncr to the bash call, e.g.
bash -x -o igncr script.sh
As per this gist, the solution is to create a ~/.bash_profile (in HOME directory) that contains:
export SHELLOPTS
set -o igncr
May be you used notepad++ for creating/updating this file.
EOL(Edit->EOL Conversion) Conversion by default is Windows.
Change EOL Conversion in Notepad++
Edit -> EOL Conversion -> Unix (LF)
I had the same problem. Solution: I edit the file with pspad editor, and give it a unix format (Menu - Format -> UNIX)
I believe you can set this format to your file with many other editors
For the Emacs users out there:
Open the file
M-x set-buffer-file-coding-system
Select "unix"
This will update the new characters in the file to be unix style. More info on "Newline Representation" in Emacs can be found here:
http://ergoemacs.org/emacs/emacs_line_ending_char.html
Note: The above steps could be made into an Emacs script if one preferred to execute this from the command line.
Issue maybe occured because of the file/script created/downloaded from a windows machine. Please try converting into linux file format.
dos2unix ./script_name.sh
or
dos2unix ~/.bashrc
If you have the vim package installed on your Cygwin install, you can use vim to fix this without find & replace. Start vim as follows: vim filename.sh (often it is aliased to vi also). Then, type :set fileformat=unix, then :wq (write & quit) to save your changes. (The : puts you in vim's edit mode.)
I recommend this over dos2unix since vim is probably more commonly installed.
However, it is probably a best practice to set your text editor to save files that you plan to use in a Unix/Linux environment to have a Unix text format. The answers given above for Notepad++ are a good example.
Additional note: If you are unsure what type a file is (DOS or Unix), you may use the file filename.sh. This can especially help in debugging more obscure issues (such as encoding issues when importing SQL dumps that come from Windows).
For other options on how to modify text file formatting, see this IU knowledge base article
More background information on Bash scripts and line endings is found on this StackOverflow question.
In EditPlus you do this from the
Document → File Format (CR/LF) → Change File Format... menu and then choose the Unix / Mac OS X radio button.
1. Choice
EditorConfig — is my choice.
2. Relevance
This answer is relevant for March 2018. In the future, the data from this answer may be obsolete.
Author of this answer personally used EditorConfig at March 2018.
3. Limitations
You need to use one of supported IDE/editors.
4. Argumentation
Simply usage. I need set my .editorconfig file 1 time, where I create my project, → I can forget some platform-, style- and IDE-specific problems.
Cross-platform, cross-languages, cross-IDE, cross-editors.
5. Example
5.1. Simple
I install EditorConfig plugin for Sublime Text → my text editor. I edit files in Sublime Text.
For example, I have sashacrlf.sh file:
echo "Sasha" &
echo "Goddess!" &
I run this file in Cygwin:
$ bash sashacrlf.sh
Sasha
sashacrlf.sh: line 1: $'\r': command not found
Goddess!
I create a file .editorconfig in same project as sashacrlf.sh.
[*.sh]
end_of_line = lf
It means, that if you save any file with .sh extension in your project in your editor, EditorConfig set UNIX line endings for this file.
I save sashacrlf.sh in my text editor again. I run sashacrlf.sh again:
$ bash sashacrlf.sh
Sasha
Goddess!
I can't get unexpected output in console.
5.2. Multiple file extensions
For example, I want to have UNIX line endings in all files with extensions .sh, .bashrc and .bash_profile.
I add these lines to my .editorconfig file:
[*.{sh,bashrc,bash_profile}]
end_of_line = lf
Now, if I save any file with .sh, .bashrc or .bash_profile extension, EditorConfig automatically set UNIX line ending for this file.
6. Additional links
EditorConfig official site.

No such file or directory errors [duplicate]

This question already has answers here:
running bash script in cygwin on windows 7 [duplicate]
(2 answers)
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
I have windows, using Cygwin, trying to set JAVA_HOME permanently through my .bashrc file.
.bashrc:
export PATH="$JAVA_HOME/bin:$PATH"
export JAVA_HOME=$JAVA_HOME:"/cygdrive/c/Program Files (x86)/Java/jdk1.7.0_05"
.bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
running cygwin:
-bash: $'\377\376if': command not found
-bash: $'then\r': command not found
: No such file or directorysu//.bashrc
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: syntax error near unexpected token `fi'
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: `fi'
I am not sure if I took the commands from a tutorial that was meant for another system or if I am missing a step. Or whitespace is causing my commands not to run properly.
I've looked at multiple similar questions but I haven't found one where the question has my error exactly.
My home path:
$ echo $HOME
/cygdrive/c/Users/jhsu
$ echo ~
/cygdrive/c/Users/jhsu/
So I believe the files should be placed in the correct spot.
When all else fails in Cygwin...
Try running the dos2unix command on the file in question.
It might help when you see error messages like this:
-bash: '\r': command not found
Windows style newline characters can cause issues in Cygwin.
The dos2unix command modifies newline characters so they are Unix / Cygwin compatible.
CAUTION: the dos2unix command modifies files in place, so take precaution if necessary.
If you need to keep the original file, you should back it up first.
Note for Mac users: The dos2unix command does not exist on Mac OS X.
Check out this answer for a variety of solutions using different tools.
There is also a unix2dos command that does the reverse:
It modifies Unix newline characters so they're compatible with Windows tools.
If you open a file with Notepad and all the lines run together, try unix2dos filename.
For those who don't have dos2unix installed (and don't want to install it):
Remove trailing \r character that causes this error:
sed -i 's/\r$//' filename
Explanation:
Option -i is for in-place editing, we delete the trailing \r directly in the input file. Thus be careful to type the pattern correctly.
For WINDOWS (shell) users with Notepad++ (checked with v6.8.3) you can correct the specific file using the option
Edit
-> EOL conversion
-> Unix/OSX format
And save your file again.
Edit: still works in v7.5.1 (Aug 29 2017)
Edit: Jan 3, 2022. As VSCode is mentioned several times. Go to settings in VSCode and type files.eol in the search field and set to \n (Unix format). Note that this changes this setting for your user or workspace for all files and it may not be what you want. YMMV.
I am using cygwin and Windows7, the trick was NOT to put the set -o igncr into your .bashrc but put the whole SHELLOPTS into you environment variables under Windows. (So nothing with unix / cygwin...) I think it does not work from .bashrc because "the drops is already sucked"
as we would say in german. ;-)
So my SHELLOPTS looks like this
braceexpand:emacs:hashall:histexpand:history:igncr:interactive-comments:monitor
SUBLIME TEXT
With sublime you just go to
View - > Line Endings -> (select)Unix
Then save the file. Will fix this issue.
Easy as that!
If you are using a recent Cygwin (e.g. 1.7), you can also start both your .bashrc and .bash_profile with the following line, on the first non commented line:
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is needed
This will force bash to ignore carriage return (\r) characters used in Windows line separators.
See http://cygwin.com/ml/cygwin-announce/2010-08/msg00015.html.
The error:
'\r': command not found
is caused by shell not able to recognise Windows-like CRLF line endings (0d 0a) as it expects only LF (0a).
Git
If you using Git on Windows, make sure you selected 'Checkout as-is' during setup. Then make sure that you run: git config --global core.autocrlf false, so Git will not perform any conversions when checking out or committing text files.
dos2unix
If you're not using Git, you simply need to convert these affected files/scripts back into Unix-like line endings (LF), either by:
dos2unix ~/.bashrc
Note: The dos2unix command is part of dos2unix package.
Ex/Vim editor + tr
If you've Vim installed, the following command should correct the files:
ex +'bufdo! %! tr -d \\r' -scxa ~/.bash*
Useful alias: alias dos2unix="ex +'bufdo! %! tr -d \\\\r' -scxa".
tr
Here is the method by using tr:
cat ~/.bashrc | tr -d '\r' > ~/.bashrc.fixed && mv -v ~/.bashrc.fixed ~/.bashrc
or:
tr -d '\r' < filename > new_filename
Note: The \r is equivalent to \015.
sed
You can try the following command:
sed -i'.bak' s/\r//g ~/.bash*
recode
The following aliases can be useful (which replaces dos2unix command):
alias unix2dos='recode lat1:ibmpc'
alias dos2unix='recode ibmpc:lat1'
Source: Free Unix Tools (ssh, bash, etc) under Windows.
perl
The following perl command can convert the file from DOS into Unix format:
perl -p -i.bak -e 's/\015//g' ~/.bash*
Source: stripping the ^M.
tofrodos
On Linux, like Ubuntu which doesn’t come standard with either dos2unix or unix2dos, you can install tofrodos package (sudo apt-get install tofrodos), and define the following aliases:
alias dos2unix=’fromdos’
alias unix2dos=’todos’
Then use in the same syntax as above.
Vagrant
If you're using Vagrant VM and this happens for provisioning script, try setting binary option to true:
# Shell provisioner, see: https://www.vagrantup.com/docs/provisioning/shell.html
config.vm.provision "shell" do |s|
s.binary = true # Replace Windows line endings with Unix line endings.
s.path = "script.sh"
end
See: Windows CRLF to Unix LF Issues in Vagrant.
try execution the following command
vim .bashrc
:set ff=unix
:wq!
You can also add the option -o igncr to the bash call, e.g.
bash -x -o igncr script.sh
As per this gist, the solution is to create a ~/.bash_profile (in HOME directory) that contains:
export SHELLOPTS
set -o igncr
May be you used notepad++ for creating/updating this file.
EOL(Edit->EOL Conversion) Conversion by default is Windows.
Change EOL Conversion in Notepad++
Edit -> EOL Conversion -> Unix (LF)
I had the same problem. Solution: I edit the file with pspad editor, and give it a unix format (Menu - Format -> UNIX)
I believe you can set this format to your file with many other editors
For the Emacs users out there:
Open the file
M-x set-buffer-file-coding-system
Select "unix"
This will update the new characters in the file to be unix style. More info on "Newline Representation" in Emacs can be found here:
http://ergoemacs.org/emacs/emacs_line_ending_char.html
Note: The above steps could be made into an Emacs script if one preferred to execute this from the command line.
Issue maybe occured because of the file/script created/downloaded from a windows machine. Please try converting into linux file format.
dos2unix ./script_name.sh
or
dos2unix ~/.bashrc
If you have the vim package installed on your Cygwin install, you can use vim to fix this without find & replace. Start vim as follows: vim filename.sh (often it is aliased to vi also). Then, type :set fileformat=unix, then :wq (write & quit) to save your changes. (The : puts you in vim's edit mode.)
I recommend this over dos2unix since vim is probably more commonly installed.
However, it is probably a best practice to set your text editor to save files that you plan to use in a Unix/Linux environment to have a Unix text format. The answers given above for Notepad++ are a good example.
Additional note: If you are unsure what type a file is (DOS or Unix), you may use the file filename.sh. This can especially help in debugging more obscure issues (such as encoding issues when importing SQL dumps that come from Windows).
For other options on how to modify text file formatting, see this IU knowledge base article
More background information on Bash scripts and line endings is found on this StackOverflow question.
In EditPlus you do this from the
Document → File Format (CR/LF) → Change File Format... menu and then choose the Unix / Mac OS X radio button.
1. Choice
EditorConfig — is my choice.
2. Relevance
This answer is relevant for March 2018. In the future, the data from this answer may be obsolete.
Author of this answer personally used EditorConfig at March 2018.
3. Limitations
You need to use one of supported IDE/editors.
4. Argumentation
Simply usage. I need set my .editorconfig file 1 time, where I create my project, → I can forget some platform-, style- and IDE-specific problems.
Cross-platform, cross-languages, cross-IDE, cross-editors.
5. Example
5.1. Simple
I install EditorConfig plugin for Sublime Text → my text editor. I edit files in Sublime Text.
For example, I have sashacrlf.sh file:
echo "Sasha" &
echo "Goddess!" &
I run this file in Cygwin:
$ bash sashacrlf.sh
Sasha
sashacrlf.sh: line 1: $'\r': command not found
Goddess!
I create a file .editorconfig in same project as sashacrlf.sh.
[*.sh]
end_of_line = lf
It means, that if you save any file with .sh extension in your project in your editor, EditorConfig set UNIX line endings for this file.
I save sashacrlf.sh in my text editor again. I run sashacrlf.sh again:
$ bash sashacrlf.sh
Sasha
Goddess!
I can't get unexpected output in console.
5.2. Multiple file extensions
For example, I want to have UNIX line endings in all files with extensions .sh, .bashrc and .bash_profile.
I add these lines to my .editorconfig file:
[*.{sh,bashrc,bash_profile}]
end_of_line = lf
Now, if I save any file with .sh, .bashrc or .bash_profile extension, EditorConfig automatically set UNIX line ending for this file.
6. Additional links
EditorConfig official site.

Shell Script -- Unexpected End of File Error [duplicate]

This question already has answers here:
running bash script in cygwin on windows 7 [duplicate]
(2 answers)
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
I have windows, using Cygwin, trying to set JAVA_HOME permanently through my .bashrc file.
.bashrc:
export PATH="$JAVA_HOME/bin:$PATH"
export JAVA_HOME=$JAVA_HOME:"/cygdrive/c/Program Files (x86)/Java/jdk1.7.0_05"
.bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
running cygwin:
-bash: $'\377\376if': command not found
-bash: $'then\r': command not found
: No such file or directorysu//.bashrc
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: syntax error near unexpected token `fi'
-bash: /cygdrive/c/Users/jhsu//.bash_profile: line 3: `fi'
I am not sure if I took the commands from a tutorial that was meant for another system or if I am missing a step. Or whitespace is causing my commands not to run properly.
I've looked at multiple similar questions but I haven't found one where the question has my error exactly.
My home path:
$ echo $HOME
/cygdrive/c/Users/jhsu
$ echo ~
/cygdrive/c/Users/jhsu/
So I believe the files should be placed in the correct spot.
When all else fails in Cygwin...
Try running the dos2unix command on the file in question.
It might help when you see error messages like this:
-bash: '\r': command not found
Windows style newline characters can cause issues in Cygwin.
The dos2unix command modifies newline characters so they are Unix / Cygwin compatible.
CAUTION: the dos2unix command modifies files in place, so take precaution if necessary.
If you need to keep the original file, you should back it up first.
Note for Mac users: The dos2unix command does not exist on Mac OS X.
Check out this answer for a variety of solutions using different tools.
There is also a unix2dos command that does the reverse:
It modifies Unix newline characters so they're compatible with Windows tools.
If you open a file with Notepad and all the lines run together, try unix2dos filename.
For those who don't have dos2unix installed (and don't want to install it):
Remove trailing \r character that causes this error:
sed -i 's/\r$//' filename
Explanation:
Option -i is for in-place editing, we delete the trailing \r directly in the input file. Thus be careful to type the pattern correctly.
For WINDOWS (shell) users with Notepad++ (checked with v6.8.3) you can correct the specific file using the option
Edit
-> EOL conversion
-> Unix/OSX format
And save your file again.
Edit: still works in v7.5.1 (Aug 29 2017)
Edit: Jan 3, 2022. As VSCode is mentioned several times. Go to settings in VSCode and type files.eol in the search field and set to \n (Unix format). Note that this changes this setting for your user or workspace for all files and it may not be what you want. YMMV.
I am using cygwin and Windows7, the trick was NOT to put the set -o igncr into your .bashrc but put the whole SHELLOPTS into you environment variables under Windows. (So nothing with unix / cygwin...) I think it does not work from .bashrc because "the drops is already sucked"
as we would say in german. ;-)
So my SHELLOPTS looks like this
braceexpand:emacs:hashall:histexpand:history:igncr:interactive-comments:monitor
SUBLIME TEXT
With sublime you just go to
View - > Line Endings -> (select)Unix
Then save the file. Will fix this issue.
Easy as that!
If you are using a recent Cygwin (e.g. 1.7), you can also start both your .bashrc and .bash_profile with the following line, on the first non commented line:
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is needed
This will force bash to ignore carriage return (\r) characters used in Windows line separators.
See http://cygwin.com/ml/cygwin-announce/2010-08/msg00015.html.
The error:
'\r': command not found
is caused by shell not able to recognise Windows-like CRLF line endings (0d 0a) as it expects only LF (0a).
Git
If you using Git on Windows, make sure you selected 'Checkout as-is' during setup. Then make sure that you run: git config --global core.autocrlf false, so Git will not perform any conversions when checking out or committing text files.
dos2unix
If you're not using Git, you simply need to convert these affected files/scripts back into Unix-like line endings (LF), either by:
dos2unix ~/.bashrc
Note: The dos2unix command is part of dos2unix package.
Ex/Vim editor + tr
If you've Vim installed, the following command should correct the files:
ex +'bufdo! %! tr -d \\r' -scxa ~/.bash*
Useful alias: alias dos2unix="ex +'bufdo! %! tr -d \\\\r' -scxa".
tr
Here is the method by using tr:
cat ~/.bashrc | tr -d '\r' > ~/.bashrc.fixed && mv -v ~/.bashrc.fixed ~/.bashrc
or:
tr -d '\r' < filename > new_filename
Note: The \r is equivalent to \015.
sed
You can try the following command:
sed -i'.bak' s/\r//g ~/.bash*
recode
The following aliases can be useful (which replaces dos2unix command):
alias unix2dos='recode lat1:ibmpc'
alias dos2unix='recode ibmpc:lat1'
Source: Free Unix Tools (ssh, bash, etc) under Windows.
perl
The following perl command can convert the file from DOS into Unix format:
perl -p -i.bak -e 's/\015//g' ~/.bash*
Source: stripping the ^M.
tofrodos
On Linux, like Ubuntu which doesn’t come standard with either dos2unix or unix2dos, you can install tofrodos package (sudo apt-get install tofrodos), and define the following aliases:
alias dos2unix=’fromdos’
alias unix2dos=’todos’
Then use in the same syntax as above.
Vagrant
If you're using Vagrant VM and this happens for provisioning script, try setting binary option to true:
# Shell provisioner, see: https://www.vagrantup.com/docs/provisioning/shell.html
config.vm.provision "shell" do |s|
s.binary = true # Replace Windows line endings with Unix line endings.
s.path = "script.sh"
end
See: Windows CRLF to Unix LF Issues in Vagrant.
try execution the following command
vim .bashrc
:set ff=unix
:wq!
You can also add the option -o igncr to the bash call, e.g.
bash -x -o igncr script.sh
As per this gist, the solution is to create a ~/.bash_profile (in HOME directory) that contains:
export SHELLOPTS
set -o igncr
May be you used notepad++ for creating/updating this file.
EOL(Edit->EOL Conversion) Conversion by default is Windows.
Change EOL Conversion in Notepad++
Edit -> EOL Conversion -> Unix (LF)
I had the same problem. Solution: I edit the file with pspad editor, and give it a unix format (Menu - Format -> UNIX)
I believe you can set this format to your file with many other editors
For the Emacs users out there:
Open the file
M-x set-buffer-file-coding-system
Select "unix"
This will update the new characters in the file to be unix style. More info on "Newline Representation" in Emacs can be found here:
http://ergoemacs.org/emacs/emacs_line_ending_char.html
Note: The above steps could be made into an Emacs script if one preferred to execute this from the command line.
Issue maybe occured because of the file/script created/downloaded from a windows machine. Please try converting into linux file format.
dos2unix ./script_name.sh
or
dos2unix ~/.bashrc
If you have the vim package installed on your Cygwin install, you can use vim to fix this without find & replace. Start vim as follows: vim filename.sh (often it is aliased to vi also). Then, type :set fileformat=unix, then :wq (write & quit) to save your changes. (The : puts you in vim's edit mode.)
I recommend this over dos2unix since vim is probably more commonly installed.
However, it is probably a best practice to set your text editor to save files that you plan to use in a Unix/Linux environment to have a Unix text format. The answers given above for Notepad++ are a good example.
Additional note: If you are unsure what type a file is (DOS or Unix), you may use the file filename.sh. This can especially help in debugging more obscure issues (such as encoding issues when importing SQL dumps that come from Windows).
For other options on how to modify text file formatting, see this IU knowledge base article
More background information on Bash scripts and line endings is found on this StackOverflow question.
In EditPlus you do this from the
Document → File Format (CR/LF) → Change File Format... menu and then choose the Unix / Mac OS X radio button.
1. Choice
EditorConfig — is my choice.
2. Relevance
This answer is relevant for March 2018. In the future, the data from this answer may be obsolete.
Author of this answer personally used EditorConfig at March 2018.
3. Limitations
You need to use one of supported IDE/editors.
4. Argumentation
Simply usage. I need set my .editorconfig file 1 time, where I create my project, → I can forget some platform-, style- and IDE-specific problems.
Cross-platform, cross-languages, cross-IDE, cross-editors.
5. Example
5.1. Simple
I install EditorConfig plugin for Sublime Text → my text editor. I edit files in Sublime Text.
For example, I have sashacrlf.sh file:
echo "Sasha" &
echo "Goddess!" &
I run this file in Cygwin:
$ bash sashacrlf.sh
Sasha
sashacrlf.sh: line 1: $'\r': command not found
Goddess!
I create a file .editorconfig in same project as sashacrlf.sh.
[*.sh]
end_of_line = lf
It means, that if you save any file with .sh extension in your project in your editor, EditorConfig set UNIX line endings for this file.
I save sashacrlf.sh in my text editor again. I run sashacrlf.sh again:
$ bash sashacrlf.sh
Sasha
Goddess!
I can't get unexpected output in console.
5.2. Multiple file extensions
For example, I want to have UNIX line endings in all files with extensions .sh, .bashrc and .bash_profile.
I add these lines to my .editorconfig file:
[*.{sh,bashrc,bash_profile}]
end_of_line = lf
Now, if I save any file with .sh, .bashrc or .bash_profile extension, EditorConfig automatically set UNIX line ending for this file.
6. Additional links
EditorConfig official site.

bash file returns unexpected token `$'do\r''

I found this script online and tried to use it:
#!/bin/sh
# Target directory
TARGET=$3
echo "Copying to $TARGET"
for i in $(git diff --name-only $1 $2)
do
# First create the target directory, if it doesn't exist.
mkdir -p "$TARGET/$(dirname $i)"
# Then copy over the file.
cp "$i" "$TARGET/$i"
done
echo "Done";
I've validated the script online, and the script is okay. I've also tried to change it in various ways, but it doesn't work for me.
I've also tried running something like:
#!/bin/sh
# Target directory
TARGET=$3
echo "Copying to $TARGET"
for i in $(ls)
do
echo "text"
done
And I still get the same error:
./git-copy.sh: line 6: syntax error near unexpected token `$'do\r''
'/git-copy.sh: line 6: `do
Why is that?
Another possible solution using unix text editor vi:
open file in vi edit with vi filename.sh command;
type in vi :set ff=unix command;
save file with :wq
It will save the file with unix line endings.
Your script has been edited on a DOS or Windows based system and contains carriage-return characters that Linux/Unix does not like (that what \r is). You could use dos2unix to convert the carriage return line endings to the correct format; if you don't have dos2unix you might use awk like
awk '{ sub("\r$", ""); print }' git-copy.sh > git-copy2.sh
mv git-copy2.sh git-copy.sh
This worked for me:
dos2unix my_script.sh
If you don't have dos2unix :
For CentOS, Fedora or RHEL:
$ sudo yum install dos2unix
For Ubuntu or Debian:
$ sudo apt-get install tofrodos
$ sudo ln -s /usr/bin/fromdos /usr/bin/dos2unix
Explanation
This commonly is caused when the executable file is created in an operative system (windows) and used in another operative system (Linux). Also rare text editors could be the problem.
Just change your line sequence in your text editor from CRLF to LF. It worked for me :)
for notepad++ : edit => EOL Conversion => Unix(LF)
This is a rather common nightmare :)
I built a script to normalize \r\n from Dos and from (old)Mac; it also removes BOM
if present:
#!/usr/bin/perl -pi
s/\012\r|\r\012|\012|\r/\012/g; ## normalize line endings
s/^(\xFF\xFE|\xFE\xFF|\xEF\xBB\xBF)//; ## remove common initial BOM
Usage: script textualfile*
Ok, ok: Choose a better name for it! chmod and install it
I am editing the same thing in the windows system and running in the Linux system . this worked for me.
for notepad++ : edit => EOL Conversion => Unix(LF)
I had a case in which a had to write (INSERT) into DB the field that had such a bash script (on Windows's version of Intellij Idea) and then I had to read out the script from DB back on Unix server.
On unix I got this subject's error $'do\r'' for the reason explained above.
The solution was - copy\paste script on Win into Notepad, change in Notepad unix line endings from Windows (CR LF) to Unix (LF), save it in a file. Open file, copy\paste into Idea's database console.
Then it worked fine on Unix for the case it was not origin from Unix.
(Idea also has options to choose line endings in its editor)

Resources