This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 6 months ago.
Any idea of what the problem could be?
My code is:
#!/bin/bash
while :
do
echo "Press [CTRL+C] to stop.."
sleep 1
done
Saved it as .sh and ran bash file.sh
CentOS 6 32-bit
What is the issue? First time EVER using BASH, need it for a simple infinite loop on something.
Run cat -v file.sh.
You most likely have a carriage return or no-break space in your file. cat -v will show them as ^M and M-BM- or M- respectively. It will similarly show any other strange characters you might have gotten into your file.
Remove the Windows line breaks with
tr -d '\r' < file.sh > fixedfile.sh
I was getting the same error on Cygwin; I did the following (one of them fixed it):
Converted TABS to SPACES
ran dos2unix on the .(ba)sh file
What is the error you're getting?
$ bash file.sh
test.sh: line 8: syntax error: unexpected end of file
If you get that error, you may have bad line endings. Unix uses <LF> at the end of the file while Windows uses <CR><LF>. That <CR> character gets interpreted as a character.
You can use od -a test.sh to see the invisible characters in the file.
$ od -a test.sh
0000000 # ! / b i n / b a s h cr nl # sp cr
0000020 nl w h i l e sp : cr nl d o cr nl sp sp
0000040 sp sp e c h o sp " P r e s s sp [ C
0000060 T R L + C ] sp t o sp s t o p " cr
0000100 nl sp sp sp sp s l e e p sp 1 cr nl d o
0000120 n e cr nl
0000124
The sp stands for space, the ht stands for tab, the cr stands for <CR> and the nl stands for <LF>. Note that all of the lines end with cr followed by a nl character.
You can also use cat -v test.sh if your cat command takes the -v parameter.
If you have dos2unix on your box, you can use that command to fix your file:
$ dos2unix test.sh
There's a way you can get this problem without having mixed newline problems (at least, in my shell, which is GNU bash v4.3.30):
#!/bin/bash
# foo.sh
function foo() {
echo "I am quoting a thing `$1' inside a function."
}
while [ "$input" != "y" ]; do
read -p "Hit `y' to continue: " -n 1 input
echo
done
foo "What could possibly go wrong?"
$ ./foo.sh
./foo.sh: line 11: syntax error near unexpected token `done'
./foo.sh: line 11: `done'
This is because bash expands backticks inside double-quoted strings (see the bash manual on quoting and command substitution), and before finding a matching backtick, will interpret any additional double quotes as part of the command substitution:
$ echo "Command substitution happens inside double-quoted strings: `ls`"
Command substitution happens inside double-quoted strings: foo.sh
$ echo "..even with double quotes: `grep -E "^foo|wrong" foo.sh`"
..even with double quotes: foo "What could possibly go wrong?"
You can get around this by escaping the backticks in your string with a backslash, or by using a single-quoted string.
I'm not really sure why this only gives the one error message, but I think it has to do with the function definition:
#!/bin/bash
# a.sh
function a() {
echo "Thing's `quoted'"
}
a
while true; do
echo "Other `quote'"
done
#!/bin/bash
# b.sh
echo "Thing's `quoted'"
while true; do
echo "Other `quote'"
done
$ ./a.sh
./a.sh: line 10: syntax error near unexpected token `done'
./a.sh: line 10: `done'
$ ./b.sh
./b.sh: command substitution: line 6: unexpected EOF while looking for matching `''
./b.sh: command substitution: line 9: syntax error: unexpected end of file
Thing's quote'
./b.sh: line 7: syntax error near unexpected token `done'
./b.sh: line 7: `done'
Might help someone else : I encountered the same kind of issues while I had done some "copy-paste" from a side Microsoft Word document, where I took notes, to my shell script(s).
Re-writing, manually, the exact same code in the script just solved this.
It was quite un-understandable at first, I think Word's hidden characters and/or formatting were the issue. Obvious but not see-able ... I lost about one hour on this (I'm no shell expert, as you might guess ...)
Sometimes this error happens because of unexpected CR characters in file, usually because the file was generated on a Windows system which uses CR line endings. You can fix this by running os2unix or tr, for example:
tr -d '\015' < yourscript.sh > newscript.sh
This removes any CR characters from the file.
Open new file named foobar
nano -w foobar
Input script
#!/bin/bash
while [ 0 = 0 ]; do
echo "Press [CTRL+C] to stop.."
sleep 1
done;
Exit and save
CTRL+X then Y and Enter
Set script executable and run
chmod +x foobar
./foobar
Had similar problems just now and these are two separate instances and solutions that worked for me:
Case 1. Basically, had a space after the last command within my newline-separated for-loop, eg. (imagining that | here represents the carat in a text editor showing where you are writing), this is what I saw when clicking around the end of the line of the last command in the loop:
for f in $pathToFiles
do
$stuff |
done
Notice the space before before the carat (so far as I know, this is something cat has no option do display visually (one way you could test is with something like od -bc yourscript.sh)). Changing the code to
for f in $pathToFiles
do
$stuff| <--- notice the carat shows no ending space before the newline
done
fixed the problem.
Case 2. Was using a pseudo try-catch block for the for-loop (see https://stackoverflow.com/a/22010339/8236733) like
{
for f in $pathToFiles
do
{ $stuff } || { echo "Failed to complete stuff"; exit 255; }
done
} || { echo "Failed to complete loop"; exit 255; }
and apparently bash did not like the nested {}s. Changing to
{
for f in $pathToFiles
do
$stuff
done
} || { echo "Failed to complete loop"; exit 255; }
fixed the problem in this case. If anyone can further explain either of these cases, please let me know more about them in the comments.
I had same problem, but solved.
I removed the following line in .bashrc
alias do="docker.exe" # this line caused the problem
I use WSL(windows subsystem for linux)
In my case, what was causing the problem was an if else statement. After re-writing the conditions, the error 'near done' got away.
Edit your code in any linux environment then you won't face this problem. If edit in windows
notepad any space take it as ^M.
I have exactly the same issue as above, and took me the whole day to discover that it doesn't like my newline approach. Instead I reused the same code with semi-colon approach instead.
For example my initial code using the newline (which threw the same error as yours):
Y=1
while test "$Y" -le "20"
do
echo "Number $Y"
Y=$[Y+1]
done
And using code with semicolon approach with worked wonder:
Y=1 ; while test "$Y" -le "20"; do echo "Number $Y"; Y=$[Y+1] ; done
I notice the same problem occurs for other commands as well using the newline approach, so I think I am gonna stick to using semicolon for my future code.
For me you had to have it do something between the do and done.
#!/bin/bash
echo "Endless Loop, to STOP press ctrl C or ctrl Z"
for ((; ;))
do
done
echo "Loop Ended"
gave me the error, but
#!/bin/bash
echo "Endless Loop, to STOP press ctrl C or ctrl Z"
for ((; ;))
do
sleep 1
done
echo "Loop Ended"
fixed it.
Related
There is a file called test.txt that contains:
ljlkfjdslkfldjfdsajflkjf word:test1
dflkdjflkdfdjls word:test2
dlkdj word:test3
word:test4
word:NewYork
dljfldflkdflkdjf word:test7
djfkd word:young
dkjflke word:lisa
amazonwle word:NewYork
dlksldjf word:test10
Now all we want is to get the strings after colon and if the result is same, print the output, in this case it is "NewYork"
Here is the script which lists the elements but when tried to push into array and compare it is failing, Please let me know my mistakes.
#!/usr/bin/sh
input="test.txt"
cat $input | while read line; do output= $(echo $line | cut -d":" -f2); done
for (( i = 0 ; i < "${#output[#]}" ; i++ ))
{
echo ${output[i]}
}
Error obtained:
./compare.sh
./compare.sh: line 11: test1: command not found
./compare.sh: line 11: test2: command not found
./compare.sh: line 11: test3: command not found
./compare.sh: line 11: test4: command not found
./compare.sh: line 11: NewYork: command not found
./compare.sh: line 11: raghav: command not found
./compare.sh: line 11: young: command not found
./compare.sh: line 11: lisa: command not found
./compare.sh: line 11: NewYork: command not found
./compare.sh: line 11: test10: command not found
Please let me know my mistakes.
output= $(..) first excutes the command $(..) inside and grabs it's output. It then sets the variable output to an empty string as-if the same as output="" and exports the variable output and executes the output of $(..) as a command. Remove the space after =.
You are setting output on the right side of a pipe inside a subshell. The changes will not be visible outside - output is unset once the pipe terminates. Use redirection while .... done < file.
And output is not an array, but a normal variable. There is no ${output[i]} (well, except ${output[0]}) as it's not an array (and output is unset, as explained above). Append an element to an array output+=("$(...)").
#!/usr/bin/sh - is invalid, sh may not be bash and support bash arrays. To use bash extension specifically use bash and use a shebang with bash - #!/bin/bash,
Now stylistic:
The format for .... { ... } is supported in bash since forever, however it's a rather undocumented syntax rather to be deprecated to be readable by people used to programming in C. Prefer the standard do ... done.
The expansions $input and ${output[i]} are unquoted.
The read will ignore leading and trailing whitespaces and also interpret/ignore \ backslash sequences (ie. see -r option to read).
echo $line will make $line undergo word splitting - multiple whitespaces will be replaced by a single space.
And using cut after read can be simpler written as just read with IFS=: and splitting on read. Also, if you're using cut, you could just cut -d: -f2 file on the whole file instead of cutting one line at a time.
Re-read basic bash introduction - notice that bash is space aware, spaces around = count. Read bashfaq how to read a file field by field, read about subshells and environments, read bashfaq I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read? and an introduction to bash arrays.
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 6 months ago.
Any idea of what the problem could be?
My code is:
#!/bin/bash
while :
do
echo "Press [CTRL+C] to stop.."
sleep 1
done
Saved it as .sh and ran bash file.sh
CentOS 6 32-bit
What is the issue? First time EVER using BASH, need it for a simple infinite loop on something.
Run cat -v file.sh.
You most likely have a carriage return or no-break space in your file. cat -v will show them as ^M and M-BM- or M- respectively. It will similarly show any other strange characters you might have gotten into your file.
Remove the Windows line breaks with
tr -d '\r' < file.sh > fixedfile.sh
I was getting the same error on Cygwin; I did the following (one of them fixed it):
Converted TABS to SPACES
ran dos2unix on the .(ba)sh file
What is the error you're getting?
$ bash file.sh
test.sh: line 8: syntax error: unexpected end of file
If you get that error, you may have bad line endings. Unix uses <LF> at the end of the file while Windows uses <CR><LF>. That <CR> character gets interpreted as a character.
You can use od -a test.sh to see the invisible characters in the file.
$ od -a test.sh
0000000 # ! / b i n / b a s h cr nl # sp cr
0000020 nl w h i l e sp : cr nl d o cr nl sp sp
0000040 sp sp e c h o sp " P r e s s sp [ C
0000060 T R L + C ] sp t o sp s t o p " cr
0000100 nl sp sp sp sp s l e e p sp 1 cr nl d o
0000120 n e cr nl
0000124
The sp stands for space, the ht stands for tab, the cr stands for <CR> and the nl stands for <LF>. Note that all of the lines end with cr followed by a nl character.
You can also use cat -v test.sh if your cat command takes the -v parameter.
If you have dos2unix on your box, you can use that command to fix your file:
$ dos2unix test.sh
There's a way you can get this problem without having mixed newline problems (at least, in my shell, which is GNU bash v4.3.30):
#!/bin/bash
# foo.sh
function foo() {
echo "I am quoting a thing `$1' inside a function."
}
while [ "$input" != "y" ]; do
read -p "Hit `y' to continue: " -n 1 input
echo
done
foo "What could possibly go wrong?"
$ ./foo.sh
./foo.sh: line 11: syntax error near unexpected token `done'
./foo.sh: line 11: `done'
This is because bash expands backticks inside double-quoted strings (see the bash manual on quoting and command substitution), and before finding a matching backtick, will interpret any additional double quotes as part of the command substitution:
$ echo "Command substitution happens inside double-quoted strings: `ls`"
Command substitution happens inside double-quoted strings: foo.sh
$ echo "..even with double quotes: `grep -E "^foo|wrong" foo.sh`"
..even with double quotes: foo "What could possibly go wrong?"
You can get around this by escaping the backticks in your string with a backslash, or by using a single-quoted string.
I'm not really sure why this only gives the one error message, but I think it has to do with the function definition:
#!/bin/bash
# a.sh
function a() {
echo "Thing's `quoted'"
}
a
while true; do
echo "Other `quote'"
done
#!/bin/bash
# b.sh
echo "Thing's `quoted'"
while true; do
echo "Other `quote'"
done
$ ./a.sh
./a.sh: line 10: syntax error near unexpected token `done'
./a.sh: line 10: `done'
$ ./b.sh
./b.sh: command substitution: line 6: unexpected EOF while looking for matching `''
./b.sh: command substitution: line 9: syntax error: unexpected end of file
Thing's quote'
./b.sh: line 7: syntax error near unexpected token `done'
./b.sh: line 7: `done'
Might help someone else : I encountered the same kind of issues while I had done some "copy-paste" from a side Microsoft Word document, where I took notes, to my shell script(s).
Re-writing, manually, the exact same code in the script just solved this.
It was quite un-understandable at first, I think Word's hidden characters and/or formatting were the issue. Obvious but not see-able ... I lost about one hour on this (I'm no shell expert, as you might guess ...)
Sometimes this error happens because of unexpected CR characters in file, usually because the file was generated on a Windows system which uses CR line endings. You can fix this by running os2unix or tr, for example:
tr -d '\015' < yourscript.sh > newscript.sh
This removes any CR characters from the file.
Open new file named foobar
nano -w foobar
Input script
#!/bin/bash
while [ 0 = 0 ]; do
echo "Press [CTRL+C] to stop.."
sleep 1
done;
Exit and save
CTRL+X then Y and Enter
Set script executable and run
chmod +x foobar
./foobar
Had similar problems just now and these are two separate instances and solutions that worked for me:
Case 1. Basically, had a space after the last command within my newline-separated for-loop, eg. (imagining that | here represents the carat in a text editor showing where you are writing), this is what I saw when clicking around the end of the line of the last command in the loop:
for f in $pathToFiles
do
$stuff |
done
Notice the space before before the carat (so far as I know, this is something cat has no option do display visually (one way you could test is with something like od -bc yourscript.sh)). Changing the code to
for f in $pathToFiles
do
$stuff| <--- notice the carat shows no ending space before the newline
done
fixed the problem.
Case 2. Was using a pseudo try-catch block for the for-loop (see https://stackoverflow.com/a/22010339/8236733) like
{
for f in $pathToFiles
do
{ $stuff } || { echo "Failed to complete stuff"; exit 255; }
done
} || { echo "Failed to complete loop"; exit 255; }
and apparently bash did not like the nested {}s. Changing to
{
for f in $pathToFiles
do
$stuff
done
} || { echo "Failed to complete loop"; exit 255; }
fixed the problem in this case. If anyone can further explain either of these cases, please let me know more about them in the comments.
I had same problem, but solved.
I removed the following line in .bashrc
alias do="docker.exe" # this line caused the problem
I use WSL(windows subsystem for linux)
In my case, what was causing the problem was an if else statement. After re-writing the conditions, the error 'near done' got away.
Edit your code in any linux environment then you won't face this problem. If edit in windows
notepad any space take it as ^M.
I have exactly the same issue as above, and took me the whole day to discover that it doesn't like my newline approach. Instead I reused the same code with semi-colon approach instead.
For example my initial code using the newline (which threw the same error as yours):
Y=1
while test "$Y" -le "20"
do
echo "Number $Y"
Y=$[Y+1]
done
And using code with semicolon approach with worked wonder:
Y=1 ; while test "$Y" -le "20"; do echo "Number $Y"; Y=$[Y+1] ; done
I notice the same problem occurs for other commands as well using the newline approach, so I think I am gonna stick to using semicolon for my future code.
For me you had to have it do something between the do and done.
#!/bin/bash
echo "Endless Loop, to STOP press ctrl C or ctrl Z"
for ((; ;))
do
done
echo "Loop Ended"
gave me the error, but
#!/bin/bash
echo "Endless Loop, to STOP press ctrl C or ctrl Z"
for ((; ;))
do
sleep 1
done
echo "Loop Ended"
fixed it.
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE:vitest filename"
else
FILENAME=$1
exec vi $FILENAME <<EOF
i
Line 1.
Line 2.
^[
ZZ
EOF
fi
exit 0
I'm trying to input the Line 1. and Line 2. with Exec vi using the here doc, and commands.
When running the script it gives me the following:
Vim(?):Warning: Input is not from a terminal
Vim: Error reading input, exiting...
Press ENTER or type command to continueVim: Finished.
Vim: Error reading input, exiting...
Vim: Finished.
You want to start vi in ex mode, with a few minor changes to the script.
vi -e "$FILENAME" <<EOF
i
Line 1.
Line 2.
.
wq
EOF
exec is almost certainly unnecessary, especially since you have an exit command following vi. exec is used to replace the current script with the given command; it is not needed simply to execute a command.
A brief history of UNIX text editors:
ed was the original editor, designed to work with a teletype rather than a video terminal.
ex was an extended version of ed, designed to take advantage of a video terminal.
vi was an editor that provided ex with a full-screen visual mode, in contrast with the line-oriented interface employed by ed and ex.
As suggested, ed
ed file << END
1i
line1
line2
.
wq
END
The "dot" line means "end of input".
It can be written less legibly as a one-liner
printf "%s\n" 1i "line1" "line2" . wq | ed file
Use cat.
$ cat file1.txt file2.txt | tee file3.txt
Line 1
Line 2
aaaa
bbbb
cccc
Using sed
If I understand correctly, you want to add two lines to the beginning of a file. In that case, as per Cyrus' suggestion, run:
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE:vitest filename"
exit 1
fi
sed -i.bak '1 s/^/line1\nline2\n/' "$1"
Notes:
When a shell variable is used, it should be in double-quotes unless you want word splitting and pathname expansion to be performed. This is important for file names, for example, as it is now common for them to contain whitespace.
It is best practice to use lower or mixed case names for shell variables. The system uses upper case names for its variables and you don't want to overwrite one of them accidentally.
In the check for the argument, the if statement should include an exit to prevent the rest of the script from being run in the case that no argument was provided. In the above, we added exit 1 which sets the exit code to 1 to signal an error.
Using vi
Let's start with this test file:
$ cat File
some line
Now, let's run vi and see what is in File afterward:
$ vi -s <(echo $'iline1\nline2\n\eZZ') File
$ cat File
line1
line2
some line
The above requires bash or similar.
What is wrong with the 5th line in this script ( I have included the snippet that gives me the error and the actual error is listed in the bottom after the code and a link to complete script)?
#! /bin/bash
INSTALLDIR=/usr/local/mapguideopensource
CLEAN_FLAG=0
while [ $# -gt 0 ]; do # Until you run out of parameters...
case "$1" in
-prefix|--prefix)
INSTALLDIR="$2"
shift
;;
-clean|--clean)
CLEAN_FLAG=1
shift
;;
-help|--help)
echo "Usage: $0 (options)"
echo "Options:"
echo " --prefix [installation directory]"
echo " --clean [clean all objects and binaries in Oem]"
echo " --help [Display usage]"
exit
;;
esac
shift # Check next set of parameters.
done
This is the error i get when i run this bash script on linux (REHL5) :
: command not founde 4:
: command not founde 8:
: command not founde 8:
: command not founde 12:
MapGuide Open Source build script for OEM components
'/build_oem.sh: line 17: syntax error near unexpected token `in
'/build_oem.sh: line 17: ` case "$1" in
Please note, that the line number above corresponds to the actual script i am running (i have included a link to that script below)
The original script i am running
From the errors, I'm pretty sure you have carriage returns (aka CR or ^M) at the end of the lines. Windows/DOS text files have carriage return AND linefeed at the end of each line, but unix programs (like bash) just expect a linefeed, and get horribly confused if there's a CR as well. The giveaway is error messages like:
: command not founde 4:
What this really is is ./build_oem.sh: line 4: ^M: command not found, but the carriage return makes the terminal go back to the beginning of the line, and write the end of the message over the beginning of the message:
./build_oem.sh: line 4:
: command not found
|
V
: command not founde 4:
To fix the script, use dos2unix to convert it to proper unix format, then switch to a text editor that saves in unix format.
What choroba says, but also note that your shebang has to be on the first line (which it is not), otherwise it is useless since it's just a plain comment then and it won't necessarily execute under bash.
In the original script, lines 4 and 8 are empty. There is probably some invisible control character on the lines. Try xxd build_oem.sh.
This question already has answers here:
Empty Body For Loop Linux Shell
(4 answers)
Closed 6 months ago.
I am trying to learn shell scripting, so I created a simple script with a loop that does nothing:
#!/bin/bash
names=(test test2 test3 test4)
for name in ${names[#]}
do
#do something
done
however, when I run this script I get the following errors:
./test.sh: line 6: syntax error near unexpected token done'
./test.sh: line 6: done'
What have I missed here? are shell scripts 'tab sensitive'?
No, shell scripts are not tab sensitive (unless you do something really crazy, which you are not doing in this example).
You can't have an empty while do done block, (comments don't count)
Try substituting echo $name instead
#!/bin/bash
names=(test test2 test3 test4)
for name in ${names[#]}
do
printf "%s " $name
done
printf "\n"
output
test test2 test3 test4
dash and bash are a bit brain-dead in this case, they do not allow an empty loop so you need to add a no op command to make this run, e.g. true or :. My tests suggest the : is a bit faster, although they should be the same, not sure why:
time (i=100000; while ((i--)); do :; done)
n average takes 0.262 seconds, while:
time (i=100000; while ((i--)); do true; done)
takes 0.293 seconds. Interestingly:
time (i=100000; while ((i--)); do builtin true; done)
takes 0.356 seconds.
All measurements are an average of 30 runs.
Bash has a built-in no-op, the colon (:), which is more lightweight
than spawning another process to run true.
#!/bin/bash
names=(test test2 test3 test4)
for name in "${names[#]}"
do
:
done
EDIT: William correctly points out that true is also a shell built-in, so take this answer as just another option FYI, not a better solution than using true.
You could replace the nothing with 'true' instead.
You need to have something in your loop otherwise bash complains.
This error is expected with some versions of bash where the script was edited on Windows and so the script actually looks as follows:
#!/bin/bash^M
names=(test test2 test3 test4)^M
for name in ${names[#]}^M
do^M
printf "%s " $name^M
done^M
printf "\n"^M
where the ^M represents the carriage-return character (0x0D). This can easily be seen in vi by using the binary option as in:
vi -b script.sh
To remove those carriage-return characters simply use the vi command:
1,$s/^M//
(note that the ^M above is a single carriage-return character, to enter it in the editor use sequence Control-V Control-M)