running bash script in cygwin on windows 7 [duplicate] - bash

This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
I am trying to run the below bash script in cygwin on windows 7
REPEATTIMES="$1"
if [ $# = 0 ]; then
echo "Usage: fetch topN repeatTimes"
exit 1
fi
for (( i=1; i<=$REPEATTIMES; i++ ))
do
echo "ITERATION: $i"
echo "GENERATING"
log=thelogs/log
bin/nutch generate crawl/segment -topN 10 > $log
batchId=`sed -n 's|.*batch id: \(.*\)|\1|p' < $log`
echo "batch id: $batchId "
# rename log file by appending the batch id
log2=$log$batchId
mv $log $log2
log=$log2
echo "FETCHING"
bin/nutch fetch crawl/segments/$batchId >> $log
echo "PARSING"
bin/nutch parse crawl/segments/$batchId >> $log
echo "UPDATING DB"
bin/nutch updatedb crawl/crawldb crawl/segments/$batchId >> $log
echo "Done "
done
But when i run it i get the error :
line 11 :syntax error near unexpected token '$'\r'
line 11 :'for (( i=1; i<= REPEATTIMES; i++ ))
The script works fine on a ubuntu server. But i need to run it now on a windows machine.

If you can't fix all your scripts, you should be able to modify the EOL behavior in Cygwin by setting an option to ignore CRs:
set -o igncr
If you add this to your .bash_profile, it will be globally set by default when you login:
export SHELLOPTS
set -o igncr
You can also do this per script internally by putting this line just after the #! line:
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is required
You need the comment to ignore the CR in that line which is read before the option takes effect.

The latest version of Cygwin seems to only support files in Unix format (i.e. with \n for newlines as opposed to the DOS/Windows \r\n newline).
To fix this, run the /bin/dos2unix.exe utility, giving your script as the argument to the command:
e.g. /bin/dos2unix.exe myScript.sh
This will convert it to Unix format and you then should be able to run it.

Related

using two variables in a bash find and replace [duplicate]

This question already has answers here:
How to pass a variable containing slashes to sed
(7 answers)
Closed 11 months ago.
I am trying to automate some patching steps and I have written a script to back up the file and then replace the path in the file in all spots, upon testing backing up the files was ok but the find and replace even though it states successful didn't work, I am trying to use said but I am not married to that so if there is a cleaner way I am not opposed, please see my code example below:
#!/bin/bash
#set -x
nodemanager="/u01/app/oracle/admin/domain/mserver/ADF_INT/nodemanager/"
bindirectory="/u01/app/oracle/admin/domain/mserver/ADF_INT/bin/"
ouilocation="/u01/app/oracle/product/fmw/middleware12c/oui/bin/"
date=$(date +"%d-%m-%y")
echo $date
read -p "Please enter the current jdk path: " oldjdk
read -p "Please enter the new jdk path: " newjdk
echo "Backing up an uploading files to remote server...."
cd $nodemanager || exit
cp nodemanager.properties nodemanager_$date.bkp
if [ $? -ne 0 ]; # Again checking if the last operation was successful if not shall exit the
script
then
echo -e "nodemanager.properties backup failed"
echo -e "Terminating script"
exit 0
fi
sed -i -e 's/${$oldjdk/$newjdk}/g' nodemanager.properties
if [ $? -ne 0 ]; # Again checking if the last operation was successful if not shall exit the
script
then
echo -e "find and replace failed for nodemanager.properties"
echo -e "Terminating script"
exit 0
fi
echo -e "nodemanager.properties operations completed successfully\n"
Thanks
JJ
To save you some trouble on this I figured it out the / is not part of sed it can be any delimiter that is not clashing with the path so I used this:
sed -i "s+$oldjdk+$newjdk+g" file
Thanks
JJ

ksh syntax error: `if' unmatched

I'm new in ksh world and I have a problem right now with a script. The script under this lines is into the .profile file of a user in a UNIX machine and when I try to connect whith him i get always the error
home/userTest/.profile: syntax error: `if' unmatched
I don't know how to solve this, because I suppose that this scripts defines the prompt for the connected user, and if I have this error the prompt only shows "$"
I tried the command
ksh -n /home/userTest/.profile
and I get the error always in the last line of the file
#!/bin/ksh
# ksh example
if [[$0 = "ksh"]];
then
bash
exit $?
fi
if [[$0 = "-ksh"]];
then
bash --login
exit $?
fi
export LOGIN=$LOGNAME
#prompt config
PS1="$LOGIN#"$(hostname)":$PWD"
if [["$(id -u)" = "0"]];
then
export PS1="$PS1# "
else
export PS1="$PS1> "
fi
#Alias utile
alias ll="ls -la"
#Set any export here
export PATH_EXAMPLE=/home/userTest
export JAVA_HOME=$PATH_EXAMPLE/games/java/current
export PATH=$JAVA_HOME/bin:$PATH
How can I solve this problem ?
Thanks.
I had the same error. Turned out it was due to DOS format newlines (CR-LF) in my *.sh file created in Windows and then transferred to a Linux server.
Commands to convert DOS format newlines (CR-LF) to UNIX format newlines (LF)
In Windows: using Notepad++, as explained here:
From the "Edit" menu, select "EOL Conversion" -> "UNIX/OSX Format".
You can also set the default EOL in notepad++ via "Settings" -> "Preferences" -> "New Document/Default Directory" then select "Unix/OSX" under the Format box.
In UNIX/Linux: using one of the techniques explained here:
Convert DOS to UNIX using sed command:
sed 's/^M$//' input.txt > output.txt
Convert DOS to UNIX using tr command:
tr -d '\r' < input.file > output.file
Convert DOS to UNIX using this Perl one-liner:
perl -pi -e 's/\r\n/\n/g' input.file
Convert DOS to UNIX using dos2unix command:
dos2unix myfile.txt or dos2unix -b myfile.txt (with a backup)
[ Bonus tip ]
Commands to convert UNIX format newlines (LF) to DOS format newlines (CR-LF)
Convert UNIX to DOS using unix2dos command:
unix2dos myfile.txt or unix2dos -b myfile.txt (with a backup)
Convert UNIX to DOS using sed command:
sed 's/$'"/`echo \\\n\\\r`/" input.txt > output.txt (you need those \\\, you do)
I am using following version
version sh (AT&T Research) 93u+ 2012-08-01
I did not received any syntax error for your above code , though there a problem with your if statement condition instead of
if [[$0 = "-ksh"]]
it should be
if [[ $0 == "-ksh" ]]
or
if [[ $0 = "-ksh" ]]
the latter is obsolete
The complete code is as below
#!/bin/ksh
# ksh example
if [[ $0 = "ksh" ]];
then
bash
exit $?
fi
if [[ $0 == "-ksh" ]];
then
bash --login
exit $?
fi
export LOGIN=$LOGNAME
#prompt config
PS1="$LOGIN#"$(hostname)":$PWD"
if [[ "$(id -u)" == "0" ]];
then
export PS1="$PS1# "
else
export PS1="$PS1> "
fi
#Alias utile
alias ll="ls -la"
#Set any export here
export PATH_EXAMPLE=/home/userTest
export JAVA_HOME=$PATH_EXAMPLE/games/java/current
export PATH=$JAVA_HOME/bin:$PATH
You script may be having some unwanted character , try to look out for then using cat -vte
you can also try command dos2unix filename and then run ksh -n

line 1: ?#!/usr/bin/sh: not found when trying to execute a shell script

I have a script called autoinstall:
#!/usr/bin/sh
echo "Installasi membutuhkan free space minimal 2MB, pastikan ada punya cukup space di router anda"
read -p "Anda yakin ingin melanjutkan installasi?(y/n) " -n 1 -r
echo ""
if [[ $REPLY = ^[Yy]$ ]]
then
cd /
cd /tmp/
tar -xvf OpenWrt_Angel_Beats_Edition_v1.3.3.tar -C /
chmod -R 744 /root/crt
chmod 744 /www/wget/wget_download.sh
chmod 744 /usr/bin/gsm
chmod 744 /usr/bin/profile
opkg update && opkg install elinks
cp /etc/rc.local /etc/rc.local.backup
cat > /etc/rc.local << END
#!bin/sh
# /etc/rc.local: Local system initialization script.
#
# Put any local startup commands in here. Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.
sh /www/wget/wget_download.sh > /dev/null 2>&1 &
exit 0
END
killall sh /www/wget/wget_download.sh
sh /www/wget/wget_download.sh > /dev/null 2>&1 &
echo "File backup /etc/rc.local.backup telah dibuat, gunakan file ini untuk mengembalikan konfigurasi rc.local anda yang dulu jika diperlukan"
echo "Installasi selesai. Jangan lupa di akun openvpn yang digunakan (/root/crt/xxx.ovpn) tambahkan baris ini:
script-security 2
up client-connect.sh"
else
echo ""
echo "Installasi dibatalkan"
fi
Every command that I put in the first line always gets the error above (line 1:xxx not found) and I'm sure I've typed in the correct command, even echo gives the error like that, how do I solve this?
There can be two problems here:
The file doesn't exist. Usually, for sh, the path is /bin/sh, so it should be #!/bin/sh
You're editing the file on Windows. Windows uses CR+LF as line ending. Unix (and Linux) uses just LF. So for Linux, the command reads "execute /bin/sh<CR> and sh<CR> doesn't exist.
Solution: When editing the file, make sure you use Unix line endings.
The file might have been edited with an editor that insert a Unicode BOM (Byte Order Mark).
Have a look to the first line contents with:
od -c autoinstall | head -1
or
hd -n 16 autoinstall
If you see unexpected characters before #!/usr/bin/sh, you might try one of the methods described here Using awk to remove the Byte-order mark to remove the BOM.

Bash syntax error: unexpected end of file

Forgive me for this is a very simple script in Bash. Here's the code:
#!/bin/bash
# june 2011
if [ $# -lt 3 -o $# -gt 3 ]; then
echo "Error... Usage: $0 host database username"
exit 0
fi
after running sh file.sh:
syntax error: unexpected end of file
I think file.sh is with CRLF line terminators.
run
dos2unix file.sh
then the problem will be fixed.
You can install dos2unix in ubuntu with this:
sudo apt-get install dos2unix
Another thing to check (just occured to me):
terminate bodies of single-line functions with semicolon
I.e. this innocent-looking snippet will cause the same error:
die () { test -n "$#" && echo "$#"; exit 1 }
To make the dumb parser happy:
die () { test -n "$#" && echo "$#"; exit 1; }
i also just got this error message by using the wrong syntax in an if clause
else if (syntax error: unexpected end of file)
elif (correct syntax)
i debugged it by commenting bits out until it worked
an un-closed if => fi clause will raise this as well
tip: use trap to debug, if your script is huge...
e.g.
set -x
trap read debug
I got this answer from this similar problem on StackOverflow
Open the file in Vim and try
:set fileformat=unix
Convert eh line endings to unix endings and see if that solves the
issue. If editing in Vim, enter the command :set fileformat=unix and
save the file. Several other editors have the ability to convert line
endings, such as Notepad++ or Atom
Thanks #lemongrassnginger
This was happening for me when I was trying to call a function using parens, e.g.
run() {
echo hello
}
run()
should be:
run() {
echo hello
}
run
I had the problem when I wrote "if - fi" statement in one line:
if [ -f ~/.git-completion.bash ]; then . ~/.git-completion.bash fi
Write multiline solved my problem:
if [ -f ~/.git-completion.bash ]; then
. ~/.git-completion.bash
fi
So I found this post and the answers did not help me but i was able to figure out why it gave me the error. I had a
cat > temp.txt < EOF
some content
EOF
The issue was that i copied the above code to be in a function and inadvertently tabbed the code. Need to make sure the last EOF is not tabbed.
on cygwin I needed:-
export SHELLOPTS
set -o igncr
in .bash_profile . This way I didn't need to run unix2dos
FOR WINDOWS:
In my case, I was working on Windows OS and I got the same error while running autoconf.
I simply open configure.ac file with my NOTEPAD++ IDE.
Then I converted the File with EOL conversion into Windows (CR LF) as follows:
EDIT -> EOL CONVERSION -> WINDOWS (CR LF)
Missing a closing brace on a function definition will cause this error as I just discovered.
function whoIsAnIidiot() {
echo "you are for forgetting the closing brace just below this line !"
Which of course should be like this...
function whoIsAnIidiot() {
echo "not you for sure"
}
I was able to cut and paste your code into a file and it ran correctly. If you
execute it like this it should work:
Your "file.sh":
#!/bin/bash
# june 2011
if [ $# -lt 3 -o $# -gt 3 ]; then
echo "Error... Usage: $0 host database username"
exit 0
fi
The command:
$ ./file.sh arg1 arg2 arg3
Note that "file.sh" must be executable:
$ chmod +x file.sh
You may be getting that error b/c of how you're doing input (w/ a pipe, carrot,
etc.). You could also try splitting the condition into two:
if [ $# -lt 3 ] || [ $# -gt 3 ]; then
echo "Error... Usage: $0 host database username"
exit 0
fi
Or, since you're using bash, you could use built-in syntax:
if [[ $# -lt 3 || $# -gt 3 ]]; then
echo "Error... Usage: $0 host database username"
exit 0
fi
And, finally, you could of course just check if 3 arguments were given (clean,
maintains POSIX shell compatibility):
if [ $# -ne 3 ]; then
echo "Error... Usage: $0 host database username"
exit 0
fi
In my case, there is a redundant \ in the like following:
function foo() {
python tools/run_net.py \
--cfg configs/Kinetics/X3D_8x8_R50.yaml \
NUM_GPUS 1 \
TRAIN.BATCH_SIZE 8 \
SOLVER.BASE_LR 0.0125 \
DATA.PATH_TO_DATA_DIR ./afs/kinetics400 \
DATA.PATH_PREFIX ./afs/kinetics400 \ # Error
}
There is NOT a \ at the end of DATA.PATH_PREFIX ./afs/kinetics400
I just cut-and-pasted your example into a file; it ran fine under bash. I don't see any problems with it.
For good measure you may want to ensure it ends with a newline, though bash shouldn't care. (It runs for me both with and without the final newline.)
You'll sometimes see strange errors if you've accidentally embedded a control character in the file. Since it's a short script, try creating a new script by pasting it from your question here on StackOverflow, or by simply re-typing it.
What version of bash are you using? (bash --version)
Good luck!
Make sure the name of the directory in which the .sh file is present does not have a space character. e.g: Say if it is in a folder called 'New Folder', you're bound to come across the error that you've cited. Instead just name it as 'New_Folder'. I hope this helps.
Apparently, some versions of the shell can also emit this message when the final line of your script lacks a newline.
In Ubuntu:
$ gedit ~/.profile
Then, File -> Save as and set end line to Unix/Linux
I know I am too late to the party. Hope this may help someone.
Check your .bashrc file. Perhaps rename or move it.
Discussion here: Unable to source a simple bash script
For people using MacOS:
If you received a file with Windows format and wanted to run on MacOS and seeing this error, run these commands.
brew install dos2unix
sh <file.sh>
If the the script itself is valid and there are no syntax errors, then some possible causes could be:
Invalid end-of-lines (for example, \r\n instead of \n)
Presence of the byte order mark (BOM) at the beginning of the file
Both can be fixed using vim or vi.
To fix line endings open the file in vim and from the command mode type:
:set ff=unix
To remove the BOM use:
:set nobomb
For those who don't have dos2unix installed (and don't want to install it):
Remove trailing \r character that causes this error:
sed -i 's/\r$//' filename
Details from this StackOverflow answer. This was really helpful.
https://stackoverflow.com/a/32912867/7286223

csh/sh for loop - how to?

i'm trying to write a for loop that executes 2 scripts on FreeBSD. I don't care if it's written in sh or csh. I want something like:
for($i=11; $i<=24; $i++)
{
exec(tar xzf 'myfile-1.0.' . $i);
// detect an error was returned by the script
if ('./patch.sh')
{
echo "Patching to $i failed\n";
}
}
Does anyone know how to do this please?
Thanks
The typical way to do this in sh is:
for i in $(seq 11 24); do
tar xzf "myfile-1.0$i" || exit 1
done
Note that seq is not standard. Depending on the availability of tools, you might try:
jot 14 11 24
or
perl -E 'say for(11..24)'
or
yes '' | nl -ba | sed -n -e 11,24p -e 24q
I've made a few changes: I abort if the tar fails and do not emit an error message, since tar should emit the error message instead of the script.
Wow! No BASH. And probably no Kornshell:
i=11
while [ $i -le 24 ]
do
tar xzf myfile-1.0.$i
i=`expr $i + 1`
if ./patch.sh
then
echo "patching to $i failed"
fi
done
Written in pure Bourne shell just like God intended.
Note you have to use the expr command to add 1 to $i. Bourne shell doesn't do math. The backticks mean to execute the command and put the STDOUT from the command into $i.
Kornshell and BASH make this much easier since they can do math and do more complex for loops.
csh does loops fine, the problem is that you are using exec, which replaces the current program (which is the shell) with a different one, in the same process. Since others have supplied sh versions, here is a csh one:
#!/bin/csh
set i = 11
while ($i &lt 25)
tar xzf "myfile-1.0.$i"
# detect an error was returned by the script
if ({./patch.sh}) then
echo "Patching to $i failed"
endif
# i = $i + 1
end
Not sure about the ./patch.sh are you testing for its existence or running it? I am running it here, and testing the result - true means it returned zero. Alternatively:
# detect an error was returned by the script
if (!{tar xzf "myfile-1.0.$i"}) then
echo "Patching to $i failed"
endif
I think you should just use bash. I don't have it here, so it cannot test it, but something along this should work:
for ((I=11; I<=24; I++)) ; do
tar xzf myfile-1.0.$I || echo Patching to $I failed
done
EDIT: Just read the comments and found out there's no bash in FreeBSD default installation. So this might not work at all - I'm not sure about the differences between (t)csh and bash.
Well what i just did is the following.
sh
Load sh shell and then for loop works like on linux.
for i in 1 2 3 4 5 6 7 8 9; do dd if=/dev/zero of=/tmp/yourfilename$i.test bs=52428800 count=15; done

Resources