src:unix programming environment
a shell script ./bundle :it works well with text file but binary code :(
#!/bin/bash
echo '# To unbundle , sh this file'
for i
do
echo "echo $i 1>&2"
echo "cat >$i <<'End of $i'"
cat $i
echo "End of $i"
done
a simple binary file named a.out,
% ./bundle a.out >> out
% chmod u+x out
% ./out # error happened
I got:
./out: line 8: warning: here-document at line 3 delimited by end-of-file (wanted `End of a.out')
yeah, I know the `End of a.out' should be in a seprate line. But I don't know how to implemetate it.
who have some idea?
thx in advance. :D
You can uuencode binary files (as shar does), or do as makeself does, appending the binary as is and reading it out with dd.
(You could obviously also use either of these tools)
Related
I have the below shell script. Please help me to understand why the values of $mft_dir and $mft_dir1 are different?
SCRIPT(test.sh)
#! /bin/sh
######## Reading properties file
function getProperty {
awk -F'=' -v k="$1" '$1==k&&sub(/^[^=]*=/,"")' $ENVIRONMENTROOT/properties/ExtServerConnection.properties
}
key=$2
######################## LOCAL Server Details ##########################
########MFT path
mft_dir=$(getProperty "${key}_mft_dir")
mft_dir1="/sjcqa01/exec/ENVIRONMENTROOT/EAIDATA/Inbound/MFT/EXL"
echo $mft_dir
echo $mft_dir1
if [ "$mft_dir" != "$mft_dir1" ] ; then
echo "Not equal"
fi
echo "------------------ END OF THE SCRIPT ---------------------"
Properties File Entry
XYZ_mft_dir=/sjcqa01/exec/ENVIRONMENTROOT/EAIDATA/Inbound/MFT/EXL
Execution Command
./test.sh -key XYZ
Output
/Inbound/MFT
/Inbound/MFT
Not equal
Replace your #! /bin/sh line with #!/bin/bash.
I did that with your code and it worked perfectly.
./so.bash -key XYZ
>>/sjcqa01/exec/ENVIRONMENTROOT/EAIDATA/Inbound/MFT/EXL<<
>>/sjcqa01/exec/ENVIRONMENTROOT/EAIDATA/Inbound/MFT/EXL<<
------------------ END OF THE SCRIPT ---------------------
Note I added >> and << characters to see the exact output of the echo commands.
Note2: some systems might require #!/usr/bin/bash. Adjust according to your system (which bash will show you where it is).
Note3: do not put a space between #! and your shell path.
Note4: well done using $() and not backticks :-)
I used the following sed command to strip off the carriage return and my issue got resolved:
sed -e 's/\r//g'
I am using a printf command to add some bytes on my file so that it acts like a Byte-Order-Mark.
the following is my SH script
title: add_bom.sh
FILE=$1
printf '\xFF\xFE' >> $1
On my PuTTY terminal, when I do directly
printf '\xFF\xFE' >> test.xls
the result is correct as expected and xxd test.xls displays ff and fe at the first line
However, when I run it via SH
sh add_bom.sh test.xls
the result is wrong and \xFF\xFE appears at the end of test.xls file as a text
Why it this so?
The >> redirection operator always appends to the end of the file.
If you want to prepend, try something like
printf '\xff\xfe' >temp
cat otherfile >>temp
mv temp otherfile
However, adding an UTF-16 BOM to a file which is not a UTF-16 text file in the first place is almost certainly an error.
I'm trying to compare the contents of two files in a bash script.
local_file=$(cat my_local_file.txt)
remote_file=$(curl -s "http://example.com/remote-file.txt")
if [ local_file == remote_file ]; then
echo "Files are the same"
else
echo "Files are different. Here is the diff:"
diff <(echo "$local_file") <(echo "$remote_file")
fi
When I run the script, I see that I have a syntax error:
./bin/check_files.sh: line 8: syntax error near unexpected token `('
./bin/check_files.sh: line 8: ` diff <(echo "$local_file") <(echo "$remote_file")'
What am I doing wrong? How can I display a diff of these two strings from a bash script?
Process substitution is a bash feature, which is usually not available in /bin/sh which is meant to be POSIX compatible.
Make sure to use the following shebang line if you want to run the script as an executable:
#!/bin/bash
instead of
#!/bin/sh
or use
bash script.sh
instead of
sh script.sh
if you run it like that
To make the script work with POSIX conform shells I would just download the file and compare it against the local file. Remove the downloaded file after the diff.
In addition to the <(command) (process substitution) syntax issue, your code if [ local_file == remote_file ] compares the literal strings local_file and remote_file, rather than the content of the variables. You need $local_file and $remote_file to compare the contents. Need to enclose them in double quotes to prevent word splitting issues.
You could do this:
#!/bin/bash
local_file=$(< my_local_file.txt) # this is more efficient than $(cat file)
remote_file=$(curl -s "http://example.com/remote-file.txt")
if [ "$local_file" = "$remote_file" ]; then
echo "Files are the same"
else
echo "Files are different. Here is the diff:"
diff <(printf '%s' "$local_file") <(printf '%s' "$remote_file")
fi
As stated by #dimo414, the limitation here is that the command substitution $(...) removes trailing newlines and that would cause a problem. So, it is better to download the remote file and compare it with the local file:
local_file=my_local_file.txt
curl -s "http://example.com/remote-file.txt" -o remote_file
if diff=$(diff -- "$local_file" remote_file); then
echo "Files are the same"
else
echo "Files are different. Here is the diff:"
printf '%s' "$diff"
fi
You can also use the following command:
cmp -b "File_1.txt" "File_2.txt"
I get the below error while trying to execute a shell script,
$'\r': command not found: line 2:
Please suggest a solution for the same.
Below are the intial lines used in the script,
#!/bin/sh
if [[ $# -lt 1 ]]; then
echo "ERROR Environment argument missing <dev,test,qa,prod>"
export RC=50
exit $RC
fi
Your problem is that the file has Windows line endings. This can be caused by editing a file in Windows and trying to run it on a non-Windows system.
You can fix this problem using dos2unix to convert the line endings:
dos2unix ConstruedTermsXMLGenerator.sh
The corresponding utility to convert in the other direction is unix2dos.
Some systems have fromdos and todos.
You can use sed -i 's/\r$//' scriptname.sh
Replace the scriptname with actual script name.
I used notepad++ to convert the line endings.
Edit > EOL Conversion > UNIX/OSX Format
I had the same error and what I did was to transform the characters '\r' to '\n'. using this line:
tr '\r' '\n' < oldfile.sh > newfile.sh
mv newfile.sh oldfile.sh
chmod +x oldfile.sh
./oldfile.sh
I think you could also delete the '\r' characters by using:
tr -d '\r' < oldfile.sh > newfile.sh
tr is the command trasnform, and the -d is delete the following character.
I think the shell actually doesn't like '\r' character.
I had this exact issue when creating a .sh file on a Mac (unix) and executing it in Linux.
Turns out that I had to set FileZilla FTP settings to 'Binary' transfer type:
"Settings>Transfers>File Types>Default transfer type" to "Binary" (instead of "Auto")
I got a different error message when running your script under /bin/sh, but when I switched to /bin/bash, it worked fine:
$ cat foo.sh
#!/bin/sh
if [[ $# -lt 1 ]];
then echo "ERROR Environment argument missing"
RC=50
exit $RC
fi
$ sh foo.sh
foo.sh: 6: [[: not found
$ bash foo.sh
ERROR Environment argument missing
You've built in a bashism. This may or may not be a big deal for your organization. If you want to keep using bash-specific features, change the shebang line to #!/bin/bash and see if that helps.
You can just replace '\r' with '' (nothing), this will do the trick.
If you have any GUI editor (Notepad++) recommended directly find/replace.
If you are only doing it on Linux server then use something like:
sed -i 's/old-text/new-text/g' input.txt
But Since '\r' comes only if you have edited it in windows as it's a new line character (\r\n), so first method should work.
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE:vitest filename"
else
FILENAME=$1
exec vi $FILENAME <<EOF
i
Line 1.
Line 2.
^[
ZZ
EOF
fi
exit 0
I'm trying to input the Line 1. and Line 2. with Exec vi using the here doc, and commands.
When running the script it gives me the following:
Vim(?):Warning: Input is not from a terminal
Vim: Error reading input, exiting...
Press ENTER or type command to continueVim: Finished.
Vim: Error reading input, exiting...
Vim: Finished.
You want to start vi in ex mode, with a few minor changes to the script.
vi -e "$FILENAME" <<EOF
i
Line 1.
Line 2.
.
wq
EOF
exec is almost certainly unnecessary, especially since you have an exit command following vi. exec is used to replace the current script with the given command; it is not needed simply to execute a command.
A brief history of UNIX text editors:
ed was the original editor, designed to work with a teletype rather than a video terminal.
ex was an extended version of ed, designed to take advantage of a video terminal.
vi was an editor that provided ex with a full-screen visual mode, in contrast with the line-oriented interface employed by ed and ex.
As suggested, ed
ed file << END
1i
line1
line2
.
wq
END
The "dot" line means "end of input".
It can be written less legibly as a one-liner
printf "%s\n" 1i "line1" "line2" . wq | ed file
Use cat.
$ cat file1.txt file2.txt | tee file3.txt
Line 1
Line 2
aaaa
bbbb
cccc
Using sed
If I understand correctly, you want to add two lines to the beginning of a file. In that case, as per Cyrus' suggestion, run:
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE:vitest filename"
exit 1
fi
sed -i.bak '1 s/^/line1\nline2\n/' "$1"
Notes:
When a shell variable is used, it should be in double-quotes unless you want word splitting and pathname expansion to be performed. This is important for file names, for example, as it is now common for them to contain whitespace.
It is best practice to use lower or mixed case names for shell variables. The system uses upper case names for its variables and you don't want to overwrite one of them accidentally.
In the check for the argument, the if statement should include an exit to prevent the rest of the script from being run in the case that no argument was provided. In the above, we added exit 1 which sets the exit code to 1 to signal an error.
Using vi
Let's start with this test file:
$ cat File
some line
Now, let's run vi and see what is in File afterward:
$ vi -s <(echo $'iline1\nline2\n\eZZ') File
$ cat File
line1
line2
some line
The above requires bash or similar.