I am using a printf command to add some bytes on my file so that it acts like a Byte-Order-Mark.
the following is my SH script
title: add_bom.sh
FILE=$1
printf '\xFF\xFE' >> $1
On my PuTTY terminal, when I do directly
printf '\xFF\xFE' >> test.xls
the result is correct as expected and xxd test.xls displays ff and fe at the first line
However, when I run it via SH
sh add_bom.sh test.xls
the result is wrong and \xFF\xFE appears at the end of test.xls file as a text
Why it this so?
The >> redirection operator always appends to the end of the file.
If you want to prepend, try something like
printf '\xff\xfe' >temp
cat otherfile >>temp
mv temp otherfile
However, adding an UTF-16 BOM to a file which is not a UTF-16 text file in the first place is almost certainly an error.
Related
I have a script which contains the file location and i am running a command to fix the files.
But i am not able to write the output to a file.
#!/bin/ksh
while read line
do
`shnfix "$line"` >> output.txt
done < filename.txt
The output generated after adding set -x in the beginning of the script.
+ < filename.txt
+ read line
+ shnfix /x01/naveen_wav/file1.wav
+ >> output.txt
Fixing [/x01/naveen_wav/file1.wav] (3:49.42) --> [file1-fixed.wav] : 100% OK
Padded last file with 1194 zero-bytes.
+ read line
+ shnfix /x01/naveen_wav/file2.wav
+ >> output.txt
Fixing [/x01/naveen_wav/file2.wav] (4:30.35) --> [file2-fixed.wav] : 100% OK
Padded last file with 644 zero-bytes.
+ read line
A more efficient version (I/O wise) of #gile's code:
#!/bin/ksh
filename="/path/to/filename.txt"
while IFS= read -r line
do
shnfix "$line"
done < filename.txt > output.txt
The output should be inside `
`shnfix $line >> output.txt`
So the script could be like this:
#!/bin/ksh
filename="/path/to/filename.txt"
while IFS= read -r line
do
# display line or do somthing on $line
echo "$line"
`shnfix $line >> output.txt`
done <"$fileName"
Just remove the backticks, they are at least confusing.
Written as it is the shell will exeute the command in a subshell and try to execute the result, adding the redirection to it.
I assume you don't want to execute the output, but you want to redirect the output.
If the output starts with, for example line, that is a correct unix command, which does not create output you don't see an error, but neither the output.
I get a test.ksh[4]: line1: not found [No such file or directory] where 'line1' is the first line in my test file.
Or keep it in a block and redirect all output from it. Makes the intention more clear and it is easier to add commnds.
#!/bin/ksh
filename="/path/to/filename.txt"
{
while IFS= read -r line
do
shnfix "$line"
done < "$filename"
} > output.txt
http://www.shellcheck.net (a big troubleshooter) will give similar hints
So I have this shell script:
#!/bin/bash -xv
PATH=${PATH[*]}:.
#filename: testScript
echo $#
It should print the number of parameters I receive from a text file.
I have a text file(named: file.txt) with one line:
I am a proud sentence
The output should be, as I understood, 5. Since there are 5 words, there are 5 parameters.
I try to run it by:
chmod +x ./testScript.txt
./testScript.txt < ./file.txt > output.txt
But I seem to get in output.txt 0, as if there were no parameters. I really barely understand when do I use $1 $2 to approach parameters, and how to actually send parameters into a script.Should I use pipe? Can it be implented with pipe, anyways?Also. When a text file is passed to a script. Is $1 the text file's name? Will echo $1 print file.txt for the above example, with only the specified arguments done?
< ./file.txt sets standard input of the command line to the content of the file.
read needs to be used to read standard input.
Maybe this script is closer to your needs
#/bin/bash --
printf "%d\n" $#
call it with
./testScript.txt $(cat ./file.txt) > output.txt
$(...) makes the shell ececute the command first. The line in the file is then passed as parameters to the script
----
Otherwise if you use
./testScript.txt ./file.txt
Then $1 is equal to ./file.txt
./testScript.txt < ./file.txt > output.txt
Here testScript.txt has no parameters, zero, none, nothing. The shell parses file redirection before the command runs, and < ./file.txt > output.txt is just file redirection, so the shell "grabs" that part away -- so testScript.txt never knows which files are being redirected from standard input and standard output.
This would work, (i.e. output "5"):
./testScript.txt I am a proud sentence
So would this:
xargs ./testScript.txt < file.txt
...and so forth (see Jay Jargot's answer).
For more info, this article by Mo Budlong should be helpful: Command line psychology 101
I have a script that stores text in a file. Some of that text has bash color espcaed characters that I would like to be used when I display the content of that file in a bash shell. How can this be achieved.
Fox example, ScriptOutput.txt contains
Server is \e[92mRUNNING\e[0m
I would normally cat the file and get the content, but cat will not color the "RUNNING" section of that line in the text file. I also tried
echo $(cat ScriptOutput.txt)
but it will print everything in that file in a single line, which is useless for me. Any ideas how to print the content of that file with the colors specified in each line?
Thank you
I ended up storing the text using echo and then I cat the file line by line and print it also using echo (echo -e)
Example:
echo 'Server1 is not \e[92mAVAILABLE\e[0m' >> scriptOutput.txt
echo 'Server2 is not \e[31mNOT AVAILABLE\e[0m' >> scriptOutput.txt
cat scriptOutput.txt | while read -r line; do echo -e "$line"; done
I needed to have the script output store in a file but I also needed to print its content in the shell later on with colors. That did the trick
Thank you anyway guys
Instead of the string \e, you want to have a literal escape character in your script. How you can enter this depends on your terminal and text editor.
For me (using nano from the OS X terminal, please withhold your disdain!) I press Esc followed by Shift-V. nano tells me it's in "Verbatim Input" mode. Then I can hit the escape key and get a literal escape character (represented on screen by ^[).
This will demonstrate a universal method to insert the escape character using printf:
printf '\033[44mfoo\033[0m\nbar\n\033[92mbaz\033[0m\n' > foo.txt
cat foo.txt
You can pipe your script output through sed and replace there the '\e' character with the hex value of ESC. The result should be colored.
cat ScriptOutput.txt | sed -e 's-\\e-\x1b-g'
What is the easiest way to append text to a file in Linux?
I had a look at this question, but the accepted answer uses an additional program (sed) I'm sure there should be an easier way with echo or similar.
How about:
echo "hello" >> <filename>
Using the >> operator will append data at the end of the file, while using the > will overwrite the contents of the file if already existing.
You could also use printf in the same way:
printf "hello" >> <filename>
Note that it can be dangerous to use the above. For instance if you already have a file and you need to append data to the end of the file and you forget to add the last > all data in the file will be destroyed. You can change this behavior by setting the noclobber variable in your .bashrc:
set -o noclobber
Now when you try to do echo "hello" > file.txt you will get a warning saying cannot overwrite existing file.
To force writing to the file you must now use the special syntax:
echo "hello" >| <filename>
You should also know that by default echo adds a trailing new-line character which can be suppressed by using the -n flag:
echo -n "hello" >> <filename>
References
echo(1) - Linux man page
noclobber variable
I/O Redirection
cat >> filename
This is text, perhaps pasted in from some other source.
Or else entered at the keyboard, doesn't matter.
^D
Essentially, you can dump any text you want into the file. CTRL-D sends an end-of-file signal, which terminates input and returns you to the shell.
Other possible way is:
echo "text" | tee -a filename >/dev/null
The -a will append at the end of the file.
If needing sudo, use:
echo "text" | sudo tee -a filename >/dev/null
Follow up to accepted answer.
You need something other than CTRL-D to designate the end if using this in a script. Try this instead:
cat << EOF >> filename
This is text entered via the keyboard or via a script.
EOF
This will append text to the stated file (not including "EOF").
It utilizes a here document (or heredoc).
However if you need sudo to append to the stated file, you will run into trouble utilizing a heredoc due to I/O redirection if you're typing directly on the command line.
This variation will work when you are typing directly on the command line:
sudo sh -c 'cat << EOF >> filename
This is text entered via the keyboard.
EOF'
Or you can use tee instead to avoid the command line sudo issue seen when using the heredoc with cat:
tee -a filename << EOF
This is text entered via the keyboard or via a script.
EOF
How can I write data to a text file automatically by shell scripting in Linux?
I was able to open the file. However, I don't know how to write data to it.
The short answer:
echo "some data for the file" >> fileName
However, echo doesn't deal with end of line characters (EOFs) in an ideal way. So, if you're going to append more than one line, do it with printf:
printf "some data for the file\nAnd a new line" >> fileName
The >> and > operators are very useful for redirecting output of commands, they work with multiple other bash commands.
#!/bin/sh
FILE="/path/to/file"
/bin/cat <<EOM >$FILE
text1
text2 # This comment will be inside of the file.
The keyword EOM can be any text, but it must start the line and be alone.
EOM # This will be also inside of the file, see the space in front of EOM.
EOM # No comments and spaces around here, or it will not work.
text4
EOM
You can redirect the output of a command to a file:
$ cat file > copy_file
or append to it
$ cat file >> copy_file
If you want to write directly the command is echo 'text'
$ echo 'Hello World' > file
#!/bin/bash
cat > FILE.txt <<EOF
info code info
info code info
info code info
EOF
I know this is a damn old question, but as the OP is about scripting, and for the fact that google brought me here, opening file descriptors for reading and writing at the same time should also be mentioned.
#!/bin/bash
# Open file descriptor (fd) 3 for read/write on a text file.
exec 3<> poem.txt
# Let's print some text to fd 3
echo "Roses are red" >&3
echo "Violets are blue" >&3
echo "Poems are cute" >&3
echo "And so are you" >&3
# Close fd 3
exec 3>&-
Then cat the file on terminal
$ cat poem.txt
Roses are red
Violets are blue
Poems are cute
And so are you
This example causes file poem.txt to be open for reading and writing on file descriptor 3. It also shows that *nix boxes know more fd's then just stdin, stdout and stderr (fd 0,1,2). It actually holds a lot. Usually the max number of file descriptors the kernel can allocate can be found in /proc/sys/file-max or /proc/sys/fs/file-max but using any fd above 9 is dangerous as it could conflict with fd's used by the shell internally. So don't bother and only use fd's 0-9. If you need more the 9 file descriptors in a bash script you should use a different language anyways :)
Anyhow, fd's can be used in a lot of interesting ways.
I like this answer:
cat > FILE.txt <<EOF
info code info
...
EOF
but would suggest cat >> FILE.txt << EOF if you want just add something to the end of the file without wiping out what is already exists
Like this:
cat >> FILE.txt <<EOF
info code info
...
EOF
Moving my comment as an answer, as requested by #lycono
If you need to do this with root privileges, do it this way:
sudo sh -c 'echo "some data for the file" >> fileName'
For environments where here documents are unavailable (Makefile, Dockerfile, etc) you can often use printf for a reasonably legible and efficient solution.
printf '%s\n' '#!/bin/sh' '# Second line' \
'# Third line' \
'# Conveniently mix single and double quotes, too' \
"# Generated $(date)" \
'# ^ the date command executes when the file is generated' \
'for file in *; do' \
' echo "Found $file"' \
'done' >outputfile
I thought there were a few perfectly fine answers, but no concise summary of all possibilities; thus:
The core principal behind most answers here is redirection. Two are important redirection operators for writing to files:
Redirecting Output:
echo 'text to completely overwrite contents of myfile' > myfile
Appending Redirected Output
echo 'text to add to end of myfile' >> myfile
Here Documents
Others mentioned, rather than from a fixed input source like echo 'text', you could also interactively write to files via a "Here Document", which are also detailed in the link to the bash manual above. Those answers, e.g.
cat > FILE.txt <<EOF` or `cat >> FILE.txt <<EOF
make use of the same redirection operators, but add another layer via "Here Documents". In the above syntax, you write to the FILE.txt via the output of cat. The writing only takes place after the interactive input is given some specific string, in this case 'EOF', but this could be any string, e.g.:
cat > FILE.txt <<'StopEverything'` or `cat >> FILE.txt <<'StopEverything'
would work just as well. Here Documents also look for various delimiters and other interesting parsing characters, so have a look at the docs for further info on that.
Here Strings
A bit convoluted, and more of an exercise in understanding both redirection and Here Documents syntax, but you could combine Here Document style syntax with standard redirect operators to become a Here String:
Redirecting Output of cat Input
cat > myfile <<<'text to completely overwrite contents of myfile'
Appending Redirected Output of cat Input
cat >> myfile <<<'text to completely overwrite contents of myfile'
This approach works and is the best
cat > (filename) <<EOF
Text1...
Text2...
EOF
Basically the text will search for keyword "EOF" till it terminates writing/appending the file
If you are using variables, you can use
first_var="Hello"
second_var="How are you"
If you want to concat both string and write it to file, then use below
echo "${first_var} - ${second_var}" > ./file_name.txt
Your file_name.txt content will be "Hello - How are you"
Can also use here document and vi, the below script generates a FILE.txt with 3 lines and variable interpolation
VAR=Test
vi FILE.txt <<EOFXX
i
#This is my var in text file
var = $VAR
#Thats end of text file
^[
ZZ
EOFXX
Then file will have 3 lines as below. "i" is to start vi insert mode and similarly to close the file with Esc and ZZ.
#This is my var in text file
var = Test
#Thats end of text file