I have two files, both of which appear to my eyes as follows.
a
a
The difference is that I created one of them with vim and the other with the wine version of Notepad. I wrote the following script to print each line of these files at a time, more or less emulating cat (That's not my end goal, of course, but it's the simplest example I've thought of.).
#!/usr/bin/env bash
readarray -t list_file < "$1"
for line in "${list_file[#]}"
do
echo "line content: \"$line\""
done
Expectedly, with the file created by vim (5 bytes as expected: a[newline][newline]a[newline]) as $1, it outputs this.
line content: "a"
line content: ""
line content: "a"
Unexpectedly, with the file created by Notepad (It's 6 bytes; I'm not sure why.) as $1, it outputs this. Why does it do this?
"ine content: "a
"ine content: "
line content: "a"
I also tried doing this completely differently, but the following script has exactly the same problem.
#!/usr/bin/env bash
for line_number in $(eval echo {1..$(wc -l < "$1")})
do
echo "line content: $(sed -n "${line_number}p" "$1")"
done
What is the matter with these scripts that causes them to behave like this with the Notepad-created file?
The file created by vim has lines ending with LF, as on Unix. The file created with Notepad has lines ending with CR LF, as on DOS/Windows. bash uses LF as its line delimiter, so when it reads from the second file it leaves the CR at the end of each line. When you echo this character, it causes the cursor to return to the left margin, without advancing to the next line.
Related
I have a simple bash script and I don't understand the return value.
My script
#!bin/bash
string=$(head -n 1 test.txt)
IFS=":"
read -r pathfile line <<< "$string"
echo "left"$line"right"
And my test.txt
filepath:file content
others lines
...
I have this return on the console.
rightfile content
The problem isn't when file only have 1 line.
I don't know why I don't have left value right to result.
Your input file has MSWin line ends (\x0D\x0A). Therefore, \x0D becomes part of $line and when printed, it moves the cursor back to the beginning, so $line"right" overwrites it.
Run dos2unix or fromdos on the input file to fix it.
BTW, you don't need to quote left and right. Quoting the variable might be needed, though.
echo left"$line"right
Each line in a given file 'a.txt' contains the directory/path to another unique file. Suppose we want to parse 'a.txt' line-by-line, extract the path in string format, and then use a tool such as vim to process the file at this path, and so on.
After going through this thread - Read a file line by line assigning the value to a variable, I wrote the following script, say 'open-file.sh' on bash (I'm new to it)
#!/bin/bash
while IFS='' read -r line || [[ -n "$line" ]]; do
vim -c ":q" -cq $line # Just open the file and close it using :q
done < "$1"
We would then run the above script as -
./open-file.sh a.txt
The problem is that although the path to a new file is correctly specified by $line, when vim opens the file, vim continues to receive the text contained in 'a.txt' as a command. How can I write a script where I can correctly obtain the path from 'a.txt', open it using vim, and then continue parsing the remaining lines in 'a.txt' ?
Replace:
vim -c ":q" -cq $line
With:
vim -c ":q" -cq "$line" </dev/tty
The redirection </dev/tty tells vim to take its standard input from the terminal. Without that, the standard input for vim is "$1".
Also, it is good practice to put $line in double-quotes to protect it from word splitting, etc.
Lastly, while vim is excellent for interactive work, if your end-goal is fully automated processing of each file, you might want to consider tools such as sed or awk.
Although I'm not sure of your ultimate goal, this shell command will execute vim once per line in a.txt:
xargs -o -n1 vim -c ':q' < a.txt
As explained in the comments to Read a file line by line assigning the value to a variable, the issue you're encountering is due to the fact that vim is an interactive program and thus continues to read input from $line.
The problem was already mentioned in a comment under the answer you based your script on.
vim is consuming stdin which is given to the loop by done < $1. We can observe the same behavior in the following example:
$ while read i; do cat; done < <(seq 3)
2
3
<(seq 3) simulates a file with the three lines 1, 2, and 3. Instead of three silent iterations we get only one iteration and the output 2 and 3.
stdin is not only passed to read in the head of the loop, but also to cat in the body of the loop. Therefore read reads one line, the loop is entered, cat reads all remaining lines, stdin is empty, read has nothing to read anymore, the loop exits.
You could circumvent the problem by redirecting something to vim, however there is an even better way. You don't need the loop at all:
< "$1" xargs -d\\n -n1 vim -c :q -cq
xargs will execute vim once for every line in the file given by $1.
I have this code
TOKEN=$(cat ./config/token)
echo "$TOKEN"
cat > variables.env <<EOF
TOKEN=`echo "$TOKEN"`
EOF
I am trying to get the content of a file and output it in a new file prefixed by some text. The first echo in the console echoes the output I want, keeping the whitespaces and newlines.
However, in the new file the output is just the first line of the original string, while I'd like the same output I can see in the console with the first echo.
Use printf %q (in ksh or bash) to escape content in such a way that it will always evaluate back to its literal value:
printf 'TOKEN=%q\n' "$(<./config/token)" >variables.env
$(<file) is a ksh and bash extension which acts as a more efficient replacement for $(cat file) (as the regular command substitution needs to fork off a subprocess, set up a FIFO, and spawn an external copy of /bin/cat, whereas the $(<file) form simply tells the shell to read the file directly).
This way a taken containing an otherwise-hostile string such as $(rm -rf ~) or content that could simply be expanded as a variable ($$) will be emitted as literal content.
Providing an explicit example of how this behaves:
printf '%s\n' "first line" "second line" >token # write two lines to the file "token"
printf 'TOKEN=%q\n' "$(<token)" >variables.env # write a shell command which assigns those
# two lines to a variable to variables.env
source variables.env # execute variables.env in the current shell
echo "$TOKEN" # emit the value of TOKEN, as given in the current shell
...when run with bash, will emit the exact output:
first line
second line
...after writing the following (with bash 3.2.48; may vary with other releases) to variables.env:
TOKEN=$'first line\nsecond line'
Useless use of echo
This is what you could write:
cat > variables.env <<EOF
TOKEN=${TOKEN}
EOF
you are doing it in a very convoluted way, there are easier methods
sed '1s/./TOKEN=&/' file > newfile
will insert TOKEN= on the first line. This has an additional benefit of not modifying empty files (at least one char should exist in the original file). If that's not intended you can use unconditional insert.
You can do:
echo "TOKEN=" > newfile && cat ./config/token >> newfile
>> appends to a file.
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE:vitest filename"
else
FILENAME=$1
exec vi $FILENAME <<EOF
i
Line 1.
Line 2.
^[
ZZ
EOF
fi
exit 0
I'm trying to input the Line 1. and Line 2. with Exec vi using the here doc, and commands.
When running the script it gives me the following:
Vim(?):Warning: Input is not from a terminal
Vim: Error reading input, exiting...
Press ENTER or type command to continueVim: Finished.
Vim: Error reading input, exiting...
Vim: Finished.
You want to start vi in ex mode, with a few minor changes to the script.
vi -e "$FILENAME" <<EOF
i
Line 1.
Line 2.
.
wq
EOF
exec is almost certainly unnecessary, especially since you have an exit command following vi. exec is used to replace the current script with the given command; it is not needed simply to execute a command.
A brief history of UNIX text editors:
ed was the original editor, designed to work with a teletype rather than a video terminal.
ex was an extended version of ed, designed to take advantage of a video terminal.
vi was an editor that provided ex with a full-screen visual mode, in contrast with the line-oriented interface employed by ed and ex.
As suggested, ed
ed file << END
1i
line1
line2
.
wq
END
The "dot" line means "end of input".
It can be written less legibly as a one-liner
printf "%s\n" 1i "line1" "line2" . wq | ed file
Use cat.
$ cat file1.txt file2.txt | tee file3.txt
Line 1
Line 2
aaaa
bbbb
cccc
Using sed
If I understand correctly, you want to add two lines to the beginning of a file. In that case, as per Cyrus' suggestion, run:
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE:vitest filename"
exit 1
fi
sed -i.bak '1 s/^/line1\nline2\n/' "$1"
Notes:
When a shell variable is used, it should be in double-quotes unless you want word splitting and pathname expansion to be performed. This is important for file names, for example, as it is now common for them to contain whitespace.
It is best practice to use lower or mixed case names for shell variables. The system uses upper case names for its variables and you don't want to overwrite one of them accidentally.
In the check for the argument, the if statement should include an exit to prevent the rest of the script from being run in the case that no argument was provided. In the above, we added exit 1 which sets the exit code to 1 to signal an error.
Using vi
Let's start with this test file:
$ cat File
some line
Now, let's run vi and see what is in File afterward:
$ vi -s <(echo $'iline1\nline2\n\eZZ') File
$ cat File
line1
line2
some line
The above requires bash or similar.
I have a script that stores text in a file. Some of that text has bash color espcaed characters that I would like to be used when I display the content of that file in a bash shell. How can this be achieved.
Fox example, ScriptOutput.txt contains
Server is \e[92mRUNNING\e[0m
I would normally cat the file and get the content, but cat will not color the "RUNNING" section of that line in the text file. I also tried
echo $(cat ScriptOutput.txt)
but it will print everything in that file in a single line, which is useless for me. Any ideas how to print the content of that file with the colors specified in each line?
Thank you
I ended up storing the text using echo and then I cat the file line by line and print it also using echo (echo -e)
Example:
echo 'Server1 is not \e[92mAVAILABLE\e[0m' >> scriptOutput.txt
echo 'Server2 is not \e[31mNOT AVAILABLE\e[0m' >> scriptOutput.txt
cat scriptOutput.txt | while read -r line; do echo -e "$line"; done
I needed to have the script output store in a file but I also needed to print its content in the shell later on with colors. That did the trick
Thank you anyway guys
Instead of the string \e, you want to have a literal escape character in your script. How you can enter this depends on your terminal and text editor.
For me (using nano from the OS X terminal, please withhold your disdain!) I press Esc followed by Shift-V. nano tells me it's in "Verbatim Input" mode. Then I can hit the escape key and get a literal escape character (represented on screen by ^[).
This will demonstrate a universal method to insert the escape character using printf:
printf '\033[44mfoo\033[0m\nbar\n\033[92mbaz\033[0m\n' > foo.txt
cat foo.txt
You can pipe your script output through sed and replace there the '\e' character with the hex value of ESC. The result should be colored.
cat ScriptOutput.txt | sed -e 's-\\e-\x1b-g'