I have a bunch of files that are incomplete: the last line is missing an EOL character.
What's the easiest way to add the newline, using any tool (awk maybe?)?
To add a newline at the end of a file:
echo >>file
To add a line at the end of every file in the current directory:
for x in *; do echo >>"$x"; done
If you don't know in advance whether each file ends in a newline, test the last character first. tail -c 1 prints the last character of a file. Since command substitution truncates any final newline, $(tail -c 1 <file) is empty if the file is empty or ends in a newline, and non-empty if the file ends in a non-newline character.
for x in *; do if [ -n "$(tail -c 1 <"$x")" ]; then echo >>"$x"; fi; done
Vim is great for that because if you do not open a file in binary mode, it will automatically end the file with the detected line ending.
So:
vim file -c 'wq'
should work, regardless of whether your files have Unix, Windows or Mac end of line style.
echo >> filename
Try it before mass use :)
Related
The content of the script is:
#!/bin/bash
tempconf="/tmp/test.file"
while read line
do
echo $line
done < test.conf > $tempconf
The content of the test.conf is:
[PORT]
tcp_ports=7000-7200
udp_ports=7000-8000, 40000-49999
[H323H]
maxSendThreads=10
maxRecvThreads=10
[SDK]
appPwd=1111111
amsAddress=192.168.222.208:8888
The content of the output file "/tmp/test.file" is:
[PORT]
tcp_ports=7000-7200
udp_ports=7000-8000, 40000-49999
2
maxSendThreads=10
maxRecvThreads=10
[SDK]
appPwd=1111111
amsAddress=192.168.222.208:8888
The question is,why [H323H] turns out to be 2. I'll be appreciated if anyone can explain it to me.
[] has a special meaning for the shell, it just means "a single character taken from any of the characters between the brackets". So when you run
echo [H323H]
the shell looks for a file named or H, or 2, or 3... If at least one file matches, [H323H] is replaced with all the matching file names in the output; otherwise it's reproduced as is.
source: https://unix.stackexchange.com/a/259385
Using quotes around $line would solve your problem without the need to check for files matching those characters (which would make the script not very robust)
#!/bin/bash
tempconf="/tmp/test.file"
while read -r line
do
echo "$line"
done < test.conf > "$tempconf"
The final.txt looks like:
build/create_changes.sh
build/create_changes.sh-meta.xml
src/aura/camping/camping.design
src/aura/camping/camping.design-meta.xml
I would like to replace only replace the files which contains /aura/ in its line to src/aura/camping
if [ -e final.txt ]
then
ARRAY=()
while read CFILE
do
echo Analyzing file `basename $CFILE`
case "$CFILE"
in
*.design) TYPENAME="AuraDefinitionBundle";;
*) TYPENAME="UNKNOWN";;
esac
if [ "$TYPENAME" == "AuraDefinitionBundle" ]
then
if [ $(contains "${ARRAY[#]}" $(basename -- "$(dirname -- "$CFILE")")) != "y" ]
then
echo baseFile from new method cFile $CFILE
CFILENAME="$CFILE"
replace="src/aura/"$(basename -- "$(dirname -- "$CFILE")")
echo checkChanges "${CFILENAME/$CFILENAME/"$replace"}"
CFILE="${CFILENAME/$CFILENAME/"$replace"}"
echo baseFile from after change method cFile $CFILE
else
continue
fi
fi
done < final.txt
else
echo Change file not found!
This works, so I can see this in the window now:
[exec] baseFile from new method cFile src/aura/camping/camping.design
[exec] checkChanges src/aura/camping
[exec] baseFile from after change method cFile src/aura/camping
But the file final.txt does not change:
build/create_changes.sh
build/create_changes.sh-meta.xml
src/aura/camping/camping.design
src/aura/camping/camping.design-meta.xml
I replaced it right there CFILE="${CFILENAME/$CFILENAME/"$replace"}"
Tried this too :
if [ $(contains "${ARRAY[#]}" $(basename -- "$(dirname -- "$CFILE")")) != "y" ]
then
CFILENAME="$CFILE"
ARRAY+=($(basename -- "$(dirname -- "$CFILE")"))
replace="src/aura/"$(basename -- "$(dirname -- "$CFILE")")
#CFILE="${CFILENAME/$CFILENAME/"$replace"}"
sed -i 's/$CFILENAME/$replace/' final.txt
else
continue
fi
Am I missing something more here?
Your attempt has been good for the most parts, but the part involving changing the file content is incorrect in both the attempts.
Problem 1
You were surprised that the below attempt did not replace the contents of the file. It won't.
CFILE="${CFILENAME/$CFILENAME/"$replace"}"
Because the above is a bash internal construct for string replacement. It just replaces the string contents stored in the variable CFILENAME and puts the result to CFILE. No file modification is done at all.
Judging from your inputs your CFILENAME input would be src/aura/camping/camping.design and the replace variable would be src/aura/camping. Doing the above would put the string src/aura/camping to the variable CFILE and not to the file pointed by the variable CFILE.
Problem 2
You seemed to have identified that sed would solve your problem, which is in the right track, but you seemed to have missed a couple of tricks.
You have variables defined in your sed search and replacement parts, but the problem is with the quotes. Variables in bash shell don't expand when single quoted but only with double-quotes.
The next problem is with the de-limiter string used in sed, which is by default /. But remember both your source and replacement strings have / present, so sed would not understand what is the original and replacement text. You need to define a de-limiter that is not / and also any meta-character that is not part of your string. I would recommend | in your case
sed "s|$CFILENAME|$replace|" final.txt
Now the biggest problem is with the -i flag in your sed command which means in-place editing of your files whiles. Remember you are reading the file in a loop with a while read construct line by line and now your editing this file after parsing each line. You are messing up with the shell re-direction in a wrong way here. The ideal way would be re-direct your line by line edits to a temporary file and move it to your original file once the loop is done.
sed "s|$CFILENAME|$replace|" final.txt >> temp_final.txt
Something like above using >> which appends to a file. And once the loop is finished, revert to your original file using mv
mv -- temp_final.txt final.txt
Description of Task & problem
I have dumped a list of files which match certain criteria to a text file. The command i used is:
find . -name "*.logic" | grep -v '.bak' | grep -v 'Project File Backup' > logic_manifest.txt
Filenames with spaces in are proving difficult to automatically open, such as :
./20160314 _ Pop/20160314 _ Pop.logic
I have replaced the spaces with '\ ' to escape them but the open command complains:
The file /Users/daniel/Music/Logic/20160314\ _\ Pop/20160314\ _\ Pop.logic does not exist.
When I copy that parsed path, type open in the terminal and paste it in, the file opens successfully.
My BASH script:
#!/bin/bash
clear
# file full of file paths, gathered using the find command
#logic_manifest.txt
# For keeping track of which line of the file I'm using
COUNTER=0
it=1
while IFS='' read -r line || [[ -n "$line" ]]; do
# Increment iterator
COUNTER=`expr $COUNTER + $it`
# replace spaces with a black-slash and space
line=${line// /<>}
line=${line//<>/'\ '}
# print the file name and the line it is on
echo "Line: $COUNTER $line"
#open the file
open "$line"
# await key press before moving on to next iterator
read input </dev/tty
done < "$1"
Encapsulating the filename in speech-marks has not helped
line=${line// /<>}
line=${line//<>/'\ '}
line="\"$line\""
The file /Users/daniel/Music/Logic/"./20160314\ _\ Pop/20160314\ _\
Pop.logic" does not exist.
Nor did passing "\${line}" to open
Question
What do I need to do to enable the open command to launch the files successfully?
Renaming the directories and filenames is not a viable option at this time.
Spaces in filenames are bad, I know, I put it down to moments of madness
There is absolutely no need whatsoever to replace any characters in line.
This simpler loop should open files just fine:
while IFS='' read -r line; do
((COUNTER++))
echo "Line: $COUNTER $line"
open "$line"
read input </dev/tty
done < "$1"
That's it. Moreover:
Spaces in filenames are bad, I know, I put it down to moments of madness.
There's nothing wrong with spaces in filenames.
You just have to use proper quoting, that's all.
That is, if the file names didn't have spaces and other special characters in them, then you could write open $line and it would work.
Since they contain spaces, you must enclose the variable in double-quotes, as in open "$line".
Actually it's strongly recommended to enclose variables in double-quotes when used in command line arguments.
Hello I am trying to add data in a file at the end but It doesn´t work I have a final.txt with this content
cat,dog,pig
I use cat file1.txt >> final.txt
but I obtain this
cat,dog,pig,car,plane,boat
and I want
cat,dog,pig
car,plane,boat
is it possible to obtain this?
echo >> final.txt && cat file1.txt >> final.txt
Let's first talk about why your original code is not working, because this is something that can cause problems for you in the future: your original final.txt file did not end with a newline (that is, there was no extra blank line at the end of the file). That is a POSIX standard (see: Why should text files end with a newline?), and many programs will not work properly when dealing with a text file that does not end with a newline.
Thus, from now on it would be advisable to always end your text files with an newline.
Now, to solve the case at hand, you could run this code:
echo >> final.txt && cat file1.txt >> final.txt
That will fix the lack of a newline on the last line of the original text and thus allow the echo command to work as expected (add the data in a new line).
Important note: the echo command, by default (in most Bash versions), will add a newline at the end of whatever it is inserting. Therefore, when you run the command "echo >> final.txt", it will add a newline at the end of the last line. This is a built-in feature of echo, and that is why running the code above "fixes" the lack of a newline on the original file.
I have a number (say, 100) of CSV files, out of which some (say, 20) are empty (i.e., 0 bytes file). I would like to concatenate the files into one single CSV file (say, assorted.csv), with the following requirement met:
For each empty file, there must be a blank line in assorted.csv.
It appears that simply doing cat *.csv >> assorted.csv skips the empty files completely in the sense that they do not have any lines and hence there is nothing to concatenate.
Though I can solve this problem using any high-level programming language, I would like to know if and how to make it possible using Bash.
Just make a loop and detect if the file is not empty. If it's empty, just echo the file name+comma in it: that will create a near blank line. Otherwise, prefix each line with the file name+comma.
#!/bin/bash
out=assorted.csv
# delete the file prior to doing concatenation
# or if ran twice it would be counted in the input files!
rm -f "$out"
for f in *.csv
do
if [ -s "$f" ] ; then
#cat "$f" | sed 's/^/$f,/' # cat+sed is too much here
sed "s/^/$f,/" "$f"
else
echo "$f,"
fi
done > $out