Bash script to remove text from each line of a txt before a : - bash

I have written this script to remove text from each line before ::
#!/bin/bash
txt=test.txt
COUNT=$(cat $txt | wc -l)
while [ $COUNT -gt 0 ]; do
data=$(sed -n ${count}p $txt)
sed '$count \c
"${data#*:}"' $txt
let COUNT=COUNT-1
done
I think I have an issue with using variables in commands without spaces. Can anyone tell me what I have done wrong?

I think you are over complicating it. To do this you just need cut:
cut -d':' -f2- file
-d sets the field separator.
-f indicates what fields to use. By saying 2- we indicate "all from the 2nd one on".
Test
$ cat a
hello
hello:man i am here:or there
and:you are here
$ cut -d':' -f2- a
hello
man i am here:or there
you are here
Some comments regarding your code:
#!/bin/bash
txt=test.txt
COUNT=$(cat $txt | wc -l) # you can directly say 'wc -l < "$txt"'
while [ $COUNT -gt 0 ]; do
data=$(sed -n ${count}p $txt) # you are using "count", not "COUNT"
sed '$count \c # same here. And I don't know what
"${data#*:}"' $txt # this sed is supposed to work like
let COUNT=COUNT-1 # you have to say let "COUNT=COUNT-1"
done
Also, it is good to indent the code, so that it shows like:
while ...
do
... things ...
done
All together, I would do:
#!/bin/bash
txt=a
count=$(wc -l < "$txt")
while (( count-- > 0 )); do
data=$(sed -n "${count}p" "$txt")
#sed '$COUNT \c "${data#*:}"' $txt # not using it
echo "${data#*:}"
done
Since you are reading the file from the bottom and done some conditions around it, you could just drop it and just use tac to print the file on reverse:
while IFS= read -r data do
echo "${data#*:}"
done < <(tac file)

Related

Shell: Add string to the end of each line, which match the pattern. Filenames are given in another file

I'm still new to the shell and need some help.
I have a file stapel_old.
Also I have in the same directory files like english_old_sync, math_old_sync and vocabulary_old_sync.
The content of stapel_old is:
english
math
vocabulary
The content of e.g. english is:
basic_grammar.md
spelling.md
orthography.md
I want to manipulate all files which are given in stapel_old like in this example:
take the first line of stapel_old 'english', (after that math, and so on)
convert in this case english to english_old_sync, (or after that what is given in second line, e.g. math to math_old_sync)
search in english_old_sync line by line for the pattern '.md'
And append to each line after .md :::#a1
The result should be e.g. of english_old_sync:
basic_grammar.md:::#a1
spelling.md:::#a1
orthography.md:::#a1
of math_old_sync:
geometry.md:::#a1
fractions.md:::#a1
and so on. stapel_old should stay unchanged.
How can I realize that?
I tried with sed -n, while loop (while read -r line), and I'm feeling it's somehow the right way - but I still get errors and not the expected result after 4 hours inspecting and reading.
Thank you!
EDIT
Here is the working code (The files are stored in folder 'olddata'):
clear
echo -e "$(tput setaf 1)$(tput setab 7)Learning directories:$(tput sgr 0)\n"
# put here directories which should not become flashcards, command: | grep -v 'name_of_directory_which_not_to_learn1' | grep -v 'directory2'
ls ../ | grep -v 00_gliederungsverweise | grep -v 0_weiter | grep -v bibliothek | grep -v notizen | grep -v Obsidian | grep -v z_nicht_uni | tee olddata/stapel_old
# count folders
echo -ne "\nHow much different folders: " && wc -l olddata/stapel_old | cut -d' ' -f1 | tee -a olddata/stapel_old
echo -e "Are this learning directories correct? [j ODER y]--> yes; [Other]-->no\n"
read lernvz_korrekt
if [ "$lernvz_korrekt" = j ] || [ "$lernvz_korrekt" = y ];
then
read -n 1 -s -r -p "Learning directories correct. Press any key to continue..."
else
read -n 1 -s -r -p "Learning directories not correct, please change in line 4. Press any key to continue..."
exit
fi
echo -e "\n_____________________________\n$(tput setaf 6)$(tput setab 5)Found cards:$(tput sgr 0)$(tput setaf 6)\n"
#GET && WRITE FOLDER NAMES into olddata/stapel_old
anzahl_zeilen=$(cat olddata/stapel_old |& tail -1)
#GET NAMES of .md files of every stapel and write All to 'stapelname'_old_sync
i=0
name="var_$i"
for (( num=1; num <= $anzahl_zeilen; num++ ))
do
i="$((i + 1))"
name="var_$i"
name=$(cat olddata/stapel_old | sed -n "$num"p)
find ../$name/ -name '*.md' | grep -v trash | grep -v Obsidian | rev | cut -d'/' -f1 | rev | tee olddata/$name"_old_sync"
done
(tput sgr 0)
I tried to add:
input="olddata/stapel_old"
while IFS= read -r line
do
sed -n "$line"p olddata/stapel_old
done < "$input"
The code to change only the english_old_sync is:
lines=$(wc -l olddata/english_old_sync | cut -d' ' -f1)
for ((num=1; num <= $lines; num++))
do
content=$(sed -n "$num"p olddata/english_old_sync)
sed -i "s/"$content"/""$content":::#a1/g"" olddata/english_old_sync
done
So now, this need to be a inner for-loop, of a outer for-loop which holds the variable for english, right?
stapel_old should stay unchanged.
You could try a while + read loop and embed sed inside the loop.
#!/usr/bin/env bash
while IFS= read -r files; do
echo cp -v "$files" "${files}_old_sync" &&
echo sed '/^.*\.md$/s/$/:::#a1/' "${files}_old_sync"
done < olddata/staple_old
convert in this case english to english_old_sync, (or after that what is given in second line, e.g. math to math_old_sync)
cp copies the file with a new name, if the goal is renaming the original file name from the content of the file staple_old then change cp to mv
The -n and -i flag from sed was ommited , include it, if needed.
The script also assumes that there are no empty/blank lines in the content of staple_old file. If in case there are/is add an addition test after the line where the do is.
[[ -n $files ]] || continue
It also assumes that the content of staple_old are existing files. Just in case add an additional test.
[[ -e $files ]] || { printf >&2 '%s no such file or directory.\n' "$files"; continue; }
Or an if statement.
if [[ ! -e $files ]]; then
printf >&2 '%s no such file or directory\n' "$files"
continue
fi
See also help test
See also help continue
Combining them all together should be something like:
#!/usr/bin/env bash
while IFS= read -r files; do
[[ -n $files ]] || continue
[[ -e $files ]] || {
printf >&2 '%s no such file or directory.\n' "$files"
continue
}
echo cp -v "$files" "${files}_old_sync" &&
echo sed '/^.*\.md$/s/$/:::#a1/' "${files}_old_sync"
done < olddata/staple_old
Remove the echo's If you're satisfied with the output so the script could copy/rename and edit the files.

I need to add a consecutive index to a searchable string inside a file

So I have a file called, say, 'test', with the contents:
39366
39371
45005
45005
216274
216277
216345
396480
396480
I need to add an index to the end of every string, which should correspond to the count of that string inside the file. It should look like this:
39366_1
39371_1
45005_1
45005_2
216274_1
216277_1
216345_1
396480_1
396480_2
396480_2
396480_3
Then I should repeat the process in another file, say 'test2' in a more complicated format, something like this:
+39366
ffasd
+39371
fasdasd
+45005
fasdfdf
+45005
asdfasdf
My first question here, I've never asked for anything, please help :)
$ cat script.sh
#!/bin/bash
FILE=$1
n=$(cat $FILE | wc -l)
for (( i=1; i<=$n; i++ ))
do
LINE=$(sed "${i}q;d" $FILE)
if [ -n "$LINE" ]
then
INDEX=$(cat test | head -n $i | grep -c "$LINE")
LINE="${LINE}_${INDEX}"
sed -i "${i}s/.*/$LINE/" $FILE
fi
done
$ cat test
39366
39371
45005
45005
216274
216277
216345
396480
396480
396480
396480
$ ./script.sh test
$ cat test
39366_1
39371_1
45005_1
45005_2
216274_1
216277_1
216345_1
396480_1
396480_2
396480_3
396480_4

How to check that a file has more than 1 line in a BASH conditional?

I need to check if a file has more than 1 line. I tried this:
if [ `wc -l file.txt` -ge "2" ]
then
echo "This has more than 1 line."
fi
if [ `wc -l file.txt` >= 2 ]
then
echo "This has more than 1 line."
fi
These just report errors. How can I check if a file has more than 1 line in a BASH conditional?
The command:
wc -l file.txt
will generate output like:
42 file.txt
with wc helpfully telling you the file name as well. It does this in case you're checking out a lot of files at once and want individual as well as total stats:
pax> wc -l *.txt
973 list_of_people_i_must_kill_if_i_find_out_i_have_cancer.txt
2 major_acheivements_of_my_life.txt
975 total
You can stop wc from doing this by providing its data on standard input, so it doesn't know the file name:
if [[ $(wc -l <file.txt) -ge 2 ]]
The following transcript shows this in action:
pax> wc -l qq.c
26 qq.c
pax> wc -l <qq.c
26
As an aside, you'll notice I've also switched to using [[ ]] and $().
I prefer the former because it has less issues due to backward compatibility (mostly to do with with string splitting) and the latter because it's far easier to nest executables.
A pure bash (≥4) possibility using mapfile:
#!/bin/bash
mapfile -n 2 < file.txt
if ((${#MAPFILE[#]}>1)); then
echo "This file has more than 1 line."
fi
The mapfile builtin stores what it reads from stdin in an array (MAPFILE by default), one line per field. Using -n 2 makes it read at most two lines (for efficiency). After that, you only need to check whether the array MAPFILE has more that one field. This method is very efficient.
As a byproduct, the first line of the file is stored in ${MAPFILE[0]}, in case you need it. You'll find out that the trailing newline character is not trimmed. If you need to remove the trailing newline character, use the -t option:
mapfile -t -n 2 < file.txt
if [ `wc -l file.txt | awk '{print $1}'` -ge "2" ]
...
You should always check what each subcommand returns. Command wc -l file.txt returns output in the following format:
12 file.txt
You need first column - you can extract it with awk or cut or any other utility of your choice.
How about:
if read -r && read -r
then
echo "This has more than 1 line."
fi < file.txt
The -r flag is needed to ensure line continuation characters don't fold two lines into one, which would cause the following file to report one line only:
This is a file with _two_ lines, \
but will be seen as one.
change
if [ `wc -l file.txt` -ge "2" ]
to
if [ `cat file.tex | wc -l` -ge "2" ]
If you're dealing with large files, this awk command is much faster than using wc:
awk 'BEGIN{x=0}{if(NR>1){x=1;exit}}END{if(x>0){print FILENAME,"has more than one line"}else{print FILENAME,"has one or less lines"}}' file.txt

count words in a file without using wc

Working in a shell script here, trying to count the number of words/characters/lines in a file without using the wc command. I can get the file broken into lines and count those easy enough, but I'm struggling here to get the words and the characters.
#define word_count function
count_stuff(){
c=0
w=0
l=0
local f="$1"
while read Line
do
l=`expr $line + 1`
# now that I have a line I want to break it into words and characters???
done < "$f"
echo "Number characters: $chars"
echo "Number words: $words"
echo "Number lines: $line"
}
As for characters, try this (adjust echo "test" to where you get your output from):
expr `echo "test" | sed "s/./ + 1/g;s/^/0/"`
As for lines, try this:
expr `echo -e "test\ntest\ntest" | sed "s/^.*$/./" | tr -d "\n" | sed "s/./ + 1/g;s/^/0/"`
===
As for your code, you want something like this to count words (if you want to go at it completely raw):
while read line ; do
set $line ;
while true ; do
[ -z $1 ] && break
l=`expr $l + 1`
shift ;
done ;
done
You can do this with the following Bash shell script:
count=0
for var in `cat $1`
do
count=`echo $count+1 | bc`
done
echo $count

Move certain lines to the preceding line

I have a list that looks like this:
sharename:shareX
comment:commentX
sharename:shareY
comment:commentY
sharename:shareZ
comment:commentZ
and so on...
And this is how I would like the list to look like:
shareX;commentX
shareY;commentY
shareZ;commentZ
How can I accomplish that in bash?
Pure Bash:
IFS=':'
while read a b; read c d; do # read 2 lines
echo -e "$b:$d"
done < "$infile"
one liner:
odd=0; for i in `cat list | cut -d":" -f2`; do if [ $odd -eq 0 ]; then echo -ne $i; odd=1; else echo $i; odd=0; fi; done
formatted:
odd=0;
for i in `cat list | cut -d":" -f2`;
do
if [ $odd -eq 0 ];
then
echo -ne $i";";
odd=1;
else
echo $i;
odd=0;
fi;
done
untested, the sed part may be wrong
paste -d ';' - - < filename | sed -r 's/(^|;)[^:]:/\1/g'
This might work for you:
sed '$!N;s/[^:]*:\([^\n]*\)\n[^:]*:/\1;/' file
shareX;commentX
shareY;commentY
shareZ;commentZ

Resources