I have this following .sh file which searchs for tons of search items in tons of files. But I want to continue the while loop at first result if one of the search items is present in $file. Currently, it is so that a query is matched in all files. The first hit is enough.
How can I do this?
while read file
do
echo $file
grep -o -f searchItems.txt "$file" >> results.txt
done < filelist.txt
Thanks.
You can use break after a successful grep return:
while read -r key; do
while read -r actualFile; do
echo "searching for $key in $actualFile"
grep -o "$key" "$actualFile" >> messageKeysInUse.txt && break
done < filelist.txt
done < allMessageKeysFromDB.txt
It will break out of inner while loop as soon as a grep succeeds.
Related
I am trying to loop through all the files in a folder and add the file name of those files with 10 lines to a txt file but I don't know how to write the if statement.
As of right now, what I have is:
for FILE in *.txt do if wc $FILE == 10; then "$FILE" >> saved_names.txt fi done
I am getting stuck in how to format the statement that will evaluate to a boolean for the if statement.
I have already tried the if statement as:
if [ wc $FILE != 10 ]
if "wc $FILE" != 10
if "wc $FILE != 10"
as well as other ways but I don't seem to get it right. I know I am new to Bash but I can't seem to find a solution to this question.
There are a few problems in your code.
To count the number of lines in the file you should run "wc -l" command. However, that command will result in the number of lines and the name of the file (so for example - 10 a.txt - you can test it by running the command on a file in your terminal). To receive only the number of lines you need to pass the file's name to the standard input of that command
"==" is used in bash to compare strings. To compare integers as in that case, you should use "-eq" (take a look here https://tldp.org/LDP/abs/html/comparison-ops.html)
In terms of brackets: To get the wc command result you need to run it in a terminal and switch the command in the code to the result. To do that, you need correct brackets - $(wc -l). To receive a result of the comparison as a bool, you need to use square brackets with spaces [ 1 -eq 1 ].
To save the name of the file in another file using >> you need to first put the name to the standard output (as >> redirect the standard output to the chosen place). To do that you can just use the echo command.
The code should look like this:
#!/bin/bash
for FILE in *.txt
do
if [ "$(wc -l < "$FILE")" -eq 10 ]
then
echo "$FILE" >> saved_names.txt
fi
done
Try:
for file in *.txt; do
if [[ $(wc -l < "$file") -eq 10 ]]; then
printf '%s\n' "$file"
fi
done > saved_names.txt
Change > to >> if you want to append the filenames.
Related docs:
Command Substitution
Conditional Constructs
Extract the actual number of lines from a file with wc -l $FILE | cut -f1 -d' ' and use -eq operator:
for FILE in *.txt; do if [ "$(wc -l $FILE | cut -f1 -d' ')" -eq 10 ]; then "$FILE" >> saved_names.txt; fi; done
I have a textfile called log.txt, and it logs the file name and the path it was gotten from. so something like this
2.txt
/home/test/etc/2.txt
basically the file name and its previous location. I want to use grep to grab the file directory save it as a variable and move the file back to its original location.
for var in "$#"
do
if grep "$var" log.txt
then
# code if found
else
# code if not found
fi
this just prints out to the console the 2.txt and its directory since the directory has 2.txt in it.
thanks.
Maybe flip the logic to make it more efficient?
f=''
while read prev
do case "$prev" in
*/*) f="${prev##*/}"; continue;; # remember the name
*) [[ -e "$f" ]] && mv "$f" "$prev";;
done < log.txt
That walks through all the files in the log and if they exist locally, move them back. Should be functionally the same without a grep per file.
If the name is always the same then why save it in the log at all?
If it is, then
while read prev
do f="${prev##*/}" # strip the path info
[[ -e "$f" ]] && mv "$f" "$prev"
done < <( grep / log.txt )
Having the file names on the same line would significantly simplify your script. But maybe try something like
# Convert from command-line arguments to lines
printf '%s\n' "$#" |
# Pair up with entries in file
awk 'NR==FNR { f[$0]; next }
FNR%2 { if ($0 in f) p=$0; else p=""; next }
p { print "mv \"" p "\" \"" $0 "\"" }' - log.txt |
sh
Test it by replacing sh with cat and see what you get. If it looks correct, switch back.
Briefly, something similar could perhaps be pulled off with printf '%s\n' "$#" | grep -A 1 -Fxf - log.txt but you end up having to parse the output to pair up the output lines anyway.
Another solution:
for f in `grep -v "/" log.txt`; do
grep "/$f" log.txt | xargs -I{} cp $f {}
done
grep -q (for "quiet") stops the output
I don't understand why this outer loop exits just because the inner loop can finish.
The $1 refers to a file with a lot of pattern/replacement lines. The $2 is a list of words. The problem is that the outer loop exits already after the first pattern/replacement line. I want it to exit after all the lines in $1 are read.
#!/bin/bash
#Receive SED SCRIPT WORDLIST
if [ -f temp.txt ];
then
> temp.txt
else
touch temp.txt
fi
while IFS='' read -r line || [[ -n "$line" ]];
do
echo -e "s/$line/p" >> temp.txt
while IFS='' read -r line || [[ -n "$line" ]];
do
sed -nf temp.txt $2
done
> temp.txt
done < $1
I understand that you want calculate de sed expressions and write it on a file, and then apply this expresions to other file.
This is so much easier than your are doing it.
First of all, you dont need to check if temp.txt already exists. When you redirect the output of a command to a file, if this file do not exist, it will be created. But if you want to reset the file, I recommend you to use truncate command.
In the body of the script, I don't understand why you put a second while loop to read from a file, but you don't put a file to read.
I think that you need is something like this:
truncate -s 0 sed_expressions.txt
while IFS='' read -r line || [[ -n "$line" ]]; do
echo -e "s/$line/p" >> sed_expressions.txt
done < $1
sed -nf sed_expressions.txt $2 > out_file.txt
Try it and tell me if is this that you need.
Bye!
I don't know why this is not working:
for g in *.txt; do for f in $(cat $g); do grep $f annotations.csv; done > ../$f_annot; done
I want to loop through each file in a folder, for each file I want to loop through each line and apply the grep command. When I do
for f in $(cat file1.txt); do grep $f annotations.csv; done > ../$f_annot
It works, it is the nested loop that doesn't output anything, it seems like it is running but it lasts forever and does nothing.
When you hava an empty txt file, grep $f annotations.csv will be translated into a grep command reading from stdin.
You might want to use something like
for g in *.txt; do
grep -f $g annotations.csv > ../$g_annot
done
SOLVED:
for file in *list.txt; do
while read -r line; do
grep "$line" annotations.csv
done < "$file"
> ${file}_annot.txt
done
:)
I get
Cannot write to a directory.
ksh: ../: 0403-005 Cannot create the specified file.
But this is because $f_annot evaluates to what we expect. It should be better with ${f}_annot:
for g in *.txt; do for f in $(cat $g); do grep $f annotations.csv; done > ../${f}_annot ; done
But there is an issue in your script because it erase the result of some loops: $f always has the last value of the loop. Maybe this suits your need better:
for g in *.txt; do for f in $(cat $g); do grep $f annotations.csv; done ; done
I need to search through an array then once I find what I'm looking, read that
element plus a couple more of the same array and write all to a file.
This is what I have so far
if [ -e "${EPH_DIR}" ]
then
i=0
while read line
do
FILE[$i]="$line"
i=$(($i+1))
done < ${EPH_DIR}
fi
for i in ${FILE[*]}
do
echo "$i"
if [[ $i == ${SAT} ]]
then
echo "Found it: $i"
fi
done
If you want to do it in a C-style for-loop:
for ((i=0; i < ${#FILES[#]}; i++)); do
if [[ ${FILES[i]} == $SAT ]]; then
printf "%s\n" "found it" "${FILES[i]}" "${FILES[i+1]}" "${FILES[i+2]}"
fi
done
Note that the array index in brackets is an arithmetic expression, so the dollar sign is not required.
You can use the built in grep switch -A:
-A NUM, --after-context=NUM
Print NUM lines of trailing context after matching lines.
Just pipe out the contents of the array into grep and use -A to give you however many extra lines you'd like printed. For example, use -A2 to print two additional lines after any line matching $STAT:
printf -- '%s\n' "${FILE[#]}" | grep -A2 "^$SAT$"
If you want this to go out to a new file, like the question's title suggests, then just redirect this to where you want:
printf -- '%s\n' "${FILE[#]}" | grep -A2 "^$SAT$" > /path/to/my/file
Since it looks like your array is just taking a file listing from the directory $EPH_DIR, instead of using a loop, you could put the listing into the array by doing:
FILE=( "$(ls $EPH_DIR)" )
Or, if this printing is the only thing you're using the array for, you can skip the array entirely and just send the directory listing directly into grep:
ls $EPH_DIR | grep -A2 "^$SAT$"