What is the simplest way to check if a string contains newline?
For example, after
FILE=$(find . -name "pattern_*.sh")
I'd like to check for newline to ensure only one file matched.
You can use pattern matching:
[[ $FILE == *$'\n'* ]] && echo More than one line
If $str contains new line you can check it by,
if [ $(echo "$str" | wc -l) -gt 1 ];
then
// perform operation if it has new lines
else
// no new lines.
fi
To refer to your example: Note that filenames could contain newlines, too.
A safe way to count files would be
find -name "pattern_*.sh" -printf '\n' | wc -c
This avoids printing the filename and prints only a newline instead.
Related
I am attempting to find multiple files with .py extension and grep to see if any of these files contain the string nn. Then return only the directory name (uniques), afterwards, if the last folder of the path ends in nn, then select this.
For example:
find `pwd` -iname '*.py' | xargs grep -l 'nn' | xargs dirname | sort -u | while read files; do if [[ sed 's|[\/](.*)*[\/]||g' == 'nn' ]]; then echo $files; fi; done
However, I cannot use sed in if-else expression, how can I use it for this case?
[[ ]] is not bracket syntax for an if statement like in other languages such as C or Java. It's a special command for evaluating a conditional expression. Depending on your intentions you need to either exclude it or use it correctly.
If you're trying to test a command for success or failure just call the command:
if command ; then
:
fi
If you want to test the output of the command is equal to some value, you need to use a command substitution:
if [[ $( command ) = some_value ]] ; then
:
fi
In your case though, a simple parameter expansion will be easier:
# if $files does not contain a trailing slash
if [[ "${files: -2}" = "nn" ]] ; then
echo "${files}"
fi
# if $files does contain a trailing slash
if [[ "${files: -3}" = "nn/" ]] ; then
echo "${files%/}"
fi
Shell loop and the [[ is superfluous here, since you use the sed anyway. This task could be accomplished by:
find "$PWD" -type f -name '*.py' -exec grep -l 'nn' {} + |
sed -n 's%\(.*nn\)/[^/]*$%\1%p' | sort -u
assuming pathnames don't contain a newline character.
I have many files not having consistent filenames.
For example
IMG_20200823_1.jpg
IMG_20200823_10.jpg
IMG_20200823_12.jpg
IMG_20200823_9.jpg
I would like to rename all of them and ensure they all follow same naming convention
IMG_20200823_0001.jpg
IMG_20200823_0010.jpg
IMG_20200823_0012.jpg
IMG_20200823_0009.jpg
Found out it's possible to change for file having only a number using below
printf "%04d\n"
However am not able to do with my files considering they mix string + "_" + different numbers.
Could anyone help me ?
Thanks !
With Perl's standalone rename or prename command:
rename -n 's/(\d+)(\.jpg$)/sprintf("%04d%s",$1,$2)/e' *.jpg
Output:
rename(IMG_20200823_10.jpg, IMG_20200823_0010.jpg)
rename(IMG_20200823_12.jpg, IMG_20200823_0012.jpg)
rename(IMG_20200823_1.jpg, IMG_20200823_0001.jpg)
rename(IMG_20200823_9.jpg, IMG_20200823_0009.jpg)
if everything looks fine, remove -n.
With Bash regular expressions:
re='(IMG_[[:digit:]]+)_([[:digit:]]+)'
for f in *.jpg; do
[[ $f =~ $re ]]
mv "$f" "$(printf '%s_%04d.jpg' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}")"
done
where BASH_REMATCH is an array containing the capture groups of the regular expression. At index 0 is the whole match; index 1 contains IMG_ and the first group of digits; index 2 contains the second group of digits. The printf command is used to format the second group with zero padding, four digits wide.
Use a regex to extract the relevant sub-strings from the input and then pad it...
For each file.
Extract the prefix, number and suffix from the filename.
Pad the number with zeros.
Create the new filename.
Move files
The following code for bash:
echo 'IMG_20200823_1.jpg
IMG_20200823_10.jpg
IMG_20200823_12.jpg
IMG_20200823_9.jpg' |
while IFS= read -r file; do # foreach file
# Use GNU sed to extract parts on separate lines
tmp=$(<<<"$file" sed 's/\(.*_\)\([0-9]*\)\(\..*\)/\1\n\2\n\3\n/')
# Read the separate parts separated by newlines
{
IFS= read -r prefix
IFS= read -r number
IFS= read -r suffix
} <<<"$tmp"
# create new filename
newfilename="$prefix$(printf "%04d" "$number")$suffix"
# move the files
echo mv "$file" "$newfilename"
done
outputs:
mv IMG_20200823_1.jpg IMG_20200823_0001.jpg
mv IMG_20200823_10.jpg IMG_20200823_0010.jpg
mv IMG_20200823_12.jpg IMG_20200823_0012.jpg
mv IMG_20200823_9.jpg IMG_20200823_0009.jpg
Being puzzled by your hint at printf...
Current folder content:
$ ls -1 IMG_*
IMG_20200823_1.jpg
IMG_20200823_21.jpg
Surely is not a good solution but with printf and sed we can do that:
$ printf "mv %3s_%8s_%d.%3s %3s_%8s_%04d.%3s\n" $(ls -1 IMG_* IMG_* | sed 's/_/ /g; s/\./ /')
mv IMG_20200823_1.jpg IMG_20200823_0001.jpg
mv IMG_20200823_21.jpg IMG_20200823_0021.jpg
Consider this mockup:
function test() {
for line in $1
do
echo $line
done
for line2 in $2
do
echo $line2
done
}
# This will give me a list of IDs
list=$(find testfolder/ -type f -exec grep "ID" {} + | sed "s/^.*ID:\ //g")
list2=$(find testfolder2/ -type f -exec grep "ID" {} + | sed "s/^.*ID:\ //g")
# this will not work
test list1 list2
# this will work
for line in $line
do
echo $line
done
for line2 in $2
do
echo $line
done
The problem with this is that the variables $1 and $2 in the function, will be (of course) the first two IDs that were retrieved in list.
Is there a way to pass list and list2 to the function and use them as I would in a non function call?
The problem with Shell scripting and file names is, that the shell splits the input stream into tokens by spaces and newlines. Which characters are used, is stored in the global variable IFS, which is the abbreviation for input field separator. The problem is, that file names may contain spaces and newlines. And if you do not quote the file names correctly as you did it in your question, then the file names get split by the shell.
Problem
Create some files with space:
$ touch a\ {1..3}
If you use a globing pattern to iterate the files, everything is fine:
$ for f in a\ *; do echo ►$f◄; done
►a 1◄
►a 2◄
►a 3◄
But when you use a sub-shell, which echos the file names, they get messed:
$ for f in $(echo a\ *); do echo ►$f◄; done
►a◄
►1◄
►a◄
►2◄
►a◄
►3◄
The same happens, when you use find:
$ for f in $(find . -name 'a *'); do echo ►$f◄; done
►./a◄
►2◄
►./a◄
►3◄
►./a◄
►1◄
Solution
The best way to read a list of files is to delimit them with a character, which is normally not in a file. This is the null character $'\0'. The program find has a special action called -print0 to print the file name with a trailing null character. And the Bash function mapfile can read a list, which is delimited with null characters:
$ mapfile -d $'\0' list < <(find . -name 'a *' -print0)
Now you can write a function, which needs a list of file names.
$ inode() { for f in "$#"; do stat -c %i "$f"; done; }
And pass the list of file names correctly quoted to the function.
$ inode "${list[#]}"
2638642
2638644
2638641
And this works even with newlines in the file name.
I want to have a conditional behavior depending on the number of files found:
found=$(find . -type f -name "$1")
numfiles=$(printf "%s\n" "$found" | wc -l)
if [ $numfiles -eq 0 ]; then
echo "cannot access $1: No such file" > /dev/stderr; exit 2;
elif [ $numfiles -gt 1 ]; then
echo "cannot access $1: Duplicate file found" > /dev/stderr; exit 2;
else
echo "File: $(ls $found)"
head $found
fi
EDITED CODE (to reflect more precisely what I need)
Though, numfiles isn't equal to 2(or more) when there are duplicate files found...
All the filenames are on one line, separated by a space.
On the other hand, this works correctly:
find . -type f -name "$1" | wc -l
but I don't want to do twice the recursive search in the if/then/else construct...
Adding -print0 doesn't help either.
What would?
PS- Simplifications or improvements are always welcome!
You want to find files and count the files with a name "$1":
grep -c "/${1}$" $(find . 2>/dev/null)
And store the result in a var. In one command:
numfiles=$(grep -c "/${1}$" <(find . 2>/dev/null))
Using $() to store data to a variable trims tailing whitespace. Since the final newline does not appear in the variable numfiles, wc miscounts by one. You can recover the trailing newline with:
numfiles=$(printf "%s\n" "$found" | wc -l)
This miscounts if found is empty (and if any filenames contain a newline), emphasizing the fact that this entire approach is faulty. If you really want to go this way, you can try:
numfiles=$(test -z "$numfiles" && echo 0 || printf "%s\n" "$found" | wc -l)
or pipe the output of find to a script that counts the output and prints a count along with the first filename:
find . -type f -name "$1" | tr '\n' ' ' |
awk '{c=NF; f=$1 } END {print c, f; exit c!=1}' c=0 |
while read count name; do
case $count in
0) echo no files >&2;;
1) echo 1 file $name;;
*) echo Duplicate files >&2;;
esac;
done
All of these solutions fail miserably if any pathnames contain whitespace. If that matters, you could change the awk to a perl script to make it easier to handle null separators and use -print0, but really I think you should stop worrying about special cases. (find -exec and find | xargs both fail to handle to 0 files matching case cleanly. Arguably this awk solution also doesn't handle it cleanly.)
How do I loop through a list of Bash file names from an input text file and grep each file in a directory for each file name (to see if the file name is contained in the file) and output to text all file names that weren't found in any files?
#!/bin/sh
# This script will be used to output any unreferenced bash files
# included in the WebAMS Project
# Read file path of bash files and file name input
SEARCH_DIR=$(awk -F "=" '/Bash Dir/ {print $2}' bash_input.txt)
FILE_NAME=$(awk -F "=" '/Input File/ {print $2}' bash_input.txt)
echo $SEARCH_DIR
echo $FILE_NAME
exec<$FILE_NAME
while read line
do
echo "IN WHILE"
if (-z "$(grep -lr $line $SEARCH_DIR)"); then
echo "ENTERED"
echo $filename
fi
done
Save this as search.sh, updating SEARCH_DIR as appropriate for your environment:
#!/bin/bash
SEARCH_DIR=some/dir/here
while read filename
do
if [ -z "$(grep -lr $filename $SEARCH_DIR)" ]
then
echo $filename
fi
done
Then:
chmod +x search.sh
./search.sh files-i-could-not-find.txt
It could be possible through grep and find commands,
while read -r line; do (find . -type f -exec grep -l "$line" {} \;); done < file
OR
while read -r line; do grep -rl "$line"; done < file
-r --> recursive
-l --> files-with-matches(Displays the filenames which contains the search string)
It will read all the filenames present inside the input file and search for the filenames which contains the readed filenames. If it found any, then it returns the corresponding filename.
You're using regular parentheses instead of square brackets in your if statement.
The square brackets are a test command. You're running a test (in your case, whether a string has zero length or not. If the test is successful, the [ ... ] command returns an exit code of zero. The if statement sees that exit code and runs the then clause of the if statement. Otherwise, if an else statement exists, that is run instead.
Because the [ .. ] are actually commands, you must leave a blank space around each side.
Right
if [ -z "$string" ]
Wrong
if [-z "$string"] # Need white space around the brackets
Sort of wrong
if [ -z $sting ] # Won't work if "$string" is empty or contains spaces
By the way, the following are the same:
if test -z "$string"
if [ test -z "$string" ]
Be careful with that grep command. If there are spaces or newlines in the string returned, it may not do what you think it does.