I got this
for file in *; do
any command
done
What I want to do is add an exception to the "for file in *; do".
Any ideas?
If you wanted to skip files with a particular extension, for example, ".pl", you could do:
for file in *
do
[ "${file##*.}" != "pl" ] && echo $file
done
One way to do what I think you are asking is to loop through and check a file name with a if statement (or just grep -v the ls cmd):
for file in `ls`; do
if [ "$file" == "something" ]; then
# do something
else
# something else
fi
done
For example, you have files
$ ls -1 ./
./M2U0001.MPG
./M2U0180.MPG
./text
Exception file is M2U0180.MPG
$> filename="M2U0180.MPG"
And
$> for file in $(ls -1 ./* | grep --invert-match "${filename}" ); do echo $file; done
./M2U0001.MPG
./text
Weird solution done :)
use bash's extended globbing
shopt -s extglob
for f in !(excluded-file); do echo "$f"; done
A simple solution :
The following code snippet will print all the files in the pwd , that have a .py extension
for i in * ;do
if [[ $i == *.py ]];then
echo "$i"
fi
done
Related
I am i need to find & remove empty files. The definition of empty files in my use case is a file which has zero lines.
I did try testing the file to see if it's empty However, this behaves strangely as in even though the file is empty it doesn't detect it so.
Hence, the best thing I could write up is the below script which i way too slow given it has to test several hundred thousand files
#!/bin/bash
LOOKUP_DIR="/path/to/source/directory"
cd ${LOOKUP_DIR} || { echo "cd failed"; exit 0; }
for fname in $(realpath */*)
do
if [[ $(wc -l "${fname}" | awk '{print $1}') -eq 0 ]]
then
echo "${fname}" is empty
rm -f "${fname}"
fi
done
Is there a better way to do what I'm after or alternatively, can the above logic be re-written in a way that brings better performance please?
Your script is slow beacuse wc reads every file to the end, which is not needed for your purpose. This might be what you're looking for:
#!/bin/bash
lookup_dir='/path/to/source/directory'
cd "$lookup_dir" || exit
for file in *; do
if [[ -f "$file" && -r "$file" && ! -L "$file" ]]; then
read < "$file" || echo rm -f -- "$file"
fi
done
Drop the echo after making sure it works as intended.
Another version, calling the rm only once, could be:
#!/bin/bash
lookup_dir='/path/to/source/directory'
cd "$lookup_dir" || exit
for file in *; do
if [[ -f "$file" && -r "$file" && ! -L "$file" ]]; then
read < "$file" || files_to_be_deleted+=("$file")
fi
done
rm -f -- "${files_to_be_deleted[#]}"
Explanation:
The core logic is in the line
read < "$file" || rm -f -- "$file"
The read < "$file" command attempts to read a line from the $file. If it succeeds, that is, a line is read, then the rm command on the right-hand side of the || won't be executed (that's how the || works). If it fails then the rm command will be executed. In any case, at most one line will be read. This has great advantage over the wc command because wc would read the whole file.
if ! read < "$file"; then rm -f -- "$file"; fi
could be used instead. The two lines are equivalent.
To check a "$fname" is a file and is empty or not, use [ -s "$fname" ]:
#!/usr/bin/env sh
LOOKUP_DIR="/path/to/source/directory"
for fname in "$LOOKUP_DIR"*/*; do
if ! [ -s "$fname" ]; then
echo "${fname}" is empty
# remove echo when output is what you want
echo rm -f "${fname}"
fi
done
See: help test:
File operators:
...
-s FILE True if file exists and is not empty.
Yet another method
wc -l ~/tmp/* 2>/dev/null | awk '$1 == 0 {print $2}' | xargs echo rm
This will break if any of your files have whitespace in the name.
To work around that, with awk still
wc -l ~/tmp/* 2>/dev/null \
| awk 'sub(/^[[:blank:]]+0[[:blank:]]+/, "")' \
| xargs echo rm
This works because the sub function returns the number of substitutions made, which can be treated as a boolean zero/not-zero condition.
Remove the echo to actually delete the files.
I have a file input.txt that looks as follows.
abas_1.txt
abas_2.txt
abas_3.txt
1fgh.txt
3ghl_1.txt
3ghl_2.txt
I have a folder ff. The filenames of this folder are abas.txt, 1fgh.txt, 3ghl.txt. Based on the input file, I would like to create and rename the multiple copies in ff folder.
For example in the input file, abas has three copies. In the ff folder, I need to create the three copies of abas.txt and rename it as abas_1.txt, abas_2.txt, abas_3.txt. No need to copy and rename 1fgh.txt in ff folder.
Your valuable suggestions would be appreciated.
You can try something like this (to be run from within your folder ff):
#!/bin/bash
while IFS= read -r fn; do
[[ $fn =~ ^(.+)_[[:digit:]]+\.([^\.]+)$ ]] || continue
fn_orig=${BASH_REMATCH[1]}.${BASH_REMATCH[2]}
echo cp -nv -- "$fn_orig" "$fn"
done < input.txt
Remove the echo if you're happy with it.
If you don't want to run from within the folder ff, just replace the line
echo cp -nv -- "$fn_orig" "$fn"
with
echo cp -nv -- "ff/$fn_orig" "ff/$fn"
The -n option to cp so as to not overwrite existing files, and the -v option to be verbose. The -- tells cp that there are no more options beyond this point, so that it will not be confused if one of the files starts with a hyphen.
using for and grep :
#!/bin/bash
for i in $(ls)
do
x=$(echo $i | sed 's/^\(.*\)\..*/\1/')"_"
for j in $(grep $x in)
do
cp -n $i $j
done
done
Try this one
#!/bin/bash
while read newFileName;do
#split the string by _ delimiter
arr=(${newFileName//_/ })
extension="${newFileName##*.}"
fileToCopy="${arr[0]}.$extension"
#check for empty : '1fgh.txt' case
if [ -n "${arr[1]}" ]; then
#check if file exists
if [ -f $fileToCopy ];then
echo "copying $fileToCopy -> $newFileName"
cp "$fileToCopy" "$newFileName"
#else
# echo "File $fileToCopy does not exist, so it can't be copied"
fi
fi
done
You can call your script like this:
cat input.txt | ./script.sh
If you could change the format of input.txt, I suggest you adjust it in order to make your task easier. If not, here is my solution:
#!/bin/bash
SRC_DIR=/path/to/ff
INPUT=/path/to/input.txt
BACKUP_DIR=/path/to/backup
for cand in `ls $SRC_DIR`; do
grep "^${cand%.*}_" $INPUT | while read new
do
cp -fv $SRC_DIR/$cand $BACKUP_DIR/$new
done
done
I have a find script that automatically opens a file if just one file is found. The way I currently handle it is doing a word count on the number of lines of the search results. Is there an easier way to do this?
if [ "$( cat "$temp" | wc -l | xargs echo )" == "1" ]; then
edit `cat "$temp"`
fi
EDITED - here is the context of the whole script.
term="$1"
temp=".aafind.txt"
find src sql common -iname "*$term*" | grep -v 'src/.*lib' >> "$temp"
if [ ! -s "$temp" ]; then
echo "ΓΈ - including lib..." 1>&2
find src sql common -iname "*$term*" >> "$temp"
fi
if [ "$( cat "$temp" | wc -l | xargs echo )" == "1" ]; then
# just open it in an editor
edit `cat "$temp"`
else
# format output
term_regex=`echo "$term" | sed "s%\*%[^/]*%g" | sed "s%\?%[^/]%g" `
cat "$temp" | sed -E 's%//+%/%' | grep --color -E -i "$term_regex|$"
fi
rm "$temp"
Unless I'm misunderstanding, the variable $temp contains one or more filenames, one per line, and if there is only one filename it should be edited?
[ $(wc -l <<< "$temp") = "1" ] && edit "$temp"
If $temp is a file containing filenames:
[ $(wc -l < "$temp") = "1" ] && edit "$(cat "$temp")"
Several of the results here will read through an entire file, whereas one can stop and have an answer after one line and one character:
if { IFS='' read -r result && ! read -n 1 _; } <file; then
echo "Exactly one line: $result"
else
echo "Either no valid content at all, or more than one line"
fi
For safely reading from find, if you have GNU find and bash as your shell, replace <file with < <(find ...) in the above. Even better, in that case, is to use NUL-delimited names, such that filenames with newlines (yes, they're legal) don't trip you up:
if { IFS='' read -r -d '' result && ! read -r -d '' -n 1 _; } \
< <(find ... -print0); then
printf 'Exactly one file: %q\n' "$result"
else
echo "Either no results, or more than one"
fi
Well, given that you are storing these results in the file $temp this is a little easier:
[ "$( wc -l < $temp )" -eq 1 ] && edit "$( cat $temp )"
Instead of 'cat $temp' you can do '< $temp', but it might take away some readability if you are not very familiar with redirection 8)
If you want to test whether the file is empty or not, test -s does that.
if [ -s "$temp" ]; then
edit `cat "$temp"`
fi
(A non-empty file by definition contains at least one line. You should find that wc -l agrees.)
If you genuinely want a line count of exactly one, then yes, it can be simplified substantially;
if [ $( wc -l <"$temp" ) = 1 ]; then
edit `cat "$temp"`
fi
You can use arrays:
x=($(find . -type f))
[ "${#x[*]}" -eq 1 ] && echo "just one || echo "many"
But you might have problems in case of filenames with whitespace, etc.
Still, something like this would be a native way
no this is the way, though you're making it over-complicated:
if [ "`wc -l $temp | cut -d' ' -f1`" = "1" ]; then
edit "$temp";
fi
what's complicating it is:
useless use of cat,
unuseful use of xargs
and I'm not sure if you really want the editcat $temp`` which is editing the file at the content of $temp
I have a list of files like so :
10_I_am_here_001.jpg
20_I_am_here_003.jpg
30_I_am_here_008.jpg
40_I_am_here_004.jpg
50_I_am_here_009.jpg
60_I_am_here_002.jpg
70_I_am_here_005.jpg
80_I_am_here_006.jpg
How can I rename all the files in a directory, so that I can drop ^[0-9]+_ from the filename ?
Thank you
Using pure BASH:
s='10_I_am_here_001.jpg'
echo "${s#[0-9]*_}"
I_am_here_001.jpg
You can then write a simple for loop in that directory like this:
for s in *; do
f="${s#[0-9]*_}" && mv "$s" "$f"
done
Using rename :
rename 's/^[0-9]+_//' *
Here's another bash idea based on files ending .jpg as shown above or whatever>
VonBell
#!/bin/bash
ls *.jpg |\
while read FileName
do
NewName="`echo $FileName | cut -f2- -d "_"`"
mv $FileName $NewName
done
With bash extended globbing
shopt -s extglob
for f in *
do
[[ $f == +([0-9])_*.jpg ]] && mv "$f" "${f#+([0-9])_}"
done
My assignment is to write a Unix shell script that asks the user for the name of a directory, and then works exactly like find.
Here is what I have so far:
#!/bin/bash
dir_lister()
{
cd "$1"
echo "$1"
list=$(ls -l ${1})
nolines=$(echo "$list" | awk 'END{printf "%d",NF}')
if [ $nolines -eq 2 ]
then
echo "$1"
return
fi
filelist=$(echo "$list" | grep ^-.*)
dirlist=$(echo "$list" | grep ^d.*)
filename=$(echo "$filelist"| awk '{printf "%s\n",$NF}')
present=$(pwd)
echo "$filename"| awk -v pres=$present '{printf "%s/%s\n",pres,$0}'
dirlist2=$(echo "$dirlist" | awk '{printf "%s\n",$NF}')
echo "$dirlist2" | while IFS= read -r line;
do
nextCall=$(echo "$present/$line");
dir_lister $nextCall;
cd ".."
done
cd ".."
}
read -p "Enter the name of the direcotry: " dName
dir_lister $dName
The problem is, after a depth of three directories, this script gets into an infinite loop, and I don't see why.
EDIT:
Here is the code i came up with after looking at your answer, it still doesn't go more than 1 directory depth:
#!/bin/bash
shopt -s dotglob # don't miss "hidden files"
shopt -s nullglob # don't fail on empty directories
list_directory()
{
cd "$2"
cd "$1"
##echo -e "I am called \t $1 \t $2"
for fileName in "$1/"*
do
##echo -e "hello \t $fileName"
if [ -d "$fileName" ];
then
echo "$fileName"
list_directory $fileName $2
else
echo "$fileName"
fi
done
}
read -p "Enter the direcotory Name: " dirName
var=$(pwd)
list_directory $dirName $var
Okay, that is completely the wrong way to list files in a directory (see ParsingLs). I'll give you the pieces and you should be able to put them together into a working script.
Put this at the top of your script:
shopt -s dotglob # don't miss "hidden files"
shopt -s nullglob # don't fail on empty directories
Then you can easily loop over directory contents with:
for file in "$directory/"* ; do
#...
done
Test if you have a directory:
if [ -d "$file" ] ; then
# "$file" is a directory, recurse...
fi