Shell script to move files in while loop with counter - bash

I am trying to move some files down a directory and rename them
the current file struct looks something like the following:
photos ->
{
1 -> Auction_Images -> <several image files>
item_image-1-1.jpg
item_image-2-1.jpg
...
2 -> Auction_Images -> <several image files>
item_image-3-1.jpg
...
3 -> Auction_Images -> <several image files>
my_script.sh
}
I want to be able to go into each numbered directory, check how many files/folders exist, if there is only one meaning the Auction_Images dir like folder 3 then I want to go into that Auction_Images dir and moved the photos out one directory and rename them. So far I have the following:
x=1
while [ $x -le 3 ]
do
cd $PWD/$x
echo Changed to dir: $PWD
count=ls | wc -l
echo $count
if [[ "$count" -eq 1 ]]
then
echo $PWD has 1 file/folder
fi
echo --------------------------------------------------
cd ..
x=$(( $x + 1 ))
done
the output I am getting is:
Changed to dir: /Users/jarvis/Desktop/imports/company/photos/1
0
--------------------------------------------------
Changed to dir: /Users/jarvis/Desktop/imports/company/photos/2
0
--------------------------------------------------
Changed to dir: /Users/jarvis/Desktop/imports/company/photos/3
0
The expected output would be:
Changed to dir: /Users/jarvis/Desktop/imports/company/photos/1
2
--------------------------------------------------
Changed to dir: /Users/jarvis/Desktop/imports/company/photos/2
1
--------------------------------------------------
Changed to dir: /Users/jarvis/Desktop/imports/company/photos/3
0
The current problem I am having is that the count doesn't seem to be captured correctly.

If I understood the question correctly, you could use something like this:
#!/usr/bin/env bash
shopt -s nullglob extglob
for dir in +([0-9])/; do
files=("$dir/"*)
count=${#files[#]}
echo "$PWD/$dir"
echo "$count"
if (( count == 1 )); then
#do something
echo "Only one file/folder"
mv -- "$files" /some/path/newname
fi
echo "=============================="
done
shopt -s nullglob = ensures you will not execute the loop if there are no directories
shopt -s extglob = enables extended globbing
for dir in +([0-9])/ = loops only over directories consisted of one or more numbers
files=("$dir/"*) = stores all files in currently searched directory into an array
count=${#files[#]} = counts the number of elements in the array -- number of files
mv -- "$files" /some/path/newname = moves the first and only element in the files array into the new location (note that $files is a shorter way of ${files[0]})
If you don't like the for loop I used and you insist on using this:
while [ $x -le 3 ]; do
x=$(( $x + 1 ))
done
you might consider using the C-style for loop:
for (( x=1; x<=3; x++ )); do
#do something
done
or brace expansion:
for x in {1..3}; do
#do something
done
That way you don't have to increment $x yourself. On top of that:
Quote expansions = echo "$PWD"
count=ls | wc -l = use command substitution instead: count=$(ls | wc -l)
Why you shouldn't parse the output of ls

Related

Why doesn't counting files with "for file in $0/*; let i=$i+1; done" work?

I'm new in ShellScripting and have the following script that i created based on a simpler one, i want to pass it an argument with the path to count files. Cannot find my logical mistake to make it work right, the output is always "1"
#!/bin/bash
i=0
for file in $0/*
do
let i=$i+1
done
echo $i
To execute the code i use
sh scriptname.sh /path/to/folder/to/count/files
$0 is the name with which your script was invoked (roughly, subject to several exceptions that aren't pertinent here). The first argument is $1, and so it's $1 that you want to use in your glob expression.
#!/bin/bash
i=0
for file in "$1"/*; do
i=$(( i + 1 )) ## $(( )) is POSIX-compliant arithmetic syntax; let is deprecated.
done
echo "$i"
That said, you can get this number more directly:
#!/bin/bash
shopt -s nullglob # allow globs to expand to an empty list
files=( "$1"/* ) # put list of files into an array
echo "${#files[#]}" # count the number of items in the array
...or even:
#!/bin/sh
set -- "$1"/* # override $# with the list of files matching the glob
if [ -e "$1" ] || [ -L "$1" ]; then # if $1 exists, then it had matches
echo "$#" # ...so emit their number.
else
echo 0 # otherwise, our result is 0.
fi
If you want to count the number of files in a directory, you can run something like this:
ls /path/to/folder/to/count/files | wc -l

bash script for putting thousands of files, into separate folders

in a directory, i have thousands of files.
i use the following bash snippet, in order to make these thousands of files, be put in folders, that have each 1000 files,
and i face two issues:
a) now each folder has a prefix of dir_ while i would like it to have a name, that will have 6 digits, if less than 6 in the folder name, leading zeros should be added appropriately.
b) current script, puts first folder, into the last one, for example, i have dir_400325 as the last folder, and in there,i find the bash script i have run, and the dir_1000 folder, which is the first folder created. How could i change this, so the first folder, is not stored into the last one?
#!/bin/bash
c=0; d=1000; mkdir -p dir_${d}
for file in *
do
if [ $c -eq 1000 ]
then
d=$(( d + 1000 )); c=0; mkdir -p dir_${d}
fi
mv "$file" dir_${d}/
c=$(( c + 1 ))
done
You can use printf and a format string to generate your 6-digit directory name with leading zeros (%06d), demonstrated in a shell:
bash-4.4$ d=1001
bash-4.4$ dir_name=$(printf "/path/to/%06d" $d)
bash-4.4$ echo $dir_name
/path/to/001001
Using an absolute path may help ensure the files end up where you're expecting them and not in some subfolder of your current working directory.
#!/bin/bash
c=0
d=1000
dir_name=$(printf "/path/to/%06d" $d)
mkdir -p $dir_name
for file in *
do
if [ $c -eq 1000 ]
then
c=0
d=$(( d + 1000 ))
dir_name=$(printf "/path/to/%06d" $d)
mkdir -p $dir_name
fi
if [[ -f "$file" ]]
then
mv "$file" $dir_name
c=$(( c + 1 ))
fi
done

bash string length in a loop

I am looping through a folder and depending on the length of files do certain condition. I seem not to come right with that. I evaluate and output the length of a string in the terminal.
echo $file|wc -c gives me the answer of all files in the terminal.
But incorporating this into a loop is impossible
for file in `*.zip`; do
if [[ echo $file|wc -c ==9]]; then
some commands
where I want to operate on files that have a length of nine characters
Try this one:
for file in *.zip ; do
wcout=$(wc -c "$file")
if [[ ${wcout%% *} -eq 9 ]] ; then
# some commands
fi
done
The %% operator in variable expansion deletes everything that match the pattern after it. This is glob pattern, not regular expression.
Opposite to natural good sense of typical programmers the == operator in BASH compares strings, not numbers.
Alternatively (following the comment) you can:
for file in *.zip ; do
wcout=$(wc -c < "$file")
if [[ ${wcout} -eq 9 ]] ; then
# some commands
fi
done
Additional observation is that if BASH cannot expand *.zip as there is no ZIP files in the current directory it will pass "*.zip" into $file and let single iteration of the loop. That leads to the error reported by wc command. So it would be recommended to add:
if [[ -e ${file} ]] ; then ...
as a prevention mechanism.
Comments leads to another form of this solution (plus I added my safety mechanism):
for file in *.zip ; do
if [[ -e "$file" && (( $(wc -c < "$file") == 9 )) ]] ; then
# some commands
fi
done
using filter outside the loop
ls -1 *.zip \
| grep -E '^.{9}$' \
| while read FileName
do
# Your action
done
using filter inside loop
ls -1 *.zip \
| while read FileName
do
if [ ${#FileName} -eq 9 ]
then
# Your action
fi
done
alternative to ls -1 that is always a bit dangereous, find . -name '*.zip' -print [ but you neet to add 2 char length or filter the name form headin ./ and maybe limit to current folder depth ]

for-loop for every folder in a directory, excluding some of them

Thank you very much in advance for helping!
I have this code in bash:
for d in this_folder/*
do
plugin=$(basename $d)
echo $plugin'?'
read $plugin
done
Which works like a charm. For every folders inside 'this_folder', echo it as a question and store the input into a variable with the same name.
But now I'd like to exclude some folders, so for example, it will ask for every folder in that directory, ONLY if they are NOT any of the following folders: global, plugins, and css.
Any ideas how can I achieve this?
Thanks!
UPDATE:
This is how the final code looks like:
base="coordfinder|editor_and_options|global|gyro|movecamera|orientation|sa"
> vt_conf.sh
echo "# ========== Base" >> vt_conf.sh
for d in $orig_include/#($base)
do
plugin=$(basename $d)
echo "$plugin=y" >> vt_conf.sh
done
echo '' >> vt_conf.sh
echo "# ========== Optional" >> vt_conf.sh
for d in $orig_include/!($base)
do
plugin=$(basename $d)
echo "$plugin=n" >> vt_conf.sh
done
If you have a recent version of bash, you can use extended globs (shopt -s extglob):
shopt -s extglob
for d in this_folder/!(global|plugins|css)/
do
plugin=$(basename "$d")
echo $plugin'?'
read $plugin
done
You can use continue to skip one iteration of the loop:
for d in this_folder/*
do
plugin=$(basename $d)
[[ $plugin =~ ^(global|plugins|css)$ ]] && continue
echo $plugin'?'
read $plugin
done
If you meant to exclude only the directories named global, css, plugins. This might not be an elegant solution but will do what you want.
for d in this_folder/*
do
flag=1
#scan through the path if it contains that string
for i in "/css/" "/plugins/" "/global/"
do
if [[ $( echo "$d"|grep "$i" ) && $? -eq 0 ]]
then
flag=0;break;
fi
done
#Only if the directory path does NOT contain those strings proceed
if [[ $flag -eq 0 ]]
then
plugin=$(basename $d)
echo $plugin'?'
read $plugin
fi
done
You could use find and awk to build the list of directories and then store the result in a variable. Something along the lines of this (untested):
dirs=$(find this_folder -maxdepth 1 -type d -printf "%f\n" | awk '!match($0,/^(global|plugins|css)$/)')
for d in $dirs; do
# ...
done
Update 2019-05-16:
while read -r d; do
# ...
done < <(gfind -maxdepth 1 -type d -printf "%f\n" | awk '!match($0,/^(global|plugins|css)$/)')
While How to exclude some files from the loop in shell script was marked as a dupe of this Q/A and closed, that Q specifically asked about excluding files in a BASH script, which is exactly what I needed (in a script to check the validity of link fragments (the part after #) in local URLs. Here is my solution.
for FILE in *
do
## https://linuxize.com/post/how-to-check-if-string-contains-substring-in-bash/
if [[ "$FILE" == *"cnp_"* ]]
then
echo 'cnp_* file found; skipping'
continue
fi
## rest of script
done
Output:
cnp_* file found; skipping
----------------------------------------
FILE: 1 | NAME: linkchecker-test_file1.html
PATH: /mnt/Vancouver/domains/buriedtruth.com/linkchecker-tests/linkchecker-test_file1.html
RAW LINE: #bookmark1
FULL PATH: /mnt/Vancouver/domains/buriedtruth.com/linkchecker-tests/linkchecker-test_file1.html#bookmark1
LINK: /mnt/Vancouver/domains/buriedtruth.com/linkchecker-tests/linkchecker-test_file1.html
FRAGMENT: bookmark1
STATUS: OK
...
My test directory contained 3 files, with one that I wanted to exclude (web scrape of an old website: an index with with tons of deprecated link fragments).
[victoria#victoria link_fragment_tester]$ tree
.
├── cnp_members-index.html
├── linkchecker-test_file1.html -> /mnt/Vancouver/domains/buriedtruth.com/linkchecker-tests/linkchecker-test_file1.html
└── linkchecker-test_file2.html -> /mnt/Vancouver/domains/buriedtruth.com/linkchecker-tests/linkchecker-test_file2.html
0 directories, 3 files
[victoria#victoria link_fragment_tester]$
Splitting the path, checking against each foldername to be ignored.
Ignoring folder this way is convenient if you want to increase quantity of folder to be ignored afterwards. (they are here stored in ignoredfolders).
Added comments in the code.
This works fine for me in Ubuntu 20.04.2
#!/bin/bash
shopt -s globstar #necessary for search ./**/*
ignoredfolders=("folder1name" "folder2name")
for i in ./**/*
do
#pattern for splitting forward slashes into empty spaces
ARRAY=(${i//\//" "} );
for k in "${ignoredfolders[#]}"; do
#does path (splitted in ARRAY) contain a foldername ($k) to be ignored?
if [[ " ${ARRAY[#]} " =~ "$k" ]]; then
#this skips this loop and the outer loop
continue 2
fi
done
# all ignored folders are ignored, you code sits here...
#code ...
#code ...
done

Bash - replacing targeted files with a specific file, whitespace in directory names

I have a large directory tree of files, and am using the following script to list and replace a searched-for name with a specific file. Problem is, I don't know how to write the createList() for-loop correctly to account for whitespace in a directory name. If all directories don't have spaces, it works fine.
The output is a list of files, and then a list of "cp" commands, but reports directories with spaces in them as individual dirs.
aindex=1
files=( null )
[ $# -eq 0 ] && { echo "Usage: $0 filename" ; exit 500; }
createList(){
f=$(find . -iname "search.file" -print)
for i in $f
do
files[$aindex]=$(echo "${i}")
aindex=$( expr $aindex + 1 )
done
}
writeList() {
for (( i=1; i<$aindex; i++ ))
do
echo "#$i : ${files[$i]}"
done
for (( i=1; i<$aindex; i++ ))
do
echo "/usr/bin/cp /cygdrive/c/testscript/TheCorrectFile.file ${files[$filenumber]}"
done
}
createList
writeList
Replace your entire script with these four lines:
find . -iname "search.file" | while read file
do
/usr/bin/cp /cygdrive/c/testscript/TheCorrectFile.file "$file"
done
(But isn't cp in /bin? Oh, on Cygwin it's in both. It's more portable, though, to use /bin.)

Resources