I have a holiday folder with all pictures unsorted in the same folder, all files are tagged with the year. I want to create a folder for the year. This I've solved with:
#!/bin/bash
SOURCE=/path/to/pictures
cd ~/pictures/sorted.by.year/
ls $SOURCE | grep -o -P '(?<!\d)\d{4}(?!\d)' | tr '/' '\n' | sort |
uniq | xargs mkdir
Now to my problem, I want to create a symlink to all year folders from the filename so it will create symlinks, like this:
~/pictures/sorted.by.year/2014/symlink-picture.on.sister.2014.jpg
~/pictures/sorted.by.year/2012/symlink-picture.on.dad.2012.jpg
I have tried with xargs but the result was not as I wanted in the example above.
Does someone have an idea how to do it?
Please check this script, I added some more line after your script:
#!/bin/bash
SOURCE=/path/to/pictures
cd ~/pictures/sorted.by.year/
ls $SOURCE | grep -o -P '(?<!\d)\d{4}(?!\d)' | tr '/' '\n' | sort | uniq | xargs mkdir
for file in /path/to/pictures/*
do
if [[ -d $file ]]; then # Skip directory
continue;
fi
DIRNAME=$(echo "$file" | grep -o -P '(?<!\d)\d{4}(?!\d)');
ln -s "$file" "./$DIRNAME/"
done
Related
I need bash script, these are my files:
./2019-01-11_15-00-29_UTC.mp4
./2019-02-10_17-42-18_UTC.mp4
./2019-01-03_14-45-43_UTC.mp4
./2018-12-24_13-00-32_UTC.mp4
./2018-12-09_19-50-59_UTC.mp4
./2019-01-11_14-51-08_UTC.mp4
./2019-01-06_16-41-54_UTC.mp4
./2019-02-03_10-33-33_UTC.mp4
./2019-02-16_18-21-30_UTC.mp4
wanna make two folder 2018 & 2019 then move files own folder.
I use this code:
ls *.mp4 | awk -F"-" '{print $1}' | while read day file ; do mkdir -p "$day"; mv "$file" "$day"; done
It makes folder but not move
Aren't you getting errors? When I run it I get
mv: cannot stat '': No such file or directory
once for each file. The reason is that file isn't being set in your loop.
ls *.mp4 | awk -F"-" '{print $1}'
Will generate a list of years
2018
2018
2019
2019
2019
2019
2019
2019
2019
That's a single column of data.
while read day file
reads the year into day (day?) and since there's no more data, leaves file empty.
mkdir -p "$day"
works fine, but
mv "$file" "$day"
evaluates to
mv "" "2018"
Try this.
for f in *.mp4
do mkdir -p "${f%%-*}" && mv "$f" "${f%%-*}"
done
The ${f%%-*} just returns $f with everything from the first dash removed. The result:
$: find
.
./2018
./2018/2018-12-09_19-50-59_UTC.mp4
./2018/2018-12-24_13-00-32_UTC.mp4
./2019
./2019/2019-01-03_14-45-43_UTC.mp4
./2019/2019-01-06_16-41-54_UTC.mp4
./2019/2019-01-11_14-51-08_UTC.mp4
./2019/2019-01-11_15-00-29_UTC.mp4
./2019/2019-02-03_10-33-33_UTC.mp4
./2019/2019-02-10_17-42-18_UTC.mp4
./2019/2019-02-16_18-21-30_UTC.mp4
You can use xargs to achieve this.
ls *.mp4 | xargs -I{} sh -c 'folder=`echo {} | cut -d"-" -f1`;mkdir -p $folder;mv {} $folder/'
Here, the entire file names are sent to xargs and, for each file the folder name is obtained using the cut command. Then the file is moved into the created folder.
More about xargs: http://man7.org/linux/man-pages/man1/xargs.1.html
Edited
for file in *.mp4 ; do
date=$(echo $file | cut -d'_' -f1)
year=$(echo $date | cut -d'-' -f1)
month=$(echo $date | cut -d'-' -f2)
day=$(echo $date | cut -d'-' -f3)
mkdir -p $year/$month/$day
mv $file $year/$month/$day/
done
I'm trying to create a script that gets an extension and reports in two columns, the user and the amount of files that user owns with that extension.
The results must be printed in report.txt
Here is my code.
#!/bin/bash
#Uncoment to create /tmp/test/ with 50 txt files
#mkdir /tmp/test/
#touch /tmp/test/arch{01..50}.txt
clear
usage(){
echo "The script needs an extension to search"
echo "$0 <extension>"
}
if [ $# -eq 0 ]; then
usage
exit 1
fi
folder="/tmp/test/"
touch report.txt
count=0
pushd $folder
for file in $(ls -l); do
grep "*.$1" | cut -d " " -f3 >> report.txt
done
popd
The program just runs endlessly. And I'm not even counting the files for each user.
How can I solve this using only grep and cut?
With GNU stat :
stat -c '%U' *."$1" | sort | uniq -c | awk '{print $2,"\t",$1}' > report.txt
As pointed out by mklement0, under BSD/OSX you must use a -f option with stat :
stat -f '%Su' *."$1" | sort | uniq -c | awk '{print $2,"\t",$1}' > report.txt
Edit :
To process many files and avoid argument number limitation, you'd better use a printf piped to the stat command (thanks again mklement0) :
printf '%s\0' *."$1" | xargs -0 stat -c '%U' | sort | uniq -c | awk '{print $2,"\t",$1}'
You don't need a loop for this (unless you later need to loop over several folders), and changing the working directory in a script is rarely necessary. Also, reading ls output is usually not recommended.
Here's a version that replaces the loop, and uses du:
ext="$1"
printf "Folder '%s':\t" "$folder" >>report.txt
du -hc "$folder"/*."$ext" | sed -n '$p' >>report.txt
I have a folder in which there are directories
ABC_1
ABC_2
ABC_3
ABC_4
ABC_5
Test
XYZ
I want to sort them by date,remove the directories which do not contain ABC in their name and cd into the first directory.
I tried
cd $(/bin/ls -t1 | head -n 1)
This is not working.
Any help would be much appreciated
Thanks.
This will list only directories and filter out any folder that does not start with ABC_:
cd "$(ls -t1 -d */ |grep "^ABC_" |head -n1)"
UPDATE:
You actually do not need grep
cd "$(ls -t1 -d ABC_*/ | head -n1)"
cd "$(/bin/ls -t1 | grep ABC | head -n 1)"
The poster is wanting the first result after the list not the first result according to modification time so the "-t" option is not needed.
You also have to make sure that you're pulling just directories and not files.
This will do what you want:
cd $(ls -d [^ABC]*/ | head -n 1)
$() run command
ls -d search for directories
[^ABC]/ do not include any directories that start with ABC
head -n 1 return the first entry
cd change to the directory
I have around 1000 files (png) and need to move them into the corresponding directory and their sub-directory.
I do have 26 directories (A - Z) and below each directory the complete alphabet A-Z again. File names are 6 characters/digits long and have a png extension, e.g. e.g. AH2BC0.png
I would need to move the file AH2BC0.png into the directory A and within that directory into the sub-directory H, e.g.A->H->AH2BC0.png.
I have created following script which is not really working as expected:
#!/bin/bash
ls >LISTE.txt
for i in LISTE.txt; do
a=$(cat $i | cut -b 1 | tr '[:lower:]' '[:upper:]')
b=$(cat $i | cut -b 2 | tr '[:lower:]' '[:upper:]')
mkdir -p $a/$b
cat $i | xargs mv $a/$b
rm $i
done
Problem is that a) the sub-directory is not created and b) the files are not moved. Any suggestions or better ideas for the script?
Thanks
PS: I guess it's obvious that it's quite some years ago that I have created any bash scripts or coded so please bear with me.
PSS: working on MAC OSX bash 3.2
There's already a post showing a better program to do what you want but I thought I'd show you how to fix yours. Hopefully you'll find it informative.
#!/bin/bash
ls >LISTE.txt
for i in LISTE.txt; do
This loops over the single value LISTE.txt; replace it with:
for i in $(cat LISTE.txt); do
to loop over the contents of the file instead.
a=$(cat $i | cut -b 1 | tr '[:lower:]' '[:upper:]')
b=$(cat $i | cut -b 2 | tr '[:lower:]' '[:upper:]')
You want to use echo rather than cat in the above two lines, as you're after the name of the file not its content.
mkdir -p $a/$b
cat $i | xargs mv $a/$b
I don't think the above line does what you think it does... It will attempt to rename the $a/$b directory to C, where C is the content of file $i. Replace it with:
mv $i $a/$b
The following line is not needed:
rm $i
So simply delete it. It would only be necessary if you copied rather than moved the files using mv.
done
Here's your complete program after the changes I've suggested.
#!/bin/bash
ls >LISTE.txt
for i in $(cat LISTE.txt); do
a=$(echo $i | cut -b 1 | tr '[:lower:]' '[:upper:]')
b=$(echo $i | cut -b 2 | tr '[:lower:]' '[:upper:]')
mkdir -p $a/$b
mv $i $a/$b
done
#!/bin/bash
for item in *; do
first=${item:0:1}
second=${item:1:1}
folder="$first/$second"
mkdir -p $folder
mv $item $folder/
done
This scripts will sort the files by date then move the first 2500 files to another directory.
When I run below scripts, system prompt out Argument list too long msg. Anyone can help me enhance the scripts ? Thanks
NUM_OF_FILES=2500
FROM_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in
DESTINATION_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in_load
if [ ! -d $DESTINATION_DIRECTORY ]
then
echo "unused_file directory does not exist!"
mkdir $DESTINATION_DIRECTORY
echo "$DESTINATION_DIRECTORY directory created!"
else
echo "$DESTINATION_DIRECTORY exist!"
fi
echo "Moving $NUM_OF_FILES oldest files to $DESTINATION_DIRECTORY directory"
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -$NUM_OF_FILES |
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
You didn't say, but I assume this is where the problem occurs:
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -2500 | \
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
(You can verify it by adding "set -x" to the top of your script.)
The problem is that the kernel has a fixed maximum size of the total length of the command line given to a new process, and your exceeding that in the ls command. You can work around it by not using globbing and instead using grep:
ls -tr $FROM_DIRECTORY/ | grep '/MSCERC\*\.Z$' |head -2500 | \
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
(grep uses regular expressions instead of globs, so the pattern looks a little bit different.)
Change
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -2500 | \
xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
do something like the following:
find "$FROM_DIRECTORY" -maxdepth 1 -type f -name 'MSCERC*.Z' -printf '%p\t%T#\n' | sort -k2,2 -r | cut -f1 | head -$NUM_OF_FILES | xargs mv -t "$DESTINATION_DIRECTORY"
This uses find to create a list of files with modification timestamps, sorts by the timestamp, then removes the unneeded field before passing the output to head and xargs
EDIT
Another variant, should work with non GNU utils
find "$FROM_DIRECTORY" -type f -name 'MSCERC*.Z' -printf '%p\t%T#' |sort -k 2,2 -r | cut -f1 | head -$NUM_OF_FILES | xargs -i mv \{\} "$DESTINATION_DIRECTORY"
First of create a backup list of the files to be treated. Then read the backup file line-by-line and heal it. For example
#!/bin/bash
NUM_OF_FILES=2500
FROM_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in
DESTINATION_DIRECTORY=/apps/data01/RAID/RC/MD/IN_MSC/ERC/in_load
if [ ! -d $DESTINATION_DIRECTORY ]
then
echo "unused_file directory does not exist!"
mkdir $DESTINATION_DIRECTORY
echo "$DESTINATION_DIRECTORY directory created!"
else
echo "$DESTINATION_DIRECTORY exist!"
fi
echo "Moving $NUM_OF_FILES oldest files to $DESTINATION_DIRECTORY directory"
ls -tr $FROM_DIRECTORY/MSCERC*.Z|head -2500 > list
exec 3<list
while read file <&3
do
mv $file $DESTINATION_DIRECTORY
done
A quick way to fix this would be to change to $FROM_DIRECTORY, so that you can refer the files using (shorter) relative paths.
cd $FROM_DIRECTORY &&
ls -tr MSCERC*.Z|head -2500 |xargs -i sh -c "mv {} $DESTINATION_DIRECTORY"
This is also not entirely fool-proof, if you have too many files that match.