bash script for apending dates onto files - bash

I am trying to write a bash script which takes in a file as $1 and then copies it and appends the year,month, and date onto the end of the file. So for example if i had a file foo.txt it would turn into foo.txt.2010.11.16. I'm not really 100% sure how to do this, any suggestions?

Simple version:
#!/bin/bash
cp "$1" "$1".`date +%Y.%m.%d`
Fancier version:
#!/bin/bash
#
# date_tag_files file1 [file2 file3 ...]
#
for f in $*
do
cp "$f" "$f".`date +%Y.%m.%d`
done

You can do this by using $(date ...) to get the current date into an environment variable.
The following script allows you to pass in any number of file names and it will attempt to copy all of them to similar files with the date appended.
It gets the date once in case you try to run it around midnight and you want them all to get the same date even if you cross into the following day during the process.
#!/usr/bin/bash
if [[ $# -lt 1 ]] ; then
echo 'Usage: datecp <filename> ...'
exit 1
fi
dt=$(date +%Y.%m.%d)
while [[ $# -ne 0 ]] ; do
if [[ ! -f "$1" ]] ; then
echo 'Warning:' $1 'is not a regular file, not copied'
else
newf="$1.$dt"
cp "$1" "$newf"
echo "$1" '-->' "$newf"
fi
shift
done
The output for the last one is along the lines of:
pax> datecp
Usage: datecp <filename> ...
pax> datecp xyz
xyz --> xyz.2010.11.17
pax> datecp xyz abc
xyz --> xyz.2010.11.17
abc --> abc.2010.11.17
pax> ./qq.sh xyz qwert abc
xyz --> xyz.2010.11.17
Warning: qwert is not a regular file, not copied
abc --> abc.2010.11.17

Related

Move just those files named as specific rows in a sample sheet

Imagine I have these files in my working directory in bash:
123.tsv 456.tsv 789.tsv 101112.tsv 131415.tsv
and that I have this sample sheet (tab separated):
sampleID tissue
123 lung
124 bone
456 lung
457 bone
Now, I want to move those files corresponding to lung samples to a new directory, so I would like to have the following files in the new directory:
123.tsv
456.tsv
I was trying to use:
awk -F"\t" '$2 == "lung"'
But I am not sure about how to include this in a for loop to select filenames included in the first column of the output file from the awk command.
How can I solve this?
If row number is larger 1 and second column contains lung then print content of first column with some text around it:
mkdir new_dir
awk 'NR>1 && $2=="lung" {print "mv", $1 ".tsv new_dir"}' sample.sheet
If output looks fine, append | sh to awk line to execute commands.
#!/bin/sh
#
#
me=$( basename "${0}" )
# Adjust these as needed. If you want to use your current
# working directory change (or remove) `/tmp/` to `./`.
old_dir="/tmp/foo"
new_dir="/tmp/bar"
list="/tmp/sample_sheet"
# Make sure all the pieces are available. Exit if not.
if [ ! -d "${old_dir}" ]
then
echo "ERROR: ${me}: Source '${old_dir}' does not exist." 1>&2
exit 1
elif [ ! -d "${new_dir}" ]
then
echo "ERROR: ${me}: Target '${new_dir}' does not exist." 1>&2
exit 2
elif [ ! -r "${list}" ]
then
echo "ERROR: ${me}: Sample sheet input '${list}' does not exist." 1>&2
exit 3
fi
# Iterate over the first column in `${list}`.
for file in $( awk 'NR>1 && $2=="lung" {print $1".tsv"}' "${list}" )
do
# If the file exists move it, if not do nothing.
if [ -f "${old_dir}/${file}" ]
then
echo "INFO: ${me}: mv ${old_dir}/${file} ${new_dir}/${file}"
mv "${old_dir}/${file}" "${new_dir}/${file}"
fi
done
Here's a script that you can run like, for example, this:
./move_files.sh lung
This works for both cases (lung and bone), and is general. Put this into a file called move_files.sh:
#!/usr/bin/env bash
files=$(sed -e "s/\([0-9]\{3\}\)\( *$1\)/\1/g" <(grep $1 eg.sheet))
if [ ! -d $1 ]; then
mkdir $1
fi
for t in ${files[#]}; do
mv "./$t.tsv" $1
done
With the following directory content:
101112.tsv 123.tsv 124.tsv 131415.tsv 456.tsv 457.tsv 789.tsv eg.sheet move_files.sh
and eg.sheet containing:
sampleID tissue
123 lung
124 bone
456 lung
457 bone
... running the script with
./move_files.sh lung
... results in 123.tsv and 456.tsv being moved into a newly created lung directory (or simply moved there if the directory already exists).
You can then simply run
./move_files.sh bone
to move 124.tsv and 457.tsv to a newly created bone directory. Of course this is then generalisable to whatever is in eg.sheet.
Side note: you must run chomd +x move_files.sh in order to use it in the way I've suggested. Otherwise, you can invoke it with bash move_files.sh lung instead.
EDIT:
To address the point raised by keithpjolley in the comments, this can still work with "tissues" such as "eye lash" just by quoting the $1 variable throughout and by calling it with a quoted string (e.g., ./move_files.sh "eye lash"):
#!/usr/bin/env bash
files=$(sed -e "s/\([0-9]\{3\}\)\( *$1\)/\1/g" <(grep "$1" eg.sheet))
if [ ! -d "$1" ]; then
mkdir "$1"
fi
for t in ${files[#]}; do
mv "./$t.tsv" "$1"
done

How to add a character to folder names when using the ls command?

I would like to display a character at the beginning of all folder names when typing the ls command.
So instead of this:
ls
Folder-1 Folder-2 file.txt
It displays this:
ls
📁Folder-1 📁Folder-2 file.txt
Is there a script I can write in my .bash_profile to do this?
You can make a custom script that makes it for you:
#!/bin/bash
lsmod=($(ls)) # Convert ls output to an array
for folder in ${lsmod[#]}; do # Iterate over ls results
if [[ -d $folder ]]; then # If this is a folder then...
echo "=> $folder" # Put your char
else # If not, display normally
echo "$folder"
fi
done
Folders are pointed out with a =>. Hope this helps.
One line approach:
for i in `ls`; do echo $([[ -d "$i" ]] && echo "=> $i" || echo "$i"); done;

How to grave the Date from this command in bash script

!/bin/bash
# When a match is not found, just present nothing.
shopt -s nullglob
# Match all .wav files containing the date format.
files=(*[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]*.wav)
if [[ ${#files[#]} -eq 0 ]]; then
echo "No match found."
fi
for file in "${files[#]}"; do
# We get the date part by part
file_date=''
# Sleep it to parts.
IFS="-." read -ra parts <<< "$file"
for t in "${parts[#]}"; do
# Break from the loop if a match is found
if [[ $t == [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] ]]; then
file_date=$t
break
fi
done
# If a value was not assigned, then show an error message and continue to the next file.
# Just making sure there is nothing in Array and date before it moves on
if [[ -z $file_date ]]; then
continue
fi
file_year=${file_date:0:4}
file_month=${file_date:4:2}
mkdir -p "$file_year/$file_month"
# -- is just there to not interpret filenames starting with - as options.
echo "Moving: ./"$file "to: " "./"$file_year"/"$file_month
mv "$file" "$file_year/$file_month"
done
Now there are files I would need to do the date to gra the date and then move it like I do now. for example there is a file called meetme.. its a wav file and I have DIR with YYYY/MM and would like to move thoses files without YYYYMMDD in file name already
If you're writing a program to do something with that information, then you might prefer seconds-since-epoch and use date to get the date in the desired format.
$ date -d #$(stat --format='%Y' testdisk.log) +%Y%m%d
20130422
You can also get the ascii representation, and then manipulate the string.
$ stat --format='%y' testdisk.log
2013-04-22 09:11:39.000000000 -0500
$ date_st=$(stat --format='%y' testdisk.log)
$ date_st=${date_st/ */}
$ date_st=${date_st//-/}
$ echo ${date_st}
20130422

Add random text before file name extension

I have this string:
2013/07./penguin pingouin pinguino 365.png
...that I want to effectively be renamed to
2013/07./penguin pingouin pinguino 365${RANDOM}.png
I'm thinking that sed might be able to insert random bits 4 spaces inward from the last character in the string. I don't know how to do that (if it's even possible or the best method).
Background: This is part of a picture sorting script that I'm trying to incorporate a rename ability when it finds duplicates. The extension is not always .png, but it will always be three characters.
Try something like:
for file in *; do
ext="${i##*.}";
mv "$file" "${file%.*}${random}.${ext}";
done
$ random="my random stuff" #
$ touch "penguin pingouin pinguino 365.png"
$ ls
penguin pingouin pinguino 365.png
$ for file in *; do
ext="${i##*.}"; mv "$file" "${file%.*}${random}.${ext}";
done
$ ls
penguin pingouin pinguino 365my random stuff.png
Here's another way to backup files into a directory:
moveto () {
file=$1
if [[ -f "$file" ]]; then
echo "no such file: $file"
return 1
fi
dir=$2;
if [[ -d "$dir" ]]; then
echo "no such directory: $dir"
return 1
fi
target="$dir/$(basename "$file")"
if [[ -f "$target" ]]; then
base=${target%.???}
ext=${target##*.}
prev=("$base"_*)
for ((i=${#prev[#]}-1; i>=0; i--)); do
if [[ ${prev[i]} =~ "$base"_0*([0-9]+) ]]; then
n=${BASH_REMATCH[1]}
mv "${prev[i]}" "$(printf "%s_%03d.%s" "$base" $((n+1)) "$ext")"
fi
done
mv "$target" "$base"_001.$ext
fi
mv "$file" "$target"
}
The first time you "moveto" a file to a dir (moveto file.txt backups), you get "backups/file.txt"
The 2nd time, you'll have "backups/file.txt" and "backups/file_001.txt"
The 3rd time, you'll have "backups/file.txt", "backups/file_001.txt" and "backups/file_002.txt"
And so on.
The following snippet should do what you need in sed sed -r 's/(.[a-zA-Z]+$)/$RANDOM\1'

bash loop between two given dates

I'm trying to create a script that will loop through files that have their filenames written in the following format: yyyymmdd.hh.filename.
The script is called with:
./loopscript.sh 20091026.00 23
./loopscript.sh 20091026.11 15
./loopscript.sh 20091026.09 20091027.17
The need is for the script to check each hour between those two given dates/hours.
e.g.
cat 20091026.00.filename |more
cat 20091026.01.filename |more
...
cat 20091026.23.filename |more
cat 20091027.01.filename |more
cat 20091027.02.filename |more
...
and so on.
any idea how to go about this? I don't have any difficulty with standard 0 - x loops. or simple for loops. Just not sure how to go about the above.
How about this:
#!/bin/bash
date1=$1
date2=$2
#verify dates
if ! date -d "$date1" 2>&1 > /dev/null ;
then echo "first date is invalid" ; exit 1
fi
if ! date -d "$date2" 2>&1 > /dev/null ;
then echo "second date is invalid" ; exit 1
fi
#set current and end date
current=$(date -d "$date1")
end=$(date -d "$date2 +1 hours")
#loop over all dates
while [ "$end" != "$current" ]
do
file=$(date -d "$current" +%Y%m%d.%H)
cat $file."filename" | more
current=$(date -d "$current +1 hours")
done
To process each file between two given date/hours, you can use the following:
#!/usr/bin/bash
#set -x
usage() {
echo 'Usage: loopscript.sh <from> <to>'
echo ' <from> MUST be yyyymmdd.hh or empty, meaning 00000000.00'
echo ' <to> can be shorter and is affected by <from>'
echo ' e.g., 20091026.00 27.01 becomes'
echo ' 20091026.00 20091027.01'
echo ' If empty, it is set to 99999999.99'
echo 'Arguments were:'
echo " '${from}'"
echo " '${to}'"
}
# Check parameters.
from="00000000.00"
to="99999999.99"
if [[ ! -z "$1" ]] ; then
from=$1
fi
if [[ ! -z "$2" ]] ; then
to=$2
fi
## Insert this to default to rest-of-day when first argument
## but no second argument. Basically just sets second
## argument to 23 so it will be transformed to end-of-day.
#if [[ ! -z "$1"]] ; then
# if [[ -z "$2"]] ; then
# to=23
# fi
#fi
if [[ ${#from} -ne 11 || ${#to} -gt 11 ]] ; then
usage
exit 1
fi
# Sneaky code to modify a short "to" based on the start of "from".
# ${#from} is the length of ${from}.
# $((${#from}-${#to})) is the length difference between ${from} and ${to}
# ${from:0:$((${#from}-${#to}))} is the start of ${from} long enough
# to make ${to} the same length.
# ${from:0:$((${#from}-${#to}))}${to} is that with ${to} appended.
# Voila! Easy, no?
if [[ ${#to} -lt ${#from} ]] ; then
to=${from:0:$((${#from}-${#to}))}${to}
fi
# Process all files, checking that they're inside the range.
echo "From ${from} to ${to}"
for file in [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9].[0-9][0-9].* ; do
if [[ ! ( ${file:0:11} < ${from} || ${file:0:11} > ${to} ) ]] ; then
echo " ${file}"
fi
done
When you create the files 20091026.00.${RANDOM} through 20091028.23.${RANDOM} inclusive, this is a couple of sample runs:
pax> ./loopscript.sh 20091026.07 9
From 20091026.07 to 20091026.09
20091026.07.21772
20091026.08.31390
20091026.09.9214
pax> ./loopscript.sh 20091027.21 28.02
From 20091027.21 to 20091028.02
20091027.21.22582
20091027.22.30063
20091027.23.29437
20091028.00.14744
20091028.01.6827
20091028.02.10366
pax> ./loopscript.sh 00000000.00 99999999.99 # or just leave off the parameters.
20091026.00.25772
20091026.01.25964
20091026.02.21132
20091026.03.3116
20091026.04.6271
20091026.05.14870
20091026.06.28826
: : :
20091028.17.20089
20091028.18.13816
20091028.19.7650
20091028.20.20927
20091028.21.13248
20091028.22.9125
20091028.23.7870
As you can see, the first argument must be of the correct format yyyymmdd.hh. The second argument can be shorter since it inherits the start of the first argument to make it the correct length.
This only attempts to process files that exist (from ls) and of the correct format, not every date/hour within the range. This will be more efficient if you have sparse files (including at the start and the end of the range) since it doesn't need to check that the files exist.
By the way, this is the command that created the test files, if you're interested:
pax> for dt in 20091026 20091027 20091028 ; do
for tm in 00 01 02 ... you get the idea ... 21 22 23 ; do
touch $dt.$tm.$RANDOM
done
done
Please don't type that in verbatim and then complain that it created files like:
20091026.you.12345
20091028.idea.77
I only trimmed down the line so it fits in the code width. :-)
One possible solution: convert dates into standard Unix representation of "Seconds passed since the epoch" and loop, increasing this number by 3600 (number of seconds in an hour) each iteration. Example:
#!/bin/bash
# Parse your input to date and hour first, so you get:
date_from=20090911
hour_from=10
date_to=20091026
hour_to=01
i=`date --date="$date_from $hour_from:00:00" +%s`
j=`date --date="$date_to $hour_to:00:00" +%s`
while [[ $i < $j ]]; do
date -d "1970-01-01 $i sec" "+%Y%m%d.%H"
i=$[ $i + 3600 ]
done

Resources