if then else statement will not loop properly - bash

I figured how to get an if then else statement to work but it now seems to have broken. =( I cannot work out what is going wrong!
There are up to 10 directories in ./ called barcode01 - 09 and one called unclassified. This script is supposed to go into each one, prep the directory for ~/Taxonomy.R (Which requires all the fastq files to be gzipped and put into a sub-directory titled "data". It then runs the ~/Taxonomy.R script to make a metadata file for each.
Edit the tmp.txt file is created using ls > tmp.txt then echo "0" >> tmp.txt to make a sacrificial list of directories for the script to chew through then stop when it gets to 0.
#!/bin/bash
source deactivate
source activate R-Env
value=(sed -n 1p tmp.txt)
if [ "$value" = "0" ]
then
rm tmp.txt
else
cd "$(sed -n 1p tmp.txt)"
gzip *fastq
#
for i in *.gz
do
mv "$i" "${i%.*}_R1.fastq.gz";
done
#this adds the direction identifier "R1" to all the fastq.gzips
mkdir Data
mv *gz Data
~/Taxonomy3.R
cd Data
mv * ..
cd ..
rm -r Data
cd ..
sed '1d' tmp.txt > tmp2.txt
mv tmp2.txt tmp.txt
fi
Currently, it is only making the metadata file in the first barcode directory.

If you indent your code, things will get a lot clearer.
On the other hand, modifying your tmp.txt file this way id slow and dangerous. Better traverse its contents only reading it.
#!/bin/bash
source deactivate
source activate R-Env
for value in $(<tmp.txt)
do
cd "$value"
gzip *fastq
for i in *.gz
do
# This adds the direction identifier "R1" to all the fastq.gzips
mv "$i" "${i%.*}_R1.fastq.gz"
done
mkdir Data
mv *gz Data
~/Taxonomy3.R
mv Data/* .
rmdir Data
cd -
done
rm tmp.txt
With this reworked script you only need to create the tmp.txt file WITHOUT adding any marker at the end (in fact, you never needed it, you could have checked for empty file).
For each folder in the script, the operations you wanted are executed. I simplified some folder changing, minimizing it to the required ones for the R script to properly run. To go back, I used cd -, which goes to the previous folder, that way you can have more than one leven in your tmp.txt file.
Hope everything else is clear.

Related

create a directory for every file and generate ā€œnā€ copies for each file

while I was looking for a solution for my files, I found something that is perfect, I include the answer here: https://unix.stackexchange.com/questions/219991/how-do-i-create-a-directory-for-every-file-in-a-parent-directory/220026#220026?newreg=94b9d49a964a4cd1a14ef2d8f6150bf8
but now, my problem is how can generate 50 copies to the directories generated by each file I was dealing with the following command line
ls -p | grep -v / | xargs -t -n1 -i bash -c 'for i in {1..50}; do cp {} "{}_folder/copy${i}_{}" ; done'
to get the following
-file1.csv---->folder_for_file1---->1copy_file1.csv,2copy_file1.csv,3copy_file1.csv........50copy_file1.csv
-file2.csv---->folder_for_file2---->1copy_file2.csv,2copy_file2.csv,3copy_file2.csv........50copy_file2.csv
-file3.csv---->folder_for_file3---->1copy_file3.csv,2copy_file3.csv,3copy_file3.csv........50copy_file3.csv
...
-file256.csv---->folder_forfile256---->1copy_file256.csv,2copy_file256.csv,3copy_file256.csv........50copy_file256.csv
How can I match this with the previous answer??, include the functional code of that answer
cd ParentFolder
for x in ./*.csv; do
mkdir "${x%.*}" && mv "$x" "${x%.*}"
done
all the credits to the person who generated this great answer and thanks in advance to everyone
Replace the move for a copy/remove and add a for loop:
cd ParentFolder
for x in ./*.csv; do
mkdir "${x%.*}"
for (( i=1;i<=50;i++ )); do # Create a loop, looping 50 times
cp "$x" "${x%.*}/copy$i_$x" # use i in the copy command
rm -f "$x" # Remove the file after the 50 copies
done
done
I have done some tests and I can publish the following code that works partially, because it effectively copies each file 50 times within the generated folder, but with the name "copy" to each new file, and also adds the extension .csv, but if someone can provide a solution to solve this would be great, I thank to #Raman Sailopal for his help and comments
code
cd pruebas
for x in ./*.csv; do
mkdir "${x%.*}"
for ((i=1;i<=50;i++)); do # Create a loop, looping 50 times
cp "$x" "${x%.*}/copy_$x_$i.csv" # use i in the copy command
#rm -f "$x" # Remove the file after the 50 copies
done
done

How to check if a file exists or not and create/delete if does/does not exist in shell

In shell, I want to check if a file exists or not then create if it doesn't exist or delete if it exists. For this I need a one liner and am trying to do something like:
ls | awk '\filename\' <if exist delete else create>
I need the ls as my problem has some command that outputs a list of strings that need to be pipelined to awk then possibly touch/mkdir.
#!/bin/bash
if [ -z "$1" ] || [ ! -f "$1" ] # $1 is input filename and -f check if $1 is a regular file
then
rm "$1" #delete the file
else
touch "$1" #create the file
fi
save the file as filecreator.sh
change the permission to allow execution with sudo chmod a+rx
while running the script use ./filecreator.sh yourfile.extension
You can see the file in your directory.
Using oc projects and oc new-project instad of ls and touch as indicated in a comment.
oc projects |
while read -r proj; do
if [ -d "$proj" ]; then
rm -rf "$proj"
else
oc new-project "$proj"
fi
done
I don't think there is a useful way to write this as a one-liner. If you like, you can replace the newlines with semicolons, except after then and else.
You really should put your actual requirements in the question itself. ls is a superbly useless example because it cannot list a file which doesn't already exist, and you should not use ls in scripts at all.
rm yourfile 2>/dev/null || touch yourfile
If the file existed before, rm will succeed and erase the file, and the touch won't be executed. You end up with no file afterwards.
If the file did not exist before, rm will fail (but the error message is not visible, since it is directed to the bitbucket), and due to the non-zero exit code of rm, the touch will be executed. You end up with an empty file afterwards.
Caveat: If the file exists, but you don't have permissions to remove it, you won't notice this error, due to the redirection of stderr. Hence, for debugging and later diagnosis, it might be better to redirect stderr to some file instead.

Replace file only if not being accessed in bash

My requirement is to replace file only when it is not being accessed. I have following snippet:
if [ -f file ]
then
while true
do
if [ -n "$(fuser "file")" ]
then
echo "file is in use..."
else
echo "file is free..."
break
fi
done
fi
{
flock -x 3
mv newfile file
} 3>file
But I have a doubt that I am not handling concurrency properly. Please give some insights and possible way to achieve this.
Thanks.
My requirement is to replace file only when it is not being accessed.
Getting requirements right can be hard. In case your actual requirement is the following, you can boil down the whole script to just one command.
My guess on the actual requirement (not as strict as the original):
Replace file without disturbing any programs reading/writing file.
If this is the case, you can use a very neat behavior: In Unix-like systems file descriptors always point to the file (not path) for which they where opened. You can move or even delete the corresponding path. See also How do the UNIX commands mv and rm work with open files?.
Example:
Open a terminal and enter
i=1; while true; do echo $((i++)); sleep 1; done > file &
tail -f file
The first command writes output to file and runs in the background. The second command reads the file and continues to print its changing content.
Open another terminal and move or delete file, for instance with
mv file file2
echo overwritten > otherFile
mv otherFile file2
rm file2
echo overwritten > file
echo overwritten > file2
While executing these commands have a look at the output of tail -f in the first terminal ā€“ it won't be affected by any of these commands. You will never see overwritten.
Solution For New Requirement:
Because of this behavior you can replace the whole script with just one mv command:
mv newfile file
Consider lsof.
mvWhenClear() {
while [[ -f "$1" ]] && lsof "$1"
do sleep $delay
done
mv "$1" "$2" # still allows race condition
}

bash script to pattern match and delete files

Problem: In a directory, I have files of the following form:
<account-number>-<invoice-number>, an example being:
123456-3456789
123456-6789023
123456-2568907
...
456789-2347890
456789-2344357
etc.
What I want to do is, if there are more than 1 invoices for the same account, delete all except the latest. If there's only one, leave it alone.
Thanks for any pointers.
You can use this awk based script:
mkdir _tmp
ls -rt *-*|awk -F'-' '{a[$1]=$0} END{for (i in a) system("mv " a[i] " _tmp/")}'
Once you're satisfied with the files in ./_tmp/ remove all files from current directory and move files over.
Here is a pure bash solution (replace echo with rm when you validate it)
for file1 in *-*
do
IFS=- arr=($file1)
for file2 in "${arr[0]}"*
do
[ "$file1" -nt "$file2" ] && echo "$file2"
done
done
A nice one in Bash:
(pseudo "in place" processing)
#!/bin/bash -e
ADIR="/path/to/account/directory"
TMP="$ADIR.tmp"
mkdir "$TMP" && rmdir "$TMP" && mv "$ADIR" "$TMP" && mkdir "$ADIR"
while IFS=- read ACCNT INVOICE < <( ls -t1 "$TMP" )
do
mv "$TMP/$ACCNT-$INVOICE" "$ADIR/$ACCNT-$INVOICE" && rm "$TMP/$ACCNT"*
done
rmdir "$ADIR.tmp"
what it does:
1 first move the a(ccounts) directory to a temporary directory. (is atomic)
2 in a loop: list newest invoice, move it to the new directory, delete invoices with same account.
3 remove temporary directory
PROs:
solid, safe, short, reasonably fast and halts on serious errors
CONs:
Very definitive, be sure to have always a backup
Comment:
You may have noticed mkdir "$TMP" && rmdir "$TMP"
This is on purpose: rmdir gives the same returnvalue for "dir not exist" as "dir not empty"
so instead of checking which of the two it is
[ -d $DIRNAME ] && { rmdir $DIRNAME || exit }
I used the above construction.
Also the ls -t1 "$TMP may be at a strange place at first sight
But it is OK, every iteration it will be executed again (but only the first line is read)

Recycle bin in bash problem

I need to make a recycle bin code using bash. Here is what I have done so far. My problem is that when I move a file with the same name into the trash folder it just overwrites the previous file. Can you give me any suggestions on how to approach this problem?
#!/bin/bash
mkdir -p "$HOME/Trash"
if [ $1 = -restore ]; then
while read file; do
mv $HOME/Trash/$2 /$file
done < try.txt
else
if [ $1 = -restoreall ]; then
mv $HOME/Trash/* /$PWD
else
if [ $1 = -empty ]; then
rm -rfv /$HOME/Trash/*
else
mv $PWD/"$1"/$HOME/Trash
echo -n "$PWD" >> /$HOME/Bash/try
fi
fi
fi
You could append the timestamp of the time of deletion to the filename in your Trash folder. Upon restore, you could strip this off again.
To add a timestamp to your file, use something like this:
DT=$(date +'%Y%m%d-%H%M%S')
mv $PWD/"$1" "/$HOME/Trash/${1}.${DT}"
This will, e.g., create a file like initrd.img-2.6.28-11-generic.20110615-140159 when moving initrd.img-2.6.28-11-generic.
To get the original filename, strip everything starting from the last dot, like with:
NAME_WITHOUT_TIMESTAMP=${file%.*-*}
The pattern is on the right side after the percentage char. (.* would also be enough to match.)
Take a look how trash-cli does it. It's written in Python and uses the same trash bin as desktop environments. Trash-cli is available at least in the big Linux distributions.
http://code.google.com/p/trash-cli/
Probably the easiest thing to do is simply add -i to the invocation of mv. That will prompt the user whether or not to replace. If you happen to have access to gnu cp (eg, on Linux), you could use cp --backup instead of mv.

Resources