I have some lines have same structure like
1000 AS34_59329 RICwdsRSYHSD11-2-IPAAPEK-93 /ifshk5/BC_IP/PROJECT/T1
1073/T11073_RICekkR/Fq/AS34_59329/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IP
AAPEK-93_1.fq.gz /ifshk5/BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_5932
9/111220_I631_FCC0E5EACXX_L4_RICwdsRSYHSD11-2-IPAAPEK-93_2.fq.gz /ifshk5/
BC_IP/PROJECT/T11073/T11073_RICekkR/Fq/AS34_59329/clean_111220_I631_FCC0E5EACXX_
L4_RICwdsRSYHSD11-2-IPAAPEK-93_1.fq.gz.total.info 11.824 0.981393
43.8283 95.7401 OK
And I want to get the Bold part to check whether in /home/jesse/ has this folder, if not create mkdir /home/jesse/AS34_59329
I use this code
! /bin/bash
myPath="/home/jesse/"
while read myline
do
dirname= echo "$myline" | awk -F ' ' '{print $2}'
echo $dirname
myPath= $myPath$dirname
echo $myPath
mkdir -p "$myPath"
done < T11073_all_3254.fq.list
But it can't mkdir and show the path name, it shows
-bash: /home/jesse/: is a directory
/home/jesse/
AS39_59324
read can read each field into a separate variable, and mkdir -p will create a dir only if it doesn't exist:
path="/home/jesse"
while read _ dir _
do
mkdir -p "$path/$dir"
done < T11073_all_3254.fq.list
for will iterate over each whitespace separated token. Try this instead.
#!/usr/bin/env bash
# Invoke with first arg as file containing the lines
# foo.sh <input_filename>
for i in `cat $1 | cut -d " " -f2`
do
if [ -d /home/jesse/$i ]
then
echo "Directory /home/jesse/$i exists"
else
mkdir /home/jesse/$i;
echo "Directory /home/jesse/$i created"
fi
done
Related
I wrote a bash script trying to generate one directory named after each file inside the directory from which I run the script.
Original directory= /home/agalvez/data//sims/phylip_format
sim1.phylip
sim2.phylip
Directories to create = sim1 sim2
The contents of these new directories should be a copy of the original file that names the new directory and an extra file called "input". This file should contain the name of the .phylip file as well as the following:
"Name of original file"
U
5
Y
/home/agalvez/data/sims/trees/tree_nodenames.txt
After that I want to run the following command (sequentially) in all these new directories:
phylip dollop < input > screenout
My approach is the following one but it is not working:
!/bin/bash
for f in *.phylip;
mkdir /home/agalvez/data/sims/dollop/$f;
cp $f /home/agalvez/data/sims/dollop/$f;
cd /home/agalvez/data/sims/dollop/$f;
echo "$f" | cat > input;
echo "U" | cat >> input;
echo "5" | cat >> input;
echo "Y" | cat >> input;
echo "/home/agalvez/data/sims/trees/tree_nodenames.txt" | cat >> input;
phylip dollop < input > screenout;
;done
Edit: The error messge looks like this:
line 4: syntax error near unexpected token `mkdir'
line 4: ` mkdir /home/agalvez/data/sims/dollop/$f;'
FINAL SOLUTION:
#!/bin/bash
for f in *.phylip;
do
mkdir /home/agalvez/data/sims/dollop/$f;
cp /home/agalvez/data/sims/phylip_format/$f /home/agalvez/data/sims/dollop/$f;
cd /home/agalvez/data/sims/dollop/$f;
echo "$f" | cat > input;
echo "U" | cat >> input;
echo "5" | cat >> input;
echo "Y" | cat >> input;
echo "/home/agalvez/data/sims/trees/tree_nodenames.txt" | cat >> input;
phylip dollop < input > screenout;
done
The immediate problem is that you are lacking a do at the beginning of the loop body; but you'll want to refactor this code to avoid hardcoding the directory structure etc.
The first line needs to start with literally the two characters # and ! in order to be a valid shebang.
Notice also When to wrap quotes around a shell variable?
The printf could be replaced with a here document; I like the compactness of printf here.
#!/bin/bash
for f in *.phylip; do
mkdir -p dollop/"$f"
cp "$f" dollop/"$f"
cd dollop/"$f"
printf "%s\n" "$f" "U" "5" "Y" \
"/home/agalvez/data/sims/trees/tree_nodenames.txt" |
phylip dollop > screenout
done
Going forward, try http://shellcheck.net/ for diagnosing many common beginner problems in shell scripts.
Assuming you have a directory named pingping in your ${HOME} folder with files 1.txt, 2.txt, 3.txt. You can accomplish that like this. Modify this code to suit your needs.
#! /bin/bash
working_directory="${HOME}/pingping/"
cd $working_directory
for f in *.txt
do
mkdir "${f%%.*}"
if [ -f "${f%%.*}.txt" ]
then
if [ -d "${f%%.*}" ]
then
cp ${f%%.*}.txt ${f%%.*}
echo "Done copying"
#phylip dollop < input > screenout
#echo "Succesfully ran the command
fi
else
echo "not found"
fi
done
I have a CSV which has three columns: object-ID, image-url1, image-url2. I'd like to be able to run a bash script that does the following for each row in the CSV:
create a new folder using 'object-ID' as the folder name
download both images into that folder
repeat for each row
I've got this code but it needs some help!
IFS=$'\n';
for file in `cat <filename.csv>`; do
echo "Creating folder $object-ID";
mkdir $object-ID
echo "Downloading image 1";
wget $image-url1
echo "Downloading image 2";
wget $image-url2
done
Try this:
while IFS=, read objid url1 url2;
do
echo "Creating folder $objid"
mkdir -p "$objid"
# Run in a subshell
(
cd "$objid"
echo "Downloading image 1"
wget "$url1"
echo "Downloading image 2"
wget "$url2";
)
done < myfile.csv
It assumes your CSV uses comma (,) as a separator. This can be adjusted by changing the IFS=, part in the while loop.
Also, if $objid contains forward slashes (/) in it, mkdir -p will treat it as a path with subdirectories and create all of them. If that's undesirable you can replace / in $objid prior to mkdir like so:
objid="${objid//\//_}"
With read :
while IFS=',' read id image_one image_two; do
[ ! -d "${id}" ] && mkdir "${id}"
for img in ${image_one} ${image_two}; do
printf "Downloading %s" "${img}"
wget -P "${id}" "${img}"
echo "---"
done
done < file.csv
For each line : creates directory based on id value if directory doesn't exist and retrieves images in created dir (with -P option of the wget).
With awk:
awk -F "," '{
print "mkdir",$1"; echo wget -P",$1,$2"; echo wget -P",$1,$3
}' filename.csv | bash
#!/bin/bash
if test $# -ne 2
then
echo "Error : Invalid number of arguments"
else
if [ -d $1 ]
then
if [[ $2 =~ ^[0-9]+$ ]]
then
ls -l $1 | while read line
do
eval "echo $line | cut -d' ' -f5" | while read ln
do
if [[ $ln -gt $2 ]]
then
echo $line
fi
done
done
else
echo $2" is not a integer"
fi
else
echo "The repertory "$1" does not exist "
fi
fi
The question was to make cpp , that works like the command cp . The script it's supposed to react correctly if we don't give 2 argument. I don't understand what this script do from line 10 .
This code is the following of this post Explain me 2 lines of this shell script.
Thanks
Without working through the code line by line and explaining it, I would point you at http://explainshell.com, which takes lines of shell code and puts commentary from the manual on each parameter.
E.g, this is part of line 12 above: http://explainshell.com/explain?cmd=echo+%24line+%7C+cut+-d%27+%27+-f5
It should help you go through it line by line and work out what is going on.
In words: It selects the lines from ls -l from a directory $1 which have a size bigger than $2.
If that code is in a file called script.sh, it is called like:
$ ./script.sh /home/user 130000
And it will print all lines of ls -l /home/user which have a size bigger than 130000.
I do not know why the eval in:
eval "echo $line | cut -d' ' -f5" | while read ln
The line will work the same as:
echo $line | cut -d' ' -f5 | while read ln
I have a file input.txt that looks as follows.
abas_1.txt
abas_2.txt
abas_3.txt
1fgh.txt
3ghl_1.txt
3ghl_2.txt
I have a folder ff. The filenames of this folder are abas.txt, 1fgh.txt, 3ghl.txt. Based on the input file, I would like to create and rename the multiple copies in ff folder.
For example in the input file, abas has three copies. In the ff folder, I need to create the three copies of abas.txt and rename it as abas_1.txt, abas_2.txt, abas_3.txt. No need to copy and rename 1fgh.txt in ff folder.
Your valuable suggestions would be appreciated.
You can try something like this (to be run from within your folder ff):
#!/bin/bash
while IFS= read -r fn; do
[[ $fn =~ ^(.+)_[[:digit:]]+\.([^\.]+)$ ]] || continue
fn_orig=${BASH_REMATCH[1]}.${BASH_REMATCH[2]}
echo cp -nv -- "$fn_orig" "$fn"
done < input.txt
Remove the echo if you're happy with it.
If you don't want to run from within the folder ff, just replace the line
echo cp -nv -- "$fn_orig" "$fn"
with
echo cp -nv -- "ff/$fn_orig" "ff/$fn"
The -n option to cp so as to not overwrite existing files, and the -v option to be verbose. The -- tells cp that there are no more options beyond this point, so that it will not be confused if one of the files starts with a hyphen.
using for and grep :
#!/bin/bash
for i in $(ls)
do
x=$(echo $i | sed 's/^\(.*\)\..*/\1/')"_"
for j in $(grep $x in)
do
cp -n $i $j
done
done
Try this one
#!/bin/bash
while read newFileName;do
#split the string by _ delimiter
arr=(${newFileName//_/ })
extension="${newFileName##*.}"
fileToCopy="${arr[0]}.$extension"
#check for empty : '1fgh.txt' case
if [ -n "${arr[1]}" ]; then
#check if file exists
if [ -f $fileToCopy ];then
echo "copying $fileToCopy -> $newFileName"
cp "$fileToCopy" "$newFileName"
#else
# echo "File $fileToCopy does not exist, so it can't be copied"
fi
fi
done
You can call your script like this:
cat input.txt | ./script.sh
If you could change the format of input.txt, I suggest you adjust it in order to make your task easier. If not, here is my solution:
#!/bin/bash
SRC_DIR=/path/to/ff
INPUT=/path/to/input.txt
BACKUP_DIR=/path/to/backup
for cand in `ls $SRC_DIR`; do
grep "^${cand%.*}_" $INPUT | while read new
do
cp -fv $SRC_DIR/$cand $BACKUP_DIR/$new
done
done
My assignment is to write a Unix shell script that asks the user for the name of a directory, and then works exactly like find.
Here is what I have so far:
#!/bin/bash
dir_lister()
{
cd "$1"
echo "$1"
list=$(ls -l ${1})
nolines=$(echo "$list" | awk 'END{printf "%d",NF}')
if [ $nolines -eq 2 ]
then
echo "$1"
return
fi
filelist=$(echo "$list" | grep ^-.*)
dirlist=$(echo "$list" | grep ^d.*)
filename=$(echo "$filelist"| awk '{printf "%s\n",$NF}')
present=$(pwd)
echo "$filename"| awk -v pres=$present '{printf "%s/%s\n",pres,$0}'
dirlist2=$(echo "$dirlist" | awk '{printf "%s\n",$NF}')
echo "$dirlist2" | while IFS= read -r line;
do
nextCall=$(echo "$present/$line");
dir_lister $nextCall;
cd ".."
done
cd ".."
}
read -p "Enter the name of the direcotry: " dName
dir_lister $dName
The problem is, after a depth of three directories, this script gets into an infinite loop, and I don't see why.
EDIT:
Here is the code i came up with after looking at your answer, it still doesn't go more than 1 directory depth:
#!/bin/bash
shopt -s dotglob # don't miss "hidden files"
shopt -s nullglob # don't fail on empty directories
list_directory()
{
cd "$2"
cd "$1"
##echo -e "I am called \t $1 \t $2"
for fileName in "$1/"*
do
##echo -e "hello \t $fileName"
if [ -d "$fileName" ];
then
echo "$fileName"
list_directory $fileName $2
else
echo "$fileName"
fi
done
}
read -p "Enter the direcotory Name: " dirName
var=$(pwd)
list_directory $dirName $var
Okay, that is completely the wrong way to list files in a directory (see ParsingLs). I'll give you the pieces and you should be able to put them together into a working script.
Put this at the top of your script:
shopt -s dotglob # don't miss "hidden files"
shopt -s nullglob # don't fail on empty directories
Then you can easily loop over directory contents with:
for file in "$directory/"* ; do
#...
done
Test if you have a directory:
if [ -d "$file" ] ; then
# "$file" is a directory, recurse...
fi