Is it possible to run bash script in a temporary folder other than the one in which it actually resides ?
My script uses a lot of filenames .I am concerned that one of the many names may coincide with others in the folder . I have named the files according to the data contained , taking reusability into consideration .
Does mktemp -d and tempfile -d do the same ? If so , can someone please illustrate its usage with an example.
Thanks in advance for the replies .
You can switch directories in a running script easily. Bash has a notion of the present working directory, which you can change at any time. For example:
dir=$(mktemp -d)
cd "$dir"
echo "Current directory changed: $PWD"
cd "$OLDPWD"
echo "Back in the old directory: $PWD"
Is it possible to run bash script in a temporary folder other than the
one in which it actually resides?
Yes, you can use cd in your script to change current directory
Does mktemp -d and tempfile -d do the same ? If so , can someone
please illustrate its usage with an example.
It does not consider contents, it creates a random name and makes sure there is no such directory:
tmpdir=$(mktemp -d)
cd $tmpdir
I don't see tempfile on a standard Linux system, and I'm not familiar with the command.
In the old days, we simply appended $$ at the end of file and directory names:
mkdir "mydir.$$"
But, mktemp replaces that with a much more secure and safer method.
Usage is generally:
$ my_temp_dir=$(mktemp -d -tmpdir=$temp_dir -t $template)
The $template is optional. It allows you to set a name. A template contains a series of XXX which the program can use to guarantee a unique name. If you don't specify $temp_dir, it will generally put the directory under /tmp.
The syntax takes advantage that mktemp creates a temporary directory and then echos out the name. Thus, you can capture the name of the temporary directory created.
Related
I am trying to loop through all of the files in a directory and
move them to a workspace (I need to do this to do this because the workspace doesn't have much storage).
Run a program which produces an output directory that contains all the files I will want to work with in the future
Delete the original file from the workspace (to save space in the workspace)and
Move the output directory out of the workspace and back to the storage space
I am able to do this for each file singly (i.e. each line works if I actually use the name of the files), but I can't get the for loop to work. I am quite new to this, so I probably did something simple wrong.
Can anyone see where I am going wrong?
for i in path_to_files; do
#copy to home directory (from scratch)
cp $i .
#Run IDBA
idba_ud -l $i -o '$i'_out
#remove file from work directory (limited space)
rm $i
#copy out directory back to scratch
cp -r '$i'_out path_to_files
done
I keep getting an error that says
syntax error near unexpected token `cp'.
I have also tried replacing cp with copy and i/$i with file/$file with no luck.
If this is indeed POSIX compatible shell (your code looks suspiciously like that, but you haven't specified the actually used shell), then:
You should always quote filenames, in case it contains spaces or other weird characters:
But you should not use single-quotes, as this will prevent shell from expanding your variables.
when appending text to substituted variables, use ${} notation (e.g. if $i expands to "murgel", then ${i}foo will expand to "murgelfoo", whereas $ifoo will expand to "" (an empty string) if there is no variable ifoo)
Thus try:
filepath=/path/to/files
for i in "${filepath}"/*; do
#copy to home directory (from scratch)
cp "${i}" .
#Run IDBA
idba_ud -l "${i}" -o "${i}_out"
#remove file from work directory (limited space)
rm $i
#copy out directory back to scratch
cp "${i}_out"/* "${filepath}"/
done
I have cobbled together a shell script to submit multiple jobs on a cluster, which it appears to without giving me an error message, but the output files are missing and the error log files are also empty. What the script supposed to do is 1.) make a bunch of new directories, 2.) copy four files to each (mainparams, extraparams, infile, and structurejobsubmissionfile) 3.) then submit each one to the cluster for it to run structure while changing one parameter in the mainparams file every tenth directory (that's the 's/changethis/'$k'/g' line).
Test running it on the front end gives no errors, the structure program is up to date on the cluster, and the cluster administrators don't see anything wrong. Thanks!
#!/bin/bash
reps=10
numK=10
for k in $(seq $numK);
do
for i in $(seq $reps);
do
#make folder name (ex. k4rep7)
tmpstr="k${k}rep${i}"
#echo "Making folder and filename $tmpstr"
#make the new folder
mkdir $tmpstr
#go to that folder
cd ./$tmpstr
#copy in the input files
cp ../str_in/* ./
#modify the recently copied input file here so source file remains the same
cp ./mainparams ./temp.txt
#change maxpops to current value of k and the directory for the files to the current directory
sed -e 's/changethis/'$k'/g' -e "s:pathforrunningstructurehere:$PWD:g" ./temp.txt > ./mainparams
#get rid of temporary file
rm ./temp.txt
#inside $i so run STRUCTURE here
qsub -q fnrgenetics -l nodes=1:ppn=1,walltime=20:00:00 structurejobsubmissionfile
#go back to parent directory
cd ../
done
done
I can't see anything obviously wrong, but I think the place that you'll find the answer lies in better logging and better error checking. Some of the things that you're not checking that you should:
Is $tmpstr created correctly? (will fail on disk full or if permissions are not set correctly)
does str_in/ exist, and is it a directory?
does it contain files?
does it contain mainparams?
is qsub in $PATH?
does the call to qsub return an error?
You can roll an error logging function of your own, or use a package like log4bash
I am new to unix and am practicing a simple script to unzip a load of files within a specified directory. I need the program to move the zipped file into another folder when it is done unzipping it (I called this oldzipped folder). For simplicity, I have removed the part of the code unzipping the file and currently have the program working for a specific file rather than the *tar.7z file extention. For some reason, the mv statement is not working. Unix is saying the following when I try to run the script. Could someone give me a hand with this? Again, I know this is the long way of doing things, but I want practice writing a script. Please be nice, as I am very new to Unix :(
unzip5: line 14: [ASDE0002.tar.7z]: command not found
#!~/bin/bash
# My program to try to unzip several files with ending of tar.7z
# I have inserted the ability to enter the directory where you want this to be done
echo "What file location is required for unzipping?"
read dirloc
cd $dirloc
mkdir oldzippedfiles
for directory in $dirloc
do
if
[ASDE0002.tar.7z]
then
mv -f ASDE0002.tar.7z $dirloc/oldzippedfiles
fi
done
echo "unzipping of file is complete"
exit 0
[ is the name of a (sometimes built-in) command which accepts arguments. As such you need to put a space after it as you would when invoking any other program. Also, you need a test. For example, to determine if the file exists and is a file, you need to use -f:
if [ -f ASDE0002.tar.7z ]
then
mv -f ASDE0002.tar.7z $dirloc/oldzippedfiles
fi
Here are some other possible tests.
I am trying to create a script that runs an another script and changes the name from the output.
Here is the script so far:
#! /bin/bash
i=1
for N in mediainput.iso mediainput2.iso
do
x264transcode $N
mv $N $((i++))
done
This don`t that well. It just moves the files and renames them.
I need to first run the x264transcode and then rename the output of that. Since they all get the same name when x264transcode as processed the files.
Its okey that the name the files are changed to are 1 then 2 and so on.
But it would be a plus if there where a method of getting the name of the folder the file was inside or the file itself. Maybe choosing between them for different scenarios.
Example below:
~/Videos/Summer Vacation 2009/dvd.iso
Output from x264: VIDEO01.mkv
Output from rename script: Summer-Vacation-2009.mkv
Does x264transcode always call its output VIDEO01.mkv? Are all the video files dvd.iso? If so, something like this, to also get the correct filename with hyphens:
cd ~/Videos
for I in */dvd.iso
do
x264transcode $I
mv VIDEO01.mkv `dirname $I|tr ' ' -`.mkv
end
This is assuming x264transcode stores VIDEO01.mkv in the current directory rather than the directory its input file is located in.
Need some help with this as my shell scripting skills are somewhat less than l337 :(
I need to gzip several files and then copy newer ones over the top from another location. I need to be able to call this script in the following manner from other scripts.
exec script.sh $oldfile $newfile
Can anyone point me in the right direction?
EDIT: To add more detail:
This script will be used for monthly updates of some documents uploaded to a folder, the old documents need to be archived into one compressed file and the new documents, which may have different names, copied over the top of the old. The script needs to be called on a document by document case from another script. The basic flow for this script should be -
The script file should create a new gzip
archive with a specified name (created from a prefix constant in the script and the current month and year e.g. prefix.september.2009.tar.gz) only if it
does not already exist, otherwise add to the existing one.
Copy the old file into the archive.
Replace the old file with the new one.
Thanks in advance,
Richard
EDIT: Added mode detail on the archive filename
Here's the modified script based on your clarifications. I've used tar archives, compressed with gzip, to store the multiple files in a single archive (you can't store multiple files using gzip alone). This code is only superficially tested - it probably has one or two bugs, and you should add further code to check for command success etc. if you're using it in anger. But it should get you most of the way there.
#!/bin/bash
oldfile=$1
newfile=$2
month=`date +%B`
year=`date +%Y`
prefix="frozenskys"
archivefile=$prefix.$month.$year.tar
# Check for existence of a compressed archive matching the naming convention
if [ -e $archivefile.gz ]
then
echo "Archive file $archivefile already exists..."
echo "Adding file '$oldfile' to existing tar archive..."
# Uncompress the archive, because you can't add a file to a
# compressed archive
gunzip $archivefile.gz
# Add the file to the archive
tar --append --file=$archivefile $oldfile
# Recompress the archive
gzip $archivefile
# No existing archive - create a new one and add the file
else
echo "Creating new archive file '$archivefile'..."
tar --create --file=$archivefile $oldfile
gzip $archivefile
fi
# Update the files outside the archive
mv $newfile $oldfile
Save it as script.sh, then make it executable:
chmod +x script.sh
Then run like so:
./script.sh oldfile newfile
something like frozenskys.September.2009.tar.gz, will be created, and newfile will replace oldfile. You can also call this script with exec from another script if you want. Just put this line in your second script:
exec ./script.sh $1 $2
A good refference for any bash scripting is Advanced Bash-Scripting Guide.
This guide explains every thing bash scripting.
The basic approach I would take is:
Move the files you want to zip to a directory your create.
(commands mv and mkdir)
zip the directory. (command gzip, I assume)
Copy the new files to the desired location (command cp)
In my experience bash scripting is mainly knowing how to use these command well and if you can run it on the command line you can run it in your script.
Another command that might be useful is
pwd - this returns the current directory
Why don't you use version control? It's much easier; just check out, and compress.
(apologize if it's not an option)