I'm currently running a Bash script called (log2csv) that runs against a specified .log file. Sitting in the desired directory I can type in terminal:
log2csv Red1_1.log
This will create Red1_1.csv
This is my current bash script:
#!/bin/bash
for path
do
base=$(basename "$path")
noext="${base/.log}"
/Users/joshuacarter/bin/read_scalepack.pl "$path" > "${noext}.csv"
done
This script is actually running a perl script on the specified log and putting the results in a CSV output.
I can alternatively run in terminal:
log2csv *.log
This will run the script against all .log files in the current directory and create .csv files for every one.
What I would like the script to do is only run on .log files that haven't had .csv files created for them. After doing some research I think I possibly can use inotifywait to achieve this, but I'm unsure how to make this work in my script? I also have read that this may be an issue if you overwrite a file. Any help or ideas would be most appreciated!
What I would like the script to do is only run on .log files that haven't had .csv files created for them.
Simply skip those .log files whose corresponding .csv files already exist:
for path
do
base=$(basename "$path")
noext="${base/.log}"
[ -e "${noext}.csv" ] && continue # <---------------
/Users/joshuacarter/bin/read_scalepack.pl "$path" > "${noext}.csv"
done
Related
I have a text file that has a list of filenames.Now I want to see if these files are present in a specific folder using bash script. I have no experience in writing bash scripts.So please help me with this
a simple for loop can do what you need :
for file in $(cat yourfilelist.txt); do ls -l /folder/to/check/"$file"; done
for missing files you'd get No such file or directory
Below is my .sh file
sh summaryByClient.sh $1 - takes around 10 mins to fetch the required data
mv summary.html ~/public_html/chats/ - **this is not happening**
exit 0
I do not understand why mv summary.html ~/public_html/chats/ this is not working inside .sh file, However I am able to mv separately using the same above command.
Could you be running into the issue from this answer with expanding the user's home directory? What happens if you write your script like this:
#!/bin/bash
# Other tasks to retrieve summary.html done here
mv ./summary.html $HOME/public_html/chats/
exit 0
Also, it's always a good idea to check whether the destination directory exists before a mv command. Examples are shown in this answer
Hi this is my shell script to copy files from one directory to another directory with timestamp.But my script shows too many arguments.I want to copy files from one directory to another.What error in my code.
Date_Val="$(date +%Y%m%d%H%M%S)";
cd /etl_mbl/SrcFiles/
if [ -f /etl_mbl/SrcFiles/SrcFiles_TEMP*.csv ]
then
cp /etl_mbl/SrcFiles/SrcFiles_TEMP/*.csv /etl_mbl/SrcFiles/Archive/*_$Date_Val.csv
fi
The reason you got "too many arguments" error is that the wildcard in the "if" statement expands to a multitude of files. Please also note that you cannot have wildcards in the destination of a "cp". You probably want something like this:
#!/bin/bash
Date_Val="$(date +%Y%m%d%H%M%S)";
for file in ./src/*.csv; do
filename=${file##*/}
basename=${filename%.*}
cp $file ./archive/$basename\_$Date_Val.csv
done
I am new to unix and am practicing a simple script to unzip a load of files within a specified directory. I need the program to move the zipped file into another folder when it is done unzipping it (I called this oldzipped folder). For simplicity, I have removed the part of the code unzipping the file and currently have the program working for a specific file rather than the *tar.7z file extention. For some reason, the mv statement is not working. Unix is saying the following when I try to run the script. Could someone give me a hand with this? Again, I know this is the long way of doing things, but I want practice writing a script. Please be nice, as I am very new to Unix :(
unzip5: line 14: [ASDE0002.tar.7z]: command not found
#!~/bin/bash
# My program to try to unzip several files with ending of tar.7z
# I have inserted the ability to enter the directory where you want this to be done
echo "What file location is required for unzipping?"
read dirloc
cd $dirloc
mkdir oldzippedfiles
for directory in $dirloc
do
if
[ASDE0002.tar.7z]
then
mv -f ASDE0002.tar.7z $dirloc/oldzippedfiles
fi
done
echo "unzipping of file is complete"
exit 0
[ is the name of a (sometimes built-in) command which accepts arguments. As such you need to put a space after it as you would when invoking any other program. Also, you need a test. For example, to determine if the file exists and is a file, you need to use -f:
if [ -f ASDE0002.tar.7z ]
then
mv -f ASDE0002.tar.7z $dirloc/oldzippedfiles
fi
Here are some other possible tests.
Need some help with this as my shell scripting skills are somewhat less than l337 :(
I need to gzip several files and then copy newer ones over the top from another location. I need to be able to call this script in the following manner from other scripts.
exec script.sh $oldfile $newfile
Can anyone point me in the right direction?
EDIT: To add more detail:
This script will be used for monthly updates of some documents uploaded to a folder, the old documents need to be archived into one compressed file and the new documents, which may have different names, copied over the top of the old. The script needs to be called on a document by document case from another script. The basic flow for this script should be -
The script file should create a new gzip
archive with a specified name (created from a prefix constant in the script and the current month and year e.g. prefix.september.2009.tar.gz) only if it
does not already exist, otherwise add to the existing one.
Copy the old file into the archive.
Replace the old file with the new one.
Thanks in advance,
Richard
EDIT: Added mode detail on the archive filename
Here's the modified script based on your clarifications. I've used tar archives, compressed with gzip, to store the multiple files in a single archive (you can't store multiple files using gzip alone). This code is only superficially tested - it probably has one or two bugs, and you should add further code to check for command success etc. if you're using it in anger. But it should get you most of the way there.
#!/bin/bash
oldfile=$1
newfile=$2
month=`date +%B`
year=`date +%Y`
prefix="frozenskys"
archivefile=$prefix.$month.$year.tar
# Check for existence of a compressed archive matching the naming convention
if [ -e $archivefile.gz ]
then
echo "Archive file $archivefile already exists..."
echo "Adding file '$oldfile' to existing tar archive..."
# Uncompress the archive, because you can't add a file to a
# compressed archive
gunzip $archivefile.gz
# Add the file to the archive
tar --append --file=$archivefile $oldfile
# Recompress the archive
gzip $archivefile
# No existing archive - create a new one and add the file
else
echo "Creating new archive file '$archivefile'..."
tar --create --file=$archivefile $oldfile
gzip $archivefile
fi
# Update the files outside the archive
mv $newfile $oldfile
Save it as script.sh, then make it executable:
chmod +x script.sh
Then run like so:
./script.sh oldfile newfile
something like frozenskys.September.2009.tar.gz, will be created, and newfile will replace oldfile. You can also call this script with exec from another script if you want. Just put this line in your second script:
exec ./script.sh $1 $2
A good refference for any bash scripting is Advanced Bash-Scripting Guide.
This guide explains every thing bash scripting.
The basic approach I would take is:
Move the files you want to zip to a directory your create.
(commands mv and mkdir)
zip the directory. (command gzip, I assume)
Copy the new files to the desired location (command cp)
In my experience bash scripting is mainly knowing how to use these command well and if you can run it on the command line you can run it in your script.
Another command that might be useful is
pwd - this returns the current directory
Why don't you use version control? It's much easier; just check out, and compress.
(apologize if it's not an option)