I have this piece of code to check if a file exists in a directory, but it exits saying the file doesn't exist -
# in another file in the same directory
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
source "${SCRIPT_DIR}/common.sh" # calling the code below
BACKUP_VARS_FILE=${BACKUP_VARS_FILE:-"${SCRIPT_DIR}"/bitbucket.diy-backup.vars.sh}
if [ -f "${BACKUP_VARS_FILE}" ]; then
source "${BACKUP_VARS_FILE}"
echo "Using vars file: '${BACKUP_VARS_FILE}'"
else
error "'${BACKUP_VARS_FILE}' not found"
bail "You should create it using '${SCRIPT_DIR}/bitbucket.diy-backup.vars.sh.example' as a template"
fi
But the file does exist in the same folder as common.sh where the code is.
Output - it doesn't even go in the if/else block
[![enter image description here][2]][2]
Any help please?
edit -
Variable output, reading the file (with less) works -
[![enter image description here][3]][3]
Since the output is "/bitbucket...", without anything before the slash, I think your ${SCRIPT_DIR} is empty, which makes it looking for the file laying under root directory /.
Your current directory is /home/ubuntu/backup, where can find bitbucket.diy-backup.vars.sh, but your script may looking for /bitbucket.diy-backup.vars.sh, not /home/ubuntu/backup/bitbucket.diy-backup.vars.sh. Check ${SCRIPT_DIR} variable first.
Related
So the issue is the following:
I have a folder with 4 .zip's and .md5's for all of them with the same base filename.
I have a FOR loop:
for S in *.zip;
do
{
check_md5
}
done
check_md5 ()
{
if md5sum -c ${S:0:-4}.md5;
then
unzip_file
else
echo "MD5 failed"
fi
}
The problem: the script for the first zip runs fine, but when the unzip_file process ends it doesn't update the S variable to the next zip's filename.
During the unzip_file process it moves out of the original folder and at the end of it moves both the current 'S' .zip and .md5 to a archive folder.
Is it possible that the bug happens, because the unzip_file script moves out of the original folder or because it moves the first .zip and .md5 out of the original folder before it jumps to the next cycle?
Any other ideas?
Thanks in advance.
The problem is at the end of the first iteration you end up in a different directory. Following is one possible fix.
CWD="$PWD" # remember current working directory path
for S in *.zip; do
check_md5
cd "$CWD" # come back to the initial working directory at the end of each iteration
done
Though not related to the question, I would prefer passing the file name to the function instead of accessing a variable set somewhere else. I find it hard to troubleshoot and more error prone.
unzip_file()
{
echo "unzipping $1"
# ...
}
check_md5()
{
if md5sum -c "${1:0:-4}.md5"; then
unzip_file "$1" # pass the argument forward
else
echo "MD5 failed"
fi
}
CWD="$PWD" # remember current working directory path
for S in *.zip; do
check_md5 "$S" # pass file name as an argument
cd "$CWD" # come back to the initial working directory
done
As another note, your function naming/modularization doe not look good. e.g. check_md5 does a lot more than just checking md5 sum.
i'm currently stuck with a bash script that should be able to manage permission on files and directories at the end of the processing part.
Actually i have 4 components:
the main script which source .conf file and libs (.sh), process things, and call a function "ApplyPermissionFromCSVFile" with a .csv file as argument at the end, to ensure that rights are correctly set. This function should handle the job for managing permission on files
a script called "permission_lib.sh" which contains several functions, including the "ApplyPermissionFromCSVFile" one. This script is SOURCED at the begining of main script
a .conf file which contains some path defined as variables which is SOURCED at the beginning of main script
a .csv file containing paths (including "dynamic path", which refer to variables defined in conf file) for files and directories which is READ by the ApplyPermissionFromCSVFile function
At the moment, the main script runs correctly, is able to source both conf file and lib file but when i put some debug point inside the "ApplyPermissionFromCSVFile", it appears that "dynamic path" is not interpreted by bash.
Extract of Main Script:
#########################################
includes
##################################################
# this section can _almost_ be copied as-is ;-)
nameOfThisScript=$(basename "${BASH_SOURCE[0]}")
directoryOfThisScript="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
configFile="$directoryOfThisScript/$nameOfThisScript.conf"
functionsFile="$directoryOfThisScript/safeScriptsFunctions.sh"
permissionLib="$directoryOfThisScript/permission_lib.sh"
permissionFile="$directoryOfThisScript/$nameOfThisScript.permissionFile.csv"
for fileToSource in "$configFile" "$functionsFile" "$permissionLib"; do
source "$fileToSource" 2>/dev/null || {
echo "File '$fileToSource' not found"
exit 1
}
done
#Main Loop to read CSV File is called in permissionLib.sh
ApplyPermissionFromCSVFile $permissionFile
Extract of conf file (real filename replaced for exemple):
totovariable="/usr/local"
tatavariable="$totovariable/bin"
Extract of csv file:
$totovariable;someuser;somegroup;0600
$tatavariable;someuser;somegroup;0600
Extract of permission lib file:
function ApplyPermissionFromCSVFile {
local csvFileName="$1"
local fieldNumberFileName=1
local fieldNumberOwner=2
local fieldNumberGroupOwner=3
local fieldNumberPermissions=4
while read csvLine
do
fileName=$(getFieldFromCsvLine "$fieldNumberFileName" "$csvLine")
fileOwner=$(getFieldFromCsvLine "$fieldNumberOwner" "$csvLine")
fileGroupOwner=$(getFieldFromCsvLine "$fieldNumberGroupOwner" "$csvLine")
filePermissions=$(getFieldFromCsvLine "$fieldNumberPermissions" "$csvLine")
permissionArray[0,0]="$fileName|$fileOwner|$fileGroupOwner|$filePermissions"
echo "${permissionArray[0,0]}"
done < "$csvFileName"
}
getFieldFromCsvLine() {
csvFieldSeparator=';'
fieldNumber="$1"
csvLine="$2"
echo "$csvLine" | cut -d "$csvFieldSeparator" -f "$fieldNumber"
}
Don't bother for the fact that the loop overwrite value at each iterations, it's not the purpose here (but optionnal answer :p).
Which output results:
$totovariable|someuser|somegroup|0600
$tatavariable|someuser|somegroup|0600
Changing owner to someuser:somegroup for file $tatavariable
chown: cannot access '$tatavariable': No such file or directory
Changing permissions to 0600 for file $tatavariable
chmod: cannot access '$tatavariable': No such file or directory
After some investigations an research, it seems normal as:
conf file is SOURCED (by main script)
lib file is SOURCED (by main script)
csv file is not SOURCED but READ (by function in lib). So bash consider the variables contents as "pure-string", and not variables
The issue, is that i can't clearly see how and where i should replace the "pure-string" variable by its value (defined in .conf file and sourced by main script): at the main script level ? at the lib level with global variables ?
Actual solutions that i've found:
sed substitute
use eval
Any help could be appreciated.
Solution used:
fileName=eval echo "$fileName"
As the pathes in .csv file may contain "$" Symbol, Bash parameter expansion will not work in every case.
Exemple:
with the following csv content:
$tatavariable;someuser;somegroup;0600
$totovariable/thisotherfile.txt;someuser;somegroup;0660
$totovariable;someuser;somegroup;0600
/home/someuser/lolzy.txt;someuser;somegroup;0666
And the following conf file:
totovariable="/home/someuser/fack"
tatavariable="$totovariable/thisfile.txt"
The following bash code (based on eval which is not highly recommended in most case) will work on every case (containing $ symbol or not):
#!/bin/bash
#########################
# Function
#########################
getFieldFromCsvLine() {
local csvFieldSeparator=';'
local fieldNumber="$1"
local csvLine="$2"
echo "$csvLine" | cut -d "$csvFieldSeparator" -f "$fieldNumber"
}
#########################
#Core Script
#########################
source configFile.conf
csvFileName="permissionFile.csv"
fieldNumberFileName=1
fieldNumberOwner=2
fieldNumberGroupOwner=3
fieldNumberPermissions=4
while read csvLine
do
fileName=$(getFieldFromCsvLine "$fieldNumberFileName" "$csvLine")
fileOwner=$(getFieldFromCsvLine "$fieldNumberOwner" "$csvLine")
fileGroupOwner=$(getFieldFromCsvLine "$fieldNumberGroupOwner" "$csvLine")
filePermissions=$(getFieldFromCsvLine "$fieldNumberPermissions" "$csvLine")
#Managing Variables used as 'Dynamic path'
fileName=$(eval echo "$fileName")
echo "Content of \$fileName is $fileName"
done < "$csvFileName"
Results:
[someuser#SAFEsandbox:~]$ ./simpletest.sh
Content of $fileName is /home/someuser/fack/thisfile.txt
Content of $fileName is /home/someuser/fack/thisotherfile.txt
Content of $fileName is /home/someuser/fack
Content of $fileName is /home/someuser/lolzy.txt
Whereas the following bash code (base on bash parameters expansion) will throw errors:
#!/bin/bash
#########################
# Function
#########################
getFieldFromCsvLine() {
local csvFieldSeparator=';'
local fieldNumber="$1"
local csvLine="$2"
echo "$csvLine" | cut -d "$csvFieldSeparator" -f "$fieldNumber"
}
#########################
#Core Script
#########################
source configFile.conf
csvFileName="permissionFile.csv"
fieldNumberFileName=1
fieldNumberOwner=2
fieldNumberGroupOwner=3
fieldNumberPermissions=4
while read csvLine
do
fileName=$(getFieldFromCsvLine "$fieldNumberFileName" "$csvLine")
fileOwner=$(getFieldFromCsvLine "$fieldNumberOwner" "$csvLine")
fileGroupOwner=$(getFieldFromCsvLine "$fieldNumberGroupOwner" "$csvLine")
filePermissions=$(getFieldFromCsvLine "$fieldNumberPermissions" "$csvLine")
#Managing Variables used as 'Dynamic path'
fileName=${!fileName}
echo "Content of \$fileName is $fileName"
done < "$csvFileName"
Behaviour example when .csv contains $ symbol:
[someuser#SAFEsandbox:~]$ ./simpletest.sh
./simpletest.sh: line 35: $tatavariable: bad substitution
Behaviour example when you remove the $ symbol in .csv file, but there is still incremental path notions in it:
[someuser#SAFEsandbox:~]$ ./simpletest.sh
Content of $fileName is /home/someuser/fack/thisfile.txt
./simpletest.sh: line 35: totovariable/thisotherfile.txt: bad substitution
Using bash parameter expansion (which is recommended in most case) is not easy here, because it forces to manage 2 cases in the script:
path containing a leading $ or serveral $ (contatenated variables paths)
path not containing a leading $ or serveral $ (contatenated variables paths)
Regards
I want to enter several directories in a for loop. I do not have the complete name of the directories, only a part of it.
I would like to do something like what you would write on the terminal, something like cd *IncompleteDirName*
This is a MVE of the loop: IncompleteDirName's are obtained from the file IncompleteDirNames.
cont=1
sum=1
while read anotherFILE; do
IncompleteDirName=$(sed "${cont}q;d" IncompleteDirNames)
cd *"${IncompleteDirName}"*
# Do stuff
cd ..
cont=$((cont + sum))
done <anotherFILE
This is not working, I don't know if this has to do with wildcard not expanding or with variable not working properly.
It is throwing me this error message:
*: No such file or directory
I suppose this means asterisk is not working as intended. It is not entering the directories, and there is a directory that matches every pattern. Anyway, no directory is being entered.
This is how IncompleteDirNames file looks like:
Alicante
Almeria
Andalucia
Avila
It is a column of names.
These are the directory names corresponding to the IncompleteDirNames above:
aa4fas_Alicante_com
mun_Almeria
comunidadde_Andalucia
ciuAvila
Try this code -
cont=1
sum=1
while read FILE; do
IncompleteDirName=$(sed "${cont}q;d" FILE)
cd *"${IncompleteDirName}"*
# Do stuff
cd ..
cont=$((cont + sum))
done <IncompleteDirNames
You can do the following:
#!/bin/bash
while read -r line; do
cd /absolute/path/to/*$line*
# do stuff
done < filename
This will enter each directory whose name partially matches a line in filename, and "does stuff".
First post, so Hi! Let me start by saying I'm a total noob regarding programming. I understand very basic stuff, but when it comes to checking exit codes or what the adequate term is, I'm at a loss. Apparently my searchfoo is really weak in this area, I guess it's a question of terminology.
Thanks in advance for taking your time to reading this/answering my question!
Description: I found a script that converts/repack .cbr files to .cbz files. These files are basically your average rar and zip files, however renamed to another extension as they are used for (comic)book applications such as comicrack, qcomicbook and what not. Surprisingly enough there no cbr -> cbz converters out there. The advantages of .cbz is besides escaping the proprietary rar file format, that one can store the metadata from Comic Vine with e. g comictagger.
Issue: Sometimes the repackaging of the files doesn't end well and would hopefully be alleviated by a integrity check & another go. I modified said script slightly to use p7zip as it can both pack/unpack 7z, zip-files and some others, i. e great for options. p7zip can test the archive by:
7z t comicfile.cbz tmpworkingdir
I guess it's a matter of using if & else here(?) to check the integrity and then give it another go, if there are any error.
Question/tl;dr: What would be the "best"/adequate approach to add a integrity file check to the script below?
#!/bin/bash
#Source: http://comicrack.cyolito.com/forum/13-scripts/30013-cbr3cbz-rar-to-zip-conversion-for-linux
echo "Converting CBRs to CBZs"
# Set the "field separator" to something other than spaces/newlines" so that spaces
# in the file names don't mess things up. I'm using the pipe symbol ("|") as it is very
# unlikely to appear in a file name.
IFS="|"
# Set working directory where to create the temp dir. The user you are using must have permission
# to write into this directory.
# For performance reasons I'm using ram disk (/dev/shm/) in Ubuntu server.
WORKDIR="/dev/shm/"
# Set name for the temp dir. This directory will be created under WORDDIR
TEMPDIR="cbr2cbz"
# The script should be invoked as "cbr2cbz {directory}", where "{directory}" is the
# top-level directory to be searched. Just to be paranoid, if no directory is specified,
# then default to the current working directory ("."). Let's put the name of the
# directory into a shell variable called SOURCEDIR.
# Note: "$1" = "The first command line argument"
if test -z "$1"; then
SOURCEDIR=`pwd`
else
SOURCEDIR="$1"
fi
echo "Working from directory $SOURCEDIR"
# We need an empty directory to work in, so we'll create a temp directory here
cd "$WORKDIR"
mkdir "$TEMPDIR"
# and step into it
cd "$TEMPDIR"
# Now, execute a loop, based on a "find" command in the specified directory. The
# "-printf "$p|" will cause the file names to be separated by the pipe symbol, rather than
# the default newline. Note the backtics ("`") (the key above the tab key on US
# keyboards).
for CBRFILE in `find "$SOURCEDIR" -name "*.cbr" -printf "%p|while read line; do
# Now for the actual work. First, extract the base file name (without the extension)
# using the "basename" command. Warning: more backtics.
BASENAME=`basename $CBRFILE ".cbr"`
# And the directory path for that file, so we know where to put the finished ".cbz"
# file.
DIRNAME=`dirname $CBRFILE`
# Now, build the "new" file name,
NEWNAME="$BASENAME.cbz"
# We use RAR file's name to create folder for unpacked files
echo "Processing $CBRFILE"
mkdir "$BASENAME"
# and unpack the rar file into it
7z x "$CBRFILE" -O"$BASENAME"
cd "$BASENAME"
# Lets ensure the permissions allow us to pack everything
sudo chmod 777 -R ./*
# Put all the extracted files into new ".cbz" file
7z a -tzip -mx=9 "$NEWNAME" *
# And move it to the directory where we found the original ".cbr" file
mv "$NEWNAME" $DIRNAME/"$NEWNAME"
# Finally, "cd" back to the original working directory, and delete the temp directory
# created earlier.
cd ..
rm -r "$BASENAME"
# Delete the RAR file also
rm "$CBRFILE"
done
# At the end we cleanup by removing the temp folder from ram disk
cd ..
echo "Conversion Done"
rm -r "$TEMPDIR"
Oh the humanity, not posting more than two links before 10 reputation and I linked the crap out of OP.. [edit]ah.. mh-mmm.. there we go..
[edit 2] I removed unrar as an dependency and use p7zip instead, as it can extract rar-files.
You will need two checks:
7z t will test the integrity of the archive
You should also test the integrity of all the image files in the archive. You can use at tools like ImageMagick for this.
A simple test would be identify file but that might read only the header. I'd use convert file -resize 5x5 png:- > /dev/null
This scales the image down to 5x5 pixels, converts it to PNG and then pipes the result to /dev/null (discarding it). For the scaling, the whole image has to be read. If this command fails with an error, something is wrong with the image file.
I'm trying to make a shell function that looks for a file existence in the current dir. If the file doesn't exists if the current dir, it should try the parent dir, and go up to the root of the filesystem (going up to ~ should be more than enough).
What I'm trying to achieve is display the current ruby on rails version used for a project on my prompt. The easy way is to call bundle show rails and parse the string, but it is quite slow; so I want to parse the file that contains the versions informations, Gemfile.lock. What I have currently only works when I'm in the root of the project (where the Gemfile.lock lives).
This could also be used to parse a git config file, though git config is pretty fast.
Thanks for your time
like this?
#!/bin/zsh
# the function
findfile_parent() {
FILE=$1
while [ ! -e "${FILE}" ]; do
if [ "$(pwd)" = "/" ]; then
return
fi
cd ..
done
echo "$(pwd)/${FILE}"
}
# usage example
SEARCHFILE=foo.pyl
UPFILE=$(findfile_parent ${SEARCHFILE})
if [ "x${UPFILE}" != "x" ]; then
echo "found file ${SEARCHFILE} as ${UPFILE}"
else
echo "couldn't find file ${SEARCHFILE}"
fi
Refer the below two helpful links to find out your solution.
Get parent directory of current directory in Ruby
How to check for file existence
):