This script1 is not working as intended. I will explain below:
#!/bin/bash
### SETUP ###
USER="MYUSER"
DIRS="MYDIR"
BUCKET="mybucket"
DOACCESS="ACCESSKEY"
DOSECRET="SECRETKEY"
NAME="FILENAME"
EXPIRE="7 days"
NOW=$(date +"%d-%m-%Y")
DAY=$(date +"%a")
# ...lots of code that is working great...
### CLEAN OLD FILE FROM BUCKET ###
### This is the line that I am having issues with.
sh ./s3-del-old.sh ''"${BUCKET}"'/backup' '"${EXPIRE}"'
END
The script2 got copied from here.
What I had prior to following some instructions on Bash: Variable in single quote linked below was:
sh ./s3-del-old.sh "$BUCKET/backup" "$EXPIRE"
This did not work and was ignored when running the bash script.
I attempted to leave out the stuff that doesn't matter to the question below, although I believe I may have confused things. For this I apologize. Very simply, I have a line in script1 that calls another script2. I use variables to meet the needs of the script. To which it is not working and I cannot find a easy to understand solution online, thus the need to post the question.
----END OF UPDATE----
I have looked at some of the answered questions, but I am not finding a solution that fits my needs or one that I can understand fully to use for my needs.
I have tried following this, although I need a little more help.
This is what I am trying to do:
I have a backup script that uses DreamHost's DreamObjects to store my backups. The annoying part with DreamObjects is that it doesn't have any built in features for removing files created x days ago. Hence my problem. I would like to add a call to a bash file from my bash file. If that makes sense. :) If not, the code in question is below, you should be able to understand then.
I would really like to be able to add the code to my current script instead of using a separate file. I just don't know how to rewrite it properly without spending more time than I have on it. I found the code at.
My variables that matter for this problem:
BUCKET="mybucket"<br>
EXPIRE="7 days"
This is the line that calls the file:
sh ./s3-del-old.sh ''"${BUCKET}"'/backup' '"${EXPIRE}"'
This provides me with an error of date:
invalid date `-"${EXPIRE}"'
The file uses the following syntax to work:
s3-del-old "bucket" "30 days"
It does work perfectly when I use it in the command line on it's own, I just would like to add the call to one file so that I can use one cronjob instead of two. Plus, this way I can use the script with any of my domains/buckets by changing the variables. :)
The "other script" that you need to call is a bash script.
A bash script (usually) will not work if called as sh, as you are doing:
sh ./s3-del-old.sh ''"${BUCKET}"'/backup' '"${EXPIRE}"'
Please call the script with bash:
bash ./s3-del-old.sh "${BUCKET}"/backup "${EXPIRE}"
Or even better, let the script choose the shell that should run it:
./s3-del-old.sh "${BUCKET}"/backup "${EXPIRE}"
With the shebang of the file s3-del-old.sh:
#!/bin/bash
Sometimes I amaze myself at how difficult I try to make things...s3cmd has an expire function for files by create date...that will be a lot easier...
Really a big thank you to all that helped!
My bad this was suppose to be a comment not an answer. :)
Related
Is there a less brute-force way to do this?
#!/bin/ksh
THIS_SCRIPT=$(/usr/bin/readlink -f $(echo $0 | /bin/sed "s,^[^/],$PWD/&,"))
echo $THIS_SCRIPT
I'm stuck using ksh but would prefer a solution that works in bash too (which I think this does).
Entry #28 in the bash FAQ:
How do I determine the location of my script? I want to read some config files from the same place.
There are two prime reasons why this issue comes up: either you want to externalize data or configuration of your script and need a way to find these external resources, or your script is intended to act upon a bundle of some sort (eg. a build script), and needs to find the resources to act upon.
It is important to realize that in the general case, this problem has no solution. Any approach you might have heard of, and any approach that will be detailed below, has flaws and will only work in specific cases. First and foremost, try to avoid the problem entirely by not depending on the location of your script!
...
Using BASH_SOURCE
The BASH_SOURCE internal bash variable is actually an array of pathnames. If you expand it as a simple string, e.g. "$BASH_SOURCE", you'll get the first element, which is the pathname of the currently executing function or script.
I've always done:
SCRIPT_PATH=$(cd `dirname ${0}`; pwd)
I've never used readlink before: is it Gnu only? (i.e. will it work on HP-UX, AIX, and Solaris out of the box? dirname and pwd will....)
(edited to add `` which I forgot in original post. d'oh!)
(edit 2 to put on two lines which I've apparently always done when I look at previous scripts I'd written, but hadn't remembered properly. First call gets path, second call eliminates relative path)
(edit 3 fixed typo that prevented single line answer from working, back to single line!)
Why didn't I think to try this before I asked the question?
THIS_SCRIPT=$(/usr/bin/readlink -nf "$0")
Works great.
In macOS I use (edit: only works if you run the script from where the script actually is!)
my_script=$(pwd)/$(basename $0)
I wrote a script with six if statements that looks like this:
#!/usr/bin/bash
if [ -n $var1 ]
then
for f in /path/*.fastq.gz
do
x=${f/%.fastq.gz/_sample1-forward.fastq.gz}
y=${f/%.fastq.gz/_sample1-forward.out}
q=${f/%.fastq.gz/_temp.fastq.gz}
command [options] -i $f -o $temp${x##*/}
cp $temp${x##*/} $temp${q##*/}
done
else
echo "no $var1"
for f in /path/*.fastq.gz
do
q=${f/%.fastq.gz/_temp.fastq.gz}
cp $f $temp${q##*/}
done
fi
The other five statements do a similar task for var2 to var6. When I run the script I get unexpected output (no errors no warnings), but when I copy paste each of the if statements to terminal I end up with the exact result I would expect. I've looked for hidden characters or syntax issues for hours now. Could this be a shell issue? Script written on OSX (default zsh) and execution on server (default bash). I have a feeling this issue is similar but I couldn't find an answer to my issue in the replies.
Any and all ideas are most welcome!
Niwatori
You should maybe look at the shebang. I think proper usage would be #!/usr/bin/env bash or #!/bin/bash.
thanks for the help, much appreciated. Shebang didn't seem to be the problem although thanks for pointing that out. Shellcheck.net reminded me to use
[[ ]]
for unquoted variables, but that didn't solve the problem either. Here's what went wrong and how I 'fixed' it:
for every variable the same command (tool) is used which relies on a support file (similar in format but different content). Originally, before every if statement I replaced the support file for the previous variable with the one needed for the current variable. For some reason (curious why, any thoughts are welcome) this didn't always happen correctly.
As a quick workaround I made six versions of the tool, all with a different support file and used PYTHONPATH=/path/to/version/:$PYTHONPATH before every if statement. Best practice would be to adapt the tool so it can use different support files or an option that deals with repetitive tasks but I don't have the time at the moment.
Have a nice day,
Niwatori
This question has been posted here many times, but it never seems to answer my question.
I have two scripts. The first one contains one or multiple variables, the second script needs those variables. The second script also needs to be able to change the variables in the first script.
I'm not interested in sourcing (where the first script containing the variables runs the second script) or exporting (using environment variables). I just simply want to make sure that the second script can read and change (get and set) the variables available in the first script.
(PS. If I misunderstood how sourcing or exporting works, and it applies to my scenario, please let me know. I'm not completely closed to those methods, after what I've read, I just don't think those things will do what I want)
Environment variables are per process. One process can not modify the variables in another. What you're asking for is not possible.
The usual workaround for scripts is sourcing, which works by running both scripts in the same shell process, but you say you don't want to do that.
I've also given this some thought. I would use files as variables. For example in script 1 you use for writing variable values to files:
echo $varnum1 > /home/username/scriptdir/vars/varnum1
echo $varnum2 > /home/username/scriptdir/vars/varnum2
And in script 2 you use for reading values from files back into variables:
$varnum1=$(cat /home/username/scriptdir/vars/varnum1)
$varnum2=$(cat /home/username/scriptdir/vars/varnum2)
Both scripts can read or write to the variables at any given time. Theoretically two scripts can try to access the same file at the same time, I'm not sure what exactly would happen but since each file only contains one value, the time to read or write should be extremely short.
In order to even reduce those times you can use a ramdisk.
I think this is much better than scripts editing each other (yuk!). Live editing of scripts can mess up scripts and only works when you initiate the script again after the edit was made.
Good luck!
So after a long search on the web and a lot of trying, I finally found some kind of a solution. Actually, it's quite simple.
There are some prerequisites though.
The variable you want to set already has to exist in the file you're trying to set it in (I'm guessing the variable can be created as well when it doesn't exist yet, but that's not what I'm going for here).
The file you're trying to set the variable in has to exist (obviously. I'm guessing again this can be done as well, but again, not what I'm going for).
Write
sudo sed -i 's/^\(VARNAME=\).*/\1VALUE/' FILENAME
So i.e. setting the variable called Var1 to the value 5, in the file
test.ini:
sudo sed -i 's/^\(Var1=\).*/\15/' test.ini
Read
sudo grep -Po '(?<=VARNAME=).*' FILENAME
So i.e. reading the variable called Var1 from the file test.ini
sudo grep -Po '(?<=Var1=).*' test.ini
Just to be sure
I've noticed some issues when running the script that sets variables from a different folder than the one where your script is located.
To make sure this always go right, you can do one of two things:
sudo sed -i 's/^\(VARNAME=\).*/\1VALUE/' `dirname $0`/FILENAME
So basically, just put `dirname $0`/ (including the backticks) in front of the filename.
The other option is to make `dirname $0`/ a variable (again including the backticks), which would look like this.
my_dir=`dirname $0`
sudo sed -i 's/^\(VARNAME=\).*/\1VALUE/' $my_dir/FILENAME
So basically, if you've got a file named test.ini, which contains this line: Var1= (In my tests, the variable can start empty, and you will still be able to set it. Mileage may vary.), you will be able to set and get the value for Var1
I can confirm that this works (for me), but since you all, with way more experience in scripting then me, didn't come up with this, I'm guessing this is not a great way to do it.
Also, I couldn't tell you the first thing about what's happening in those commands above, I only know they work.
So if I'm doing something stupid, or if you can explain to me what's happening in the commands above, please let me know. I'm very curious to find out what you guys think if this solution.
I have an ultra-simple script that houses the Linux program shred, and it contains the parameters that have always worked from the command line (bash). Specifically 'shred -uzn 35'
The script, named D, has execute permissions set.
When I run the script, bash prints an error:
$ D some_file_to_delete
shred: missing file operand
I realize that the solution to the problem is probably as simple as the program itself. Please help?
Thanks in advance.
EDIT: The error "missing file operand" was due to the fact that the script was not set to take arguments, such as via "$#". Also, as stated in the accepted answer, I agree that an alias makes total sense for such a scenario (much more sense than, say, a script somewhere in $PATH).
Since you are using a script, not an alias, you need to pass the arguments through
shred -uzn 35 "$#"
In this case, however, I suggest you do make it an alias. In your .bashrc file, add this:
alias D='shred -uzn 35'
Is there a less brute-force way to do this?
#!/bin/ksh
THIS_SCRIPT=$(/usr/bin/readlink -f $(echo $0 | /bin/sed "s,^[^/],$PWD/&,"))
echo $THIS_SCRIPT
I'm stuck using ksh but would prefer a solution that works in bash too (which I think this does).
Entry #28 in the bash FAQ:
How do I determine the location of my script? I want to read some config files from the same place.
There are two prime reasons why this issue comes up: either you want to externalize data or configuration of your script and need a way to find these external resources, or your script is intended to act upon a bundle of some sort (eg. a build script), and needs to find the resources to act upon.
It is important to realize that in the general case, this problem has no solution. Any approach you might have heard of, and any approach that will be detailed below, has flaws and will only work in specific cases. First and foremost, try to avoid the problem entirely by not depending on the location of your script!
...
Using BASH_SOURCE
The BASH_SOURCE internal bash variable is actually an array of pathnames. If you expand it as a simple string, e.g. "$BASH_SOURCE", you'll get the first element, which is the pathname of the currently executing function or script.
I've always done:
SCRIPT_PATH=$(cd `dirname ${0}`; pwd)
I've never used readlink before: is it Gnu only? (i.e. will it work on HP-UX, AIX, and Solaris out of the box? dirname and pwd will....)
(edited to add `` which I forgot in original post. d'oh!)
(edit 2 to put on two lines which I've apparently always done when I look at previous scripts I'd written, but hadn't remembered properly. First call gets path, second call eliminates relative path)
(edit 3 fixed typo that prevented single line answer from working, back to single line!)
Why didn't I think to try this before I asked the question?
THIS_SCRIPT=$(/usr/bin/readlink -nf "$0")
Works great.
In macOS I use (edit: only works if you run the script from where the script actually is!)
my_script=$(pwd)/$(basename $0)