I'm trying to add a service under the name of $1.service, however, the service file being created is not getting the $1, instead, only .service
also there is a part where $1 needs to be pasted inside the $1.service file but it's not passing that information through as well.
this is basically how my bash script look like;
#! /bin/bash
function addService {
cat << EOF > /etc/systemd/system/$1.service
(all that service stuff here)
PIDFile=${_var}/$1.pid
EOF
}
cfg_file=~/config/"$cfg.conf"
if [ -f "$cfg_file" ]; then
. "$cfg_file"
addService $1
fi
so you run the script as ./script.sh test and it should create a service called test.service in this example, but it doesn't seem to be working properly. however, the variables like ${_conf} are passing through without any problems.
and also, do I have to use EOF specifically for this task or echo would do the job alone?
EDIT:
The config file exists and it is $1+.conf and this is the content of test.conf file;
_var=var1
and the .service file that is created passing this information without any problems. which means if $1 wasn't working, it wouldn't fetch the config file as well. but apparently, it is working.
First, you are checking for the existence of a file in ~/conf that ends in .conf? What is the value of $cfg? Does ~/conf/${cfg}.conf exist? If not, are you even going into the if clause? Using "set -x" will help debug these things.
Second, you have EOF indented. For HERE documents, the delimiter must start in the first column. You should have gotten an error when running this script about that. Something like, here-document at line X delimited by end-of-file (wanted EOF). The delimiter string can be anything (e.g. EOSD for end of service definition). It needs to start in column 1 though.
Here is what I quickly did to make sure things work.
#! /bin/bash
set -x
function addService {
cat << EOF > ./$1.service
(all that service stuff here)
PIDFile=${_conf}/$1.pid
EOF
}
cfg_file=./conf.in
if [ -f "$cfg_file" ]; then
. "$cfg_file"
addService $1
fi
Hope this helps.
The problem has been solved by changing
cat << EOF > /etc/systemd/system/$1.service
(service content here)
EOF
to
echo "
(service content here)
" > /etc/systemd/system/$1.service
Related
I am setting up a directory structure with many different R & bash scripts in it. They all will be referencing files and folders. Instead of hardcoding the paths I would like to have a text file where each script can search for a descriptor in the file (see below) and read the relevant path from that.
Getting the search-append to work in R is easy enough for me; I am having trouble getting it to work in Bash, since I don't know the language very well.
My guess is it has something to do with the way awk works / stores the variable, or maybe the way the / works on the awk output. But I'm not familiar enough with it and would really appreciate any help
Text File "Master_File.txt":
NOT_DIRECTORY "/file/paths/Fake"
JOB_TEST_DIRECTORY "/file/paths/Real"
ALSO_NOT_DIRECTORY "/file/paths/Fake"
Bash Script:
#! /bin/bash
master_file_name="Master_File.txt"
R_SCRIPT="RScript.R"
SRCPATH=$(awk '/JOB_TEST_DIRECTORY/ { print $2 }' $master_file_name)
Rscript --vanilla $SRCPATH/$R_SCRIPT
The last line, $SRCPATH/$R_SCRIPT, seems to be replacing part of SRCPath with the name of $R_SCRIPT which outputs something like /RScript.Rs/Real instead of what I would like, which is /file/paths/Real/RScript.R.
Note: if I hard code the path path="/file/paths/Real" then the code $path/$R_SCRIPT outputs what I want.
The R Script:
system(command = "echo \"SUCCESSFUL_RUN\"", intern = FALSE, wait = TRUE)
q("no")
Please let me know if there's any other info that would be helpful, I added everything I could think of. And thank you.
Edit Upon Answer:
I found two solutions.
Solution 1 - By Mheni:
[ see his answer below ]
Solution 2 - My Adaptation of Mheni's Answer:
After seeing a Mehni's note on ignoring the " quotation marks, I looked up some more stuff, and found out it's possible to change the character that awk used to determine where to separate the text. By adding a -F\" to the awk call, it successfully separates based on the " character.
The following works
#!/bin/bash
master_file_name="Master_File.txt"
R_SCRIPT="RScript.R"
SRCPATH=$(awk -F\" -v r_script=$R_SCRIPT '/JOB_TEST_DIRECTORY/ { print $2 }' $master_file_name)
Rscript --vanilla $SRCPATH/$R_SCRIPT
Thank you so much everyone that took the time to help me out. I really appreciate it.
the problem is because of the quotes around the path, this change to the awk command ignores them when printing the path.
there was also a space in the shebang line that shouldn't be there as #david mentioned
#!/bin/bash
master_file_name="/tmp/data"
R_SCRIPT="RScript.R"
SRCPATH=$(awk '/JOB_TEST_DIRECTORY/ { if(NR==2) { gsub("\"",""); print $2 } }' "$master_file_name")
echo "$SRCPATH/$R_SCRIPT"
OUTPUT
[1] "Hello World!"
in my example the paths are in /tmp/data
NOT_DIRECTORY "/tmp/file/paths/Fake"
JOB_TEST_DIRECTORY "/tmp/file/paths/Real"
ALSO_NOT_DIRECTORY "/tmp/file/paths/Fake"
and in the path that corresponds to JOB_TEST_DIRECTORY i have a simple hello_world R script
[user#host tmp]$ cat /tmp/file/paths/Real/RScript.R
print("Hello World!")
I would use
Master_File.txt :
NOT_DIRECTORY="/file/paths/Fake"
JOB_TEST_DIRECTORY="/file/paths/Real"
ALSO_NOT_DIRECTORY="/file/paths/Fake"
Bash Script:
#!/bin/bash
R_SCRIPT="RScript.R"
if [[ -r /path/to/Master_File.txt ]]; then
. /path/to/Master_File.txt
else
echo "ERROR -- Can't read Master_File"
exit
fi
Rscript --vanilla $JOB_TEST_DIRECTORY/$R_SCRIPT
Basically, you create a configuration file Key=value, source it then use the the keys as variable for whatever you need throughout the script.
I'm working on a program to process requests in bash which are requested by users in a WebInterface. To give users flexibility they can specify several parameters per each job, at the end the request is saved in a file with a specific name, so the bash script could perform the requested task.
This file at the end is filled like this:
ENVIRONMENT="PRO"
INTEGRATION="J050_provisioning"
FILE="*"
DIRECTORY="out"
So the bash script will source this file to perform the needed tasks user requested. And it works great so far, but I see a security issue with this, if user enters malicious data, something like:
SOMEVAR="GONNAHACK $(rm -f some_important_file)"
OTHERVAR="DANGEROUZZZZZZ `while :; do sleep 10 & done`"
This will cause undesirable effects when sourcing the file :). Is there a way to prevent a source file execute any code but variable initializations? Or the only way would be just grep the source file before sourcing it to check it is not dangerous?
Just do not source it. Make it a configuration file composed of name=value lines (without the double quotes), read each name/value pair and assign value to name. In order not to overwrite critical variables like PATH, prefix the name with CONF_ for example.
Crude code:
while IFS='=' read -r conf_name conf_value; do
printf -v "CONF_$conf_name" '%s' "$conf_value" \
|| echo "Invalid configuration name '$conf_name'" >&2
done < your_configuration_file.conf
Test it works:
$ echo "${!CONF_*}"
CONF_DIRECTORY CONF_ENVIRONMENT CONF_FILE CONF_INTEGRATION CONF_OTHERVAR CONF_SOMEVAR
$ printf '%s\n' "$CONF_SOMEVAR"
GONNAHACK $(rm -f some_important_file)
I wish to create a single file with some contents known to me.
How do I do this in couple lines of bash?
this command will be used inside of a single script, so it should create file, add text, save, and quit automatically by itself without human intervention.
I know that
cat >> some.text
type some stuff
ctrl + D
will work. But is there a pure command line way of doing it?
Thanks
Use a "here document":
cat >> some.text << 'END'
some stuff here
more stuff
END
The delimiter (here END) is an arbitrary word. Quoting this delimiter after the << will ensure that no expansion is performed on the contents.
You could also do the following:
echo 'some stuff' > your/file.txt
For multiline, here's another example:
printf "some stuff\nmore stuff" >> your/file.txt
For making it multiline its also possilbe to echo in "execution mode":
echo -e "line1\nline2" > /tmp/file
so the \n will make a carriage return.
Great answer from #that-other-guy, also important to note that you can include the directory of the file in there and not to forget your bin/bash stuff at the start, and that it works for more than just text files.
See below my example for yaml files. And remember to make your bash files executable after with: chmod u+x fileName.sh
#!/usr/bin/bash
cat >> ~/dir/newDir/yamlFiles/testing.yaml << 'END'
service:
- testing
testing:
setOpts:
podCount: 2
END
I have recently just made this script:
if test -s $HOME/koolaid.txt ; then
Billz=$(grep / $HOME/koolaid.txt)
echo $Billz
else
Billz=$HOME/notkoolaid
echo $Billz
fi
if test -d $Billz ; then
echo "Ok"
else touch $Billz
fi
So basically, if the file $HOME/koolaid.txt file does NOT exist, then Billz will be set as $HOME/koolaid.txt. It then sucesfully creates the file.
However, if I do make the koolaid.txt then I get this
mkdir: cannot create directory : No such file or directory
Any help would be appreciated
Here is a difference between content of a variable and evaluated content...
if your variable contains a string $HOME/some - you need expand it to get /home/login/same
One dangerous method is eval.
bin=$(grep / ~/.rm.cfg)
eval rbin=${bin:-$HOME/deleted}
echo "==$rbin=="
Don't eval unless you're absolutely sure what you evaling...
Here are a couple things to fix:
Start your script with a "shebang," such as:
#!/bin/sh
This way the shell will know that you want to run this as a Bourne shell script.
Also, your conditional at the top of the script doesn't handle the case well in which .rm.cfg exists but doesn't contain a slash character anywhere in it. In that case the rbin variable never gets set.
Finally, try adding the line
ls ~
at the top so you can see how the shell is interpreting the tilde character; that might be the problem.
I'm writing shell scripts where quite regularly some stuff is written
to a file, after which an application is executed that reads that file. I find that through our company the network latency differs vastly, so a simple sleep 2 for example will not be robust enough.
I tried to write a (configurable) timeout loop like this:
waitLoop()
{
local timeout=$1
local test="$2"
if ! $test
then
local counter=0
while ! $test && [ $counter -lt $timeout ]
do
sleep 1
((counter++))
done
if ! $test
then
exit 1
fi
fi
}
This works for test="[ -e $somefilename ]". However, testing existence is not enough, I sometimes need to test whether a certain string was written to the file. I tried
test="grep -sq \"^sometext$\" $somefilename", but this did not work. Can someone tell me why?
Are there other, less verbose options to perform such a test?
You can set your test variable this way:
test=$(grep -sq "^sometext$" $somefilename)
The reason your grep isn't working is that quotes are really hard to pass in arguments. You'll need to use eval:
if ! eval $test
I'd say the way to check for a string in a text file is grep.
What's your exact problem with it?
Also you might adjust your NFS mount parameters, to get rid of the root problem. A sync might also help. See NFS docs.
If you're wanting to use waitLoop in an "if", you might want to change the "exit" to a "return", so the rest of the script can handle the error situation (there's not even a message to the user about what failed before the script dies otherwise).
The other issue is using "$test" to hold a command means you don't get shell expansion when actually executing, just evaluating. So if you say test="grep \"foo\" \"bar baz\"", rather than looking for the three letter string foo in the file with the seven character name bar baz, it'll look for the five char string "foo" in the nine char file "bar baz".
So you can either decide you don't need the shell magic, and set test='grep -sq ^sometext$ somefilename', or you can get the shell to handle the quoting explicitly with something like:
if /bin/sh -c "$test"
then
...
Try using the file modification time to detect when it is written without opening it. Something like
old_mtime=`stat --format="%Z" file`
# Write to file.
new_mtime=$old_mtime
while [[ "$old_mtime" -eq "$new_mtime" ]]; do
sleep 2;
new_mtime=`stat --format="%Z" file`
done
This won't work, however, if multiple processes try to access the file at the same time.
I just had the exact same problem. I used a similar approach to the timeout wait that you include in your OP; however, I also included a file-size check. I reset my timeout timer if the file had increased in size since last it was checked. The files I'm writing can be a few gig, so they take a while to write across NFS.
This may be overkill for your particular case, but I also had my writing process calculate a hash of the file after it was done writing. I used md5, but something like crc32 would work, too. This hash was broadcast from the writer to the (multiple) readers, and the reader waits until a) the file size stops increasing and b) the (freshly computed) hash of the file matches the hash sent by the writer.
We have a similar issue, but for different reasons. We are reading s file, which is sent to an SFTP server. The machine running the script is not the SFTP server.
What I have done is set it up in cron (although a loop with a sleep would work too) to do a cksum of the file. When the old cksum matches the current cksum (the file has not changed for the determined amount of time) we know that the writes are complete, and transfer the file.
Just to be extra safe, we never overwrite a local file before making a backup, and only transfer at all when the remote file has two cksums in a row that match, and that cksum does not match the local file.
If you need code examples, I am sure I can dig them up.
The shell was splitting your predicate into words. Grab it all with $# as in the code below:
#! /bin/bash
waitFor()
{
local tries=$1
shift
local predicate="$#"
while [ $tries -ge 1 ]; do
(( tries-- ))
if $predicate >/dev/null 2>&1; then
return
else
[ $tries -gt 0 ] && sleep 1
fi
done
exit 1
}
pred='[ -e /etc/passwd ]'
waitFor 5 $pred
echo "$pred satisfied"
rm -f /tmp/baz
(sleep 2; echo blahblah >>/tmp/baz) &
(sleep 4; echo hasfoo >>/tmp/baz) &
pred='grep ^hasfoo /tmp/baz'
waitFor 5 $pred
echo "$pred satisfied"
Output:
$ ./waitngo
[ -e /etc/passwd ] satisfied
grep ^hasfoo /tmp/baz satisfied
Too bad the typescript isn't as interesting as watching it in real time.
Ok...this is a bit whacky...
If you have control over the file: you might be able to create a 'named pipe' here.
So (depending on how the writing program works) you can monitor the file in an synchronized fashion.
At its simplest:
Create the named pipe:
mkfifo file.txt
Set up the sync'd receiver:
while :
do
process.sh < file.txt
end
Create a test sender:
echo "Hello There" > file.txt
The 'process.sh' is where your logic goes : this will block until the sender has written its output. In theory the writer program won't need modifiying....
WARNING: if the receiver is not running for some reason, you may end up blocking the sender!
Not sure it fits your requirement here, but might be worth looking into.
Or to avoid synchronized, try 'lsof' ?
http://en.wikipedia.org/wiki/Lsof
Assuming that you only want to read from the file when nothing else is writing to it (ie, the writing process has finished) - you could check whether nothing else has file handle to it ?