I am using a chef recipe to update a configuration file on my node.The contents of the file look something like follows:
server server1.domain.com
server server2.domain.com
I have a ruby array defined in my attribute file as follows:
default['servers'] = %w(xyz.domain.com abc.domain.com)
I want to use sed recursively to replace the server values in the file, such that my file is updated as such:
server xyz.domain.com
server abc.domain.com
I tried the following ruby loop in my recipe:
(node['servers']).each_with_index do |ntserver,index|
bash "server set" do
code <<-EOH
sed -i 's|server .*|server #{node['servers'].at(index)}|' /etc/ntp.conf
EOH
end
end
But after the chef-client is ran and the changes are applied respectively, the contents of configuration file are as follows:
server abc.domain.com
server abc.domain.com
I am new to sed command so can't figure out where i'm going wrong.
Any help will be appreciated.
By design you should not modify files with Chef. Instead you overwrite the whole file with cookbook_file resource or, if you need to insert some dynamic values into the file, with template resource.
The sed command (the way you use it) is quite simple; it only performs (inplace in the given file due to the -i option) a substitution of each string matching the pattern server .* by the string server #{node['servers'].at(index)}. It does this throughout the whole file, so each loop changes all occurrences in the whole file.
What bothers me is that you write that in the original version you've got server1.domain.com but in the pattern you've got server .* (meaning server, followed by a space , and any amount of other characters .*). Because of the space, this should not match anything, so nothing should be changed at all. But maybe you just put that space in there by mistake when posting your question. I'll assume that there was no such space in your actual code because this way it would fit the observed phenomenon.
So, to change only one line at a time, you should have a counter in your loop and have the number of the iteration in the search pattern, so that it is server1.* for the first iteration, server2.* for the second and so on. Then each iteration will change only exactly one line and you should get your required result.
Related
This question already has answers here:
Editing/Replacing content in multiple files in Unix AIX without opening it
(2 answers)
Closed 6 years ago.
I wonder know if there is a way to edit a conf file without getting in the file and changing the lines?
In my case, I need to edit zabbix-agent conf file (located in /etc/zabbix/zabbix_agentd.conf) and there are some parameters in the file that I need to change, such as Server Name, DebugLevel, and others.
Normally I edit the file using vim and change the lines, but my idea is to edit the file directly from bash, but I don`t know if this is possible.
For example, if I need to change the parameter DebugLevel, at bash I would run:
# /etc/zabbix/zabbix_agentd.conf.DebugLevel=3
This actually doesn`t works, but I would like something like this for my problem...
Does anyone knows??
I tested what David said, but it didn`t solved my problem... There are some lines in the file that is commented and I need to uncomment them, and there are some lines that I just need to change.
For example, the line above:
# DebugLevel=3
I need to change to:
DebugLevel=3
And this line:
Server=127.0.0.1
I need to change for the IP of zabbix server name, like this:
Server=172.217.28.142
Is there any other way?
If I understand your question correctly, then you want sed -i (the -i option allows sed to edit the file in place, and -i.bak will do the same but create a backup of the original file with the .bak extension)
To change DebugLevel=3 to DebugLevel=4 in file /etc/zabbix/zabbix_agentd.conf, you can do:
# sed -i.bak "/DebugLevel/s/[0-9][0-9]*$/4/" /etc/zabbix/zabbix_agentd.conf
If I misinterpreted your question, please let me know.
To Change Values at the End
Example Input File
$ cat file.txt
DebugLevel=3
Example Use
$ sed -i "/DebugLevel/s/[0-9][0-9]*$/4/" file.txt
$ cat file.txt
DebugLevel=4
To Remove Comments
You can do virtually the same thing to uncomment the parameters of interest, for example:
# sed -i.bak "/DebugLevel/s/^#//" /etc/zabbix/zabbix_agentd.conf
In each case, sed is searching for the label in the first set of forward slashes /.../, next the substitute command is called s and then the expression between the next set of forward slashes, e.g. /^#/ (^ match at the beginning), if matched, is replaced by what follows in the next set // (nothing in the remove comment case).
You will need to adjust the values as required to match each parameter you need to find and replace. Let me know if you have further problems and exactly what the problem is.
A config file that the last line contains data that I want to assign everything to the RIGHT of the = sign into a variable that I can display and call later in the script.
Example: /path/to/magic.conf:
foo
bar
ThisOption=foo.bar.address:location.555
What would be the best method in a bash shell script to read the last line of the file and assign everything to the right of the equal sign? In this case, foo.bar.address:location.555.
The last line always has what I want to target and there will only ever be a single = sign in the file that happens to be the last line.
Google and searching here yielded many close but non-relative results with using sed/awk but I couldn't come up with exactly what I'm looking for.
Use sed:
variable=$(sed -n 's/^ThisOption=//p' /path/to/magic.conf)
echo "The option is: $variable")
This works by finding and removing the ThisOption= marker at the start of the line, and printing the result.
IMPORTANT: This method absolutely requires that the file be trusted 100%. As mentioned in the comments, anytime you "eval" code without any sanitization there are grave risks (a la "rm -rf /" magnitude - don't run that...)
Pure, simple bash. (well...using the tail utility :-) )
The advantage of this method, is that it only requires you to know that it will be the last line of the file, it does not require you to know any information about that line (such as what the variable to the left of the = sign will be - information that you'd need in order to use the sed option)
assignment_line=$(tail -n 1 /path/to/magic.conf)
eval ${assignment_line}
var_name=${assignment_line%%=*}
var_to_give_that_value=${!var_name}
Of course, if the var that you want to have the value is the one that is listed on the left side of the "=" in the file then you can skip the last assignment and just use "${!var_name}" wherever you need it.
So one of our developers needs me to batch a bunch of information and process it into an OVA to be presented back for download. This is an easy process using the long method (ie writing to the filesystem), but the developers want a cleaner, streamlined solution that will scale better. They have therefore requested that I stream the entire processes which is proving difficult. Can someone please give me some direction. Here are the steps that need to be accomplished:
Get input from webserver (Webserver will pass these as stream eventually.)
Random password
XML file
Modify boot script on file system (ie insert random password generated by server)
Create ISO of XML file and boot script
Calculate the SHA1 sum of ISO
Append SHA1 sum of ISO to manifest file in OVF directory
Create OVA from OVF directory
Here is an example directory structure (I outlined this in / just for simplicity)
/--
|
|--ISO/
| |
| |--boot.sh (Where the random password gets inserted)
| |--config.xml (This is handed from the web server. Needs to stream from server)
|
|--OVF/
|
|--disk.vmdk
|--ovf.xml
|--manifest.mf (Contains SHA1 of all files in OVF directory)
|--boot.iso (This file will exist once created from ISO directory)
Here is what I have so far (I'll explain the issues afterwards. Yes... there are a lot of issues):
cat /ISO/boot.sh | sed "s%DEFAULT%RANDOM%" | mkisofs /ISO/* | echo "SHA1(boot.iso)= " && sha1sum >> manifest.mf | tar -cvf success.ova /OVF/*
NOTE
In boot.sh there is a variable set to DEFAULT like this (Just for testing purposes):
PASSWORD="DEFAULT"
NOTE
This is what a line in the manifest file should look like:
SHA1(boot.iso)= 5fbc0d70 BLAH BLAH BLAH a91c9121bb
So I've never tried to write an entire script in one stream before. Usually I write to the filesystem a lot as I go. The first issue I see with this is that sed is replacing the string, but what it's piping over to mkisofs will not be used as mkiosfs is just going to make an iso of what it finds in /ISO. I dont even know if you can pass something like that to mkisofs. Piping is sometimes weird to think about.
Next, I think mkisofs is ok because I didnt specify a file output, therefore it should output to stdout which will be passed to sha1sum, but and here is the next problem I see. I need to append some additional text to the file before the SHA1 sum gets added which kinda interrupts the stream.
Finally, the last problem I see is how to pass everything to be tar into OVA without writing to the filesystem first (writing to manifest.mf).
Oh and the last BIG problem which I should have mentioned first is the config.xml file. Right now im dealing with it as just a file. The dev guys want to pass it to this script as a stream as well. I dont have a clue how to handle that.
Any help would be greatly appreciated. These concepts are a little beyond my knowledge.
Thanks!
UPDATE 12/11/13 2:11PM EST
Testing each part individually right now. Will report findings below soon.
UPDATE 12/11/13 2:14PM EST
The following works:
cat /ISO/boot.sh | sed "s%DEFAULT%RANDOM%"
and produces the following output:
RANDOM="RANDOM"
Exactly as expected.
You are correct NeronLeVelu, I will have to come back later and look at sed more carefully when real random passwords are being generated. ie. Making sure proper characters are escaped. Right now though, I'm just testing the logic. I will worry about regex and escaping later. We have not even decided on random password yet. It's only temporary and will most likely be alphanumeric.
Moving onto next part. Still not sure how to take the output from sed (stdout) and use it to include in ISO creation without actually creating a file that gets written to the file system. It may not be possible without writing to file system. More to come soon
# for the password if it contain & \ and separator used in your sed (default is /)
Password4Sed="`echo \"${PASSWORD} | sed \"s/[\\/&]/\\\\&/g\"`"
# no need of a cat with a sed
sed "s/DEFAULT/${Password4Sed}/"/ISO/boot.sh > /tmp/mkisofs.input
Treat rest from this input and put some test to validate each step like empty crc value or mkisofs.input. This will help at runtime when production error occur
I am trying to redirect emails that match a particular pattern to a shell script which will create files containing the texts, with datestamped filenames.
First, here is the routine from .procmailrc that hands the emails off to the script:
:0c:
* Subject: ^Ingest_q.*
| /home/myname/procmail/process
and here is the script 'process':
#!/bin/bash
DATE=`date +%F_%N`
FILE=/home/myname/procmail/${DATE}_email.txt
while read line
do
echo "$line" 1>>"$FILE";
done
I have gotten very frustrated with this because I can pipe text to this script on the command line and it works fine:
mybox-248: echo 'foo' | process
mybox-249: ls
2013-07-31_856743000_email.txt process
The file contains the word 'foo.'
I have been trying to get an email text to get output as a date-stamped file for hours now, and nothing has worked.
(I've also turned logging on in my .procmailrc and that isn't working either -- I'm not trying to ask a second question by mentioning that, just wondering if that might provide some hint as to what I might be doing wrong ...).
Thanks,
GB
Quoting your attempt:
:0c:
* Subject: ^Ingest_q.*
| /home/myname/procmail/process
The regex is wrong, ^ only matches at beginning of line, so it cannot occur after Subject:. Try this instead.
:0c:process.lock
* ^Subject: Ingest_q
| /home/myname/procmail/process
I also specified a named lockfile; I do not believe Procmail can infer a lock file name from just a script name. As you might have multiple email messages being delivered at the same time, and you don't want their logging intermingled in the log file, using a lock file is required here.
Finally, the trailing .* in the regex is completely redundant, so I removed it.
(The olde Procmail mini-FAQ also addresses both of these issues.)
I realize your recipe is probably just a quick test before you start on something bigger, but the entire recipe invoking the process script can be completely replaced by something like
MAILDIR=/home/myname/procmail
DATE=`date +%F_%N`
:0c:
${DATE}_email.txt
This will generate Berkeley mbox format, i.e. each message should have a From_ pseudo-header before the real headers; if you are not sure whether this is already the case, you should probably use procmail -Yf- to make sure to make it so (otherwise there is really no way to tell where one message ends and another begins; this applies both to your original solution, and this replacement).
Because Procmail sees the file name you are delivering to, it can infer a lockfile name now, as a minor bonus.
Using MAILDIR to specify the directory is the conventional way to do this, but you can specify a complete path to an mbox file if you prefer, of course.
I have a web application that is deployed to a server. I am trying to create a script that amoing other things reads the current version of the web application from a properties file that is deployed along with the application.
The file looks like this:
//other content
version=[version number]
build=[buildnumber]
//other content
I want to create a variable that looks like this: version-buildnumber
Here is my script for it:
VERSION_FILE=myfile
VERSION_LINE="$(grep "version=" $VERSION_FILE)"
VERSION=${VERSION_LINE#$"version="}
BUILDNUMBER_LINE=$(grep "build=" $VERSION_FILE)
BUILDNUMBER=${BUILDNUMBER_LINE#$"build="}
THEVERSION=${VERSION}-${BUILDNUMBER}
The strange thing is that this works in some cases but not in others.
The problem I get is when I am trying to concatenate the strings (i.e. the last line above). In some cases it works perfectly, but in others characters from one string replace the characters from the other instead of being placed afterwards.
It does not work in these cases:
When I read from the deployed file
If I copy the deployed file to another location and read from there
It does work in these cases:
If I write a file from scratch and read from that one.
If I create my own file and then copy the content from the deployed file into my created file.
I find this very strange. Is there someone out there recognizing this?
It is likely that your files have carriage returns in them. You can fix that by running dos2unix on the file.
You may also be able to do it on the fly on the strings you're retrieving.
Here are a couple of ways:
Do it with sed instead of grep:
VERSION_LINE="$(sed -n "/version=/{s///;s/\r//g;p}" $VERSION_FILE)"
and you won't need the Bash parameter expansion to strip the "version=".
OR
Do the grep as you have it now and do a second parameter expansion to strip the carriage return.
VERSION=${VERSION_LINE#$"version="}
VERSION=${VERSION//$'\r'}
By the way, I recommend habitually using lowercase or mixed case variable names in order to reduce the chance of name collisions.
Given this foo.txt:
//other content
version=[version number]
build=[buildnumber]
//other content
you can extract a version-build string more easily with awk:
awk -F'=' '$1 == "version" { version = $2}; $1 == "build" { build = $2}; END { print version"-"build}' foo.txt
I don't know why your script doesn't work. Can you provide an example of erroneous output?
From this sentence:
In some cases it works perfectly, but in others characters from one string replace the characters from the other instead of being placed afterwards.
I can't understand what's actually going on (I'm not a native English speaker so it's probably my fault).
Cheers,
Giacomo