I have few file, the name of the files if vhost-<someport>.conf, I want to replace the line with the port number in each file by the filename $port vhost-$port.conf.
For example vhost-8081.conf file will contain the line listen 8081, vhost-8082.conf file will contain the line listen 8082;, etc..
vhost-8081.conf
vhost-8082.conf
etc..
server {
listen 8081;
}
server {
listen 8082;
}
If you have files named vhost-NUM.conf, then you can strip the vhost- prefix and the .conf suffix to extract the port number, and then use sed -i to replace listen .* with the number in-place, updating the file:
for file in vhost-*.conf; do
num=${file#vhost-}
num=${num%.conf}
sed -i "s/listen .*/listen $num;/" "$file"
done
This may be what you want:
for file in vhost-*.conf
do num=${file#vhost-}
sed -i "s/listen .*/vhost-$num;/" "$file"
done
cat vhost*
server {
vhost-8081.conf;
}
server {
vhost-8082.conf;
Related
I have python script generating AWS Signature key for S3. It generates two values:
GZkXNl6Leat71ckcwfxGuiHxt9fnkj47F1SbVjRu/t0=
20190129/eu-west-2/s3/aws4_request
Both are valid for 7 days. What I want is to run that script for every five days using cron inside the Docker container, grab the output and place/replace values in the Nginx config
config:
server {
listen 80;
aws_access_key 'AKIDEXAMPLE';
aws_signing_key FIRST_VALUE;
aws_key_scope SECOND_VALUE;
aws_s3_bucket s3_bucket_name;
location / {
aws_sign;
proxy_pass http://s3_bucket_name.s3.amazonaws.com;
}
Then restart nginx in the container
Assuming the values are stored in val_file, slotting them into nginx.conf, this simplistic solution ought to do -
$: cat script
#! /bin/env bash
{ read -r val1 # read the first line
read -r val2 # read the second line
sed -i "s!FIRST_VALUE!$val1!;
s!SECOND_VALUE!$val2!;
" nginx.conf
} < val_file
$: script
$: cat nginx.conf
server {
listen 80;
aws_access_key 'AKIDEXAMPLE';
aws_signing_key GZkXNl6Leat71ckcwfxGuiHxt9fnkj47F1SbVjRu/t0=;
aws_key_scope 20190129/eu-west-2/s3/aws4_request;
aws_s3_bucket s3_bucket_name;
location / {
aws_sign;
proxy_pass http://s3_bucket_name.s3.amazonaws.com;
}
The curlies make the single input supply both reads. Then it's just a sed, using !'s because you have standard forward slashes in your data. Double quotes on the sed to allow embedded vars - not ideal, but seems ok here.
I would like to write a script that reads a text file that has all the nodes listed in there:
node1
node2
node3
.
.
.
It creates a .conf file for each node in the
/etc/icinga2/zones.d/master/hosts/new/ directory
Copies the content of the file name windows-template into each
new conf file.
Then finds the phrase "hostname.hostdomain.com" in each conf file
and replaces that with the filename minus the .conf. So for example,
for node1, I will have node1.conf in which there is a phrase
"hostname.hostdomain.com" which needs to be replaced with node1
Then pings the hostname which is technically the filename minus
".conf" and replaces the 10.20.20.1 with the correct hostname.
I tried wrirting the script and part 1 and 2 work, part 3 works too but it replaces the hostname.hostdomain.com with "$f" which is not right. And I have no clue how to do number 4.
Can you please help?
Thank you
This is my windows-template.conf file:
object Host "hostname.hostdomain.com" {
import "production-host"
check_command = "hostalive"
address = "10.20.20.1"
vars.client_endpoint = name
vars.disks["disk C:"] = {
disk_partition = "C:"
}
vars.os = "Windows"
}
object Zone "hostname.hostdomain.com" {
endpoints = [ "hostname.hostdomain.com" ];
parent = "master";
}
object Endpoint "hostname.hostdomain.com" {
host = "10.20.20.1"
}
And this is my script:
#!/bin/bash
cd /etc/icinga2/zones.d/master/hosts/new
while read f; do
cp -v "$f" /etc/icinga2/zones.d/master/hosts/new/"$f.conf"
cp windows-template.conf "$f.conf"
chown icinga:icinga "$f.conf"
sed -i 's/hostname.hostdomain.com/$f/g' "$f.conf"
# git add "$f.conf"
# git commit -m "Add $f"
done < windows-list.txt
Thank you
You need double quotes for the shell to expand your variable. Try
sed -i "s/hostname.hostdomain.com/$f/g" "$f.conf"
Does this work for you?
#!/bin/bash
cd /etc/icinga2/zones.d/master/hosts/new
while read f; do
cp -v "$f" /etc/icinga2/zones.d/master/hosts/new/"$f.conf"
cp windows-template.conf "$f.conf"
chown icinga:icinga "$f.conf"
sed -i "s/hostname.hostdomain.com/$f/g" "$f.conf"
hostname=$( ssh -o StrictHostKeyChecking=no "username#$f" -n "hostname" )
mv "$f.conf" "${hostname}.conf"
# git add "$f.conf"
# git commit -m "Add $f"
done < windows-list.txt
Where username is your username, and I assume you copy your pub key to the hosts.
I want to write a script to go through my UFW firewall logs for blocked connections, and pull out the source IP and port they were trying to connect to. I have managed to put together scripts to pull these out individually, and am having difficulty in getting the script now to output the IP and port on one line. Below is a sample of the firewall logs I have and the script. Currently it output all IPs, followed by all port numbers. What I want on each line is the source IP address and corresponding destination port from each log entry.
The ultimate aim is to see what ports each IP address is trying to connect to. My plan was to use uniq -c, once I can output each IP and port from the logs.
input
Nov 26 06:25:11 vps123456 kernel: [620802.845897] [UFW BLOCK] IN=ens3 OUT= MAC=fa:16:3e:9f:c7:5d:11:22:33:44:55:66:77:88 SRC=85.93.20.253 DST=10.20.30.40 LEN=40 TOS=0x00 PREC=0x00 TTL=245 ID=5830 PROTO=TCP SPT=51639 DPT=735 WINDOW=1024 RES=0x00 SYN URGP=0
Nov 26 06:27:44 vps123456 kernel: [620955.012996] [UFW BLOCK] IN=ens3 OUT= MAC=fa:16:3e:9f:c7:5d:11:22:33:44:55:66:77:88 SRC=51.15.51.140 DST=10.20.30.40 LEN=433 TOS=0x00 PREC=0x00 TTL=50 ID=42044 DF PROTO=UDP SPT=5088 DPT=5062 LEN=413
./script
file="input"
for line in $file; do
addr=$(awk '{match($0,/SRC=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}' $file)
port=$(awk '{match($0,/DPT=[0-9]{0,5}/); port = substr($0,RSTART,RLENGTH); print port}' $file)
echo $addr, $src >> output
done
output
SRC=85.93.20.253 SRC=51.15.51.140, DPT=735 DPT=5062
A quick php solution for you problem, that will print on each line the source ip address and the corresponding destination port. The script reads your input logfile line by line and process each row. Match ip address and port number into variables using regex.
<?php
$infile = $argv[1];
$handle = fopen($infile, "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
preg_match('/.*SRC=(?<ip>[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)(.*DPT=)(?<port>[0-9]+).*/', $line, $m);
if ($m) {
print $m['ip'] . ' ' . $m['port'] . "\n";
}
}
fclose($handle);
} else {
exit("Unable to open file ($finfile)");
}
After saving the above script into a file (ie.: get-uwf-ip-port.php), you can run in the following way:
php get-uwf-ip-port.php your_logfile
The output (source file was your two example lines):
85.93.20.253 735
51.15.51.140 5062
UPDATE:
And here is a native bash solution. The main idea here is to read the file line by line, read each line into a shell variable, extract the ip and the port number using shell parameter expansion.
#!/bin/bash
infile="$1"
while read line; do
ip="${line##*SRC=}"
ip="${ip%% *}"
port="${line##*DPT=}"
port="${port%% *}"
echo $ip $port
done < "$infile"
I slightly modified your awk code, putting the ip and port matching together it will produce the same result as above:
awk '{match($0,/SRC=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART+4,RLENGTH-4); match($0,/DPT=[0-9]{0,5}/); port = substr($0,RSTART+4,RLENGTH-4); print ip,port}' your_logfile
I am adding several lines to a file like this:
echo "server {
listen 80;
server_name ${brand}.mydomain.com;
root /srv/www/clients/${brand}/soon;
}" >> default
The result is like this (brand is passed via parameter) :
server {
listen 80;
server_name cola.mydoman.com;
root /srv/www/clients/cola/soon;
}
The question is how to remove this specifically if this file contains lots of similar other values. Is it possible to do this with sed, awk or something else?
I have 100s of config files, each 10,000 to 20,000 lines long. These are config files for hardware. I need to search through all the config files to find the "profile" associated with a given cert name. There are several different versions of hardware software so the configs files are somewhat different. However the profile name is always above the cert. The profile name does not necessarily contain the cert name.
Example of Profile Names:
clientssl_www.profile-cert
clientssl_www.example.com-cert
Example of Cert Name:
www.example.com.crt
Example sections of config:
profile clientssl clientssl_www.profile-cert {
defaults from clientssl
key "www.example.com.key"
cert "www.example.com.crt"
chain "Intermediate-bundle.crt"
options {
cipher server preference
dont insert empty fragments
no sslv2
}
}
ltm profile client-ssl /Common/clientssl_www.example.com-cert {
app-service none
cert /Common/www.example.com.crt
cert-key-chain {
www.example.com_www.example.com {
cert /Common/www.example.com.crt
chain /Common/Intermediate-bundle.crt
key /Common/www.example.com.key
}
}
chain /Common/Intermediate-bundle.crt
ciphers
key /Common/www.example.com.key
options { dont-insert-empty-fragments cipher-server-preference no-sslv2 }
}
I cannot read the config files line by line as there are millions of lines and it simply takes too long.
I can find the cert name with grep using something like this:
$ grep www.example.com *file.conf | egrep 'cert "|cert /Common'
Which gives me something like this:
cert "www.example.com.crt"
cert /Common/www.example.com.crt
cert /Common/www.example.com.crt
I need to find the 'profile name' that is above my search for a given cert name.
Any suggestions?
Thanks!
You can use -B option of grep which comes handy in such cases. From the man pages for grep:
-B NUM, --before-context=NUM
Print NUM lines of leading context before matching lines. Places a line containing a group separator (--) between contiguous groups
of matches. With the -o or --only-matching option, this has no effect and a warning is given.
So, the pattern match will now be:
$ grep www.example.com *file.conf | egrep -B3 'cert "|cert /Common'
Output:
profile clientssl clientssl_www.profile-cert {
defaults from clientssl
key "www.example.com.key"
cert "www.example.com.crt"
--
ltm profile client-ssl /Common/clientssl_www.example.com-cert {
app-service none
cert /Common/www.example.com.crt
cert-key-chain {
www.example.com_www.example.com {
cert /Common/www.example.com.crt
However, you will still need to figure out some common pattern in the line containing profile name to single them out. It becomes difficult in your example to filter it further because in the first case, the profile name is three lines before the cert " pattern whereas in the second example, it is two lines before cert / pattern.
Another approach which i find better is to find some pattern in the profile name itself. If all profile names contain the string profile or if they have a pattern such as clientssl.*-cert, then the following pattern match will do what you need:
$ grep www.example.com *file.conf | egrep 'profile|clientssl.*-cert'
Output:
profile clientssl clientssl_www.profile-cert {
ltm profile client-ssl /Common/clientssl_www.example.com-cert {
Even better, if you know that the profile name starts with clientssl_ and ends with -cert, then
$ grep www.example.com *file.conf | grep -o clientssl_.*-cert
Output:
clientssl_www.profile-cert
clientssl_www.example.com-cert
This may be madness, but whenever I see sample data that fits Tcl's syntax rules, I look to produce a Tcl solution:
#!/usr/bin/env tclsh
proc unknown {cmdname args} {
set data [lindex $args end]
if {[set idx [lsearch -exact $data "cert"]] != -1 && [string match $::cert_pattern [lindex $data [incr idx]]]} {
set idx [expr {$cmdname eq "profile" ? 1 : [lsearch -exact $args "profile"] + 2}]
puts [lindex [split [lindex $args $idx] /] end]
}
}
set cert_pattern "*[lindex $argv 0]*"
foreach file [lrange $argv 1 end] {
source $file
}
Then
$ ./cert.tcl www.example.com file.conf
file.conf
clientssl_www.profile-cert
clientssl_www.example.com-cert
I won't bother to explain how it works unless there's a hue and cry.