I'm trying to cut certain virtual host from file which has a lot of them using bash.
Eg. in my script I would like to get virtual host which uses the_one_known2.example.com, but another time the_one_known3.example.com, namely I would like to get some part form apache config file with beforehand known URL (present in ServerName or ServerAlias) which is set in bash script as a parameter.
<VirtualHost *:80 *:${SERVER_PORT}>
ServerName test1.example.com
ServerAlias test2.example.com
ServerAlias the_one_known1.example.com
ServerAlias test3.example.com
</VirtualHost>
<VirtualHost *:80 *:${SERVER_PORT}>
ServerName test4.example.com
ServerAlias the_one_known2.example.com
ServerAlias test5.example.com
ServerAlias test6.example.com
</VirtualHost>
<VirtualHost *:80 *:${SERVER_PORT}>
ServerName the_one_known3.example.com
ServerAlias test7.example.com
ServerAlias test8.example.com
ServerAlias test9.example.com
</VirtualHost>
So I my URL variable would change to eg. the_one_known2.example.com I would get:
<VirtualHost *:80 *:${SERVER_PORT}>
ServerName test4.example.com
ServerAlias the_one_known2.example.com
ServerAlias test5.example.com
ServerAlias test6.example.com
</VirtualHost>
[EDIT]
So far I tried to find first line of a selected virtual host:
url_line=$(grep -n -m 1 "${URL}" ./apache.conf" | sed 's/\([0-9]*\).*/\1/')
vitual_host_start_line="$((app_line-1))" // this is an assumption that Virtual Host starts just one line before the first occurance of URL
echo $vitual_host_line // the place it starts
But I have a problem to find a last line of this virtual host because it is first occurence of </VirtualHost> after vitual_host_start_line
With awk:
awk -v name="the_one_known2.example.com" 'BEGIN{RS=ORS="</VirtualHost>\n"} $0~name{print; exit}' file
I assume the_one_known2.example.com is no substring of another domain in your file.
See: 8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR
With your shown samples, please try following awk code. Written and tested in GNU awk.
awk -v RS='<VirtualHost[^/]*/' -v ORS="VirtualHost>\n" 'RT~/[[:space:]]+the_one_known2\.example\.com\n/{print RT}' Input_file
Explanation: Simple explanation would be, making RS(record separator) from <VirtualHost till next / occurrence(non-greedy match). Then checking if it has [[:space:]]+the_one_known2\.example\.com\n if yes then print matched value, this will print VirtualHost> at ending of passage since its set as ORS value in program.
Hey I am new to Ansible and working with it for a project one of the steps is to input our rhel7 instance public ip address into the Ansible hosts file. We weren't given any other instruction and I wanted to make sure I did so correctly (line 11 of my Hosts File), any feedback would be greatly appreciated. Hosts File
# This is the default ansible 'hosts' file.
#
# It should live in /etc/ansible/hosts
#
# - Comments begin with the '#' character
# - Blank lines are ignored
# - Groups of hosts are delimited by [header] elements
# - You can enter hostnames or ip addresses
# - A hostname/ip can be a member of multiple groups
# Ex 1: Ungrouped hosts, specify before any group headers.
3.82.200.205
## green.example.com
## blue.example.com
## 192.168.100.1
## 192.168.100.10
# Ex 2: A collection of hosts belonging to the 'webservers' group
##[webservers]
## alpha.example.org
## beta.example.org
## 192.168.1.100
## 192.168.1.110
# If you have multiple hosts following a pattern you can specify
# them like this:
## www[001:006].example.com
# Ex 3: A collection of database servers in the 'dbservers' group
## [dbservers]
##
## db01.intranet.mydomain.net
## db02.intranet.mydomain.net
## 10.25.1.56
## 10.25.1.57
# Here's another example of host ranges, this time there are no
# leading 0s:
## db-[99:101]-node.example.com
To add-on Vladimir's answer, the servers created in AWS normally come with Elastic IP and it can be changed after you restart the server
So you need to make it dynamic inventory when your system becomes bigger and more complex, take an example from this article: https://medium.com/happy5/aws-dynamic-inventory-and-ansible-thank-god-i-can-sleep-more-4d2aeadbc6f. This approach works for me and I use it to manage more than 50 AWS EC2 instances with dynamic IPs
And looks like there is AWS plugin here: https://docs.ansible.com/ansible/latest/plugins/inventory/aws_ec2.html, I haven't give a try on this solution yet, but it may have some advantage as this is from Ansible official document
Yes. It's correct. Here is a link to Working with Inventory (for the record).
My Hosts file looks like:
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
# 102.54.94.97 rhino.acme.com # source server
# 38.25.63.10 x.acme.com # x client host
# localhost name resolution is handled within DNS itself.
127.0.0.1 localhost
::1 localhost
192.168.10.10 ingeniaticupct.ddns.net
I am working with Laravel. Every 1 hour approximately (without touch PC) (or always when I restart Homestead, or I do any change on Laravel files, etc...) Hosts file looks like:
# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
# 102.54.94.97 rhino.acme.com # source server
# 38.25.63.10 x.acme.com # x client host
# localhost name resolution is handled within DNS itself.
127.0.0.1 localhost
::1 localhost
# 192.168.10.10 ingeniaticupct.ddns.net
Ergo, my domain ingeniaticupct.ddns.net is autocommented (self-reported) (# 192.168.10.10 ingeniaticupct.ddns.net) and I do not understand why. I have been searching in several sites but I have not found solution yet.
Hope if you could help me.
Thank you all.
Ergo, my domain ingeniaticupct.ddns.net is autocommented (self-reported) (# 192.168.10.10 ingeniaticupct.ddns.net) and I do not understand why.
That’s most likely some kind of security application trying to protect your machine (from malware that would simply change or add entries in there, trying to redirect your requests for popular sites to their own servers.)
Might be Bitdefender, according to windows 10 hosts file entries been commented out on superuser.com; but there’s probably other programs that do this as well.
We installed Varnish but it was not work out like supposed to be.
I setup Varnish to listen on Port 80 and Apache to listen on 8080.
When i restart varnish and apache the pages keep loading and loading..
I also tried to disable CSF..
After that it shows:
Error 503 Backend fetch failed
Guru Meditation:
XID: 2818051
Do someone knows how to fix this issue?
Specs: 2.6.32-604.30.3.lve1.3.63.el6.x86_64 #1 SMP Sun Sep 27 06:34:10 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
CentOS - Cloudlinux 6.7
Below the config files:
httpd.conf
#
# This is the main Apache HTTP server configuration file. It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.4> for detailed information.
# In particular, see
# <URL:http://httpd.apache.org/docs/2.4/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do. They're here only as hints or reminders. If you are unsure
# consult the online docs. You have been warned.
#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# Do not add a slash at the end of the directory path. If you point
# ServerRoot at a non-local disk, be sure to specify a local disk on the
# Mutex directive, if file-based mutexes are used. If you wish to share the
# same ServerRoot for multiple httpd daemons, you will need to change at
# least PidFile.
#
ServerRoot "/etc/httpd"
HostnameLookups Off
#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
#
# Change this to Listen on specific IP addresses as shown below to
# prevent Apache from glomming onto all bound IP addresses.
#
#Listen 12.34.56.78:80
Listen 80
<IfModule unixd_module>
#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.
#
# User/Group: The name (or #number) of the user/group to run httpd as.
# It is usually good practice to create a dedicated user and group for
# running httpd, as with most system services.
#
User apache
Group apache
</IfModule>
#LoadModule dummy_module /usr/lib/apache/mod_dummy.so
Include /etc/httpd/conf/extra/httpd-phpmodules.conf
#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed. This address appears on some server-generated pages, such
# as error documents. e.g. admin#your-domain.com
#
ServerAdmin admin#localhost
DocumentRoot "/var/www/html"
<IfModule dir_module>
<IfModule dir_module>
DirectoryIndex index.html index.htm index.shtml index.php index.php5 index.php4 index.php3 index.phtml index.cgi
</IfModule>
#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<Files ".ht*">
Require all denied
</Files>
#
# The following lines prevent .user.ini files from being viewed by Web clients.
#
<Files ".user.ini">
Require all denied
</Files>
#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here. If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog /var/log/httpd/error_log
#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#
LogLevel warn
<IfModule log_config_module>
#replace %b with %O for more accurate logging
<IfModule mod_logio.c>
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%a %l %u %t \"%r\" %>s %O" common
LogFormat "%O %I" bytes
LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog /var/log/httpd/access_log common
</IfModule>
<IfModule alias_module>
# Include some DirectAdmin alias
Include conf/extra/httpd-alias.conf
</IfModule>
#DefaultType text/plain
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-gzip .tgz
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddHandler cgi-script .cgi
AddHandler type-map var
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
AddType video/x-ms-asf .avi
AddType video/mpeg .mpg
AddType video/mpeg .mpeg
AddType video/quicktime .mov
AddType video/x-ms-wmv .wmv
</IfModule>
#
# MaxRanges: Maximum number of Ranges in a request before
# returning the entire resource, or one of the special
# values 'default', 'none' or 'unlimited'.
# Default setting is to accept 200 Ranges.
#MaxRanges unlimited
#
# EnableMMAP and EnableSendfile: On systems that support it,
# memory-mapping or the sendfile syscall may be used to deliver
# files. This usually improves server performance, but must
# be turned off when serving from networked-mounted
# filesystems or if support for these functions is otherwise
# broken on your system.
# Defaults: EnableMMAP On, EnableSendfile Off
#
#EnableMMAP off
#EnableSendfile off
#######################################################################################
# For user configurations not maintained by DirectAdmin. Empty by default.
#######################################################################################
Include conf/extra/httpd-includes.conf
#######################################################################################
# Supplemental configuration
#######################################################################################
# Options and AllowOverrides
Include conf/extra/httpd-directories.conf
# Nginx reverse proxy configuration
Include conf/extra/httpd-nginx.conf
# Server-pool management (MPM specific)
Include conf/extra/httpd-mpm.conf
# Multi-language error messages
Include conf/extra/httpd-multilang-errordoc.conf
# Fancy directory listings
Include conf/extra/httpd-autoindex.conf
# Language settings
Include conf/extra/httpd-languages.conf
# User home directories
#Include conf/extra/httpd-userdir.conf
# Real-time info on requests and configuration
Include conf/extra/httpd-info.conf
# Suphp
Include conf/extra/httpd-suphp.conf
# Local access to the Apache HTTP Server Manual
#Include conf/extra/httpd-manual.conf
# Distributed authoring and versioning (WebDAV)
Include conf/extra/httpd-dav.conf
# Various default settings
Include conf/extra/httpd-default.conf
# Secure (SSL/TLS) connections
Include conf/extra/httpd-ssl.conf
# Deflate module settings
Include conf/extra/httpd-deflate.conf
#######################################################################################
# Do not change anything in files below, because they are rewritten by DirectAdmin #
#######################################################################################
# This is needed for PHP
Include conf/extra/httpd-php-handlers.conf
# Virtual hosts
Include conf/extra/httpd-vhosts.conf
# All the DirectAdmin vhosts
Include conf/extra/directadmin-vhosts.conf
#######################################################################################
# End of included files that are rewritten by DirectAdmin #
#######################################################################################
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
# Alias for RoundCube webmail
# Alias /roundcube /var/www/html/roundcube/
default.vcl
#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.
# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "149.210.175.70";
.port = "80";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
}
sub vcl_backend_response {
# Happens after we have read the response headers from the backend.
#
# Here you clean the response headers, removing silly Set-Cookie headers
# and other mistakes your backend does.
}
sub vcl_deliver {
# Happens when we have all the pieces we need, and are about to send the
# response to the client.
#
# You can do accounting or modifying the final object here.
Sysconfig/varnish
# Configuration file for varnish
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
# Maximum number of open files (for ulimit -n)
NFILES=131072
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"
# Maximum size of corefile (for ulimit -c). Default in Fedora is 0
# DAEMON_COREFILE_LIMIT="unlimited"
# Set this to 1 to make init script reload try to switch vcl without restart.
# To make this work, you need to set the following variables
# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,
# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE, or in short,
# use Alternative 3, Advanced configuration, below
RELOAD_VCL=1
# This file contains 4 alternatives, please use only one.
## Alternative 1, Minimal configuration, no VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# content server on localhost:8080. Use a fixed-size cache file.
#
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-b localhost:8080 \
-u varnish -g varnish \
-s file,/var/lib/varnish/varnish_storage.bin,1G"
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request. Use a
# fixed-size cache file.
#
#DAEMON_OPTS="-a :6081 \
# -T localhost:6082 \
# -f /etc/varnish/default.vcl \
# -u varnish -g varnish \
# -S /etc/varnish/secret \
# -s file,/var/lib/varnish/varnish_storage.bin,1G"
## Alternative 3, Advanced configuration
#
# See varnishd(1) for more information.
#
# # Main configuration file. You probably want to change it :)
VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=
VARNISH_LISTEN_PORT=80
#
# # Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
#
# # Shared secret file for admin interface
VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # The minimum number of worker threads to start
VARNISH_MIN_THREADS=50
#
# # The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000
#
# # Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=256M
#
# # Backend storage specification
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
#
# # Default TTL used when the backend does not specify one
VARNISH_TTL=120
#
# # DAEMON_OPTS is used by the init script. If you add or remove options, make
# # sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
-f ${VARNISH_VCL_CONF} \
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
-t ${VARNISH_TTL} \
-p thread_pool_min=${VARNISH_MIN_THREADS} \
-p thread_pool_max=${VARNISH_MAX_THREADS} \
-p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \
-u varnish -g varnish \
-S ${VARNISH_SECRET_FILE} \
-s ${VARNISH_STORAGE}"
#
## Alternative 4, Do It Yourself. See varnishd(1) for more information.
#
# DAEMON_OPTS=""
Your httpd.conf tells Apache to listen on port 80 too. Either Apache or Varnish will not be able to start.
Varnish listens on port 80 on your public server. Apache can run on port 80 - but on a separate server. If you are running them both, Varnish and Apache, on the same server than Apache needs to run on a different port, e.g., port 8080.
I have just created my first Rightscale ServerTemplate and Deployment using a bunch of Rightscripts. One of the scripts I created was to add a virtual host to apache.
#!/bin/bash -e
if [ $RS_DISTRO = ubuntu ]; then
export apache=apache2
export apache_extra_conf_dir=/etc/apache2/conf.d
elif [ $RS_DISTRO = centos ]; then
export apache=httpd
export apache_extra_conf_dir=/etc/httpd/conf.d
fi
server_name=$SERVER_NAME
echo "Adding virtual hosts to ${apache_extra_conf_dir}/vh-${server_name}.conf"
cat > $apache_extra_conf_dir/vh-${server_name}.conf <<EOF
NameVirtualHost $SITE_IP:$SITE_PORT
<VirtualHost $SITE_IP:$SITE_PORT>
ServerName $SERVER_NAME
ServerAlias $SITE_DOMAIN *.$SITE_DOMAIN
UseCanonicalName Off
ServerAdmin $ADMIN_EMAIL
DocumentRoot $APACHE_WWW_DIR
<Directory "$APACHE_WWW_DIR">
Options FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all
</Directory>
</VirtualHost>
EOF
service $apache restart
exit 0
My question is can I use the same Rightscript twice on the ServerTemplate but set different inputs for each (IP, Port, www dir and Servername)? eg.
ServerTemplate:
Execute Rightscript vhost: *:80 /www-x/ x.com
Execute Rightscript vhost: *:80 /www-y/ y.com
OR do I have to create a special Rightscript just for this server deployment that has both virtual hosts defined in the same script?
Execute Rightscript vhost: *:80 /www-x/ x.com | *:80 /www-y/ y.com
You can use the same RightScript with different inputs only if you put that script in the "Operational Scripts" section of your ServerTemplate. Then if you leave the inputs blank, you can execute that operational script with different inputs many times.
If you want that behavior in the "Boot Scripts" section, you'll have to either create a single script which handles two sets of inputs (for two vhosts) or create a clone of the RightScript with a different name and different input names.
You might consider taking a look at the Chef based tools. You can create a Chef recipe which takes an array of vhost names as input, then executes the "apache_site" resource that sets up a vhost. You can see an example in the RightScale Chef code linked below.
https://github.com/rightscale/cookbooks_public/blob/master/cookbooks/web_apache/recipes/setup_frontend_http_vhost.rb