rsyslog logging to multiple servers with different TLS configurations - rsyslog

Is it possible to have rsyslog log to multiple servers with different TLS configurations? We're currently logging to a local syslog server using the following:
$DefaultNetstreamDriver gtls
$DefaultNetstreamDriverCAFile /etc/pki/rsyslog/ca.pem
$DefaultNetstreamDriverCertFile /etc/pki/rsyslog/local-cert.pem
$DefaultNetstreamDriverKeyFile /etc/pki/rsyslog/local-key.pem
$ActionSendStreamDriverAuthMode anon
$ActionSendStreamDriverMode 1
*.* ##10.50.59.241:6514
We're now in the process of setting up logging to a third party and want to use TLS there as well. They state that we should set up rsyslog like this:
$DefaultNetstreamDriverCAFile /path/to/their/ca.crt
$ActionSendStreamDriver gtls
$ActionSendStreamDriverMode 1
$ActionSendStreamDriverAuthMode x509/name
$ActionSendStreamDriverPermittedPeer *.theirhost.theirdomain
*.* ##theirhost.theirdomain:6514
I figure that I can simply combine the CA's into a single file and set DefaultNetstreamDriverCAFile to that. But if I simply add the remaining second set of options to the bottom of my rsyslog.conf then the permitted peer causes a conflict with the first host. So is there any way to configure rsyslog (we're currently using 7.4.8) to use vastly different TLS setups to two different targets?

Well after a bunch of head-banging I figured this out on my own. First off, there's a bug in some versions of rsyslog that will prevent this from working (you'll never see a connection established to one or more of the target servers) so make sure you're using version 7.6 or later of rsyslog.
Make sure your CA file has any CA's needed for all targets listed in it. Order isn't important. Then your conf file should look something like this:
$DefaultNetstreamDriverCAFile /etc/pki/rsyslog/ca.pem
*.* action(type="omfwd"
protocol="tcp"
Target="10.50.59.241"
Port="6514"
StreamDriverMode="1"
StreamDriver="gtls"
StreamDriverAuthMode="anon"
)
*.* action(type="omfwd"
Protocol="tcp"
Target="some.other.host.com"
Port="6514"
StreamDriverMode="1"
StreamDriver="gtls"
StreamDriverAuthMode="x509/name"
StreamDriverPermittedPeers="*.some.other.host.com"
)

Related

Jython script to modify queue connection factories

I have queue manager already configured in Queue connection factories of websphere
currently settings are with out SSl enabled.
I want to enable this two things
Use SSL to secure communication with IBM MQ
specify the SSl to be used
Need your help to edit this things through jython
Use modifyWMQConnectionFactory command for this task.
See below example:
#Set QCF Name
qcfName='MYQCF'
#Set custom SSL config name
sslConfig='CUSTOMSSLSettings'
#Get the list of all QCF's in the environment
qcfList=AdminConfig.list('MQQueueConnectionFactory').splitlines()
#You can futher refine this command and restrict the query to a specific resource scope
#eg: qcfList=AdminConfig.list('MQQueueConnectionFactory', clusterId).splitlines()
#Iterate the list and update the config for MYQCF
for qcf in qcfList:
if qcfName == AdminConfig.showAttribute(qcf, 'name'):
print 'Enable SSL config for QCF: '+qcfName+' and use '+sslConfig+' SSL configuration'
AdminTask.modifyWMQConnectionFactory(qcf, '[-sslType SPECIFIC -sslConfiguration '+sslConfig+']')
else:
print 'Skip SSL config update for QCF: '+AdminConfig.showAttribute(qcf, 'name')
#save the changes
AdminConfig.save()

ClickHouse Question For DB::Exception URL “XXX” is not allowed in config.xml

I use the remote function to import data from remote servers in another clickhouse cluster, when I use the sql like :
INSERT INTO dataplugin.ods_stb_info_all_local
SELECT evtTime,evtCode,pVer,sdkVer,sdkSortId,stbID,manufacturer,model,cpName,macAddress,wifiMacAddress,romVer,apkVer,accessMethod,provinceCode,cityCode,now() as writeTime
FROM remote('xxx.xxx.xxx.xxx:19000','dataplugin','ods_stb_info_all','default','');
the error happens like this:
Code: 491. DB::Exception: Received from localhost:9000. DB::Exception: URL "xxx.xx.xxx.xxx:19000" is not allowed in config.xml.
I really can not understand what happened, I wish for someone's helps
It needs to define remote_url_allow_hosts-section in config.xml:
<yandex>
<!-- The list of hosts allowed to use in URL-related storage engines and table functions.
If this section is not present in configuration, all hosts are allowed.
-->
<remote_url_allow_hosts>
<!-- Host should be specified exactly as in URL. The name is checked before DNS resolution.
Example: "yandex.ru", "yandex.ru." and "www.yandex.ru" are different hosts.
If port is explicitly specified in URL, the host:port is checked as a whole.
If host specified here without port, any port with this host allowed.
"yandex.ru" -> "yandex.ru:443", "yandex.ru:80" etc. is allowed, but "yandex.ru:80" -> only "yandex.ru:80" is allowed.
If the host is specified as IP address, it is checked as specified in URL. Example: "[2a02:6b8:a::a]".
If there are redirects and support for redirects is enabled, every redirect (the Location field) is checked.
-->
<!-- Regular expression can be specified. RE2 engine is used for regexps.
Regexps are not aligned: don't forget to add ^ and $. Also don't forget to escape dot (.) metacharacter
(forgetting to do so is a common source of error).
-->
</remote_url_allow_hosts>
</yandex>
Example from test test_allowed_url_from_config:
<remote_url_allow_hosts>
<host>host:80</host>
<host_regexp>^[a-z]*\.ru$</host_regexp>
</remote_url_allow_hosts>
Steps:
open config.xm for editing
sudo nano /etc/clickhouse-server/config.d/config.xml
define the required settings
config.xml:
<yandex>
..
<remote_url_allow_hosts>..</remote_url_allow_hosts>
..
</yandex>
restart service to apply settings
sudo service clickhouse-server restart

How can we run connect-standalone with the same connector with two different configurations?

I am using a sink connector with two different sinks(of the same type), the connector needs to be invoked with two different configuration. Currently, we are dealing with that using screen:
launchScript.sh
screen -dmS conn-one runConnector.sh config/connect-sink-conn-one.properties config/conn-one.properties
screen -dmS conn-two ./bin/runConnector.sh config/conn-two.properties config/conn-two.properties
runConnector.sh
connect-standalone $1 $2
Is there a way of achieving this without having to use GNU Screen?
The bin/connect-standalone.sh script can take multiple properties files with different connector configurations. You just have to use different names for each connector. So for example you can do something like this:
bin/connect-standalone.sh config/connect-standalone.properties config/connect-file-sink.properties config/connect-file-sink2.properties
Where config/connect-file-sink.properties is:
name=local-file-sink
connector.class=FileStreamSink
tasks.max=1
file=test.sink.txt
topics=connect-test
and config/connect-file-sink2.properties is:
name=local-file-sink2
connector.class=FileStreamSink
tasks.max=1
file=test.sink.txt2
topics=connect-test
connect-standalone already takes one or more connector property files in addition to one worker property files.
Or you can run connect-distributed, as its meant to be a long running service, then you POST as many JSON configs as you want to port 8083, by default

Configure ini file for HHVM 3.0 via socket with nginx

I'm able to start the HipHop VM to use a unix socket. I can accomplish this via:
/usr/bin/hhvm --config /etc/hhvm/server.ini --mode daemon -vPidFile=/var/run/hhvm/pid -vServer.Type=fastcgi -vServer.FileSocket=/var/run/hhvm/hhvm.sock
However, I can't find a reference anywhere with how to set this in the ini file I'm specifying for my config. To use a TCP port the line in server.ini is:
hhvm.server.port = 9000
I've tried both
hhvm.server.filesocket=/var/run/hhvm/hhvm.sock
hhvm.server.socket=/var/run/hhvm/hhvm.sock
Both fail. Anyone know the file setting or where a reference for these settings can be found?
Although I can't find any documentation--they haven't yet written the updated version for the ini file format (as of 2014-05-01): https://github.com/hhvm/hack-hhvm-docs/issues/156
Regardless I figured it out and they confirmed it should be:
hhvm.server.file_socket=/var/run/hhvm/hhvm.sock
It looks like you take the camel case command line argument -vServer.FileSocket and drop the v, lowercase it, split it with underscores instead of camel case.
If y ou follow the above rewrite rules you can convert the old format to the new.

How to clone a WebSphere 8 profile

I would like to clone an existing profile, start the server and modify it via the Admin Console.
I already read the IBM documentation about
manageprofiles.bat
but the manageprofiles tool does not contain something like:
# would be nice if a clone action exists
manageprofiles.bat -clone -profileName base -targetProfileName base1
This is what I need and I don't see a way to achieve this.
The tool can create, delete, backup and restore a profile.
What I already tried
Copied the profile directory and renamed it
Edited the paths in the bin/setupCmdLine.bat
Added the profile to the AppServer/properties/profileRegistry.xml
Executed manageprofiles -validateAndUpdateRegistry
But the profile is still not recognized by WAS. I can verify this by executing
manageprofiles -listProfiles
How do you clone or copy profiles?
Is there a manual way?
If so, which files in the profile's dir must be edited?
Solved
Here are the manual steps that I did to clone an existent profile.
Make shure that the server is shutdown.
Copy the existent profile from Profiles/<oldProfile> to Profiles/<newProfile>
Update Path WAS_USER_SCRIPT in Profiles\<newProfile>\bin\setupCmdLine.bat
Update Path USER_INSTALL_ROOT in Profiles\<newProfile>\bin\setupCmdLine.bat
Update property user.root in Profiles\<newProfile>\properties\ssl.client.props
Replace all occurences of <oldProfile> with <newProfile> in Profiles\<newProfile>\firststeps\firststeps.bat
Edit AppServer\properties\profileRegistry.xml. Make a copy of the <oldProfile> and update the tag values with the <newProfile>. Should look something like this: <profile isAReservationTicket="false" isDefault="false" name="newProfile" path=".....\Profiles\newProfile" template=".......\AppServer\profileTemplates\default"/>
Copy AppServer\properties\fsdb\<oldProfile>.bat to AppServer\properties\fsdb\<newProfile>.bat. This step will make the profile available to "AppServer\bin\manageprofiles.bat -listProfiles"
Edit config/cells/<cell>/nodes/<node>/variables.xml. Update the USER_INSTALL_ROOT path.
Update the path of WAS_USER_SCRIPT in AppServer\properties\fsdb\<newProfile>.bat
This worked for me. Please comment or vote to let me know if it also worked for you.
Here is a build-in WebSphere solution. This works fine on WAS 8.5.5.3.
Run your application server in source profile
Run command line utility by running script <WAS_INSTAL_ROOT>/AppServer/profiles/<YOUR_SOURCE_PROFILE>/bin/wsadmin.sh or <WAS_INSTAL_ROOT>/AppServer/profiles/<YOUR_SOURCE_PROFILE>/bin/wsadmin.bat (on Windows machines)
Execute command in this utility:
$AdminTask exportWasprofile {-archive <PATH/TO/PROFILE/EXPORT/ARCHIVE>}
Create new target profile or use existing and run application server on it
Run command line utility for target profile by running script <WAS_INSTAL_ROOT>/AppServer/profiles/<YOUR_TARGET_PROFILE>/bin/wsadmin.sh or <WAS_INSTAL_ROOT>/AppServer/profiles/<YOUR_TARGET_PROFILE>/bin/wsadmin.bat (on Windows machines)
Execute command to import profile settings into target profile in this utility:
$AdminTask importWasprofile {-archive <PATH/TO/PROFILE/EXPORT/ARCHIVE> -deleteExistingServers}
Execute command to save new settings into target profile in this utility:
$AdminConfig save
I needed to do this today on WebSphere 8.5 Network Deployment, so I tracked it down and wrote a script. Here's how I do it.
Create a WebSphere profile using the GUI based Profile Management Tool.
Each one you create will write a log file to:
C:\Program Files (x86)\IBM\WebSphere\AppServer\logs\manageprofiles\XXXX_create.log
In the log, look for the -create switch. Clean up and copy that line into a script, modifying or parameterizing the fields as you wish:
SET PROFILENAME=Profile2
SET HOSTNAME=MyHostName
SET WAS_BIN=C:\Program Files (x86)\IBM\WebSphere\AppServer\bin
SET PROFILE_PATH=C:\Program Files (x86)\IBM\WebSphere\AppServer\profiles\%PROFILENAME%
SET WAS_TEMPLATE=C:\Program Files (x86)\IBM\WebSphere\AppServer\profileTemplates\default
SET WAS_CELL=%HOSTNAME%%PROFILENAME%Cell
SET WAS_PORTS_FILE=C:\Program Files (x86)\IBM\WebSphere\AppServer\logs\manageprofiles\was_default_portdef.props
SET A1=-create -serverName %PROFILENAME% -profileName %PROFILENAME% -cellName %WAS_CELL% -nodeName %PROFILENAME% -hostName %HOSTNAME%
SET A2=-profilePath "%PROFILE_PATH%" -templatePath "%WAS_TEMPLATE%"
SET A3=-applyPerfTuningSetting development -enableAdminSecurity false -winserviceCheck false -omitAction defaultAppDeployAndConfig
SET A4=-portsFile "%WAS_PORTS_FILE%"
SET A5=-signingCertDN "cn=10.0.2.15\\,ou=Root Certificate\\,ou=%WAS_CELL%\\,ou=%PROFILENAME%\\,o=IBM\\,c=US" -personalCertDN "cn=10.0.2.15\\,ou=%WAS_CELL%\\,ou=%PROFILENAME%\\,o=IBM\\,c=US"
SET A6=-personalCertValidityPeriod 10 -signingCertValidityPeriod 15
SET WAS_ARGS=%A1% %A2% %A3% %A4% %A5% %A6%
call "%WAS_BIN%\manageprofiles.bat" %WAS_ARGS%
You will need to look for the "C:\Program Files (x86)\IBM\WebSphere\AppServer\logs\manageprofiles*_portdef.props" files and copy one of them for your use as I did.
The output of my script was thus a cloned/tweaked version of a profile I had created before.
I then ran it at the commandline:
"C:\Program Files (x86)\IBM\WebSphere\AppServer\bin\manageprofiles.bat" -create -serverName Unica9103 -profileName Unica9103 -cellName UnicaVMUnica9103Cell -nodeName Unica9103 -hostName UnicaVM -profilePat
h "C:\Program Files (x86)\IBM\WebSphere\AppServer\profiles\Unica9103" -templatePath "C:\Program Files (x86)\IBM\WebSphere\AppServer\profileTemplates\default" -a
pplyPerfTuningSetting development -enableAdminSecurity false -winserviceCheck false -omitAction defaultAppDeployAndConfig -portsFile "C:\Program Files (x86)\IB
M\WebSphere\AppServer\logs\manageprofiles\was_default_portdef.props" -signingCertDN "cn=10.0.2.15\\,ou=Root Certificate\\,ou=UnicaVMUnica9103Cell\\,ou=Unica9103
\\,o=IBM\\,c=US" -personalCertDN "cn=10.0.2.15\\,ou=UnicaVMUnica9103Cell\\,ou=Unica9103\\,o=IBM\\,c=US" -personalCertValidityPeriod 10 -signingCertValidityPeri
od 15
Notice that the commas need to be escaped with double backslashes or you get cryptic errors.
You will get something like the following on success:
INSTCONFSUCCESS: Success: Profile Unica9103 now exists. Please consult C:\Program Files (x86)\IBM\WebSphere\AppServer\profiles\Unica9103\logs\AboutThisProfile.txt for more information about this profile.
I have not tested the method but wsadmin has the following command:
createAppServerTemplate
This script creates a new application server template in your
configuration. A server template is used to define the configuration
settings for a new application server. When you create a new
application server, you either select the default server template or a
template you previously created, that is based on another, already
existing application server. The default template is used if you do
not specify a different template when you create the server.
You could create a template from your initial profile and create a new profile using the template.
I believe that createAppServerTemplate creates e template for the server not the profile. Therefore if you already have configured a profile you can than use the template to create additional servers within this profile.
You would then use createApplicationServer to create new servers based on the above created template.
I guess if you want to copy a profile the best method is to record the configurations and set them in a script file.
Just for information In MyEclipseBlue version 10 we can export server profile configuration & import in new profile; hence retaining all the configuration.
René answer is correct but i faced issues while Running the server in Debug Mode.
\WebSphere8552\AppServer\properties\fsdb\AppSrv01.bat change was_user_script path
\WebSphere8552\AppServer\properties\fsdb_was_profile_defauldefault.bat change was_user_script path with actual
Also change path in : \AppServer\properties\profileRegistry.xml for validating license

Resources