I'm trying to organize my metrics into grafana with collectd.
When I use the GenericJMX plugin collectd includes GenericJMX as part of the metric name.
Query Grafana
How do I configure collectd to not include GenericJMX?
You might work with a rename rule like this:
<Chain "PreCache">
<Rule "rename_jmx">
<Match "regex">
Plugin "^GenericJMX$"
</Match>
<Target "set">
Plugin "java"
</Target>
</Rule>
</Chain>
This will rename 'GenericJMX' to 'java'.
Hope it helps!
You can control it with PluginName on a per bean basis. Unfortunately it does not seem to work on the top-level:
LoadPlugin java
<Plugin "java">
# ...
<Plugin "GenericJMX">
# Memory usage by memory pool.
<MBean "memory_pool">
ObjectName "java.lang:type=MemoryPool,*"
InstancePrefix "memory_"
InstanceFrom "name"
<Value>
Type "memory"
PluginName "jmx"
#InstancePrefix ""
#InstanceFrom ""
Table true
Attribute "Usage"
</Value>
</MBean>
# ...
</Plugin>
</Plugin>
See PluginName in the docs
Related
I am working on a data migration project, from Visualfoxpro to Oracle...!
I have successfully created the meta data and the packages script in BIML but when i tried to generate the SSIS package in Visual Studio, it give an error "Could not resolve reference to '' in property 'Database'. '' is invalid.".
I am not sure what Database name should be used for and Oracle DB as it's schema driven(In my understanding).
I have followed the article http://bimlscript.com/Walkthrough/Details/73
Changed 1-2-Environment.biml in the following way
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Connections>
<OleDbConnection Name="Source" ConnectionString="Provider=vfpoledb.1;Data Source=<Path to VFP database folder>;Exclusive=false;Nulls=false;" />
<OleDbConnection Name="Target" ConnectionString="Data Source=ServerName;User ID=UserName;Password=Password;Provider=OraOLEDB.Oracle.1;Persist Security Info=True;" />
</Connections>
<Databases>
<Database Name="NameForTheDatabase" ConnectionName="Target" />
</Databases>
<Schemas>
<Schema Name="OracleSchemaName" DatabaseName="" />
</Schemas>
</Biml>
Since i am new to this technical stack, I am struggling to get a solution for this issue. Any input will be very helpful.
I'd think you just need the same DatabaseName in Databases and Schemas.
<Databases>
<Database Name="NameForTheDatabase" ConnectionName="Target" />
</Databases>
<Schemas>
<Schema Name="OracleSchemaName" DatabaseName="NameForTheDatabase" />
</Schemas>
Our project structure regarding the logback.xmls looks like this:
src\main\resources\logback.xml
src\main\resources\config\dev\logback.xml
src\main\resources\config\sjngm\dev\logback.xml
src\main\resources\config\int\logback.xml
src\main\resources\config\local\logback.xml
src\main\resources\config\prod\logback.xml
where the first one references to the environment specific one:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="30 seconds">
<contextName>sjngm</contextName>
<jmxConfigurator />
<include resource="config/${extra}${env:-local}/logback.xml" />
</configuration>
Note that extra is not defined most of the times, which is why used this for a while:
<include resource="config/${extra:-}${env:-local}/logback.xml" />
This stopped working at one point, can't remember which version of logback. So we changed it to
<include resource="config/${extra:-./}${env:-local}/logback.xml" />
which also worked for quite a while.
Now we switched to Spring Boot 1.5.4 (contains logback-classic 1.1.11 and logback-core 1.1.11) and it stopped working again. The latest error message is:
11:08:15,020 |-WARN in ch.qos.logback.core.joran.action.IncludeAction - Could not find resource corresponding to [config/./local/logback.xml]
If I go back to
<include resource="config/${extra:-}${env:-local}/logback.xml" />
the message is
11:19:28,778 |-WARN in ch.qos.logback.core.joran.action.IncludeAction - Could not find resource corresponding to [config/extra_IS_UNDEFINEDlocal/logback.xml]
Note that logback still uses "local" as a default string for env, so not all is broken.
What do I do now? Basically I want to tell logback that I want an empty string where extra would be.
This also doesn't work:
<property name="defaultExtra" value="" />
<include resource="config/${extra:-${defaultExtra}}${env:-local}/logback.xml" />
as an empty string seems to always result in an undefined property.
The only working thing I can come up with is this:
<if condition='isDefined("extra")'>
<then>
<include resource="config/${extra}${env:-local}/logback.xml" />
</then>
<else>
<include resource="config/${env:-local}/logback.xml" />
</else>
</if>
plus this into the pom.xml:
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
</dependency>
Isn't this nice?! So why did they have to break what was working nicely???
This worked for me:
<property name="extra" value="${logback.myApp.extra:- }" />
Logback seems to trim Whitespace out of the value. So the default value of Space did the trick.
Embedded Whitespace is preserved, which may lead to a FileNotFoundException if Tabs were embedded, but embedded Spaces were ok.
Setting a Property in a Java Initialiser had the desired effect:
System.setProperty("logback.myApp.extra", "\t \tEXTRA_EXTRA_EXTRA\t \t");
The Tabs & Spaces were removed from the Property, which was assigned the value EXTRA_EXTRA_EXTRA
(the Java Initialiser must be invoked before any Logging has taken place & may contain no Logging itself)
You could of course set the Property on the Java Command line.
P.S. if the Property is undefined & you omit the Space (${logback.myApp.extra:-}), it is assigned the value:
logback.myApp.extra_IS_UNDEFINED
...so it may be wise to add a suitable comment:
<property name="extra" value="${logback.myApp.extra:- }" /><!-- N.B. Empty Default value must contain # least 1 Space!! -->
I have recently spent several weeks trying to get WildFly to successfully connect to a Kerberized Apache Phoenix data source. There is a surprisingly limited amount of documentation on how to do this, but now that I have cracked it, I'm sharing.
Environment:
WildFly 9+. An equivalent JBoss version should also work (but untested). WildFly 8 does not contain the required org.jboss.security.negotiation.KerberosLoginModule class (but you can hack it, see Kerberos sql server datasource in Wildfly 8.2). I used WildFly 10.1.0.Final, and used a standalone deployment.
Apache Phoenix 4.2.0.2.2.4.10. I have not tested any other version.
Kerberos v5. My KDC is running on Windows Active Directory, but this should not make a noticable difference.
My Hadoop environment is a HortonWorks version, and maintained by Ambari. Ambari ensures that all of the configuration files and Kerberos implementation settings are correct.
Firstly, you'll want to add a system property to WildFly's standalone.xml to specify the location of the Kerberos configuration file:
...
</extensions>
<system-properties>
<property name="java.security.krb5.conf" value="/path/to/krb5.conf"/>
</system-properties>
...
I'm not going to go into the format of the krb5.conf file here, as it is dependent on your own implementation of Kerberos. What is important is that it contains the default realm and network location of the KDC. On Linux you can normally find it at /etc/krb5.conf or /etc/security/krb5.conf. If you're running WildFly on Windows, then make sure you use forward-slashes in your path, e.g. "C:/Source/krb5.conf"
Secondly, add two new security domains to standalone.xml - one called "Client" which is used by ZooKeeper, and another called "host", which is used by WildFly. Do not ask me why (it caused me so much pain) but the name of the "Client" security domain must match that defined in Zookeeper's JAAS client configuration file on the server. If you've set up with Ambari, "Client" is the default name. Also note that you cannot simply provide a jaas.config file as a system property, you must define it here:
<security-domain name="Client" cache-type="default">
<login-module code="com.sun.security.auth.module.Krb5LoginModule" flag="required">
<module-option name="useTicketCache" value="true"/>
<module-option name="debug" value="true"/>
</login-module>
</security-domain>
<security-domain name="host" cache-type="default">
<login-module code="org.jboss.security.negotiation.KerberosLoginModule" flag="required" module="org.jboss.security.negotiation">
<module-option name="useTicketCache" value="true"/>
<module-option name="debug" value="true"/>
<module-option name="refreshKrb5Config" value="true"/>
<module-option name="addGSSCredential" value="true"/>
</login-module>
</security-domain>
The module options will vary depending on your implementation. I'm getting my tickets from the default Java ticket cache, which is defined in the java.security file of your JRE, but you can supply a keytab here if you want. Note that setting storeKey to true broke my implementation. Check the Java documentation for all of the options. Note that each security domain uses a different login module: this is not by accident - Phoenix does not know how to use the org.jboss... version.
Now you need to provide WildFly with the org.apache.phoenix.jdbc.PhoenixDriver class in phoenix-<version>-client.jar. Create the following directory tree under the WildFly directory:
/modules/system/layers/base/org/apache/phoenix/main/
In the main directory, paste the phoenix--client.jar which you can find on the server (e.g. /usr/hdp/<version>/phoenix/client/bin) and create a module.xml file:
<?xml version="1.0" ?>
<module xmlns="urn:jboss:module:1.1" name="org.apache.phoenix">
<resources>
<resource-root path="phoenix-<version>-client.jar">
<filter>
<exclude-set>
<path name="javax" />
<path name="org/xml" />
<path name="org/w3c/dom" />
<path name="org/w3c/sax" />
<path name="javax/xml/parsers" />
<path name="com/sun/org/apache/xerces/internal/jaxp" />
<path name="org/apache/xerces/jaxp" />
<path name="com/sun/jersey/core/impl/provider/xml" />
</exclude-set>
</filter>
</resource-root>
<resource-root path=".">
</resource-root>
</resources>
<dependencies>
<module name="javax.api"/>
<module name="sun.jdk"/>
<module name="org.apache.log4j"/>
<module name="javax.transaction.api"/>
<module name="org.apache.commons.logging"/>
</dependencies>
</module>
You also need to paste the hbase-site.xml and core-site.xml from the server into the main directory. These are typically located in /usr/hdp/<version>/hbase/conf and /usr/hdp/<version>/hadoop/conf. If you don't add these, you will get a lot of unhelpful ZooKeeper getMaster errors! If you want the driver to log to the same place as WildFly, then you should also create a log4j.xml file in the main directory. You can find an example elsewhere on the web. The <resource-root path="."></resource-root> element is what adds those xml files to the classpath when deployed by WildFly.
Finally, add a new datasource and driver in the <subsystem xmlns="urn:jboss:domain:datasources:2.0"> section. You can do this with the CLI or by directly editing standalone.xml, I did the latter:
<datasource jndi-name="java:jboss/datasources/PhoenixDS" pool-name="PhoenixDS" enabled="true" use-java-context="true">
<connection-url>jdbc:phoenix:first.quorumserver.fqdn,second.quorumserver.fqdn:2181/hbase-secure</connection-url>
<connection-property name="phoenix.connection.autoCommit">true</connection-property>
<driver>phoenix</driver>
<validation>
<check-valid-connection-sql>SELECT 1 FROM SYSTEM.CATALOG LIMIT 1</check-valid-connection-sql>
</validation>
<security>
<security-domain>host</security-domain>
</security>
</datasource>
<drivers>
<driver name="phoenix" module="org.apache.phoenix">
<xa-datasource-class>org.apache.phoenix.jdbc.PhoenixDriver</xa-datasource-class>
</driver>
</drivers>
It's important that you replace first.quorumserver.fqdn,second.quorumserver.fqdn with the correct ZooKeeper quorum string for your environment. You can find this in hbase-site.xml in the HBase configuration directory: hbase.zookeeper.quorum. You don't need to add Kerberos information to the connection URL string!
tl;dr
Make sure that hbase-site.xml and core-site.xml are in your classpath.
Make sure that you have a <security-domain> with a name that ZooKeeper expects (probably "Client"), that uses the com.sun.security.auth.module.Krb5LoginModule.
The Phoenix connection URL must contain the entire ZooKeeper quorum. You can't miss one server out! Make sure it matches the value in hbase-site.xml.
References:
Using Kerberos for Datasource Authentication
Phoenix data source configuration by Mark S
I currently have collectd 5.4 installed on my linux host and attempting to use percent to trigger a notification when the threshold has been exceeded but it doesn't appear to work. The threshold for CPU does work however. My current config:
LoadPlugin threshold
<Plugin threshold>
<Plugin "cpu">
<Type "cpu">
Instance "idle"
DataSource "value"
WarningMin 10
FailureMin 5
Hits 25
Persist true
</Type>
</Plugin>
<Plugin "df">
Instance "usr"
<Type "df">
Instance "free"
WarningMin 50
FailureMin 55
Percentage true
Persist true
</Type>
</Plugin>
Anyone ever gotten this to work?
Percentage inside threshold blocks only works for values with multiple datasources, which is not the case for the df plugin for collectd 5.x.
Fortunately since 5.4 there is the ValuesPercentage option for the df plugin which will let you achieve your goal using the percent_bytes type.
Here's a full example config that will poll filesystem usage and emit notifications to syslog when there is no free space on any filesystem:
<Plugin syslog>
LogLevel notice
NotifyLevel OKAY
</Plugin>
LoadPlugin df
<Plugin df>
ValuesPercentage true
</Plugin>
LoadPlugin "threshold"
<Plugin "threshold">
<Plugin df>
<Type "percent_bytes">
Instance free
WarningMin 10
FailureMin 1
</Type>
</Plugin>
</Plugin>
Suppose I have a Web.config like such:
<configuration>
<elmah>
...
</elmah>
</configuration>
Is it possible to remove the <elmah> node with config transforms? So far I've tried something like:
<configuration>
<elmah xdt:Transfrom="RemoveAll"/>
</configuration>
Which doesn't work (according to Preview Transform). Althought this type of thing does seem to work on other nodes. Does anyone know how this can be removed?
Thanks
You need to have a xdt:Locator to get the match.
Try using the following:
Debug:
<configuration>
<elmah name="debug" />
</configuration>
Release:
<configuration>
<elmah name="debug" xdt:Locator="Match(name)" xdt:Transform="RemoveAll" />
</configuration>
Or without the need for name matching:
<configuration>
<elmah name="debug" xdt:Locator="XPath(//elmah)" xdt:Transform="RemoveAll" />
</configuration>
or
<configuration>
<elmah name="debug" xdt:Locator="XPath(configuration/elmah)" xdt:Transform="RemoveAll" />
</configuration>
As a note:
Currently the Web.config transforms are only applied during the Web Publish Pipleline (WPP) that is on Publish, not during debug, to enable them during debug check the following link: http://sedodream.com/2010/10/21/ASPNETWebProjectsWebdebugconfigWebreleaseconfig.aspx .
Hope it helps.
You have a typo in your xdt syntax – it should be xdt:Transform, not xdt:Transfrom.