I am reading the configuration reference for the JanusGraph https://docs.janusgraph.org/0.2.0/config-ref.html and I wonder - in which file should I write those values? Under conf directory of the JanusGraph there is no single common file in which I can write those options. I am so confused and documentation does not specify it!
These configuration options should be put into a .properties configuration file. The JanusGraph distribution zip archive already contains example configuration files for different backends that can be used as a basis, for example for Cassandra with Elasticsearch: conf/janusgraph-cql-es.properties.
How these configuration files are provided to JanusGraph depends on whether you want to use JanusGraph embedded or via JanusGraph Server (which is the recommended approach). If you want to use JanusGraph Server, then you need a section that specifies this configuration file for JanusGraph in the configuration file of the server which is conf/gremlin-server/gremlin-server.yaml by default:
graphs: {
graph: conf/janusgraph-cql-es.properties
}
The chapter Using Configuration of the JanusGraph docs contains more information about how the configuration can be applied.
Related
I am using Kafka connect to sink data into elastic search. Usually, we ignore the empty fields when persisting to the elastic search. Can we do the same using Kafka connect?
Sample input
{"field1":"1","field2":""}
In the elastic index
{"field1":"1"}
In kafka connect there is a term called SMT, single message transformation, it has several types of supported functions, but none of them doing what you wish for, you can however write your on SMT doing that action ,
● Create your JAR file.
● Install the JAR file. Copy your custom SMT JAR file (and any non-Kafka JAR files required by the transformation) into a directory that is under one of the directories listed in the plugin.path property in the Connect worker configuration
refer to further instructions
https://docs.confluent.io/platform/current/connect/transforms/custom.html
I was trying to run spark(1.6.0) application which was using com.databricks.spark.csv jar to load csv file on yarn client mode from eclipse. It was throwing
CSVRelatio$annonfunc$func not found exception. That was resolved by setting
spark.hadoop.yarn.application.classpath
property in SparkConf.
My question is spark.hadoop.yarn.application.classpathproperty was not
listed in any of the spark official documents. So where can I find all such
properties? I know it is silly questions but there are many beginners who
refer official
documents (https://spark.apache.org/docs/1.6.0/configuration.html) and they
are not at all aware about these properties.
There are not listed because there are not Spark properties. spark. prefix is used only, so Spark recognizes, that these should be parsed, and put into org.apache.hadoop.conf.Configuration.
Where to look for a documentation? You should check Hadoop documentation for a corresponding component. For example for YARN: https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
You should also not that Spark has its own classpath related properties including:
spark.jars
spark.packages
spark.driver.extraClassPath / spark.executor.extraClassPath
....
When we update any configuration for a component in the Configurations tab in the Apache Felix Web OSGi Console, where are these configuration settings saved? This is with respect to AEM 6.0 or above.
The manually saved configuration settings are stored in the the crx-quickstart/launchpad/config directory (in your AEM installation folder in the local file system) and, on top of that, as .config files in the Content Respository at /apps/system/config.
For example, if you're looking for the configuration of com.example.MyComponent, you can find it as a text file in crx-quickstart/launchpad/config/com/example/MyComponent.config (in the local file system) and at /apps/system/config/com.example.MyComponent.config inside your Content Repository.
The files at these two locations are updated when you change the settings manually in the OSGi console.
In AEM you can also store configuration in JCR nodes of the type sling:OsgiConfig. These will not be created or updated when you manually save the config but they offer a neat way of managing configuration as content.
When AEM looks for the configuration, the following order of resolution is used:
nodes under /apps/*/config, be it property files or JCR nodes with the node jcr:primaryType of sling:OsgiConfig
nodes with type sling:OsgiConfig under /libs/*/config, these are OOTB definitions for components that come with AEM
.config files from <aem-installation-directory>/crx-quickstart/launchpad/config/ on the local file system.
Depending on the OSGi configuration of Apache Sling JCR Installer, the sling:OsgiConfig nodes can be written back to when you manually change the config. There's also a number of settings affecting the lookup of said configurations so it's a good idea to familiarise oneself with this config.
Please check out the official documentation for more comprehensive information.
If you're trying to get a config stored in a sling:OsgiConfig node to work as expected and you're unsure if another config for the same service has been cached somewhere, here's a sequence of steps that has worked for me consistently in AEM 6.4:
Delete the sling:OsgiConfig node you just created
Go to the Felix console's configMgr, find the relevant service and Delete the configuration.
Recreate the sling:OsgiConfig node (I usually keep those in a CRX package that I can simply reinstall)
The configurations are stored by the felix configadmin bundle inside the felix bundle cache. Not sure where this cache is located in AEM. In Apache Karaf the configs in my case are in:
data/cache/bundle7/date/config
Where bundle id 7 is the id of the configadmin bundle.
in Sling, configuration can be deployed either via a sling:osgiConfig node and via a nt:file node having the configuration values.
When i make some changes in Felix Console in some configuration deployed via sling:osgiConfig node, it gets converted to nt:file format.
Why there are these two different formats for configurations in Sling. is there any significant difference between the two?
I'd say this is mostly for historical reasons, in some cases it's more convenient to provide configurations as hierarchical resources (sling:OsgiConfig) and if the config is coming from a filesystem for example, files are more convenient.
#Shashi sling:osgiConfig changing to nt:file when you make changes in felix console is expected behaviour. This will not cause any issue when you try to read the config value from java class. You will just not be able to edit the run mode config via crxde when it has changed to nt:file as it stores data as binary content.
However there is a way to disable this behaviour, you will have to uncheck "Enable Write Back" at /system/console/configMgr/org.apache.sling.installer.provider.jcr.impl.JcrInstaller as mentioned in this thread.
OSGi config best practices
Does anyone know how to put/load configuration files required for a target in Apache ACE other than configurator? I am using apache configurator to pick .cfg files from conf directry but i always get the error:
*ERROR* Unexpected problem providing configuration xxxxx to service [xxx, id=xx, bundle=xx/osgi-dp:xxx]
Does anyone knows how to fix this?
The configurator bundle was designed to provide configuration via "cfg" files to the ACE server, relay and client. It was not designed for provisioning configuration data to a target. From your question, it looks like you are trying to do that, so...
For doing that, ACE supports the Auto Configuration Service as described in chapter 115 of the Compendium. In short, that specification uses configuration files in an XML format. These files, along with the resource processor, then get provisioned with the end result that the configuration data is sent to the Configuration Admin service automatically. More documentation about that process, and using "template" configuration files (in case you need to configure many targets with only a few differences between them) is described here:
http://ace.apache.org/user-doc/user-guide.html#using-the-template-engine-for-targets