I'm totally new of neo4j and maven.
I'm using a batch-import tool to convert my mysql to neo4j.
After compiling and converting from .csv, a target file containing graph.db is generated.
Does it mean that I have successfully converted my .csv into neo4j?
What should I do now?
Hope there is someone who can help me.
You have a couple of options to inspect your converted data.
Java REST server
Download and install the Neo4j server. Connect to it via http://localhost:7474. Launch a cypher query to inspect your data. E.g. START n=node(*) RETURN n;.
Java embedded server
You could write a small java application that connects to your data directory. E.g.
public static void main(String[] args) throws Exception {
final GraphDatabaseService graphDb = new GraphDatabaseFactory().newEmbeddedDatabase("db");
final ExecutionEngine engine = new ExecutionEngine(graphDb);
System.out.println(engine.execute("START n=node(*) RETURN n").dumpToString());
graphDb.shutdown();
}
Related
I'm working currently on a Spring boot project, and we have an application.properties file that defines a path where we have 2 jks files. (the original file has more key/value pairs, but I'm focusing in the main problem with these two pairs)
example
file1.example=folder/file1.jks
file12.example=folder/file2.jks
But now, I need to read an environment variable that contains an encoded value that I need to take, decode and use to create these files. The main problem is that I need to create these two files before the properties are "loaded" because these files do not exist, if I first don't build them, any idea?
Normally Spring boot launches from the following main method. You can define a static method before the launcher is called so that in that method you use whatever environment variable you want and make I/O operations to prepare the files. Then if this is done successfully you can let Spring Boot start the server and application normally.
#SpringBootApplication
public class ConfigurationApplication {
public static void main(String[] args) {
createPropertiesFiles(); <---------------
SpringApplication.run(ConfigurationApplication.class, args);
}
private static void createPropertiesFiles(){
// I/O operations to create those files ...
}
}
I would like to set up a simple "Hello, World!" to get an understanding of how to use basic Hadoop functionality such storing/reading files using HDFS.
Is it possible to:
Run an embedded Hadoop as part of my application?
Run an embedded Hadoop as part of my tests?
I would like to put together a minimal Spring Boot set up for this. What is the minimal Spring configuration required for this? There are sufficient examples illustrating how to read/write files using HDFS, but I still haven't been able to work out the what I need as Spring configuration. It's a bit hard to figure out what libraries one really needs, as it seems that the Spring Hadoop examples are out of date. Any help would be much appreciated.
You can easily use Hadoop Filesystem API 1 2 with any local POSIX filesystem without Hadoop cluster.
The Hadoop API is very generic and provides many concrete implementations for different storage systems such as HDFS, S3, Azure Data Lake Store, etc.
You can embed HDFS within your application (i.e run Namenode and Datanodes withing single JVM process), but this is only reasonable for tests.
There is Hadoop Minicluster which you can start from command-line (CLI MiniCluster) 3 or via Java API in your unit-tests with MiniDFSCluster class 4 found in hadoop-minicluster package.
You can start Mini Cluster with Spring by making a separate configuration for it and using it as #ContextConfiguration with your unit tests.
#org.springframework.context.annotation.Configuration
public class MiniClusterConfiguration {
#Bean(name = "temp-folder", initMethod = "create", destroyMethod = "delete")
public TemporaryFolder temporaryFolder() {
return new TemporaryFolder();
}
#Bean
public Configuration configuration(final TemporaryFolder temporaryFolder) {
final Configuration conf = new Configuration();
conf.set(
MiniDFSCluster.HDFS_MINIDFS_BASEDIR,
temporaryFolder.getRoot().getAbsolutePath()
);
return conf;
}
#Bean(destroyMethod = "shutdown")
public MiniDFSCluster cluster(final Configuration conf) throws IOException {
final MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.clusterId(String.valueOf(this.hashCode()))
.build();
cluster.waitClusterUp();
return cluster;
}
#Bean
public FileSystem fileSystem(final MiniDFSCluster cluster) throws IOException {
return cluster.getFileSystem();
}
#Bean
#Primary
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
public Path temp(final FileSystem fs) throws IOException {
final Path path = new Path("/tmp", UUID.randomUUID().toString());
fs.mkdirs(path);
return path;
}
}
You will inject FileSystem and a temporary Path into your tests, and as I've mentioned above, there's no difference from API stand point in either it's a real cluster, mini-cluster, or local filesystem. Note that there is a startup cost of this, so you likely want to annotated your tests with #DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_EACH_TEST_METHOD) in order to prevent cluster restart for each test.
In you want this code to run on Windows you will need a compatibility layer called wintuils 5 (which makes possible to access Windows filesystem in a POSIX way).
You have to point environment variable HADOOP_HOME to it, and depending on version load its shared library
String HADOOP_HOME = System.getenv("HADOOP_HOME");
System.setProperty("hadoop.home.dir", HADOOP_HOME);
System.setProperty("hadoop.tmp.dir", System.getProperty("java.io.tmpdir"));
final String lib = String.format("%s/lib/hadoop.dll", HADOOP_HOME);
System.load(lib);
I have created a JDBCProvider service in an IBM integration bus (IIB v10) in Windows called TESTDDBB, which is also the name of the database. I have a JavaCompute node where I'm trying to generate a connection to call an oracle function
TESTDDBB
connectionUrlFormat='jdbc:oracle:thin:#x.x.x.x:1521:TESTDDBB;'
connectionUrlFormatAttr1=''
connectionUrlFormatAttr2=''
connectionUrlFormatAttr3=''
connectionUrlFormatAttr4=''
connectionUrlFormatAttr5=''
databaseName='TESTDDBB'
databaseSchemaNames='PROM'
databaseType='Oracle'
databaseVersion='11.2.0.4.0'
description='default_Description'
environmentParms='default_none'
jarsURL='C:\jdbc\lib'
jdbcProviderXASupport='true'
maxConnectionPoolSize='200'
portNumber='1521'
securityIdentity='devCredentials'
serverName='x.x.x.x'
type4DatasourceClassName='oracle.jdbc.xa.client.OracleXADataSource'
type4DriverClassName='oracle.jdbc.OracleDriver'
useDeployedJars='true'
public class GetUserData_JavaCompute extends MbJavaComputeNode {
public void evaluate(MbMessageAssembly inAssembly) throws MbException {
...
Connection conn = getJDBCType4Connection("TESTDDBB",JDBC_TransactionType.MB_TRANSACTION_AUTO);
try(CallableStatement callableStmt = conn.prepareCall("{ ? = call PROM.pkg_prop_2.getUserData(?)}");) {
...
}
...
}
}
The problem is that when IIB tried to get the conenction, it isn't finding the datasource java class
...
com.ibm.broker.jdbctype4.jdbcdbasemgr.JDBCType4Connection#-53d4c850.createXAConnection 'java.lang.ClassNotFoundException: oracle.jdbc.xa.client.OracleXADataSource at java.net.URLClassLoader.findClass(URLClassLoader.java:609) at com.ibm.broker.classloading.JavaResourceClassLoader.findClass(JavaResourceClassLoader.java:181) at com.ibm.broker.classloading.SharedClassLoader.findClass(SharedClassLoader.java:215) at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:925)
...
I have the ojdbc6.jar driver in the folder C:\jdbc\lib and deployed in a shared library in the integration server, library that is referenced by the RESTAPI app that contains the JavaCompute node. What am I missing? I have tried using useDeployedJars true and false, and jarsURL also with 'C:\jdbc\lib\ojdbc6' without success. Where are the common libraries of the integration server in windows?
From one side you can place the jar to root of you java source folder and connect this jar to Integration Toolkit. This give you ability to use this jar. From other side you can configure JDBC provider for Integration node and use created alias in getJDBCType4Connection call. More information about working with databases from JavaCompute you can see here 1
I have a need for a java-cache with file storage, that survives JVM crashes.
Previously I used ehcache, configured with .heap().disk().
However, it has a problem with unclear JVM shutdowns - next startup clears the store.
My only requirement is that at least parts of the data survive a restart.
I tried to use hazelcast, however with following code snippet, even subsequent run of the program returns prints "null".
Please suggest how to configure hazelcast, so that cache.put is written to a disk and loaded on startup.
public class HazelcastTest {
public static void main(String[] args) throws InterruptedException {
System.setProperty("hazelcast.jcache.provider.type", "server");
Config config = new Config();
HotRestartPersistenceConfig hotRestartPersistenceConfig = new HotRestartPersistenceConfig()
.setEnabled(true)
.setBaseDir(new File("cache"))
.setBackupDir(new File("cache/backup"))
.setParallelism(1)
.setClusterDataRecoveryPolicy(HotRestartClusterDataRecoveryPolicy.FULL_RECOVERY_ONLY);
config.setHotRestartPersistenceConfig(hotRestartPersistenceConfig);
HazelcastInstance instance = Hazelcast.newHazelcastInstance(config);
CacheConfig<String, String> cacheConfig = new CacheConfig<>();
cacheConfig.getHotRestartConfig().setEnabled(true);
cacheConfig.getHotRestartConfig().setFsync(true);
CachingProvider cachingProvider = Caching.getCachingProvider();
Cache<String, String> data = cachingProvider.getCacheManager().createCache("data", cacheConfig);
System.out.println(data.get("test"));
data.put("test", "value");
data.close();
instance.shutdown();
}
}
Suggestions for other frameworks that could complete the task are also welcome.
#Igor, Hot Restart is an Enterprise Feature of Hazelcast. You need to use Hazelcast Enterprise edition with a valid License Key.
Do you really need to store in a file, or just persist cache data somewhere else? If you can use a database, you can use MapStore which is available in Open Source version & write data to a persistent data store. You can even use write-behind mode to speed up writes.
See these sample project: https://github.com/hazelcast/hazelcast-code-samples/tree/master/distributed-map/mapstore
i'm using spring and at the moment running my tests create new objects in my "real" embedded database. I want to create a new one or a temporary db just for testing. I'm new with spring and neo4j so could anyone please help?
thanks a lot
If you are using the embedded driver with SDN/OGM you just need to configure it without providing a path. Then it will create embedded database in /tmp/.. which gets deleted on jvm exit.
E.g. if you are using java configuration
#Bean
public Configuration getConfiguration() {
Configuration config = new Configuration();
config
.driverConfiguration()
.setDriverClassName("org.neo4j.ogm.drivers.embedded.driver.EmbeddedDriver");
return config;
}
See docs for full documentation
http://docs.spring.io/spring-data/data-neo4j/docs/current/reference/html/#_configuring_the_embedded_driver