I know that both Memgraph and Neo4j use Cypher as query language. I see that Neo4j has APOC (Awesome Procedures on Cypher). Is there a Memgraph version or replacement for APOC?
I've looked at Memgraph documentation but I couldn't find info on this.
APOC is a custom Neo4j library of stored procedures and functions that depends on Java runtime/API.
Memgraph doesn’t have Java runtime (it will probably never have) so it’s not possible to run APOC procedures; hence there is nothing in the docs.
But there are a few ways to use/approach Memgraph:
Similar to APOC, Memgraph provides a set of the shelf modules/functions called MAGE (Memgraph Advanced Graph Extensions), you can write your own query modules by using MAGE.
There is also a C API, which allows users to write extensions for Memgraph in any programming language, so far Memgraph have a native API for C/C++/Python/Rust so that any developer feels like at home because these native APIs are all specific for a given language and ecosystem.
Memgraph has Python runtime embedded, which allows users to easily integrate Memgraph and any library from the vast Python ecosystem; NetworkX, Pytorch, Pandas just to name a few.
Related
I'm currently working on a setup that uses Google Cloud Dataflow to transform data and save into a Neo4j Database hosted on a Compute Engine VM. The current setup uses a JdbcIO to write to Neo4j by running a prepared statement, but it seems also possible to use a Neo4j driver directly in Java, which allows more flexibility on creating the query dynamically.
I wonder if anyone had compared the 2 approaches and notice any differences? I would guess the Jdbc approach would be more efficient since it only need to run the prepared statement, but that is just my guess.
Update
I'm going to post my findings as I experiment more with the two methods
One difference I found was that when using JdbcIO, I run into a deadlock when multiple transaction is trying to write to the same node at the same time. However that issue did not occur when using Neo4j driver directly.
I am learning about JDBC. When I am learning about it, I have come to known we can use 4 drivers. In each of these architecture I find the vendor database libraries in jdbc-odbc bridge driver and also native API calls in other type 3, 4.
I am unable to know about those concepts in the architecture.
So I request anyone to say about those two concepts which deals with the JDBC drivers.
I have referred the below link :
http://www.javatpoint.com/jdbc-driver
Vendor based database libraries means you need to use libraries provided by vendors to connect to database.
For example
You need to use ojdbc6.jar for Oracle , postgresql-9.4.1207.jar for postgresql etc.
Short answer: database is called a vendor database if you interact with it through JDBC using special drivers.
JDBC is an API, which allows you to work with different databases (such as MySQL, PostgreSQL etc.) using the same methods to query and update data in a database. This happens by using special drivers, which are unique for every database. In this case the database you are working with is called a vendor database.
Progress/OpenEdge is not a SQL database and although it does have support for SQL (through the ODBC interface) that is not the natural way to access OpenEdge data. Progress applications are built using "ABL" (formerly known as "Progress 4gl") and some SQLish things like referential integrity constraints are implemented through procedural logic in 4gl code. Which makes things difficult if you are not really interested in learning ABL.
Has anybody successfully managed to use Apache Drill (https://drill.apache.org) to access Progress data? Are there any pointers particular to the combination of Progress and Apache Drill to share?
Short answer: No, this is impossible.
Longer answer: my admittedly cursory read of what Apache/Drill does lead to the mistaken conclusion that you can just access Progress (or anything else) via ODBC and then use Drill on top of it.
Apache/Drill is exposing itself via ODBC but currently the only datasources it can use are:
HBase
Hive
MapR-DB
Avro
Parquet
File system:
CSV (Comma-Separated-Values)
TSV (Tab-Separated-Values)
PSV (Pipe-Separated-Values)
Therefore at the time being Progress cannot be accessed by Apache/Drill.
We are using liquibase as evolutionary DB change management tool in our applications, it works great when we use it in "common" database schemas.
But we also work with GIS applications using esri arcSDE 9.3 platform over Oracle and in this case, all (or almost all) tables (both GIS and 'alphanumeric' tables) in the schema are managed (create table, grants, etc.) through arcSDE. So when we want to create new feature classe now we use arcCatalog, and this way, it's not possible to manage the feature classes’ changes directly through SQL using liquibase or other automate refactoring tool.
So if we cannot use liquibase to manage changes, at least we want to execute the management operations over our features through command line.
We’ve started looking for tools that avoid the use of arcCatalog, and then try to automate the changes using scripts, we are investigating these possibilities:
Try to capture the SQL that arcCatalog/arcSDE is executing each time we make a change in one Feature Class monitoring the oracle connection. It outcomes us a too complex set of SQL instructions that involves indexes, versioning tables, etc. so we give up this way.
Use the sdelayer and sdetable admin commands installed on arcSDE server.
Use the data management tool: a python based library to manage the feature classes, but it has to be executed from a machine with the desktop version installed.
These last two options will provide a way to manage features from command-line, but our target is to find/develop a tool to manage changes similar to the way liquibase do. But with these tools we will have to find a tool that let us map each SQL DDL operation to an arcSDE command, and currently no db refactoring tool provides this (currently we have check liquibase, dbdeploy, flyway).
Anybody had solved this problem of evolutionary change management with arcSDE? Any insight of another way to face this problem?
I'll take a stab at this, although I'm unfamiliar with one of the products you mention (liquibase specifically - I have used Oracle and I'm very familiar with ArcGIS (ArcMap & ArcCatalog).
Here's just some additional information that may help and my interpretation of your question.
My interpretation - "What's a simple way to manage or enable us to automate the management of our tables of GIS data in our Oracle database without having to use ArcCatalog all the time?"
So - I'll throw this concept back in the ring - I know SQL Server has spatial datatypes "geometry", etc. and that you can bypass SDE and let ArcGIS directly connect and interpret this data without even installing SDE. I also know Oracle has compatible spatial types. So I would possibly consider migrating my data from the managed FC's that ArcCatalog creates and push them into oracle-native geometry based tables. This way you can treat them like regular tables, cut ESRI out of the solution, and manage them with liquibase, etc. Hopefully that helps.
I would also consider upgrading to 10.1 or at least 10.0 (I promise I'm not an undercover salesman), although that will require your users to come with you on the client side (http://resources.arcgis.com/en/help/main/10.1/index.html#//002q000000n8000000) because the newer python APIs are much easier and faster to use (arcpy vs. the GP model), if you do choose to use Python to manage your stuff. (Regardless, either API isn't very well developed and isn't intuitive to code in or fast.)
Good luck.
I'm trying to move snapshots of data from our MongoDB into our Oracle BI data store.
From the BI team I've been asked to make the data available for ODI, but I haven't been able to find an example of that being done.
Is it possible and what do I need to implement it?
If there is a more generic way of getting MongoDB data into Oracle then I'm happy to propose that as well.
Versions
MongoDB: 2.0.1
ODI: 11.1.1.5
Oracle: 11.2g
Edit:
This is something that will be queried once a day, maybe twice but at this stage the BI report granularity is daily
In ODI, under the Topology tab and Physical Architecture sub-tab, you can see all technologies that are supported out of the box. MongoDB is not one of them. There are also no Knowledge Modules available for importing/exporting from/to MongoDB.
ODI supports implementing your own technologies and your own Knowledge Modules.
This manual will get you started with developing your won Knowledge module, and in one of the other manuals i'm sure you can find an explanation on how to implement your own technologies. (Ctrl-F for "Data integrator")
If you're lucky, you might find someone else who has already implemented it. Your best places to look would be The Oracle Technology Network Forum, or a forum related to MongoDB.
Instead of creating a direct link, you could also take an easier workaround. Export the data from the MongoDB to a format that ODI supports, and MongoDB can extract to. CSV or XML maybe? Then load the data trough ODI into the oracle database. I think... that will be the best option, unless you have to do this frequently...
Look at the blog post below for an option;
https://blogs.oracle.com/dataintegration/entry/odi_mongodb_and_a_java
Cheers
David