Does Neo4j spatial server plugin 3.0.2 work with Neo4j 3.0.3 community? - maven

I've been struggling to install Neo4j spatial for quite some time now using several methods I've found on the Web.
I am using neo4j-community-3.0.3, but I don't remember the link I got it from. It was a pre-compiled version from an ftp site (I believe it was an /archives folder somewhere on neo4j's website but I can't find it in google for the life of me). If someone has a link for downloading precompiled versions of neo4j that would be greatly appreciated. The neo4j other-releases webpage only provides recent versions: https://neo4j.com/download/other-releases/
I tried compiling my own version of neo4j from github but to be honest it is very confusing, as the directory tree is extremely dense. It seems like both community and enterprise versions are included in the same repo, without READMEs, so I don't even know where to begin.
As far as the plugin goes, I have tried both the precompiled version, and compiling my own. For the precompiled, I have followed instructions on the git page to the tee.
https://github.com/neo4j-contrib/spatial#using-the-neo4j-spatial-server-plugin
I downloaded the jar file, and copied it over to $NEO4J_HOME/plugins/
Then I restarted the neo4j server. Finally I make the rest call to see if the plugin has been loaded, but I do not see it.
$ http :7474/db/data/ -a neo4j
http: password for neo4j#localhost:7474:
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Length: 795
Content-Type: application/json; charset=UTF-8
Date: Fri, 01 Jul 2016 19:49:44 GMT
Server: Jetty(9.2.9.v20150224)
{
"batch": "http://localhost:7474/db/data/batch",
"constraints": "http://localhost:7474/db/data/schema/constraint",
"cypher": "http://localhost:7474/db/data/cypher",
"extensions": {},
"extensions_info": "http://localhost:7474/db/data/ext",
"indexes": "http://localhost:7474/db/data/schema/index",
"neo4j_version": "3.0.3",
"node": "http://localhost:7474/db/data/node",
"node_index": "http://localhost:7474/db/data/index/node",
"node_labels": "http://localhost:7474/db/data/labels",
"relationship": "http://localhost:7474/db/data/relationship",
"relationship_index": "http://localhost:7474/db/data/index/relationship",
"relationship_types": "http://localhost:7474/db/data/relationship/types",
"transaction": "http://localhost:7474/db/data/transaction"
}
The compiled version gave me the same result, only it takes longer to achieve. I cloned the git repo for version 3.0.2, and run the following:
git clone git://github.com/neo4j/spatial.git spatial
cd spatial
mvn clean package -Dmaven.test.skip=true install
Note: This mvn command actually failed for me at one point, but after some googling I found that this command worked
mvn clean compile package assembly:single -Dmaven.test.skip=true install
Finally I run
cp target/neo4j-spatial-0.17-neo4j-3.0.2-server-plugin.jar $NEO4J_HOME/plugins
$NEO4J_HOME/bin/neo4j restart
And voila, the exact same results as before (no plugin listing).
I have never had so much trouble installing something. I really do not want to go back to versions 2.* because I want to take advantage of the new bolt driver with python, and get the latest and greatest performance. Please, any help is greatly appreciated. (Even just finding an archive of direct links to precompiled versions of neo4j would help me).

Okay so I figured out several issues which were probably the cause to my confusion.
Issue 1:
If you start the server as root (sudo), you must stop the server as root!
Issue 2:
Make sure you do not have another version simultaneously running (with default port 7474).
I believe a combination of these 2 issues were the real culprit to my problem. It would be great if Neo4J had some sort of check upon startup whether or not that port is already being used.
Also, it is very confusing when attempting to stop the service as someone other than the original user, Neo4J shows the following output:
$ sudo bin/neo4j start
Starting Neo4j.
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
Started neo4j (pid 25418). By default, it is available at http://localhost:7474/
There may be a short delay until the server is ready.
See /opt/neo4j/neo4j-community-3.0.2/logs/neo4j.log for current status.
$ bin/neo4j stop
Neo4j not running
rm: remove write-protected regular file ‘/opt/neo4j/neo4j-community-3.0.2/run/neo4j.pid’? ^C
That last line caught my attention and then after running
$ ps aux | grep neo
I found that Neo4j WAS actually running.
As far as the download link goes, credit to William in the comments above.
He pointed me to http://dist.neo4j.org/neo4j-community-3.0.2-unix.tar.gz,
I suppose one can just change the version number in the url if they want other ones.
So figuring this out, I found that the 3.0.2 spatial plugin indeed does show up in the response from http://localhost:7474/db/data/ for neo4j version 3.0.3. However, I am going to stick to using neo4j version 3.0.2 just to be safe for now.

Related

Ambari server installation on centos7.4 get repos error

I'm trying to get my own cluster with Ambari following this official guide https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide#space-menu-link-content. But I cannot copy repository on this command wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.1.0/ambari.repo
I am using centos7.4
As a result I have problem 403:Forbidden. I also tried to get this page via browser but I got an error.
I also tried to use one of this repos https://docs.cloudera.com/HDPDocuments/Ambari-2.2.0.0/bk_Installing_HDP_AMB/content/_ambari_repositories.html or similar but there's still nothing.
Could you help me guys?)
"Effective Jan 31, 2021, all Cloudera software requires a subscription and must be accessed via the paywall", please check https://www.cloudera.com/downloads.html

Open Source Greenplum: GPFDIST error 'Segmentation fault' when selecting from external table

I'm trying to simply setup an Open Source Greenplum instance and have been hitting the same issue regarding GPFDIST for days! Simply put, I do a full installation from scratch on CentOS 7.6 (can provide further details regarding setup if needed) installing OS GPDB software version 5.18 with GPORCA disabled. Full command for the compile is:
./configure --prefix=/usr/local/gpdb --with-perl --with-python --with-libxml --with-gssapi --with-includes=/usr/local/gpdb/include --with-libs=/usr/local/gpdb/lib --disable-orca
This compiles successfully, and the following make/make install commands too complete without issue. The initialisation of the Greenplum database itself also succeeds, and I can then go into a database and create tables, insert data and run queries like normal.
But if I try to select from an external table, such as the following:
create external table test_external_table
(testing smallint
)
location ('gpfdist://mdw:8080/test_data.csv')
format 'csv' (header delimiter '|')
;
with GPFDIST run as follows:
gpfdist -d /home/gpadmin/test/ -p 8080 -l /home/gpadmin/greenplum/logs/gpfdist_log 2>&1 &
then I get two errors; one from the external table, and one from GPFDIST. These are as follows:
External Table Returns:
ERROR: connection with gpfdist failed for gpfdist://mdw:8080/test_data.csv. effective url: http://127.0.0.1:8080/test_data.csv. error code = 104 (Connection reset by peer) (seg0 slice1 127.0.0.1:6000 pid=27962)
GPFDIST Returns:
[1]+ Segmentation fault gpfdist -d /home/gpadmin/test -p 8080 -l /home/gpadmin/greenplum/logs/gpfdist_log 2>&1
I have removed everything that isn't on the OS GPDB GitHub installation guide (for a 'bare-bones' setup), so I don't think that is causing the issue. I have tried everything to do with the hostname and network firewall, and it's all perfect as far as I can see.
I have also downloaded the same version of GPDB (5.18) from Pivotal and installed that version on the same instance simultaneously, and GPFDIST works perfectly fine.
I have also tried OS GPDB 5.17, 6 beta and 7 beta, and I get the same issue for all of them.
Any ideas at all on what might be causing this is VERY much appreciated, as I'm slowly going insane trying to figure this out now.
Thank you very much in advance for any help.
-- Edit --
Okay.. Having nearly chewed my own arm off in sheer frustration at trying to install debuginfo stuff on CentOS 7, I've finally generated a core dump with gdb. I then run:
gdb -c core_dump.<pid>
and get the following output:
Core was generated by `gpfdist -d /home/gpadmin/test -p 8080 -l /home/gpadmin/greenplum/logs/gpfdist_log'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f4f2c07bdff in ?? ()
But I have absolutely no idea what that means... Totally honest, I'm a little over my head with this now and really am stuck on how to proceed.
The connection reset by peer only indicates that the other end of the socket had dropped (...in this case, gpfdist because it crashed out).
Setup your gpfdist and try a wget to a hosted file adding:
--header='X-GP-PROTO:0'
You will need to add the header to avoid having the request rejected.
Are you able to retrieve a file there? Or does that also crash out?
If that crashes out, it's nothing to do with the database - and you will likely need a core dump to determine what the segfault is about (r/w permissions, memory, ...).
I've finally managed to solve this issue. Should anyone come looking with a similar problem, make sure you are installing Libevent version 1.4[.15], and nothing above that.
I had 2.2.0 installed, and whilst Greenplum sees this as fine, it actually doesn't work for it. Unfortunately, I did have to do an entire system installation from scratch to seemingly get it to work, as just installing Libevent 1.4 on the old system with Greenplum already compiled did not work for me.

two greenplum installation on the same machine

I have an old version of greenplum and I would like to upgrade to version 5.0.0 since it has been released. https://github.com/greenplum-db/gpdb/releases/tag/5.0.0.
I have a huge machine, and I can not simply have an equivalent one. So I would like to know how can I run both version on the same machine. I have seen for example gpseginstall distribute binaries to the /usr/local/gpdb which is already there for the old version.
Regards
I have run multiple versions in parallel on a single node system.
You need to set your config file you use for the gpinitsystem for different segment/mirror directories, master port, starting port, etc..
You will also need two different OS profiles to source, so when you log as gpadmin you can source your 4.3 or 5.0 paths ($GPHOME, $MASTER_DATA_DIRECTORY) for gpstart, gpstop, psql, etc..
Hope this makes sense... I haven't tried it on a multi node system, but the setup should be the same
i.e.
GPDB 4.3
ARRAY_NAME="GPDB"
MACHINE_LIST_FILE=./hostsfile
SEG_PREFIX=seg
PORT_BASE=40000
declare -a DATA_DIRECTORY=(/gpsegment4 /gpsegment4 /gpsegment4 /gpsegment4)
MASTER_HOSTNAME=mdw
MASTER_DIRECTORY=/gpmaster4
MASTER_PORT=5432
TRUSTED_SHELL=ssh
CHECK_POINT_SEGMENTS=8
ENCODING=UNICODE
DATABASE_NAME=gpadmin
#MIRROR_PORT_BASE=50000
REPLICATION_PORT_BASE=41000
#MIRROR_REPLICATION_PORT_BASE=51000
#declare -a MIRROR_DATA_DIRECTORY=(/mirror4 /mirror4 /mirror4 /mirror4)
GPDB 5.0
ARRAY_NAME="GPDB"
MACHINE_LIST_FILE=./hostsfile
SEG_PREFIX=seg
PORT_BASE=60000
declare -a DATA_DIRECTORY=(/gpsegment5 /gpsegment5 /gpsegment5 /gpsegment5)
MASTER_HOSTNAME=mdw
MASTER_DIRECTORY=/gpmaster5
MASTER_PORT=7432
TRUSTED_SHELL=ssh
CHECK_POINT_SEGMENTS=8
ENCODING=UNICODE
DATABASE_NAME=gpadmin
#MIRROR_PORT_BASE=70000
REPLICATION_PORT_BASE=61000
#MIRROR_REPLICATION_PORT_BASE=71000
#declare -a MIRROR_DATA_DIRECTORY=(/mirror5 /mirror5 /mirror5 /mirror5)
I have seen where you can have different versions installed, then change the greenplum-db link to point to the one you want to run. That link is referenced when you enter gpstart. Not sure how you could have two different versions running at the same time on the same machine.
If your goal is to do upgrade dry runs and test on the new release, another alternative could be to deploy a Greenplum cluster using Microsoft Azure. This would deploy the latest version (5.0).
Sounds like you know how to build your own greenplum so you could delete that 5.0 install then install the version you are currently using, then practice the upgrade/migration as well as just kick the tires of 5.0.
You could also easily have side by side systems in Azure; one running your current release and the other running 5.0.
The smallest cluster you can deploy is 1 master and 1 segment which could be adequate depending on your requirements.
Hope this helps

Unable to disable Google Analytics via Spring to address DS-2718 (failed GA connections prevent file downloads) when building DSpace 5.3 with Mirage 2

I'm working on a fresh installation of stock DSpace 5.3 (Windows Server 2012, Tomcat 8.0, Maven 3.2.5, Ant 1.9.6). This particular instance will be a dark archive without Google Analytics enabled; we don't currently have a GA account or analytics key, although we plan to register one eventually for a separate public-facing instance.
As per the problem described in JIRA ticket DS-2718, DSpace hangs with the following message in dspace.log when I attempt to download a bitstream:
2015-10-20 09:52:02,324 INFO org.apache.http.impl.execchain.RetryExec
# I/O exception (java.net.SocketException) caught when processing
request to {s}->https://www.google-analytics.com:443: Network is
unreachable: connect
2015-10-20 09:52:02,324 INFO org.apache.http.impl.execchain.RetryExec
# Retrying request to {s}->https://www.google-analytics.com:443
Since we won't be using GA on this instance, disabling it in Spring is a good workaround until the issue is resolved. As per the instructions, I commented out the Google Analytics entry in dspace-5.3-src-release\dspace-xmlui\src\main\webapp\WEB-INF\spring\applicationContext.xml, disabled Tomcat and rebuilt DSpace. An initial attempt running mvn package -Dmirage2.on=true still produced the problem, so I tried a "ground up" rebuild:
cd d:\dspace-5.3-src-release\dspace
mvn clean package -U -Dmirage2.on=true
[successful build]
cd d:\dspace-5.3-src-release\dspace\target\dspace-installer
ant update
[successful update]
[copy webapps to Tomcat 8.0\webapps and start Tomcat]
Even after the rebuild, however, I'm still getting the same error, with the same java.net.SocketException in dspace.log.
Not sure why this isn't working. Have I missed a step or setting in the rebuild process so that the change to applicationContext.xml isn't being applied?
FWIW, I tried grepping for "google" in dspace-5.3-src-release\dspace-xmlui-mirage2 to see if this could be a Mirage 2 problem, but I don't see anything that looks relevant.
This isn't an answer to why you're still seeing the SocketException, but the real fix for the problem you're describing is to remove the default GA key from dspace-services/src/main/resources/config/dspace-defaults.cfg, see https://github.com/DSpace/DSpace/commit/5b84fef1ad789443d06c338558a92f854b20c8ef. Have you tried doing that?
The issue resolved itself after I ran mvn clean -Dmirage2.on=true in both [dspace-src] and [dspace-src]\dspace. I'm guessing that the issue originated on our end due to someone running a maven build from the wrong directory.
I've also removed the default key from dspace-defaults.cfg as suggested. Everything's now working.

asp.net not running on Apache 2 on OSX Lion - Getting a Forbidden 403 response

As of the current time of writing (2012-03-05) I'm running the following components:
OSX Lion
Default Apache installation (Apache2)
SSLEngine on Apache is "on" (doing some dev, but not trying to access .net through ssl yet)
Mono 2.10.8
xsp 2.10.2
mod_mono 2.10
I'm an Win/IIS guy so this is all new to me, but trying to get the xsp test ASP.NET app running on my mac (the app is in the xsp folder and is referenced in the mod_mono install guide).
I've run through the INSTALL guide for xsp and mod_mono.
As far as I can tell everything is running. Appache seems to have loaded the mono module, as it's ok with the mono configuration elements in it's httpd.cong file.
However, when I try to browse to an ASP.NET page, for example, the xsp test one (http://127.0.0.1/demo/index.aspx), I get:
Forbidden
You don't have permission to access /demo/index.aspx on this server.
Apache/2.2.20 (Unix) mod_ssl/2.2.20 OpenSSL/0.9.8r DAV/2 mod_mono/2.10 Server at 127.0.0.1 Port 80
The mono_mod troubleshooting guides say this might happen if Apache doesn't have read acess, but read access on the xsp test folder is set to:
everyone: Read & Write
I'm thinking it's not a directory permissions issue?
I wonder if the mono mod isn't running properly so actually Apache is trying to do a directory listing, which would give a 403?
What could be causing this? And is there any way to diagnose if all the mono mod stuff is installed and running correctly with Apache?
thanks
A script might also require execute permission, try from the command line:
chmod +x /path/to/script.aspx
This let's OS X (and other unixes) know that the file can be run.

Resources