I am exploring how best to access a PostgreSQL/PostGIS DB from NodeJS. All I need is simple SQL SELECT queries. Nothing more complex than:
SELECT *
FROM portal.catalog AS cat
WHERE ST_Intersects(st_geogfromtext('SRID=4326;POLYGON((20 50 ,19 50,19 49,20 50 ))'), cat.gpoly)
LIMIT 5000;
This will be on a windows7 or windows2008 server, running PostgreSQL 9.2/PostGIS 2.0, The traffic will be pretty light (only a few requests per minute).
Some preliminary research I have done has come up with the following potential directions. But I was interested in hearing from others what is working for them (as an easy implementation).
https://github.com/brianc/node-postgres (But I am having trouble building it do to firewall issues), Supposed the "pure" solution is better, but I am having issues there also) https://github.com/brianc/node-postgres-pure
http://www.infoq.com/articles/the_edge_of_net_and_node (And then I guess I would write my own ADO.NET adapter to PostgreSQL)
I have also seen references to ODBC for NodeJS (unclear if this is the way to go)
Is there something like the SQL adapter for NodeJS? http://blogs.msdn.com/b/sqlphp/archive/2012/06/08/introducing-the-microsoft-driver-for-node-js-for-sql-server.aspx
There was also a full blown ORM by EntitySpaces (which went bankrupt). Now a defunct opensource project: https://github.com/EntitySpaces/entityspaces.js
I've used node-postgres in the past, but recently opted for any-db, which has support for PostgreSQL.
Both have worked well, although I prefer any-db, particularly with respect to pooling and transactions. I believe any-db deserves more recognition.
Any-db is layered on top of BrianC's node-postgres.
But I just got the https://github.com/brianc/node-postgres-pure working, and it is a pleasure.
EntitySpaces is the way to go, not defunct at all.
http://download.cnet.com/EntitySpaces-Studio/3000-10250_4-10590953.html?tag=mncol;1
I got BrianC's PostgresPure system working (must have been a dependent module malfunctioning, since I did not do anything special.
Works just great.
See: https://github.com/brianc/node-postgres-pure
Related
I've recently noticed that Couch DB is back in heavy development. One of the more interesting features to have been added in the last iteration (https://blogs.apache.org/couchdb/entry/apache_couchdb_1_5_0) is a NodeJS query server.
As the default Spidermonkey query server is known to be notoriously slow under mid/heavy loads, I was wondering if anyone knows what kind of performance benefit can be achieved with the Nodejs server, and how it might compare to writing views directly in Erlang.
Anyone have any experience with it?
I have no experience, but since CouchDB is built in erlang I feel any other query servers would be slower because of the interface tasking between core and others. Still I think we should have facts to prove it I have read quite
May be this gives you some idea.. and I nerd on NodeJS on this
http://wiki.apache.org/couchdb/Performance#View_generation
I'm interested in the best practice to query the performance state of the mongo cluster (on mongohq) using a ruby script.
I would like to build some ruby script that checks if the mongo is idle (or near idle) and if so, start to do some work (lots of queries and updates) on it.
I suggest instead of writing this yourself to have a look at MMS. MongoHQ supports this for their dedicated database plans. See https://mms.10gen.com/docs/faq for information.
If you really want to do this yourself, you need to call the serverStatus command.
Dru:
Also, there are some additional tools that MongoHQ can make available to you. Please drop the team a note and they can give you some things to try out. But yes, as was stated above, MMS is a good solution as well.
Jason
MongoHQ
Hy,
I'm implementing an IMAP client as a Mac OSX application using MacRuby.
For the sake of offline availability, I wanted to allow fulltext indexing and attribute based indexing of all messages. Attributes include common E-Mail stuff like from:, to:, etc...
This would allow for advanced results sprinkled with faceting, analytic calculations and such.
Now I'm unsure about the choices and good practices when it comes to integrating such a search feature. I have a strong web development background, therefore my intuitive action would be to setup a Solr server and start feeding it with data. This might just work theoretically, as I could write an Agent that manages the solr instance for my application in the background. But to me, this approach seems like an infrastructure hassle.
On the other side, I've read about people using the FTS3 functionality from SQLite. This approach is easily accessible by CoreData. I haven't used SQLite's FTS3 but I don't think it is as powerful as Solr can be.
What is your weapon of choice for a use case like mine?
I'm mainly interested in solutions that are actually in use by Objective-C/Cocoa/MacRuby developers.
In you're going to develop the app with Ruby give a try to picky. It is very simple to use.
There is an Objective-C Lucene port
http://svn.gna.org/viewcvs/etoile/trunk/Etoile/Frameworks/LuceneKit/
I have not used it, but in your situation, I'd at least check it out. In my experience, SQL based full text search can't compete with Lucene, but haven't tried SQLite for this.
EDIT: just noticed the ruby tag -- this started out as port of Lucene
https://github.com/dbalmain/ferret
I don't really plan on using active record or any of the built in database constructs native to CodeIgniter for database access. I have Oracle, SQL Server, and others. I want to use PHP PDO (unless anyone thinks that's bad) because of the universal aspect of it.
I mainly want CI because of some of the built in libraries and MVC. I also like that it is small and easy to work with.
2.x if it matters.
I did see other questions but none exactly about databases.
Thanks.
edit: It's not that I don't think CI and PHP can take it with large websites. This is solely about using multiple databases of varying companies. I have mostly seen MySQL used with it. I know I can use other databases but again, I don't know if it is more trouble than worth or what.
MySQL is the default just because of how widely-adopted it is, especially in the PHP world. Almost everyone has a *AMP stack to work on so it ends up being the main driver used in almost every example out there.
If you're not planning on using the database class, then it really doesn't matter what type of database you are using, just don't load the class. You can still use routing, helpers, libraries, and other CI features.
So yes, I do think it is suitable for your purposes.
CodeIgniter was built with the idea of being the framework closest to native PHP that doesn't tell you what to do. The entire framework is modular and you are not required to use any single component.
Yes, it is absolutely suited to what you are doing. You can plug and play whatever DB driver you want and CI will not complain one bit.
I think CI is more suited for this role than any other of the 'big' frameworks.
I've seen a couple of SQLite performance questions here on Stackoverflow, but the focus was on websites, and I'm considering using this DB in a client-server scenario:
I expect 1-10 clients for one server for now, could go up to 50 or more in the future.
slightly more reads than writes
the DB would sit behind a server process (i.e: not using direct DB access through a network)
Would using SQLite make the app less responsive as opposed to using PostgreSQL? My intuition tells me that it should be ok for these loads, but maybe someone has some practical experience with this kind of scenario.
I did use SQLite for a major client/server product used with ~10 concurrent users and I deeply regret that decision. In my opinion - PostgreSQL is much more suitable for client/server scenarios than SQLite due to its fine locking granularity.
You simply can't get very far when the entire database is locked whenever someone needs to write something ..
I like SQLite very much (I even wrote a commercial utility for comparing SQLite databases - SQLite Compare but I don't think it fits the bill when you have client/server scenarios.
Even SQLite's author says that it should be used as a replacement for custom file formats and not as a full blown database server. I wish I took his advice more seriously..
You didn't mention what operating system and Postgres versions you are using. However, before considering change of database engine, try to do some logging and benchmarking your current database with typical usage, then optimize "heaviest" questions. And maybe your backend processing load makes DB question time irrelevant? As SQLite is a file-based DBMS, concurrent access from multiple processes will degrade performance when client number grows up (edited after comment)
Following question may be helpful: How Scalable is SQLite?
I would confirm to S.Lott's answer.
I dont know how SQLite performs in comparison to PostgreSQL, since I don't know any newer meassurements, but my own experience with SQLite in a rather similar environment is rather good.
The only thing that might cause troubles in my view is that you have rather many writes. But it all depends on the total number per second I would say.
Also your setting to have one server process is optimal for SQLite in my opinion -- so you circumvent its weakness in multi-tasking.