How to Import Data to MongoDB.Atlas on Windows - windows

I need to import 500K+ small records into a MongoDB Atlas collection. I've no access to linux/mac machines and need to do it on Windows 10.
Is there any way to achieve it?

The mongoimport utility works well on Windows. It can be installed with the Community Edition MSI.
You can import from EJSON, CSV or TSV, depending on your needs. Take a look at the official docs for working examples.
If you have trouble post-install, share a sample source record for additional pointers.

Related

How to read from an EDB database file

Microsoft Edge and other Microsoft products use an Extensible Storage Engine. If you have edge installed on windows, you can find the database file here:
C:\Users\username\AppData\Local\Packages\Microsoft.MicrosoftEdge_xxxxxxx\AC
\MicrosoftEdge\User\Default\DataStore\Data\nouser1\xxxxxxxx\DBStore\spartan.edb
I would like to read this database from my .NET app.
The only tool I have seen for viewing this data appears to be deprecated:
http://www.woanware.co.uk/forensics/esedbviewer.html
I can't seem to find any relevant nuget packages for querying this type of database. Does anyone have experience working with this type of database?
The database engine is esent.dll, and you can access it in several different ways:
C API. https://msdn.microsoft.com/en-us/library/gg269259(v=exchg.10).aspx
C#. https://github.com/microsoft/managedesent
Simplified C# (Isam layer). Easier to use, but not everything is exposed. https://github.com/Microsoft/ManagedEsent/tree/master/isam
(Disclosure: I've worked on the above products.)
That being said, if you just modify a random database, you can impact the host process's integrity, and it might end up crashing.
If you're a law-enforcement agent and it's for forensic purposes, Microsoft should be able to assist you (I haven't done it myself, but I've heard that they'll do it).
-martin

Hadoop Hive web interface options

I've been experimenting with Hive for some data mining activities and would like to make it easily available to less command line orientated colleagues.
Hive does now ship with a web interface (http://wiki.apache.org/hadoop/Hive/HiveWebInterface) but it's very basic at this stage.
My question is does a visually polished and fully featured interface (either desktop or preferably web based) to Hive exist yet? Are their any open source efforts outside the Hive project working on this?
Now with new version of Cloudera's Hadoop Distribution comes HUE (Hadoop User Experience) with plugin called Beeswax, which most likely all you would need.
It's pretty tricky to configure, but one you get over it, it provides something like phpmyadmin interface, but is much nicer and easier. It supports writing queries, importing data, storing results, etc.
Web based opensource GUI for Hive
HWI - Shipped in Hive. with basic features.
Hue - Nice query editor with autocompletion. Support parameterized query. Latest version includes basic visualization of query result. Includes many other useful tools like managing HDFS, JobFlows, etc. Thus, heavy and little bit tricky to install and configure.
Zeppelin - Only includes Hive tool compare to Hue. Support query template. Pluggable visualization architecture and it's online archive, so easily create custom visualization and share it. Lightweight and easier to install than Hue while it does not include any feature for non-hive related things.
Other alternatives
Excel - Microsoft Excel is capable of making hive query and fetch data from hive. http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.1/bk_dataintegration/content/ch_using-hive-2.html has guide for doing it.
Commercial BI tool - Commercial BI tool like Tableau, Datameer, Karmasphere support connection to Hadoop or Hive. They have nice dashboards, charts. All they offer trial/community/personal edition.
HUE is usefull and good but you should also try "Karmasphere Analyst Free/community Edition". It is very easy to use and well documented. Free version is very capable. It is not web based but it has different OS support (windows,linux...etc). YOu can check the GUI from documents to see how it looks.

deploy application with OraOLEDB provider

I developed an application that uses Delphi 7, ADO and ORACLE, the provider I use is OraOLEDB (I need use this provider because the BLOB fields support). now I want to distribute this application with the provider. I search the web to download the Oracle provider, but has a size of 174 mb. I need to install this file on all client machines ? there is a smaller distribution of this file?
UPDATE
i am looking for a lightweigth (small) distribution of the OraOLEDB.
Thanks in advance.
I would opt to let the user supply it's client library himself. You avoid problems where there might me different oracle OleDb versions for different Oracle versions, licensing issues, installation issues etc. etc.
As fas as I know you need the package of 170 MB+.
It seems the distribution license (http://www.oracle.com/technology/software/popup-license/distribution-license.html) from that page allows you to redistribute the OleDB driver as long as you comply fully with that license. That is the official Oracle distribution - usually Oracle setups are large - if you don't want to include them on your distribution media you can simply point them to the download page.
Update: there is a smaller package in the InstanClient download page, look for Instant Client Package - ODAC here: http://www.oracle.com/technology/software/tech/oci/instantclient/htdocs/winsoft.html
I know a little bit about installing OLE DB providers, but I do not know anything about the Oracle provider, so this may not be practical. I see that the 174MB download includes a number of drivers (ODBC, .NET, OLE DB, etc.). I should be possible (but maybe not useful in the real world) to create your own distribution with the necessary files. One very nice thing about OLE DB is that it is typically "simple" to install. Running regsvr32 /i provider.dll will make the necessary registry updates to make it usable on a system. The providers I have used do not require a bucket full of registry hits like ODBC drivers often need.
So it might be possible to create your own distribution package. This site lists the files that are apparently necessary for the provider. I do not know if there are other "generic" files common to all Oracle client kits that might be necessary (that may be the part that would make this idea impractical).
This is the main problem when developing for oracle, except if you use ODAC, which has direct connection to oracle, without installing oracle client on clients' machines.
and it's much faster than using ADO or OLE DB providers.
Distributing Oracle client application can be a nightmare, even more today when you have 64 bit Windows.
Which version of the Client would you have to install? You need a 32 bit version for your Delphi application. But what if other programs do need having access to the 64 bit version? You need several ORACLE_HOME, with duplicated settings, or force you DotNet code to run in 32 bit mode.
I first wrote a Delphi wrapper using OleDB, then I realized how difficult it was to deploy it when using the Oracle DB. The same exact issue as yours...
Then I wrote a dedicated version, calling directly the OCI library. Speed was there (2 to 5 times faster than OleDB), with easy deployment.
You can use the latest version of the Oracle Instant Client provided by Oracle - see this download link - which allows you to run your applications without installing the standard (huge) Oracle client or having an ORACLE_HOME. Just deliver the dll files in the same directory than your application, and it will work.
The drawback of this solution is that it's not compatible with the DB aware components. But if you are using TQuery directly, then map the results in Delphi classes, it could be a great solution.

Simple database application for Windows

I need to build a simple, single user database application for Windows. Main requirements are independence from windows version and installed software. What technologies (language/framework) would you recommend? My preference for language is the Visual Basic.
EDIT: What about VB.Net and SQL Server Compact Edition?
I would recommend Sqlite. It's completely self-contained, and public domain so there are no license issues at all.
Single user or multi user?
For single user, the answer would be SQLite
For multi user (or multithread), try MySQL or PostgreSQL.
Since your requirement is a windows based application i would suggest that you go with sql server 2005 express edition which is a free tool, but with certain small limitations. you can upgrade to a bigger version when you go with a paid version.
There are other DB engines like SQL Lite or FireBird, choose them if the support and growth options they provide are good enough for you
Additionally, Visual Basic is eof lifed. VB.NET might be a better windows based platform currently. It would give a better platform / features to start with and when you want to expand the talent you have working on the project, i assume .NET talent might be more available than programmers who want to work with a dead language.
duplicate of What options are there for a quick embedded DB in .NET?
I'll repeat my answer from there:
"Or theres Esent, the built in database that exists in every copy of windows. Read about it here: http://ayende.com/Blog/archive/2008/12/23/hidden-windows-gems-extensible-storage-engine.aspx" and http://www.codeplex.com/ManagedEsent
SQLite will work for a local desktop application. If you want several users, a few gigas of data, and multiple connections I would use mysql or Firebird.
http://www.mysql.com/
http://www.firebirdsql.org/
FireBird SQL server will be thing of choice. It can be used in both embedded and multiuser mode like traditional databases. It implements many of the SQL standards and has strong community base. It is available for Windows, Linux, Solaris, OS X, HP-UX
As mentioned, SQLite is a great single-user database. This page has VB/SQLite examples. Once concerns is that SQLite parses foreign key constraints, but does not enforce them. You can use this code to generate "foreign key triggers" for SQLite, thus gaining an easy to use database with FK constraints.
Depending on how demanding your database needs are, though, you might want to consider MS Access.
I used SQL Server Compact Edition. It's like sqllite. A single SDF file accessed using ADO.NET.
You can develop your application using Visual Basic .NET and manage you database (add tables, columns, constraints, etc...) using Visual Studio.
SQLite may be what you are looking for. http://www.sqlite.org/
Depending on your needs for the application.
You could use SQLLite which is a very nice database with no installation required.
You could also use Microsoft SQL Server: SQL Server Compact 3.5.
Both are free!
It's not quite clear from your post whether you want a web application or not.
For a web application, MySQL works effectively on the Windows platform. You also have nearly limitless options for development environment including, PHP, Ruby on Rails, Django, and .Net.
If you are looking at a desktop application, MS Access might be suitable ... incredible easy for simple applications.
Well, assuming you don't have any prior experience...
You need some kind of persistence storage (for example a database) and a client.
For the storage you could use almost anything. For example you could create your DB in MS Access and just ship it as a file, using ADO to access it.
Other options are MS SQL Express edition (comes pre-installed on some machines or could be installed for free) and plenty of open source databases like SQLite
For the client side you could not go wrong with VBScript and ADO (using OLE DB drivers). They come with every Windows installation since Dark Ages, you will have plenty of references/tutorials/answers online.
A drawback: no UI to speak of, so you'll have to build a command line interface (which was for a 'simple' application).
If you want to build a UI I would suggest using .NET WinForms. The overhead will be substantially bigger but .NET is now installed on all XP/Vista machines and even if it is not you could always install the framework with you application.
If you want to build application that can move to other pc easily,I prefer Microsoft Access it is small database easy to use and no need to install.It suites for application like Addressbook,mini crud system.
But if you want to develop enterprise database system you should use MySQL instead.
I do not understand what you mean with "independence form [...] installed software". You ever need at least the DBMS installed as well as one client or user interface.
I recommend using MS Access. It is easy and cheap for simple, single user tasks and rapid prototyping development. Only development version have to be bought ("normal" Access) to create DBs. Runtime version of Access 2007 can be downloaded free of cost from Microsoft Homepage - for using only the database you created.
Also it combines DBMS and GUI frontend in same tool.
Dare I mention MS Access...?
If you are looking for small footprint (up to a few MB) and easy deployment (end-user should only install your application to get it working), then your options are SQLite and Firebird embedded.
Of those two, I'd pick Firebird any time, because of it's full support for SQL (you can't, for example, drop a column in SQLite), ACID compliance, and ability to go client/server without any changes (just change the connection string from embedded to server) to the code if you ever decide to let multiple users work on the same database.
Not to mention that you can use full server to develop (which means your application and database administration tool can be connected to database at the same time).
I'm successfully using Turbo Delphi (free for commercial and no commercial use) + ZeosLib (zeos.firmos.at).
The only things you need to distribute with your .exe are the database client dlls (no need to install the client, just put the dlls in the same directory).
Would Kexi work?
I can recommend from personal experience "My Visual database"
free, no code, no sql, just drag and drop.
http://myvisualdatabase.com/
Best Option would be to create a Win32 native application using Delphi and use SQLLite as the database.
Reason being Delphi can produce native win32 applications without any other product being installed on the machine.

Excel report framework

Is there any Excel report framework available? We need to export some of the reports into Excel format. Our application is java application hence anything supporting java would be great. I have tried Apache POI API, however that is not good enough. Any framework based on Windows API would be better.
SQL Server Reporting Services has options to export to Excel.
FYI, JasperReports uses POI for Excel-conversion.
Can you elaborate on what you don't like about Apache POI? I've been using POI for years now and haven't found anything that it couldn't do with a tweak here or there or taking an creative approach. IMHO, it's the best open-source (and free) Excel generation/reporting framework out there.
If you are willing to pay money, then Actuate has probably the best solution. Actuate's e.Spreadsheet Engine and the Excel API, you can read, write, modify and generate entire spreadsheets or parts of spreadsheets. I've used it and their API is richer and simpler then POI. POI, while powerful feels like an API that's grown up over time and has many developers involved in creating functionality and patches.
Try xlsgen, supports Java (but can only run under Windows).
Why is poi not good enough for you?
An alternative might be jasper reports. I've used this instead of poi a couple of times and the experience was pleasant.
You can also try jxl, but honestly it's API is more confusing than POI.
Xylophone is LGPL Java library and command line utility that uses Apache POI, but mitigates most of its drawbacks.
It consumes data in XML format, spreadsheet templates in XLS(X) format and makes producing of complex Excel reports more fun. You can read about it in this post. Because of licensing and security issues this must be better choice for Java backend than Windows API.

Resources