Which method do you suggest to efficiently secure your embedded database in Delphi applications?
Here are the methods I've tested:
Using Molebox Pro
Pros - Fast, unpacking is not child's play, no additional task/coding
Cons - Database will be read-only, same drawbacks as exe compressors
Using DISQLite3
Pros - Overcome Molebox's read-only issue
Cons - 50% or more performance fall on encrypting
So I'd like to know if you have used anything like this in your projects and if you satisfied with speed and encryption etc. Please share your techniques.
The fact that Molebox Pro leaves your DB read-only while DISQLite3 does not seems to be the deciding factor. Likewise if the performance penalty on encrypting is the only con for DISQLite3, then it is irrelevant since Molebox Pro is read-only (thus no encrypting during operation). It really comes down to your requirements.
If you are looking for other options then I would suggest checking out ElevateDB or DBISAM from ElevateSoft. They are both embedded databases with built in encryption. I've used DBISAM, but ElevateDB is their newer and preferred database. Also check out Advantage DB from Sybase, which is less embedded but also has encryption.
If you have other requirements that may impact your choice let me know!
Related
Are there any performance benchmarks between the managed and unmanaged Oracle ODP.Net drivers?
(i.e. is there any advantage to moving to the managed driver other than for architectural/deployment simplicity)
I would like to share some results. I think that the small lack of performance is worth compared to the easiness of deployment.
Note: seg means seconds. Sorry about that.
Of course that it is a simple test, and there are several topics that is not covered like connection pool, stability, reliability and so on...
It is important to mention, that the scenarios were executed 100 times. So the time quantities are the average of that 100 executions.
Bullets from the quick start video:
Fewer files (1 or 2 dlls at most)
Smaller footprint (10 MB compared to 200 MB)
Easier side by side deployment
Same assembly for 32 and 64 bit (except for second MTS assembly).
Code Access Security
I'm not sure about performance but I doubt it will be much different either way. My guess is that the two drivers communicate in an identical way over "Oracle Net." While there might be minor differences in the in-memory client side operations done to prepare a command and process the results, this overhead typically only represents a fraction of the time relative to the entire transaction. Most of the cost/time is spent on the server in physical IO and transferring the data back to the client. This simply isn't the same as going from the oledb provider or the System.DataAccess.OracleClient driver. This is another release from the same RDBMS company - they're going to exploit all the same performance tricks that their other client used. I wish I could post a study, but i'd guess such a thing doesn't exist because in the end it would be unremarkable. A case of no news is good news - if the new provider was somehow worse you would be reading about it.
Simplicity is enough reason to switch to this IMO. The vast majority of developers and administrators do not fully understand the provider and its relationship to the unmanaged client. Confusion about oracle home preference, version mismatch, upgrades, etc comes up constantly. To eliminate these questions would be a welcome change.
Here is a gotcha for all you folks. Took me a couple weeks to figure out why Oracle Managed drivers would not connect using ef6. If your database has the following data integrity algorithms then you MUST use the unmanaged drivers!!
buried deep in the oracle documentation!!! THANKS ORACLE!!!!!
The easier deployment and bitness independence are really nice benefits, but you should rather evaluate your typical driver usage thoroughly. I faced almost 50% performance handicap when using the new managed driver in 64bit processes. Other people are reporting memory leaks etc on Oracle forum: https://forums.oracle.com/community/developer/english/oracle_database/windows_and_.net/odp.net . It looks like it's kind of typical Oracle buggy product which needs some more months/years to settle back :/
Keep in mind that Custom Types are not supported yet. This could be a reason not to switch to the managed driver.
See this Oracle doc for the differences between the managed and unmanaged version:
http://docs.oracle.com/cd/E16655_01/win.121/e17732/intro004.htm
I am just wondering if there are any advantages and disadvantages such as performance when choosing between NSXMLStoreType and NSSqliteStoreType for Core Data persistent type?
I searched around, and only found NSXMLStoreType is not supported in iOS.
Not supported in OS X? That's news to me (and to the apps in which I used the type for development).
NSXMLStoreType is GREAT for development purposes because you can actually look into the persistent store with a text editor (XML == human-readable) while SQLite or Binary store types are not. SQLite, however, is fastest because it supports atomic updates (update individual records without writing the entire store back out for one little change). XMLStoreType, by contrast trades this speed for readability.
I use XML type for debugging but SQLite in deployed apps. Well, those in which I still use Core Data, that is. CD isn't a good fit for everything. In fact, it's a so-so or poor fit for many when a custom-designed system works just fine (especially where filtering or property setting speed is important).
We have a server solution written entirely in unmanaged Visual C++. It contains complicated methods for really heavy data processing.
The whole thing contains millions lines of code, so rewritning it all in some other language is not an option. We could write some extra code or make isolated changes, but rewriting everything is out of the question.
Now we'd like to put it on a cloud. Which platform do we choose - Amazon EC2 or Windows Azure - and why?
Does it require Administrative rights on the box (e.g. writing to registry, changing box configuration, installing components, etc)? If it does, you can't use Windows Azure today.
If it doesn't require admin privileges then the other things you need to think about are:
What is the architecture? How does it interact with the world? Files? databases?
What dependencies you have?
What is the usage pattern (burst? continuous?)
What would be the cost based on usage and the pricing of both offerings?
That would hopefully give you some more datapoints to help you make a decision.
That depends largely on how you think about costs, future value of the platforms, etc... Azure is still in its very early stages. There are definitely more people using EC2 today. I would recommend computing the costs between the two platforms as a starting point given your estimated usage. Do you want to use features that one platform has over the other? How does your app benchmark between the two platforms? Do you want to take advantage of spot pricing?
In either case I would recommend adding a thin shim layer to abstract you from whichever you choose and enable you to move in the future if you need to.
This is like windows vs. linux....there are no universal right answers, only opinions.
I was looking for ETL tool and on google found lot about Pentaho Kettle.
I also need a Data Analyzer to run on Star Schema so that business user can play around and generate any kind of report or matrix. Again PentaHo Analyzer is looking good.
Other part of the application will be developed in java and the application should be database agnostic.
Is Pentaho good enough or there are other tools I should check.
Pentaho seems to be pretty solid, offering the whole suite of BI tools, with improved integration reportedly on the way. But...the chances are that companies wanting to go the open source route for their BI solution are also most likely to end up using open source database technology...and in that sense "database agnostic" can easily be a double-edged sword. For instance, you can develop a cube in Microsoft's Analysis Services in the comfortable knowledge that whatver MDX/XMLA your cube sends to the database will be intrepeted consistently, holding very little in the way of nasty surprises.
Compare that to the Pentaho stack, which will typically end interacting with Postgresql or Mysql. I can't vouch for how Postgresql performs in the OLAP realm, but I do know from experience that Mysql - for all its undoubted strengths - has "issues" with the types of SQL that typically crops up all over the place in an OLAP solution (you can't get far in a cube without using GROUP BY or COUNT DISTINCT). So part of what you save in licence costs will almost certainly be used to solve issues arising from the fact the Pentaho doesn't always know which database it is talking to - robbing Peter to (at least partially) pay Paul, so to speak.
Unfortunately, more info is needed. For example:
will you need to exchange data with well-known apps (Oracle Financials, Remedy, etc)? If so, you can save a ton of time & money with an ETL solution that has support for that interface already built-in.
what database products (and versions) and file types do you need to talk to?
do you need to support querying of web-services?
do you need near real-time trickling of data?
do you need rule-level auditing & counts for accounting for every single row
do you need delta processing?
what kinds of machines do you need this to run on? linux? windows? mainframe?
what kind of version control, testing and build processes will this tool have to comply with?
what kind of performance & scalability do you need?
do you mind if the database ends up driving the transformations?
do you need this to run in userspace?
do you need to run parts of it on various networks disconnected from the rest? (not uncommon for extract processes)
how many interfaces and of what complexity do you need to support?
You can spend a lot of time deploying and learning an ETL tool - only to discover that it really doesn't meet your needs very well. You're best off taking a couple of hours to figure that out first.
I've used Talend before with some success. You create your translation by chaining operations together in a graphical designer. There were definitely some WTF's and it was difficult to deal with multi-line records, but it worked well otherwise.
Talend also generates Java and you can access the ETL processes remotely. The tool is also free, although they provide enterprise training and support.
There are lots of choices. Look at BIRT, Talend and Pentaho, if you want free tools. If you want much more robustness, look at Tableau and BIRT Analytics.
Would I expect to see any performance gain by building my native C++ Client and Server into 64 bit code?
What sort of applications benefit from having a 64 bit specific build?
I'd imagine anything that makes extensive use of long would benefit, or any application that needs a huge amount of memory (i.e. more than 2Gb), but I'm not sure what else.
Architectural benefits of Intel x64 vs. x86
larger address space
a richer register set
can link against external libraries or load plugins that are 64-bit
Architectural downside of x64 mode
all pointers (and thus many instructions too) take up 2x the memory, cutting the effective processor cache size in half in the worst case
cannot link against external libraries or load plugins that are 32-bit
In applications I've written, I've sometimes seen big speedups (30%) and sometimes seen big slowdowns (> 2x) when switching to 64-bit. The big speedups have happened in number crunching / video processing applications where I was register-bound.
The only big slowdown I've seen in my own code when converting to 64-bit is from a massive pointer-chasing application where one compiler made some really bad "optimizations". Another compiler generated code where the performance difference was negligible.
Benefit of porting now
Writing 64-bit-compatible code isn't that hard 99% of the time, once you know what to watch out for. Mostly, it boils down to using size_t and ptrdiff_t instead of int when referring to memory addresses (I'm assuming C/C++ code here). It can be a pain to convert a lot of code that wasn't written to be 64-bit-aware.
Even if it doesn't make sense to make a 64-bit build for your application (it probably doesn't), it's worth the time to learn what it would take to make the build so that at least all new code and future refactorings will be 64-bit-compatible.
Before working too hard on figuring out whether there is a technical case for the 64-bit build, you must verify that there is a business case. Are your customers asking for such a build? Will it give you a definitive leg up in competition with other vendors? What is the cost for creating such a build and what business costs will be incurred by adding another item to your accounting, sales and marketing processes?
While I recognize that you need to understand the potential for performance improvements before you can get a handle on competitive advantages, I'd strongly suggest that you approach the problem from the big picture perspective. If you are a small or solo business, you owe it to yourself to do the appropriate due diligence. If you work for a larger organization, your superiors will greatly appreciate the effort you put into thinking about these questions (or will consider the whole issue just geeky excess if you seem unprepared to answer them).
With all of that said, my overall technical response would be that the vast majority of user-facing apps will see no advantage from a 64-bit build. Think about it: how much of the performance problems in your current app comes from being processor-bound (or RAM-access bound)? Is there a performance problem in your current app? (If not, you probably shouldn't be asking this question.)
If it is a Client/Server app, my bet is that network latency contributes far more to the performance on the client side (especially if your queries typically return a lot of data). Assuming that this is a database app, how much of your performance profile is due to disk latency times on the server? If you think about the entire constellation of factors that affect performance, you'll get a better handle on whether your specific app would benefit from a 64-bit upgrade and, if so, whether you need to upgrade both sides or whether all of your benefit would derive just from the server-side upgrade.
Not much else, really. Though writing a 64-bit app can have some advantages to you, as the programmer, in some cases. A simplistic example is an application whose primary focus is interacting with the registry. As a 32-bit process, your app would not have access to large swaths of the registry on 64-bit systems.
Continuing #mdbritt's comment, building for 64-bit makes far more sense [currently] if it's a server build, or if you're distributing to Linux users.
It seems that far more Windows workstations are still 32-bit, and there may not be a large customer base for a new build.
On the other hand, many server installs are 64-bit now: RHEL, Windows, SLES, etc. NOT building for them would be cutting-off a lot of potential usage, in my opinion.
Desktop Linux users are also likely to be running the 64-bit versions of their favorite distro (most likely Ubuntu, SuSE, or Fedora).
The main obvious benefit of building for 64-bit, however, is that you get around the 3GB barrier for memory usage.
According to this web page you benefit most from the extra general-purpose registers with 64 bit CPU if you use a lot of and/or deep loops.
You can expect gain thanks to the additional registers and the new passing parameters convention (which are really linked to the additional registers).