We are switching from Oracle to Hadoop due to slow performance with Oracle DB, built a universe with Cloudera Simba ODBC connections scheduled a report expecting a faster performance compare to Oracle DB but the report took more than 2 hours, took the same query and ran in HUE SQL editor the result got back in less than 2 mins
We tested in DEV, TEST, & PROD, & also tried switching to JDBC connection no difference, we feel its the network's latency issue and opened a Case with SAP
Point to note here that our Hadoop servers and BO servers are in two different locations NCAL and SCAL, we have 3.5 million records to pull
I am looking for some tested advice here on this issue if anyone has already faced such issue
I'm currently using a vps plan at vpsdime.com as my development server. I move a lot and use different computers so didn't want to develop locally.
Soon, I'll be able to launch my webapp (approx 5-10 users to start with). Should I simply install my production app on my same vps server, or would you advise to get another server? Why?
You can safely use the same server. Just make sure everything as separated per environment:
Different Redis database
Different MySQL database
Different Elasticsearch server
Different location to store session data
Different caching location
Different queues (Redis/Beanstalk, ...)
Different AWS bucket
Different ... you get the gist.
It should be straightforward to setup different vhosts with Apache or Nginx.
Just downloaded latest version (2.1.7) of Apache Cassandra from official site.
Then I started the server without any changes on localhost and created table via GettingStarted Guide
I noticed, that all queries to the Cassandra server are very slow.
For example, this trivial query takes about 250ms:
SELECT * FROM users where user_id=1745;
Is it normal performance? I see much better performance for other database systems on the same machine.
May be I should tweak something?
I have:
Intel Core i5 CPU 2.27GHz
8GB RAM
Windows 8.1
Edit1:
Well.. I see something strange.
The trace log looks pretty nice (6ms):
But when I execute this query in DataStax DevStudio, it shows 476ms:
It cannot be network latency, because I use server on localhost.
Setup:
We're using AppFabric 1.1 on Windows 2008 Enterprise Edition VMs.
We setup a cluster with three nodes using SQL server for cluster configuration and also using offloading so SQL server is supposed to do the cluster management by making sure to create the cluster with: New-AFCacheCluster -Offloading true. We then add the three nodes and start the cluster up. All is good.
We then setup a single cache instance, call it "Test", with HA using the -Secondaries 1 option.
Test Scenario:
We then use a test app to put some test data into the cache and access that data and everything is working great. So then we go to the VM host and down the NIC for one of the nodes in the cluster to simulate that node's failure.
Results:
As soon as the NIC is disabled on the one node, when we go to read from the cache we get timeouts instead of a clean failover.
If we go run Get-AFCacheHostStatus on either of the other two hosts that are still up, the first time after the NIC is disabled, this call will take a very long time to return the status of the hosts. Once it finally does return status, it shows the node on which we yanked the NIC as being in UNKNOWN status. Subsequent calls to Get-AFCacheHostStatus will return quickly, but always showing the error message that the one node is unreachable and shows it in the UNKNOWN status.
Ok, so AF itself detects that node is in UNKNOWN status, but the test app is still getting timeouts at this point. Some minutes later, somewhere btwn 5-10mins, the app will eventually start working again with only the two nodes we have left.
Sooo, what's going on here? Are we configuring something incorrectly? Why is the cluster taking so long to recover from this basic kind of failure?
I am planning to deploy my web app to EC2. I have several webserver instances. I have 1 primary database instance. I have 1 failover database instance. I need a strategy to redirect the webservers to the failover database instance IP when the primary database instance fails.
I was hoping I could use an Elastic IP in my connection strings. But, the webservers are not able to access/ping the Elastic IP. I have several brute force ideas to solve the problem. However, I am trying to find the most elegant solution possible.
I am using all .Net and SQL Server. My connection strings are encrypted.
Does anybody have a strategy for failing over a database instance in EC2 using some form of automation or DNS configuration?
Please let me know.
http://alestic.com/2009/06/ec2-elastic-ip-internal
tells you how to use the Elastic IP public DNS.
Haven't used EC2 but surely you need to either:
(a) put your front-end into some custom maintenance mode, that you define, while you switch the IP over; and have the front-end perform required steps to manage potential data integrity and data loss issues related to the previous server going down and the new server coming up when it enters and leaves your custom maintenance mode
OR, for a zero down-time system:
(b) design the system at the object/relational and transaction levels from the ground up to support zero-down-time fail-over. It's not something you can bolt on quicjkly to just any application.
(c) use some database support for automatic failover. I am unaware whether SQL Server support for failover suitable for your application exists or is appropriate here. I suggest adding a "sql-server" tag to the question to start a search for the right audience.
If Elastic IPs don't work (which sounds odd to say the least - shouldn't you talk to EC2 about that), you mayhave to be able to instruct your front-end which new database IP to use at the same time as telling it to go from maintenance mode to normal mode.
If you're willing to shell out a bit of extra money, take a look at Rightscale's tools; they've built custom server images and supporting tools that handle database failover (among many other things). This link explains how to do it with MySQL, so will hopefully show you some principles even though it doesn't use SQL Server.
I always thought there was this possibility in the connnection string
This is taken (but not yet tested) from How to add Failover Partner to a connection string in VB.NET :
If you connect with ADO.NET or the SQL
Native Client to a database that is
being mirrored, your application can
take advantage of the drivers ability
to automatically redirect connections
when a database mirroring failover
occurs. You must specify the initial
principal server and database in the
connection string and the failover
partner server.
Data Source=myServerAddress;Failover Partner=myMirrorServerAddress;
Initial Catalog=myDataBase;Integrated Security=True;
There is ofcourse many other ways to
write the connection string using
database mirroring, this is just one
example pointing out the failover
functionality. You can combine this
with the other connection strings
options available.
To broaden gareth's answer, cloud management softwares usually solve this type of problems. RightScale is one of them, but you can try enStratus or Scalr (disclaimer: I work at Scalr). These tools provide failover solutions like:
Backups: you can schedule automated snapshots of the EBS volume containing the data
Fault-tolerant database: in the event of failure, a slave is promoted master and mounted storage will be switched if the failed master and new master are in the same AZ, or a snapshot taken of the volume
If you want to build your own solution, you could replicate the process detailed below that we use at Scalr:
Is there a slave in the same AZ? If so, promote it, switch EBS
volumes (which are limited to a single AZ), switch any ElasticIP you
might have, reconfigure replication of the remaining slaves.
If not, is there a slave fully replicated in another AZ? If so, promote it,
then do the above.
If there are no slave in same AZ, and no slave fully
replicated in another AZ, then create a snapshot from master's
volume, and use this snapshot to create a new volume in an AZ where a
slave is running. Then do the above.