I was wondering whether MySQL commands like NOLOCK or creation of indexes is actually supported on Aurora AWS? Also, can I do performance issue investigation or is Aurora AWS a "black-box" to me?
The Aurora it is completely different database than MySQL, but when it comes to the compability with MySQL or PostgreSQL Amazon team did a lot so that there would not be a lot of differences. "The Amazon Aurora MySQL is designed to be wire-compatible with MySQL 5.6 and 5.7 using the InnoDB storage engine. Certain MySQL features like the MyISAM storage engine are not available with Amazon Aurora." Amazon Aurora MySQL 2.01 does not currently also support the following MySQL 5.7 features:
Global transaction identifiers (GTIDs)
Group replication plugin
Increased page size
InnoDB buffer pool loading at startup
InnoDB full-text parser plugin
Multisource replication
Online buffer pool resizing
Password validation plugin
Query rewrite plugins
Replication filtering
The CREATE TABLESPACE SQL statement
X Protocol
(source:https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.2022.html)
But it changes from release to release.
Answering to you question about a performance monitoring. You can monitor Aurora database from AWS Console or by using queries directly on Aurora database, the same as in case of MySQL. Staring from version 2.03 there is also available the performance schema.
UNLOCK command - I am not sure whether it also exists in native MySQL, but there is possiblity to set transaction isolation level.
Indexes - yes, you can also create indexes on Aurora.
I hope the information will be useful for you.
Related
We are currently setting up AWS hosting for our Web Application.
This Laravel Web Application will have a Schema per company that registers, meaning it will have a large sized mySql server.
I have gone through the motions of setting up a VPC with EC2 instances and and RDS for this mySql server.
However we are currently looking at using Laravel Forge as a tool to host.
What Forge does differently is that it includes the mySql Server on the EC2 instance not on an RDS.
The question I have come to ask here is, what are the implications if any of having the mySql server on the EC2 instance rather then an RDS.
Would there be performance issues?
Is it better practice to have an RDS?
Or is Forges out the box way of packaging this all together on an EC2 server fine?
By running this on an EC2 instance you will taking more of the responsibility of managing the database, not just installation but also patching, backups, recovery. Harder to maintain functionality such as replication and HA will also be on you to implement and monitor.
By running on RDS AWS is going to take the heavy lifting of this and implement a best practice version of MySQL which offers the flexibility of allowing you to run a MySQL stack in the cloud without having to really think about the implementation details under the hood other than deciding do you want it to be HA and how many replicas do you want.
In saying this by using RDS you're also giving up the ability to run it however you want, you are limited to the versions of the database that RDS supports (although this is now quite soon after release). In addition not all plugins or extensions will be active so check this functionality before deciding.
Is there a way to upgrade from Aurora 1 (MySQL 5.6) to Aurora 2 (MySQL 5.7) without downtime on an active database? This seems like a simple task given we should be able to simply do major version upgrades from either the CLI or the Console, but that is not the case.
We tried:
Creating a snapshot of the database
Creating a new cluster using Aurora 2 (MySQL 5.7) from the snapshot
Configure replication to the new cluster from the primary cluster
However, because you can't run commands that require SUPER user privileges in Aurora you're not able to stop transactions long enough to get a good binlog pointer from the master, which results in a ton of SQL errors that are impossible to skip on an active database.
Also, because Aurora is not doing binlog replication to its Read replicas I can't necessarily stop replication to that read replica and get the pointer.
I have seen this semi-related question, but it certainly requires downtime: How to upgrade AWS RDS Aurora MySQL 5.6 to 5.7
UPDATE: AWS just announced in-place upgrade option available for 5.6 > 5.7:
https://aws.amazon.com/about-aws/whats-new/2021/01/amazon-aurora-supports-in-place-upgrades-mysql-5-6-to-5-7/
Simple as Modify and choose version with 2.x. :)
I tested this Aurora MySQL 5.6 > 5.7 on a 25Gb db, many minor versions behind and it took 10 min, with 8 min of downtime. Not zero downtime, but a very easy option, and it can be scheduled in AWS to happen automatically during off-peak times (maintenance window).
Additionally consider RDS Proxy to reduce downtime. During small windows of db unavailable time (eg. reboot for minor updates), the proxy will hold connections open, instead of completely unavailable, simply appearing as a brief delay/latency, only.
Need was to upgrade the AWS RDS Aurora MySQL from 5.6 to 5.7 without causing any downtime to our production. Being a SaaS solution, we could not afford any downtime.
Background
We have distributed architecture based on micro services running in AWS Fargate and AWS Lambda. For data persistency AWS RDS Aurora MySQL is used. While there are other services being used, those are not of interest in this use case.
Approach
After a good deliberation on in place upgrade by declaring a downtime and maintenance window, we realized that having zero downtime upgrade is the need. As without which we would have created a processing backlog for us.
High level approach was:
Create an AWS RDS Cluster with the required version and copy the data from the existing RDS Cluster to this new Cluster
Setup AWS DMS(Data Migration Service) between these two clusters
Once the replication is done and is ongoing then switch the application to point to the new DB. In our case, the micro-services running in AWS Fargate has to upgraded with the new end point and it took care of draining the old and using the new.
For Complete post please check out
https://bharatnainani1997.medium.com/aws-rds-major-version-upgrade-with-zero-downtime-5-6-to-5-7-b0aff1ea1f4
When you create a new Aurora cluster from a snapshot, you get a binlog pointer in the error log from the point at which the snapshot was taken. You can use that to set up replication from the old cluster to the new cluster.
I've followed a similar process to what you've described in your question (multiple times in fact) and was able to keep the actual downtime in the low seconds range.
I am pretty new to Oracle Golden Gate, wanted to understand if it possible to create a bidirectional sync between Oracle 12x and Cassandra(DSE) using Oracle Golden Gate? Searched several places in internet but most examples are replicating data between Oracle databases. I started wondering if it is even possible to do so. Can anyone help me with any documentation?
There is a separate module called Oracle GoldenGate for BigData. It supports many NoSQL replication targets.
One of the supported BigData databases is also Apache Cassandra.
There is a separate manual explaining how to use it.
There is no separate module that allows you to connect Apache Cassandra as the source of your replication. If you need such replication you need to provide some intermediate step. The source of replication for Oracle GoldenGate can only be a database (Oracle, TimesTen, DB2, Informix, MySQL, MS SQL Server, NonStop SQL/MX, SAP/Sybase ASE, Teradata) or a JMS queue.
We have an Oracle Database that resides tables. We would like to implement a new project as I mentioned in title; Oracle to Cassandra real-time replication.
But this new Cassandra environment would be as a reporting service. From the application (in-house), datas is inserted to Oracle production environment. Then our custom service (or what ever) will read delta and insert to Cassandra (this would be like Goldengate may be).
Briefly, does the Cassandra will answer our needs for this scenario?
In our case, we have 20 oracle DBs in different locations (these 20 dbs has similar implementation) 1 central report DB that is daily refresh from these 20 DBs. We use "outdated" snapshot technology, every night our central single report DB (REPORTDB) with fast refresh option, we gather the daily delta from these 20 dbs within oracle ss. we need a structure that reads data from 20 dbs and real-time injection to new cassandra database just like REPORDB
These days you can run spark jobs on Cassandra, thanks to Datastax so yes it can be used as a reporting tool. It's best utilized as a key value store if your number of writes are high compared to your reads.
Reading delta is not real time so you should try using Oracle's AQs. I've been doing real time replication of Oracle to Cassandra using Oracle's AQ and Apache Storm for almost 4 years now and it's running flawlessly.
I don't understand this Oracle/Cassandra architecture running alongside.
Either Oracle suits your needs then you should stick with it. Or it doesn't and you need scalability/high availability then switch to Cassandra.
Can you elaborate on the reasons that make you choose Cassandra for the reporting service ?
I have been assigned to move 10g database to AWS Oracle 12c. Size of data is around 20 GB of actual data but 900 GB has been allocated. So we will need to decrease that. Any suggestions on how to migrate and reduce the size of the database?
This is a good summary of several options:
Importing Data Into Oracle on Amazon RDS
Also here is the Oracle migration Whitepaper.
The solution depends on the size of your data and the accepted downtime.
Personally I used Data Dump in a project, it works well for the size of data you have. I'm not sure about the resizing.