I have {TrueIfReplica} for read_only in parameter group for Aurora. I setup my aurora to replicate from an external master that is vanilla MySQL. When I look at read_only in show variables it is 0; even though it is a replica. Why is this?
Related
Is there a way to upgrade from Aurora 1 (MySQL 5.6) to Aurora 2 (MySQL 5.7) without downtime on an active database? This seems like a simple task given we should be able to simply do major version upgrades from either the CLI or the Console, but that is not the case.
We tried:
Creating a snapshot of the database
Creating a new cluster using Aurora 2 (MySQL 5.7) from the snapshot
Configure replication to the new cluster from the primary cluster
However, because you can't run commands that require SUPER user privileges in Aurora you're not able to stop transactions long enough to get a good binlog pointer from the master, which results in a ton of SQL errors that are impossible to skip on an active database.
Also, because Aurora is not doing binlog replication to its Read replicas I can't necessarily stop replication to that read replica and get the pointer.
I have seen this semi-related question, but it certainly requires downtime: How to upgrade AWS RDS Aurora MySQL 5.6 to 5.7
UPDATE: AWS just announced in-place upgrade option available for 5.6 > 5.7:
https://aws.amazon.com/about-aws/whats-new/2021/01/amazon-aurora-supports-in-place-upgrades-mysql-5-6-to-5-7/
Simple as Modify and choose version with 2.x. :)
I tested this Aurora MySQL 5.6 > 5.7 on a 25Gb db, many minor versions behind and it took 10 min, with 8 min of downtime. Not zero downtime, but a very easy option, and it can be scheduled in AWS to happen automatically during off-peak times (maintenance window).
Additionally consider RDS Proxy to reduce downtime. During small windows of db unavailable time (eg. reboot for minor updates), the proxy will hold connections open, instead of completely unavailable, simply appearing as a brief delay/latency, only.
Need was to upgrade the AWS RDS Aurora MySQL from 5.6 to 5.7 without causing any downtime to our production. Being a SaaS solution, we could not afford any downtime.
Background
We have distributed architecture based on micro services running in AWS Fargate and AWS Lambda. For data persistency AWS RDS Aurora MySQL is used. While there are other services being used, those are not of interest in this use case.
Approach
After a good deliberation on in place upgrade by declaring a downtime and maintenance window, we realized that having zero downtime upgrade is the need. As without which we would have created a processing backlog for us.
High level approach was:
Create an AWS RDS Cluster with the required version and copy the data from the existing RDS Cluster to this new Cluster
Setup AWS DMS(Data Migration Service) between these two clusters
Once the replication is done and is ongoing then switch the application to point to the new DB. In our case, the micro-services running in AWS Fargate has to upgraded with the new end point and it took care of draining the old and using the new.
For Complete post please check out
https://bharatnainani1997.medium.com/aws-rds-major-version-upgrade-with-zero-downtime-5-6-to-5-7-b0aff1ea1f4
When you create a new Aurora cluster from a snapshot, you get a binlog pointer in the error log from the point at which the snapshot was taken. You can use that to set up replication from the old cluster to the new cluster.
I've followed a similar process to what you've described in your question (multiple times in fact) and was able to keep the actual downtime in the low seconds range.
I was wondering whether MySQL commands like NOLOCK or creation of indexes is actually supported on Aurora AWS? Also, can I do performance issue investigation or is Aurora AWS a "black-box" to me?
The Aurora it is completely different database than MySQL, but when it comes to the compability with MySQL or PostgreSQL Amazon team did a lot so that there would not be a lot of differences. "The Amazon Aurora MySQL is designed to be wire-compatible with MySQL 5.6 and 5.7 using the InnoDB storage engine. Certain MySQL features like the MyISAM storage engine are not available with Amazon Aurora." Amazon Aurora MySQL 2.01 does not currently also support the following MySQL 5.7 features:
Global transaction identifiers (GTIDs)
Group replication plugin
Increased page size
InnoDB buffer pool loading at startup
InnoDB full-text parser plugin
Multisource replication
Online buffer pool resizing
Password validation plugin
Query rewrite plugins
Replication filtering
The CREATE TABLESPACE SQL statement
X Protocol
(source:https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.2022.html)
But it changes from release to release.
Answering to you question about a performance monitoring. You can monitor Aurora database from AWS Console or by using queries directly on Aurora database, the same as in case of MySQL. Staring from version 2.03 there is also available the performance schema.
UNLOCK command - I am not sure whether it also exists in native MySQL, but there is possiblity to set transaction isolation level.
Indexes - yes, you can also create indexes on Aurora.
I hope the information will be useful for you.
I’m researching my options to publish microservices under AWS lambda. As we will be billed by the microsecond and memory used, performance of our lambda functions is pretty important.
My DB is Cassandra or other NOSQL distributed on several AWS nodes, on several data centres.
How can I configure lambda to be sure that my lambda function, including DB access, is using a DB node on the same data centre?
I know that Cassandra drivers will redirect intelligently queries to the nearest DB node, but what I need is to the restrict lambda execution to just nodes on the same data centres where I have some DB node.
You can configure or create a lambda role using IAM roles which will have access to only a specific aws resource (you can be restrictive or open on access control by specifying that in the policies).
Then assign that role to your lambda in a VPC or default VPC.
We're planning to move our Tomcat/MySQL app onto the Amazon cloud employing 2 EC2 instances (inst_1 & inst_2) running in different availability zones whereby inst_1 will contain the master RDS db and inst_2 the slave RDS db.
If we employ elastic load balancing to balance traffic between the two instances, will traffic directed to inst_2 that includes insert/update/delete db transactions first update the master RDS db in inst_1 followed by a synchronous update of the slave in inst_2; thereby ensuring that the two RDS instances are always synchronized?
Amazon's published info (whitepapers) suggests such, but doesn't explicitly state it. If not, how does one ensure that the two RDS instances remain synchronized?
Additional note: We're planning to employ Amazon's Elastic Beanstalk. Thanks!
You have to take a few things into consideration
AWS RDS instances are simple managed EC2 instances which run a MySQL server.
If you add a slave ( I think Amazon calls them read-replica) this is a read-only slave
Amazon doesn't manage the distribution of writing queries to the master server automatically.
Replication will ensure that your read slave always is up-to-date automatically ( with minimal delay which is increasing with write-load on the master )
This behavior is MySQL-specific
This means that you have to delegate manipulating queries to the master exclusively.
This can either be done by your application or by a MySQL proxy running on a extra machine.
The proxy then is the only interface your application servers will talk to. It is able to manage balancing between your RDS instances and the direction of any manipulation query to the master instance.
When RDS is used in multi-az mode you have no access to the secondary instance. There is only ever one instance that is visible to you, so most if your question doesn't apply. In case of failover the DNS address you are given will start resolving to a different ip. Amazon doesn't disclose how the two instances are kept in sync.
If instead of using a multi-az instance you use a single-az instance + a replica then it is up to you to direct queries appropriately - any attempt to alter data on the replica will fail. Since this is just standard MySQL replication, the replica can lag behind the master (in particular with current versions of MySQL the replica only runs a single thread)
How different is the Amazons RDS DB Instance different from The normal EC2 Instance other than the fact that RDS DB Instance has a Database server running on it?
When an EC2 Instance goes down all the data associated with it also vanishes(when you dont attach an EBS). Is this true for RDS DB Instance as well?
I have already set up my database server with the following: 1 small Instance?(m1.small) with Mysql and attached a 10GB EBS and routed the Mysql Data Directories to EBS.
Is the small Instance of EC2 RDS any Different from the above?
An RDS db instance can be configured to not lose any data during downtime, either planned or unplanned. For unplanned downtime, AWS keeps transaction logs which are replayed automatically on a failover instance. These logs can also be used to get an instance to a specific point in time.
For planned downtime, you create a DB snapshot prior to stopping the instance, and can later start a new instance with the saved snapshot.
RDS is a unmanaged MySQL service, means you only start and load data into it and your ready to go.
Is the small Instance of EC2 RDS any Different from the above?
The small instance of RDS is a 64bit, which support multi AZ failover and pricing obviously is a little expensive compared to EC2 MySQL.
EC2 Mysql needs more administration, but you can setup it up to do replication and you can customize it to have better performance compared to RDS.
See also http://www.dotdeb.org/2010/05/04/mysql-on-amazon-benchmarks-rds-vs-ec2/