I've seen examples of DynamoDB as the data source for AWS AppSync but I'm wondering if Aurora (specifically PostgreSQL) can be used? If yes, what would the resolvers look like for a basic example? Are there any resources that demonstrate doing this for Aurora PostgreSQL or even MySQL?
It can not. You can use Aurora Serverless as the data source which is driven by Data API (still in beta), this allows you to configure resolvers as database queries. That being said Data API is still very slow and Aurora Serverless has a cold-start of 30sec or so as it needs to run from VPC. I would recommend avoiding production but worth playing around.
You are much better of using Lambdas as resolvers or running HTTP RestFul calls from within the resolvers.
Ignore the comments provided in the answer, no disrespect, but the comments are coming from people who never managed production at scale. The fact you have a fully manage GraphQL service at scale as well as with high-security posture will save you months of maintenance nightmares when your product(s) will reach anything close to 1MIL revenue.
You can use the AWS Lambda resolver available in AWS AppSync to access Aurora Postgres. The code is similar to how you would access a relational database using any language. For example, you could use node-postgres with NodeJS to implement the Lambda function.
yes this can be done.
Do take a look at this open-source repo that does exactly that: https://github.com/wednesday-solutions/appsync-rds-todo
As of time of writing, yes but only if it is a Serverless Aurora RDS cluster set to Postgres compatibility. The reason for this is it's the only RDS instance type that supports the Data API. Other RDS instances would have to be configured as a different data source type, most commonly Lambda.
Related
I am not an expert in AWS but have some experience. Got a situation where Angular UI (host on EC2) would have to talk to RDS DB instance. All set so far in the stack except API(middle ware). We are thinking of using Lambda (as our traffic is unknown at this time). Again here we have lot of choices to make on programming side like C# or Python or Node. (we are tilting towards C# or Python based on the some research done and skills also Python good at having great cold start and C# .NET core being stable in terms of performance).
Since we are with Lambda offcourse we should go in the route of API GATEWAY. all set but now, can all the business logic of our application can reside in Lambda? if so wouldnt it Lambda becomes huge and take performance hit(more memory, more computational resources thus higher costs?)? then we thought of lets have Lambda out there to take light weight processing and heavy lifting can be moved to .NET API that host on EC2?
Not sure of we are seeing any issues in this approach? Also have to mention that, Lambda have to call RDS for CRUD operations then should I think to much about concurrency issues? as it might fall into state full category?
The advantage with AWS Lambda here is scaling. As you know already , cuz Lambda is fully managed so we can take advantages of it for this case.
If you host API on EC2, so we don't have "scaling" part in place. And of course, you can start using ECS, Auto Scaling Group ... but it is bring you to another route.
Are you building this application for learning or for production?
Anyone successfully integrated AWS AppSync with Laravel?
I am new to AWS AppSync but good experience with laravel.
I am trying to implement an offline-app feature in my mobile app and the mobile API part is what Laravel handles.
I looked into AWS AppSync but all they are talking about is dynamoDB and graphQL. Someplace it says i need to use AWS Lambda.
I really can't get a grip on how to properly implement this.
Any suggestions or pieces of advice are greatly appreciated.
I have basic experience with graphQL
Thanks
I checked a few video sessions and found HTTP endpoint can be used as a resolver. is this the proper way?
If I use HTTP as resolver can I still use the real-time features?
links
https://aws.amazon.com/appsync/
Laravel is a PHP framework, so I think the two options you would want to consider would be HTTP and Lambda data sources.
Lambda can be something of a catch-all for data sources: you have absolute control over what you call, how you do it, and in what language you do it. You just have to set up a Lambda function and create a data source in the AppSync console pointing to it, then have your Lambda function interact with your framework however it needs to.
I'm not terribly familiar with Laravel myself, but I believe HTTP is also a totally viable option. I would think this would be the way you want to go, as it cuts out the added complexity and latency of a Lambda function between AppSync and your end destination. A tutorial for setting one up is available here: https://docs.aws.amazon.com/appsync/latest/devguide/tutorial-http-resolvers.html
In either case, real time updates will be absolutely available to you.
Is better to use DynamoDB with Lambda functions? (and if yes why? is the connection faster for some reason, maybe Lambda is designed to be more compatible with DynamoDB than MySQL or other databases?)
Because in Amazon Lambda official main page it makes reference to "DynamoDB" but not to "MySQL" or any other DB system:
https://aws.amazon.com/lambda/?nc1=h_ls
But I've found some tutorials tu connect MySQL from Lambda functions:
https://aws.amazon.com/blogs/database/query-your-aws-database-from-your-serverless-application/
NOTE: I'm not asking for the difference between DynamoDB and MySQL, relational vs non-relational DB, etc. This is about "Lambda and Databases". I need my Lambda function to read and write in some DB, I was thinking in MySQL first but because the only reference on the main page is to DynamoDB I'm a bit confused if I should choose that for some reason (performance, connection speed, or some other limitations).
First of all AWS Lambda can connect with almost any database system.
There are several benefits when using Lambda with Dynamodb.
Using IAM policies for fine grained Access Control.
Both are Serverless offerings from AWS.
Simplicity in provisioning with AWS SAM
Dynamodb supports streams with Lambda for data driven workflows.
On the other hand using MySQL or any other database will require to place the database in a private subnet inside a VPC for security best practices. This will require placing Lambda functions also within a VPC that has a negative impact for performance since an Elastic Network Interface (ENI) needs to be attached to each Lambda function upon provisioning. This increases the cold start time of Lambda.
Its also challenging in managing stateful connections with other databases since Lambda is stateless which also impacts performance.
The AWS Dynamodb is complete managed service in which there is no minimum fee to use DynamoDB.you will pay only for the resources you provision.The AWS will takecare of millisecond latency at any scale.
When it comes to RDS MYSQL lets you to set up, operate and scale database on AWS.so you need to choose Multi A-Z or single A-Z deployment,Db class instance (micro, small, large, xlarge), storage etc.
Dynamodb is a distributed nosql solution designed for very large datastore/extremely high throughput nosql application, while RDS shines in smaller scale flexible traditional RDBMS for far more query and design flexibility.
For simple application and small data set you can go with Dynamodb, For large & complex application,go for Dynamodb if you look for high throughput or you can choose RDS if you look for cheaper option.
I am new to prometheus, and so I am not sure if high availability is part of Prometheus data store tsdb. I am not looking into something like having two prometheus server instances scraping data from the same exporter as that has high chance of having two tsdb data store which are out of sync.
It really depends on your requirements.
Do you need highly available alerting on your metrics? Prometheus can do that.
Do you need a highly available monitoring system that contains the last few hours of data for operational triage? Two prometheus instances are pretty good for that too.
Do you need long-term storage of timeseries data? Prometheus is not designed to accomplish this on its own. Either use the remote write functionality of prometheus to ship data to another TSDB that supports redundant storage (InfluxDB and Clickhouse are pretty promising here) but you are on the hook for de-duping data. Alternatively, consider Cortex.
For Kubernetes setup Using kube-prometheus (prometheus-operator), you can configure it using values.
and including thanos Would help in this situation
There is prometheus-postgresql-adapter that allows you to use PostgreSQL / TimescaleDB as a remote storage. The adapter enables multiple Prometheus instances (HA setup) to write to a single remote storage, so you have one source of truth. Recently, I've published a blog post about it [How to manage Prometheus high-availability with PostgreSQL + TimescaleDB] (https://blog.timescale.com/blog/prometheus-ha-postgresql-8de68d19b6f5/).
Disclaimer: I am one of the engineers behind the adapter
I am looking for general advice from anyone who has experience monitoring Oracle RDS databases in AWS. The system that I am working with will involve several enterprise Oracle RDS databases (on the order of a few dozen) in AWS. My organization is considering two options for monitoring:
Setting up Cloud Control in AWS, by housing the OMS and the repository database on an EC2 instance and enabling the OEM_AGENT on our RDS instances.
Relying entirely on EM Express/CloudWatch and any other third party software that we can use without the overhead of Cloud Control.
The concern with option 1 is that it undermines our reasons for moving to RDS, namely, to remove some of the administrative overhead of maintaining traditional on-premises Oracle databases. The OEM repository database cannot be housed in RDS as the OMS requires SYS-level access to the repository and RDS does not allow for this. As a result, having Cloud Control would require a lot of the kind of maintenance we were hoping to move away from.
The problem with option 2 is mainly the lack of metric alerting. CloudWatch/Enhanced Monitoring provide some basic metrics for alerts, but more specific metrics and alerts, such as those on alert log errors, tablespaces, long-running queries, archive area used, etc are lacking. We do not mind the lack of centralization as we will simply create an internal page with links to all of the different databases, and EM Express gives us what we need from a performance monitoring standpoint. The only concern really is the lack of metrics alerting. If there is not some other way of doing this, we may also simply write our own PL/SQL scripts to trigger the alerts.
However, I am curious to know how others solved this problem or even just generally, what kinds of AWS-based Oracle monitoring systems have been set up and how they work.
A problem almost all the enterprises which moving to cloud are facing today. Companies moving to cloud to get rid of some of their admin tasks and then they are figuring out they can't do all the customization that they had in on-prem.
So, here is how you can make the option 2 better. Especially to address your concern
The only concern really is the lack of metrics alerting
RDS events are a good way for monitoring. You can subscribe to the events and get notified in multiple ways such as group email, slack channels or to a third part monitoring tool like pagerduty.
Using RDS Events integration with Lambda. I strongly suggest to have a look on Lambda. As I mentioned above, apart from subscribing to the events, you can also call/trigger a lambda function to take actions for certain events. We use Lambda for overcoming the slave skip error in mysql.
Another use case of Lambda is an alternative to cron job. Things like checking disk space every day, to make sure incremental backups are taken over night.
Let me know, if you have specific question on "how to implement" these options. I will be glad to add more information.