In looking at the sqitch docs, there’s a situation I don’t immediately understand how to deal with.
Like probably many organizations, we progress changes through several environments before they reach production. In our situation, we have a different DBA user on a different Oracle server for each environment, each with its own credentials.
As I understand it, sqitch uses database tables to track what changes have been applied to a server. Maybe I’m dumb, but it just doesn’t jump out at me how sqitch can tell me if a change has been applied to a UAT server, but not yet to a production server.
So basically, I’d like to organize a repository to move changes from one DB environment to the next. Might this be what “sqitch target” and plan files are for? Are there examples I can look at?
If I were you, I would create a centralized DB with DB links to points to each database. After that, I would create an Union of all Repositories and a View (with PIVOT function) to see the deployement path of each patch.
To deploy to multiple environments it would depend on how you're running your sqitch deploy command.
You use the sqitch.conf to declare various targets.
eg.
[core]
engine = oracle
top_dir = SQL
[engine "snowflake"]
reworked_dir = SQL/rework
[target "DEV"]
uri = "db:oracle:DEV_DB"
[target "QA"]
uri = "db:oracle:QA_DB"
[target "PROD"]
uri = "db:oracle:PROD_DB"
With that sqitch.conf setup you can now run deploy to target the required environments.
eg.
sqitch deploy --target DEV
sqitch deploy --target QA
sqitch deploy --target PROD
You won't be able to compare deployments from one environment to another unfortunately.
You can use the sqitch check --target <xxx> command to check for divergences between the planned and deployed changes as stated here: https://sqitch.org/docs/manual/sqitch-check/.
However I've found this to not work properly at times. I haven't been able to determine the exact cause as yet but you're welcome to run the command to check.
Related
I followed the documentation here https://devcenter.heroku.com/articles/pipelines and setup a pipeline with two stages:
The above link only talks about how to promote an app, without specifying how to deploy, this section specifically (we can read beyond this point and see there is no instruction on how to deploy):
Pipelines let you define how your deployed code flows from one
environment to the next. For example, you can deploy code to your
staging app (which builds it into a slug) and later promote that same
slug to production. This promotion flow ensures that production
contains the exact same code that you tested in staging, and it’s also
much faster than rebuilding the slug.
So I used the same command we use for deploying non-pipeline based apps (as instructed here):
docker push registry.heroku.com/ultimate-hello-world-cra-stage/web
This appears to be a wrong approach. You can see in snapshot above that STAGING app did get deployed, but when I try to promote the app using this command heroku pipelines:promote --app ultimate-hello-world-cra-stage, I get this error: Error: Pipeline promotions are not supported on apps pushed via Heroku.
What is the correct way to deploy to a pipeline and where can I find it in the docs?
I have Azure Devops for Azure databases using Dacpacs.
I can easily deploy schema from Dev to Test and Prod.
However I have a issue. Dev databases have several Dev only tables that I don't want to deploy to Test and Prod.
Excluding certain tables manually with Visual Studio have resulted to human errors and certain non wanted tables have been deployed to prod.
It there are solution for making sure that Dev only tables are automatically excluded from Dacpac?
Possible to automatically filter if table name starts with "Temp*"?
No, you won't be able to request that it just include certain object kinds. Yes, you can ask it to exclude certain object types (/p:ExcludeObjectTypes), allowing you to filter to exactly what you want while eliminating anything else. Using the DacFx API's programmatic paradigm, you can accomplish more targeted/convenient things, but it requires programming code.
You can use sqlPackage.exe to limit the modifications by using the /p:ExcludeObjectTypes argument to indicate the kinds you don't want to deploy.
Use the following as an example:/p:ExcludeObjectTypes="StoredProcedures;ScalarValuedFunctions;TableValuedFunctions"
The following is a list of potential ExcludeObjectTypes arguments: https://learn.microsoft.com/en-us/dotnet/api/microsoft.sqlserver.dac.objecttype?view=sql-dacfx-150
Im looking at Sqitch and so far it seems like a great tool, however I have an existing project that I want to use it with, Is there a way to create a baseline?
For example, I take a backup of my schema then add it to the deploy script, I then want to run a command that will not run the this script on the database as it already exists, but would apply everything after this point?
I need the full base schema in there so that we can re-deploy the whole schema if required
You can use the --log-only option of sqitch deploy command
From the docs: https://sqitch.org/docs/manual/sqitch-deploy/
--log-only
Log the changes as if they were deployed, but without actually running the deploy scripts. Useful for an existing database that is being converted to Sqitch, and you need to log changes as deployed because they have been deployed by other means in the past.
I have provisioned postgres on my heroku app and also installed postgres locally to maintain parity (as the documentation recommends) with the online database but I'm also not understanding how this will work. Am I supposed to be accessing a local copy of a database when running on my own computer (while building and before deploying) and then using heroku's separate postgres database once it is deployed? If it is parity, shouldn't they both be using the heroku postgres database?
In other words, will my local app (during production) and heroku app (deployed and live) be using the same online postgres database?
Thanks.
Am I supposed to be accessing a local copy of a database when running on my own computer (while building and before deploying) and then using heroku's separate postgres database once it is deployed?
Yes, that's exactly it. Without seeing what bit of documentation you're referencing it's hard to say what they mean but perhaps there's another way to explain it.
In your local development environment, you may find that you need to test database schema changes (this is just one example, there are many). If you only had the one heroku postgres database you'd be forced to test these changes in production, which might result in poor usability for your users and that doesn't even account for the possibility of making a mistake and accidentally destroying your production data. There are a number of other shortcomings and challenges with this single database configuration.
For these reasons and more, it's best to keep your production data completely separated from your development/staging/test environment by creating a local/staging database. You might reasonably ask, "What about the data? I need data to test!". There are many ways to put together your test database and which you choose will likely depend on your needs. A shortlist of possibilities:
Use a seed file to generate mock data in your db
Use a model factory (usually runs in conjunction with your testing framework)
Take a dump of your production database, anonymize and redact sensitive information and use that for local testing.
I am planning to build an enterprise application using aws lambda and serverless framework.
I want to separate the dev, test and prod environments and I am planning to use AWS Parameter store for it.
I don't want my production environment configuration be exposed to developers. If the developer runs the command serverless offline -s production start then the production configuration should not be obtained.
It should be obtained only when the serverless function has been successfully deployed to aws lambda.
Here are few considerations based on your question:
To have different environments on Serverless framework you have to set up the stage. This value can be passed as a parameter when executing sls commands.
If you are keeping your code in a repo, the developers will have access to all the configurations. If this is really important, you could keep the production configuration in a diff repo where only very specific people will have access to it, and then you make a reference to in in your serverless.yml. Ex:
custom: ${file(./config/${opt:stage, 'dev'}.json)} and then in your config folder you create the prod.json file, but pointing to the real one of the new repo you created. Note: this would make your project harder to maintain.
Considering you don't want your developers to execute your production environment locally. You can use the global variable of serverless offline to block the execution. You could also inform then to not do so.
Here is what should be a good practice and solution based on your problem:
Considering you have a production environment you want to isolate from a given group in your company, you should create VPC's and configure their resources access, accordingly.
Then you create users to have diff access. When your developer try to execute the code accessing a resource (dynamoDB for example) in a VPC they don't have access, they will be blocked.
AWS configure to define which user will execute the SLS command.
Your development team will still have access to your configuration file.
Note: In this case the person/group with access to the production VPC will have to do the deploy.
If the answer does not suffice, could you please reinforce which type of resource(s) are sensitive across your Serverless project? I am taking for granted it is the DB as it is the most common scenario.