thanks for reading...
Here's my problem:
The Background: I have been tasked with being a Build Manager (first time doing so). Among other things, this includes creating scripts to update the staging DB. I would like to utilize a Database Project for this so I may keep it in source control and also have it apart of TFS Labels to create a snapshot of the DB at the time of the build. This seems pretty straight forward to me.
The Setup: In our dev environment, we have a copy of our production DB. This DB is the one where our new data base objects will be deployed when we go live. We also have a copy of it in staging for testing.
The Goal: The only schemas (and related objects) I need in the Database Projects are limited only to the ones that aren't in production.
The Problem: When I create a new Database Project, I have no option to select only certain schemas to import (that I am aware of). Is there a way to only import Schemas A, B, and D and leave out dbo and C? Along with this, is it possible when using the 'Schema Comparison' to compare only selected schemas within the database?
Please let me know if my question and information are incomplete.
Thanks.
-Justin
Related
We are working in a very big organization, many Databases (of many types), many schemas, many users.
Does LB has to work with some Source Control (for locking the files
when many users exist in the organization and using the same DB,
same Schema, etc)?
What is the best practice of working with LB in a very big
organization, many concurrent users?
Can SQLCL general sql format type or just xml format type?
Is there some integration with SQL Developer? I mean, suppose a user
changes an objects via sql developer, what happens then?
We get this type of question all the time, after folks get a handle of how to automate DB changes, next step is typically to add it into an existing CI/CD workflow.
Yes, Liquibase works with any source control. Most users are using
Git. But you can use Git, TFS, SVN, CVS... Once you are up and
running with Liquibase, you just need to make sure that your scripts
are in source control and you are good to go.
Besides 3rd party source control tools, Liquibase has tracking tables called "DATABASECHANGELOG" tables that keep track of the changes applied to your database when using Liquibase deployments.
Here is some more information about getting started and How Liquibase Works. https://www.liquibase.org/get_started/how-lb-works.html
Liquibase has one more table that it uses internally called "DATABASECHANGELOGLOCK" table.
This table was designed to prevent multiple Liquibase users running deployments concurrently - potentially leaving the Database in a bad state. Once the Liquibase deployment (the liquibase update command) is done, the "DATABASECHANGELOGLOCK" will allow the next Liquibase user to deploy.
You can use both SQL and XML formats (or even JSON and YAML formats).
When using SQL, you have a few options:
Best option is to use Formatted SQL changeLogs https://www.liquibase.org/documentation/sql_format.html
https://www.liquibase.org/get_started/quickstart_sql.html
You can use plain raw SQL files referenced from an XML changeLog
https://www.liquibase.org/documentation/changes/sql_file.html
When using XML, can find all the available change types (also called changeSets) available in the following page (on the left of the page)
https://www.liquibase.org/documentation/changes/
XML changeLog are more agnostic and sometimes can be used for different Database platforms when doing migrations. Also, many of the change types in XML have the ability to be rolled back automatically. The reason that this is possible with XML is because Liquibase uses it own built in functions to figure out inverse statements like "create table" to be "drop table".
For each of those changeSets you can find out if they are auto rollback eligible (at the bottom of the page). For example, create table changeSet will be Auto Rollback = yes.
https://www.liquibase.org/documentation/changes/create_table.html
I have been building my ASP.NET Core MVC web application since last year and there are currently 100 migration files in my project, as the database has evolved along with features and capabilities. My development / test database is obviously in sync with this migration.
Now, the time has come to create a Production environment, and I want to have an empty database with only the table schema. So, I just can't copy my test database to create a production database.
So, the question / dilemma I have regarding creating this Production database is as follows;
1. I can either create my Production database by running Add-Migration command (for which I need to delete existing migration files from the project), or
2. Create table schema in SQL Management Server, and keeping the __EFMigrationsHistory from test database
With [1], I am not sure how I will be able to manage my test database with this same project, going forward.
With [2], I am not sure if there is any drawback.
So, what is the standard or best practice for Production deployment?
You can certainly use Visual Studio to create or update database tables, but this would normally just be in your development environment.
If you are worried about the number of migrations you have, you still have the option of starting over by deleting them, and deleting your migration history (and manually deleting your tables and data) and creating a fresh 'Initial create' migration. If you do this you will probably want to export any test or config data first, or make sure you have a way to recreate it.
For the staging and production deployments, best to use SQL Server Management Studio to generate a script to build your tables. The database administrator - or you if it's just you - can create the database and run the script to generate the tables.
This is because staging and production environments tend to be more tightly controlled than development, so best to understand the processes that are most appropriate for the environment.
You need to check that your dev and staging/production SQL databases are set at a matching 'compatibility level', and also need to decide whether you need to add any seed or configuration data on create.
In SQL Server Management Studio, select the appropriate option to generate a script for the required tables.
https://learn.microsoft.com/en-us/sql/ssms/scripting/generate-scripts-sql-server-management-studio?view=sql-server-ver15
You can also use this process to export, and import data - this is ideal for config or test data.
https://dzone.com/articles/generate-database-scripts-with-data-in-sql-server
I recently moved my screwturn installation to a completely new environment (site on new IIS/server and database moved to new SQL Server).
I can succesfully open the Screwturn wiki but instead of using the existing database (that I configured in the web.config), Screwturn wants to create the DB from scratch and hence states the error that certain tables already exist in the DB.
If I create a blank DB and use this in the web.config, Screwturn will happily create its tables and use that DB. Of course, that's not what I want, I want it to use the existing, moved DB on the server.
Is there a way to tell Screwturn not to create the DB and simply "use" the existing one?
If this is not possible, are there scripts available to copy the content from the old DB to the new one (simply copying it with SQL Servers Import/Export feature gives errors (logical) that certain fields cannot be created as that will cause PK violations.
Did you use your old web.config or just update the SQL connection info in the web.config from the new version? I think you should use your old web.config (maybe compare to the new one to see if anything was added). I have done this several times with several installations and haven't had a problem.
The only other thing I can think of off-hand is if your old version is v2, then you should review the instructions here as a few things changed from v2 to v3.
Old Screwturn Wiki Site - Upgrade from v2 to v3
I'm a bit new to database projects, but I find that with a bigger team it is a good way to synchronize development.
I had a single Database Project that contained all the tables used by a web app.
The powers that be want to display data from another database in the web app.
I create another Database Project that represents the tables I need in the other database.
I create views in the original database that represent tables in the second database (for linq to sql joins mainly)
Questions
Do I create a Sql Server 2008 Server Project and merge both of these databases into it? From what I can tell, a Server Project still only represents a single database.
If a Server Project isn't meant to represent multiple databases, what is it for?
I have added steps in our TFS Workflow to deploy changes from the first Database Project into our Dev environment on check-in (CI). Assuming a Server Project is meant to contain multiple database projects, can I deploy it and have it deploy changes in both databases?
Thanks!
A server project is for server scope objects needed by your solution: endpoints, server principals (logins), server level permissions, linked servers and other similar constructs that are not scoped to any particular database.
Seems to me that what you need is a solution with two projects:
your original database project
you reference database project
You need a third project in this solution, a server project, only if you must add server scoped objects.
For more details, see Using References in Database Projects.
Hey guys,
I need to figure out a way to back up and also migrate our Oracle database from our production schema to the dev schema and the other way around.
We have bunch of config tables that drive how systems on our platform run, and when setting up new systems or doing maintenance, we need to update our config tables. We want to be able to work on the dev schemas and after setting up a system/feature, we want to be able to migrate all those configs to the dev schemas.
I thought of running a procedure where we give the ID of the system (from the main table) and i would go through all the tables and select nvl(..) and if it doesn't exist, i would insert into, and if it does exist then i just run an update on that row.
This code will get very messy and complicated especially since the whole config schema is very complex and it might be hard to handle all the keys properly.
Another option i was looking at was triggers, so when setting up a new system, there would be a log of all the statements we ran while setting up/editing a system, then we would run it on our production schema.
I'm on a coop term, and have only been working with databases for 6 months, so i don't know that much and any information/advice would be greatly appericiated.
(We use pl/sql)
What about using export / import (or datapump) to bring over the config tables?
Check out data comparison tools like this
Think TOAD has one built in. I'm sure there are others out there too.
It is common to have tables in a schema that are what we call "static data", i.e. the users don't change it because it controls how the application works.
Each change to config data should not be run ad-hoc in the target environment. Instead, you design and code your DML carefully in one or more scripts, which get tested in a dev environment, checked into change control, and can be re-run in any environment when required.