I came across the below document for liquibase but confused about the wording on how it identifies the unique change.
https://docs.liquibase.com/concepts/basic/changeset.html
It says:
'If it has been run, the changeset will be skipped unless there is a runAlways tag set to true in that changeset. After all the changes in the changesets are run, Liquibase will insert a new row with the id/author/filepath along with an MD5Sum of the changeset in the DATABASECHANGELOG.'.
My question here is:
If change is uniquely identified when ALL of these 3 attributes are different from what is stored in DATABASECHANGELOG table? Or
Change is uniquely identified when ANY of these 3 attributes is different from what is stored in the changelog table.
From the DATABASECHANGELOG table docs:
The table tracks each changeset as a row, identified by a combination of the “id”, “author”, and “filename” columns. There is no primary key on the table. This is to avoid any database-specific restrictions on key lengths. The composite of “id”, “author”, and “filename” is unique across all rows of the table.
Meaning if any of the attributes are different, it is considered as a new, unique entry and Liquibase will not compare MD5Sum with any already existing rows in that table. Even though the ID values are the same for example, which may not come as obvious when you are starting to work with Liquibase.
The primary column of the table DATABASECHANGELOG consists of the three columns you mentioned (at least in liquibase version 2.0.5, which is quite ancient):
ALTER TABLE databasechangelog
CONSTRAINT pk_databasechangelog
PRIMARY KEY (id, author, filename);
As #veljkost pointed out, the current version 4.3.2 hasn't got a primary key as some database vendors limit the length of an index.
Related
I have a table where i can chose to delete a row. e.g i have table having 5 records numbered from 1-5, after deleting lets say 3, am looking a way that the remaining records will be 1,2,3,4 and not 1,2,4,5.
If you want to reorder by IDs, please think if it is necessary. Here is explanation https://laracasts.com/discuss/channels/eloquent/how-do-you-handle-with-reordering-items Also if you are using soft deletes it can cause a problem concerning unique IDs.
If you are using other column to reorder by, you can iterate through each entity and setting value to iterator value after deleting.
Warning: If you use this approach on Primary Key columns it might bring inconsistency and mess up your relations if you don't take care of it properly. Also most of the time it is unnecessary to reorder the primary key column.
The process can be applied from database itself. The general procedure is as following:
Make sure if the column is used as foreign keys in other tables, the definitions must have ON UPDATE CASCADE
(On production server) to reduce inconsistency put lock on the table.
Apply reordering
SQL:
For example you can run the following commands for MySQL (inspired by this answer)
-- if on production we lock the table write
LOCK TABLES my_reordering_table WRITE;
SET #count = 0;
UPDATE `my_reordering_table` SET `my_reordering_table`.`id` = #count:= #count + 1;
ALTER TABLE `my_reordering_table` AUTO_INCREMENT = 1;
Warning: Again make sure you do not run this on production server and if you run, just make sure you have all the foreign keys ON UPDATE CASCADE.
I'm trying to auto-generate ID's for my entity, but it's not generating. Instead, it's starting from 1 when there already exists an entry with id "1" in my DB. Why is it not generating id "9" for my new entity?
Typically when creating a table with GenerationType.IDENTITY on postgres, Hibernate will setup the id column plus a database sequence to manage this id.
By convention the sequence name will be "tablename_id_seq". E.g., for the table ad_group_action there will be a corresponding sequence ad_group_action_id_seq. You can connect to the database to double-check the actual sequence name created.
The sequence just starts from 1 and increments each time a row is inserted by Hibernate.
But if there are pre-existing rows -- or if rows with existing IDs are inserted "manually" into the table -- those rows can conflict with the sequence.
One solution is to simply reset the sequence (from pgAdmin or another database client) to start at a higher number (say 100), using something like:
ALTER SEQUENCE ad_group_action_id_seq RESTART WITH 100;
Now Hibernate will not conflict with the existing rows (assuming their max id is < 100).
Alternatively, when inserting rows manually, omit the id column and let postgres automatically set them. This way the table and the sequence will always be in sync.
I need to update the some tables in my application from some other warehouse tables which would be updating weekly or biweekly. I should update my tables based on those. And these are having foreign keys in another tables. So I cannot just truncate the table and reinsert the whole data every time. So I have to take the delta and update accordingly based on few primary key columns which doesn't change. Need some inputs on how to implement this approach.
My approach:
Check the last updated time of those tables, views.
If it is most recent then compare each row based on the primary key in my table and warehouse table.
update each column if it is different.
Do nothing if there is no change in columns.
insert if there is a new record.
My Question:
How do I implement this? Writing a PL/SQL code is it a good and efficient way? as the expected number of records are around 800K.
Please provide any sample code or links.
I would go for Pl/Sql and bulk collect forall method. You can use minus in your cursor in order to reduce data size and calculating difference.
You can check this site for more information about bulk collect, forall and engines: http://www.oracle.com/technetwork/issue-archive/2012/12-sep/o52plsql-1709862.html
There are many parts to your question above and I will answer as best I can:
While it is possible to disable referencing foreign keys, truncate the table, repopulate the table with the updated data then reenable the foreign keys, given your requirements described above I don't believe truncating the table each time to be optimal
Yes, in principle PL/SQL is a good way to achieve what you are wanting to
achieve as this is too complex to deal with in native SQL and PL/SQL is an efficient alternative
Conceptually, the approach I would take is something like as follows:
Initial set up:
create a sequence called activity_seq
Add an "activity_id" column of type number to your source tables with a unique constraint
Add a trigger to the source table/s setting activity_id = activity_seq.nextval for each insert / update of a table row
create some kind of master table to hold the "last processed activity id" value
Then bi/weekly:
retrieve the value of "last processed activity id" from the master
table
select all rows in the source table/s having activity_id value > "last processed activity id" value
iterate through the selected source rows and update the target if a match is found based on whatever your match criterion is, or if
no match is found then insert a new row into the target (I assume
there is no delete as you do not mention it)
on completion, update the master table "last processed activity id" to the greatest value of activity_id for the source rows
processed in step 3 above.
(please note that, depending on your environment and the number of rows processed, the above process may need to be split and repeated over a number of transactions)
I hope this proves helpful
Found this here:
In general, consider creating an index on a column in any of the following situations:
A referential integrity constraint exists on the indexed column or
columns. The index is a means to avoid a full table lock that would
otherwise be required if you update the parent table primary key,
merge into the parent table, or delete from the parent table.
I don't understand why a full table lock would occurr in such situation. I would've thought that if I tried to delete/update the primary key in the parent table that a full table scan would be performed on the child table.
Where does the lock come from?
Have a look at this Tom Kyte blog entry. In it, he refers to the Oracle documentation, where this explanation is offered:
Prevents a full table lock on the child table. Instead, the database acquires a row lock on the index.
Removes the need for a full table scan of the child table. As an illustration, assume that a user removes the record for department 10 from the departments table. If employees.department_id is not indexed, then the database must scan employees to see if any employees exist in department 10.
In the first scenario, if the column is not indexed, the entire table must be locked because Oracle does not know which rows must be updated in the child table. With an index, Oracle can identify the rows in question and just lock them. Without the full table lock, it would be possible to modify the parent and have another session modify the child to something that violated the constraint.
I need to rename a table in Oracle but I want to be sure that any foreign keys, constraints, triggers and sequences that reference the table are updated to use the new name.
How can I be sure that I have not broken anything?
Note that I want to preserve any existing data that the table contains.
If you
ALTER TABLE old_table_name
RENAME TO new_table_name;
all the existing constraints (foreign key and other constraints) and triggers will reference the newly renamed object. Sequences have no relationship to tables so there will be no impact on the sequences (though if you mean that you are referencing the sequence in a trigger on the table, the trigger will continue to reference the same sequence after the rename). Any stored procedures that you have written that reference the old table name, however, will need to be updated to reference the new table name.
Now, while the constraints and triggers will continue to work correctly, they will retain their original names. If you have naming conventions for these objects that you want to maintain after the table name, you'd need to do more. For example, if you want a row-level before insert trigger on table FOO to be named TRG_BI_FOO and you rename the table to BAR, you'd need to alter the trigger explicitly to change its name
ALTER TRIGGER trg_bi_foo
RENAME TO trg_bi_bar;
Similarly, you'd need to rename your constraints and indexes
ALTER TABLE bar
RENAME CONSTRAINT pk_foo TO pk_bar;
It depends on what you mean by "any foreign keys, constraints, triggers and sequences that reference the table are updated to use the new name."
Any existing indexes, constraints, and triggers against the table being renamed will automatically reference the new name.
However, any naming conventions used for those objects won't automatically use the updated name. For example, if the primary key for TABLE_NAME is generally named TABLE_NAME_PK, renaming TABLE_NAME to NEW_TABLE_NAME won't automatically rename the primary key constraint to NEW_TABLE_NAME_PK.
What will need to be checked is code - packages, procedures, and functions - which referenced the old table name, as well as any triggers which referenced the old table name. Similarly, views against the old table name will break as well. The view ALL_DEPENDENCIES can help identify which of those objects need to be updated.
ALTER TABLE oldName RENAME TO newName
Will preserve the table's dependencies and data but there can always be a piece of PL/SQL that references the old name which is going to become invalid.