I want to know if I can configure retention policy per database?
I have several databases, about 100 and I need a different retention policy for each database. Is that possible?
If possible, How do is see the retention policies per database?
SELECT value from v$rman_configuration WHERE name = 'RETENTION POLICY';
The above sql just returns one single row.
you can add line in you backup script, base on different database you connected.
======================for example===
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
It very much depends on what you mean by "database":
RMAN only works with physical database files for the entire "database", i.e. all users/schemas contained in those files. Your retention policy in RMAN then applies to all data files and all users/schemas within those files. Each physical Oracle database (all tablespaces, archive logs, etc.) can have its own retention policy, which is stored in the database control files or in the RMAN repository. There will only ever be a single retention policy at any given time, which is what your query is showing.
If by "database" you mean individual logical schemas within a physical database, then the answer is "no", you cannot have separate RMAN retention policies at that level.
Related
Is there anyway to change the backup retention? I have not found any way to change that and we want to confirm if it is possible.
ADB stores backups (database and archivelogs) for 60 days as documented here. You can restore a database in-place, or you can create a new database from any point in time in the last 60 days.
Currently, there's no way to change the backup retention.
As an alternative, you can do Data Pump exports to the object store and keep the dump files based on your retention requirements.
We recently started the process of continuous migration (initial load + CDC) from an Oracle database on RDS to S3 using AWS DMS. The DB is using LogMiner.
the problem that we have detected is that the CDC records of type Update only contain the data that was updated, leaving the rest of the fields empty, so the possibility of simply taking as valid the record with the maximum timestamp value is lost.
Does anyone know if this can be changed or in what part of the DMS or RDS configuration to touch so that the update contains the information of all the fields of the record?
Thanks in advance.
Supplemental Logging at table level may increase what is logged, but that will also increase total volume of log data written for a given workload.
Many Log Based Data Replication products from various vendors require additional supplemental logging at the table level to ensure the full row data for updates with before and after change data is written to the database logs.
re: https://docs.oracle.com/database/121/SUTIL/GUID-D857AF96-AC24-4CA1-B620-8EA3DF30D72E.htm#SUTIL1582
Pulling data through LogMiner may be possible, but you will need to evaluate if it will scale with the data volumes you need.
DMS-FULL/CDC also supports Binary Reader better option to LogMiner. In order to capture updates WITH all the columns use "ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS" on Oracle side.
This will push all the columns in a update record to endpoint from Oracle RAC/non-RAC dbs. Also, a pointer for CDC is use TRANSACT_ID in DMS side to generate a unique sequence for each record. Redo will be little more but, it is what it is; you can keep an eye on it and DROP the supplemental logging if require at the table level.
Cheers!
It's kinda real-world problem and I believe the solution exists but couldn't find one.
So We, have a Database called Transactions that contains tables such as Positions, Securities, Bogies, Accounts, Commodities and so on being updated continuously every second whenever a new transaction happens. For the time being, We have replicated master database Transaction to a new database with name TRN on which we do all the querying and updating stuff.
We want a sort of monitoring system ( like htop process viewer in Linux) for Database that dynamically lists updated rows in tables of the database at any time.
TL;DR Is there any way to get a continuous updating list of rows in any table in the database?
Currently we are working on Sybase & Oracle DBMS on Linux (Ubuntu) platform but we would like to receive generic answers that concern most of the platform as well as DBMS's(including MySQL) and any tools, utilities or scripts that can do so that It can help us in future to easily migrate to other platforms and or DBMS as well.
To list updated rows, you conceptually need either of the two things:
The updating statement's effect on the table.
A previous version of the table to compare with.
How you get them and in what form is completely up to you.
The 1st option allows you to list updates with statement granularity while the 2nd is more suitable for time-based granularity.
Some options from the top of my head:
Write to a temporary table
Add a field with transaction id/timestamp
Make clones of the table regularly
AFAICS, Oracle doesn't have built-in facilities to get the affected rows, only their count.
Not a lot of details in the question so not sure how much of this will be of use ...
'Sybase' is mentioned but nothing is said about which Sybase RDBMS product (ASE? SQLAnywhere? IQ? Advantage?)
by 'replicated master database transaction' I'm assuming this means the primary database is being replicated (as opposed to the database called 'master' in a Sybase ASE instance)
no mention is made of what products/tools are being used to 'replicate' the transactions to the 'new database' named 'TRN'
So, assuming part of your environment includes Sybase(SAP) ASE ...
MDA tables can be used to capture counters of DML operations (eg, insert/update/delete) over a given time period
MDA tables can capture some SQL text, though the volume/quality could be in doubt if a) MDA is not configured properly and/or b) the DML operations are wrapped up in prepared statements, stored procs and triggers
auditing could be enabled to capture some commands but again, volume/quality could be in doubt based on how the DML commands are executed
also keep in mind that there's a performance hit for using MDA tables and/or auditing, with the level of performance degradation based on individual config settings and the volume of DML activity
Assuming you're using the Sybase(SAP) Replication Server product, those replicated transactions sent through repserver likely have all the info you need to know which tables/rows are being affected; so you have a couple options:
route a copy of the transactions to another database where you can capture the transactions in whatever format you need [you'll need to design the database and/or any customized repserver function strings]
consider using the Sybase(SAP) Real Time Data Streaming product (yeah, additional li$ence is required) which is specifically designed for scenarios like yours, ie, pull transactions off the repserver queues and format for use in downstream systems (eg, tibco/mqs, custom apps)
I'm not aware of any 'generic' products that work, out of the box, as per your (limited) requirements. You're likely looking at some different solutions and/or customized code to cover your particular situation.
We are planning for a large Greenplum DB (growing from 10 to 100TB over the first 18 months). Traditional backup and restore tools aren't going to help as we have 24hr RPO/RTOs to deal with.
Is there a way to replicate the DB across to our DR site without resorting to block replication (i.e. place a segment on SAN and mirror)?
You've got a number of options to choose:
Dual ETL. Replicate input data and run the same ETL on two sites. Synchronize them with backup-restore every week or so
Backup-restore. Simple backup-restore can be not that efficient. But if you use DataDomain it can perform deduplication on the block level and store only changed blocks. It can offload the deduplication task to run on the Greenplum cluster (DDBoost). Also in case of replication to remote site it would replicate only changed blocks, which would greatly reduce replication time. In my experience, if clean backup on DD takes 12 hours, subsequent DDBoost backup will take 4 hours + 4 hours to replicate the data
Custom solution. I know the case when the data replicatioin to remote site is made as a part of ETL process. For the ETL job you know which tables are changed, they are added to the replication queue and moved to the remote site using external tables. Analysts are working in a special sandbox and their sandbox is replicated with backup-restore daily
At the moment Greenplum does not have built-in WAN replication solution so this is almost all the options to choose from.
I do some investigation on this. here is my result
I. Using EMC Symmetrix VMAX SAN(Storage Area Network) Mirror and SRDF (Symmetrix Remote Data Facility) remote replication software
Please refer to h12079-vnx-replication-technologies-overview-wp.pdf for details
Preconditions
1. Having EMC Symmetrix VMAX SAN installed
2. Having SRDF softeware
Advantages of 3 different modes
1. Symmetrix Remote Data Facility / Synchronous (SRDF/S)
Provides a no data loss solution (Zero RPO).
No server resource contention for remote mirroring operation.
Can perform restoration of primary site with minimal impact to application. Performance on remote site. Enterprise
disaster recovery solution. Supports replicating over IP and Fibre
Channel protocols.
2. Symmetrix Remote Data Facility / Asynchronous (SRDF/A) Extended-distance data replication that supports longer distances
than SRDF/S. SRDF/A does not affect host performance, because host
activity is decoupled from the remote copy process. Efficient link
utilization that results in lower link-bandwidth requirements.
Facilities to invoke failover and restore operations. Supports
replicating over IP and Fibre Channel protocols.
3. Symmetrix Remote Data Facility / Data Mobility (SRDF/DM)
II. Using Backup Tools
Please refer to http://gpdb.docs.pivotal.io/4350/admin_guide/managing/backup.html for details
Parallel Backup
Parallel backup utility gpcrondump
Non-parallel backup
It is not recommended. It is used for migrate PostgreSQL databases to GreenPlum databases
Parallel Restore
Support system with the same configuration and different configuration with the source GreenPlum database configuration
Non-Parallel Restore
pg_restore requires to modified the create statement to add distributed by clause
Disadvantages
1. The backup process locks table, it put an EXCLUSIVE lock on table pg_class. It means that read permission is only allowed in this period.
2. After releasing the EXCLUSIVE lock on table pg_clas, it will put an ACCESS SHARE lock on all the tables, it only allows read access during the lock period.
III. Replay DDL statements
In PostgreSQL, there is a parameters to log all the sql statements to a file.
In the data/postgresql.conf, modify log_statement to ‘all’
Write an application to get the DML and DDL statement, and run them in the DR servers.
Advantage
1. Easy to configure and maintain
2. No decrease in the performance
Disadvantage
1. Need additional storage for the statement logging
IV. Parse the wal log of PostgreSQL
Parse the wal log to extract the DDL statement from the log and then run all the generated DDL statements in the DR GreenPlum
Advantage
1. Doesn’t impact the source GreenPlum Database
Disadvantage
1. Write code to parse the wal log
2. Not easy to parse the log, there are not enough documents about the wal log.
3. Don’t know if it is feasible for GreenPlum, as it is one solution for PostgreSQL.
I'd like to figure out the best way to archive the data that is no needed anymore, in order to improve the application performance and also to save disk space. In your experience what is the best way to implement this, what kind of tools can I use? It is better to develop an specific in house application for that purpose?
One way to manage archiving:
Partition your tables on date range, e.g. monthly partitions
As partitions become obsolete, e.g. after 36 months, those partitions can now be moved to your data warehouse, which could be another database of just text files depending upon your access needs.
After moving, the obsolete partitions can be removed from your primary database, so always maintaining just (e.g.) 36 months of current data.
All this can be automated using a mix of SQL/Shell scripts.
The best way to archive old data in ORACLE database is:
Define an archive and retention policy based on date or size.
Export archivable data to an external table (tablespace) based on a defined policy.
Compress the external table and store it in a cheaper storage medium.
Delete the archived data from your active database using SQL DELETE.
Then to clean up the space execute the below commands:
alter table T_XYZ enable row movement;
alter table T_XYZ shrink space;
If you still want to free up some disk space back to the OS (As Oracle would have now reserved the total space that it was previously using), then you may have to resize the datafile itself:
SQL> alter database datafile '/home/oracle/database/../XYZ.dbf' resize 1m;
For more details, please refer:
http://stage10.oaug.org/file/sroaug080229081203621527.pdf
I would export the data to a comma-delimited file so it can be exported into almost any database. So if you change versions of Oracle or go to something else years later you can restore it without much concern.
Use the spool file feature of SQL*Plus to do this: http://cisnet.baruch.cuny.edu/holowczak/oracle/sqlplus/#savingoutput