I am going to perform an upgrade on a Hadoop cluster but am wanting to backup the Ambari metastore schema first in case anything goes wrong.
Oracle is used to store the data, so I looked at using expdp to make a quick backup of the schema in its current state. However, I see in several different documents it is mentioned this is used to "unload" data. Does that mean the data will be removed from the database during the dump process? I want to keep everything in place and just make a quick backup, similar to the Postgres command pg_dump.
Don't worry, your data will stay where it is.
Here's a simple example: I'm exporting Scott's DEPT table. You'll see that data is in the table before and after EXPDP was executed.
SQL> select * from dept;
DEPTNO DNAME LOC
---------- -------------- -------------
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
SQL> $expdp scott/tiger#xe tables=dept directory=ext_dir
Export: Release 11.2.0.2.0 - Production on Pon O×u 5 21:21:24 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Connected to: Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
Starting "SCOTT"."SYS_EXPORT_TABLE_01": scott/********#xe tables=dept directory=ext_dir
Estimate in progress using BLOCKS method...
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 64 KB
Processing object type TABLE_EXPORT/TABLE/TABLE
Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX
Processing object type TABLE_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported "SCOTT"."DEPT" 5.929 KB 4 rows
Master table "SCOTT"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SCOTT.SYS_EXPORT_TABLE_01 is:
C:\TEMP\EXPDAT.DMP
Job "SCOTT"."SYS_EXPORT_TABLE_01" successfully completed at 21:21:34
SQL> select * from dept;
DEPTNO DNAME LOC
---------- -------------- -------------
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
SQL>
Related
I'm using Spring Data R2DBC with Oracle 11g and I have the following error using method findById of R2dbcCrudRepository
executeMany; bad SQL grammar [SELECT GAME_PHASE.* FROM GAME_PHASE WHERE GAME_PHASE.ID = :P0_id FETCH FIRST 2 ROWS ONLY]
This is the repository declaration
public interface ReactiveGamePhaseRepository extends R2dbcRepository<GamePhase, Long> {
}
I don't understand why FETCH FIRST 2 ROWS ONLY is added to the query and it's the cause of problem.
I have the same problem writing the query using R2dbcEntityTemplate like below:
r2dbcEntityTemplate.selectOne(query(where("id").is(id)), GamePhase.class);
And these are the used dependancies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-r2dbc</artifactId>
<version>2.5.12</version>
</dependency>
<dependency>
<groupId>com.oracle.database.r2dbc</groupId>
<artifactId>oracle-r2dbc</artifactId>
<version>1.0.0</version>
</dependency>
I don't know anything about products you tagged (except Oracle) so I don't know whether this will help or not. Hopefully, it will).
FETCH FIRST 2 ROWS ONLY is added to the query and it's the cause of problem.
That's because - I presume - your Oracle database version doesn't support that syntax. I guess it is 11g or lower. See a demo:
Oracle 11g:
SQL> select banner from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE 11.2.0.4.0 Production
TNS for Linux: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production
SQL> SELECT *
2 FROM emp
3 WHERE deptno = 10
4 FETCH FIRST 2 ROWS ONLY;
FETCH FIRST 2 ROWS ONLY
*
ERROR at line 4:
ORA-00933: SQL command not properly ended --> that's your "bad grammar"
Oracle 18c:
SQL> select banner from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 18c Express Edition Release 18.0.0.0.0 - Production
SQL> SELECT *
2 FROM emp
3 WHERE deptno = 10
4 FETCH FIRST 2 ROWS ONLY;
EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO
---------- ---------- --------- ---------- -------- ---------- ---------- ----------
7782 CLARK MANAGER 7839 09.06.81 2450 10
7839 KING PRESIDENT 17.11.81 5000 10
SQL>
So, what can you do? Rewrite the query (if possible). For example:
SQL> WITH
2 temp
3 AS
4 (SELECT e.*, ROW_NUMBER () OVER (ORDER BY NULL) rn
5 FROM emp e
6 WHERE deptno = 10)
7 SELECT *
8 FROM temp
9 WHERE rn <= 2;
EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO RN
---------- ---------- --------- ---------- -------- ---------- ---------- ---------- ----------
7782 CLARK MANAGER 7839 09.06.81 2818 1 10 1
7839 KING PRESIDENT 17.11.81 5750 1 10 2
SQL>
(Note that - without the ORDER BY clause - you can't really know which rows will be returned. If you're happy with any two rows, then OK. If not, include ORDER BY).
I think Littlefoot did a great job explaining why this is causing problems for you. Let me explain why that clause is there in the first place.
selectOne has the following contract:
Execute a SELECT query and convert the resulting item to an entity ensuring exactly one result.
This means it needs to try to fetch at least 2 elements to ensure that there actually isn't a second element.
This could be done by selecting all matching rows.
But it would never look at anything beyond the second row, so all the other rows would be selected in vain.
And possibly worse, the execution plan might differ significantly when the database tries to load all rows vs when it only tries to fetch the first few.
Therefore the limit is applied to the query.
It uses the syntax that works for later versions of Oracle because those are the only ones supported by Spring Data JDBC.
You could avoid the problem by creating and registering your own Dialect that does not support any kind of limiting or paging and replaces ist with a noop.
Of course you need to make sure that you don't try to use such features in your application, since they obviously won't work.
Due to the interesting syntax required by older Oracle versions to implement this correctly it is not easily possible to properly implement paging and limiting for older Oracle versions.
The better option would be to migrate to a recent version of Oracle, although it is understood, that this often isn't easy for a variety of reasons.
Of course you always have the alternative to create your SQL statements in your code using either #Query annotations or custom methods.
If you're willing to use a third party library, you could translate the Oracle 12c compatible SQL using FETCH FIRST to Oracle 11g compatible SQL using jOOQ's ParsingConnection, which acts like an R2DBC Connection proxy, translating all incoming SQL to the target dialect. On Oracle 11g, the FETCH clause will be generated via ROWNUM filtering.
You could obviously also replace your Spring Data usage by jOOQ API usage to get the correct behaviour automatically. jOOQ supports R2DBC out of the box.
If that's not an option, then you'll have to write your SQL manually instead of relying on the DSL API of Spring Data or other third parties.
Disclaimer: I work for the company behind jOOQ.
I have an Oracle SQL Loader control file based on position in a text file. One particular field periodically gets an ellipsis '...' from the source, which causes a carriage return in the loading table. No matter how many times I request '...' to NOT be used by these users, there is eventually someone who forgets, or due to staff turnover, etc. Here is the current control file line for that field:
TRAN_DESC POSITION(153 : 202) Char,
Is there any command that can be added to this line in order to ignore special characters such as an ellipsis?
I'd think of REPLACE. Here's an example.
Sample table:
SQL> create table test (id number, tran_desc varchar2(10));
Table created.
Control file:
load data
infile *
into table test
(id position(1:2),
tran_desc position(3:12) char "replace(:tran_desc, '...', '')"
)
begindata
10LittleFoot
11Big...foot
Loading session and result:
SQL> $sqlldr scott/tiger control=test2.ctl log=test2.log
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Tra 5 17:03:39 2021
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 2
SQL> select * from test;
ID TRAN_DESC
---------- ----------
10 LittleFoot
11 Bigfoot
SQL>
I am using this copy command.
COPY FROM username/[pwd]#identifier INSERT SCHEMA_NAME.TABLE_NAME
USING SELECT * FROM SCHEMA_NAME.TABLE_NAME;
Note : Both the source and target tables are in different databases. The source table has around 19 million records. Both tables have 198 columns.
and I am getting the below message when the copy command is executed (I am not seeing any error message but no rows are copied).
Array fetch/bind size is 5000. (arraysize is 5000)
Will commit after every 100 array binds. (copycommit is 100)
Maximum long size is 80. (long is 80)
0 rows selected from username/[pwd]#identifier.
0 rows inserted into SCHEMA_NAME.TABLE_NAME.
0 rows committed into SCHEMA_NAME.TABLE_NAME at DEFAULT HOST connection.
Please help me on this or any possible guidance to tackle above issue is deeply appreciated.
I tried it on my local 11g XE database; it works OK.
SQL> create table test as select * From dept where 1 = 2;
Table created.
SQL> copy from scott/tiger#xe insert test using select * from dept;
Array fetch/bind size is 15. (arraysize is 15)
Will commit when done. (copycommit is 0)
Maximum long size is 80. (long is 80)
4 rows selected from scott#xe.
4 rows inserted into TEST.
4 rows committed into TEST at DEFAULT HOST connection.
SQL> select * From test;
DEPTNO DNAME LOC
---------- -------------- -------------
10 ACCOUNTING NEW YORK
20 RESEARCH DALLAS
30 SALES CHICAGO
40 OPERATIONS BOSTON
SQL>
Your query selected nothing and inserted nothing, which looks as if query doesn't do what you thought it would. Did you make sure that it is correctly written and that it fetches some rows?
As of disadvantages: Oracle 19c documentation says that
The COPY command will be deprecated in future releases of SQL*Plus. After Oracle 9i, no new datatypes are supported by COPY.
so you'd probably rather use other options to move data around. That would be e.g.
INSERT INTO
MERGE
data pump export & import
Is it possible to load csv file with sqlldr and use sequence at the same time?
Say for example I would like to use command
sqlldr id/pass#'ip:1521/ORCL' data=path\csv_test.csv
control=path\control.ctl log=path\log.log bad=path\bad.csv
to load csv file into database but at the same time use sequence to create an extra column that increments for every csv file insert (so every bulk insert of a csv file)
Sure there's an option; it is called a sequence. More info about it in Field List Reference documentation.
Here's an example.
Data will be loaded into the TEST table:
SQL> create table test
2 (id number,
3 ename varchar2(20)
4 );
Table created.
SQL>
A sequence will be used for the ID column. Control file looks like this:
load data
infile *
replace
into table test
(
id sequence,
ename char terminated by whitespace
)
begindata
Little
Foot
Stack
Over
Flow
Loading session:
M:\a1_maknuto>sqlldr scott/tiger#orcl control=test21.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Vel 19 07:20:29 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
Commit point reached - logical record count 5
Results:
SQL> select * From test;
ID ENAME
---------- --------------------
1 Little
2 Foot
3 Stack
4 Over
5 Flow
SQL>
[EDIT: aha, all rows should share the same "sequence"]
OK then, try something like this (note expression used for the ID column):
load data
infile *
append
into table test
(
id expression "userenv('sessionid')",
ename char(30) terminated by whitespace
)
begindata
Little
Foot
Stack
Over
Flow
A few loading sessions:
SQL> $sqlldr scott/tiger#orcl control=test21.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Vel 19 08:13:23 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
Commit point reached - logical record count 5
SQL> select * From test;
ID ENAME
---------- --------------------
4530297 Little
4530297 Foot
4530297 Stack
4530297 Over
4530297 Flow
SQL> $sqlldr scott/tiger#orcl control=test21.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Vel 19 08:13:30 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
Commit point reached - logical record count 5
SQL> select * From test;
ID ENAME
---------- --------------------
4530297 Little
4530297 Foot
4530297 Stack
4530297 Over
4530297 Flow
4530301 Little
4530301 Foot
4530301 Stack
4530301 Over
4530301 Flow
10 rows selected.
SQL>
Alternatively, you could use a sequence (an Oracle object). That's a little bit more "complex" (you'd need a function too) but - if you need it, I can create an example. Say so.
When I'm trying to gather table statistics using GATHER_TABLE_STATS procedure, I'm getting the following error:
ORA-20001: Invalid input values for pname
ORA-06512: at "SYS.DBMS_STATS", line 31513
ORA-06512: at line 2
The code I'm running to gather statistics is
BEGIN
DBMS_STATS.gather_table_stats ('OWNER', 'TABLE_NAME');
END;
/
My Oracle version is Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
I guess you might have misspelled OWNER or TABLE_NAME parameter. Its working fine for me as shown below.
SQL> conn jay/jay
Connected.
SQL> select table_name from tabs;
TABLE_NAME
------------------------------
ROAD
EVENT
ALL_OBS
ACCOUNT
NVARCHAR2_EMAIL
TABLEA
T2
MYTABLE
8 rows selected.
SQL> exec dbms_stats.gather_table_stats('JAY','ROAD');
PL/SQL procedure successfully completed.
Update
As per the My Oracle Support Doc:755577.1, It is possible that the post installation scripts for the patch were not run correctly after a patch was applied.
You may need to reinitialize the DBMS_STATS package using execstat.sql under $ORACLE_HOME/rdbms/admin directory. Or reinstall DBMS_STATS.
Or you might hit the bug- Bug 14479079 : ORA-20001 GATHERING STATS AFTER CPU JULY 2012 PATCH