I have a sequence number in my Oracle database that I am trying to fetch and increment using the following query: SELECT TABLE.SEQ.NEXTVAL FROM DUAL where TABLE is the table and SEQ is the sequence. It gets the value and increments it in DBeaver. However, when I run the following code:
QUERY = "SELECT TABLE.SEQ.NEXTVAL FROM DUAL"
sqlContext.read.format("jdbc").options(url=URL, driver=DRIVER,QUERY=QUERY, user=USER,password=PASSWORD).load()
It gives me error:
py4j.protocol.Py4JJavaError: An error occurred while calling o98.load.
: java.sql.SQLSyntaxErrorException: ORA-02287: sequence number not allowed here
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:61)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
The solution is to not query the sequence. Starting with 12.2 Oracle allows an identity column, where the ID is assigned in the INSERTstatement without a need of querying the nextval.
Unfortunately you can't modify existing primary key column to identity column, but you may workaroud the problem of getting the sequence value by setting the sequence as a DEFALULT value of the column - see here for further details.
Example - Table with PK Colun ID filled from sequence SEQ
CREATE TABLE TAB
(id INTEGER,
col1 VARCHAR2(5),
PRIMARY KEY (id));
select * from tab;
ID COL1
---------- -----
1 y
2 z
So the nextvalof the sequence is 3
alter table TAB modify id default SEQ.nextval;
Now you may insert new records in the table and the PK will be assigned from the sequence automatically
insert into tab (col1) values('u');
select * from tab;
ID COL1
---------- -----
1 y
2 z
3 u
You may still use the (same) sequence value explicite in the insert statement (as if you provide a value the DEFAULTclause is not firing).
insert into tab(id,col1) values(seq.nextval,'c');
If you want to assign a sequence nextval in case your application passes NULL such as
insert into tab(id,col1) values(null,'d');
you must use the default on null option
alter table TAB modify id default on null SEQ.nextval;
Related
A column in Oracle DB - table has a NULLABLE = NO and DATA_DEFAULT = 'T'
Using EntifyFramework, When i'm trying to pass a NULL(this is a string property on the entity where NULL is a default value in it) into this field, i'm expecting it to insert a 'T' into this column but instead it is throwing an error.
"The COL_NAME field is required."
That is the expected behaviour. The default value is applied when that column is not referenced in the insert statement. When you do reference the column you are overriding the default - whether that is with null or some other not-null value.
From the documentation for insert:
If you omit one or more of the table's columns from this list, then the column value of that column for the inserted row is the column default value as specified when the table was created or last altered. If any omitted column has a NOT NULL constraint and no default value, then the database returns an error indicating that the constraint has been violated and rolls back the INSERT statement. Refer to CREATE TABLE for more information on default column values.
So you have to omit the column to pick up the default.
Default does not mean 'replace null with this'; it means 'use if not set at all'. Set to null is not the same as not set.
Bear in mind that if that wasn't the case, if you had a nullable column with a default value, it would be impossible to explicitly set it to null on insert.
However, you can override that, at least in later versions, as the create table documentation mentions:
DEFAULT
The DEFAULT clause lets you specify a value to be assigned to the column if a subsequent INSERT statement omits a value for the column.
...
ON NULL
If you specify the ON NULL clause, then Oracle Database assigns the DEFAULT column value when a subsequent INSERT statement attempts to assign a value that evaluates to NULL.
When you specify ON NULL, the NOT NULL constraint and NOT DEFERRABLE constraint state are implicitly specified. If you specify an inline constraint that conflicts with NOT NULL and NOT DEFERRABLE, then an error is raised.
Quick demo:
create table t42 (
id number,
col_a varchar2(1) default 'A' not null,
col_b varchar2 (1) default on null 'B'
);
Table T42 created.
insert into t42 (id) values (1);
1 row inserted.
-- this will fail - the scenario from the question
insert into t42 (id, col_a) values (2, null);
ORA-01400: cannot insert NULL into ("STACK"."T42"."COL_A")
insert into t42 (id, col_b) values (3, null);
1 row inserted.
insert into t42 (id, col_a, col_b) values (4, 'X', null);
1 row inserted.
insert into t42 (id, col_a, col_b) values (5, 'Y', 'Z');
1 row inserted.
select * from t42;
ID C C
---------- - -
1 A B
3 A B
4 X B
5 Y Z
The on null clause was added in 12cR1, so if you're still on an older version you won't be able to use it. But if your on a version that supports it, changing your table definition will allow your explicit null to do what you want.
Hi I want to capture all the Oracle Errors for my DML operations in the manually created table with columns as ErrorID and Error_Descr.
How to get ORA_ERR_NUMBER$ and ORA_ERR_MESG$ values in the above columns?
This table contains user defined errors as well so I do not want to limit it to the Oracle Errors.
Is there any way of capturing Oracle as well as User Defined Errors in the User Defined Tables?
Thanks in Advance!
As per documentation Link,
Oracle allows you to use a manually created table for LOGGING only if you have included these mandatory columns.
ORA_ERR_NUMBER$
ORA_ERR_MESG$
ORA_ERR_ROWID$
ORA_ERR_OPTYP$
ORA_ERR_TAG$
If you want other columns to capture the information in those two columns, you could make them as virtual columns.
CREATE TABLE my_log_table (
ORA_ERR_NUMBER$ NUMBER
,ORA_ERR_MESG$ VARCHAR2(2000)
,ORA_ERR_ROWID$ ROWID
,ORA_ERR_OPTYP$ VARCHAR2(2)
,ORA_ERR_TAG$ VARCHAR2(2000)
,ErrorID NUMBER AS (COALESCE(ORA_ERR_NUMBER$, ORA_ERR_NUMBER$))
,Error_Descr VARCHAR2(2000) AS (COALESCE(ORA_ERR_MESG$, ORA_ERR_MESG$))
);
using COALESCE is a hack because Oracle doesn't allow you to have one column default to another directly.
Now, you could run your error logging dml normally mentioning the logging table name.
INSERT INTO t_emp
SELECT employee_id * 10000
,first_name
,last_name
,hire_date
,salary
,department_id
FROM hr.employees
WHERE salary > 10000 LOG ERRORS
INTO my_log_table('ERR_SAL_LOAD') REJECT LIMIT 25
0 row(s) inserted.
select ORA_ERR_TAG$,ErrorID,Error_Descr FROM my_log_table ;
ORA_ERR_TAG$ ERRORID ERROR_DESCR
ERR_SAL_LOAD 1438 ORA-01438: value larger than specified precision allowed for this column
ERR_SAL_LOAD 1438 ORA-01438: value larger than specified precision allowed for this column
I created Hive custom UDTF. My new UDTF returns new 3 columns based on input 2 columns.
But, I can't any INSERT operation by using generated values.
For example,
INSERT OVERWRITE DIRECTORY 'generated_data.csv' SELECT udtf(one, two) FROM table_orig;
INSERT OVERWRITE TABLE test_table SELECT udtf(one, two) FROM table_orig;
Both of INSERT queries returns NullPointerException like following:
2017-05-30T08:02:45,209 ERROR [main([])]: exec.Task (:()) - Failed to execute tez graph.
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:430)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:451)
at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:396)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:134)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Any suggestion is welcome, thank you !
Add columns list after UDTF:
INSERT OVERWRITE TABLE test_table SELECT udtf(one, two) as (col1, col2, col3) FROM table_orig;
I have an insert statement that I am trying to run against an Oracle database. The insert statement is very simple, and I only want to insert a value into one column. The table itself has more than one column, but I am only interested in filling one of the columns with data. The format of my query is similar to the one below:
insert into myTable (col1) Values (val1)
However, this throws the following error:
ORA-00904: "col1": invalid identifier
I've checked to make sure that the column is named correctly, and my only other thought is that something is wrong with my syntax. There are no constraints on the table such as primary keys. Is it possible to only insert values into certain columns when executing an insert statement in Oracle?
Check that you didn't quote the column name on creation of the table. If you did, the column name is stored as you quoted it. For example:
create table table1 (
id number(2)
)
has a different column name to this
create table table2 (
"id" number(2)
)
Oracle by default stores the (unquoted) column names in uppercase. Quoted are stored as is.
You can use a DESC table_name to see how the columns are stored.
The following
select id from table1
select iD from table1
select ID from table1
fetch the records, while only
select "id" from table2
will fetch records.
select id from table2
will throw the ORA-00904 : "ID" : invalid identifier error.
You may have inadvertently done the quoting during creation while using tools as established below:
https://community.oracle.com/thread/2349926
Btw: Yes it is possible to insert records for only one column, as long as the other columns do not have a NOT NULL constraint on them.
Actually, I think you may be double quoting the column in the insert statement (not on table creation), although your example is misleading. I guess this because of your error, which says "col1" is invalid, not "COL1" is invalid. Consider this simple test:
SQL> create table mytable
(
col1 varchar2(10)
)
Table created.
SQL> -- incorrect column name, without double quotes
SQL> insert into mytable(col2) values ('abc')
insert into mytable(col2) values ('abc')
Error at line 9
ORA-00604: error occurred at recursive SQL level 1
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at line 11
ORA-00904: "COL2": invalid identifier
SQL> -- incorrect column name, with double quotes
SQL> insert into mytable("col2") values ('abc')
insert into mytable("col2") values ('abc')
Error at line 12
ORA-00604: error occurred at recursive SQL level 1
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at line 11
ORA-00904: "col2": invalid identifier
SQL> -- correct column name, without double quotes (works)
SQL> insert into mytable(col1) values ('abc')
1 row created.
SQL> -- correct column name, with double quotes (fails)
SQL> insert into mytable("col1") values ('abc')
insert into mytable("col1") values ('abc')
Error at line 18
ORA-00604: error occurred at recursive SQL level 1
ORA-06502: PL/SQL: numeric or value error: character string buffer too small
ORA-06512: at line 11
ORA-00904: "col1": invalid identifier
The last failed insert attempt is what I think you may be doing:
insert into mytable("col1") values ...
based on the error message:
ORA-00904: "col1": invalid identifier
So the solution would simply be remove the double quoting around the column name in the insert.
It's most probably a syntax error
Desc myTable;
insert into myTable (col1) Values ('val1')
Ensure col1 is a valid column in the table, and you're simply not trying to say 'select the left-most column.
edit: Is it possible to only insert values into certain columns when executing an insert statement in Oracle?
Yes if you want to only insert into certain columns then simply specify it
e.g.
insert into myTable (col1, col2, col6) Values ('val1', 'val2', 'val3');
This will only work if the column itself doesn't have a NOT NULL constraint - in which case it will not allow a value to be enetered (unless again there's a default value).
I have external tables. And I'd like to extract the data from those tables and insert/merge that data in other tables.
Now when a select from => insert into query or merge query runs then it is possible (and likely possible) that the data might be in a bad quality, which will result into breaking query. Say there is 000000 as date in external table which will result breaking query if I am merging data.
How can I log these errors in a table (for example) error table which will log the error, reason of error, line number and column name? Just like you see in SQL Loader logs. For example:
Record 2324: Rejected - Error on table AA_STAG_VR_01, column KS1.
ORA-01843: not a valid month
And the query shouldn't break upon rather. Rather log the error and move on like it happens in SQL Loader.
Is it possible? I tried to look around net but I wasn't unable to find anything, or maybe I simply don't know the magic words
Thanks in advance :-)
EDIT:
Ok, I was able to solve the problem (well, partly) using the following approach.
CREATE TABLE error_table (
ora_err_number$ NUMBER,
ora_err_mesg$ VARCHAR2(2000),
ora_err_rowid$ ROWID,
ora_err_optyp$ VARCHAR2(2),
ora_err_tag$ VARCHAR2(2000)
)
INSERT INTO destination_table (column)
SELECT column FROM external_table
LOG ERRORS INTO error_table REJECT LIMIT UNLIMITED
This gives me:
SELECT * FROM error_table;
----------------------------------------------------------------------------------------------------------------------------------------------------------
ORA_ERR_NUMBER$ | ORA_ERR_MESG$ | ORA_ERR_ROWID$ | ORA_ERR_OPTYP$ | ORA_ERR_TAG$ |
----------------------------------------------------------------------------------------------------------------------------------------------------------
12899 |ORA-12899: value too large for column "SYSTEM"."destination_table"."column"
So far, so good. However, I would like to know what record number (line number in external_table) has this error. Because it is possible that fist 10 records went ok but 11th record was bad.
Check out FORALL + SAVE EXCEPTIONS clause. It might help you.
15:57:02 #> conn hr/hr#vm_xe
Connected.
15:57:15 HR#vm_xe> create table err_test(unique_column number primary key);
Table created.
Elapsed: 00:00:01.51
15:57:46 HR#vm_xe> EXECUTE DBMS_ERRLOG.CREATE_ERROR_LOG('err_test', 'errlog');
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.46
15:59:22 HR#vm_xe> insert into err_test select mod(rownum, 2) from dual connect by rownum < 10
16:00:00 2 log errors into errlog ('test') reject limit unlimited;
2 rows created.
Elapsed: 00:00:00.87
16:00:27 HR#vm_xe> commit;
Commit complete.
Elapsed: 00:00:00.00
16:02:37 HR#vm_xe> col ora_err_mesg$ for a75
16:02:43 HR#vm_xe> col unique_column for a10
16:02:47 HR#vm_xe> select unique_column, ora_err_mesg$ from errlog;
UNIQUE_COL ORA_ERR_MESG$
---------- ---------------------------------------------------------------------------
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
0 ORA-00001: unique constraint (HR.SYS_C007056) violated
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
0 ORA-00001: unique constraint (HR.SYS_C007056) violated
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
0 ORA-00001: unique constraint (HR.SYS_C007056) violated
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
7 rows selected.
Elapsed: 00:00:00.03
Below some syntax, you have reject limit as in sqlloader, you have log files, bad files etc.
CREATE TABLE <table_name> (
<column_definitions>)
ORGANIZATION EXTERNAL
(TYPE oracle_loader
DEFAULT DIRECTORY <oracle_directory_object_name>
ACCESS PARAMETERS (
RECORDS DELIMITED BY newline
BADFILE <file_name>
DISCARDFILE <file_name>
LOGFILE <file_name>
[READSIZE <bytes>]
[SKIP <number_of_rows>
FIELDS TERMINATED BY '<terminator>'
REJECT ROWS WITH ALL NULL FIELDS
MISSING FIELD VALUES ARE NULL
(<column_name_list>))\
LOCATION ('<file_name>'))
[PARALLEL]
REJECT LIMIT <UNLIMITED | integer>;
See here some examples, and here the docs