I have a place.file text file;
place.file
New Hampshire
New Jersey
New Mexico
Nevada
New York
Ohio
Oklahoma
....
There are 4000 place names in this file. I will match my my_place table in oracle and place.file . So I want to insert the place.file into the Oracle . Maybe I should use bulk insert, how can I do bulk insert ?
You can use SQL Loader from Oracle.
The syntax is:
sqlldr *connection_string* control=*control_file.ctl*
The control file contains:
LOAD DATA
INFILE names.file
INTO TABLE <table_name>
FIELDS TERMINATED BY <delimiter>
OPTIONALLY ENCLOSED BY <enclosing character>
(<column_name>[, <column_name>, <column_name>])
No mention of an Oracle version. (For the best possible answer, always include Oracle version, Oracle edition, OS, and OS version.)
However, you should investigate using an external table for this purpose. Once you have that set up correctly, you can do:
insert into db_table select ... from external_table;
Optionally, you could use the APPEND hint on the INSERT statement, to use direct load.
Also,optionally, you could set the NOLOGGING attribute on the table you're loading the data into, for best performance. But, consider the recovery implications before you enable NOLOGGING.
Hope that helps,
-Mark
Related
Is there any subprogram similar to DBMS_METADATA.GET_DDL that can actually export the table data as INSERT statements?
For example, using DBMS_METADATA.GET_DDL('TABLE', 'MYTABLE', 'MYOWNER') will export the CREATE TABLE script for MYOWNER.MYTABLE. Any such things to generate all data from MYOWNER.MYTABLE as INSERT statements?
I know that for instance TOAD Oracle or SQL Developer can export as INSERT statements pretty fast but I need a more programmatically way for doing it. Also I cannot create any procedures or functions in the database I'm working.
Thanks.
As far as I know, there is no Oracle supplied package to do this. And I would be skeptical of any 3rd party tool that claims to accomplish this goal, because it's basically impossible.
I once wrote a package like this, and quickly regretted it. It's easy to get something that works 99% of the time, but that last 1% will kill you.
If you really need something like this, and need it to be very accurate, you must tightly control what data is allowed and what tools can be used to run the script. Below is a small fraction of the issues you will face:
Escaping
Single inserts are very slow (especially if it goes over a network)
Combining inserts is faster, but can run into some nasty parsing bugs when you start inserting hundreds of rows
There are many potential data types, including custom ones. You may only have NUMBER, VARCHAR2, and DATE now, but what happens if someone adds RAW, BLOB, BFILE, nested tables, etc.?
Storing LOBs requires breaking the data into chunks because of VARCHAR2 size limitations (4000 or 32767, depending on how you do it).
Character set issues - This will drive you ¿¿¿¿¿¿¿ insane.
Enviroment limitations - For example, SQL*Plus does not allow more than 2500 characters per line, and will drop whitespace at the end of your line.
Referential Integrity - You'll need to disable these constraints or insert data in the right order.
"Fake" columns - virtual columns, XML lobs, etc. - don't import these.
Missing partitions - If you're not using INTERVAL partitioning you may need to manually create them.
Novlidated data - Just about any constraint can be violated, so you may need to disable everything.
If you want your data to be accurate you just have to use the Oracle utilities, like data pump and export.
Why don't you use regular export ?
If you must you can generate the export script:
Let's assume a Table myTable(Name VARCHAR(30), AGE Number, Address VARCHAR(60)).
select 'INSERT INTO myTable values(''' || Name || ','|| AGE ||',''' || Address ||''');' from myTable
Oracle SQL Developer does that with it's Export feature. DDL as well as data itself.
Can be a bit unconvenient for huge tables and likely to cause issues with cases mentioned above, but works well 99% of the time.
I've been searching for a while now but can't seem to find answers so here goes...
I've got a CSV file that I want to import into a table in Oracle (9i/10i).
Later on I plan to use this table as a lookup for another use.
This is actually a workaround I'm working on since the fact that querying using the IN clause with more that 1000 values is not possible.
How is this done using SQLPLUS?
Thanks for your time! :)
SQL Loader helps load csv files into tables: SQL*Loader
If you want sqlplus only, then it gets a bit complicated. You need to locate your sqlloader script and csv file, then run the sqlldr command.
Another solution you can use is SQL Developer.
With it, you have the ability to import from a csv file (other delimited files are available).
Just open the table view, then:
choose actions
import data
find your file
choose your options.
You have the option to have SQL Developer do the inserts for you, create an sql insert script, or create the data for a SQL Loader script (have not tried this option myself).
Of course all that is moot if you can only use the command line, but if you are able to test it with SQL Developer locally, you can always deploy the generated insert scripts (for example).
Just adding another option to the 2 already very good answers.
An alternative solution is using an external table: http://www.orafaq.com/node/848
Use this when you have to do this import very often and very fast.
SQL Loader is the way to go.
I recently loaded my table from a csv file,new to this concept,would like to share an example.
LOAD DATA
infile '/ipoapplication/utl_file/LBR_HE_Mar16.csv'
REPLACE
INTO TABLE LOAN_BALANCE_MASTER_INT
fields terminated by ',' optionally enclosed by '"'
(
ACCOUNT_NO,
CUSTOMER_NAME,
LIMIT,
REGION
)
Place the control file and csv at the same location on the server.
Locate the sqlldr exe and invoce it.
sqlldr userid/passwd#DBname control=
Ex : sqlldr abc/xyz#ora control=load.ctl
Hope it helps.
Somebody asked me to post a link to the framework! that I presented at Open World 2012. This is the full blog post that demonstrates how to architect a solution with external tables.
I would like to share 2 tips: (tip 1) create a csv file (tip 2) Load rows from a csv file into a table.
====[ (tip 1) SQLPLUS to create a csv file form an Oracle table ]====
I use SQLPLUS with the following commands:
set markup csv on
set lines 1000
set pagesize 100000 linesize 1000
set feedback off
set trimspool on
spool /MyFolderAndFilename.csv
Select * from MYschema.MYTABLE where MyWhereConditions ;
spool off
exit
====[tip 2 SQLLDR to load a csv file into a table ]====
I use SQLLDR and a csv ( comma separated ) file to add (APPEND) rows form the csv file to a table.
the file has , between fields text fields have " before and after the text
CRITICAL: if last column is null there is a , at the end of the line
Example of data lines in the csv file:
11,"aa",1001
22,"bb',2002
33,"cc",
44,"dd",4004
55,"ee',
This is the control file:
LOAD DATA
APPEND
INTO TABLE MYSCHEMA.MYTABLE
fields terminated by ',' optionally enclosed by '"'
TRAILING NULLCOLS
(
CoulmnName1,
CoulmnName2,
CoulmnName3
)
This is the command to execute sqlldr in Linux. If you run in Windows use \ instead of / c:
sqlldr userid=MyOracleUser/MyOraclePassword#MyOracleServerIPaddress:port/MyOracleSIDorService DATA=datafile.csv CONTROL=controlfile.ctl LOG=logfile.log BAD=notloadedrows.bad
Good luck !
From Oracle 18c you could use Inline External Tables:
Inline external tables enable the runtime definition of an external table as part of a SQL statement, without creating the external table as persistent object in the data dictionary.
With inline external tables, the same syntax that is used to create an external table with a CREATE TABLE statement can be used in a SELECT statement at runtime. Specify inline external tables in the FROM clause of a query block. Queries that include inline external tables can also include regular tables for joins, aggregation, and so on.
INSERT INTO target_table(time_id, prod_id, quantity_sold, amount_sold)
SELECT time_id, prod_id, quantity_sold, amount_sold
FROM EXTERNAL (
(time_id DATE NOT NULL,
prod_id INTEGER NOT NULL,
quantity_sold NUMBER(10,2),
amount_sold NUMBER(10,2))
TYPE ORACLE_LOADER
DEFAULT DIRECTORY data_dir1
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY '|')
LOCATION ('sales_9.csv') REJECT LIMIT UNLIMITED) sales_external;
Here is my problem, I wants to create a baseline on our development Dateabase (Oracle 10g), and check into our svn for version control, and after this we will use liquibase to help us manage the incremental database changes.
My problem is how should I create baseline of Oracle 10g? the database now consists of 500+ tables, with large amount of configuration data, and I wants my db baseline to base on a set SQL scripts to check into subversion, rather then check in Oracle dump..
I have try use liquibase generateChangeLog, but it have some performance problem.. can anyone can recommends me any tools that will help me
1. Scan any Oracle Schema
2. Generate a set of SQL Scripts (With Table structures, and Data)..
Thanks in advance
James!
Something like
SELECT DBMS_METADATA.GET_DDL('TABLE',table_name) FROM USER_TABLES;
is a good start. You can tweak it with PL/SQL and UTL_FILE to get it to write each table to a different file. You will probably need to do sequences too (though versioning them is fairly pointless), and maybe triggers/procedures/functions/packages etc.
Don't forget grants.
Have you tried Oracle's free SQLDeveloper tool? It gives you the possibility of exporting DDL and data.
EXPDP with CONTENT=METADATA_ONLY option, then IMPDP with SQLFILE=your_script.sql ?
Nicolas.
More general solution would be to dump DDL sql for selected list of tables, but additionally also other types of objects. This could be done by using all_objects and all_users views.
Example that worked for me:
select dbms_metadata.GET_DDL(u.object_type,u.object_name, u.owner)
from all_objects u
where 1=1
-- filter only selected object types
and u.object_type in ('TABLE', 'INDEX', 'FUNCTION', 'PROCEDURE', 'VIEW',
'TYPE', 'TRIGGER', 'SEQUENCE')
-- don't want system objects, generated, temp, invalid etc.
and u.object_name not like 'SYS_%'
and temporary!='Y'
and generated!='Y'
and status!='INVALID'
and u.object_name not like 'TMP_%'
and u.object_name not like '%$%'
-- if you want to filter only changed from some date/timestamp:
-- and u.last_ddl_time > '2014-04-02'
-- filter by owner
and owner in (
select username from dba_USERS where DEFAULT_TABLESPACE not like 'SYS%'
and username not in ('ORACLE_OCM')
and username not like '%$%'
)
;
I wrote a python script that refreshes db schema in incremental mode based on similar sql:
runs sql with last_ddl_time>=max(last_ddl_time from last refresh)
at the end stores last_ddl_time somewhere in filesystem for next refresh
References:
oracle dbms_metadata.GET_DDL function
oracle all_objects view
Can any one give me the syntax to truncate a table in IBM DB2.
I m running the following command: truncate table tableName immediate;
The eror is DB2
SQLCODE=-104, SQLSTATE=42601, SQLERRMC=table;truncate ;JOIN , DRIVER=3.50.152
Message: An unexpected token "table" was found following "truncate ". Expected tokens may include: "JOIN ".. SQLCODE=-104, SQLSTATE=42601, DRIVER=3.50.152
The syntax matches the one specified in the reference docs of IBM : http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.db29.doc.sqlref/db2z_sql_truncate.htm
There is a great article about truncating, here is the Gist of DB2 stuff
Almost follows the standard.(since version 9.7)
DB2 requires that the IMMEDIATE keyword be added the the ordinary TRUNCATE TABLE statement, e.g.:
TRUNCATE TABLE someschema.sometable IMMEDIATE
TRUNCATE TABLE must be the first statement in a transaction. A transaction starting with TRUNCATE TABLE may include other statements, but if the transaction is rolled back, the TRUNCATE TABLE operation is not undone.
DB2s TRUNCATE TABLE operation has a number of optional arguments, see the documentation for more on this; especially, the REUSE STORAGE argument may be important for ad-hoc DBA tasks.
In DB2 versions < 9.7, you may abuse the IMPORT statement. Unfortunately, you need to know which operating system the command is executed from for this to work:
On unix-like systems:
IMPORT FROM /dev/null OF DEL REPLACE INTO tablename
On Windows:
IMPORT FROM NUL OF DEL REPLACE INTO tablename
IMPORT cannot be abused in all contexts. E.g., when working with dynamic SQL (from Java/.NET/PHP/...—not using the db2 command line processor), you need to wrap the IMPORT command in a call to ADMIN_CMD, e.g.:
CALL ADMIN_CMD('IMPORT FROM /dev/null OF DEL REPLACE INTO tablename')
IMPORT seems to be allowed in a transaction involving other operations, however it implies an immediate COMMIT operation.
The ALTER TABLE command may also be abused to quickly empty a table, but it requires more privileges, and may cause trouble with rollforward recovery.
This was taken from the website:
http://troels.arvin.dk/db/rdbms/#bulk-truncate_table-db2
If you are using DB2 for AS400, IMMEDIATE TRUNCATE TABLE will NOT work. The equivallent work around is to either:
DELETE FROM [tableName] then if it is an auto increment equivalant column, run:
ALTER TABLE ALTER COLUMN RESTART WITH 1
OR the faster (most efficient way)
Pass a command to the system to clear out the Physical File
Java syntax:
CommandCall command = new CommandCall(new AS400(AS400SystemName, AS400JavaUser, AS400JavaPwd));
try {
command.run("CLRPFM FILE(as400SchemaName/" + tableName + ")");
Which version of DB2 are you using? The truncate table command was introduced in DB2 v9 (at least on the mainframe, which appears to be what you're asking about based on the link).
You may have to resort to the delete from option although this article gives a stored procedure way of doing it in DB2 v8.
use truncate 'table_name' immediate
This is the exact reference documentation available for TRUNCATE in DB2 from 9.7 version
DB2 Reference for TRUNCATE
DB2 on z/OS V10
Empty one of your tables: truncate table; followed by commit work. Ex. truncate temp;
Someone else table: truncate owner.table Ex: truncate student.work ;
I have not tried this on a linked DB2. I do not know if truncate node2.student.work; is good.
SQL for creating list of tables automatically. Substring (substr) used because column width for table name and creator are sooo long. Your values may be different.
select 'truncate table '||substr(creator,1,9)||'.'||substr(name,1,20)
from sysibm.systables
where creator = 'Student';
in Java make sure it's the first statement in the transaction
This question already has answers here:
Load Excel data sheet to Oracle database
(6 answers)
Closed 8 years ago.
How to copy the data from Excel to oracle?
There are many different methods, depending
upon the amount of data, the repetitiveness
of the process, and the amount of programming
I am willing to invest.
First, create the Oracle table, using the
SQL CREATE TABLE statement to define the table's
column lengths and types. Here's an example of a
sqlplus 'CREATE TABLE' statement:
CREATE TABLE SPECIES_RATINGS
(SPECIES VARCHAR2(10),
COUNT NUMBER,
RATING VARCHARC2(1));
Then load the data using any of the following
methods or an entirely new method you invent:
--------------------------------------------
First load method:
I use the SQL*Loader method.
You will need to save a copy of your spreadsheet
in a text format like CSV or PRN.
SQL*Loader Control file for CSV file:
load data
infile 'c:\data\mydata.csv'
into table emp
fields terminated by "," optionally enclosed by '"'
( empno, empname, sal, deptno )
There are some GUIs that have wizards to walk you through the
process (Enterprise Manager -> Maintenance -> Data Movement ->
Move Row Data -> Load Data from User Files) for the
ad-hoc imports. Toad for Oracle has a SQL*Loader Wizard as
well. (DBA -> Data Import/Export -> SQL*Loader Wizard)
You can save your Excel data in PRN format if you are
planning to use positional data (fixed length) in your
control file.
SQL*Loader Control file for PRN file:
load data
infile 'c:\data\mydata.prn'
replace
into table departments
( dept position (02:05) char(4),
deptname position (08:27) char(20) )
Position(02:05) will give the 2nd to the 5th character
Once I've gone through the EM or Toad wizard, I save
the control file, tweak it as needed in a text editor,
and reuse it in SQL*Plus scripts.
SQL*Loader is handy also since it allows you to
skip certain data and call filter functions (i.e.
native functions as in DECODE() or TO_DATE() or
user defined functions) in your control .ctl file.
You can load from multiple input files provided
they use the same record format by repeating the
INFILE clause. Here is an example:
LOAD DATA
INFILE file1.prn
INFILE file2.prn
INFILE file3.prn
APPEND
INTO TABLE emp
( empno POSITION(1:4) INTEGER EXTERNAL,
ename POSITION(6:15) CHAR,
deptno POSITION(17:18) CHAR,
mgr POSITION(20:23) INTEGER EXTERNAL
)
You can also specify multiple "INTO TABLE" clauses
in the SQL*Loader control file to load into multiple
tables.
LOAD DATA
INFILE 'mydata.dat'
REPLACE
INTO TABLE emp
WHEN empno != ' '
( empno POSITION(1:4) INTEGER EXTERNAL,
ename POSITION(6:15) CHAR,
deptno POSITION(17:18) CHAR,
mgr POSITION(20:23) INTEGER EXTERNAL
)
INTO TABLE proj
WHEN projno != ' '
( projno POSITION(25:27) INTEGER EXTERNAL,
empno POSITION(1:4) INTEGER EXTERNAL
)
With SQL*Loader, you can selectively load only
the records you need (see WHEN clause), skip
certain columns while loading data (see FILLER
columns) and load multi-line records (see
CONCATENATE and CONTINUEIF)
Once you've created the control file, you need
to start sql loader from the command line like this:
sqlldr username/password#connect_string control=ctl_file.ctl log=log.log
You can create a batch file to call sqlldr.
For more examples, see
http://examples.oreilly.com/orsqlloader/
That's it for the versatile SQL*Loader.
--------------------------------------------
Second load method:
In this scenario, I have full control of the
spreadsheet, but less control of the data because
users send me the spreadsheets back with data.
I create another worksheet within the same Excel
file, which has locked down INSERT statements
referring back to the sheet with the data. When
I receive the spreadsheet, I copy and paste the
INSERT statements directly into SQL*Plus, or
indirectly staging them in a SQL script.
Excel is a great tool for composing dynamic
SQL statements dynamically. (see Excel functions)
--------------------------------------------
Third load method:
If you need a utility to load Excel data into
Oracle, download quickload from sourceforge at
http://sourceforge.net/projects/quickload
--------------------------------------------
Fourth load method:
In theory, this should work.
Configure Generic Database connectivity (Heterogeneous Database HS)
Connect to the Excel spreadsheet from Oracle through ODBC.
Describe it (see DESC command) or
CREATE TABLE AS SELECT col1, col2 FROM ExcelTable
to make a copy and see what data types Oracle assigns
the columns by default.
http://www.e-ammar.com/Oracle_TIPS/HS/configuring_generic_database_con.htm
--------------------------------------------
References:
http://209.85.173.132/search?q=cache:GJN388WiXTwJ:www.orafaq.com/wiki/SQL*Loader_FAQ+Oracle+control+file+columns&cd=3&hl=en&ct=clnk&gl=us
http://forums.oracle.com/forums/thread.jspa?threadID=305918&tstart=0
http://techrepublic.com.com/5208-6230-0.html?forumID=101&threadID=223797&messageID=2245485
http://examples.oreilly.com/orsqlloader/
A DBA once showed me an easy trick:
In someplace like another sheet, create a formula like:
INSERT INTO my_table (name, age, monkey) VALUES ('" & A1 & "', " & B1 & ", '" & C1 & "');"
Copy/paste it into the appropriate rows (Excel automatically changes your formula to A2, A3, etc.)
Then copy/paste the result into sqlplus.
The simplest way I can think of is to put Access in the middle. Attach to Excel (or import the data into Access); then attach to the destination Oracle tables and copy. The Access Export facility also works pretty well.
Use external tables
Perhaps some combination of DBD::Oracle, DBD::Excel and DBIx::Copy? But surely there's an easier way...
If its a once off, or rare thing, and you can export to csv, then the Application Express or SQL Loader facilities would work fine. If its a regular thing, then Chris's suggestion is what I'd go with.