I wanted to create an external table, but did not have the CREATE ANY DIRECTORY permission (and could not have it granted). Fair enough, I asked the DBAs to run the following:
CREATE OR REPLACE DIRECTORY ext_data_files AS '/data/ext_data_files';
GRANT ALL ON DIRECTORY ext_data_files TO MYAPPUSER;
They did, and the final object has the following script:
CREATE OR REPLACE DIRECTORY
EXT_DATA_FILES AS
'/data/ext_data_files';
GRANT READ, WRITE ON DIRECTORY SYS.EXT_DATA_FILES TO MYAPPUSER;
(I got that from asking a desc with Toad)
I was then hoping to use this directory to create my external table with the script as follows:
CREATE TABLE MYAPPUSER.MY_EXT_TABLE
(
ID VARCHAR2(100 BYTE),
LOGIN VARCHAR2(100 BYTE),
CODE VARCHAR2(100 BYTE),
CREATED_AT VARCHAR2(100 BYTE)
)
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY SYS.EXT_DATA_FILES
ACCESS PARAMETERS
( RECORDS DELIMITED BY NEWLINE
NOBADFILE
NOLOGFILE
FIELDS TERMINATED BY ';'
MISSING FIELD VALUES ARE NULL
( ID, LOGIN, CODE, CREATED_AT) )
LOCATION (SYS.EXT_DATA_FILES:'the_external_file.txt')
)
REJECT LIMIT 0
PARALLEL ( DEGREE DEFAULT INSTANCES DEFAULT )
NOMONITORING;
but then when I SELECT * FROM MY_EXT_TABLE, the result is the infamous
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file the_external_file.txt in EXT_DATA_FILES not found
ORA-06512: at "SYS.ORACLE_LOADER", line 19
(which has quite a few hits on google, but none seem related)
I'm confident of the syntax since this is the exact same script used in our DEV environment. Also, the permissions of all files and directories involved were checked and there is nothing lower than 775.
The only difference I have here from DEV (where it works) is that the directory EXT_DATA_FILES was not created by MYAPPUSER. I tried to create a synonym for it.. but had no effect.
Maybe worth mentioning, it is Oracle 10g we are talking about.
Am I missing something obvious? Is this allowed?
All directories are in fact owned by SYS. That's why there is no CREATE DIRECTORY privilege, only CREATE ANY DIRECTORY.
So try the command without prefixing the directory name with the SYS schema and see what happens.
The error message reads:
"file the_external_file.txt in EXT_DATA_FILES not found"
Are you sure it's there?
Related
I am trying to get all XML file names present in a directory in order to feed them to a procedure which pulls data out of those files. Could anyone help with how I can get the file name using the EXTERNAL TABLE.
I am having trouble with ACCESS PARAMETERS and LOCATION file. Don't know what exactly would go there.
Thanks
CREATE TABLE S7303786.XML_FILES
(
FILE_NAME VARCHAR2(255 CHAR)
)
ORGANIZATION EXTERNAL
(
TYPE ORACLE_LOADER
DEFAULT DIRECTORY AUTOACCEPT_XMLDIR
ACCESS PARAMETERS
(
RECORDS DELIMITED BY NEWLINE
PREPROCESSOR AUTOACCEPT_XMLDIR: 'list_file.sh'
FIELDS TERMINATED BY WHITESPACE
)
LOCATION ('list_file.sh')
)
REJECT LIMIT UNLIMITED;
list_files.sh just contains the directory where the files are present.
sticky.txt has nothing in it
error I am getting are :
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-29400: data cartridge error
KUP-04004: error while reading file /home/transfer/stu/nshstrans/sticky.txt
Error you got might have something to do with directory, Oracle object which points to physical directory on database server's disk. It is created by a privileged user - SYS, who then grants read and/or write privileges on it to users who will use it.
If you missed to do anything of above mentioned things, your external table won't work.
So:
SQL> show user
USER is "SYS"
SQL>
SQL> create directory mydir as 'c:\temp';
Directory created.
SQL> grant read, write on directory mydir to scott;
Grant succeeded.
SQL>
Connect to Scott and create external table:
SQL> connect scott/tiger
Connected.
SQL> create table extusers
2 (username varchar2(20),
3 country varchar2(20)
4 )
5 organization external
6 (type oracle_loader
7 default directory mydir --> this is directory I created
8 access parameters
9 (records delimited by newline
10 fields terminated by ';'
11 missing field values are null
12 (username char(20),
13 country char(20)
14 )
15 )
16 location ('mydata.txt') --> name of the file that contains data
17 ) -- located in c:\temp, which is MYDIR
18 reject limit unlimited -- directory
19 /
Table created.
SQL>
Contents of the sample text file:
SQL> $type c:\temp\mydata.txt
Littlefoot;Croatia
Michel;France
Maaher;Netherlands
SQL>
Finally, let's select from the external table:
SQL> select * from extusers;
USERNAME COUNTRY
-------------------- --------------------
Littlefoot Croatia
Michel France
Maaher Netherlands
SQL>
Works OK, doesn't it? Now, try to do what I did.
On a second reading,
it appears that you don't want to read file contents, but directory contents. If that's so - apparently, it is - then see whether this helps.
In order to make it work, privileged user has to grant additional privilege - EXECUTE - to the directory.
SQL> show user
USER is "SYS"
SQL> grant execute on directory mydir to scott;
Grant succeeded.
Next step is to create an operating system executable (on MS Windows I use, it is a .bat script; on Unix, that would be a .sh, I think) which will list the directory. Note the first line - I have to navigate to a directory which is source for Oracle directory object. If you don't do that, it won't work. The .bat file is simple:
SQL> $type c:\temp\directory_contents.bat
cd c:\temp
dir /b *.txt
SQL>
Create external table:
SQL> create table extdir
2 (line varchar2(50))
3 organization external
4 (type oracle_loader
5 default directory mydir
6 access parameters
7 (records delimited by newline
8 preprocessor mydir:'directory_contents.bat'
9 fields terminated by "|" ldrtrim
10 )
11 location ('directory_contents.bat')
12 )
13 reject limit unlimited
14 /
Table created.
SQL> connect scott/tiger
Connected.
Let's see what it returns:
SQL> select * From extdir;
LINE
-----------------------------------------------
c:\Temp>dir /b *.txt
a.txt
dept.txt
emp.txt
emps.txt
externalfile1.txt
lab18.txt
mydata.txt
p.txt
parfile_01.txt
sofile.txt
test.txt
test2.txt
15 rows selected.
SQL>
Well ... yes, those are my .txt files located in c:\temp directory.
As you use *nix, I think that problem you got is related to list_files.sh script. You didn't post its contents (which would probably help - not necessarily help me as I forgot almost everything I knew about *.nix), but - regarding Preprocessing External Tables (written by Michael McLaughlin), you might need to
prepend /usr/bin before the ls, find, and sed programs: /usr/bin/ls ...
See if it helps.
Preface
This script has worked previously when importing a csv file into an external table when using a local path for the created directory (with a local drive like F:\stats) instead of a UNC path (\\station-1\...)
Script
CREATE OR REPLACE DIRECTORY csvdir AS '\\Station-1\mainFolder\stats';
DROP TABLE EXT_index;
CREATE TABLE EXT_index(
SESSION_STATION NVARCHAR2(60),
"user" NVARCHAR2(20),
"type" NCHAR(4),
"date" DATE,
"hour" NVARCHAR2(8),
BATCH NUMBER,
STEP NUMBER,
VARIANTE NUMBER,
TIME_MS NUMBER,
NB_FOLDER NUMBER,
NB_DOC NUMBER,
NB_FIELDS NUMBER,
NB_FIELDS_SHOW NUMBER,
NB_FIELDS_CONFIRM NUMBER,
NB_FIELDS_EMPTY NUMBER,
NB_KEY NUMBER,
NB_CHAR NUMBER,
NB_USEFUL_CHAR NUMBER
)organization external (
type oracle_loader
default directory csvdir
access parameters (
records delimited by newline skip 1
fields terminated by ';' lrtrim
missing field values are null (
SESSION_STATION,
"user",
"type",
"date" date 'yyyy-mm-dd',
"hour",
BATCH,
STEP,
VARIANTE,
TIME_MS,
NB_FOLDER,
NB_DOC,
NB_FIELDS,
NB_FIELDS_SHOW,
NB_FIELDS_CONFIRM,
NB_FIELDS_EMPTY,
NB_KEY,
NB_CHAR,
NB_USEFUL_CHAR
)
)
location('INDEX.csv')
)
reject limit unlimited;
DROP TABLE TMP_index;
CREATE TABLE TMP_index AS(SELECT * FROM EXT_index);
Information
Database used : OracleXE 18c
The folder containing the csv we need to import is '\\Station-1\mainFolder\stats' and is situated on a remote station, different than my local computer running the database
Problem
As said previously, this script works fine when csvdir is a local path (F:\mainFolder\stats), the csv is imported into the external table and the temp table can be created from it afterwards just fine.
When csvdir is a UNC path the error is the following : KUP-04027: file name check failed: INDEX.csv
When csvdir is a network drive mapped to the previous UNC path (J:\ mapped to \\Station-1\mainFolder, such as csvdir=J:\stats) the error is the following : KUP-04040: file INDEX.csv in CSVDIR not found
Both these errors occur when trying to create TMP_index from EXT_index :
CREATE TABLE TMP_index AS(SELECT * FROM EXT_index)
Error report -
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
--> Either KUP-04027 or KUP-04040 here, depending on how csvdir was created <--
When checking in SQL Developer, EXT_index is always empty, the table doesn't seem to be created properly (i.e. the Data tab of that table doesn't even show column names)
Steps taken for attempted resolution
The folder '\\Station-1\mainFolder' is shared with 2 users having full control over it : Everyone and User1 (my account)
On the computer that has the database (My local computer) this '\\Station-1\mainFolder' is mapped to a network drive : J:\
When using J:\, csvdir is created as 'J:\stats'
When using \\Station-1\mainFolder, csvdir is created as '\\Station-1\mainFolder\stats'
As per this link both services TNSListener and OracleServiceXE are started logging in to my account, User1
Question
How can I import a csv file situated on a remote server into a local external table?
I have created an external table :
CREATE TABLE XX_Lookup_EXT
(
LOOKUP_TYPE varchar2(200),
LOOKUP_CODE varchar2(200),
MEANING varchar2(200),
ENABLED_FLAG varchar2(10)
)
ORGANIZATION EXTERNAL
( TYPE ORACLE_LOADER
DEFAULT DIRECTORY INTF_DIR1
ACCESS PARAMETERS
( RECORDS DELIMITED BY NEWLINE SKIP 1
NODISCARDFILE
FIELDS TERMINATED BY '|'
OPTIONALLY ENCLOSED BY '"'
MISSING FIELD VALUES ARE NULL
REJECT ROWS WITH ALL NULL FIELDS
)
LOCATION (INTF_DIR1:'LOOKUP_CODE.csv')
)
REJECT LIMIT UNLIMITED
NOPARALLEL
nomonitoring;
When I am querying this table it is giving me the following error :
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
error opening file /orabin/tst/test/XX_LOOKUP_EXT_30723.log
29913. 00000 - "error in executing %s callout"
*Cause: The execution of the specified callout caused an error.
*Action: Examine the error messages take appropriate action.
I have tried everything out. Still I am getting this error.
#alex poole is right. The /orabin/tst/test/ directory must be local to the database and the database server account, usually 'oracle', needs read and write permissions within the directory.
I am trying to create a the following table in postgresql
CREATE TABLE retail_demo.categories_dim_hawq
(
category_id integer NOT NULL,
category_name character varying(400) NOT NULL
)
WITH (appendonly=true, compresstype=quicklz) DISTRIBUTED RANDOMLY;
I am getting the following error:
ERROR: cannot get table space location for content 0 table space 1663
(catalog.c:97)
I tried to create a new tablespace, I got the following:
ERROR: syntax error at or near "LOCATION" LINE 1: create TABLESPACE
moha LOCATION "/tmp/abc";
Thanks in advance,
Moha.
I got the answer
you’ll need to create a filespace, tablespace, database, and then create the table to do this follow the following steps:
12. If you are on the default database (using plsql command), you can get out to root db user (gpadmin) using CTRL + D.
13. gpfilespace -o .
14. enter the name of the filespace: hawqfilespace3
15. Choose filesystem name for this filespace: hdfs
16. Enter replica num for filespace: 0
17. Specify the HDFS location for the segments: bigdata01.intrasoft.com.jo:8020/xd
Note that /xd is one of Hadoop directories which has read write access.
18. The system will generate a configuration command to you, just execute it.
19. Copy and paste the command and click on enter to execute it.
20. The file space is now created successfully.
21. Now connect to the Database using the psql command.
22. Now create a tablespace on the file space you created.
create TABLESPACE hawqtablespace3 FILESPACE hawqfilespace3;
23. Create a database on this tablespace using the command.
CREATE DATABASE hawqdatabase3 WITH OWNER gpadmin TEMPLATE=template0 TABLESPACE hawqtablespace3;
24. Now you need to connect to the database you created, but first click CTRL + D to exit the user you are in.
25. Enter the command psql hawqdatabase3
I'm trying to execute this code in PL/SQL:
create or replace directory ext_tab_dir as 'C:/mydir';
GRANT READ,WRITE ON DIRECTORY ext_tab_dir TO PUBLIC;
DROP TABLE emp_load;
CREATE TABLE emp_load (v1 VARCHAR2(4000),
v2 VARCHAR2(4000)
)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER DEFAULT DIRECTORY ext_tab_dir
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
BADFILE ext_tab_dir:'bad.bad'
LOGFILE ext_tab_dir:'log.log'
FIELDS TERMINATED BY ','
)
LOCATION ('testfile.csv')
);
-- INSERT INTO tablename(v1,v2)
SELECT * From emp_load
and then getting next errors:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error error opening file C:/mydir/log.log
I do get that it has to do something with permissions, but I'm the one who created that directory, so how do I grant priveleges to myself if it is set like this by default? Is there any way to perform that sort of operation from PL/SQL?
Try something like this.
GRANT SELECT, INSERT, UPDATE, DELETE ON emp_load TO NikitaBuriak;
Replace 'NikitaBuriak' with the ID you used when you created the table..
You should grant all the privileges to the directories and files in the location pointing to the file you are trying to access.
eg : if the file you are trying to excess is in /home/dummy_folder/new_folder/file.txt
then you should grant all the administrative privileges to dummy_folder, new folder and file.txt as well