I have a shell with one command line opening sql plus and calling a sql file
$ORACLE_HOME/bin/sqlplus id/psw #SQL_DIRECTORY/myfile.sql
in this file I write this :
spool myfile.csv
select 'column 01', ';', 'column 02' from dual;
select myColumn1, ';', mycolum2 from mytable
spool off
It works good BUT on the first line of the the first column of each select there is a special character.
Why is that? how can I get rid of this
Thanks
From the description it sounds like you have a login.sql or glogin.sql user profile that is issuing set newpage 0:
Sets the number of blank lines to be printed from the top of each page to the top title. A value of zero places a formfeed at the beginning of each page (including the first page) and clears the screen on most terminals. If you set NEWPAGE to NONE, SQL*Plus does not print a blank line or formfeed between the report pages.
In Excel that does show up as an unprintable character with a question mark in a small square; in Notepad++ it seems to show as FF in reverse colouring; and in some editors (e.g. Vim) it shows as ^L. This is the ASCII form feed character, decimal 12 or 0xC.
You can either reset that with set newpage none, or make it irrelevant with set pagesize 0, which has the convenient side effect of removing the column headers and separators. You may also want to set feedback off if you aren't already.
Samples with contrasting settings:
set newpage 0;
set feedback on;
set pagesize 100;
spool ctrl.csv
select 'head 1;head 2' from dual;
select sysdate, ';', systimestamp from dual;
spool off
^L'HEAD1;HEAD2'
-------------
head 1;head 2
1 row selected.
^LSYSDATE ' SYSTIMESTAMP
------------------- - ---------------------------------------------------------------------------
2015-09-16 15:45:42 ; 16-SEP-15 15.45.42.333627 +01:00
1 row selected.
And
set newpage none;
set feedback off;
set pagesize 0;
spool ctrl.csv
select 'head 1;head 2' from dual;
select sysdate, ';', systimestamp from dual;
spool off
head 1;head 2
2015-09-16 15:46:11 ; 16-SEP-15 15.46.11.274863 +01:00
To make a long story short, I started getting ORA-00001 primary key violations and I tracked down the issue to the fact that some of my INSERT INTO statements were running twice. I then discovered that the offending commands had a C-style comment afterwards:
WHENEVER SQLERROR EXIT FAILURE
SET ECHO OFF
SET HEADING OFF
SET PAGESIZE 0
SET FEEDBACK OFF
SET TIMING OFF
SET TIME OFF
SET TRIMSPOOL ON
SET TRIMOUT ON
SET LINESIZE 120
SET SQLBLANKLINES ON
SET SERVEROUTPUT ON
[...]
INSERT INTO INF_FIELD (FIELD_ID, CATEGORY_ID, COLUMN_EXPRESSION, DISPLAY_NAME, SORT_ORDER) VALUES (17, 1, 'FOO.NAME', 'Name of the foo', 17);
/*This is a comment*/
It was then easily fixed by switching to this syntax:
--This is a comment
What's the exact reason why /*...*/ comments were making SQL*Plus run the statement twice?
/* This is a comment */
Just make sure you have a space after /* ,
So it is treated as a single/multi line comment. And not mean to execute the last stored PL/SQL or SQL
To put it in detail.
What ever SQL*Plus interprets after / is ignored and it blindly pushes it's cached block into the server. Except for /* followed by a new line or space.
SQL> SELECT * FROM DUAL;
D
-
X
SQL> /*t*/
D
-
X
SQL> /*
SQL> */
SQL> /
D
-
X
SQL> /*s
D
-
X
From Document:
You must enter a space after the slash-asterisk(/*) beginning a
comment.
I am required to spool a csv from a table in Oracle, using sqlplus. Following is the format required:
"HOST_SITE_TX_ID","SITE_ID","SITETX_TX_ID","SITETX_HELP_ID"
"664436565","16","2195301","0"
"664700792","52","1099970","0"
Following is the relevant piece of the shell script I wrote:
sqlplus -s $sql_user/$sql_password#$sid << eof >> /dev/null
set feedback off
set term off
set linesize 1500
set pagesize 11000
--set colsep ,
--set colsep '","'
set trimspool on
set underline off
set heading on
--set headsep $
set newpage none
spool "$folder$filename$ext"
select '"'||PCL_CARRIER_NAME||'","'||SITETX_EQUIP_ID||'","'||SITETX_SITE_STAT||'","'||SITETX_CREATE_DATE||'","'||ADVTX_VEH_WT||'"'
from cvo_admin.MISSING_HOST_SITE_TX_IDS;
spool off
(I have used some commented statements in, to signify the things that I tried but couldn't get to work)
The output I receive is:
'"'||PCL_CARRIER_NAME||'","'||SITETX_EQUIP_ID||'","'||SITETX_SITE_STAT||'","'||SITETX_CREATE_DATE||'","'||ADVTX_VEH_WT||'"'
"TRANSPORT INC","113","00000000","25-JAN-13 10.17.51 AM",""
"TRANSPORT INC","1905","00000000","25-JAN-13 05.06.44 PM","0"
Which shows that the header is messed up - it is literally printing the whole string that should have been interpreted as an sql statement, as is the case with the data displayed.
Options I am considering:
1) Using colsep
set colsep '","'
spool
select * from TABLE
spool off
This introduces other problems as the data having leading and trailing spaces, first and the last values in the files are not enclosed by quotes
HOST_SITE_TX_ID"," SITE_ID"
" 12345"," 16"
" 12345"," 21
I concluded that this method gives me more heartburn than the one I described earlier.
2) Getting the file and use a regex to modify the header.
3) Leaving the header altogether and manually adding a header string at the beginning of the file, using a script
Option 2 is more doable, but I was still interested in asking, if there might be a better way to format the header somehow, so it comes in a regular csv, (comma delimited, double quote bounded) format.
I am looking to do as less hard coding as possible - the table I am exporting has around 40 columns and I am currently running the script for around 4 million records - breaking them in a batch of around 10K each. I would really appreciate any suggestions, even totally different from my approach - I am a programmer in learning.
One easy way to have a csv with just one header is to do
set embedded on
set pagesize 0
set colsep '|'
set echo off
set feedback off
set linesize 1000
set trimspool on
set headsep off
the embedded is a hidden option but it is important to have JUST one header
This is how I created a header:
set heading off
/* header */
SELECT '"'||PCL_CARRIER_NAME||'","'||SITETX_EQUIP_ID||'","'||SITETX_SITE_STAT||'","'||SITETX_CREATE_DATE||'","'||ADVTX_VEH_WT||'"'
FROM
(
SELECT 'PCL_CARRIER_NAME' AS PCL_CARRIER_NAME
, 'SITETX_EQUIP_ID' AS SITETX_EQUIP_ID
, 'SITETX_SITE_STAT' AS SITETX_SITE_STAT
, 'SITETX_CREATE_DATE' AS SITETX_CREATE_DATE
, 'ADVTX_VEH_WT' AS ADVTX_VEH_WT
FROM DUAL
)
UNION ALL
SELECT '"'||PCL_CARRIER_NAME||'","'||SITETX_EQUIP_ID||'","'||SITETX_SITE_STAT||'","'||SITETX_CREATE_DATE||'","'||ADVTX_VEH_WT||'"'
FROM
(
/* first row */
SELECT to_char(123) AS PCL_CARRIER_NAME
, to_char(sysdate, 'yyyy-mm-dd') AS SITETX_EQUIP_ID
, 'value3' AS SITETX_SITE_STAT
, 'value4' AS SITETX_CREATE_DATE
, 'value5' AS ADVTX_VEH_WT
FROM DUAL
UNION ALL
/* second row */
SELECT to_char(456) AS PCL_CARRIER_NAME
, to_char(sysdate-1, 'yyyy-mm-dd') AS SITETX_EQUIP_ID
, 'value3' AS SITETX_SITE_STAT
, 'value4' AS SITETX_CREATE_DATE
, 'value5' AS ADVTX_VEH_WT
FROM DUAL
) MISSING_HOST_SITE_TX_IDS;
This is how you add a pipe delimited header to SQL statements. Once you spool it out that "something" wont be there
-- this creates the header
select 'header_column1|header_column2|header_column3' as something
From dual
Union all
-- this is where you run the actual sql statement with pipes in it
select
rev.value1 ||'|'||
rev.value2 ||'|'||
'related_Rel' as something
from
...
In Oracle 19 you can use set markup csv on to ensure that csv outputs are created.
You can also set the delimiter and optional quote or even spool html if you prefer
You can read more here
set markup csv on
spool "$folder$filename$ext"
select q'|wow, I can't beleive he said "hello, how are you?", can you beleive it!|' as text
from dual;
spool off
quit;
What is the difference between these two variable declarations?
1: num number:='&&num';
2: variable num1 number;
Since in both cases I can reference num by using &num or &&num in other files also,
and in the case of bind variables :num1.
Moreover I have one more confusion: whether any of the below statements differ somehow, are they both valid and do they mean the same thing?
1: variable num1 number;
2: var num1 number;
You appear to have some confusion about the differences between bind variables in Oracle and substitution variables in SQL*Plus.
Let's start with substitution variables. Substitution variables are unique to SQL*Plus and are not part of the database. They won't work if you try to use them with JDBC, for example.
Substitution variables can only hold a piece of text. If SQL*Plus encounters a substitution variable in a line of input, it will replace the variable with its text contents:
SQL> define subvar=X
SQL> select * from dual where dummy = &subvar;
old 1: select * from dual where dummy = &subvar
new 1: select * from dual where dummy = X
select * from dual where dummy = X
*
ERROR at line 1:
ORA-00904: "X": invalid identifier
Note that SQL*Plus replaced our substitution variable with its text value with no regard for whether it gave us valid SQL. In the example above, we omitted the single quotes around &subvar and it gave us invalid SQL, so we got an error.
The lines beginning old and new show us the line we entered before and after SQL*Plus applied the substitution variables. The new line is the line the database tried to run.
You can enable or disable the display of the old and new lines using SET VERIFY ON and SET VERIFY OFF. You can also turn the replacement of substitution variables on or off by using SET DEFINE ON and SET DEFINE OFF.
If we want to run the above query using the substitution variable, we must put quotes around it:
SQL> select * from dual where dummy = '&subvar';
old 1: select * from dual where dummy = '&subvar'
new 1: select * from dual where dummy = 'X'
D
-
X
If &subvar happens to contain a string that was a valid number (e.g. 5), then we can get away without using the quotes, but that's only because taking out the text &subvar and replacing it with the text 5 happens to give us valid SQL.
For example, suppose we have a table called test with the following data in it:
A
----------
1
2
3
4
5
Then we can do
SQL> define subvar=5
SQL> select * from test where a = &subvar;
old 1: select * from test where a = &subvar
new 1: select * from test where a = 5
A
----------
5
Bind variables, on the other hand, have types. They are not simple text values. Their values are sent to the database, and the database can also set their values.
SQL> variable bindvar varchar2(1);
SQL> exec :bindvar := 'X';
PL/SQL procedure successfully completed.
You don't put quotes around a bind variable when you want to use it:
SQL> select * from dual where dummy = :bindvar;
D
-
X
SQL> select * from dual where dummy = ':bindvar';
no rows selected
In the second example above, we got no rows returned because the DUAL table has no rows with the DUMMY column containing the text :bindvar.
You'll get an error if you attempt to assign a value of the wrong type to a bind variable:
SQL> variable bindvar number;
SQL> exec :bindvar := 'X';
BEGIN :bindvar := 'X'; END;
*
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error: character to number conversion error
ORA-06512: at line 1
Bind variables are a standard part of the database, and you can use them with JDBC or whichever method of connecting to the database you choose.
Finally, variable num1 number and var num1 number both mean the same thing. They both define a bind variable num1 of type number. var is just an abbreviation for variable.
I need the count of records of a data base table from unix.
I am calling one sql script from unix and need the record count to any log file .
You can put the following into a test.sql file:
SET HEADING OFF;
SELECT COUNT(*) FROM dual;
QUIT;
and call it via SQL*Plus via script.
It will output:
1
since table dual has only one row. You should be able to write that into a log file.
You cant to append the output off your script to a named file by rediecting it like this.
$ sqlplus username/password#SID #your_script.sql >> /tmp/whatever.log
If your want to have more than a bald count in the output you'll need to include the boilerplate in the projectors:
SQL> select to_char(sysdate, 'YYYYMMDDHH24MISS')||'::Number of emps = '
2 , count(*)
3 from emp
4 group by to_char(sysdate, 'YYYYMMDDHH24MISS')||'::Number of emps = '
5 /
TO_CHAR(SYSDATE,'YYYYMMDDHH24MISS COUNT(*)
--------------------------------- ----------
20100210133747::Number of emps = 16
SQL>