I have a below test ksh file
#!/bin/ksh
(
sqlplus -s << EOF
${DATABASE}
SET SERVEROUTPUT ON;
SET HEAD OFF
SET FEED OFF
SET DEFINE OFF
SET PAGES 0
SET COLSEP ";"
WHENEVER SQLERROR EXIT SQL.SQLCODE
SELECT 'aaaa vvvv cccc' FROM DUAL;
EXIT
EOF
) | while read sql_out
do
echo ${sql_out}
done
The expected output is
aaaa vvvv cccc
however the output observed is
aaaa vvvv cccc
Sqlplus is stripping out extra white spaces from my output and I want to preserve it.
It isn't SQL*Plus doing that, it's the shell. You need to enclose your variable in quotes:
To preserve whitespace within a string or in a variable, use quoting.
So instead of
echo ${sql_out}
do either of these:
echo "${sql_out}"
printf "%s\n" "${sql_out}"
and the output will then be:
aaaa vvvv cccc
We are using SQL*Plus to export data to a csv file. We have both decimal and text columns, and each field is delimited by comma (,) but some text columns also contain commas. While importing data into some other DB we are seeing that these commas in the text are treated as column separators. Can anyone tell me how I can overcome this?
set tab off
SET head OFF
SET feedback OFF
SET pagesize 0
SET linesize 3000;
SET colsep ,
set trimspool on
set trimout on
set trims on
set null ""
set rowprefetch 2
set feedback off
set arraysize 1000
set PAGESIZE 50000
set STATEMENTCACHE 20
set numwidth 15
column coulmnName format 999999999.99
column coulmnName format 999999999.99
column coulmnName format 999999999.99
column coulmnName format 999999999.99
If you are able to upgrade to SQL*Plus 12.2, you can use set markup csv on:
SQL> set markup csv on
SQL> select * from departments;
"DEPARTMENT_ID","DEPARTMENT_NAME","MANAGER_ID","LOCATION_ID"
10,"Administration",200,1700
20,"Marketing",201,1800
30,"Purchasing",114,1700
40,"Human Resources",203,2400
50,"Shipping",121,1500
60,"IT",103,1400
70,"Public Relations",204,2700
80,"Sales",145,2500
90,"Executive",100,1700
100,"Finance",108,1700
Just concatenate a double quote character to the start and end of columns with a string data type:
SELECT '"' || stringColumnA || '"' AS stringColumnA,
numberColumnB,
'"' || stringColumnC || '"' AS stringColumnC
-- ...
FROM table_name;
If your column already contains double quotes then escape them by doubling them up:
SELECT '"' || REPLACE( stringColumnA, '"', '""' ) || '"' AS stringColumnA,
numberColumnB,
'"' || REPLACE( stringColumnC, '"', '""' ) || '"' AS stringColumnC
-- ...
FROM table_name;
Don't set column separator to a comma, but to something else (such as pipe | or exclamation ! or hash # or some other character that doesn't exist in data you're about to export). You'd, of course, use that sign as a separator while loading data into the target database.
If you are in a linux environment, you can use ~ as the separator and then sed to fix up the output. with this method, you don't have to know what the table content is in order to create the csv file.
The script below wraps every field in double-quotes and drops the trailing delimiter:
ORDER="01"
TABLE="MY_DATA_TBL"
CONN="127.0.0.1:31521/abc0008.world" #via ssh tunnel
sqlplus -L login/pswd#//${CONN}<<EOF >/dev/null
set pagesize 4000;
set verify off;
set feedback off;
set long 99999;
set linesize 32767;
set trimspool on;
col object_ddl format A32000;
set colsep ~;
set underline off;
set headsep off;
spool ${ORD}${TABLE}.tmp1;
select * from ${TABLE};
EOF
cat ${ORD}${TABLE}.tmp1 | sed -e "s/\"/'/" -e 's/ * / /g' -e "s/^ //" -e "s/ ~/~/g" -e "s/~ /~/g" | tail -n +11 | head -n -1 > ${ORD}${TABLE}.tmp2
head -n 1 ${ORD}${TABLE}.tmp2 | sed -e "s/$/~/" > ${ORD}${TABLE}.tmp3
tail -n +2 ${ORD}${TABLE}.tmp2 >> ${ORD}${TABLE}.tmp3
cat ${ORD}${TABLE}.tmp3 | sed -e "s/^/\"/" -e "s/~$/\"/" -e "s/~/\",\"/g" > ${ORD}${TABLE}.csv
#cat file.txt
12354
13456
13498
#!/bin/bash
for i in `cat file.txt`
do
sqlplus XXXXX/XXXXX#DB_NAME << EOF
select *from TABLE_NAME where id="$i"
EOF
done
This is not working for me. Help me how I can solve this.
The solution given by #codeforester works. However I was unable to use it because it created as many DB connections as the number of lines in your file which is a potential impact.
To overcome this, I chose the below solution which may not be ideal but does the job with just one DB connection.
Considering the same data in file.txt
12354
13456
13498
I used the below sed command to populate the above to a single variable "12354,13456,13498"
myvariable=$(echo "`cat file.txt | sed '$!s/$/,/g' | tr -d '\n' | tr -d ' '`")
Now below script will pass this variable to the SQL query and spool the data into a text file:
#!/bin/bash
myvariable=$(echo "`cat file.txt | sed '$!s/$/,/g' | tr -d '\n' | tr -d ' '`")
echo #myvariable
sqlplus /nolog << EOF
CONNECT user#dbname/dbpassword
SPOOL dboutput.txt
select column1 from table_name where id in ($myvariable);
SPOOL OFF
EOF
The output is stored in dboutput.txt (along with the SQL query)
cat dboutput.txt
SQL> select column1 from table_name where id in (12354,13456,13498);
NAME
---------------------------------------------------------------------------- ----
data1
data2
data3
SQL> spool off
Here is the right way to use the heredoc <<, along with the choice of while read instead of for to read the file:
#!/bin/bash
while read -r value; do
sqlplus xxxxx/xxxxx#db_name << EOF
select * from table_name where id='$value';
# make sure heredoc marker "EOF" is not indented
EOF
done < file.txt
See also:
How can I write a here doc to a file in Bash script?
BashFAQ001 to understand why for loop is not the best way to read text lines from a file.
the below script gives me an error. Basically I am trying to delete the records that I got from the first query. I have put them in a text file, formatted them and used them in the delete operation.
After executing the script I am getting the below error:-
: line 5: syntax error at line 27: `<<' unmatched
Can't tell because the code you dumped is unformatted, but my first guess would be you have leading spaces in front of the EOF in your here document.
This should work (note that there are no leading spaces in front of the EOF.:
sqlplus -s $dbcreds << EOF > output.txt
SET SERVEROUTPUT OFF
select empname from emp where dept_no=123;
EOF
if [ -s "output.txt" ]
then
echo " Found the below employees....Deleting them from Database ..............!!!! \n"
cat output.txt
sed "s/(.*)/'\1'/" output.txt| tr '\n' ','|sed 's/.$//' >final_employees.txt
while read line
do
sqlplus -s $dbcreds <<EOF
SET SERVEROUTPUT OFF
Delete from emp where empname in ($line);
EOF
done < final_employees.txt
else
echo " No employees found....!!!"
fi
When I try running the script, I am gettin the error line 45: syntax error: unexpected end of file. I am relatively new to scripting. Please help me resolve it.
#!/bin/ksh
set -xv
export HOME=/home/mine
. $HOME/.env.ksh
BIS_SPOOL=/tmp/bis_table_mine.spl
BIS_REPORT_MINE=/tmp/bis_table_report_mine.txt
touch $BIS_SPOOL
rm $BIS_SPOOL
touch $BIS_SPOOL
exec 5< $BIS_REPORT_MINE
while read -u5 REC_MINE
do
TBLENAME=`echo "$REC_MINE" | awk '{print $3}' | tr '[:upper:]' '[:lower:]'`
sqlplus -s ${USER_ID}/${USER_PASS}#${ORACLE_SID} <<- EOF
set feedback off
set hea ON
set pagesize 9999
set linesize 9999
set trimspool ON
set termout off
spool $BIS_SPOOL append
Column C1 Heading 'Job Name' Format a30
Column C2 Heading 'Table Name' Format a30
SELECT job_name C1,
table_name C2,
FROM table_usage
WHERE table_name like 'TBLENAME%'
/
exit;
EOF
done
exec 5<& -
The <<- EOS is interpreted literally. Make it
sqlplus -s ${USER_ID}/${USER_PASS}#${ORACLE_SID} <<-EOF
without the space char.
Also make sure there is no space char before or after your closing EOS , but leading tab chars are allowed.
IHTH