I have an array of say empId's which are of typr String , now i want to pass it in IN clause of SQL from bash.
I had tried below
#make an array of empIds
empIdsarray=(e123 e456 e675 e897)
for j in "${empIdsarray[#]}"
do
inclause=\"$j\",$inclause
done
#remove the trailing comma below
inclause=`echo $inclause|sed 's/,$//'
`isql -U$user -P$pwd -D$db -S$server <<< QRY > tempRS.txt
select * from emp where empId IN ($inclause)
go
quit
go
QRY`
i had tried IN('$inclause') as well but nothing is working , the output is blank although when i run in DB directly it gives result .
Any help is appreciated .
#it should execute like
select * from emp where empId IN ("e123", "e456", "e675", "e897")
thanks in advance .
In BASH you can do:
#!/bin/bash
#make an array of empIds
empIdsarray=(e123 e456 e675 e897)
printf -v inclause '"%s",' "${empIdsarray[#]}"
isql -U$user -P$pwd -D$db -S$server <<QRY > tempRS.txt
select * from emp where empId IN (${inclause%,})
go
quit
go
QRY
Related
I am trying to accomplish a fairly simple goal: react to the possible error of a previous command with a second command. The wrench in the spokes is the need to use a heredoc syntax in the first command.
this (trivialized example) would produce the result I want to catch:
psql -c "select * from table_that_doesnt_exist" || echo "error"
except, the SQL I need to execute is multiple commands and for my circumstance, I must do this with heredocs:
psql << SQL
select * from good_table;
select * from table_that_doesnt_exist
SQL
and when trying to successfully read the stderr from this type of configuration (I've tried a million ways) I cannot seem to figure it out. These kind of methods do not work:
( psql << SQL
select * from good_table;
select * from table_that_doesnt_exist
SQL
) || echo "error"
or
psql << SQL || echo "error"
select * from good_table;
select * from table_that_doesnt_exist
SQL
Trying to figure out a way to backfill partitions of a ds partitioned Hive table.
I know how to run a Hive command from CLI, e.g.
$HIVE_HOME/bin/hive -e 'select a.col from tab1 a'
What I would like to do is provide a .txt file of different DS and have a new job run for each of those DS's, e.g.
$HIVE_HOME/bin/hive -e 'INSERT OVERWRITE PARTITION ds = $DS_VARIABLE_HERE
select a.col from tab1 a where ds = $DS_VARIABLE_HERE'
But I'm not so sure how to do this
I'm thinking of trying out
cat date_file.txt | hive -e 'query here'
But I'm not sure how to place the variable from the date_file file into the Hive query string.
My suggestion is to use shell command to iterate through the values:
Option 1:
If you have fixed set of values you want to iterate through then
DS_VARIABLE_HERE=('val1' 'val2' 'val3')
for ((i=0;i<${#DS_VARIABLE_HERE[#]};i++))
do
$HIVE_HOME/bin/hive -e "INSERT OVERWRITE PARTITION ds = ${DS_VARIABLE_HERE[$i]} select a.col from tab1 a where ds = ${DS_VARIABLE_HERE[$i]}"
done
Option 2:
if you want to iterate through lets say 1 to 10
for ((i=1;i<=10;i++))
do
$HIVE_HOME/bin/hive -e "INSERT OVERWRITE PARTITION ds = ${i} select a.col from tab1 a where ds = ${i}"
done
I'm passing max(pay_date) to a variable Max_date in Shell from Hive table.
The datatype of pay_date field is Date.
I want to extract 7 days of pay_date from Max_date of pay_date from the table.
I used below script to get...
#!/bin/bash
Max_date=$(hive -e "select max(pay_date) from dbname.tablename;")
hive -e "select pay_date from dbname.tablename where pay_date >= date_sub(\"$Max_date\",7);"
It's not giving me any output.
I'm stuck with passing a variable which has date value and use that in date_sub function for last 7 days of rows.
Please let me know if I'm missing some absolute basics.
You can run query like this:
hive -e "select o1.order_date from orders o1 join (select max(order_date) order_date from orders) o2 on 1=1 where o1.order_date >= date_sub(o2.order_date, 7);
Also you can use this as part of shell script:
max_date=$(hive -e "select max(order_date) from orders" 2>/dev/null)
hive -e "select order_date from orders where order_date >= date_sub('$max_date', 7);"
I want make a query wiht this bash code.
##fist I will extrac the table name from table.txt file
table=$(head -n 1 tables.txt)
#In this code is where I will make the substitution on the query
result=`echo "select 'result '||segment_name||':'||MB||':'||TOTAL_MB from (
select TSEG.segment_name,round(TSEG.bytes/1024/1024) MB,l.segment_name as LSEGMENT_NAME,nvl(l.MB,0) as LOB_MB,nvl(l.MB,0)+round(TSEG.bytes/1024/1024) as TOTAL_MB
from dba_segments tseg, (select LOBS.table_name,round(bseg.bytes/1024/1024) MB,lobs.SEGMENT_NAME
from dba_lobs lobs,dba_segments bseg
where LOBS.SEGMENT_NAME=bseg.segment_name
order by bseg.bytes asc
) l
where TSEG.segment_type='TABLE'
and TSEG.segment_name='$table'
and TSEG.SEGMENT_NAME=l.table_name(+)
order by TOTAL_MB
)where rownum=1;`
my problem is on line TSEG.segment_name='$table', I need the table name on format 'TABLE_NAME'.
this is my actual ouput with table named "AABLG":
select 'result '||segment_name||':'||MB||':'||TOTAL_MB from (
select TSEG.segment_name,round(TSEG.bytes/1024/1024) MB,l.segment_name as LSEGMENT_NAME,nvl(l.MB,0) as LOB_MB,nvl(l.MB,0)+round(TSEG.bytes/1024/1024) as TOTAL_MB
from dba_segments tseg, (select LOBS.table_name,round(bseg.bytes/1024/1024) MB,lobs.SEGMENT_NAME
from dba_lobs lobs,dba_segments bseg
where LOBS.SEGMENT_NAME=bseg.segment_name
order by bseg.bytes asc
) l
where TSEG.segment_type='TABLE'
' and TSEG.segment_name='AABLG
and TSEG.SEGMENT_NAME=l.table_name(+)
order by TOTAL_MB
)where rownum=1;
you can see that the " ' " is on the first position, and I don't know why.
regards.
Marco.
This would be much better done without echo at all. Consider, for instance:
IFS=$'\r\n ' read -r table <tables.txt
IFS= read -r -d '' result <<EOF
select 'result '||segment_name||':'||MB||':'||TOTAL_MB from (
select TSEG.segment_name,round(TSEG.bytes/1024/1024) MB,l.segment_name as LSEGMENT_NAME,nvl(l.MB,0) as LOB_MB,nvl(l.MB,0)+round(TSEG.bytes/1024/1024) as TOTAL_MB
from dba_segments tseg, (select LOBS.table_name,round(bseg.bytes/1024/1024) MB,lobs.SEGMENT_NAME
from dba_lobs lobs,dba_segments bseg
where LOBS.SEGMENT_NAME=bseg.segment_name
order by bseg.bytes asc
) l
where TSEG.segment_type='TABLE'
and TSEG.segment_name='$table'
and TSEG.SEGMENT_NAME=l.table_name(+)
order by TOTAL_MB
) where rownum=1;
EOF
This also fixes the bug observed in your question by setting IFS to a value that includes $'\r', the carriage-return character found in DOS-formatted newlines, and thus stripping such characters when they exist at the end of the first line of tables.txt.
Hi I'm pretty new using BTEQ,
I'm looking to schedule a query that runs using teradata connection, the query results should go to an excel or txt with separators (so it can be formatted using excel)
I need to to this through windows, so I guess it should be a *.bat scheduled using windows scheduler
The things is that I don't have a clue how to open the connection, running the query and exporting the result to a *.xls or *.csv or *.txt
I have set already the ODBCs to connect to TD (I use the TD administrator and run the query manually everyday).
Any ideas?
BTEQ doesn't use ODBC but a CLI connection.
You can simply create a script like this:
.logon TDPID/username,password;
.OS if exist bla.txt del bla.txt; -- remove the file if it exists (otherwise BTEQ appends)
.export report file = bla.txt;
SELECT
TRIM(DataBaseName) || ',' ||
TRIM(TableName) || ',' ||
TRIM(Version) || ',' ||
TRIM(TableKind) || ',' ||
TRIM(ParentCount) (TITLE '')
FROM dbc.TablesV
SAMPLE 10
;
.export reset;
.EXIT;
TDPID is the name of your TD system (or an IP-address).
You need to manually format the csv in the SELECT as shown above using TRIM and || and you have to take care of possible NULLs using COALESCE(TRIM(col), '').
You might also try the ancient DIF format, no need to care about NULLs, etc.
.export DIF file = bla.dif;
SELECT
DataBaseName
,TableName
,Version
,TableKind
,ParentCount
FROM dbc.TablesV
SAMPLE 10
;
In TD14 there's a table UDF named CSV which takes care for NULLs and quoting strings instead of TRIM/COALESCE/||. Thy syntax is a bit lengthy, too:
WITH cte
(
DataBaseName
,TableName
,Version
,TableKind
,ParentCount
)
AS
(
SELECT
DataBaseName
,TableName
,Version
,TableKind
,ParentCount
FROM dbc.TablesV
SAMPLE 10
)
SELECT *
FROM TABLE(CSV(NEW VARIANT_TYPE
(
cte.DataBaseName
,cte.TableName
,cte.Version
,cte.TableKind
,cte.ParentCount
), ',', '"')
RETURNS (op VARCHAR(32000) CHARACTER SET UNICODE)) AS t1;
Finally you run BTEQ and redirect the file (you can put this is an BAT file):
BTEQ < myscript.txt
There might be other options, too, e.g. a TPT/FastExport script or putting the SQL inside an Excel file which automatically runs the query when opened...