query = "COPY s_clids.o(cid,utype,uid,mid,aid,b,o,d,ad,adat,trid) FROM LOCAL '/database/clids/o.csv' PARSER fcsvparser(header='true') DIRECT REJECTMAX 20000 EXCEPTIONS '/tmp/o_exception.csv' REJECTED DATA '/tmp/o_rejected.csv'"
this works fine in vsql shell but gets stuck when using python cursor.
im using vertica python
Related
I have a Cloudera cluster on which multiple impala jobs are running all the time (i.e. cronjobs containing impala-shell commands). However I have a few INSERT INTO queries that are unusually long: they contain a lot of 'CASE...WHEN...THEN' lines. When these queries are run in impala-shell the command fails with an error "Argument list too long". They run just fine in Hue, but can't get them to run on the commandline.
Are there any workarounds for this?
I've tried running the command via Hive beeline (instead of impala) and setting 'hive.script.operator.truncate.env = true'. Beeline failed with the same error. I've tried if calling the query from a separate file made any difference (it doesn't). Can I save the 'CASE...WHEN...THEN' lines in separate vars perhaps (using 'set') and call those in the query? Or would that be another dead-end? A colleague mentioned a User Defined Function (UDF) might help, but I'm not sure. Opinions?
I have an Informatica data synchronization task that creates a table in Oracle. I am trying to include a call to an Oracle stored procedure in the postprocessing command of Informatica Cloud that will update a variety of tables at the completion of the task. The procedure that I am trying to call is in the same schema as the target of the synchronization task. The procedure runs correctly when I run it directly in Oracle SQL Developer, but I can't get it to run via Informatica Cloud. I know I'm not using the right syntax to make the call, but these are some examples that I have tried so far:
BEGIN
(PROCEDURE_NAME)
END;
CALL(PROCEDURE_NAME);
EXEC(PROCEDURE_NAME);
PROCEDURE_NAME;
Would designing a mapping in Informatica Cloud help me with this? Or is there a prefix that I should be appending to the stored procedure call, even though the procedure is in the same schema as the target of the task?
None of these options will work in Synchronisation task. Call (procedure_name); will work in Mappings which is the easiest way to do it
In Synchronisation task you need to create a batch file to run a SQL script. The script has to contain the connection details.
Example:
connect user/password#
exec (procedure_name);
disconnect;
exit SQL.SQLCODE
SQL.SQLCODE is used so that it will commit all the transactions.
After that, it will be necessary to use a .bat to run the above sql script using sqlplus client.
Example:
cd \bin
call cmd.exe /C "sqlplus /nolog < {path to sql script created in step
2}\filename.sql"
exit /b 0;
I have a locally running oracle thin client and have successfully created a ruby script to connect to a remote oracle database. I successfully make a call (select * from table_name) to the database to get the content of a table:
begin
con = OCI8.new('<user>', <password>, '<host>:<port>/XE')
con.exec('select name from actor') do |records|
puts records
end
rescue OCIError
puts "Database Connection Error"
end
I also want to run an sql script that resides in the oracle directory on the remote host.
Usually I perform the following:
su - oracle
sqlplus <user>/<password>
<SQL> #<script_name>
and this will run the script
In the ruby script I try the following:
con.exec('#<script_name>')
Yet, I get the following error:
stmt.c:230:in oci8lib_200.bundle: ORA-00900: invalid SQL statement (OCIError)
#<script_name> is an sqlplus command.
When sqlplus find #<script_name>, it opens <script_name>, divides its contents to SQL statements and executes them.
If you want to run SQL statements in a script by ruby, you need to write code which opens the script, divides its contents and pass SQL statements to con.exec one by one.
I also want to run an sql script that resides in the oracle directory on the remote host. Usually I perform the following:
No, it can't. sqlplus reads the sql script that resides on the local host.
You can put this script in a function (function_name) and execute
con.exec("select function_name from dual")
I have been trying to execute PSQL from system() within R in RStudio. I have PSQL setup in my PATH and can execute PSQL from the cmd line. I cannot for the life of me figure out the correct method for executing psql from within R on windows. I have code supplied from a ubuntu environment. I have not used system() previously before this and researching for this specific issue has been unsuccessful.
The hardest part is not receiving any output after executing system in R. I have tried a few different setting from looking at ?system. With no luck.
This code should execute a simple sql statement and pass the output to a local file. Ultimately this will be more robust to include dynamic elements in an application. Just having the basics working seems like the hardest part.
system(paste("export PGPASSWORD=db_password;psql -h db_host -d db_name -c 'copy(select * from large_table limit 1000) to stdout csv' > C:/temp_data/db_test.dat", sep=""))
I am curious as to if anyone has a working windows environment using PSQL in R. My greenplum server is not local.
My echo %PATH% includes C:\Program Files (x86)\pgAdmin III\1.12
included in both system and user vars.
There are a few problems with your command.
system cannot be used with redirects, you must use shell
You cannot use single quotes to quote commands in Windows, you must use double quotes.
To concatenate commands, you use the & operator, not a ; like in Unix.
So your command would look like (it appears to be necessary to include this in one line):
cmd<-'set PGPASSWORD=db_password& psql -h db_host -d db_name -c "copy(select * from large_table limit 1000) TO STDOUT CSV;" > C:/temp_data/db_test.dat'
shell(cmd)
But, have you considered using the RPostgresql driver, which is a much simpler, platform-independent way to do your task?
# Load up the driver
library(RPGsql)
drv <- dbDriver("PostgreSQL")
# Create a connection
con <- dbConnect(drv, dbname="db_name", host='db_host',password='db_password',user='db_user')
# Query the database
db_test=dbGetQuery(con, 'select * from large_table limit 1000')
# Write your file
write.csv(db_test,'C:/temp_data/db_test.dat')
I would like to use strace to debug a strange behaviour I have with db2. I have a SQL function myFoo() implemented in C that doesn't get called for some reason (some code access path not existing or not authorized see here). My Sql Function call a function Fooin shared library in /usr/local/lib/libmyfoo (so in db2 term /usr/local/lib/libmyfoo!Foo).
If I use strace directly with db2 and the query I have an error saying
A database connection does not exist
so i created a script call debug.sh with the following. The idea is to have a shell with the db2 connection active and trace it.
db2 "connect to MYDB"
db2 "select * from table(myFoo())" # this calls /usr/local/lib/libmyfoo!Foo
db2 "disconnect MYDB"
It doesn't work cos I realized that strace works with binary so I have the error
Exec Format Error
Probably you are calling each DB2 command in different subshells.
You can fix that problem by executing everything in just one subshell, for example
VALUE=$(. /home/db2inst1/sqllib/db2profile ; db2 connect to MYDB ; db2 "select * from table(myFoo())")
My Sql Function call a function Foo in shared library in /usr/local/lib/libmyfoo (so in db2 term libmyfoo!Foo).
Is this really how you defined your function? libmyfoo!Foo will point to the library $INSTANCE_HOME/sqllib/function/libmyfoo. If your library is elsewhere you need to provide the absolute path to it.