Haskell, HDBC, ODBC, MySQL and Mac OS X - macos

I'm trying to use Haskell (version 6.10.3) and HDBC to connect to a MySQL Database. I've chosen to do it with Haskell ODBC. I've used cabal to install HDBC (2.1.1) and HDBC-ODBC (2.1.0.0). I've downloaded and installed the MySQL ODBC driver (5.1.5). I used macports to install unixODBC (2.2.14_1). All of this on top of Mac OS X (10.5.8).
I've mostly been using the instructions on this page http://en.wikibooks.org/wiki/Haskell/Database. At around this point:
"# Add the mysql driver to odbcinst.ini file (under $ODBC_HOME/etc/) and your data source in $HOME/.odbc.ini."
It looks like the macports version of unixODBC installs everything under /opt/local/. I've put an odbcinst.ini into /opt/local/etc/ and I've created a .odbc.ini in my home directory which looks something like this (note that I've experimented with UID vs. USERNAME and PWD vs PASSWORD):
[ODBC Data Sources]
myodbc = MySQL ODBC 5.1 Driver
[ODBC]
Trace = 0
TraceAutoStop = 0
TraceFile =
TraceLibrary =
[myodbc]
Driver = /usr/local/lib/libmyodbc5.so
DATABASE = [hidden]
DESCRIPTION = [hidden]
SERVER = localhost
PORT = 3306
UID = [hidden]
PWD = [hidden]
PASSWORD = [hidden]
USER = [hidden]
And I've written and compiled this Haskell Program:
import Database.HDBC.ODBC
import Database.HDBC
import System
main = do
args <- getArgs
c <- connectODBC (args!!0)
tables <- getTables c
mapM_ putStrLn $ tables
When I try a DSN of "DSN=myodbc" it errors out with:
Database: SqlError
{seState = "[\"HY000\"]",
seNativeError = -1,
seErrorMsg = "connectODBC/sqlDriverConnect:
[\"1045: [unixODBC][MySQL][ODBC 5.1 Driver]Access
denied for user 'jamie'#'localhost' (using password: YES)\"]"}
However, when I try a DSN of "DSN=myodbc;UID=[hidden];PWD=[hidden]", it lists all the tables in the database.

This may be a unixODBC problem rather than a Haskell / HDBC / HDBC-ODBC problem. Running "isql myodbc" results in a "Bus Error". Running "isql -v myodbc" doesn't give any more information. Running isql [uid] [pwd] connects just fine.

iODBC, maintained and supported by my employer, has shipped as part of Mac OS X since Jaguar (10.2.x).
You'll be better off updating iODBC with all the latest patches (Apple tends to be a bit behind on these), than shifting to UnixODBC.
It's generally best to keep all your ODBC configuration in the default file locations for Mac OS X --
/Library/ODBC/odbc.ini
/Library/ODBC/odbcinst.ini
/Users/*/Library/ODBC/odbc.ini
/Users/*/Library/ODBC/odbcinst.ini
You can create symlinks from anywhere else you may want to have these files, e.g. --
ln -s ~/Library/ODBC/odbc.ini ~/.odbc.ini
Last, you may benefit from testing with a commercial ODBC driver for MySQL, such as one of my employer's offerings (two week free trial provided as part of download).

Related

Mac OS X: how to enable ODBC log?

I have installed some ODBC drivers in Mac OS X. One of them works incorrectly, i.e. my app cannot connect to PostgreSQL database via the ODBC driver.
I'd like to enable logging for it and check what is the reason.
How I can do it?
If you were using the iODBC driver manager that ships with macOS, tracing would be controlled through the iODBC Administrator.app, found in /Applications/iODBC/, and the standard configuration file (odbc.ini) found in the standard locations (/Library/ODBC/ for system-level config; /Users/*/Library/ODBC/ for user-level).
You can also manually edit (or create) a stanza like the one below in that file.
[ODBC]
Trace = 1
TraceFile = iodbc.$U.$p.$T.log
TraceAutoStop = 0
;TraceDLL =
Note the $u, $p, $t, and/or $h tokens may be used to automatically insert UserID, ProcessID, Timestamp, and/or $HOME in the log filename.
With UnixODBC as an after-market, third-party add-on, the configuration files may be located anywhere. UnixODBC tracing is enabled/disabled via this stanza of the odbcinst.ini (not odbc.ini) file.
[ODBC]
Trace = yes
TraceFile = trace_file_path

DBD::Oracle: load_file:The specified module could not be found

It's a story as old as time itself. No, I'm not talking about the story of a man's wife running off with his best friend. I'm talking about getting DBD::Oracle to work. Although both stories have about the same amount of pain associated with them.
Let us start with the facts:
Running Windows 7 -64 bit
Running Strawberry Perl (v. 5.16.3 - 32 bit) (the reason we have 32 bit is because there is a module that is only 32 bit and we are stuck with it - don't ask)
Oracle Client (version 12.1.0 - 32 bit) successfully installed.
The bin directory of said Oracle Client has been included at the front of the PATH
the addition of the following environmental variables:
ORACLE_HOME - pointing to Oracle Client path
TNS_ADMIN -pointing to Oracle Client path
LD_LIBRARY_PATH - pointing to Oracle Client path
ORACLE_USER_ID - set to a valid oracle user in the form: user/pass#system:port\sid
ORACLE_SID - set to valid oracle sid
Successful connection of sqlplus to the desired database.
"Successful creation" of DBD::Oracle into Strawberry Perl using cpanm. Although I had to use --force because the tests failed.
An entry entered into tnsnames.ora, that I assume is correct:
nameIMadeUp.WORLD =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = servername)(PORT = port#))
)
(CONNECT_DATA =
(SID = sid)
)
)
After all of that, we try to run this code (snipped for the sake of brevity):
use DBI;
use DBD::ODBC;
use Log4Perl;
#logger set up here.
my $user = "username";
my $passwd = "password";
my $DSN = "dbi:Oracle:host=servername;sid=sid;port=port#";
my $oracleDbh = DBI->connect($DSN,$user,$passwd) or $logger->logdie("$DBI::errstr\n at line:".__LINE__);
And we get this glorious error:
install_driver(Oracle) failed: Can't load 'C:/strawberry/perl/site/lib/auto/DBD/Oracle/Oracle.dll' for module DBD::Oracle: load_file:The specified module could not be found at C:/strawberry/perl/lib/DynaLoader.pm line 190.
at (eval 84)[C:/strawberry/perl/vendor/lib/DBI.pm:770] line 3.
Compilation failed in require at (eval 84)[C:/strawberry/perl/vendor/lib/DBI.pm:770] line 3.
Perhaps a required shared library or dll isn't installed where expected
...stacktrace...
So, I have read that 'C:/strawberry/perl/site/lib/auto/DBD/Oracle/Oracle.dll' is looking for oci.dll which is causing the error. However oci.dll is in [oracle_client_path]/bin/. Theoretically, it should be able to find it.
Or is it looking for another dll? are there tools in windows or maybe even visual studio that can help here?
Or is there some magical step that I am missing?
I thank you in advance for any wise sage knowledge you put forth.
Apparently, a reboot was needed. After the reboot, connections through Perl were possible. This is here in case someone is as stupid as me. I'm going back my AS/400 now...

Connect to Oracle database using ROracle with tnsnames.ora on Mac OS 10.10

Followup to Installation of RODBC/ROracle packages on OS X Mavericks...
First of all, I have installed ROracle on Mac OS 10.10.3 (Yosemite) using the answer provided by #joran. Additionally, using the start-up plist file to set DYLD_LIBRARY_PATH, I can run library(ROracle), and it loads just fine. However, I am unable to connect to my database with a tnsnames.ora file. I have added the TNS_ADMIN variable to the .Renviron file, which RStudio seems to pick up:
> Sys.getenv("TNS_ADMIN")
[1] "opt/oracle/instantclient_11_2/network/admin"
When I run the following, for example
con <- dbConnect(drv = dbDriver("Oracle"), dbname = "db", username = "user", password = "pw")
, I get the error
Error in .oci.Connect(.oci.drv(), username = username, password = password, :
ORA-12154: TNS:could not resolve the connect identifier specified
In addition, I have also added the TNS_ADMIN environment variables to .bash_profile, but that didn't help.
NOTE 1: I have already used the tnsnames.ora file to connect to the database with SQL Developer, so I'm fairly confident the issue is something external to the content of the file.
NOTE 2: I can in fact connect using ROracle with something like:
# see example at http://www.oralytics.com/2015/05/loading-json-data-into-oracle-using.html
host <- "localhost"
port <- 1521
service <- "pdb12c"
drv <- dbDriver("Oracle")
connect.string <- paste(
"(DESCRIPTION=",
"(ADDRESS=(PROTOCOL=tcp)(HOST=", host, ")(PORT=", port, "))",
"(CONNECT_DATA=(SERVICE_NAME=", service, ")))", sep = "")
con <- dbConnect(drv, username = "dmuser", password = "dmuser", dbname = connect.string)
I double checked my tnsnames.ora file and it's in the exact same format as connect.string, so I'm thinking it's just not actually being seen by RStudio, even though Sys.getenv("TNS_ADMIN") gives me the correct path. Alternatively, it could be that the name required by the dbname argument on Mac is different than Windows.
Any help would be greatly appreciated! Thanks!
I've seen many recommendations online for adding environment variables on Yosemite to be accessed by RStudio. The only one that has fully worked for me, at least so far, is to add all environment variables to the plist file. You can add as many variables as you want, which is described by #MortimorGoro in Setting environment variables via launchd.conf no longer works in OS X Yosemite/El Capitan/macOS Sierra?.
So my solution here was to just add TNS_ADMIN to plist!
For those of you that got here but are on Windows 7, on my workstation I have to use a TNSnames.ora file (also utilizing TNS_ADMIN environment variable)(Located in: C:\app - see 2nd screenshot), I got the following to work:
library(RODBC)
channel <- odbcConnect("PERMIT_DEV_odbc",
uid = "POWDERED_TOAST_MAN",
pwd = "dev_NONE_OF_YOUR_BEEZNEEZ",
believeNRows = FALSE)
testsql <- "select sysdate from dual;"
query <- sqlQuery(channel = channel,
query = testsql)
1 Click on the Start button (in windows 7)
2 Start typing, "odbc" and look for "set up data sources (ODBC)" or something like that.
3 Add your connection based off your TNS file.
4 there's also a "test connection" button you should use to verify connectivity!
Hope this helps!

How do I install RODBC on Mac with unixodbc and freetds?

After a fairly extensive search, I noticed that a number of people are having a hard time finding a start-to-finish guide that answers this question. (At least one question notes that a solution exists, but the proposed solution does not get around the fact that, by default, RODBC attempts to compile against iODBC, which is not included with Yosemite.) I just went through this process, so I thought I would document it here in the hope that it will benefit others. I am connecting to a SQL Server database.
Using Homebrew as my OS X package manager, I can install RODBC with the following steps (assuming I have already installed R).
Install unixodbc:
$ brew install unixodbc
Install freetds (replacing /usr/local/Cellar/unixodbc/2.3.2_1 with your unixodbc directory, if necessary):
$ brew install --with-tdsver=8.0 --with-msdblib --with-unixodbc=/usr/local/Cellar/unixodbc/2.3.2_1 freetds
Configure your freetds installation (the following is a minimal configuration file):
freetds.conf
# server specific section
[global]
; tds version = 8.0
; dump file = /tmp/freetds.log
; debug flags = 0xffff
; timeout = 10
; connect timeout = 10
text size = 64512
[TESTSQL]
# insert the actual host below
host = <xxx.xx.x.xx>
port = 1433
tds version = 8.0
Test the freetds config:
$ tsql -H `<xxx.xx.x.xx>` -p 1433 -U `<username>` -P `<password>`
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
1> exit
Configure your unixodbc installation (the following is a minimal configuration file):
$ sudo vim /usr/local/Cellar/unixodbc/2.3.2_1/etc/odbcinst.ini
odbcinst.ini
[MSSQL]
Description = Microsoft SQL Server driver
Driver = /usr/local/Cellar/freetds/0.95.18/lib/libtdsodbc.so
(and another minimal installation file):
$ sudo vim /usr/local/Cellar/unixodbc/2.3.2_1/etc/odbc.ini
odbc.ini
[ODBC Data Sources]
TESTSQL = Test database
[TESTSQL]
Driver = MSSQL
Servername = TESTSQL
Port = 1433
Database = TMSEPRD
TDS_Version = 8.0
Test the new configuration with isql:
$ isql TESTSQL `<username>` `<password>`
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
SQL> quit
Create a symbolic link to the files in your home directory:
$ ln -vs /usr/local/Cellar/freetds/0.95.18/etc/freetds.conf ~/.freetds.conf
$ ln -vs /usr/local/Cellar/unixodbc/2.3.2_1/etc/odbc.ini ~/.odbc.ini
$ ln -vs /usr/local/Cellar/unixodbc/2.3.2_1/etc/odbcinst.ini ~/.odbcinst.ini
Find and modify your RProfile file by appending the following line(s) of code to the file (replacing /usr/local/include with the include directory that contains your sql.h and sqlext.h files; the second line may be unnecessary if the directory does not exist):
$ vim /Library/Frameworks/R.framework/Versions/3.2/Resources/library/base/R/Rprofile
Sys.setenv(ODBC_INCLUDE="/usr/local/include")
Sys.setenv(ODBC_LIBS="/usr/local/lib")
Now download the RODBC package source (which you an download here) to your Downloads folder.
Open a new R console session and install the package (replacing RODBC_1.3-12.tar.gz with the name of your package source):
install.packages("~/Downloads/RODBC_1.3-12.tar.gz", repos=NULL, type="source")
The package should now work:
> library(RODBC)
> myconn <- odbcConnect("TESTSQL", uid="<userid>", pwd="<password>")
Thanks to Jared Folkins and Gabi Huiber for help with figuring out what directories R looks in by default for the requisite files for RODBC.

Performance issue when using pyodbc with Python 3 to access MS SQL Server 2008

I have a performance problem when accessing an MS SQL Server 2008 using Python 3.X and PyODBC (FreeTDS as ODBC driver).
When I run the script with Python 2.7 it retrieves about 30000 rows per second. When using Python 3 the speed is reduced to approximately 7000 rows per second.
I've also tried to query MySQL through ODBC with Python2 and Python3 and the speed is surprisingly about the same in both versions.
Did anyone experience the same problems? Could it be solved by changing some configuration files?
Here is the configuration:
#odbcinst.ini
[FreeTDS]
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
FileUsage = 1
CPTimeout =
CPReuse =
client charset = utf-8
#freetds.conf
[global]
tds version = 7.2
port = 1433
client charset = utf-8
The connection string looks like this:
conn = pyodbc.connect('DRIVER={FreeTDS};SERVER=[OUR_SERVER];PORT=1433;UID=[USER];PWD=[PASSWORD];DATABASE=[DATABASE];TDS_VERSION=auto')
Some additional information:
I'm using an Ubuntu 12.04 server with freetds v0.91-1 and pyodbc v3.0.7.
Thank you very much for some hints or inspiration

Resources