Sphinx + Oracle : Data source name not found error - oracle

I want to connect to remote oracle database server and index some data from there with sphinx search engine. My OS is ubuntu 16.04 and I havc installed sphinx on it and tested it with local mysql database and everthing was ok (All the data indexed and I could search and results was correct) . I also have installed unixODBC and tested it with isql tool to remote access to oracle database server and every thing was ok, but when I want to index data with indexer command of sphinx this error occure:
sql_connect: [unixODBC][Driver Manager]Data source name not found, and no default driver specified
Here is source block of my sphinx.conf file:
source src2
{
type = odbc
sql_host = hostName
sql_user = user
sql_pass = pass
sql_db = dbname
sql_port = 1521
odbc_dsn = DSN = mydsn; Driver={Oracle};Dbq=hostname:1521/dbname;Uid=user;Pwd=pass
sql_query = \
SELECT tableId, Name \
FROM sampleTable
}
And odbc.ini file:
[mydsn]
Application Attributes = T
Attributes = W
BatchAutocommitMode = IfAllSuccessful
BindAsFLOAT = F
CloseCursor = F
DisableDPM = F
DisableMTS = T
Driver = Oracle
DSN = mydsn
EXECSchemaOpt =
EXECSyntax = T
Failover = T
FailoverDelay = 10
FailoverRetryCount = 10
FetchBufferSize = 64000
ForceWCHAR = F
Lobs = T
Longs = T
MaxLargeData = 0
MetadataIdDefault = F
QueryTimeout = T
ResultSets = T
ServerName = MYDATABASE
SQLGetData extensions = F
Translation DLL =
Translation Option = 0
DisableRULEHint = T
UserID = user
Password = pass
StatementCache=F
CacheBufferSize=20
UseOCIDescribeAny=F
SQLTranslateErrors=F
MaxTokenSize=8192
AggregateSQLType=FLOAT
and odbcinst.ini file :
[Oracle]
Description= ODBC for Oracle
Driver = /opt/oracle/instantclient_12_2/libsqora.so.12.1
Setup =
FileUsage = 1
CPTimeout =
CPReuse = /usr/local/etc/odbcinst.ini

Try
odbc_dsn = DSN=mydsn;
i.e. w/o spaces around = after the DSN and since you have everything else specified in the ini file just the DNS should be enough. You also need only sql_query out of the rest sql_*. Like this:
source src2
{
type = odbc
odbc_dsn = DSN=mydsn;
sql_query = \
SELECT tableId, Name \
FROM sampleTable
}

Related

Oracle ODBC Issues with Centos7

Installation of Oracle ODBC originated from Centos7 was successful. However, the error message im004 unixodbc driver manager driver's sqallochhandle on sql_handle_henv failed was encountered and a solution was not found.
For a list of installation files, see
unixODBC-2.3.11.tar.gz
instantclient-basic-linux.x64-19.18.0.0.0dbru.zip
instantclient-odbc-linux.x64-19.18.0.0.0dbru.zip
First, this is the installation process.
tar -xvzf ./unixODBC-2.3.11.tar.gz -\> ./configure -\> make -\> make install
unzip ./instantclient-basic-linux.x64-19.18.0.0.0dbru.zip
unzip ./instantclient-odbc-linux.x64-19.18.0.0.0dbru.zip
cd instantclient_19_18/
./odbc_update_ini.sh /usr/local /usr/lib64 Oracle
The location and contents of each important installation file.
/usr/local/etc/odbc.ini
\[Oracle226\] Application Attributes = T Attributes = W BatchAutocommitMode = IfAllSuccessful CloseCursor = F DisableDPM = F DisableMTS = T Driver = Oracle EXECSchemaOpt = EXECSyntax = T Failover = T FailoverDelay = 10 FailoverRetryCount = 10 FetchBufferSize = 2000 ForceWCHAR = F Lobs = F Longs = T MetadataIdDefault = F QueryTimeout = T ResultSets = T ServerName = //IP:PORT/DB SQLGetData extensions = F Translation DLL = Translation Option = 0 UserID = DB_ID Password = DB_PW Port = PORT DatabaseCharacterSet = AL32UTF8 #DatabaseCharacterSet = euckr
/usr/local/etc/odbcinst.ini
\[Oracle\] Description = Oracle ODBC driver for Oracle 19 Driver = /usr/local/libsqora.so.19.1 Setup = FileUsage = CPTimeout = CPReuse =
/etc/profile
export ODBCSYSINI=/usr/local/etc export ODBCINI=/usr/local/etc/odbc.ini export LD_LIBRARY_PATH=/usr/lib64:/usr/local/etc
odbcinst -j unixODBC 2.3.11 DRIVERS............: /usr/local/etc/odbcinst.ini SYSTEM DATA SOURCES: /usr/local/etc/odbc.ini FILE DATA SOURCES..: /usr/local/etc/ODBCDataSources USER DATA SOURCES..: /usr/local/etc/odbc.ini SQLULEN Size.......: 8 SQLLEN Size........: 8 SQLSETPOSIROW Size.: 8
In the middle, commands such as source /etc/profile or ldconfig were executed. It's hard to use yum because it's a closed net.
Please help me.
The additional DB you are trying to connect to is an Oracle DB of a different IP.
I tried Googleing, but there was nothing I could do.

Creating cross region autonomous database failing with 'message': "The following tag namespaces / keys are not authorized or not found: 'oracle-tags'"

Need help in creating cross region standby database via python have tried creating with
oci.database.models.CreateCrossRegionAutonomousDatabaseDataGuardDetails
I am unable to find an example for the same so i tried with whatever i can find through sdk documentation
response = oci_client.get_autonomous_database(autonomous_database_id=primary_db_id)
primary_db_details = response.data
def create_cross_region_standby_db(db_client, primary_db_details: oci.database.models.AutonomousDatabase):
adw_request = oci.database.models.CreateCrossRegionAutonomousDatabaseDataGuardDetails()
adw_request.compartment_id = primary_db_details.compartment_id
adw_request.db_name = primary_db_details.db_name
adw_request.data_storage_size_in_tbs = primary_db_details.data_storage_size_in_tbs
adw_request.data_storage_size_in_gbs = primary_db_details.data_storage_size_in_gbs
adw_request.cpu_core_count = primary_db_details.cpu_core_count
adw_request.db_version = primary_db_details.db_version
adw_request.db_workload = primary_db_details.db_workload
adw_request.license_model = primary_db_details.license_model
adw_request.is_mtls_connection_required = primary_db_details.is_mtls_connection_required
adw_request.is_auto_scaling_enabled = primary_db_details.is_auto_scaling_enabled
adw_request.source_id = primary_db_details.id
adw_request.subnet_id = <standby subnet id>
adw_response = db_client.create_autonomous_database(create_autonomous_database_details=adw_request)
print(adw_response.data)
adw_id = adw_response.data.id
oci.wait_until(db_client, db_client.get_autonomous_database(adw_id), 'lifecycle_state', 'AVAILABLE')
print("Created ADW {}".format(adw_id))
return adw_id
create_cross_region_standby_db is done using standby region credentials. Creation of primary db in the same region works fine.

Using variables in cx_Oracle.connect method

I'm trying to read env.properties file to get the credentials to connect to Oracle DB using cx_Oracle.
Variables have correct values but I guess issue is with the way I'm using connect method.
Is there something wrong?
config = ConfigParser.RawConfigParser()
config.read('env.properties')
user = config.get('DatabaseSection', 'database.user')
pwd = config.get('DatabaseSection', 'database.password')
host = config.get('DatabaseSection', 'database.db_host')
port = config.get('DatabaseSection', 'database.db_port')
instance = config.get('DatabaseSection', 'database.db_instance')
conn = cx_Oracle.connect(user,pwd,host:port/instance)
cur = conn.cursor()
Managed with below piece of code -
dsn_tns = cx_Oracle.makedsn(host, port, instance)
conn = cx_Oracle.connect(user=user, password=pwd, dsn=dsn_tns)

How to add dynamic code/ruby code inside .conf file

I'm using Sphinx search engine for my application, and I would like to add ruby code inside the sphinx.conf file to update some values dynamically based on my application server hostname? Is it possible to execute the ruby code inside sphinx.conf? Something like below
{
type = mysql
sql_host = testHost
sql_user = test
sql_pass = <%= D.decrypt("encrypted password") %> // Ruby CODE
sql_db = {{database_name}}
sql_query_pre = SET TIME_ZONE = '+0:00'
sql_query_pre = SET SESSION group_concat_max_len = 32768
sql_query_pre = SET NAMES utf8
}
In short, no it's not possible. Sphinx is not capable of executing Ruby directly in its .conf file.
That being said, you can use environment variables to keep the password out of your config file:
{
type = mysql
sql_host = testHost
sql_user = test
sql_pass = $SQL_PASS
sql_db = {{database_name}}
sql_query_pre = SET TIME_ZONE = '+0:00'
sql_query_pre = SET SESSION group_concat_max_len = 32768
sql_query_pre = SET NAMES utf8
}
Then you can either set the environment variable in the commandline when you run Sphinx:
SQL_PASS="mypassword123" indexer -c /path/to/config
Or set it in your .bashrc:
export SQL_PASS="mypassword123"
Or set it in ruby:
ENV["SQL_PASS"] = "mypassword123"
Keep in mind that no matter what method you choose, you must set the environment variable before you run Sphinx.

Add ports on Observium

I've installed Observium and it's working great, except for one thing: no ports are showing up.
I've added 3 devices with snmpd installed (Debian 7), all the graphs are displayed (cpu, mem) except for the network traffic ones, and on the front page, it shows:
Ports 0 0 up 0 down 0 ignored 0 disabled
I've also installed the agent on one device, it didn't change anything.
The config used is here (all the private informations are noted xx):
$config['db_host'] = 'localhost';
$config['db_user'] = 'xx';
$config['db_pass'] = 'xx';
$config['db_name'] = 'xx';
// Base directory
$config['install_dir'] = "/opt/observium";
// Default community list to use when adding/discovering
$config['snmp']['community'] = array("xx");
// Authentication Model
$config['auth_mechanism'] = "mysql"; // default, other options: ldap, http-auth, please see documentation for config he$
// Enable alerter (not available in CE)
#$config['poller-wrapper']['alerter'] = TRUE;
// Set up a default alerter (email to a single address)
$config['alerts']['alerter']['default']['descr'] = "Observium - Alert";
$config['alerts']['alerter']['default']['type'] = "email";
$config['alerts']['alerter']['default']['contact'] = "xx#xx.com";
$config['alerts']['alerter']['default']['enable'] = TRUE;
$config['poller_modules']['unix-agent'] = 1;
$config['collectd_dir'] = '/var/lib/collectd/rrd';
$config['int_customers'] = 1; # Enable Customer Port Parsing
$config['int_transit'] = 1; # Enable Transit Types
$config['int_peering'] = 1; # Enable Peering Types
$config['int_core'] = 1; # Enable Core Port Types
$config['int_l2tp'] = 0; # Enable L2TP Port Types
$config['show_locations'] = 1; # Enable Locations on menu
$config['show_locations_dropdown'] = 1; # Enable Locations dropdown on menu
$config['show_services'] = 0; # Enable Services on menu (Disabled by default as this option is deprecated)
$config['ports_page_default'] = "details/"; ## eg "details/" "graphs/bits/"
$config['show_overview_tab'] = true;
$config['overview_show_sysDescr'] = true;
$config['frontpage']['device_status']['ports'] = true;
$config['device_traffic_iftype'] = array('/loopback/','/tunnel/','/virtual/','/mpls/');
$config['device_traffic_descr'] = array('/loopback/','/vlan/','/tunnel/','/:\d+/');
// End config.php
Is the configuration the problem ? Is it only from Professional editions ?
Thank you for your help, I'm new to Observium and I really find it awesome (excepted this one little problem...)
Problem solved !
Easy fix on my MySQL Observium database:
ALTER TABLE `ports` CHANGE `port_label_short` `port_label_short` VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL ;
Explanation:
I've ran ./discovery.php -d -m ports -h 4 (4 being my host id), and it showed a lot of errors when running queries:
ERROR[Error in query: (1364) Field 'port_label_short' doesn't have a default value]
This error was the key, as I only needed to set a default value on this column.

Resources