I would like to query a list of values from the database, then use it as input for my custom C++ UDF.
The idea being that I'd grab the mapping from the database when the query started. This is the part I don't know how to do from within C++.
SELECT 'foo',val ...
Then the return value of my custom_scalar_func, I could map the similar names and derive my new value.
SELECT custom_scalar_func(some_other_val,'foo') ...
Heard back from their support, Vertica does not support being able to run queries from within a UDx.
A mandatory requirement is to have Vertica SDK.
Also, it is highly recommneded you will have the Vertica documentation,
especially Programmer's Guide and SDK doumnetation.
Both of them can be downloaded from here (after registration):
https://my.vertica.com
Additional great source would be the Vertica contributed packages located in github here:
Vertica github home page
I found great example of SUDF (Scalar User Defined Function) in here:
Vertica Scalar word count function
According to the Vertica documentation:
"You create your UDSF by subclassing two classes defined by the Vertica SDK: Vertica::ScalarFunction and Vertica::ScalarFunctionFactory."
This is the basic, I think after you look at the link above (wordcount), you will understand how it works.
Related
These are three simple questions which was surprisingly hard to find definite answers.
Does ElasticSearch support indexing data in RDBMS tables ( Oracle/SQLServer/Informix) out of the box?
If yes, can you please point me to documentation on how to do it
If not, what are alternate ways (plugins like Rivers - deprecated) with good reputation
I'm surprised there isn't any solid answer as yet for this. So here's the solution. Logstash directly gives us the ability to push data from a RDBMS into Elasticsearch.
Here's a link to a tutorial which tell you how to go about it. Briefly(all details in link 1), you simply need a JDBC driver for the relational database you'll be using (Postgres, MySQL etc) and make a config file specifying your input as the Relational Database and your output as Elasticsearch. You can also specify a cron which would allow you to keep updating one regular intervals.
Here's the article which mentions the configuration and gets you started (See Example 2): https://www.elastic.co/blog/logstash-jdbc-input-plugin
Here's the article which tells you how to configure the Cronjob as such: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_scheduling
I'm learning how to implement change data capture in oracle. However, not being a DB specialist but rather a DEV, i find the process tedious with respect to other things that i have to do. I end up doing it because my DBA/DEVOP don't want to take care of it.
Hence i was wondering if there is any tool that can help set oracle change data capture. In other words a simple graphical user interface that would write the all code for me. Creation of change table, PL/SQL Script and etc....
Many thanks
topic duplicated in: dba.stackexchange
What problem are you trying to solve?
How (when) will the CDC data be consumed
Are you planning to use something akin to: Oracle 11.1 CDC doc
Be sure to heed: Oracle 11.2 CDC Warning
"Oracle Change Data Capture will be de-supported in a future release of Oracle Database and will be replaced with Oracle GoldenGate. Therefore, Oracle strongly recommends that you use Oracle GoldenGate for new applications."
The company I work for, Attunity, has a pretty slick GUI CDC tool called "Replicate".
It can directly apply changes to a selected target DB, or store changes to be applies.
Many sources (Oracle, SQLserver, DB2...) many targets (Oracle. SQLserver, Netezza, Vertica,...)
Define your source and target DB, Search/Select source table, and one click to go.
Optional transformations such as: table and column names, drop and add columns, calculate values.
Regards,
Hein.
I'm working on a Hadoop / Cassandra integration I have a couple of questions I was hoping someone could help me with.
First, I seem to require the source table/cf to have been created with the option WITH COMPACT STORAGE otherwise I get an error can't read keyspace in map/reduce code.
I was wondering if this is just how it needs to be?
And if this is the case, my second question was, is it possible/how do I add the WITH COMPACT STORAGE option on to a pre-exsting table? .. or am I going to have to re-create them and move data around.
I am using Cassandra 1.2.6
thanks in advance
Gerry
I'm assuming you are using job.setInputFormatClass(ColumnFamilyInputFormat.class);
Instead, try using job.setInputFormatClass(CqlPagingInputFormat.class);
The Mapper input for this is Map<String, ByteBuffer>, Map<String,ByteBuffer>
Similarly, if you need to write out to Cassandra us CqlPagingOutputFormat and the appropriate output types.
See http://www.datastax.com/dev/blog/cql3-table-support-in-hadoop-pig-and-hive for more info.
#Gerry
The "WITH COMPACT STORAGE" thing is CQL3 syntax to create tables structure compatible with Thrift clients and legacy column families.
Essentially, when using this option, the table, or should I say column family, is created without using any Composite.
You should know that CQL3 tables heavily rely on composites to work.
Now to answer your questions:
I was wondering if this is just how it needs to be?
Probably because your map/reduce code cannot deal with Composites. But I believe in version 1.2.6 of Cassandra, you have all the necessary code to deal with CQL3 tables. Look at classes in package org.apache.cassandra.hadoop.
is it possible/how do I add the WITH COMPACT STORAGE option on to a pre-exsting table?
No, it's not possible to modify/change table structure once created. You'll need some kind of migrations.
We have an Oracle Enterprise Edition 10 installation and as its been explained to me by our DBAs, Oracle Enterprise installs include all extensions and you're simply licensed by what you use.
We've discovered we're using Oracle Spatial but we don't want to be. I can confirm for myself that its being used with this SQL:
select * from dba_feature_usage_statistics;
Unfortunately that's all I can find out. We have a large number of applications which use Spatial elements, but having asked all of our vendors they assure us their apps are using Oracle Locator (which is the free subset of Spatial).
So my question is simple - how do I discover exactly which app is using the Oracle Spatial extension?
Alternately (brought to light by ik_zelf's answer), how do I prove I'm only using the Locator subset of Spatial.
Check the sdo metadata:
select * from mdsys.sdo_geom_metadata_table where sdo_owner not in ('MDSYS', 'OE')
when you dig a little deeper in the dba_feature_usage_statistics you will find this query as part of the determination of what is being used and what not. The schema's MDSYS and OE are not counted, even when they have sdo objects.
There is a list of functionality that is part of Oracle Spatial vs. Oracle Locator on the Oracle website: http://docs.oracle.com/cd/B19306_01/appdev.102/b14255/sdo_locator.htm#SPATL340 - specifically pay attention to the section that lists things only available in Oracle Spatial.
The short story is that (basically) the following things are off the table for Locator:
Topology
Network data model
GeoRaster
Geocoding
In-built data mining functions
Linear referencing
Some spatial aggregation functionality
Some parts of the sdo_geom package
Storage, indexing, partitioning, sdo_util package, coordinate transformations and more are all fully within Locator. I would simply check the dba_source view for any stored procedures that use any of the prohibited functions.
For code outside of the database, I guess you have to take someone's word for it, but in my experience external applications tend to use their own methods rather than Oracle in-built features.
I want to load data into text file that is generated after executing "views" in Oracle?How can I achieve this in oracle using UNIX.for example-
I want the same in Oracle on unix box.Please help me out as it alredy cosume lots of time.
your early response is highly appreciated!!
As Thomas asked, we need to know what you are doing with the "flat file". For example, if you're loading it into spreadsheet or doing some other processing that expects a defined format, then you need to use SQL*Plus and spool to a file. If you're looking to save a table (data + table definition) for moving it to another Oracle database then EXP/IMP is the tool to use.
We generally describe the data retrieval process as "selecting" from a table/view, not "executing" a table/view.
If you have access to directories on the database server, and authority to create "Directory" objects in Oracle, then you have lots of options.
For example, you can use the UTL_FILE package (part of the PL/SQL built-ins) to read or write files at the operating system level.
Or use the "external table" functionality to define objects that look like single tables to Oracle but are actually flat files at the OS level. Well documented in the Oracle docs.
Also, for one-time tasks, most of the tools for working SQL and PL/SQL provide facilities for moving data to and from the database. In the Windows environment, Toad's good at that. So is Oracle's free SQLDeveloper, which runs on many platforms. You wouldn't want to use those for a process that runs every day, but they're fine for single moves. I've generally found these easier to use than SQLPlus spooling, but that's a primitive version of the same functionality.
As stated by others, we need to know a bit more about what you're trying to do.