how to connect Amazon Athena with Ruby and execute query over Amazon Athena and get result.
we not able to find any gem or example with help us to connect Amazon Athena in ruby.
Please provide any reference using which we can use to build connection with Amazon Athena and build a custom query executor in Ruby.
just to clarify my application on production so changing SDK from Ruby to JRuby is not a suitable option for me.
Per May 19th 2017, Amazon Athena supports query execution via SDK and CLI.
Ruby API client for Athena documentation # docs.aws.amazon.com
Source code of aws-sdk-athena # github.com/aws/aws-sdk-ruby
I found that the official Amazon SDK for athena was a bit complicated, so I made a new gem called Athens that wraps the SDK in a nicer interface:
conn = Athens::Connection.new(database: 'sample')
query = conn.execute("SELECT * FROM mytable")
If using JRuby is not acceptable, there is another option that could work - but be warned that it's not 100% Ruby!
You could set up a Java Lambda function that encapsulates the query logic, taking in search parameters and then connecting directly to Athena using the JDBC driver.
Then call the Lambda function from Ruby - either via HTTP or through the Ruby client.
Using Lambda function is good alternative but if some one not like to pay additional service amount then batter to implment small application with jetty using rest service in JAVA with sql query as parameter and response text as output(your prefer format) will give you workaround to move on.
JRuby is required. Athena offers only JDBC driver. It works only in JRE.
Related
Very new to Datadog and need some help. I have crafted 2 SQL queries (one for on-prem database and one for cloud database) and I would like to run those queries through Datadog and be able display the query results and validate that the daily results fall within an expected variance between the two systems.
I have already set up Datadog on the cloud environment and believe I should use DogStatsD to create a custom metric but I am pretty lost with how I can incorporate my necessary SQL queries in the code to create the metric for eventual display on a dashboard. Any help will be greatly appreciated!!!
You probably want to be using the MySQL integration, and configure the 'custom queries' option: https://docs.datadoghq.com/integrations/faq/how-to-collect-metrics-from-custom-mysql-queries
You can follow those instructions after you configure the base integration https://docs.datadoghq.com/integrations/mysql/#pagetitle (This will give you a lot of use metrics in addition to the custom queries you want to run)
As you mentioned, DogStatsD is a library you can import to whatever script or application in order to submit metrics. But it really isn't a common practice in the slightest to modify the underlying code of your database. So instead it makes more sense to externally run a query on the database, take those results, and send them to datadog. You could totally write a python script or something to do this. However the Datadog agent already has this capability built in, so it's probably easier to just use that.
I am also just assuming SQL refers to MySQL, there are other integration for things like SQL Server, and PostgreSQL, and pretty much every implementation of sql. And the same pattern applies where you would configure the integration, and then add an extra line to the config file where you have the check run your queries.
Is there a gem other than:
https://github.com/wellbredgrapefruit/asari
https://github.com/spokesoftware/aws_cloud_search
for using aws cloud search in ruby? I'm trying to evaluate which one to use and want to make sure I'm not missing the_gem_one_that_everyone_uses.
There's also Cloudsearchable: https://github.com/awslabs/cloudsearchable
It provides AR-like query building syntax and a simple API for mapping ActiveModel-like objects to a Cloudsearch index.
I am new to Cassandra and was trying to achieve some simple
operations like inserting data into cassandra. I am using cassandra gem
to achieve this.
client = Cassandra.new('tags_logs', 'ec2-xxx-xxx-xxx.com:9160')
client.disable_node_auto_discovery!
client.get('tag_data','red')
And I get the following error:
ThriftClient::NoServersAvailable - No live servers in ...
I'm running this code from my local machine. And while I've no problem connecting using cassandra-cli (so it is not a firewall issue), the code refuses to work. It works perfectly when accessing Cassandra on my own local machine.
Any ideas?
Thanks,
Eden.
I recommend you to use this gem I'm developing: https://github.com/hsgubert/cassandra_migrations
It gives access to Cassandra through CQL3 and manages schema with migrations.
Note: it requires Rails.
For future generations: simply change the timeout ...
client = Cassandra.new('tags_logs', 'ec2-example-example-example.com:9160',:connect_timeout => 10000)
Could I use the Mongo client shell queries/commands from inside Ruby?
I know there is the Ruby driver DSL, but I was thinking about something similar to running a SQL query from within PHP.
Just for the sake of knowing.
You can always work directly with the MongoDB Ruby driver. Read this tutorial for more information.
I'm pretty sure the answer is "no" but I thought I'd check.
Background:
I have some legacy data in Access, need to get it into MySQL, which will be the DB server for a Ruby application that uses this legacy data.
Data has to be processed and transformed. Access and MySQL schemas are totally different. I want to write a rake task in Ruby to do the migration.
I'm planning to use the techniques outlined in this blog post: Using Ruby and ADO to Work with Access Databases. But I could use a different technique if it solves the problem.
I'm comfortable working on Unix-like computers, such as Macs. I avoid working in Windows because it fills me with deep existential horror.
Is there a practical way that I can write and run my rake task on my Mac and have it reach across the network to the grunting Mordor that is my Windows box and delicately pluck the data out like a team of commandos rescuing a group of hostages? Or do I have to just write this and run it on Windows?
Why don't you export it from MS-Access into Excel or CSV files and then import it into a separate MySQL database? Then you can rake the new one to your heart's content.
Mac ODBC drivers that open Access databases are available for about $30.00
http://www.actualtechnologies.com/product_access.php is one. I just run access inside vmware on my mac and expore to csv/excel as CodeSlave mentioned.
ODBC might be handy in case you want to use the access database to do a more direct transfer.
Hope that helps.
I had a similar issue where I wanted to use ruby with sql server. The best solution I found was using jruby with the java jdbc drivers. I'm guessing this will work with access as well, but I don't know anything about access