Connecting to unknown number of databases using ActiveRecord - ruby

I am writing a ruby script that needs to connect to a primary database in order to retrieve a list of secondary databases to connect to. The steps I need to carry out are:
Connect to primary DB
Retrieve list of secondary databases to connect to
Iterate through list of secondary databases, extracting a single model from each database
I don't need to be connected to multiple secondary databases at once, so open -> retrieve object -> close is fine.
All of the examples I have seen so far describe multiple connections where the databases are described in database.yml, which is not possible here as the number of databases I need to connect to can vary.
This blog post describes using a connection pool as follows:
spec = database_a
ActiveRecord::ConnectionPool.open(spec) do |conn|
User.find_all
end
spec = database_b
ActiveRecord::ConnectionPool.open(spec) do |conn|
User.find_all
end
However, ConnectionPool seems to have changed and the .open method no longer exists.
I would appreciate any pointers.

The problem lies with this:
[1] pry(main)> ActiveRecord::ConnectionPool
NameError: uninitialized constant ActiveRecord::ConnectionPool
ActiveRecord::ConnectionPool doesn't exist. However, on my journey to find it, I found ActiveRecord::ConnectionAdapters::ConnectionPool, which looks like something that you want.

Related

How to parameterise the data connection in Tableau in AWS (cloudformation or otherwise)?

I have a simple web app UI (which stores certain dataset parameters (for simplicity, assuming they are all data tables in a single Redshift database, but the schema/table name can vary, and the Redshift is in AWS). Tableau is installed on an EC2 instance in the same AWS account.
I am trying to determine an automated way of passing 'parameters' as a data source (i.e. within the connection string inside Tableau on EC2/AWS) rather than manually creating data source connections and inputting the various customer requests.
The flow for the user would be say 50 users select various parameters on the UI (for simplicity suppose the parameters are stored as a JSON file in AWS) -> parameters are sent to Tableau and data sources created -> connection is established within Tableau without the customer 'seeing' anything in the back end -> customer is able to play with the data in Tableau and create tables and charts accordingly.
How may I do this at least through a batch job or cloud formation setup? A "hacky" solution is fine.
Bonus: if the above is doable in real-time across multiple users that would be awesome.
** I am open to using other dashboard UI tools which solve this problem e.g. QuickSight **
After installing Tableau on EC2 I am facing issues in finding an article/documentation of how to pass parameters into the connection string itself and/or even parameterise manually.
An example could be customer1 selects "public_schema.dataset_currentdata" and "public_scema.dataset_yesterday" and one customer selects "other_schema.dataser_currentdata" all of which exist in a single database.
3 data sources should be generated (one for each above) but only the data sources selected should be open to the customer that selected it i.e. customer2 should only see the connection for other_schema.dataset_currentdata.
One hack I was thinking is to spin up a cloud formation template with Tableau installed for a customer when they make a request, creating the connection accordingly, and when they are done then just delete the cloud formation template. I am mainly unsure how I would get the connection established though i.e. pass in the parameters. I am not sure spinning up 50 EC2's though is wise. :D
An issue I have seen so far is creating a manual extract limits the number of rows. Therefore I think I need a live connection per customer request. Hence I am trying to get around this issue.
You can do this with a combination of a basic embed and applying filters. This would load the Tableau workbook. Then you would apply a filter based on whatever values your user selects from the JSON.
The final missing part is that you would use a parameter instead of a filter and pass those values to the database via initial sql.

How to refresh DB connection with Sequel

I use Sequel::Model.DB to interact with my DB, but for some reason, the DB structure was changed, for example, via the DB console.
This method:
Sequel::Model.db.schema('table_name')
still returns the old DB, cached from the first connection I guess.
How can I reset that cache or, ideally, ensure the actual DB connection on each request?
I tried to use a new connection every time:
def db
#db ||= Sequel.connect(Sequel::Model.db.opts)
end
but, predictably, I got this error eventually:
Sequel::DatabaseConnectionError - PG::ConnectionBad: FATAL: sorry, too many clients already
You shouldn't be changing the structure of the database in an incompatible way while Sequel is running. The easiest way to solve this issue is just to restart the process after changing the database schema, and Sequel will pick up the new database structure.
If you really wanted to try to do this without restarting the process, you could remove the cached schemas (#db.instance_variable_get(:#schemas).clear), and reset the dataset for all model classes (ModelClass.dataset = ModelClass.dataset for each Sequel::Model). However, that doesn't necessarily result in the same thing, since if you remove a column, the old column name will still have a method defined for it.

Batch updates with Sequel gem

I need to do multiple updates on a PostgreSQL database at once. To make it faster, I would like to avoid continually changing context between my ruby application and the database.
I've learned about the #update_sql method, however my strategy to queue those queries is not working. I'm making a list sqls which holds many update strings, and then db[sqls.join(';') + ';']. If instead of executing this line I decide to print the resulting sql and copy/paste to the database it works.
Why doesn't this work? How can I send multiple updates in a single sql?
Sequel doesn't offer an API that supports multiple queries in a single statement. If the driver you are using supports, it, you can use Sequel::Database#synchronize to get access to the driver connection object, and call methods directly on that.

Edit Connection Strings MVC3 LINQ

We have been running one server for the past few months and it contains all the files, SQL data, and is running as our server. We have recently bought 2 more servers to use replication because our database load was so high.
We are going to use a simple master slave replication using transaction replication in MSSQL however our methods that we use to acess LINQ entities must be changes.
For all functions that update they need to connect to the master, but all the ones that select need to query the slave.
How can we edit the connection string based on the function that needs to be done?
Any help would be appreciated.
Thanks
The simplest approach would be;
Create two connection strings on the web.config <connectionStrings> section for read and write.
When querying data, pass the read connection string name to the context's constructor.
and, pass the write connection string name when updating.
If you are using LINQ to entities, you can pass the connection string to the instance of the context i.e ModelContext ctx = new ModelContext("[edmx format connectionstring]");

How connect to db depending on #request.host value?

How connect to different db depending on #request.host value?
Using Sinatra and MongoDB with Mongoid.
I need to read a Sintra application's menu, data ... from different databases. I wish to deploy it only in one place and depending on request.host(subdomain) value to serve the specific pages.
You're probably better off storing all your data in one database marking/tagging/categorizing it depending on the subdomain you're on.
If you setup your Mongoid connection manually already, you could do something like this:
connection = Mongo::Connection.new
Mongoid.database = connection.db(#request.host)
But still, I think you're better of with one database.

Resources