I'm trying to use the Ruby Sequel::Model ORM functionality for a web service, in which every user's data is stored in a separate MySQL database. There may be thousands of users and thus databases.
On every web request I want to construct the connection string to connect to the user's data, do the work, and then close the connection.
When using Sequel, I can specify the database to use for a particular block of code:
Sequel.connect(:adapter=>'mysql', :host=>'localhost', database=>'test1') do |db|
db.do_something()
end
This is all very good, I can perform Sequel operations on the particular user's database. However, when using Sequel::Model, when I come to do my db operations it looks like this:
Supplier.create(:field1 => 'TEST')
I.e. it doesn't take db as a parameter, so just uses some shared database configuration.
I can configure the database Model uses in two ways, either set the global DB variable:
DB = Sequel.connect(:adapter=>'mysql', :host=>'localhost', database=>'test1')
class Supplier < Sequel::Model
end
Or, I can set the database just for Model:
Sequel::Model.db = Sequel.connect(:adapter=>'mysql', :host=>'localhost', database=>'test1')
class Supplier < Sequel::Model
end
In either case, setting a shared variable like this is no good - there may be multiple requests processed concurrently, each of which needs its own database configuration.
Is there any way around this? Is there a way of specifying per-request db configuration using Sequel::Model?
As an aside, I've run into a similar problem with DataMapper, I'm now wondering whether having a single multi-tenanted database is going to be the only option if using Ruby, although I'd prefer to avoid this as it limits scalability.
A solution, or any pertinent discussion would be much appreciated.
Thanks
Pete
Use Sequel's sharding support for this: http://sequel.jeremyevans.net/rdoc/files/doc/sharding_rdoc.html
Actually in your case it's probably better to use arbitrary_servers extension than sharding:
DB.with_server(:host=>'hash_host_b', :database=>'backup') do
DB.synchronize do
# All queries here default to the backup database on hash_host_b
end
end
See:
http://sequel.jeremyevans.net/rdoc/files/doc/sharding_rdoc.html#label-arbitrary_servers+Extension
Related
Is there a way to override the multi_insert_sql_strategy that is specified when using methods like multi_insert? I am using the ODBC adapter which falls back to :separate as the strategy. The database that I am connecting to (Snowflake) supports multiple rows in the VALUES clause and as such, I'd like to leverage :values as the strategy instead. I have not found this to be an option that I can pass in.
Default strategy:
https://github.com/jeremyevans/sequel/blob/9202d780b92626646c9faeff90a7f7b9d7b6c10d/lib/sequel/dataset/sql.rb#L1340
multi_insert code:
https://github.com/jeremyevans/sequel/blob/ff5d77cb60a61b41d3eb500344f287f0b9fbdb97/lib/sequel/dataset/actions.rb#L484
Options available for import which is used by multi_insert:
https://www.rubydoc.info/github/jeremyevans/sequel/Sequel%2FDataset:import
Yes, you can override the strategy:
DB.extend_datasets do
def multi_insert_sql_strategy; :values; end
end
In general, you may want to consider working on an Sequel adapter for Snowflake, as this is something the adapter is supposed to take care of.
I want to use FactoryGirl to build in-memory stubs of models, then have all ActiveRecord queries run against only those. For example:
# Assume we start with an empty database, a Foo model,
# and a Foo factory definition.
#foo_spec.rb
stubbed_foo = FactoryGirl.build_stubbed(:foo)
# Elsewhere, deep in the guts of application
Foo.first() # Ideally would return the stubbed_foo we created
# in the test. Currently this returns nil.
The solution might be to use an in-memory database. But is the above scenario possible?
If your reason for avoiding the database, is to speed up your tests, then there are better ways.
Use FactoryGirl.build as much as possible instead of create. This works as long as the record won't be fetched from the database by your code. This works well for unit tests with well-structured code. (For example, it helps to use Service Objects and unit test them independently.
For tests that actually need to read from the database (as in your Foo.first example call), you can use FactoryGirl.create and use transactional fixtures. This creates a database transaction at the beginning of each test example, and then rolls back the transaction at the end of the example. This can cause problems when you use callbacks in your ActiveRecord models such as after_commit.
If you use after_commit or other callbacks in your models that require the database transaction to close (or you use explicit transactions in your code), I recommend setting up DatabaseCleaner. Here's an example of to configure and use it: https://gist.github.com/RobinDaugherty/9f4e5f782d9fdbe191a23de30ad8b539
I like the simplicity of Pundit gem and I would like to make policies dynamic by storing them to database.
Basically I'm looking for a way to be able to change policies without need to redeploy the application.
1st way
Pundit policy is pure ruby code, so if you don't want to keep code inside database and evaluate it dynamically, I'd say the answer is no. It's unsafe. You may give it a go, though.
2nd way
But nothing prevents you from creating model which keeps rules in simple json and compare them using Pundit, e.g.:
class PostPolicy < ApplicationPolicy
def update?
access_setting = PolicySetting.find_by(key: self.class_name)
user.role.in?(access_setting['roles'])
end
end
Of course, complexity and flexibility of the tool directly depends on each other.
3rd way
Is just work around. You may set you authorisation project apart from the main one, so that it's deploys (zero-downtime, of course) would not affect the main big project.
4th way
Create your own DSL to be stored in Database
5th way
Use something like json-logic-ruby to store logic in database
Every time I call Model.new, and before calling .save, ActiveRecord seems to get a database connection (which might make sense since it needs to get the field names).
How do I prevent this from happening? I don't intend to save the model into the database. I'm just creating it and then passing it to other functions.
Why don't you create a version of the model that doesn't inherit ActiveRecord::Base Then you can pass it around as a data object and leave your database alone until you actually need it.
I have a class that reads from a DB on startup. I'd prefer to be able to store it in the session, but I get the following error when trying to do so:
ERROR TypeError: no marshal_dump is defined for class Mutex
Is what I'm doing possible/reasonable? If so how should I go about doing it? If not, whats a good alternative to storing the class instance in the session? Currently my workaround is just instantiating it whenever I need to use it, but that doesn't strike me as a good solution or one that will be able to scale.
A good alternative is to store the id of the record in the session. Then when you need that data again you'd use a helper to return the data either from memory or from the database. A perfect example is the pattern used in current_user helper methods found in many ruby authentication gems. You could modify this helper to use a cache layer if you find it to be a bottleneck, but I'd leave that as an optimization after the fact.
Issues of having to get the object into a marshaled format that will live happily in a session, there are issues with storage space, stale data and possibly unintentional exposure to confidential data.