I'm looking for recommendations on how to implement multi-tenancy with couchrest model in a rails app. For my multi-tenant app, I'm thinking of two options:
{ edit - removed my ugly options because they'll only confuse future readers }
I would like this to work well with 10K users.
SOLUTION:
Based on Sam's advice, here's what I did and it's working well -
In my case, I needed to override the proxy_database method because the standard naming for proxy databases didn't match my naming.
created the master
class Site < CouchRest::Model::Base
property :name
property :slug
proxy_for :users
proxy_for ...(all the other multi-tenant models)
# Databases are on same server in this example
def proxy_database
#db ||= self.server.database!(slug)
end
end
Then in each multi-tenant model
class User < CouchRest::Model::Base
...
proxied_by :site
In ApplicationHelper create a 'site' method that you can reuse in all your controllers.
module ApplicationHelper
def site
db_name = current_user.db_name
#site ||= Site.create(slug: "#{db_name}_#{Rails.env}" )
end
Then controller might do something like:
def show
user = site.users.find(params[:id])
render :json => user
end
You might want to checkout the Proxying feature of CouchRest Model for this. More details can be found here:
http://www.couchrest.info/model/proxying.html
Although I have no personal experience, I understand that CouchDB handles >10k databases. Here is a good thread of ways of scaling the number of users:
http://comments.gmane.org/gmane.comp.db.couchdb.user/13862
A few considerations to take into account when dealing with lots of databases:
File system sub-directory count, not a problem with Ext4.
Namespace databases to split between sub-directories and/or servers.
System open file limit. Usually around 10k. Probably fine if not all databases are accessed at the same time.
Hope that helps.
Related
I use multible databases with ActiveRecord. I must put establish_connection db in all models. But i want to call the connection of libraries file. While put establish_connection db in for all models Database connection count is too much . How can I do other methods?
My project is a Ruby on Sinatra.
Build a global hash of {db_name => connection_instance } for all possible connections you have and retrieve them from your models intelligently:
def get_or_establish_connection db
$connections[db] ||= establish_connection db
end
That way you’ll reuse all already established connections.
NB I wrote “global” because there is no way to suggest anything more robust without knowledge of your architecture details. In the real life one would put this hash as a class variable somewhere accessible from everywhere.
What we did in our projects that used multiple databases was creating one class per database, that establish the connection, and make models inherit from it. That way, only one connection is created per database.
class UsageTable < ActiveRecord::Base
self.abstract_class = true
establish_connection :usage
end
class User < UsageTable
I'm little new to Ruby, but I have the following coded using Grape API in Ruby. I have #data = YAML.load() called every time it hits GET /api/v1/foo, is there a way in Grape to only load it once and use it? This way is more optimized and not calling YAML.load() at every time. Should I override the initialize method and put a super() for this operation?
Thanks,
require 'grape'
require 'json'
require "yaml"
module MyProject
CONFIG_FILE = "./config.yml"
class Api < Grape::API
rescue_from :all
prefix 'api'
version 'v1'
format :json
resources :foo do
get do
#data = YAML.load(File.open(MyProject::CONFIG_FILE))
end
end
end
end
The short answer is that Grape doesn't work quite how you think, and attribute variables of the MyProject::Api are not the way forward for your new web service. However, it is an interesting question, and worth exploring why this is so.
If you add a puts self.inspect inside the resources :foo block, and run using rackup, when you call the route you should see that self is in fact a Grape::Endpoint object. Also, no matter what you try to do with instance variables, they will always start in the same state for each request. That is because Grape turns your route definitions into prepared Grape::Endpoint objects, with a lot of the definition data and setup put into a quickly-accessible form (so that it is not figured out on each request). Eventually, on each request, the matching Grape::Endpoint object including your block (and other details you defined for the route) is duplicated before being called, meaning that state is not maintained between requests.
This may seem complicated, but most frameworks covering web service requests will do something similar. Generally you don't want request-handling state to persist between requests. Frameworks with larger scope - e.g. Rails - have places to put more persistent data planned out for you. Grape does not have this defined, which has its pros and cons. One obvious plus point is that you are more free to use whatever other data persistence approach that you wish to.
23tux's answer will sort you out immediately for loading config. Although I'm not entirely sure how ##data becomes accessible to the endpoint block (it may even be creating a closure around the variable).
Longer term, you should look to moving config management out of your MyProject::Api class, and including it as a module via Grape's helpers method (I'm happy to provide an example if you are interested).
Edit: Example based on your current code, but moving config management to a separate module:
require 'grape'
require 'json'
require "yaml"
module MyProject
module Config
CONFIG_FILE = "./config.yml"
##data = nil
def config
##data ||= YAML.load( File.open( CONFIG_FILE ) )
end
end
class Api < Grape::API
rescue_from :all
prefix 'api'
version 'v1'
format :json
helpers MyProject::Config
resources :foo do
get do
config
end
end
end
end
This is one step further, structurally, than 23tux's answer, but is still not completely separating concerns of storage (and caching etc) versus api access. As you progress towards a more sophisticated web service, you will want to keep the Grape route definitions simple, with only a small amount of logic that manages or manipulates the data - well, at least as seen directly in the blocks.
One way to link between your Grape definitions, and other gems that might manage config, logging, authentication and other services, is via Grape's helpers method. Grape also has some built-in helper methods for common tasks.
The main exception to using helpers MyModule to add shared functions into Grape is when you want to manage displaying data objects (aka "models") from your core application. For that you have a few choices, but the grape-entity gem and the present method is not a bad place to start.
If the #data is the same for the whole api, and doesn't change at any time, just use a class variable
require 'grape'
require 'json'
require "yaml"
module MyProject
CONFIG_FILE = "./config.yml"
class Api < Grape::API
##data = YAML.load(File.open(MyProject::CONFIG_FILE))
rescue_from :all
prefix 'api'
version 'v1'
format :json
resources :foo do
get do
puts ##data
end
end
end
end
Not tested, but with this way, you ensure that the data is only loaded once, when your Api class is loaded
I'm attempting to create a restful, json api in ruby - so I'm using grape (https://github.com/intridea/grape) inside of Rack.
I'm not using Rails for this project, so cancan, sorcery, etc... don't seem to be the best options. Plus, I'd hate to mix in a bunch of imperative logic into grape's declarative DSL.
While grape has built in authentication support, I do not see anything about authorization. It seems like this would be a common enough use case that this road would have been traveled before, but after some pretty thorough digging in google and the grape codebase itself I have turned up nothing.
Has anyone implemented something like this for their project in grape? What did you use?
This may be a little too late, but anyway. I'd recommend you use Pundit for authorization, it's deadly simple. To use it in your Grape API endpoints, you would need to include Pundit helpers:
class API < Grape::API
format :json
helpers Pundit
helpers do
def current_user
resource_owner
end
end
mount FoosAPI
end
Now in you API endpoints, you should be able to use authorize foo, action? as you would always do in Rails controllers:
class FoosAPI < Grape::API
get ':id' do
foo = Foo.find(params[:id])
authorize foo, :show?
present foo, with: FooEntity
end
end
Hope it helps!
I thought I can give short comment on this, but the field is to short, sorry if it will not be right answer but:
You mentioned sorcery - I think it is authentication system and got nothing to do with an authorization. (I do not know sorcery gem implementation - just repeating statement from documentation and assuming that description enumerates such systems it replaces and it is valid definition). I guess it is just mistake.
Fundamental question you should ask yourself is...
How much role-based system do you develop? I think if this is only matter of public/private/admin roles probably you should consider just moving it to different APIs.
That can be cumbersome in some circumstances but worth a try for not complicated none additive roles. Simple mount in grape will solve the problem OOTB.
Real problem is if you think about some expandable/dynamic role system or you want to be just DRY. That can be painful ;-). I think Rayan's Bytes cancan gem implementation should help you understand how such problem can be solved on higher abstract level. For particular (without higher abstraction - such as dynamic roles) implementation it should be fine to just use currently given helpers from grape and delegate their responsibilities to model (basic usage).
helpers do
def current_user
#current_user ||= User.authorize!(env)
end
def authenticate!
error!('401 Unauthorized', 401) unless current_user
end
end
so all the story is about how to implement User.authorize!(env) and I believe that should be done in your model and depends strictly on your needs.
I don't know whether my answer is in time for you. I recently have the same problem with the Grape and authorization in a Rails4 project. And after trying, I found out a way for this.
In my project, I'm using the pundit for authorization, it asks me to create a policy folder, and create the authorize rules for each Model, each rule is a Ruby class, something like this(from pundit Github page)
class PostPolicy < ApplicationPolicy
def update?
user.admin? or not record.published?
end
end
then in the Grape API, I just use this class for the authorization, code like this:
desc "hide a post"
post :hide do
authenticate!
error!( "user unauthorized for this" ) unless PostPolicy.new(current_user, #post).hide_post?
#post.update hidden: true
{ hidden: #post.hidden }
end
the authenticate! and current_user helpers are customized helpers. By this way, I can reuse the authorize rules created when developing website parts.
This works for me. Hope the Pundit way can solve your problems for Grape authorization
as all of us, I have some application. In database I store sensitive data and I want to overwrite ActiveRecord (or somewhere else?) to always add AND user_id = current_user statement to all SQL queries sent to database (I will add user_id column to all tables). I basically want to ensure, that all operations done by user, are performed ONLY on his data.
Regards,
Mateusz
Sounds like what you're after is a strategy for implementing multitenancy in your application.
You might want to checkout the multitenant gem which can help you to isolate your queries to data belonging to the current tenant.
Alternatively, you can enforce this at the database level using PostgreSQL schemas. Gay Naor has an excellent talk on implementing this strategy among others.
If you search on "multi-tenant rails apps" you'll find several other discussions on this problem.
There is no way to enforce it on application level, the only way to enforce it is to forbid to team to use Class.find/where/etc methods and allow to call them only on collections. So instead of Task.find you should always use current_user.tasks.find etc.
You can add AbstractModel with default scope and then inherit all other models from it:
class AbstractModel < ActiveRecord::Base
self.abstract_class = true
default_scope where(:user_id => Thread.current[:current_user])
end
class SomeModel < AbstractModel
...
end
I've used Thread.current[:current_user] because of current_user is inaccessible inside of models. You should assign current_user to Thread.current in some before_filter method in ApplicationController etc.
I am having some trouble getting routing to play nicely with Single Table Inheritance in my Ruby on Rails application. I am using Ruby 1.9.2 and Rails 3.0.6. This is in development so the back-end is SQLite3, in case that makes a difference.
Let's say I have two products, widgets and sprockets. My application keeps track of bug numbers and support case tickets for both the products but the bugs and support tickets themselves are stored in other systems. There are two separate teams that work on these two products.
I have implemented Single Table Inheritance for the two types of bug records because the validation rules for widget bug numbers and sprockets bug numbers are different (the two teams use different bug tracking systems) and there is a possibility that I will have to add further products to the application that behave wildly differently. Using STI gives me the flexibility to implement additional methods and properties as required.
The widgets team only cares about widgets information and the sprockets team only cares about sprockets information. There is a third team that needs to be able to view the information on both the widgets and the sprockets. The widgets team will access the application using the path /widgets and the sprockets team will access the application using the path /sprockets. I set this up in routes.rb using namespaces:
resources :bugs
namespace "widgets" do
resources :bugs
end
namespace "sprockets" do
resources :bugs
end
I have set up the following models which work as expected when I fire up irb and use WidgetBug.create() or SprocketBug.create():
bug.rb
class Bug < ActiveRecord::Base
end
widget_bug.rb
class WidgetBug < Bug
# Some validation rules
end
sprocket_bug.rb
class SprocketBug < Bug
# Some different validation rules
end
I used scaffolding to create the controller and the view for the bug object, then modified the controller to try to generalize it so it could be used with both widgets bugs and sprockets bugs. For example, the index method looks like this:
def index
# The scaffold code lists all bugs, which is not what we want
# #bugs = Bug.all
# Only return bugs of the subclass we're looking for
#bugs = eval("#{params[:controller].classify}.all")
respond_to do |format|
format.html # index.html.erb
format.xml { render :xml => #bugs }
end
end
I then used create() to populate the database with several bugs of each type. Unfortunately when I browse to /widgets/bugs, the bugs for both products appear. After some debugging, I determined that the classify call is returning Widgets::Bugs or Sprockets::Bugs, so when I call all on it, it appears to be running against the superclass instead of running against the subclass.
I've reviewed the routing documentation and done quite a bit of searching on Google but I'm still at a loss as to how I can change the routing or the controller to make this work correctly. Any help would be greatly appreciated.
Checkout this post: STI, one controller
Routes
resources :widgets, :controller => "bugs", :type => "Widget"
resources :sprockets, :controller => "bugs", :type => "Sprocket"
# And I don't know if you need this
resources :bugs, :type => "Bug"
Controller
def index
#bugs = params[:type].constantize.all
end
UPD
namespace "widgets" do
resources :bugs, :type => "Widget"
end
namespace "sprockets" do
resources :bugs, :type => "Sprocket"
end
I wrote a blog post on STI in Rails 3 that discusses some common pitfalls and proper work arounds, including the problem you mention. http://www.christopherbloom.com/2012/02/01/notes-on-sti-in-rails-3-0/