MongoDB MongoMapper Ruby Replica Set Config - ruby

I'm using this config to connect to MongoDB with MongoMapper in my Sinatra application:
MongoMapper.connection = Mongo::Connection.new('localhost', 27017)
Now I have a replica set with 2 mongos on separate servers, 10.5.5.5, and 10.5.5.6. How do I setup the connection with both mongos? How do I add authentication to this connection?

I ended up doing this:
MongoMapper.connection = Mongo::MongoReplicaSetClient.new(
['10.5.5.5:27017', '10.5.5.6:27017'],
:read => :primary, :rs_name => 'name', :connect_timeout => 30, :op_timeout => 30
)
MongoMapper.database = "db_name"
MongoMapper.database.authenticate("user", "test123")
Works beautifully.

You should be able to set a different connection per model. But I guess this is not exactly what you trying to do.
class MyModel
include MongoMapper::Document
connection(Mongo::Connection.new('localhost', 27017))
set_database_name "my_database"
# ...
end
Or there is ReplSetConnection with this you can set your replications sets:
MongoMapper.connection = Mongo::ReplicaSetConnection.new(['10.5.5.5', 30000], [' 10.5.5.6', 30000])
And the authentication is simple:
MongoMapper.connection = Mongo::Connection.new('localhost', 27017)
MongoMapper.database = "DBNAME"
MongoMapper.database.authenticate("USERNAME", "PASSWORD")

Related

Ruby Datamapper connection with SSL to MariaDB on Amazon RDS

How do I establish a ruby Datamapper connection to MariaDB on Amazon RDS with SSL?
Here's what I did:
A non-SSL connection works when testing with:
uri = 'mysql://user:pass#host:port/db_name'
connection = DataObjects::Connection.new(uri)
=> #<DataObjects::Mysql::Connection:0x000056179a3a5921
connection.secure?
=> false
According to the MySQL datamapper wiki, an ssl connection requires the following options: :ssl_ca, :client_key, and :client_cert.
This would result in the following code:
uri = 'mysql://user:pass#host:port/db_name?'
ssl_opts = 'ssl[ssl_ca]=file&ssl[client_key]=file&ssl[client_cert]=file'
connection = DataObjects::Connection.new(uri + ssl_opts)
connection.secure?
=> false
However the only files get is the RDS combind CA bundle, refered from the RDS docs
I do not have a client_cert at all.
Connecting with the mysql client on cli works with SSL:
mysql --ssl -h host -u user -p pass db_name
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 1638
Server version: 10.1.26-MariaDB MariaDB Server
In doc
https://github.com/datamapper/do/wiki/MySQL
It also says
as tested only ca_cert was required to connect to RDS.
So try adding only ca_cert path and do a test
There's only one parameter required: :ssl => {:ca_cert => 'pem_file'}.
However it looks like using uri string for configuration does not work. The reason is a limitation in Addressable::Uri. It cannot handle query strings which aim to represent hashes with more than 1 level.
The good news is that it works using DataMapper.setup with a config Hash:
DataMapper.setup(:default,
:adapter => 'mysql',
:user => 'user',
:database => 'db_name',
:host => 'host',
:password => 'pass',
:ssl => {
:ca_cert => '/path/to/rds-combined-ca-bundle.pem'
}
)

Ruby net/ssh(/multi) nested connections

I'm attempting to use ruby to connect to mongodb. But first I need to connect to the server hosting mongo.
There is a bastion host that is able to be connected to publicly. The database host is only able to be connected to through the bastion.
Here is my code:
require 'net/ssh/gateway'
require 'mongo'
include Mongo
ops_host = '<PUBLIC IP/DNS>'
db_host = '<PRIVATE IP>'
port = '27017'
user = 'ubuntu'
key = File.read("<PATHTOKEY>")
gateway = Net::SSH::Gateway.new( ops_host, user, :key_data => key, :keys_only => TRUE)
gateway.ssh(db_host, user, :key_data => key, :keys_only => TRUE) do |ssh|
puts ssh.exec!("hostname")
client = MongoClient.new(db_host, port)
db = client.db('<DBNAME>')
coll = db.collection('profiles')
puts coll.find().count()
end
I'm able to connect to the database host as the puts ssh.exec!("hostname") command returns the correct hostname, but I cannot get mongo to connect as I keep recieving Failed to connect to a master node errors.

Reuse mongodb connection in ruby script

I have a ruby script with several connections to a MongoDB I've since found that each query/insert is opening a new connection leaving me lots of connections in use.
ruby-2.0.0p195-2.el6.x86_64 & mongodb-2.4.12-1.el6.x86_64
I have several connections as per the examples below throughout my script. How do I use a single, or as few as possible connections so that I don't end up with 100's in use at any one time. My script is split into def foo / end chunks where some blocks have just 1 db action, others have 3 or 4.
#Insert into db
db = Mongo::Connection.new.db("room1-web")
coll = db.collection("room1")
coll.update({"_id" =>a}, rabbitdb, {upsert:true})
#Insert into db
db = Mongo::Connection.new.db("room1-web")
coll = db.collection("room1")
coll.update({"_id" =>b}, chickendb, {upsert:true})
#Do query on db to update indicators etc.
db = Mongo::Connection.new.db("room1-indi-lookup")
coll = db.collection("elements")
kitty = coll.find({"_id" => table[address][i], "state" => char}, :fields => {"_id" => 0, "state" => 0}).to_a
#conn = Mongo::Connection.new("localhost", 27017, :pool_size => 5, :pool_timeout => 5)
db = #conn.db(.....)

Mongodb not inserting Ruby time.new consistantly on Heroku

Built a small app to grab Tweets from political candidates for the upcoming election. Using Ruby, Twitterstream, Mongodb and Heroku.
The time is being inserted into the database inconsistantly. Sometimes it works, sometimes it doesn't. Is this my code, Heroku or Mongodb (Mongohq). I have a support question in.
Working
{
_id: ObjectId("52556b5bd2d9530002000002"),
time: ISODate("2013-10-09T14:42:35.044Z"),
user: "Blondetigressnc",
userid: 1342776674,
tweet: "RT #GovBrewer: Mr. President #BarackObama, reopen America’s National Parks or let the states do it. #GrandCanyon #Lead http://t.co/kkPKt9B7…",
statusid: "387951226866110464"
}
Not working
{
_id: ObjectId("52556c2454d4ad0002000016"),
user: "PeterMcC66",
userid: 1729065984,
tweet: "#GovBrewer #Blondetigressnc #BarackObama Time to impeach surely?",
statusid: "387952072223506432"
}
Seems random. See anything wrong or stupid in my code?
require 'rubygems'
require 'tweetstream'
require 'mongo'
# user ids
users = 'list of Twitter user ids here'
# connect to stream
TweetStream.configure do |config|
config.consumer_key = ENV['T_KEY']
config.consumer_secret = ENV['T_SECRET']
config.oauth_token = ENV['T_TOKEN']
config.oauth_token_secret = ENV['T_TOKEN_SECRET']
config.auth_method = :oauth
end
# connection to database
if ENV['MONGOHQ_URL']
uri = URI.parse(ENV['MONGOHQ_URL'])
conn = Mongo::Connection.from_uri(ENV['MONGOHQ_URL'])
DB = conn.db(uri.path.gsub(/^\//, ''))
else
DB = Mongo::Connection.new.db("tweetsDB")
end
# creation of collections
tweets = DB.create_collection("tweets")
deleted = DB.create_collection("deleted-tweets")
#client = TweetStream::Client.new
#client.on_delete do | status_id, user_id |
puts "#{status_id}"
timenow = Time.new
id = status_id.to_s
deleted.insert({ :time => timenow, :user_id => user_id, :statusid => id })
end
#client.follow(users) do |status|
puts "[#{status.user.screen_name}] #{status.text}"
timenow = Time.new
id = status.id
tweets.insert({ :time => timenow, :user => status.user.screen_name, :userid => status.user.id, :tweet => status.text, :statusid => id.to_s })
end
The issue is that you need to use a UTC time, not your local timezone. This is not a MongoDB or a Ruby driver issue, its a constraint of the BSON spec and the ISODate BSON type.
http://docs.mongodb.org/manual/reference/bson-types/#date
http://bsonspec.org/#/specification
Also, just a good practice though.
General word of advice: Use UTC on the back-end of whatever you're building always anyway, regardless of what datastore you're using (not a MongoDB specific thing). This is especially true if this data is something you want to query on directly.
If you need to convert to a local timezone, its best to handle that when you display or output the data rather than trying to manage that elsewhere. Some of the most fantastic bugs I've ever seen were related to inconsistent handling of timezones in the persistence layer of the application.
Keep those times consistent on the back-and, deal with local timezone conversion when in your application and life will be much easier for you.
Here is an examples of how to work with times in MongoDB using Ruby:
require 'time' # required for ISO-8601
require 'mongo'
include Mongo
client = MongoClient.new
coll = client['example_database']['example_collection']
coll.insert({ 'updated_at' => Time.now.utc })
doc = coll.find_one()
doc['updated_it'].is_a?(Time) #=> true
doc['updated_at'].to_s #=> "2013-10-07 22:43:52 UTC"
doc['updated_at'].iso8601 #=> "2013-10-07T22:43:52Z"
doc['updated_at'].strftime("updated at %m/%d/%Y") #=> "updated at 10/07/2013"
I keep a gist of this available here:
https://gist.github.com/brandonblack/6876374

Redis equivalent of ActiveRecord::Base.logger = Logger.new(STDOUT)?

What's the redis equivalent of ActiveRecord::Base.logger = Logger.new(STDOUT)? I want to see what redis is up to.
$redis.client.logger = Rails.logger
It's better!
I usually want to set some additional attributes (such as the redis DB name). You can pass on arguments in the initialize method like this
$redis = Redis.new(db: Rails.config.redis.db, logger: Rails.logger)
Never mind. It's easy:
$redis = Redis.new(:host => 'localhost', :port => 6379)
$redis.client.logger = Logger.new(STDOUT)
I wrote a gem call RedisLogger:
https://github.com/hellolucky/redis_logger

Resources