Environment: Ruby 1.9.2
I am new to Ruby/Sinatra and am creating a proof of concept web application. The purpose is fairly simple: A user inputs a list of domains and the script first checks their mx records, and if they meet the condition, pulls out the domain contact information. I am fairly sure that I am not going about storing the data appropriately and am looking for a more elegant solution that will enable me to style the results such that domain, email, and name are grouped together.
#!/usr/bin/env ruby
require "sinatra/base"
require 'rubygems'
require 'haml'
require 'sinatra'
require 'whois'
get '/' do
haml :index
end
post '/' do
#host = params[:host]
#host.split('\n')
#email = Array.new
#name = Array.new
#domain = Array.new
#host.each_line {|i|
if %x[dig -t mx #{i.chomp.gsub('www.', '')} | grep -i mx | grep -i google].empty?
puts "empty"
else
#domain << i.chomp.gsub('www.','')
#email << (Whois.whois(i.chomp.gsub('www.',''))).technical_contact.email
#name << (Whois.whois(i.chomp.gsub('www.',''))).technical_contact.name
end
}
haml :index
end
__END__
## layout
%html
%head
%title Gcrawl
%body
#header
%h1 Gcrawl
#content
=yield
%footer
## index
%p
Welcome to Gcrawl
%form(action='/' method='POST')
%textarea{:rows => '12', :cols => '40', :name => 'host'}
%input(type='submit')
- if defined?(#email)
%h3= #domain
%h3= #email
%h3= #name
Create a class Record which will contain #name, #domain and #email for a particular entry.
So, each instance of Record will have it's own name, domain and email.
Replace the array implementation with a Class. If you need to store the records in a database, use ActiveRecord.
It's good that you started with Sinatra, but if you are in a hurry, you can get your app running on Rails in an hour.
EDIT
Tutorials/Guides for getting started with Rails:
Rails Guides - Getting Started
rubyonrailstutorials.com
railstutor.com
Related
I've been working for a while to try to use the .uniq method to generate a unique list of URL's from a website (within the /informatics path). No matter what I try I get a method error when trying to generate the list. I'm sure it's a syntax issue, and I was hoping someone could point me in the right direction.
Once I get the list I'm going to need to store these to a database via ActiveRecord, but I need the unique list before I get start to wrap my head around that.
require 'nokogiri'
require 'open-uri'
require 'active_record'
ARGV[0]="https://www.nku.edu/academics/informatics.html"
ARGV.each do |arg|
open(arg) do |f|
# Display connection data
puts "#"*25 + "\nConnection: '#{arg}'\n" + "#"*25
[:base_uri, :meta, :status, :charset, :content_encoding,
:content_type, :last_modified].each do |method|
puts "#{method.to_s}: #{f.send(method)}" if f.respond_to? method
end
# Display the href links
base_url = /^(.*\.nku\.edu)\//.match(f.base_uri.to_s)[1]
puts "base_url: #{base_url}"
Nokogiri::HTML(f).css('a').each do |anchor|
href = anchor['href']
# Make Unique
if href =~ /.*informatics/
puts href
#store stuff to active record
end
end
end
end
Replace the Nokogiri::HTML part to select only those href attributes that matches with /*.informatics/ and then you can use uniq, as it's already an array:
require 'nokogiri'
require 'open-uri'
require 'active_record'
ARGV[0] = 'https://www.nku.edu/academics/informatics.html'
ARGV.each do |arg|
open(arg) do |f|
puts "#{'#' * 25} \nConnection: '#{arg}'\n #{'#' * 25}"
%i[base_uri meta status charset content_encoding, content_type last_modified].each do |method|
puts "#{method.to_s}: #{f.send(method)}" if f.respond_to? method
end
puts "base_url: #{/^(.*\.nku\.edu)\//.match(f.base_uri.to_s)[1]}"
anchors = Nokogiri::HTML(f).css('a').select { |anchor| anchor['href'] =~ /.*informatics/ }
puts anchors.map { |anchor| anchor['href'] }.uniq
end
end
See output.
I have a basic banking application running on Heroku using Sinatra.
I have tried implementing sessions to ensure each user that visits has a different version of the app. However, at the moment, if I visit it with two separate browsers, I have the same data.
There is no backend database implemented but data I add via the interface persists in every browser I visit in.
Here is my app.rb:
require 'sinatra/base'
require 'tilt/erb'
require 'require_all'
require_all 'lib'
require 'rufus-scheduler'
class BankingApp < Sinatra::Base
enable :sessions
set :session_secret, 'super secret'
get '/' do
session[:accounts] = AccountsController.instance
session[:holders] = HoldersController.instance
session[:loans] = LoansController.instance
erb :index
end
get '/holders' do
#holders = session[:holders].store
erb :holders
end
get '/holders_accounts' do
#holder = session[:holders].find(params[:id].to_i)
message = session[:accounts].get_accounts_of(params[:id].to_i)
#accounts = message.accounts
erb :holders_accounts
end
get '/new_holder' do
erb :new_holder
end
post '/new_holder' do
#message = session[:holders].create(params[:name])
#holders = session[:holders].store
erb :holders
end
get '/create_account' do
erb :create_account
end
post '/create_account' do
type = :Current
id = params[:id].to_i
#message = session[:accounts].open(type, with: id)
erb :index
end
get '/accounts' do
#accounts = session[:accounts].store
erb :accounts
end
get '/transactions' do
message = session[:accounts].get_transactions_of(params[:id].to_i)
#transactions = message.transactions
erb :transactions
end
get '/deposit' do
erb :deposit
end
post '/deposit' do
#accounts = session[:accounts].store
#message = session[:accounts].deposit(params[:amount].to_i, into: params[:id].to_i)
erb :accounts
end
get '/withdraw' do
erb :withdraw
end
post '/withdraw' do
#accounts = session[:accounts].store
#message = session[:accounts].withdraw(params[:amount].to_i, from: params[:id].to_i)
erb :accounts
end
get '/transfer' do
erb :transfer
end
post '/transfer' do
#accounts = session[:accounts].store
#message = session[:accounts].transfer(params[:amount].to_i, from: params[:donar].to_i, to: params[:recipitent].to_i)
erb :accounts
end
get '/add_holder' do
erb :add_holder
end
post '/add_holder' do
#accounts = session[:accounts].store
#message = session[:accounts].add_holder(params[:holder_id].to_i, to: params[:account_id].to_i)
erb :accounts
end
get '/enable_overdraft' do
erb :enable_overdraft
end
post '/enable_overdraft' do
#accounts = session[:accounts].store
#message = session[:accounts].activate_overdraft(params[:id].to_i, params[:amount].to_i)
erb :accounts
end
get '/disable_overdraft' do
erb :disable_overdraft
end
post '/disable_overdraft' do
#accounts = session[:accounts].store
#message = session[:accounts].deactivate_overdraft(params[:id].to_i)
erb :accounts
end
get '/loans' do
#loans = session[:loans].store
erb :loans
end
get '/loan_view' do
message = session[:loans].show(params[:id].to_i)
#transactions = message.transactions
erb :loan_view
end
get '/new_loan' do
erb :new_loan
end
post '/new_loan' do
#loans = session[:loans].store
id = params[:id].to_i
options = { borrowed: params[:amount].to_i, term: params[:term].to_i, rate: params[:rate].to_f }
#message = session[:loans].create_loan(id, options)
erb :loans
end
get '/pay_loan' do
erb :pay_loan
end
post '/pay_loan' do
#message = session[:loans].pay(params[:amount].to_i, off: params[:id].to_i)
#loans = session[:loans].store
erb :loans
end
# start the server if ruby file executed directly
run! if app_file == $0
end
I do not have a great deal of experience with Sinatra so apologies if this is an oversight on my part.
Any help greatly appreciated.
So I'm pretty sure the core of your problem is this line which you have in multiple places throughout your code. As per the ruby docs
This ensures that only one instance of Klass can be created.
You've explicitly told ruby to only ever let one copy of each class/module ever exist. I don't think that's what you want.
It's a bit hard to infer what it is you're trying to achieve exactly but I don't think using Singleton in a web app is going to be the right solution. The assumptions it imposes break down as soon as you run a 2nd instance (or dyno in Heroku parlance) of your app.
I would recommend using this
get '/logout' do
session.clear
end
I am trying to take data from a path in Sinatra, and use it to look up a particular record using Datamapper. The Datamapper docs seem to indicate that.
get "/test/:test_path" do
test_get = Intake.get( params[:test_path] )
# Do stuff
erb :blah_blah_blah
end
should find any records associated with the symbol :test_path
This does not work. test_get gets nil.
Meanwhile, what does work is
get "/test/:test_path" do
test_all = Intake.all(:test_path => params[:test_path] )
# Do stuff
erb :blah_blah
end
My two questions are:
What am I doing wrong with the .get() call in Datamapper?
Is the .all(:name => value) method slower than .get(), or does it not matter which I use?
Here's a Sinatra script pared down to demonstrate the behavior.
#!/usr/bin/env ruby
require 'rubygems'
require 'sinatra'
require 'dm-core'
require 'dm-timestamps'
DataMapper.setup(:default, {:adapter => 'yaml', :path => 'db'})
class Intake
include DataMapper::Resource
property :id, Serial
property :created_at, DateTime
property :test_path, String
end
get "/test/:test_path" do
test_all = Intake.all(:test_path => params[:test_path] )
puts 'test_all:' test_all.inspect
test_get = Intake.get( params[:test_path] )
puts 'test_get:' test_get.inspect
"Hello World!"
end
#get only does a lookup based on primary key, with is the id. So
Intake.get(params[:test_path])
looks for something with id params[:test_path], which will fail. Use
Intake.first(test_path: params[:test_path])
I'm using Sinatra, EventMachine, DataMapper, SQLite3 and the Twitter Stream API to capture and save tweets. When I run the application from my command line, it seems to continually fail at tweet 50. If I'm not saving the tweets, it can run seemingly forever.
Below is the app code to capture tweets with 'oscar' in them, which provided a very quick stream. Just enter your twitter username and password and run at the command line.
require 'rubygems'
require 'sinatra'
require 'em-http'
require 'json'
require 'dm-core'
require 'dm-migrations'
USERNAME = '<your twitter username>'
PASSWORD = '<your secret password>'
STREAMING_URL = 'http://stream.twitter.com/1/statuses/filter.json'
DataMapper.setup(:default, ENV['DATABASE_URL'] || "sqlite3://#{Dir.pwd}/db/development.db")
class Tweet
include DataMapper::Resource
property :id, Serial
property :tweet_id, String
property :username, String
property :avatar_url, String
property :text, Text
end
DataMapper.auto_upgrade!
get '/' do
#tweets = Tweet.all
erb :index
end
def rip_tweet(line)
#count += 1
tweet = Tweet.new :tweet_id => line['id'],
:username => line['user']['screen_name'],
:avatar_url => line['user']['profile_image_url'],
:text => line['text']
if tweet.save
puts #count
else
puts "F"
end
end
EM.schedule do
#count = 0
http = EM::HttpRequest.new(STREAMING_URL).get({
:head => {
'Authorization' => [ USERNAME, PASSWORD]
},
:query => {
'track' => 'oscars'
}
})
buffer = ""
http.stream do |chunk|
buffer += chunk
while line = buffer.slice!(/.+\r?\n/)
rip_tweet JSON.parse(line)
end
end
end
helpers do
alias_method :h, :escape_html
end
I'm not sure you can safely mix EM and Sinatra in the same process. You might want to try splitting the Sinatra viewer and the EventMachine downloader into separate programs and processes.
I wrote this little application :
require 'rubygems'
require 'sinatra'
require 'bson'
require 'mongoid'
Mongoid.configure do |config|
name = "articles"
host = "localhost"
config.master = Mongo::Connection.new.db(name)
config.persist_in_safe_mode = false
end
class Article
include Mongoid::Document
field :title
field :content
end
get '/' do
#articles = Article.all
end
get '/show/:id' do
#article = Article.find(params[:id])
end
get '/new' do
haml :new
end
post '/create' do
#article = Article.new(params['article'])
if #article.save
redirect '/'
else
redirect '/new'
end
end
The following error occur when i post an article with a content "Test d'un article en français"
BSON::InvalidStringEncoding at /create String not valid UTF-8
How i can fix this error ?
Thanks
This is a known issue with Ruby 1.9 and Sinatra. Wait for Sinatra 1.1 to be released or use Sinatra edge version from github.