Rails 6 ActionMailbox with Mailgun returning 404 - ruby

I am developing an in-house expenses application on Rails 6 with ActionMailbox to replace Excel spreadsheets. As many of our receipts are now in email form (for example airline tickets), the idea is that users will be able to simply forward a receipt to the application and it will automatically be associated with the expense entry.
I am using ActionMailbox with Mailgun as the email receiver. As suggested by the Gorails Pro tutorial I have exposed my app using localtunnel to the general Internet. I have used Mailgun's facility to send a test email to my application.
My post address is:
https://xxxxxxxx.localtunnel.me/rails/action_mailbox/mailgun/inbound_emails/mime
However, I have run into an issue where the incoming email from Mailgun is not being processed correctly but is being returned with a 404 error. The Rails log shows the message being received as a POST. The last two entries in the log are:
2019-10-15T07:50:07.646Z 10260 TID-gn609ivg8 INFO: Filter chain halted as :ensure_configured rendered or redirected
2019-10-15T07:50:07.646Z 10260 TID-gn609ivg8 INFO: Completed 404 Not Found in 0ms (ActiveRecord: 0.0ms | Allocations: 144)
My configuration is:
config/routes.rb
Rails.application.routes.draw do
devise_for :users
resources :categories
resources :expense_claims do
get 'export_excel', on: :member
post 'barclay_csv_import', on: :collection
end
resources :expense_entries
root 'expense_claims#index'
# Enable the sidekiq console.
require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'
end
config/application.rb
module Expenses
class Application < Rails::Application
# Initialize configuration defaults for originally generated Rails version.
config.load_defaults 5.2
# Settings in config/environments/* take precedence over those specified here.
# Application configuration can go into files in config/initializers
# -- all .rb files in that directory are automatically loaded after loading
# the framework and any gems in your application.
# Set the ActionMailbox ingress here for now.
config.action_mailbox.ingress = :mail_gun
end
end
config/environments/development.rb
Rails.application.configure do
[... lot of stuff removed as not relevant]
# Settings specified here will take precedence over those in config/application.rb.
config.action_mailer.default_url_options = { host: '0.0.0.0', port: 3000 }
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = { address: '0.0.0.0', port: 1025 }
# Set the active job queue adapter to Sidekiq/Redis
# config.active_job.queue_adapter = :sidekiq
# Alternatively, when debugging, you can set to in-line (or :async)
config.active_job.queue_adapter = :inline
# Don't care if the mailer can't send.
config.action_mailer.raise_delivery_errors = false
config.action_mailer.perform_caching = false
# Set so we can test Devise self registration
config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }
# Allow traffic from localtunnel
config.hosts << 'xxxxxx.localtunnel.me'
end
app/mailboxes/application_mailbox.rb
class ApplicationMailbox < ActionMailbox::Base
routing :all => :receipt
end
app/mailboxes/receipt_mailbox.rb
class ReceiptMailbox < ApplicationMailbox
# mail => Mail object
# inbound_email => ActionMailboxEmail record
def process
end
end

Well, turns out it was the most stupid of mistakes. In config/application.rb the final configuration was wrong. I needed to replace :mail_gun with :mailgun:
# Set the ActionMailbox ingress here for now.
config.action_mailbox.ingress = :mailgun
Unfortunately, the Rails error message was not very helpful here.
To give a little more info, ActionMailbox has a number of different routes defined for different email processors. Therefore, the URL to which the email processor posts the incoming email defines the controller to use. On receiving an email, Rails checks that the URL to which the email has been posted matches the ingress class that is set in the config. If not, you get the error I saw.

Related

Rails 6 Active Job. Why I can't send async tasks?

I was following the Action Mailer guideon https://guides.rubyonrails.org/action_mailer_basics.html#calling-the-mailer
I did almost the same of what tutorial show. The controller:
def create
#user = User.create(user_params)
if #user && UserMailer.with(user: #user).welcome_email.deliver_later
token = encode_token({ user_id: #user.id })
render json: { token: token }, status: :created
else
render json: #user.errors.messages, status: :bad_request
end
end
The Mailer:
class UserMailer < ApplicationMailer
default from: 'notifications#example.com'
def welcome_email
#user = params[:user]
#url = 'http://example.com/login'
mail(to: #user.email, subject: 'Welcome to My Awesome Site')
end
end
But when I make the request Active Job yells:
ActiveJob::SerializationError => "Unsupported argument type: User"
It works correctly using deliver_now
This shows that we have a problem with :async adapter. But as the guide says:
Active Job's default behavior is to execute jobs via the :async adapter. So, you can use deliver_later to send emails asynchronously. Active Job's default adapter runs jobs with an in-process thread pool. It's well-suited for the development/test environments, since it doesn't require any external infrastructure, but it's a poor fit for production since it drops pending jobs on restart. If you need a persistent backend, you will need to use an Active Job adapter that has a persistent backend (Sidekiq, Resque, etc).
So what I'm missing here?
It's not working because ActiveJob doesn't support the objects. To make it accessible you have to either convert it into json string and then deserialise in the mailer method.
Try sending the object as json string and retrieve the id value, then use:
#user = User.find(params[:user_id])
Another approach is to use Resque or Sidekiq to process these jobs. They come really handy.
Another sources to help people out:
https://stackoverflow.com/a/40896988/5832250
If you want to go with rails guides and learn more:
https://guides.rubyonrails.org/active_job_basics.html#globalid

devise reset password token invalid

controller
def create
# admin manually creates user
UserMailer.reset_password_instructions(#user).deliver
end
user.rb
class User < ActiveRecord::Base
before_create :generate_reset_password_token # generating devise reset token
# Include default devise modules. Others available are:
# :confirmable, :lockable and :omniauthable
# :registerable,
# :trackable,
devise :database_authenticatable,
# :confirmable,
:rememberable,
:validatable,
:recoverable,
:trackable,
:timeoutable
private
# Generates a new random token for confirmation, and stores
# the time this token is being generated
def generate_reset_password_token
raw, enc = Devise.token_generator.generate(self.class, :reset_password_token)
#raw_confirmation_token = raw
self.reset_password_token = enc
self.reset_password_sent_at = Time.now.utc
end
end
user_mailer.rb
class UserMailer < ApplicationMailer
include Devise::Mailers::Helpers
default from: 'no-reply#identt.co'
def reset_password_instructions(resource, opts={})
#resource = resource
#token = #resource.reset_password_token
mail(to: #resource.email, subject: "Reset Password Instructions")
end
end
reset_password_instructions.html.erb
<p>Hello <%= #resource.email %>!</p>
<p>Someone has requested a link to change your password. You can do this through the link below.</p>
<p><%= link_to 'Change my password', edit_password_url(#resource, reset_password_token: #token) %></p>
<p>If you didn't request this, please ignore this email.</p>
<p>Your password won't change until you access the link above and create a new one.</p>
At this point, when user is created manually by admin, Password reset Link is going to the email address, which I can see using MailCatcher or letter_opener.
http://lvh.me:3000/users/password/edit?reset_password_token=6a8bc4683fc9e5dfcc789f94f9b6bd2b1c44fd857f13662d0f0d1f6212022f81
I click on the link and it successfully took me to
edit password page. When I submit form, ivalidation failed with Reset password token is invalid message.
What am I missing here....
UPDATE:
My Development.rb looks like:
Rails.application.configure do
# Settings specified here will take precedence over those in config/application.rb.
# In the development environment your application's code is reloaded on
# every request. This slows down response time but is perfect for development
# since you don't have to restart the web server when you make code changes.
config.cache_classes = false
# Do not eager load code on boot.
config.eager_load = false
# Show full error reports and disable caching.
config.consider_all_requests_local = true
config.action_controller.perform_caching = false
# Don't care if the mailer can't send.
config.action_mailer.raise_delivery_errors = false
# Print deprecation notices to the Rails logger.
config.active_support.deprecation = :log
# Raise an error on page load if there are pending migrations.
config.active_record.migration_error = :page_load
# Debug mode disables concatenation and preprocessing of assets.
# This option may cause significant delays in view rendering with a large
# number of complex assets.
config.assets.debug = true
# Asset digests allow you to set far-future HTTP expiration dates on all assets,
# yet still be able to expire them through the digest params.
config.assets.digest = true
# Adds additional error checking when serving assets at runtime.
# Checks for improperly declared sprockets dependencies.
# Raises helpful error messages.
config.assets.raise_runtime_errors = true
# Raises error for missing translations
# config.action_view.raise_on_missing_translations = true
# Configure letter opener to open email in browser
# config.action_mailer.delivery_method = :letter_opener
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = { :address => "lvh.me", :port => 1025 }
config.action_mailer.default_url_options = { host: 'lvh.me', port: 3000 }
config.domain = 'lvh.me'
end
There was a one line code for my solution, which I have made complicated by adding manual mailer, actions, etc.
To solve this problem I just have to call devise's send_reset_password_instructions in user object:
In controller
#user.send_reset_password_instructions
Solved my problem.
I cleaned up my code by removing (as per my question:)
user_mailer.rb file is no more required, so deleted it
views/user_mailer/reset_password_instructions.html.erb file is not required, so deleted it.
In User.rb model, remove before_action :generate_reset_password_token as well as generate_reset_password_token private method.
Remove below mailer line from controller
UserMailer.reset_password_instructions(#user).deliver
I went crazy deep with this, and finally found the answer:
def generate_reset_password_token
raw, enc = Devise.token_generator.generate(self.class, :reset_password_token)
#raw_confirmation_token = raw
self.reset_password_token = enc
self.reset_password_sent_at = Time.now.utc
end
This code is right, you want the user to have enc as the reset_password_token. It's also good that you keep the raw variable handy.
class UserMailer < ApplicationMailer
include Devise::Mailers::Helpers
default from: 'no-reply#identt.co'
def reset_password_instructions(resource, opts={})
#resource = resource
#token = #resource.reset_password_token
mail(to: #resource.email, subject: "Reset Password Instructions")
end
end
For this part, you want #token = #raw_confirmation_token (raw from the token generator), not #resource.reset_password_token (which is enc from the generator).
I believe this solution is for devise 3.1+, it seems they changed their setup for added security, without explaining the two tokens.

heroku: ActionController::RoutingError (No route matches [GET] "/newrelic")

ERROR
ActionController::RoutingError (No route matches [GET] "/newrelic")
# and I am getting error page for both staging and production heroku servers
Documentation
https://devcenter.heroku.com/articles/newrelic
GEM
https://github.com/newrelic/rpm
# ruby 2.1.0p0
# rails 4.0.1
Both environment variables NEW_RELIC_LICENSE_KEY and NEW_RELIC_APP_NAME are set to heroku config variable
Gemfile
gem "newrelic_rpm", "~> 3.5.7.59"
config/newrelic.yml
common: &default_settings
license_key: <%= ENV['NEW_RELIC_LICENSE_KEY'] %>
app_name: <%= ENV["NEW_RELIC_APP_NAME"] %>
monitor_mode: true
developer_mode: false
log_level: info
browser_monitoring:
auto_instrument: true
audit_log:
enabled: false
capture_params: false
transaction_tracer:
enabled: true
transaction_threshold: apdex_f
record_sql: obfuscated
stack_trace_threshold: 0.500
error_collector:
enabled: true
capture_source: true
ignore_errors: "ActionController::RoutingError,Sinatra::NotFound"
development:
<<: *default_settings
monitor_mode: true
developer_mode: true
test:
<<: *default_settings
monitor_mode: false
production:
<<: *default_settings
monitor_mode: true
staging:
<<: *default_settings
monitor_mode: true
app_name: <%= ENV["NEW_RELIC_APP_NAME"] %> (Staging)
[NOTE: I have two application hosted to heroku:]
(staging).herokuapp.com
(production).herokuapp.com
And I want to configure new-relic for both environments/servers.
Also, Note that this configuration is working fine in development(localhost) environmet.
EDITED
config/routes.rb
Demo::Application.routes.draw do
root :to => "home#index"
devise_for :users,:controllers => {:sessions => "sessions",:omniauth_callbacks => "omniauth_callbacks" }
post '/tinymce_assets' => 'tinymce_assets#create'
resources :home
namespace :admin do
resources :dashboards
resources :users do
member do
get :reset
put :reset_pw
put :delete_record
put :restore_user
end
end
end
resources :drives do
member do
put :destroy_drive
post :add_consolidation
put :delete_consolidation
post :add_driveorganizer
put :delete_drive_organizer
put :restore_drirve
end
collection do
get :recalculate_consolidation
end
resources :drive_details do
resources :images
end
end
resources :products do
member do
post :add_category
put :destroy_pc
put :delete_product
put :restore_products
end
end
resources :stores do
member do
put :delete_store
end
end
resources :store_products do
member do
put :delete_storeproduct
post :add_package_items
put :delete_package_item
put :restore_store_product
get :get_product_price
end
collection do
get :autocomplete_others
end
end
resources :orders do
member do
put :delete_order
put :restore_order
get :charge_stripe
end
resources :order_items do
collection do
post :display_price
end
member do
put :delete_record
end
end
end
resources :categories do
member do
put :delete_category
put :restore_category
end
collection do
get :move
end
end
namespace :user do
resources :campaigns do
member do
get :single_campaign
end
resources :stores
resources :carts do
collection do
post :carts_update
get :checkout_final
get :payment
post :payment
get :update_payment
get :update_payment_and_redirect
get :confirmation
post :confirmation
get :finish
post :confirmation_update
put :empty_cart
post :shelter_survey
end
member do
put :destroy_oi
put :checkout
end
end
end
end
# The priority is based upon order of creation: first created -> highest priority.
# See how all your routes lay out with "rake routes".
# You can have the root of your site routed with "root"
# root 'welcome#index'
# Example of regular route:
# get 'products/:id' => 'catalog#view'
# Example of named route that can be invoked with purchase_url(id: product.id)
# get 'products/:id/purchase' => 'catalog#purchase', as: :purchase
# Example resource route (maps HTTP verbs to controller actions automatically):
# resources :products
# Example resource route with options:
# resources :products do
# member do
# get 'short'
# post 'toggle'
# end
#
# collection do
# get 'sold'
# end
# end
# Example resource route with sub-resources:
# resources :products do
# resources :comments, :sales
# resource :seller
# end
# Example resource route with more complex sub-resources:
# resources :products do
# resources :comments
# resources :sales do
# get 'recent', on: :collection
# end
# end
# Example resource route with concerns:
# concern :toggleable do
# post 'toggle'
# end
# resources :posts, concerns: :toggleable
# resources :photos, concerns: :toggleable
# Example resource route within a namespace:
# namespace :admin do
# # Directs /admin/products/* to Admin::ProductsController
# # (app/controllers/admin/products_controller.rb)
# resources :products
# end
end
Thanks
Based on the error you're getting, it looks like you're trying to access the route to the New Relic Ruby agent's developer mode in your staging and production environments. Developer mode installs a middleware in your app that responds to any URL prepended with /newrelic. Because you've enabled Developer mode in your development (localhost) environment (the developer_mode key is set to true in your newrelic.yml under development), accessing this route succeeds there, but it fails in staging and production because you don't have developer mode enabled in those environments.
Your current configuration is usually desirable, since Developer mode introduces a large amount of overhead that is generally unacceptable in production. Rather than attempt to access the route in staging or production, use it during development only.
You may also want to consider upgrading the version of the agent you are using, since version 3.5.7 does not fully support Ruby 2.0 or Rails 4. More information on releases can be found at https://docs.newrelic.com/docs/releases/ruby.

Creating a variable per instance, rather than per request, with Sinatra modular style

I have a Sinatra app, written in modular style, running on Heroku. It uses Redis and I have a limited number (10) of Redis connections. I found that it would often throw errors complaining that it had run out of Redis connections. So I started using connection_pool in the hope that would fix things; a single pool of Redis connections and the app would choose one of those each time, rather than try to create a new connection on each request.
But I'm still getting the same issue. I can do loads of Redis queries on a single query without complaints. But if I reload a single test page, which just does some Redis queries, several times in fairly quick succession, I get the "Redis::CommandError - ERR max number of clients reached" error again.
So I'm assuming, maybe, it's creating a new instance of connection_pool on each request... I don't know. But it's not "pooling" as I would expect it to.
I have this kind of thing:
# myapp.rb
$LOAD_PATH.unshift(File.dirname(__FILE__))
$stdout.sync = true
require 'thin'
require 'myapp/frontend'
MyApp::Frontend.run!
And the Sinatra app:
# myapp/frontend.rb
require 'sinatra/base'
require 'redis'
require 'connection_pool'
require 'uuid'
module MyApp
class Frontend < Sinatra::Base
helpers do
def redis_pool
#redis_pool ||= ConnectionPool.new(:size => 8, :timeout => 5) do
redis_uri = URI.parse(ENV['REDISCLOUD_URL'])
client = ::Redis.new(:host => redis_uri.host,
:port => redis_uri.port,
:password => redis_uri.password)
end
end
end
get '/tester/'
redis_pool.with do |r|
id = UUID.generate
r.hset(:user, id, "Some data")
r.hget(:user, id)
r.hdel(:user, id)
end
p "DONE"
end
end
end
The Procfile looks like:
web: ruby myapp.rb
Any ideas? The current site is pretty low traffic, so this should be possible.
A new instance of #redis_pool is created every time a get request for /tester/ is processed because the helper method redis_pool is called every time.
You can use sinatra's settings helper to initialize a redis connection only once:
config do
redis_uri = URI.parse(ENV['REDISCLOUD_URL'])
set :redis, Redis.new(:host => redis_uri.host,
:port => redis_uri.port,
:password => redis_uri.password)
end
Now the each instance of the app has one redis connection that persists for all requests. Access the setting like so
get '/tester/'
id = UUID.generate
settings.redis.hset(:user, id, "some data")
settings.redis.hget(:user, id)
settings.redis.hdel(:user, id)
p "DONE"
end

Rails3 omniauth google authentication on Returns User Identity

My Omniauth integration works on local development but fails for google on staging.
require 'omniauth/openid'
require 'openid/store/memcache'
Rails.application.config.middleware.use OmniAuth::Builder do
OmniAuth.config.full_host = "http://xx.xx.xxx/"
# dedicated openid
provider :open_id, OpenID::Store::Memcache.new(Dalli::Client.new), :name => 'google', :identifier => 'https://www.google.com/accounts/o8/id'
end
I get a this error message:
Started GET "/auth/failure?message=invalid_credentials" for 58.71.19.178 at 2011-12-01 02:22:20 +0000
Processing by ErrorsController#routing as HTML
Parameters: {"message"=>"invalid_credentials", "a"=>"auth/failure"}
Rendered public/404.html (0.1ms)
Completed 404 Not Found in 1ms (Views: 0.6ms | ActiveRecord: 0.0ms)
Also the ip in for is not the same in my OmniAuth.config.full_host maybe that could be causing the issue?
The culprit was that apache sending and returning on different ips
This monkey patch fixed the issue.
module OmniAuth
module Strategies
# OmniAuth strategy for connecting via OpenID. This allows for connection
# to a wide variety of sites, some of which are listed [on the OpenID website](http://openid.net/get-an-openid/).
class OpenID
protected
def callback_url
uri = URI.parse(request.url)
uri.path += '/callback'
# by KirylP: to overcome hosting subdomain forwarding to rails port
uri.port = '' if request.env.has_key? 'HTTP_X_FORWARDED_SERVER'
uri.to_s
end
end
end
end
module Rack
class OpenID
SERVER_PORT_TO_AVOID = 12002
private
def realm_url(req)
url = req.scheme + "://"
url << req.host
scheme, port = req.scheme, req.port
if scheme == "https" && port != 443 ||
scheme == "http" && port != 80
url << ":#{port}" if port != SERVER_PORT_TO_AVOID # KirylP
end
url
end
end
end
module OpenID
class Consumer
def complete(query, current_url)
message = Message.from_post_args(query)
current_url.sub!(":#{Rack::OpenID::SERVER_PORT_TO_AVOID}", '') # KirylP
mode = message.get_arg(OPENID_NS, 'mode', 'invalid')
begin
meth = method('complete_' + mode)
rescue NameError
meth = method(:complete_invalid)
end
response = meth.call(message, current_url)
cleanup_last_requested_endpoint
if [SUCCESS, CANCEL].member?(response.status)
cleanup_session
end
return response
end
end
end
I had a similar problem. Seems like your google authentication fails (can be for different reasons - invalid credentials, or user denied access), therefore you receive callback to /auth/failure -- and then you get 404.
Did you implement a route for /auth/failure in your routes.rb? In my current project:
in routes.rb
match '/auth/failure', :to => 'sessions#failure'
in sessions_controller
def failure
redirect_to session[:return_uri] || root_path, alert: "Sorry, we were not able to authenticate you using your chosen sign on method"
end

Resources