Elastic Beanstalk and Rails – initializing multiple times? - ruby

When I deploy my RoR (4.2.6) application to ElasticBeanstalk, it appears that the initialization process is getting run four times. This is impacting the way I rely on a singleton instance of a job Scheduler object (using Rufus Scheduler).
In a couple of initializer files and in application.rb, I added a few log statements:
Here:
# /config/initializers/scheduler.rb
require 'rufus-scheduler'
::Rufus_lockfile = "/tmp/.rufus-scheduler.lock"
::Scheduler = Rufus::Scheduler.singleton(
:lockfile => Rufus_lockfile
)
Rails.logger.info "1: started Scheduler #{Scheduler.object_id}"
And here:
# /config/initializers/cookies_serializer.rb
Rails.logger.info "2: some other initializer"
And here:
# /config/application.rb
require File.expand_path('../boot', __FILE__)
require 'rails/all'
Bundler.require(*Rails.groups)
module MyApp
class Application < Rails::Application
...
config.after_initialize do
Rails.logger.info "3: after app is initialized"
end
end
end
After I run eb deploy and it completes, this is what I see at the top of app/log/production.log:
I, [2016-05-31T10:31:08.756302 #18753] INFO -- : 2: some other initializer
I, [2016-05-31T10:31:08.757252 #18753] INFO -- : 1: started Scheduler 47235057343600
I, [2016-05-31T10:31:08.896353 #18753] INFO -- : 3: after app is initialized
I, [2016-05-31T10:31:23.669517 #18817] INFO -- : 2: some other initializer
I, [2016-05-31T10:31:23.670380 #18817] INFO -- : 1: started Scheduler 46989489069800
I, [2016-05-31T10:31:23.806154 #18817] INFO -- : 3: after app is initialized
D, [2016-05-31T10:31:23.969103 #18817] DEBUG -- : ^[[1m^[[36mActiveRecord::SchemaMigration Load (1.3ms)^[[0m ^[[1mSELECT "schema_migrations".* FROM "schema_migrations"^[[0m
I, [2016-05-31T10:31:33.108449 #18897] INFO -- : 2: some other initializer
I, [2016-05-31T10:31:33.109513 #18897] INFO -- : 1: started Scheduler 47156425207060
I, [2016-05-31T10:31:33.116500 #18901] INFO -- : 2: some other initializer
I, [2016-05-31T10:31:33.117374 #18901] INFO -- : 1: started Scheduler 47156425216940
I, [2016-05-31T10:31:33.790266 #18901] INFO -- : 3: after app is initialized
I, [2016-05-31T10:31:33.844517 #18897] INFO -- : 3: after app is initialized
So it looks like the initializer files and even the code in my after_initializer block are getting run four times... and I can't figure out why.

Question got asked on rufus-scheduler issue system, answered there: https://github.com/jmettraux/rufus-scheduler/issues/208

Related

How to not exit a for loop inside a pytest although few items fail

I would like to run the pytest for all the items in the for loop. The pytest should fail in the end but it should run all the elements in the for loop.
The code looks like this
#pytest.fixture
def library():
return Library( spec_dir = service_spec_dir)
#pytest.fixture
def services(library):
return list(library.service_map.keys())
def test_properties(service, services):
for service_name in services:
model = library.models[service_name]
proxy = library.get_service(service_name)
if len(model.properties ) != 0 :
for prop in model.properties:
try:
method = getattr(proxy, f'get_{prop.name}')
method()
except exception as ex:
pytest.fail(ex)
The above code fails if one property of one service fails. I am wondering if there is a way to to run the test for all the service and get a list of failed cases for all the services.
I tried parametrize But based on this stackoverflow discussion. The parameter list should be resolved during the collection phase and in our case the library is loaded during the execution phase. Hence I am also not sure if it can be parametrized.
The goal is run all the services and its properties and get the list of failed items in the end.
I moved the variables to the global scope. I can parametrize the test now\
library = Library( spec_dir = service_spec_dir)
service_names = list(library.service_map.keys())
#pytest .mark.paramertize("serivce_name", service_names)
def test_properties(service):
pass
Don't use pytest.fail, but pytest_check.check instead.
fail point, is that you stop test execution on condition, while check made for collect how much cases were failed.
import logging
import pytest
import pytest_check as check
def test_000():
li = [1, 2, 3, 4, 5, 6]
for i in li:
logging.info(f"Test still running. i = {i}")
if i % 2 > 0:
check.is_true(False, msg=f"value of i is odd: {i}")
Output:
tests/main_test.py::test_000
-------------------------------- live log call --------------------------------
11:00:05 INFO Test still running. i = 1
11:00:05 INFO Test still running. i = 2
11:00:05 INFO Test still running. i = 3
11:00:05 INFO Test still running. i = 4
11:00:05 INFO Test still running. i = 5
11:00:05 INFO Test still running. i = 6
FAILED [100%]
================================== FAILURES ===================================
__________________________________ test_000 ___________________________________
FAILURE: value of i is odd: 1
assert False
FAILURE: value of i is odd: 3
assert False
FAILURE: value of i is odd: 5
assert False

Chef error on attributes- string not matched

I cannot seem to get around a chef error that deals with similarly named attributes in my attributes/default.rb file.
I have 2 attributes:
default['test']['webservice']['https']['keyManagerPwd'] = 'password'
...
...
default['test']['webservice']['https']['keyManagerPwd']['type'] = 'encrypted'
Notice that, up until the last bracket (['type']), the names are identical.
I am referencing these attributes in a template and in a template block in the recipe. When I go to run it, I receive this error:
==================================================[0m
I, [2015-01-28T13:36:43.668692 #7920] INFO -- core-14-2-centos-65:
[31mRecipe Compile Error
in /tmp/kitchen/cache/cookbooks/avx/attributes/default.rb[0m
I, [2015-01-28T13:36:43.669192 #7920] INFO -- core-14-2-centos-65:
=================================================================[0m
I, [2015-01-28T13:36:43.669192 #7920] INFO -- core-14-2-centos-65:
I, [2015-01-28T13:36:43.669692 #7920] INFO -- core-14-2-centos-65:
[0mIndexError[0m
I, [2015-01-28T13:36:43.669692 #7920] INFO -- core-14-2-centos-65: -------
--[0m
I, [2015-01-28T13:36:43.669692 #7920] INFO -- core-14-2-centos-65: string not matched[0m
I, [2015-01-28T13:36:43.670192 #7920] INFO -- core-14-2-centos-65:
I, [2015-01-28T13:36:43.670192 #7920] INFO -- core-14-2-centos-65:
[0mCookbook Trace:[0m
I, [2015-01-28T13:36:43.670692 #7920] INFO -- core-14-2-centos-65: --------[0m
I, [2015-01-28T13:46:05.101875 #8332] INFO -- core-14-2-centos-65:
[0m113>> default['webservice']['https']['keyManagerPwd']['type'] =
'encrypted'
It seems as if Chef cannot distinguish between 2 attributes when the only differentiation is the ending.
If I modify the same attributes by placing some unique text at the front of the name, there is not issue with the recipe at all:
default['test']['1']['webservice']['https']['keyManagerPwd'] = 'password'
...
...
default['test']['2']['webservice']['https']['keyManagerPwd']['type'] = 'encrypted'
By putting the ['1'] and ['2'] there, it solves the problem.
I am fairly new to Chef so I'm thinking its just something simple I'm overlooking. Does anyone have any ideas or suggestions? Thanks.
Simple answer: You cannot do this. That's not a Chef problem, nor a ruby problem - it's a general problem of like most programming languages.
Let's use foo as a variable instead of the lengthy default['test']['webservice']['https']['keyManagerPwd'].
What you effectively do is
1: foo = "password"
2: foo['type'] = "encrypted"
In line 1, foo is a string. In line 2, it is treated as hash (called array in some other languages). The second line automatically overwrites your foo = "password" assignment. It's effectively the same as
1: foo = "password"
2: foo = {}
3: foo['type'] = "encrypted"
The alternative would be to use
foo['something'] = "password"
foo['type'] = "encrypted"
Or translated to your code:
default['test']['webservice']['https']['keyManagerPwd']['something'] = 'password'
default['test']['webservice']['https']['keyManagerPwd']['type'] = 'encrypted'
This should work.

ExecJS::ProgramError: /bootstrap-theme.css.map:1:14: Unknown word

rails 4.1.7
ruby-2.1.4
Using bootstrap-sass 3.3.1.0 gem
When I try to push to Heroku I get the following error:
$ git push heroku master
...
-----> Preparing app for Rails asset pipeline
Running: rake assets:precompile
I, [2014-11-21T01:06:34.863128 #1054] INFO -- : Writing /tmp/build_5e094815d056adf463e7c75e987d5e63/public/assets/glyphicons-halflings-regular-3ba17edd9d971727ac7482518167c417.eot
I, [2014-11-21T01:06:34.865429 #1054] INFO -- : Writing /tmp/build_5e094815d056adf463e7c75e987d5e63/public/assets/glyphicons-halflings-regular-9fdfa71474063341410181a0580f8cc9.svg
I, [2014-11-21T01:06:34.867462 #1054] INFO -- : Writing /tmp/build_5e094815d056adf463e7c75e987d5e63/public/assets/glyphicons-halflings-regular-b1be6b5ab502863bf4997057c9e81c47.ttf
I, [2014-11-21T01:06:34.869377 #1054] INFO -- : Writing /tmp/build_5e094815d056adf463e7c75e987d5e63/public/assets/glyphicons-halflings-regular-d1bdcb334cbe33c43cd1644d34c214bc.woff
I, [2014-11-21T01:06:39.656791 #1054] INFO -- : Writing /tmp/build_5e094815d056adf463e7c75e987d5e63/public/assets/application-16641e09fe68375d82a616882c87cd48.js
rake aborted!
ExecJS::ProgramError: /bootstrap-theme.css.map:1:14: Unknown word
...
On my config/application.rb file I have:
require File.expand_path('../boot', __FILE__)
require 'rails/all'
Bundler.require(*Rails.groups)
module Evolution
class Application < Rails::Application
config.assets.initialize_on_precompile = false
end
end
Any idea how to fix this error?
UPDATE #1
If I search for the word "bootstrap-theme" in the App directory, I find:
4 matches across 2 files:
/Users/andreucasadella/rails_projects/evolution/app/assets/stylesheets/bootstrap-theme.css:
468 box-shadow: inset 0 1px 3px rgba(0, 0, 0, .05), 0 1px 0 rgba(255, 255, 255, .1);
469 }
470: /*# sourceMappingURL=bootstrap-theme.css.map */
And
/Users/andreucasadella/rails_projects/evolution/app/assets/stylesheets/bootstrap-theme.css.map:
1: {"version":3,"sources":["less/theme.less","less/mixins/vendor-prefixes.less","bootstrap-theme.css","less/mixins/gradients.less","less/mixins/reset-filter.less"],"names":...
I understand that the "Unknown word" is on the file /bootstrap-theme.css.map and on line 1:14.
How do I identify space 14?
I have not been able to find an answer, therefore I decided to delete both map files:
- bootstrap-theme.css.map
- bootstrap.css.map
I will not be able live-edit my preprocessor source files in the DevTools Sources panel, but at least I can go on. Hope to find a better answer soon.
what are the .map files used for in Bootstrap 3.1?

Calling Sinatra from Sinatra produces results different from external request

Following my question here.
So I am trying to do a smart redirect using this:
get "/category/:id/merge" do
#... setting #catalog_id and category
call env.merge("PATH_INFO" => "/catalog/#{#catalog_id}/category/#{category.id}", "REQUEST_METHOD"=>"PATCH","QUERY_STRING"=>"merge=1")
status 200
end
But when I look in logs, I see something that is not only frustrating but also completely absurd:
# this one is from internal call
I, [2013-03-21T15:55:54.382153 #29569] INFO -- : Processing GET /catalog/1/category/2686/merge
I, [2013-03-21T15:55:54.382239 #29569] INFO -- : Parameters: {}
...
I, [2013-03-21T15:55:54.394992 #29569] INFO -- : Processing PATCH /catalog/1/category/2686
I, [2013-03-21T15:55:54.395041 #29569] INFO -- : Parameters: {"merge"=>"1"}
I, [2013-03-21T15:55:54.395560 #29569] INFO -- : Processed PATCH /catalog/1/category/2686?merge=1 with status code 404
I, [2013-03-21T15:55:54.395669 #29569] INFO -- : Processed GET /catalog/1/category/2686/merge with status code 200
# this one is a direct request
I, [2013-03-21T15:56:36.246535 #29588] INFO -- : Processing PATCH /catalog/1/category/2686
I, [2013-03-21T15:56:36.246629 #29588] INFO -- : Parameters: {"merge"=>"1"}
...
I, [2013-03-21T15:56:36.286216 #29588] INFO -- : Processed PATCH /catalog/1/category/2686?merge=1 with status code 204
And the body of internal 404 request is just Sinatra's standard 404 error page. How the hell can he tell me straight in the eye that he doesn't know this route if I caught him serving exactly the same URL with acceptable 204?
UPDATE
It gets even more exciting when I change REQUEST_METHOD to GET - works like a charm.
I, [2013-03-21T17:09:37.718756 #3141] INFO -- : Processing GET /catalog/1/category/2686/merge
I, [2013-03-21T17:09:37.718838 #3141] INFO -- : Parameters: {}
...
I, [2013-03-21T17:09:37.735632 #3141] INFO -- : Processing GET /catalog/1/category/2686
I, [2013-03-21T17:09:37.735678 #3141] INFO -- : Parameters: {"merge"=>"1"}
...
I, [2013-03-21T17:09:37.773033 #3141] INFO -- : Processed GET /catalog/1/category/2686?merge=1 with status code 200
I, [2013-03-21T17:09:37.773143 #3141] INFO -- : Processed GET /catalog/1/category/2686/merge with status code 200

Handling chunked request in Sinatra

In Sinatra I have a route setup similar to the following:
put '/test' do
begin
logger.info 'In put begin block'
write_to_file(request.body.read)
[200, '']
rescue RuntimeError => e
[500, 'some error']
end
end
def write_to_file(data)
logger.info "writing data with size #{data.size}"
# Code to save file...
end
When I send data that is < ~500 MBytes it everything seems to work correctly but when I attempt to send data that is >= 500 MBytes I get some weird log output and then the client eventually errors out with the following error: Excon::Errors::SocketError: EOFError (EOFError)
The logs from the server (Sinatra) look like the following:
For data < 500 MBytes:
I, [2013-01-07T21:33:59.386768 #17380] INFO -- : In put begin block
I, [2013-01-07T21:34:01.279922 #17380] INFO -- : writing data with size 209715200
xxx.xxx.xxx.xxx - - [07/Jan/2013 21:34:22] "PUT /test " 200 - 22.7917
For data > 500 MBytes:
I, [2013-01-07T21:47:37.434022 #17386] INFO -- : In put begin block
I, [2013-01-07T21:47:41.152932 #17386] INFO -- : writing data with size 524288000
I, [2013-01-07T21:48:16.093683 #17380] INFO -- : In put begin block
I, [2013-01-07T21:48:20.300391 #17380] INFO -- : writing data with size 524288000
xxx.xxx.xxx.xxx - - [07/Jan/2013 21:48:39] "PUT /test " 200 - 62.4515
I, [2013-01-07T21:48:54.718971 #17386] INFO -- : In put begin block
I, [2013-01-07T21:49:00.381725 #17386] INFO -- : writing data with size 524288000
I, [2013-01-07T21:49:33.980043 #17267] INFO -- : In put begin block
I, [2013-01-07T21:49:41.990671 #17267] INFO -- : writing data with size 524288000
xxx.xxx.xxx.xxx - - [07/Jan/2013 21:50:06] "PUT /test " 200 - 110.2076
xxx.xxx.xxx.xxx - - [07/Jan/2013 21:51:22] "PUT /test " 200 - 108.5339
Not entirely sure whats going on here so I guess my question is two fold, A. What is fundamentally different between these two cases that would cause them to behave this way? B. Is there a better way to handle data to mitigate against this?
The problem turned out to be with apache/passenger. Running the server with WEBrick alleviated the issue.

Resources