Why is assets:precompile so slow on Heroku? - heroku

My deployment set up is to have Heroku precompile assets. I want them precompiled so I can send them to a CDN (via asset_sync) and I want that done on Heroku so I don't have any compiled assets in my repo. I set that up using Heroku's guide and that had been working great. I added turbo-sprockets-rails3 for a speed boost. It had all been working fine and then stopped; I can't figure out why.
Now when I deploy (without public/assets) the assets:precompile Rake task times out on Heroku. To see how long it's really taking, I went into a shell (heroku run bash):
~ $ time bundle exec rake assets:precompile
AssetSync: YAML file found /app/config/asset_sync.yml settings will be merged into the configuration
AssetSync: Syncing.
Using: Directory Search of /app/public/assets
AssetSync: Done.
real 19m29.575s
user 17m43.690s
sys 0m28.480s
Twenty minutes! But when I run it locally (2011 iMac), it's only only a few minutes:
$ env RAILS_ENV=production time bundle exec rake assets:precompile --trace
** Invoke assets:precompile (first_time)
** Execute assets:precompile
~/.rbenv/versions/1.9.3-p327-perf/bin/ruby ~/.rbenv/versions/1.9.3-p327-perf/bin/rake assets:precompile:all RAILS_ENV=production RAILS_GROUPS=assets --trace
** Invoke assets:precompile:all (first_time)
** Invoke assets:cache:clean (first_time)
** Invoke assets:environment (first_time)
** Execute assets:environment
AssetSync: YAML file found ~/Code/Ruby/myproject-rails/config/asset_sync.yml settings will be merged into the configuration
** Execute assets:cache:clean
** Execute assets:precompile:all
AssetSync: Syncing.
Using: Directory Search of ~/Code/Ruby/myproject-rails/public/assets
Uploading: assets/application-3a6de939348195e9610f1321df27837a.js
Uploading: assets/application-3a6de939348195e9610f1321df27837a.js.gz
Uploading: assets/jquery.min-959a5819c76e1508f5a8655c289c0de8.map
Uploading: assets/jquery.min.map
Uploading: assets/rails_admin/rails_admin-76cfbf31a605916a55eee29464ca3e6d.js
Uploading: assets/rails_admin/rails_admin-76cfbf31a605916a55eee29464ca3e6d.js.gz
AssetSync: Done.
179.92 real 19.29 user 3.98 sys
I wrote Heroku support over a week ago and after some days they said they'd escalate it to their Ruby buildback team, but I still haven't heard anything. Do any of the smart people on Stack Overflow have possible solutions?

The solution was to use tell Heroku to use Ruby 1.9.3, like my local machine. Apparently 1.9.3 is much faster at assets compilation than 1.9.2 which Heroku defaults to.
To fix it, make sure your Gemfile starts:
source "https://rubygems.org"
ruby "1.9.3"
# ...
More at Heroku docs on Ruby versions.

Related

Capistrano cap production deploy fails on migration but cap production deploy:migrate passes

I've come across a strange problem with Capistrano while getting a production server ready. When I run cap production deploy, it always fails on the deploy:migrate step with
ActiveRecord::AdapterNotSpecified: 'production' database is not
configured. Available: ["defaul…
Yet, when I run cap production deploy:migrate, Capistrano completes successfully.
On the actual server, I am able to run RAILS_ENV=production bundle exec rake:migrate without any problems either.
The Capistrano log spits out the same command for both:
[deploy:migrate] Run rake db:migrate
My database config looks like this:
production:
adapter: mysql2
encoding: utf8
database: foo
host: localhost
pool: 5
timeout: 5000
username: bar
password: password
socket: /opt/bitnami/mysql/tmp/mysql.sock
I should also mention that I have set the rails env in my deploy.rb like so: set :rails_env, :production
Here is the relevent information in my gemfile:
gem 'mysql2', '~> 0.4.5'
group :development do
gem 'capistrano', '~> 3.6', '>= 3.6.1'
gem 'capistrano-rvm'
gem 'capistrano-bundler'
gem 'capistrano-rails'
gem 'capistrano-passenger'
end
Can anyone shed some light on what I am missing here? I am new to Capistrano.
After a lot of searching I found the root cause.
I didn't realise that the "current" directory (or symlink to be fair) is created at the end of the Capistrano process and since I had an earlier trial run with RAKE_ENV=development, I already had the current symlink on the server.
Before the deploy:migrate step, I upload my database.yml from a secure location to the server via a custom task that uses this command:
upload! "#{SECURE_DATA}database.yml", "#{current_path}/config/"
Which at the time of before deploy:migrate, which to my confusion, I thought the rake db:migrate was being run from the current symlink, but it was in fact being run from the latest release directory. The latest release directory did not have the production database information as the file was uploaded to the current symlink which pointed to an old release.
All in all, my custom task caused this problem and if anyone else comes across a similar scenario, us this command instead:
upload! "#{SECURE_DATA}database.yml", "#{release_path}/config/"
The key being the release_path variable instead of the current_path variable.

Heroku: where did my gems go?

I have been deploying a Rails 3.2 application to heroku for several weeks now. I have also been executing rake tasks on the Cedar stack where my application is located.
One day after a deploy I noticed that rake was no longer working. I get, for example, the following:
$:~/dev/my_project$ heroku run rake -T
Running `rake -T` attached to terminal... up, run.7566
bundler: command not found: rake
Install missing gem executables with `bundle install`
Trying to run commands with bundle exec yields the same results.
What I've tried:
heroku run bundle install. This works and informs me that gems were installed into ./vendor/bundle. However, heroku run ls ./vendor/bundle yields only the following:
$:~/dev/my_project$ heroku run ls ./vendor/bundle/
Runningls ./vendor/bundle/attached to terminal... up, run.3458
bin ruby
bundle package. Although the deployment works it does not help my problem.
fiddling around with the rubygems-bundler gem (although I think this is now part of core bundler). This does not seem to have any effect.
On Heroku, gems are installed within the vendor/bundle/ruby/<version>/gems directory. I just checked my Heroku instance and confirmed this.
You are going to want to use bundle exec rake task because the gems are not in the users PATH.

Could not detect rake tasks

ensure you can run `$ bundle exec rake -P` against your app with no environment variables present
and using the production group of your Gemfile.
This may be intentional, if you expected rake tasks to be run
cancel the build (CTRL+C) and fix the error then commit the fix:
rake aborted!
I don't have any rake tasks that I'd like to run automatically. Should I just ignore this warning?
Effective December 05, 2013, Heroku added debug output to deploys using the Ruby build pack.
The error is triggered:
if the assets:precompile task does not exist or
if there is an error in the Rakefile.
Two common causes of errors in the Rakefile are referencing a dependency not available in production (such as rspec) or
expecting an environment variable to be present.
It's counter-intuitive because the app never runs without config vars, but Heroku's build process executes without config vars.
As per the error message, ensure you can run bundle exec rake -P RAILS_ENV=production with no environment variables present before pushing to Heroku (e.g. comment out your environment variables while running the aforementioned command).
Also, rest assured that rake's -P switch is harmless, so you can run it as much as you want until you fix this issue. This switch is used to display a list of all tasks and their immediate prerequisites. See Rake Command Line Usage if you want to double-check. The output may have over 200 lines, and look something like this:
rake about
environment
rake assets:clean
environment
rake assets:clobber
environment
rake assets:environment
rake assets:precompile
environment
rake db:_dump
...
rake tmp:pids:clear
rake tmp:sessions:clear
rake tmp:sockets:clear
I started getting this strange error all of a sudden yesterday. Heroku confirmed making an update to the Ruby buildpack...
It has to do with the Rakefile. Does your Rakefile require any files? Does it require your app files? If so, then the app should not raise exceptions when it is loaded without any config vars set.
It's counter-intuitive because the app never runs without the config vars set.
In my case, the Sinatra app was looking for database urls in the init file:
uri = URI.parse( ENV[ "REDISTOGO_URL" ])
This will raise an exception if there are no env vars set.
You may have the same issue with other database URL's, such as Mongo or Postgres.
So, protect against missing env vars:
if ENV[ "REDISTOGO_URL" ]
uri = URI.parse( ENV[ "REDISTOGO_URL" ])
...
You can check if it will work before pushing to Heroku by running bundle exec rake -P
Also, make sure all your tests pass after updating your init. Remove any cached init state by restarting Spork or similar.
Reference: Show Rakefile errors in Ruby deploys
you can also try enabling user-env-compile
heroku labs:enable user-env-compile
FWIW I just ran into this issue also.
It turned out that I had config.assets.css_compressor = :sass commented out, when there was a rake task referring to it in production.rb.
Very simple oversight, but that would cause rake assets:precompile to fail and thus cause this error.
Old question, but I ran into this issue just now, and there is a new fix for it. If you're using Ruby version <= 2.6.1, and Bundler 2.0.1, update Ruby to 2.6.3 ($ rvm install "ruby-2.6.3") and Bundler to 2.0.2 ($ gem install bundler '2.0.2'). Make sure to specify the new Ruby version in your Gemfile.
Unfortunately I can't tell you why this works, but it's worked for 3 other people on my team so far, so it's worth a shot.

windows heroku run rake db:migrate error "/usr/bin/env: ruby.exe: No such file or directory"

I'm pretty new to Rails here and I've followed Ruby on Rails Tutorial for most of it. I have since decided to make my own application, but using the Rails 4 gem in beta. I have installed PostgreSQL on my local machine (Windows 7) and done development. Everything works great! (I love Rails much more than C#, ASP, .NET work that I do for a living!)
However, once I decided to push to Heroku, I've been pulling my hair out. Prior to pushing to Heroku, I did the standard:
c:\Sites\elms>git add .
c:\Sites\elms>git commit -m "My comment"
c:\Sites\elms>git push
c:\Sites\elms>git push heroku master
I've downloaded and installed the Heroku Toolbelt (not heroku gem.) The first error told me I needed to make sure that my Gemfile specified Ruby >= 1.9.3, so I added "ruby 1.9.3" to the top of my Gemfile:
source 'https://rubygems.org'
ruby "1.9.3"
gem 'rails', '4.0.0.beta1'
gem 'pg'
gem 'haml-rails'
gem 'httparty'
gem 'devise', git: 'git://github.com/plataformatec/devise.git', branch: 'rails4'
gem 'cancan'
gem 'bootstrap-sass'
gem 'formtastic'
gem 'formtastic-plus-bootstrap'
group :development do
gem 'annotate'
gem 'better_errors'
gem 'binding_of_caller'
gem 'guard-spork'
gem 'guard-rspec'
gem 'meta_request'
gem 'rb-notifu'
gem 'schema_to_scaffold'
gem 'wdm', :platforms => [:mswin, :mingw], :require => false
gem 'win32console'
end
group :development, :test do
gem 'rspec-rails'
gem 'pry'
gem 'spork-rails', git: 'git://github.com/sahilm/spork-rails.git', branch: 'rails-4'
gem 'spork', '~> 1.0.0rc3'
end
group :test do
gem 'capybara'
gem 'factory_girl_rails'
gem "shoulda-matchers"
end
group :assets do
gem 'sass-rails', '~> 4.0.0.beta1'
gem 'coffee-rails', '~> 4.0.0.beta1'
gem 'uglifier', '>= 1.0.3'
end
gem 'jquery-rails'
gem 'turbolinks'
gem 'jbuilder', '~> 1.0.1'
The push finally worked without any issues, but when I tried to migrate the database, I got the following:
c:\Sites\elms>heroku run rake db:migrate
Running `rake db:migrate` attached to terminal... up, run.2205
/usr/bin/env: ruby.exe: No such file or directory
I have also tried: heroku run bin/rake db:migrate, and it gives same error. I am using Getting Started with Rails 4.x on Heroku to push my Rails 4 app to Heroku. Any help would be greatly appreciated!
Oh, I also had an issue after running:
c:\Sites\elmx>gem install rails --pre
I solved this by adding C:\RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\bin; to my environment Path variable in Windows 7 (it was giving me an error that it couldn't find rails when trying to execute anything such as:
rails -v
rails g controller User
But if I typed just:
rails
It would give me all of the commands I could use (as if running rails -h)
I set up my machine by using the Windows Rails Installer, then installing the Rails 4 gem. This is extremely frustrating and I've scoured the internet trying to find someone with the same problem.
EDIT 1:
I just deleted my user Path environment variable and modified my system Path environment variable to look like such:
C:\RailsInstaller\Ruby1.9.3\bin;C:\RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\bin;C:\Program Files (x86)\git\cmd;C:\Program Files (x86)\Heroku\bin
Updated my path variable to the above and uninstalled Heroku Toolbelt, then re-installed it (hence why the Heroku piece is in the path variable) and tried the following:
c:\Sites\elms>ruby -v
ruby 1.9.3p125 (2012-02-16) [i386-mingw32]
c:\Sites\elms>rails -v
Rails 4.0.0.beta1
c:\Sites\elms>heroku -v
! `-v` is not a heroku command.
! Perhaps you meant `-h`.
! See `heroku help` for a list of available commands.
c:\Sites\elms>heroku run rake db:migrate
Running `rake db:migrate` attached to terminal... up, run.2100
/usr/bin/env: ruby.exe: No such file or directory
c:\Sites\elms>heroku run bin/rake db:migrate
Running `bin/rake db:migrate` attached to terminal... up, run.2872
/usr/bin/env: ruby.exe: No such file or directory
EDIT 2:
So I got a little further in this venture. I found out that I could execute:
c:\Sites\elms>heroku run bash
~ $ cd bin
~/bin $ ruby rake db:migrate
And that would then run rake and migrate my database on Heroku. But the problem gets worse! I try to navigate to the page and it tells me that there is something wrong, so I run the log file and this is what I get:
c:\Sites\elms>heroku logs
2013-03-16T05:34:20+00:00 heroku[api]: Enable Logplex by
2013-03-16T05:34:20+00:00 heroku[api]: Release v2 created by
2013-03-16T05:34:36+00:00 heroku[slugc]: Slug compilation started
2013-03-16T05:35:37+00:00 heroku[web.1]: Starting process with command `bin/rails server -p 13423 -e $RAILS_ENV`
2013-03-16T05:35:37+00:00 heroku[slugc]: Slug compilation finished
2013-03-16T05:35:37+00:00 app[web.1]: bash: bin/rails: No such file or directory
2013-03-16T05:35:39+00:00 heroku[web.1]: Process exited with status 127
2013-03-16T05:35:39+00:00 heroku[web.1]: Starting process with command `bin/rails server -p 33539 -e $RAILS_ENV`
2013-03-16T05:35:40+00:00 app[web.1]: /usr/bin/env: ruby.exe: No such file or directory
2013-03-16T05:35:41+00:00 heroku[web.1]: Process exited with status 127
2013-03-16T05:35:41+00:00 heroku[web.1]: State changed from starting to crashed
2013-03-16T05:35:41+00:00 heroku[web.1]: State changed from crashed to starting
2013-03-16T05:35:44+00:00 heroku[web.1]: Starting process with command `bin/rails server -p 23452 -e $RAILS_ENV`
2013-03-16T05:35:44+00:00 app[web.1]: /usr/bin/env: ruby.exe: No such file or directory
This is getting really frustrating. I have no clue what the heck /usr/bin/env is. The only thing I can guess is that it's like PATH for Windows. If this is the case, why does the server not know where ruby.exe is?
I did make a non Rails 4 application and was able to post it to Heroku and view it with no issues. This is really getting on my nerves.
You need to change in your application first line of the 3 files:
bin/bundle
bin/rails
bin/rake
Instead
#!/usr/bin/env ruby.exe
must be
#!/usr/bin/env ruby
That's why is error:
/usr/bin/env: ruby.exe: No such file or directory
Then you do successfully:
heroku run rake db:migrate
It's work for me.
I had a similar issue and as others have already stated, changing the below helped.
#!/usr/bin/env ruby.exe to #!/usr/bin/env ruby
However, I also needed to remove some hidden carriage returns which were still present from developing on Windows originally (I guess?).
I used dos2unix to do this.
dos2unix bin/bundle bin/rake bin/rails
After that, I committed my changes, pushed to heroku and all was good.
I had this same problem, changed ruby.exe to ruby in the three files as described above, executed the
git push heroku master
step, which told me everything was up to date. Trying the
heroku run rake db:migrate
step again, it still didn't work; I got the same error about no such file or directory for ruby.exe. I examined my gemfile.lock as directed, but didn't see anything that looked windows-y.
I then executed these:
git add .
git commit -m "My comment"
and tried again with
git push heroku master
This time, boatloads of messages went by, including:
Removing Gemfile.lock because it was generated on Windows.
so perhaps there was something wrong with it and I just didn't know what I should be looking for in it.
Trying again to
heroku run rake db:migrate
looks to have worked, as I got
Running rake db:migrate attached to terminal... up, run.3260
== CreateUsers: migrating ====================================================
-- create_table(:users)
-> 0.0409s
== CreateUsers: migrated (0.0412s) ===========================================
My best guess is that the change to remove the .exe from bundle, rake, and rails did not actually get pushed up on the first try of
git push heroku master
and the
git add .
and
git commit -m "My comment"
were necessary to make it get pushed up. This is my first attempt at anything with git/heroku/ruby, so I don't know enough to say why the other two commands were required, but leave my experience documented here in case it helps anybody else.
Thanks so much for the clue above that it was the .exe in those three files that caused the problem.
bin/bundle
bin/rails
bin/rake
Instead
#!/usr/bin/env ruby.exe
must be
#!/usr/bin/env ruby
(thanks to previous post)
make sure there are no references to windows in the gemfile.lock - they can just be deleted
then
git push heroku master
if there are still problems use
heroku logs (copying text to a text editor makes it easier to cut and paste for solutions)
Try this:
sudo apt-get install build-essential patch ruby-dev zlib1g-dev liblzma-dev
and then install Nokogiri:
gem install nokogiri

rake assets:precompile failing during push to Heroku

I'm currently using asset_sync in my Rails app, and I have the environment variables set in my Heroku app. When I run heroku config I get:
AWS_ACCESS_KEY_ID: XXXXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY: XXXXXXXXXXXXXXXXXX
FOG_DIRECTORY: MY-BUCKET-NAME
FOG_PROVIDER: AWS
etc...
When I push my app to Heroku, it tries to run rake assets:precompile and I get the following message:
Preparing app for Rails asset pipeline
Running: rake assets:precompile
/usr/local/bin/ruby /tmp/build_2pa7aisux9av8/vendor/bundle/ruby/1.9.1/bin/rake assets:precompile:nondigest RAILS_ENV=production RAILS_GROUPS=assets
AssetSync: using /tmp/build_2pa7aisux9av8/config/initializers/asset_sync.rb
rake aborted!
Fog directory can't be blank, Aws access key can't be blank, Aws secret access key can't be blank
But then I run:
heroku run rake assets:precompile --app my-app-name
...and it processes everything and syncs to S3 just fine:
Running `rake assets:precompile` attached to terminal... up, run.1
/usr/local/bin/ruby /app/vendor/bundle/ruby/1.9.1/bin/rake assets:precompile:all RAILS_ENV=staging RAILS_GROUPS=assets
AssetSync: using /app/config/initializers/asset_sync.rb
/usr/local/bin/ruby /app/vendor/bundle/ruby/1.9.1/bin/rake assets:precompile:nondigest RAILS_ENV=staging RAILS_GROUPS=assets
AssetSync: using /app/config/initializers/asset_sync.rb
AssetSync: Syncing.
Using: Directory Search of /app/public/assets
Uploading: assets/application-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.css
Uploading: assets/application-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.css.gz
Uploading: assets/application-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.css.gz
Uploading: assets/application-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.css
AssetSync: Done.
Any ideas why it wouldn't work during the push but it would work fine when I heroku run rake assets:precompile?
I had the same problem on one of our servers, until I found the documentation on the asset_sync github page that says you need to run
heroku labs:enable user-env-compile --app <appname>
in order for it to work.
Heroku also has AssetSync documentation
It's so nice to only be compiling assets once now
I can see you're running the assets:precompile rake task with --app my-app-name option. Just to be sure, do you have multiple apps on Heroku? (eg. staging and production). If you do make sure make sure running heroku config --app my-app-name results in the output you had with heroku config.
If you had the expected results with the above command, it's likely the ENV vars aren't available on git push as suggested here asset_sync_test github readme . You can go around that by using the following in your config/environments/*.rb file:
config.asset_sync.aws_access_key = ENV['AWS_ACCESS_KEY_ID']
config.asset_sync.aws_access_secret = ENV['AWS_SECRET_ACCESS_KEY']
config.asset_sync.aws_bucket = ENV['FOG_DIRECTORY']
config.asset_sync.fog_provider = ENV['FOG_PROVIDER']

Resources