I've got a Ruby application that has been deployed to an AWS instance using CodeDeploy via CircleCI. I added a gem to the application. All tests are passing both locally and in CircleCI, but the deployment stage fails with this error:
$ create_application_revision /tmp/codedeploy_applications.json /tmp/codedeploy_revisions.json
create_application_revision loaded: {"applications":[{"application_root":"/","region":"us-west-2","revision_location":{"revisionType":"S3","s3Location":{"bucket":"something","key":"etl-scripts-testdeploy-3a050b1"}},"deployment_group":"EtlScriptsFleet","application_name":"EtlScripts"}]}
Bundling EtlScripts from /home/ubuntu/etl-scripts
Unhandled exception
[Errno 2] No such file or directory: '/home/ubuntu/etl-scripts/vendor/bundle/ruby/2.2.0/gems/regexp-examples-1.1.4/db/unicode_ranges_2.1.pstore'
It's true that file doesn't exist. It doesn't exist locally either, but all tests pass. I just can't figure out why the deployment process thinks it needs this file?
I noticed that in the source code for this gem (https://github.com/tom-lord/regexp-examples/tree/master/db) is a symbolic link and that's probably why it's not being created. But I'm still confused about how the deployment bundle process would know about a non-existant symbolic link.
Any help greatly appreciated!
It seems the deployment stage is being handled by the deploy push command from aws CLI, and it cannot handle broken symlinks.
I could reproduce the problem by trying to deploy an app containing a broken symlink using the push command:
(Command - aws deploy push --application-name --s3-location s3:///.zip --source ./)
(Output - [Errno 2] No such file or directory: '/path/to/source/')
I guess you will need to manually remove all broken symlinks for the deployment to succeed.
For what it's worth, this gem was later refactored to not use symbolic links; as of v1.3.0.
Upgrading from your current version (1.1.4) should almost certainly require no code change.
Related
I have created Pipeline in GitLabs and I am using docker as gitlab-runner. I want to push Jekyll website on s3 website. And to do so, I am using s3_website gem. I have 4 stages defined in my pipeline. Where I am building Jekyll, creating Artifacts using Gulp, executing test on my jekyll site and then deploying.
All steps are working fine but while doing deployment, I'm getting following error. And i could not figure it how to get this solve.
[fail] Could not load the site: Failed to parse ERB in /builds/myproject/s3_website.yml:
(SyntaxError) /usr/local/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_warn.rb:15: syntax error, unexpected tLABEL
module_function define_method(:warn) {|*messages, uplevel: nil|
It is working perfectly fine on my local machine when I'm not on Docker. But when I try to do the same thing using docker. It is giving me above error.
I tried it with ruby 2.3, 2.4, 2.5, 2.6 on my docker machine. However no luck.
bundle exec s3_website push
I am expecting this to deploy the site on S3 bucket and corresponding cloudfront.
Any clue would be appreciated.
We ran into this same error on CircleCI. If I understand correctly, the s3_website gem wraps a Java .jar that's using JRuby 1.7, and something must have changed in one of the Docker images or Ruby gems that causes it to start inheriting the system's Ruby 2+ path. As a result, its JRuby 1.7 tries to load Ruby gems that only work in Ruby 2.0 and above, so it runs into errors.
As a workaround, instead of letting the s3_website gem invoke the .jar file itself, I tell the s3_website gem to only download the .jar file, then I manually invoke it:
bundle exec s3_website install
java -cp $(bundle show s3_website)/*.jar s3.website.Push
I reported this on the s3_website project's GitHub page.
Same error message but different solution.
The root of my problem was that the S3_ACCESS_KEY_ID wasn't set properly. I source'd my .env file
source .env
And that loaded the access key variable and the deploy worked.
Using docker-compose, I'm trying to build a Ruby application (ruby-app). However near the end of the build it errors out:
There was an error while trying to write to `/ruby-app/Gemfile.lock`. It is likely
that you need to grant write permissions for that path.
ERROR: Service 'ruby-app' failed to build: The command '/bin/sh -c bundle install --jobs 4' returned a non-zero code: 23
Not sure why this permissions error is cropping up now, it has been building fine for some time.
The problem was that I modified the Gemfile and I was expecting bundler inside of the container to generate the new Gemfile.lock based on the changes.
To fix the issue, I ran bundle install outside of the container. This generated a new Gemfile.lock on my local filesystem and then I was able to build the container as expected.
I manged to get my spring boot website online on Heroku. But I also use wkhtmltopdf to create a pdf. This works locally but now I have some problems.
Offline it works as follow :
ProcessBuilder pb = new ProcessBuilder
("cmd.exe",
"/c",
" cd C:\\Program Files\\wkhtmltopdf\\bin && wkhtmltopdf.exe "
+ "http://google.com C:\\MainWebApps\\TestApp\\src\\main\\resources\\userstorage\\Google2.pdf");
But how do I install this on Heroku?
Where do I store the temporarily html page so I can create a pdf from it ?
And where is wkhtmltopdf installed on Heroku ?
Can I call the wkhtmltopdf with a processbuilder on heroku?
EDIT
So after the comment of ceejayoz I googled a bit more and did find some interesting stuff.
So for Compile the binaries on Heroku I used this:
heroku run /bin/bash
Then I did a curl of wkhtmltopdf like this:
curl -O http://download.gna.org/wkhtmltopdf/0.12/0.12.0/wkhtmltox-linux-amd64_0.12.0-03c001d.tar.xz
Then I tried to extract it on the server but without success:
$ tar -xjvf wkhtmltox-linux-amd64_0.12.0-03c001d.tar.xz
tar (child): wkhtmltox-linux-amd64_0.12.0-03c001d.tar.xz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
EDIT2
I also found this https://github.com/dscout/wkhtmltopdf-buildpack on github.
So I did following :
heroku buildpacks:set 'https://github.com/heroku/heroku-buildpack-multi.git'
echo 'https://github.com/dscout/wkhtmltopdf-buildpack.git' >> .buildpacks
This created a file named .buildpacks but how do I proceed from there on ?
I also found this post but vulcan is deprecated and uses ruby
Using Wkhtmltopdf with Nodejs on Heroku
Can somebody provide me with good information because I am completely stuck with this?
You actually have two problems that you need to solve -
How to install/invoke the executable
How to handle the generated .pdf
Assuming you have the basics of Heroku deployment (push to the Heroku git remote), for #1, #ceejayoz is right - check the binary into your git repository. For example, under a ./bin directory. The root of your project (where the Procfile is) will be your working directory, and you should be able to invoke the program with ProcessBuilder using relative paths.
Caveat - since it looks like you are developing on Windows, you will need to pay attention to ensuring both platform-specific binaries are available, and add some logic to know which one to invoke (for example, by setting/checking a specific environment variable).
I recommend against trying to build with a custom build pack - you will spend a lot of energy for little to no benefit. Aside from the platform issue, you don't need to rebuild a third party tool when your code changes...
The second problem is that you can't leave the generated PDF in place. It will go away when the dyno is restarted (see https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem). Thus, the first thing you should do when the process completes is move the generated file to an external storage service (Amazon S3 is a good starting point).
Hope this helps.
You might want to use wkhtmltopdf-binary. With that solution, you do not need to put wkhtmltopdf executable into your VCS. You can use it for example with Maven or Gradle.
In my Gemfile I have
gem 'slim', :git => 'git://github.com/brennancheung/slim.git', :branch => 'angularjs_support'
which is a branch of the slim gem required for me to run AngularJS correctly with my views. I've pushed my code to my beanstalk application but am unable to bundle install according to the logs shown below...
sh: git: command not found
Git error: command `git clone 'git://github.com/brennancheung/slim.git'
"/usr/share/ruby/1.9/gems/1.9.1/cache/bundler/git/slim-700ed452e752ccb6baf9de9d0a46fbded8bb2da5"
--bare --no-hardlinks` in directory /var/app/ondeck has failed.
I'm new to Beanstalk and have no idea how to fix this. Any help on how to get bundle to install successfully would be greatly appreciated. Thanks.
Since git is not installed on by default on EC2 instance, you would have to find a workaround solution:
a. Install git on instance with configuration file and command.
It is the most obvious way to solve the problem, although not be the most efficient.
b. Clone slim repository into your project, so it will be deployed together.
Seems that slim is not being actively developed lately, so having the copy in your project might not be a bad idea. It protects you from github.com being down, yet you will have extra files to carry around.
c. Use configuration file and commands to pull the data from github.com directly with http.
Too many files to work with, and also dependency on third party service.
d. Use a combination of above. Clone slim repository and copy files to S3. Use configuration and commands to copy files from S3 to your instance.
It seems like the most elegant and efficient way to solve the problem.
It might look something like:
$ cat .ebextensions/myapp.config
commands:
10-copy-slim-from-s3
command: "aws s3 cp s3://mybucket/slim slim --recursive"
I'm trying to put together a custom buildpack with NodeJS and the CouchBase module/libraries
I've gotten as far as using Vulcan to build libcouchbase and libvbucket and getting the buildpack to retrieve and unpack the tgz files for both.
Everything looks ok there, but I receive errors when npm tries to install the couchbase module:
I get a bunch of errors, but this line:
"../src/couchbase_impl.h:52:36: warning: libcouchbase/couchbase.h: No such file or directory"
leads me to think that it can't find the libcouchbase libraries (which is possible since they aren't in the usual place).
I've tried to add the correct path using CPPFLAGS="-I/app/vendor/couchbase/include/libcouchbase" in both the Config Vars and just exporting that as part of the compile phase, but still no luck.
Here is the gist with the Heroku deploy output and the compile/release buildpack files:
https://gist.github.com/ahamidi/5620503
Any help would be greatly appreciated.
Thanks,
Ali
[Update 1]
I've made some progress and I can now get the slug to compile when deploying to Heroku.
The key was figuring out the ENV Variables that CouchNode looks for when adding custom directories to include.
In this case, the Env Variables were EXTRA_CPPFLAGS and EXTRA_LDFLAGS.
So I updated the compile file to include the following:
export EXTRA_CPPFLAGS="-I$BUILD_DIR/vendor/couchbase/include"
export EXTRA_LDFLAGS="-L$BUILD_DIR/vendor/couchbase/lib -Wl,-rpath,$BUILD_DIR/vendor/couchbase/lib"
The slug compiles and the app is deployed, but I now get a different error in the logs:
Error: libcouchbase.so.2: cannot open shared object file: No such file or directory
So it looks like Node can't see the libcouchbase libraries directory.
For anyone who is curious or experiencing a similar issue, here's what worked for me.
In order to get the couchbase npm module to install I had to tell it where to find the libcouchbase libraries (in compile file):
export EXTRA_CPPFLAGS="-I$BUILD_DIR/vendor/couchbase/include"
export EXTRA_LDFLAGS="-L$BUILD_DIR/vendor/couchbase/lib -Wl,-rpath,$BUILD_DIR/vendor/couchbase/lib"
Then in order to require couchbase in my app I had to set the following Env Variable:
LD_LIBRARY_PATH="/app/vendor/couchbase/lib:$LD_LIBRARY_PATH"
With the command:
heroku config:add LD_LIBRARY_PATH="/app/vendor/couchbase/lib:$LD_LIBRARY_PATH"
You should set CPPFLAGS="-I/app/vendor/couchbase/include" LDFLAGS="-L/app/vendor/couchbase/include -lcouchbase"
from your script it seems like you just unpacking libcouchbase without any further work. you should also build it and install. typical magic spell for node.js client will be ./configure --disable-plugins --disable-examples && make && sudo make install. I'm not sure if sudo part needed on heroku, probably just make install