I created a Lambda function on AWS and selected the runtime: Ruby version 2.7.0.
Then I deployed my Ruby function with .zip file archives to this AWS Lambda function using following commands:
bundle config set --local path 'vendor/bundle'
bundle install
zip -r my_function.zip lambda_function.rb vendor
aws lambda update-function-code --function-name test-function --zip-file fileb://my_function.zip
Deploying was successful, however all the libraries are added to vendor/bundle/ruby/2.4.0/..., but not vendor/bundle/ruby/2.7.0/...
adding: vendor/bundle/ruby/2.4.0/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/build_info/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/doc/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/gems/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/CHANGELOG.md (deflated 50%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/LICENSE.txt (deflated 65%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/lib/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/lib/aws-eventstream.rb (deflated 63%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/lib/aws-eventstream/ (stored 0%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/lib/aws-eventstream/types.rb (deflated 62%)
adding: vendor/bundle/ruby/2.4.0/gems/aws-eventstream-1.2.0/lib/aws-eventstream/header_value.rb (deflated 57%)
...
I could not find what causes the problem. How can I deploy the lambda with adding the libraries to vendor/bundle/ruby/2.7.0/...?
The GEMFILE.lock file is:
GEM
remote: https://rubygems.org/
specs:
aws-eventstream (1.2.0)
aws-partitions (1.547.0)
aws-sdk-core (3.125.1)
aws-eventstream (~> 1, >= 1.0.2)
aws-partitions (~> 1, >= 1.525.0)
aws-sigv4 (~> 1.1)
jmespath (~> 1.0)
aws-sdk-sqs (1.49.0)
aws-sdk-core (~> 3, >= 3.125.0)
aws-sigv4 (~> 1.1)
aws-sigv4 (1.4.0)
aws-eventstream (~> 1, >= 1.0.2)
bindata (2.4.10)
colorize (0.8.1)
jmespath (1.4.0)
json (2.6.1)
redis (4.5.1)
PLATFORMS
x86_64-darwin-20
DEPENDENCIES
aws-sdk-sqs
bindata
colorize
json
redis
BUNDLED WITH
2.3.4
You're using Ruby 2.4.0 locally to bundle your Ruby application which makes Bundler (correctly) believe that you're targeting 2.4.0, which then results in the vendor/bundle/ruby/2.4.0/ folder being populated.
You should instead install and use Ruby 2.7.0 to sync your local Ruby version to the Lambda runtime you are targeting.
After verifying with ruby -v that you are indeed on 2.7.0, reinstall Bundler (gem install bundler) and then rerun the above commands.
You should then have the vendor/bundle/ruby/2.7.0/ folder populated instead after running bundle install, with the output of zip -r also showing that.
Related
Context
Deployments to Cloud Functions have been failing since a few days ago.
Gemfile
source "https://rubygems.org"
ruby "~> 2.7.0"
git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
gem "google-cloud-firestore"
Deployment script
$ gcloud functions deploy my_func --region=us-central1 --memory=128MB --runtime=ruby27
(snip)
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Bundler found conflicting requirements for the Ruby version:
In Gemfile:
Ruby (~> 2.7.0)
google-cloud-firestore was resolved to 2.4.1, which depends on
google-cloud-firestore-v1 (~> 0.0) was resolved to 0.4.0, which depends on
gapic-common (~> 0.3) was resolved to 0.4.0, which depends on
google-protobuf (~> 3.15, >= 3.15.2) was resolved to 3.15.5, which
depends on
Ruby (< 3.1.dev, >= 2.3)
google-cloud-firestore was resolved to 2.4.1, which depends on
google-cloud-firestore-v1 (~> 0.0) was resolved to 0.4.0, which depends on
gapic-common (~> 0.3) was resolved to 0.4.0, which depends on
grpc (~> 1.36) was resolved to 1.36.0, which depends on
Ruby (< 3.1.dev, >= 2.4); Error ID: af32a539
Why?
This is bundler's regression since bundler v2.2.8.
https://github.com/rubygems/rubygems/issues/4366
And this is fixed at bundler v2.2.10
https://github.com/rubygems/rubygems/blob/master/bundler/CHANGELOG.md#2210-february-15-2021
https://github.com/rubygems/rubygems/pull/4371
So I want the bundler version to be less than v2.2.8 or more than v2.2.10.
Cloud Functions deployment ( gcloud functions deploy ) automatically runs bundle install and uses bundler which installed in deployment task.
Current bundler version in deployment task is v2.2.9
Verification code
Gemfile
source "https://rubygems.org"
ruby "~> 2.7.0"
git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
gem "bundler", "!= 2.2.8", "!= 2.2.9"
Deployment script
$ gcloud functions deploy my_func --region=us-central1 --memory=128MB --runtime=ruby27
(snip)
Resolving dependencies...
Bundler could not find compatible versions for gem "bundler":
In Gemfile:
bundler (!= 2.2.8, != 2.2.9)
Current Bundler version:
bundler (2.2.9)
Main subject
gcloud functions deploy doesn't have some arguments to change bundler version...
https://cloud.google.com/sdk/gcloud/reference/functions/deploy?hl=ja
How to change the version of bundler used in Cloud Functions deployment?
I ran into a similar issue as well, however the error output was slightly different. One thing in common was the line:
Bundler found conflicting requirements for the Ruby version:
After doing some digging and troubleshooting, I found that Google Cloud Functions locks the bundled platform on our behalf. I'm not a ruby expert, but I suspect this will impact how information from the Gemfile will be read along with the Ruby version.
Example Log from Google Cloud Functions:
2021-03-10 13:13:01.764 PST
Step #4 - "builder": Done "bundle lock --add-platform x86_64-linux"
Solution:
Update bundler locally (I updated to 2.2.14). Delete your Gemfile.lock file and rerun bundle install to generate Gemfile.lock. This should fix dependencies if there is anything to fix.
Most importantly, it will update the Platforms section in your Gemfile.lock to match what Google expects.
In my case, the platform was updated from Ruby to x86_64-darwin-19.
Or the other approach may be to just update the platform with bundle if possible.
Eventually I have successfully deployed with followings.
gem install bundler --no-doc
Delete Gemfile.lock
re-create Gemfile.lock ( bundle install )
diff
$ git --no-pager diff main
diff --git a/Gemfile.lock b/Gemfile.lock
index 5e7502b..b5642ab 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
## -57,7 +57,6 ## GEM
gapic-common (~> 0.3)
google-cloud-errors (~> 1.0)
grpc-google-iam-v1 (>= 0.6.10, < 2.0)
- google-protobuf (3.15.5)
google-protobuf (3.15.5-universal-darwin)
googleapis-common-protos (1.3.11)
google-protobuf (~> 3.14)
## -72,9 +71,6 ## GEM
multi_json (~> 1.11)
os (>= 0.9, < 2.0)
signet (~> 0.14)
- grpc (1.36.0)
- google-protobuf (~> 3.14)
- googleapis-common-protos-types (~> 1.0)
grpc (1.36.0-universal-darwin)
google-protobuf (~> 3.14)
googleapis-common-protos-types (~> 1.0)
## -160,7 +156,6 ## GEM
PLATFORMS
ruby
- x86_64-darwin-17
DEPENDENCIES
dotenv
## -180,4 +175,4 ## RUBY VERSION
ruby 2.7.2p137
BUNDLED WITH
- 2.1.4
+ 2.2.14
when I execute this command to compile my project:
sudo bundle exec fastlane beta
shows this error:
Could not find rake-12.3.3 in any of the sources
Run `bundle install` to install missing gems.
(base)
and I install rake:
$ sudo gem install rake
Password:
Successfully installed rake-13.0.3
Parsing documentation for rake-13.0.3
Done installing documentation for rake after 0 seconds
1 gem installed
(base)
but still not fix the problem. what should I do to make it work? I already execute bundle install, still not fix it:
~/Documents/GitHub/cruise-open/ios on main! ⌚ 17:49:28
$ sudo bundle install
Password:
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.
Using rake 12.3.3
Using CFPropertyList 3.0.3
Using public_suffix 4.0.6
Using addressable 2.7.0
Using atomos 0.1.3
Using aws-eventstream 1.1.0
Using aws-partitions 1.402.0
Using aws-sigv4 1.2.2
Using jmespath 1.4.0
Using aws-sdk-core 3.110.0
Using aws-sdk-kms 1.39.0
Using aws-sdk-s3 1.86.0
Using babosa 1.0.4
Using bundler 2.2.9
Using claide 1.0.3
Using colored 1.2
Using colored2 3.1.2
Using highline 1.7.10
Using commander-fastlane 4.4.6
Using declarative 0.0.20
Using declarative-option 0.1.0
Using digest-crc 0.6.2
Using unf_ext 0.0.7.7
Using unf 0.1.4
Using domain_name 0.5.20190701
Using dotenv 2.7.6
Using emoji_regex 3.2.1
Using excon 0.78.1
Using multipart-post 2.0.0
Using ruby2_keywords 0.0.2
Using faraday 1.1.0
Using http-cookie 1.0.3
Using faraday-cookie_jar 0.0.7
Using faraday_middleware 1.0.0
Using fastimage 2.2.0
Using gh_inspector 1.1.3
Using jwt 2.2.2
Using memoist 0.16.2
Using multi_json 1.15.0
Using os 1.1.1
Using signet 0.14.0
Using googleauth 0.14.0
Using httpclient 2.8.3
Using mini_mime 1.0.2
Using uber 0.1.0
Using representable 3.0.4
Using retriable 3.1.2
Using google-api-client 0.38.0
Using google-cloud-env 1.4.0
Using google-cloud-errors 1.0.1
Using google-cloud-core 1.5.0
Using google-cloud-storage 1.29.1
Using json 2.3.1
Using mini_magick 4.11.0
Using plist 3.5.0
Using rubyzip 2.3.0
Using security 0.1.3
Using naturally 2.2.0
Using simctl 1.6.8
Using slack-notifier 2.3.2
Using terminal-notifier 2.0.0
Using unicode-display_width 1.7.0
Using terminal-table 1.8.0
Using tty-screen 0.8.1
Using tty-cursor 0.7.1
Using tty-spinner 0.9.3
Using word_wrap 1.0.0
Using nanaimo 0.3.0
Using xcodeproj 1.19.0
Using rouge 2.0.7
Using xcpretty 0.3.0
Using xcpretty-travis-formatter 1.0.0
Using fastlane 2.170.0
Using fastlane-plugin-pgyer 0.2.2
Using xcode-install 2.6.8
Bundle complete! 3 Gemfile dependencies, 75 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
(base)
this is my Gemfile:
source "https://rubygems.org"
gem "xcode-install"
gem "fastlane"
plugins_path = File.join(File.dirname(__FILE__), 'fastlane', 'Pluginfile')
eval_gemfile(plugins_path) if File.exist?(plugins_path)
I want to enable pagination on my Jekyll site, but this happens:
jekyll serve
WARN: Unresolved specs during Gem::Specification.reset:
eventmachine (>= 0.12.9)
listen (~> 3.0)
rouge (< 4, >= 1.7)
rb-fsevent (>= 0.9.4, ~> 0.9)
rb-inotify (>= 0.9.7, ~> 0.9)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
Configuration file: /Users/sean/dev/sh78.github.io/_config.yml
Source: /Users/sean/dev/sh78.github.io
Destination: /Users/sean/dev/sh78.github.io/_site
Incremental build: disabled. Enable with --incremental
Generating...
Liquid Exception: Liquid syntax error (line 1): Unknown tag 'paginator' in log/index.html
jekyll 3.8.3 | Error: Liquid syntax error (line 1): Unknown tag 'paginator'
I think I followed the docs to a tee:
In _config.yml:
plugins:
- jekyll-feed
- jekyll-seo-tag
- jekyll-paginate
paginate: 2
paginate_path: "/log/page:num/"
The index.html file is in place as defined in paginate_path
ls
404.html README.md _posts/ favicon.ico
Gemfile _config.yml _sass/ favicon.png
Gemfile.lock _includes/ _site/ index.md
LICENSE.txt _layouts/ assets/ log/
ls log/
index.html
Contents of log/index.html, where I'm trying to at least print the paginator.posts object. There is no permalink in the front matter per the docs:
---
layout: log
title: log
icon: pencil-alt
description: >
Sean's blog. Articles about software and other things.
---
{% paginator.posts %}
Here is the parent log (blog) template:
---
layout: default
---
<div class="blog">
{{ content }}
{%- if site.posts.size > 0 -%}
{% comment %} *truncated* (it works fine) {% endcomment %}
I have jekyll-paginate in my Gemfile:
# If you have any plugins, put them here!
group :jekyll_plugins do
gem "jekyll-feed", "~> 0.6"
gem "jekyll-paginate"
end
And ran bundle install successfully:
bundle install
Using public_suffix 3.0.2
Using addressable 2.5.2
Using bundler 1.16.2
Using colorator 1.1.0
Using concurrent-ruby 1.0.5
Using eventmachine 1.2.7
Using http_parser.rb 0.6.0
Using em-websocket 0.5.1
Using ffi 1.9.25
Using forwardable-extended 2.6.0
Using i18n 0.9.5
Using rb-fsevent 0.10.3
Using rb-inotify 0.9.10
Using sass-listen 4.0.0
Using sass 3.5.6
Using jekyll-sass-converter 1.5.2
Using ruby_dep 1.5.0
Using listen 3.1.5
Using jekyll-watch 2.0.0
Using kramdown 1.17.0
Using liquid 4.0.0
Using mercenary 0.3.6
Using pathutil 0.16.1
Using rouge 3.1.1
Using safe_yaml 1.0.4
Using jekyll 3.8.3
Using jekyll-feed 0.9.3
Using jekyll-paginate 1.1.0
Using jekyll-seo-tag 2.5.0
Using minimaterialize 1.1.2
Bundle complete! 5 Gemfile dependencies, 30 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
I uninstalled all versions of jekyll except the one in use, like this answer.
I also tried like gem cleanup && bundle exec jekyll serve like in this answer, and got the same exception listed above.
Environment:
jekyll -v
WARN: Unresolved specs during Gem::Specification.reset:
eventmachine (>= 0.12.9)
listen (~> 3.0)
rouge (< 4, >= 1.7)
rb-fsevent (>= 0.9.4, ~> 0.9)
rb-inotify (>= 0.9.7, ~> 0.9)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
jekyll 3.8.3
ruby -v
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin15]
gem -v
2.5.1
What am I missing?
Well, yeah, I made an oops. I wrote paginator as a liquid tag, aka command (like if) instead of just calling a variable/object.
Changing {% paginator %} to {{ paginator }} is the solution.
When I add a gem to a project's Gemfile for the first time, but I have installed the gem previously while working on another project, it uses the existing version of the gem, rather than the latest version of the gem available.
For example, using bundler version 1.11.2, I added gem 'rubocop' to a project's Gemfile, and running bundle install resulted in it using RuboCop version 0.42.0, rather than the current (as of 21 October 2016) version of 0.44.1:
rubocop (0.42.0)
parser (>= 2.3.1.1, < 3.0)
powerpack (~> 0.1)
rainbow (>= 1.99.1, < 3.0)
ruby-progressbar (~> 1.7)
unicode-display_width (~> 1.0, >= 1.0.1)
Running bundle update rubocop resulted in version 0.44.1 being used, without any pre-existing gems having their version changed. This indicates there weren't any constraints shopping me from using RuboCop version 0.44.1.
rubocop (0.44.1)
parser (>= 2.3.1.1, < 3.0)
powerpack (~> 0.1)
rainbow (>= 1.99.1, < 3.0)
ruby-progressbar (~> 1.7)
unicode-display_width (~> 1.0, >= 1.0.1)
When running bundle install, how can I tell it, for gems that aren't mentioned in Gemfile.lock, to download the newest compatible version of the gem, rather than use an older version which happens to be available on the local machine?
I tried looking at http://bundler.io/v1.13/man/bundle-install.1.html but nothing there seemed to be relevant.
If you want to have newest version gem, remove version in Gemfile and run bundle install again.
I am trying to install a Gem named spiceweasel (https://github.com/mattray/spiceweasel). When running it I get the following error:
$ sudo gem install spiceweasel
ERROR: While executing gem ... (Gem::DependencyError)
Unable to resolve dependencies: ridley requires buff-extensions (~> 0.3); buff-config requires buff-extensions (~> 0.3); varia_model requires buff-extensions (~> 1.0)
Is there a way to circumvent this?
Looks like you have a conflict in your dependencies:
ridley requires buff-extensions (~> 0.3)
varia_model requires buff-extensions (~> 1.0)
Ie: two gems depends on conflicting versions of a third one ("~>" in Bundler means that another minor version is correct, but not a major one).
What you can do:
If your project does not require both ridley & varia_model, you may use RVM to create a specific Gemset for it.
Update ridley (the newest version is also using buff-extension 1.0).
Install the buff-extensions gem. It's currently at version 1.0, which should satisfy all of the other items that have dependencies on it.