I have the following script:
require 'httparty'
require 'aws-sdk-s3'
include HTTParty
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new('...', '...')
})
client = Aws::S3::Resource.new
movies = self.class.get("#{HOST}/movies")
When I run it I'm met with the following error:
Traceback (most recent call last):
10: from script.rb:33:in `<main>'
9: from script.rb:33:in `new'
8: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-athena-1.22.0/lib/aws-sdk-athena/resource.rb:14:in `initialize'
7: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-core-3.78.0/lib/seahorse/client/base.rb:99:in `new'
6: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-athena-1.22.0/lib/aws-sdk-athena/client.rb:262:in `initialize'
5: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-core-3.78.0/lib/seahorse/client/base.rb:19:in `initialize'
4: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-core-3.78.0/lib/seahorse/client/base.rb:62:in `build_config'
3: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-core-3.78.0/lib/seahorse/client/configuration.rb:149:in `build!'
2: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-core-3.78.0/lib/seahorse/client/configuration.rb:158:in `empty_struct'
1: from /Users/me/.rvm/gems/ruby-2.6.5/gems/aws-sdk-core-3.78.0/lib/seahorse/client/configuration.rb:158:in `new'
/Users/me/.rvm/gems/ruby-2.6.5/gems/httparty-0.17.1/lib/httparty/module_inheritable_attributes.rb:42:in `inherited': undefined method `each' for nil:NilClass (NoMethodError)
After going through AWS's documentation and tutorials I still can't find the issue. What could be wrong?
I found the issue to be the line include HTTParty. I declared the S3 client before that line and it works now:
require 'httparty'
require 'aws-sdk-s3'
Aws.config.update({
region: 'us-east-1',
credentials: Aws::Credentials.new('...', '...')
})
client = Aws::S3::Resource.new
include HTTParty
movies = self.class.get("#{HOST}/movies")
Related
ruby .\bot.rb
#: /stop
Traceback (most recent call last):
8: from ./bot.rb:7:in `<main>'
7: from C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/telegram_bot-0.0.8/lib/telegram_bot/bot.rb:32:in `get_updates'
6: from C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/telegram_bot-0.0.8/lib/telegram_bot/bot.rb:32:in `loop'
5: from C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/telegram_bot-0.0.8/lib/telegram_bot/bot.rb:34:in `block in get_updates'
4: from C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/telegram_bot-0.0.8/lib/telegram_bot/bot.rb:34:in `each'
3: from C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/telegram_bot-0.0.8/lib/telegram_bot/bot.rb:37:in `block (2 levels) in get_updates'
2: from ./bot.rb:11:in `block in <main>'
1: from C:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/telegram_bot-0.0.8/lib/telegram_bot/message.rb:16:in `reply'
./bot.rb:19:in `block (2 levels) in <main>': undefined local variable or method `commands' for main:Object (NameError)
Did you mean? command``
the whole code:
require 'telegram_bot'
token ="my bot token"
bot = TelegramBot.new(token: token)
bot.get_updates(fail_silently: true) do |message|
puts "##{message.from.username}: #{message.text}"
command = message.get_command_for(bot)
message.reply do |reply|
case command
when /start/i
reply.text = "All I can do is say hello. Try the /greet command."
when /greet/i
greetings = ['bonjour', 'hola', 'hallo', 'sveiki', 'namaste', 'salaam', 'szia', 'halo', 'ciao']
reply.text = "#{greetings.sample.capitalize}, #{message.from.first_name}!"
else
reply.text = "I have no idea what #{commands.inspect} means."
end
puts "sending #{reply.text.inspect} to ##{message.from.username}"
reply.send_with(bot)
end
end
I've got sample app for hijack proxy server
io_lambda = lambda{ |io|
3.times do |i|
puts i
io.write "David\r\n"
end
io.close
}
run lambda{ |req|
[
200,
[ [ "rack.hijack", io_lambda ] ],
[""]
]
}
and starting it
rackup config.ru -p 3000
Now I trying to code client for it. First, just curl it:
curl http://localhost:3000 -vv
and get following client output:
* Rebuilt URL to: http://localhost:3000/
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3000 (#0)
> GET / HTTP/1.1
> Host: localhost:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
<
David
David
* transfer closed with outstanding read data remaining
* stopped the pause stream!
* Closing connection 0
curl: (18) transfer closed with outstanding read data remaining
now, with httparty:
require 'rubygems'
require 'bundler/setup'
Bundler.require(:default)
response = HTTParty.get('http://localhost:3000')
puts response.body, response.code, response.message, response.headers.inspect
I get:
→ ruby httparty-client.rb
Traceback (most recent call last):
19: from httparty-client.rb:5:in `<main>'
18: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/httparty-0.17.1/lib/httparty.rb:627:in `get'
17: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/httparty-0.17.1/lib/httparty.rb:508:in `get'
16: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/httparty-0.17.1/lib/httparty.rb:594:in `perform_request'
15: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/httparty-0.17.1/lib/httparty/request.rb:145:in `perform'
14: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1470:in `request'
13: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:920:in `start'
12: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1472:in `block in request'
11: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1479:in `request'
10: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1517:in `transport_request'
9: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:166:in `reading_body'
8: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:229:in `body'
7: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:204:in `read_body'
6: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:283:in `read_body_0'
5: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:278:in `inflater'
4: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:285:in `block in read_body_0'
3: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:324:in `read_chunked'
2: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:324:in `ensure in read_chunked'
1: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/protocol.rb:159:in `read'
/Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/protocol.rb:225:in `rbuf_fill': end of file reached (EOFError)
last try with net/http:
require 'net/http'
streamURL = 'http://localhost:3000'
uri = URI.parse(streamURL)
Net::HTTP.start(uri.host, uri.port) do |http|
request = Net::HTTP::Get.new uri.request_uri
http.request request do |response|
response.read_body do |chunk|
#We get the data here chunk-by-chunk
puts chunk
end
end
end
I get:
→ ruby net-http-client.rb
David
David
David
David
Traceback (most recent call last):
16: from net-http-client.rb:7:in `<main>'
15: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:605:in `start'
14: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:920:in `start'
13: from net-http-client.rb:10:in `block in <main>'
12: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1479:in `request'
11: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1517:in `transport_request'
10: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:165:in `reading_body'
9: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http.rb:1518:in `block in transport_request'
8: from net-http-client.rb:11:in `block (2 levels) in <main>'
7: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:204:in `read_body'
6: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:283:in `read_body_0'
5: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:278:in `inflater'
4: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:285:in `block in read_body_0'
3: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:324:in `read_chunked'
2: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/http/response.rb:324:in `ensure in read_chunked'
1: from /Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/protocol.rb:159:in `read'
/Users/khataev/.rbenv/versions/2.6.1/lib/ruby/2.6.0/net/protocol.rb:225:in `rbuf_fill': end of file reached (EOFError)
Could someone explain me:
1) Why curl return David x2, httparty - nothing and net/http - David x4, while server sent 5? Also I'm curious why httparty and net/http results differ, because one uses another under the hood.
2) Why in all cases we have an error (transfer closed with outstanding read data remaining and end of file reached (EOFError))?
Have figured it out. Headers do matter. This version makes it work for all clients
io_lambda = lambda{ |io|
5.times do |i|
puts i
io.write "David\r\n"
sleep 1
end
io.close
}
app = lambda do |env|
response_headers = {}
response_headers['Transfer-Encoding'] = 'binary'
response_headers["Content-Type"] = "text/plain"
response_headers["rack.hijack"] = io_lambda
[200, response_headers, nil]
end
run app
I installed the gem webmock, which blocks external connections during the test suite. After I installed it, I got an unexpected result when running the test suite.
I'm somewhat confused why the aws-sdk needs to connect to 169.254.169.254 on initialization?
rspec spec/models/concerns/posconcern_spec.rb
[Coveralls] Set up the SimpleCov formatter.
[Coveralls] Using SimpleCov's 'rails' settings.
An error occurred while loading ./spec/models/concerns/posconcern_spec.rb.
Failure/Error: require File.expand_path('../../config/environment', __FILE__)
WebMock::NetConnectNotAllowedError:
Real HTTP connections are disabled. Unregistered request: GET http://169.254.169.254/latest/meta-data/iam/security-credentials/ with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Ruby'}
You can stub this request with the following snippet:
stub_request(:get, "http://169.254.169.254/latest/meta-data/iam/security-credentials/").
with(headers: {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Ruby'}).
to_return(status: 200, body: "", headers: {})
============================================================
# /home/andey/.rvm/gems/ruby-2.3.4/gems/webmock-3.1.0/lib/webmock/http_lib_adapters/net_http.rb:114:in `request'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:109:in `http_get'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:90:in `block (2 levels) in get_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:105:in `open_connection'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:88:in `block in get_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:121:in `retry_errors'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:87:in `get_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:73:in `block in refresh'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:121:in `retry_errors'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:72:in `refresh'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/refreshing_credentials.rb:20:in `initialize'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/instance_profile_credentials.rb:51:in `initialize'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:90:in `new'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:90:in `instance_profile_credentials'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:12:in `block in resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:11:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/credential_provider_chain.rb:11:in `resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/aws-sdk-core/plugins/request_signer.rb:37:in `block in <class:RequestSigner>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:70:in `call'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:205:in `block in resolve_defaults'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:57:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:57:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:204:in `resolve_defaults'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:200:in `value_at'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:189:in `block in resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:189:in `resolve'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:177:in `apply_defaults'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/configuration.rb:150:in `build!'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/base.rb:68:in `build_config'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/base.rb:19:in `initialize'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/aws-sdk-core-2.10.52/lib/seahorse/client/base.rb:105:in `new'
# ./config/initializers/aws.rb:1:in `<top (required)>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:268:in `load'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:268:in `block in load'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:268:in `load'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:652:in `block in load_config_initializer'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/activesupport-4.2.9/lib/active_support/notifications.rb:166:in `instrument'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:651:in `load_config_initializer'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:616:in `block (2 levels) in <class:Engine>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:615:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/engine.rb:615:in `block in <class:Engine>'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:30:in `instance_exec'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:30:in `run'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:55:in `block in run_initializers'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:44:in `each'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:44:in `tsort_each_child'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/initializable.rb:54:in `run_initializers'
# /home/andey/.rvm/gems/ruby-2.3.4/gems/railties-4.2.9/lib/rails/application.rb:352:in `initialize!'
# ./config/environment.rb:5:in `<top (required)>'
# ./spec/rails_helper.rb:3:in `require'
# ./spec/rails_helper.rb:3:in `<top (required)>'
# ./spec/models/concerns/posconcern_spec.rb:1:in `require'
# ./spec/models/concerns/posconcern_spec.rb:1:in `<top (required)>'
Relevant link to repo:
https://github.com/aws/aws-sdk-ruby/blob/master/gems/aws-sdk-core/lib/aws-sdk-core/instance_profile_credentials.rb#L45
It is trying to connect to the AWS metadata server to get the AWS credentials. All AWS servers should be able to connect to the internal metadata server.
I installed the gem webmock, which blocks external connections during
the test suite
How are you blocking it? Firewall? Add a rule to allow traffic to 169.254.169.254 or stub it.
The script is trying to connect to AWS services. For that it needs credentials which can be provided in many ways. One way is to use AWS IAM role and fetch the credentials dynamically from the metadata server (169.254.169.254). Your script is connecting to 169.254.169.254 to fetch the credentials. It can be used to connect to AWS services later.
This is a follow up to #helloV 's answer.
Full documentation on
http://169.254.169.254/latest/meta-data/iam/security-credentials/
An application on the instance retrieves the security credentials
provided by the role from the instance metadata item
iam/security-credentials/role-name. The application is granted the
permissions for the actions and resources that you've defined for the
role through the security credentials associated with the role. These
security credentials are temporary and we rotate them automatically.
We make new credentials available at least five minutes prior to the
expiration of the old credentials.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#instance-metadata-security-credentials
Had a tough time researching on how to stub it. Will share my findings here hope it'll help other people.
Since AWS uses service name to fetch credentials for multiple services such as S3, any IAM Role, SQS etc.
But when dealing with IAM roles AWS hits http://169.254.169.254/latest/meta-data/iam/security-credentials/ which return the role name attached onto the machine and using the response of the same it hits the curl http://169.254.169.254/latest/meta-data/iam/security-credentials/#{role}.
So we need to stub both of the curls in order to successfully mock the IAM role. Below is the code for the same.
def self.auth(params = {})
status = params[:status].presence || 200
url = 'http://169.254.169.254/latest/meta-data/iam/security-credentials/'
WebMock.stub_request(:get, url)
.with({headers: request_headers}.compact)
.to_return(status: status, body: 'SampleRole', headers: {})
end
def self.role_auth(role = 'SampleRole')
url = "http://169.254.169.254/latest/meta-data/iam/security-credentials/#{role}"
WebMock.stub_request(:get, url)
.with({headers: request_headers}.compact)
.to_return(status: 200, body: sample_response, headers: {})
end
private
def self.request_headers
{
'Accept': '*/*',
'Accept-Encoding': 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
'User-Agent': 'aws-sdk-ruby3/3.21.2'
}.with_indifferent_access
end
def self.sample_response
{
Code: 'Success',
LastUpdated: '2020-05-08T05:55:23Z',
Type: 'AWS-HMAC',
AccessKeyId: Faker::Lorem.characters(10),
SecretAccessKey: Faker::Lorem.characters(20),
Token: Faker::Lorem.characters(60),
Expiration: '2020-05-08T12:00:45Z'
}.to_json
end
So whenver any rspec uses InstanceCredentials provider you need to stub both of the above defined requests in before block of each of the rspec.
Example:
before do
ApiStub::IamCred.auth
ApiStub::IamCred.role_auth
end
Hope it helps.
Happy Coding
There are some good answers in this thread on how to stub this. Another alternative (a better one?) is to eliminate the need for thE SDK to check for credentials from the instance meta data service. You can do this by providing credentials. These can be fake. A simple way to do this would be to provide credential via the ENV (e.g. ENV['AWS_ACCESS_TOKEN'] and ENV['SECRET_ACCESS_TOKEN']).
I am following the tutorial here: http://docs.aws.amazon.com/AmazonS3/latest/dev/UploadObjSingleOpRuby.html
to learn about using Ruby to upload file to S3, I wrote this:
require 'aws-sdk'
s3 = Aws::S3::Client.new(
access_key_id: '.......',
secret_access_key: '........'
)
s3 = Aws::S3::Resource.new(region:'us-east-1')
obj = s3.bucket('aws-adms-whitelisting-rules').object('test')
obj.upload_file('~/Desktop/ruby/test.rb')
But got following error:
/Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.10.7/lib/aws-sdk-core/plugins/regional_endpoint.rb:34:in `after_initialize': missing region; use :region option or export region name to ENV['AWS_REGION'] (Aws::Errors::MissingRegionError)
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.10.7/lib/seahorse/client/base.rb:84:in `block in after_initialize'
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.10.7/lib/seahorse/client/base.rb:83:in `each'
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.10.7/lib/seahorse/client/base.rb:83:in `after_initialize'
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.10.7/lib/seahorse/client/base.rb:21:in `initialize'
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-core-2.10.7/lib/seahorse/client/base.rb:105:in `new'
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-resources-2.10.7/lib/aws-sdk-resources/resource.rb:169:in `extract_client'
from /Library/Ruby/Gems/2.0.0/gems/aws-sdk-resources-2.10.7/lib/aws-sdk-resources/resource.rb:15:in `initialize'
from test.rb:35:in `new'
from test.rb:35:in `<main>'
Could you show me a path forward? Thanks
A more standard way to do this would be to pass the region as an argument to the instance of Aws::S3::Client you create.
You can then just pass the instance of Aws::S3::Client(docs) to the Aws::S3::Resource(docs) you're creating.
For example:
require 'aws-sdk'
s3 = Aws::S3::Client.new(
region: 'us-east-1',
access_key_id: '.......',
secret_access_key: '........'
)
resource = Aws::S3::Resource.new(client: s3)
obj = resource.bucket('aws-adms-whitelisting-rules').object('test')
obj.upload_file('~/Desktop/ruby/test.rb')
I'm trying to download all of my class notes from coursera. I figured that since I'm learning ruby this would be a good practice exercise, downloading all the PDFs they have for future use. Unfortunately though, I'm getting an exception saying ruby can't connect for some reason. Here is my code:
require 'net/http'
module Coursera
class Downloader
attr_accessor :page_url
attr_accessor :destination_directory
attr_accessor :cookie
def initialize(page_url,dest,cookie)
#page_url=page_url
#destination_directory = dest
#cookie=cookie
end
def download
puts #page_url
request = Net::HTTP::Get.new(#page_url)
puts #cookie.encoding
request['Cookie']=#cookie
# the line below is where the exception is thrown
res = Net::HTTP.start(#page_url.hostname, use_ssl=true,#page_url.port) {|http|
http.request(request)
}
html_page = res.body
pattern = /http[^\"]+\.pdf/
i=0
while (match = pattern.match(html_page,i)) != nil do
# 0 is the entire string.
url_string = match[0]
# make sure that 'i' is updated
i = match.begin(0)+1
# we want just the name of the file.
j = url_string.rindex("/")
filename = url_string[j+1..url_string.length]
destination = #destination_directory+"\\"+filename
# I want to download that resource to that file.
uri = URI(url_string)
res = Net::HTTP.get_response(uri)
# write that body to the file
f=File.new(destination,mode="w")
f.print(res.body)
end
end
end
end
page_url_string = 'https://class.coursera.org/datasci-002/lecture'
puts page_url_string.encoding
dest='C:\\Users\\michael\\training material\\data_science'
page_url=URI(page_url_string)
# I copied this from my browsers developer tools, I'm omitting it since
# it's long and has my session key in it
cookie="..."
downloader = Coursera::Downloader.new(page_url,dest,cookie)
downloader.download
At runtime the following is written to console:
Fast Debugger (ruby-debug-ide 0.4.22, debase 0.0.9) listens on 127.0.0.1:65485
UTF-8
https://class.coursera.org/datasci-002/lecture
UTF-8
Uncaught exception: A socket operation was attempted to an unreachable network. - connect(2)
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:878:in `initialize'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:878:in `open'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:878:in `block in connect'
C:/Ruby200-x64/lib/ruby/2.0.0/timeout.rb:52:in `timeout'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:877:in `connect'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:862:in `do_start'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:851:in `start'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:582:in `start'
C:/Users/michael/Documents/Aptana Studio 3 Workspace/practice/CourseraDownloader.rb:20:in `download'
C:/Users/michael/Documents/Aptana Studio 3 Workspace/practice/CourseraDownloader.rb:52:in `<top (required)>'
C:/Ruby200-x64/bin/rdebug-ide:23:in `load'
C:/Ruby200-x64/bin/rdebug-ide:23:in `<main>'
C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:878:in `initialize': A socket operation was attempted to an unreachable network. - connect(2) (Errno::ENETUNREACH)
from C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:878:in `open'
from C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:878:in `block in connect'
from C:/Ruby200-x64/lib/ruby/2.0.0/timeout.rb:52:in `timeout'
from C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:877:in `connect'
from C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:862:in `do_start'
from C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:851:in `start'
from C:/Ruby200-x64/lib/ruby/2.0.0/net/http.rb:582:in `start'
from C:/Users/michael/Documents/Aptana Studio 3 Workspace/practice/CourseraDownloader.rb:20:in `download'
from C:/Users/michael/Documents/Aptana Studio 3 Workspace/practice/CourseraDownloader.rb:52:in `<top (required)>'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/ruby-debug-ide-0.4.22/lib/ruby-debug-ide.rb:86:in `debug_load'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/ruby-debug-ide-0.4.22/lib/ruby-debug-ide.rb:86:in `debug_program'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/ruby-debug-ide-0.4.22/bin/rdebug-ide:110:in `<top (required)>'
from C:/Ruby200-x64/bin/rdebug-ide:23:in `load'
from C:/Ruby200-x64/bin/rdebug-ide:23:in `<main>'
I was following instructions here to write all the HTTP code. As far as I can see I'm following them ver-batim.
I'm using Windows 7, ruby 2.0.0p481, and Aptana Studio 3. When I copy the url into my browser it goes straight to the page without a problem. When I look at the request headers in my browser for that url, I don't see anything else I think I'm missing. I also tried setting the Host and Referer request headers, it made no difference.
I am out of ideas, and have already searched Stack Overflow for similar questions but that didn't help. Please let me know what I'm missing.
So, I had this same error message with a different project and the problem was that my machine literally couldn't connect to the IP / Port. Have you tried connecting with curl? If it works in your browser, it could be using a proxy or something to actually get there. Testing the URL with curl solved the problem for me.