Re-asking same question because:
Database Secrets
warning
Database secrets are currently deprecated and use a legacy Firebase token generator. Update your source code with the Firebase Admin SDK.
when getting the "secret" -- by which I infer they mean secret key.
for reference:
thufir#dur:~/ruby/firebase$
thufir#dur:~/ruby/firebase$ ./quickstart.rb
true
200
{"name"=>"-Kxf9rMd9p1F0cb2HTeM"}
thufir#dur:~/ruby/firebase$
thufir#dur:~/ruby/firebase$ cat quickstart.rb
#!/usr/bin/env ruby
require 'rubygems'
require 'firebase'
require 'pp'
require_relative 'config'
config = Config.new
#firebase = Firebase::Client.new(config.database_url)
firebase = Firebase::Client.new(config.database_url,config.database_secret)
response = firebase.push("todos", { :name => 'Pick the milk', :priority => 1 })
pp response.success? # => true
pp response.code # => 200
pp response.body # => { 'name' => "-INOQPH-aV_psbk3ZXEX" }
response.raw_body # => '{"name":"-INOQPH-aV_psbk3ZXEX"}'
thufir#dur:~/ruby/firebase$
this approach is relatively stable? I copied the information from the Google console GUI:
Add Firebase to your web app
Copy and paste the snippet below at the bottom of your HTML, before other script tags.
into the config file for reference. (Probably should use yaml or similar; works for now.)
Related
I am trying to get a Ruby Oauth2.0 client talking with the Ory Hydra docker-compose 5 minute demo. I am stuck on the client app's authentication code exchange for the token. LOG is below. Looks like the main problem is the following "hashedPassword is not the hash of the given password".
DEBUG LOG FROM HYDRA SERVER
time="2019-06-04T21:32:09Z" level=info msg="started handling request" method=POST remote="172.19.0.2:35482" request=/oauth2/token
hydra_1
time="2019-06-04T21:32:09Z" level=error msg="An error occurred" debug="crypto/bcrypt: hashedPassword is not the hash of the given password" description="Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)" error=invalid_client
hydra_1
time="2019-06-04T21:32:09Z" level=info msg="completed handling request" measure#hydra/public: http://127.0.0.1:4444/.latency=92931900 method=POST remote="172.19.0.2:35482" request=/oauth2/token status=401 text_status=Unauthorized took=92.9319ms
I've read up here and it would appear this 'is' possible.
Here is how I register my client 'test-app9' in hydra:
docker-compose -f quickstart.yml exec hydra hydra clients create --endpoint http://127.0.0.1:4445 --id test-app9 --secret secret--skip-tls-verify --grant-types authorization_code,refresh_token,client_credentials,implicit --response-types token,code,id_token --scope profile --callbacks http://127.0.0.1:8088/auth/callback --token-endpoint-auth-method client_secret_post -g client_credentials
I can indeed see the client appearing in the postgres DB from the docker-compose demo. The password 'secret' is hashed in the DB.
Here is my single Sinatra file acting as the Oauth2.0 client:
require 'rubygems'
require 'sinatra'
require 'oauth2'
require 'json'
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
# If you add your authentication in the header then use ~Sclient_secret_basic~T
# If you add your authentication details in the post use ~Sclient_secret_post~T
def client
OAuth2::Client.new('test-app9',
'secret',
# 'c2VjcmV0',
:site => "http://127.0.0.1:4445",
:logger => Logger.new('example.log'),
:authorize_url => "http://127.0.0.1:4444/oauth2/auth",
:token_url => "http://hydra:4444/oauth2/token")
end
set :root, File.dirname(__FILE__)
set :views, Proc.new { File.join(root, "views") }
set :run, true
set :port, 80
get "/" do
erb :index
end
get '/auth' do
authorization_url = client.auth_code.authorize_url(:redirect_uri => redirect_uri, :response_type => "code", :scope => "profile", :state => "pqrst1234")
puts "Redirecting to URL: #{authorization_url.inspect}"
redirect authorization_url
end
get '/auth/callback' do
begin
access_token = client.auth_code.get_token(params[:code], :redirect_uri => redirect_uri, :client_id => "test-app9", :client_secret => 'secret', :headers => {'Authorization' => 'basic_auth_header', 'client_id' => 'test-app9', 'client_secret' => 'c2VjcmV0'} )
api_url = "/me.json"
me = JSON.parse(access_token.get(api_url).body)
erb "<p>Your data:\n#{me.inspect}</p>"
rescue OAuth2::Error => e
erb %(<p>Wassup #{$!}</p><p>Retry</p>)
end
end
get '/auth/failure' do
erb "<h1>Authentication Failed:</h1><h3>message:<h3> <pre>#{params}</pre>"
end
def redirect_uri(path = '/auth/callback', query = nil)
uri = URI.parse(request.url)
uri.path = path
uri.query = query
uri.to_s
end
__END__
So some interesting notes:
It apparently does not matter what secret I initiate the OAuth2 client with. I can use 'secret' or the base64 encoded 'c2VjcmV0' string. Either way I get up to the token exchange portion.
I was shotgunning this and ended up putting the client_id and the client_secret in what I believe is the proper way to set in the headers as well as the body.
I have tried many variations of this. Can't seem to get the correct syntax that the author of this apparently succeeded with OR I'm hitting a bug (doubtful).
Anyone able to help here?
UPDATE
Fixed this myself. Problem was syntax issue when creating my client 'app'. This is the corrected version.
docker-compose -f quickstart.yml exec hydra hydra clients create --endpoint http://127.0.0.1:4445 --id test-app10 --secret secret --skip-tls-verify --grant-types authorization_code,refresh_token,client_credentials,implicit --response-types token,code,id_token --scope profile --callbacks http://127.0.0.1:8088/auth/callback --token-endpoint-auth-method client_secret_post -g client_credentials
I've managed to fix the original issue below by removing the .readonly from the URL.
I'm now being presented with this error:
{"error"=>{"errors"=>[{"domain"=>"global", "reason"=>"notFound", "message"=>"Resource Not Found: domain"}], "code"=>404, "message"=>"Resource Not Found: domain"}}
Again, I've taken a look at this error online but all the links seem to go to an error page on Google and nothing to help determine a fix for this error.
Original Issue
I'm trying to use the below script to access to Google Directory API using Ruby but I'm receiving an error when I try to execute the script.
Script
#!/usr/bin/env ruby
require 'rubygems'
require 'json'
require 'google/api_client'
require 'active_support/core_ext/hash'
# The value of client_email from the JSON file goes here
SERVICE_ACCOUNT_EMAIL = 'xxxxxxxxx-bk7gbhr8k2dfvtidm0cc0msnipo34c8p#developer.gserviceaccount.com'
# This should be an Admin
ACT_ON_BEHALF_EMAIL = 'xxxxxx#gmail.com'
# File path to the certificate
SERVICE_ACCOUNT_PKCS12_FILE_PATH = '/home/xxxx/3ce8a60d43a8fa5cb747441fdfd90c3b91d6f45b-privatekey.p12'
key = Google::APIClient::PKCS12.load_key(SERVICE_ACCOUNT_PKCS12_FILE_PATH, 'notasecret')
asserter = Google::APIClient::JWTAsserter.new(SERVICE_ACCOUNT_EMAIL, 'https://www.googleapis.com/auth/admin.directory.user.readonly', key)
client = Google::APIClient.new(:application_name => 'GAAccountAudit')
client.authorization = asserter.authorize(ACT_ON_BEHALF_EMAIL)
api = client.discovered_api('admin', 'directory_v1')
result = client.execute(
:api_method => api.users.list,
:parameters => {
'domain' => 'http://www.xxxxxx.com',
'orderBy' => 'givenName',
'maxResults' => 500,
'fields' => 'users(id,etag,primaryEmail,name,suspended)',
}
)
users = JSON.parse(result.body, {:symbolize_names => true})[:users]
users.each do |u|
puts "#{u[:name][:fullName]} <#{u[:primaryEmail]}>"
end
Error
{"error" : "unauthorized_client","error_description" : "Unauthorized client or scope in request."}
I've read up on this error but I've not been able to find a conclusive answer to this. Could anyone please advise how I resolve this error?
I'm trying to run a Query against Google Big Query, using the Ruby API.
This is my first project with Ruby and I'm still learning the language.
This is also my first project using the Google API.
ENVIORNMENT:
Windows 7
Ruby 1.9
Faraday 0.90
Goolge API - Service Account Authentication
My Code runs without giving any warnings or error messages through:
#client.authorization.fetch_access_token!
doc = File.read('bigQueryAPI.json')
#bigQuery = #client.register_discovery_document('bigquery', 'v2', doc)
NOTE: #bigQuery is loaded from a file because when I try to load #bigquery with
#bigquery = #client.discovered_api('bigquery', 'v2')
I get Google::APIClient::ClientError: Not Found and inspect only prints
#<Google::APIClient::API:0x17c94cc ID:bigquery:v2>
However If I save the Big Query API as a text file from
https://www.googleapis.com/discovery/v1/apis/bigquery/v2/rest
and then load it as a text file with
doc = File.read('bigQueryAPI.json')
#bigQuery = #client.register_discovery_document('bigquery', 'v2', doc)
then #bigQuery.inspect actually returns something useful.
#bigQuery.inspect output.
However, When I try to actually run a query, like so:
result = #client.execute!(
:api_method => #bigQuery.batch_path.query,
:body_object => { "query" => "SELECT count(DISTINCT repository_name) as repository_total, " +
"count(payload_commit) as commits_total, " +
"count(DISTINCT repository_name) / count(payload_commit) as average, " +
"FROM [githubarchive:github.timeline]" }, #,
:parameters => { "projectId" => #project_id })
I get the following error:
NoMethodError: undefined method `query_values' for nil:NilClass
Here's the Full Stacktrace of the error:
1) Error:
test_averages(Test_GitHub_Archive):
NoMethodError: undefined method `query_values' for nil:NilClass
C:/Ruby193/lib/ruby/gems/1.9.1/gems/google-api-client-0.7.1/lib/google/api_client/request.rb:145:in `uri='
C:/Ruby193/lib/ruby/gems/1.9.1/gems/google-api-client-0.7.1/lib/google/api_client/request.rb:101:in `initialize'
C:/Ruby193/lib/ruby/gems/1.9.1/gems/google-api-client-0.7.1/lib/google/api_client.rb:518:in `new'
C:/Ruby193/lib/ruby/gems/1.9.1/gems/google-api-client-0.7.1/lib/google/api_client.rb:518:in `generate_request'
C:/Ruby193/lib/ruby/gems/1.9.1/gems/google-api-client-0.7.1/lib/google/api_client.rb:583:in `execute!'
C:/Users/tfburton/Documents/private/ProjectSuggestor/RubyStats/GitHub_Archive.rb:39:in `get_averages'
C:/Users/tfburton/Documents/private/ProjectSuggestor/RubyStats/TestSpec/test_GitHub_Archive.rb:26:in `test_averages'
Here is the results for #client.inspect
NOTE: I would have pasted here, but my post ended over the length limit.
After doing some digging. It looks like I'm not passing the proper #bigQuery prameter to get the query function.
Looking at the dump for #bigQuery.inspect I need to pass the method at line 751.
However I can't seem to figure out how to pass that method.
If you strip out the rest of the inspect output the "path" looks like this:
{ "resources => { "jobs" => { "methods" => { "query"
I've tried #bigQuery.Jobs.query and that results in an error stating that #bigQuery.Jobs doesn't exist.
So am I creating #bigQuery correctly?
Why doesn't #bigQuery.Jobs.query work?
Here's how I got it to work with the bigquery.jobs.query method, which is probably what you need.
I had to set OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE because otherwise the authorization process would fail miserably, but that might be specific to min Win7/MgitSys environment. In any case, this specific line is not safe in prod.
require 'google/api_client'
require 'google/api_client/client_secrets'
require 'google/api_client/auth/installed_app'
require 'google/api_client/auth/file_storage'
require 'openssl'
require 'json'
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
# Initialize the client.
client = Google::APIClient.new(
:application_name => 'Example Ruby application',
:application_version => '1.0.0'
)
CREDENTIAL_STORE_FILE = "#{$0}-oauth2.json"
file_storage = Google::APIClient::FileStorage.new(CREDENTIAL_STORE_FILE)
# Initialize Google+ API. Note this will make a request to the
# discovery service every time, so be sure to use serialization
# in your production code. Check the samples for more details.
#bigQuery = client.discovered_api('bigquery', 'v2')
# Load client secrets from your client_secrets.json.
client_secrets = Google::APIClient::ClientSecrets.load
file_storage = Google::APIClient::FileStorage.new(CREDENTIAL_STORE_FILE)
if file_storage.authorization.nil?
client_secrets = Google::APIClient::ClientSecrets.load
# The InstalledAppFlow is a helper class to handle the OAuth 2.0 installed
# application flow, which ties in with FileStorage to store credentials
# between runs.
flow = Google::APIClient::InstalledAppFlow.new(
:client_id => client_secrets.client_id,
:client_secret => client_secrets.client_secret,
:scope => ['https://www.googleapis.com/auth/cloud-platform','https://www.googleapis.com/auth/bigquery']
)
client.authorization = flow.authorize(file_storage)
else
client.authorization = file_storage.authorization
end
puts "authorized, requesting"
# Make an API call.
result = client.execute!(
:api_method => #bigQuery.jobs.query,
:body_object => { "query" => "SELECT count(DISTINCT repository_name) as repository_total, " +
"count(payload_commit) as commits_total, " +
"count(DISTINCT repository_name) / count(payload_commit) as average, " +
"FROM [githubarchive:github.timeline]" }, #,
:parameters => { "projectId" => "845227657643" })
puts JSON.dump result.data
I need to change some metadata (Content-Type) on hundreds or thousands of objects on S3. What's a good way to do this with ruby? As far as I can tell there is no way to save only metadata with fog.io, the entire object must be re-saved. Seems like using the official sdk library would require me rolling a wrapper environment just for this one task.
You're right, the official SDK lets you modify the object metadata without uploading it again. What it does is copy the object but that's on the server so you don't need to download the file and re-upload it.
A wrapper would be easy to implement, something like
bucket.objects.each do |object|
object.metadata['content-type'] = 'application/json'
end
In the v2 API, you can use Object#copy_from() or Object.copy_to() with the :metadata and :metadata_directive => 'REPLACE' options to update an object's metadata without downloading it from S3.
The code in Joost's gist throws this error:
Aws::S3::Errors::InvalidRequest: This copy request is illegal because
it is trying to copy an object to itself without changing the object's
metadata, storage class, website redirect location or encryption
attributes.
This is because by default AWS ignores the :metadata supplied with a copy operation because it copies metadata. We must set the :metadata_directive => 'REPLACE' option if we want to update the metadata in-place.
See http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#copy_from-instance_method
Here's a full, working code snippet that I recently used to perform metadata update operations:
require 'aws-sdk'
# S3 setup boilerplate
client = Aws::S3::Client.new(
:region => 'us-east-1',
:access_key_id => ENV['AWS_ACCESS_KEY'],
:secret_access_key => ENV['AWS_SECRET_KEY'],
)
s3 = Aws::S3::Resource.new(:client => client)
# Get an object reference
object = s3.bucket('my-bucket-name').object('my-object/key')
# Create our new metadata hash. This can be any hash; in this example we update
# existing metadata with a new key-value pair.
new_metadata = object.metadata.merge('MY_NEW_KEY' => 'MY_NEW_VALUE')
# Use the copy operation to replace our metadata
object.copy_to(object,
:metadata => new_metadata,
# IMPORTANT: normally S3 copies the metadata along with the object.
# we must supply this directive to replace the existing metadata with
# the values we supply
:metadata_directive => "REPLACE",
)
For easy re-use:
def update_metadata(s3_object, new_metadata = {})
s3_object.copy_to(s3_object,
:metadata => new_metadata
:metadata_directive => "REPLACE"
)
end
For future readers, here's a complete sample of changing stuff using the Ruby aws-sdk v1 (also see this Gist for a aws-sdk v2 sample):
# Using v1 of Ruby aws-sdk as currently v2 seems not able to do this (broken?).
require 'aws-sdk-v1'
key = YOUR_AWS_KEY
secret = YOUR_AWS_SECRET
region = YOUR_AWS_REGION
AWS.config(access_key_id: key, secret_access_key: secret, region: region)
s3 = AWS::S3.new
bucket = s3.buckets[bucket_name]
bucket.objects.with_prefix('images/').each do |obj|
puts obj.key
# Add metadata: {} to next line for more metadata.
obj.copy_from(obj.key, content_type: obj.content_type, cache_control: 'max-age=1576800000', acl: :public_read)
end
after some search this seems to work for me
obj.copy_to(obj, :metadata_directive=>"REPLACE", :acl=>"public-read",:content_type=>"text/plain")
Using the sdk to change the content type will result in x-amz-meta- prefix. My solution was to use ruby + aws cli. This will directly write to the content-type instead of x-amz-meta-content-type.
ids_to_copy = all_object_ids
ids_to_copy.each do |id|
object_key = "#{id}.pdf"
command = "aws s3 cp s3://{bucket-name}/#{object_key} s3://{bucket-name}/#{object_key} --no-guess-mime-type --content-type='application/pdf' --metadata-directive='REPLACE'"
system(command)
end
This API appears to be available now:
Fog::Storage.new({
:provider => 'AWS',
:aws_access_key_id => 'foo',
:aws_secret_access_key => 'bar',
:endpoint => 'https://s3.amazonaws.com/',
:path_style => true
}).put_object_tagging(
'bucket_name',
's3_key',
{foo: 'bar'}
)
Anyone got a working example of using ruby to post to a presigned URL on s3
I have used aws-sdk and right_aws both.
Here is the code to do this.
require 'rubygems'
require 'aws-sdk'
require 'right_aws'
require 'net/http'
require 'uri'
require 'rack'
access_key_id = 'AAAAAAAAAAAAAAAAA'
secret_access_key = 'ASDFASDFAS4646ASDFSAFASDFASDFSADF'
s3 = AWS::S3.new( :access_key_id => access_key_id, :secret_access_key => secret_access_key)
right_s3 = RightAws::S3Interface.new(access_key_id, secret_access_key, {:multi_thread => true, :logger => nil} )
bucket_name = 'your-bucket-name'
key = "your-file-name.ext"
right_url = right_s3.put_link(bucket_name, key)
right_scan_command = "curl -I --upload-file #{key} '#{right_url.to_s}'"
system(right_scan_command)
bucket = s3.buckets[bucket_name]
form = bucket.presigned_post(:key => key)
uri = URI(form.url.to_s + '/' + key)
uri.query = Rack::Utils.build_query(form.fields)
scan_command = "curl -I --upload-file #{key} '#{uri.to_s}'"
system(scan_command)
Can you provide more information on how a "presigned URL" works? Is it like this:
AWS::S3::S3Object.url_for(self.full_filename,
self.bucket_name, {
:use_ssl => true,
:expires_in => ttl_seconds
})
I use this code to send authenticated clients the URL to their S3 file. I believe this is the "presigned URL" that you're asking about. I haven't used this code for a PUT, so I'm not exactly sure if it's right for you, but it might get you close.
I know this is an older question, but I was wondering the same thing and found an elegant solution in the AWS S3 Documentation.
require 'net/http'
file = "somefile.ext"
url = URI.parse(presigned_url)
Net::HTTP.start(url.host) do |http|
http.send_request("PUT", url.request_uri, File.read(file), {"content-type" => "",})
end
This worked great for my Device Farm uploads.
Does anything on the s3 library page cover what you need? There are loads of examples there.
There are some generic REST libraries for Ruby; Google for "ruby rest client". See also HTTParty.
I've managed to sort it out. Turns out the HTTP:Net in Ruby is has some short comings. Lot of Monkeypatch later I got it working.. More details when I have time. thank