I checked the whole azure-storage-blob gem and didn't find any way to get the URI for a blob. Is there some way to construct it correctly and in a generic way that will work for any other blob in any region?
I used S3 SDK before and I'm well grounded in S3 but new to Azure.
There is a protected method called blob_uri that looks like this:
def blob_uri(container_name, blob_name, query = {}, options = {})
if container_name.nil? || container_name.empty?
path = blob_name
else
path = ::File.join(container_name, blob_name)
end
options = { encode: true }.merge(options)
generate_uri(path, query, options)
end
So you could take the short cut of:
blob_client = Azure::Storage::Blob::BlobService.create(storage_account_name: 'XXX' , storage_access_key: 'XXX')
blob_client.send(:blob_uri, container_name,blob_name)
However, the actual URI is simply:
https://[storage_account_name].blob.core.windows.net/container/[container[s]]/[blob file name]
So since you have to know the blob name and the container to access to blob.
File.join(blob_client.host,container,blob_name)
Is the URI to the blob
Related
How do I get a media instance through the REST api? I'm looking to either download the file or fetch the url for that media. I'm using Ruby
You can get the media SID from a message resource. For example:
account_sid = ENV['TWILIO_ACCOUNT_SID']
auth_token = ENV['TWILIO_AUTH_TOKEN']
#client = Twilio::REST::Client.new(account_sid, auth_token)
messages = #client.conversations
.conversations('CHXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX')
.messages
.list(order: 'desc', limit: 20)
messages.each do |message|
puts message.sid
message.media.each do |media|
puts "#{media.sid}: #{media.filename} #{media.content_type}"
end
end
I've not actually tried the above, the media objects may just be plain hashes and you would access the sid with media['sid'] instead.
Once you have the SID, you can fetch the media by constructing the following URL using the Chat service SID and the Media SID:
https://mcs.us1.twilio.com/v1/Services/<chat_service_sid>/Media/<Media SID>
For downloading files in Ruby, I like to use the Down gem. You can read about how to use Down to download images here. Briefly, here's how you would use Down and the URL above to download the image:
conversation_sid = "CHXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
media_sid = "MEXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
account_sid = ENV["TWILIO_ACCOUNT_SID"]
auth_token = ENV["TWILIO_AUTH_TOKEN"]
url = "https://#{account_sid}:#{auth_token}#mcs.us1.twilio.com/v1/Services/#{conversation_sid}/Media/#{media_sid}"
tempfile = Down.download(url)
I want to upload data binary directly to GCP storage, without writing the file to disk. Below is the code snippet I have created to get to the state that I am going to be at.
require 'google/cloud/storage'
bucket_name = '-----'
data = File.open('image_block.jpg', 'rb') {|file| file.read }
storage = Google::Cloud::Storage.new("project_id": "maybe-i-will-tell-u")
bucket = storage.bucket bucket_name, skip_lookup: true
Now I want to directly put this data into a file on gcs, without having to write a file to disk.
Is there an efficient way we can do that?
I tried the following code
to_send = StringIO.new(data).read
bucket.create_file to_send, "image_inder_11111.jpg"
but this throws an error saying
/google/cloud/storage/bucket.rb:2898:in `file?': path name contains null byte (ArgumentError)
from /home/inder/.gem/gems/google-cloud-storage-1.36.1/lib/google/cloud/storage/bucket.rb:2898:in `ensure_io_or_file_exists!'
from /home/inder/.gem/gems/google-cloud-storage-1.36.1/lib/google/cloud/storage/bucket.rb:1566:in `create_file'
from champa.rb:14:in `<main>'
As suggested by #stefan, It should be to_send = StringIO.new(data), i.e. without .read (which would return a string again)
I'm trying to write some Ruby code to update GitLab CI/CD variables using the REST endpoint update variable. When I perform a curl with the same path, the same private token, and the same --form data it updates the variable as expected. When I use the Ruby code that I put together based on reading stackoverflow and the net::http docs, it fails with a 404 URL not found.
I can use a similar piece of code to create a new CI/CD variable successfully. I can also delete an existing variable, and re-create it, but it I would like to know the mistake I am making in the update call.
Can someone point out what I did wrong?
#!/usr/bin/env ruby
require 'net/http'
require 'uri'
token = File.read(__dir__ + '/.gitlab-token').chomp
host = 'https://gitlab.com/'
variables_path = 'api/v4/projects/123456/variables'
env_var = 'MY_VAR'
update_uri = URI(host + variables_path + '/' + env_var)
# I've written the above this way because my actual code
# has a delete and create in order to "update" the variable
response = Net::HTTP.start(update_uri.host, update_uri.port, use_ssl: true) do |http|
update_request = Net::HTTP::Post.new(update_uri)
update_request['PRIVATE-TOKEN'] = token
form_data = [
['value', 'a new value']
]
update_request.set_form(form_data, 'multipart/form-data')
response = http.request(update_request)
response.body
end
I have tried to create publicly readable openstack object store containers like this:
os = OpenStack::Connection.create(...)
container = os.create_container(container_name)
container.set_metadata({'X-Container-Read' => '.r:*'})
Using my code above, the newly created containers are private.
What is the correct way to create containers with public read permissions with the ruby openstack gem?
You can try the following way.
You can redfine the create_container method
then
class MyStack < OpenStack::Swift::Connection
def create_container(containername)
super
#connection.req("PUT", path, {:headers=>{"Content-Length"=>"0", "X-Container-Read" => ".r:*", "X-Container-Write" => ".r:*}})
OpenStack::Swift::Container.new(self, containername)
end
end
These "X-Container-Read" => ".r:*", "X-Container-Write" => ".r:*" header value you need to set.
or
container.set_metadata({"X-Container-Read" => ".r:*", "X-Container-Write" => ".r:*"})
Here's what I ended up doing:
module PubliclyRedableContainerMonkeyPatch
def create_publicy_readable_container(containername)
raise OpenStack::Exception::InvalidArgument.new("Container name cannot contain '/'") if containername.match("/")
raise OpenStack::Exception::InvalidArgument.new("Container name is limited to 256 characters") if containername.length > 256
path = "/#{URI.encode(containername.to_s)}"
#connection.req("PUT", path, {:headers=>{"Content-Length"=>"0", "X-Container-Read" => ".r:*"}})
OpenStack::Swift::Container.new(self, containername)
end
end
OpenStack::Swift::Connection.include PubliclyRedableContainerMonkeyPatch
os = OpenStack::Connection.create(...)
container = os.create_publicy_readable_container(container_name)
Worksforme. :)
I am using aws-sdk-ruby for deleting a file saved in a bucket in my amazon s3 account, but i can't figure out why i am able to delete the desired file from S3 bucket using the following code.
this is my code
require 'aws-sdk-v1'
require 'aws-sdk'
ENV['AWS_ACCESS_KEY_ID'] = "XXXXXXX"
ENV["AWS_SECRET_ACCESS_KEY"] = '/ZZZZZZZZ'
ENV['AWS_REGION'] = 'us-east-1'
s3 = Aws::S3::Resource.new
bucket = s3.bucket('some-bucket')
obj = bucket.object('https://s3.amazonaws.com/some-bucket/38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg')
obj.delete
The documentation tells that it should look like this:
s3 = Aws::S3.new
bucket = s3.buckets['some-bucket']
object = bucket.objects['38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg']
object.delete
Please note:
the square brackets,
that the object's key doesn't include the domain and
instead of creating an instance of Aws::S3::Resource create an instance of AWS::S3
If you use API version 3 (aws-sdk-s3 (1.81.1)) you should do something like below:
s3 = Aws::S3::Client.new
s3.delete_object(bucket: 'bucket_name', key: 'bucket_folder/file.txt')
it should be:
objs = bucket.objects('https://s3.amazonaws.com/some-bucket/38ac8226-fa72-4aee-8c3d-a34a1db77b91/some_image.jpg')
objs.each {|obj| obj.delete}
With the aws-sdk v2: I had to do this: (see Doc)
$s3.bucket("my-bucket").objects(prefix: 'my_folder/').batch_delete!
(delete is deprecated in favor of batch_delete)
Useful post: https://ruby.awsblog.com/post/Tx1H87IVGVUMIB5/Using-Resources