I am very new to ruby. I am able to connect to AWS S3 using ruby. I am using following code
filePath = '/TMEventLogs/stable/DeviceWiFi/20160803/1.0/20160803063600-2f9aa901-2ce7-4932-aafd-f7286cdb9871.csv'
s3.get_object({bucket: "analyticspoc", key:"TMEventLogs/stable/DeviceWiFi/20160803/1.0/"}, target:filePath ) do |chunk|
puts "1"
end
In above code s3 is client. "analyticspoc" is root bucket. My path to csv file is as follows All Buckets /analyticspoc/TMEventLogs/stable/DeviceWiFi/20160803/1.0/20160803063600-2f9aa901-2ce7-4932-aafd-f7286cdb9871.csv.
I have tried above code. I above code I was getting error Error getting objects: [Aws::S3::Errors::NoSuchKey] - The specified key does not exist. Using above code I want to read the contents of a file. How to do that ? Please tell me what is the mistake in above code
Got the answer. You can use list_objects for accessing array of file names in chunk(1000 at a time) where as get_object is used for accessing the content of a single file as follows
BUCKET = "analyticspoc"
path = "TMEventLogs/stable/DeviceWiFi/20160803/1.0/"
s3.list_objects(bucket:BUCKET, prefix: path).each do |response|
contents = response.contents
end
file_name = "TMEventLogs/stable/DeviceWiFi/20160803/1.0/012121212121"
response = s3.get_object(bucket: BUCKET, key: file_name)
As far as I can tell you're passing in the arguments incorrectly. It should be a single options hash according to the documentation for get_object:
s3.get_object(
bucket: "analyticspoc",
key: "TMEventLogs/stable/DeviceWiFi/20160803/1.0/",
target: filePath
) do |chunk|
puts "1"
end
I believe it was trying to use your hash as a string key which is obviously not going to work.
With Ruby the curly braces { } are only necessary in method calls if additional arguments follow that need to be in another hash or are non-hash in nature. This makes the syntax a lot less ugly in most cases where options are deliberately last, and sometimes first and last by virtue of being the only argument.
Related
I am trying to save some class objects to a csv file, everything works fine. I can save and read back from the csv file, there is only a 'minor' problem with an attribute that is an Array of Strings.
When I save it to the file it appears like this: "[""Dan Brown""]"
CSV.open('documents.csv', "w") do |csv|
csv << %w[ISBN Titre Auteurs Type Disponibilité]
#docs.each { |doc|
csv << [doc.isbn, doc.titre, doc.auteurs, doc.type, doc.empruntable ? "Disponible" : "Emprunté"]
}
end
And when I try to extract the data from the file I end up with something like this: ["[\"Dan Brown\"]"].
table = CSV.parse(File.read("documents.csv"), headers: true)
table.each do |row|
doc = Document.new(row['Titre'], row['ISBN'], row['Type'])
doc.auteurs << row['Auteurs'] #This the array where there is a 'problem'
if row['Disponibilité'] == "Disponible"
doc.empruntable = true
else
doc.empruntable = false
end
#docs.push(doc) #this an array where I save my objects
end
I tried many things to solve this but without any luck. I would be thankful if you can help me find a solution.
Since a CSV file, by it's nature, contains in its fields only strings, not arrays or other data types, the CSV class is applying the to_s method of the objects to turn them into a string before putting them into the CSV.
When you later read them back, you just get this - the string representation of what once had been your array. The only one who knows that 'Auteurs' should end up as an array of strings, is the application, i.e. you.
Hence on reading the CSV, after having extracted the autheurs string, you need to convert it manually back to an Array, because there is no automatic "inverse method" to reverse the to_s.
A cheap, but dangerous way to do it, is to use eval, which indeed would reconstruct your array. However, you need to be sure that nobody had a chance to fiddle manually with the CSV data, because an eval allows sneaking in arbitrary code.
A safer way would be to either write your own conversion function to and from String representation, or use a format such as YAML or JSON for representing the Array as String, instead of using to_s.
I have a text file (objects.txt) which contains Objects and its attributes.
The content of the file is something like:
Object.attribute = "data"
On a different file, I am Loading the objects.txt file and if I type:
puts object.attribute it prints out data
The issue comes when I am trying to access the object and/or the attribute with a string. What I am doing is:
var = "object" + "." + "access"
puts var
It prints out object.access and not the content of it "data".
I have already tried with instance_variable_get and it works, but I have to modify the object.txt and append an # at the beginning to make it an instance variable, but I cannot do this, because I am not the owner of the object.txt file.
As a workaround I can parse the object.txt file and get the data that I need but I don't want to do this, as I want take advantage of what is already there.
Any suggestions?
Yes, puts is correctly spitting out "object.access" because you are creating that string exactly.
In order to evaluate a string as if it were ruby code, you need to use eval()
eg:
var = "object" + "." + "access"
puts eval(var)
=> "data"
Be aware that doing this is quite dangerous if you are evaluating anything that potentially comes from another user.
I'm using the following to open a file within a directory that contains 'abc' in the filename. The wildcard is used as the rest of the filename includes a timestamp (time and date the file was created).
required = ["this", "two"]
log = Dir['*.log'].select { |f| f.include?('abc')}
result = required.all? do |phrase|
log.any? { |line| line.include?(phrase) }
end
index = result ? "Pass" : "Fail"
The above script is part of an automation process, so the full name will not be known (file created in one of the initial steps)
Is there a way to sort files by mtime prior to the .select?
I've tried .sort and .max_by prior to .select and get private method error
log = Dir['*.log'].max_by {|f| File.mtime(f)}.select { |f| f.include?('abc')}
I've tried it after .select, which gets the latest file but throws and underfined method 'any?' - this seems incorrect anyway as i should be sorting before selecting.
any assistance would be appreciated
#max_by is going to return at most one element. You should be able to use #sort_by to arrange the files according to their mtimes, #reverse this list, and then use #find (or its alias #detect) to pluck out the first element matching the predicate ({ |f| f[/abc/] }, in this case).
I am using the ruby Dir method to get all the filenames within a directory. Like this:
dir_files = Dir["/Users/AM/Desktop/07/week1/dailies/regionals/*.csv"]
This gives me an array with each element listed below:
/Users/AM/Desktop/07/week1/dailies/regionals/ch002.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch014.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch90.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch112.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch234.csv
Im trying to extract just the part of the above strings that matches: "regionals/*.csv"
How do I do that in Ruby?
The following didn't work
#files_array.each do |f|
f = f.split("/").match(/*.csv/)
i = f.include?(".csv")
puts "#{i.inspect}"
#self.process_file(f[i])
end
Whats a clever way of doing this? I intend to pass the returned string of each filename to a helper method for processing. But as you can see all the csv files are located in a different directory as my executing script.
My script thats executing this is located at
/Users/AM/Desktop/07/week1/dailies/myScript.rb
Thanks
This will always post back the final directory and file name, regardless of the file pattern:
#files_array.map { |f| f.split("/")[-2..-1].join("/") }
#=> ["regionals/ch002.csv", "regionals/ch014.csv", "regionals/ch90.csv", "regionals/ch112.csv", "regionals/ch234.csv"]
This gives you the desired values :)
dir_files.map {|path| path[/regionals\/.*.csv/]}
#=> ["regionals/ch002.csv", "regionals/ch014.csv", "regionals/ch90.csv", "regionals/ch112.csv", "regionals/ch234.csv"]
comics = load_comics( '/comics.txt' )
Popup.make do
h1 "Comics on the Web"
list do
comics.each do |name, url|
link name, url
end
end
end
I am new to ruby. This is a piece of code from a ruby website.
I cant find what 'link' and 'list' keyword in the menu.
can someone explain it a little bit those two keywords, and where is the definition of those two keyword .
I am also confused on how they read the variables name and url, they are reading it by the space at the same line or what?
so if I have
Comics1 link_of_comics_site_1
Comics2 link_of_comics_site_2
Comics3 link_of_comics_site_3
so for the first iteration, name=Comics1, and url =link_of_comics_site_1
Thanks.
That's not just Ruby. That's a template for a webpage using ruby add-on methods for HTML generation.
But presumably, the result of the call to load_comics is a Hash, where the keys are names and the values are URLs. You could make one of those yourself:
my_comics_hash = { "name1" => "url1", "name2" => "url2" }
which you can then iterate over the same way:
my_comics_hash.each do |name, url|
puts "Name #{name} goes with URL #{url}"
end
In your code, it's building up an HTML list inside a popup window, but it's the same idea. The each method iterates over a collection - in this case a Hash - and runs some code on every item in that collection - in this case, each key/value pair. When you call each, you pass it a block of code inside do ... end; that's the code that gets run on each item. The current item is passed to the code block, which declares a variable to hold it inside the pipes right after the word do. Since we're iterating over key/value pairs, we can declare two variables, and the key goes in the first and the value in the second.
In ruby function, parenthesis is optional and the ";" end of statement is also optional. ej
link "click here" , "http://myweb.com"
is equivalent to :
link("click here", "http://myweb.com");
But If you have more than one statement in a line the ";" is a must, ej
link("click here1", "http://myweb.com"); link("click here2", "http://myweb.com");
In your code it could be written in
link(name, url)
or just
link(name, url);
or
link name, url
But it is highly recommended to put parenthesis around function parameters for readability unless you have other reason . The ";" is not common in ruby world .