I'm using the following to open a file within a directory that contains 'abc' in the filename. The wildcard is used as the rest of the filename includes a timestamp (time and date the file was created).
required = ["this", "two"]
log = Dir['*.log'].select { |f| f.include?('abc')}
result = required.all? do |phrase|
log.any? { |line| line.include?(phrase) }
end
index = result ? "Pass" : "Fail"
The above script is part of an automation process, so the full name will not be known (file created in one of the initial steps)
Is there a way to sort files by mtime prior to the .select?
I've tried .sort and .max_by prior to .select and get private method error
log = Dir['*.log'].max_by {|f| File.mtime(f)}.select { |f| f.include?('abc')}
I've tried it after .select, which gets the latest file but throws and underfined method 'any?' - this seems incorrect anyway as i should be sorting before selecting.
any assistance would be appreciated
#max_by is going to return at most one element. You should be able to use #sort_by to arrange the files according to their mtimes, #reverse this list, and then use #find (or its alias #detect) to pluck out the first element matching the predicate ({ |f| f[/abc/] }, in this case).
Related
I have this input repeated in 1850 files:
[
{
"id"=>66939,
"login"=>"XXX",
"url"=>"https://website.com/XX/users/XXX"
},
...
{}
]
And I wanted to make a list in a way that by looking for the login I can retrieve the ID using a syntax like:
users_list[XXX]
This is my desired output:
{"XXX"=>"66570", "XXX"=>"66570", "XXX"=>"66570", "XXX"=>"66570", ... }
My code is:
i2 = 1
while i2 != users_list_raw.parsed.count
temp_user = users_list_raw.parsed[i2]
temp_user_login = temp_user['login']
temp_user_id = temp_user['id']
user = {
temp_user_login => temp_user_id
}
users_list << user
i2 += 1
end
My output is:
[{"XXX":66570},{"XXX":66569},{"XXX":66568},{"XXX":66567},{"XXX":66566}, ... {}]
but this is not what I want.
What's wrong with my code?
hash[key] = value to add an entry in a hash. So I guess in your case users_list[temp_user_login] = temp_user_id
But I'm unsure why you'd want to do that. I think you could look up the id of a user by having the login with a statement like:
login = XXX
user = users_list.select {|user| user["login"] == login}.first
id = user["id"]
and maybe put that in a function get_id(login) which takes the login as its parameter?
Also, you might want to look into databases if you're going to manipulate large amounts of data like this. ORMs (Object Relational Mappers) are available in Ruby such as Data Mapper and Active Record (which comes bundled with Rails), they allow you to "model" the data and create Ruby objects from data stored in a database, without writing SQL queries manually.
If your goal is to lookup users_list[XXX] then a Hash would work well. We can construct that quite simply:
users_list = users_list_raw.parsed.each.with_object({}) do |user, list|
list[user['login']] = user['id']
end
Any time you find yourself writing a while loop in Ruby, there might be a more idiomatic solution.
If you want to keep track of a mapping from keys to values, the best data structure is a hash. Be aware that assignment via the array operator will replace existing values in the hash.
login_to_id = {}
Dir.glob("*.txt") { |filename| # Use Dir.glob to find all files that you want to process
data = eval(File.read(filename)) # Your data seems to be Ruby encoded hash/arrays. Eval is unsafe, I hope you know what you are doing.
data.each { |hash|
login_to_id[hash["login"]] = hash["id"]
}
}
puts login_to_id["XXX"] # => 66939
I am very new to ruby. I am able to connect to AWS S3 using ruby. I am using following code
filePath = '/TMEventLogs/stable/DeviceWiFi/20160803/1.0/20160803063600-2f9aa901-2ce7-4932-aafd-f7286cdb9871.csv'
s3.get_object({bucket: "analyticspoc", key:"TMEventLogs/stable/DeviceWiFi/20160803/1.0/"}, target:filePath ) do |chunk|
puts "1"
end
In above code s3 is client. "analyticspoc" is root bucket. My path to csv file is as follows All Buckets /analyticspoc/TMEventLogs/stable/DeviceWiFi/20160803/1.0/20160803063600-2f9aa901-2ce7-4932-aafd-f7286cdb9871.csv.
I have tried above code. I above code I was getting error Error getting objects: [Aws::S3::Errors::NoSuchKey] - The specified key does not exist. Using above code I want to read the contents of a file. How to do that ? Please tell me what is the mistake in above code
Got the answer. You can use list_objects for accessing array of file names in chunk(1000 at a time) where as get_object is used for accessing the content of a single file as follows
BUCKET = "analyticspoc"
path = "TMEventLogs/stable/DeviceWiFi/20160803/1.0/"
s3.list_objects(bucket:BUCKET, prefix: path).each do |response|
contents = response.contents
end
file_name = "TMEventLogs/stable/DeviceWiFi/20160803/1.0/012121212121"
response = s3.get_object(bucket: BUCKET, key: file_name)
As far as I can tell you're passing in the arguments incorrectly. It should be a single options hash according to the documentation for get_object:
s3.get_object(
bucket: "analyticspoc",
key: "TMEventLogs/stable/DeviceWiFi/20160803/1.0/",
target: filePath
) do |chunk|
puts "1"
end
I believe it was trying to use your hash as a string key which is obviously not going to work.
With Ruby the curly braces { } are only necessary in method calls if additional arguments follow that need to be in another hash or are non-hash in nature. This makes the syntax a lot less ugly in most cases where options are deliberately last, and sometimes first and last by virtue of being the only argument.
I'm saving a bunch of data as a hash and saving it as a json file. Code for the saving:
def write_to_file(id, data)
Dir.chdir(File.dirname(__FILE__)+"/specs")
filename = "./"+id+".json"
File.open("#{filename}", 'w') do |f2|
f2.write(data.to_json)
end
end
I want to save it as id.json, but the file is getting saved with a "?" at the end. For example, 199015806906670?.json where the original value of "id" is 199015806806670.
If I search for 19901580606670 I am also unable to use TAB to autocomplete.
Does anyone know why this is happening?
EDIT:
Sample from file containing the ids:
104184946332304
131321736945390
693134284084652
146974018804301
288608807960773
Code to get these:
url = File.open("newapps/curlist.txt","r")
url.each_line do |line|
func1(line) #func1 calls write_to_file, no changes to line in func1
end
As has been mentioned in the comments you likely have a newline as the last character of your id field. The character (being invalid) is being replaced with a question mark.
Use this to remove the newline...
filename = "./"+id.chomp+".json"
I am using the ruby Dir method to get all the filenames within a directory. Like this:
dir_files = Dir["/Users/AM/Desktop/07/week1/dailies/regionals/*.csv"]
This gives me an array with each element listed below:
/Users/AM/Desktop/07/week1/dailies/regionals/ch002.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch014.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch90.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch112.csv
/Users/AM/Desktop/07/week1/dailies/regionals/ch234.csv
Im trying to extract just the part of the above strings that matches: "regionals/*.csv"
How do I do that in Ruby?
The following didn't work
#files_array.each do |f|
f = f.split("/").match(/*.csv/)
i = f.include?(".csv")
puts "#{i.inspect}"
#self.process_file(f[i])
end
Whats a clever way of doing this? I intend to pass the returned string of each filename to a helper method for processing. But as you can see all the csv files are located in a different directory as my executing script.
My script thats executing this is located at
/Users/AM/Desktop/07/week1/dailies/myScript.rb
Thanks
This will always post back the final directory and file name, regardless of the file pattern:
#files_array.map { |f| f.split("/")[-2..-1].join("/") }
#=> ["regionals/ch002.csv", "regionals/ch014.csv", "regionals/ch90.csv", "regionals/ch112.csv", "regionals/ch234.csv"]
This gives you the desired values :)
dir_files.map {|path| path[/regionals\/.*.csv/]}
#=> ["regionals/ch002.csv", "regionals/ch014.csv", "regionals/ch90.csv", "regionals/ch112.csv", "regionals/ch234.csv"]
This is a sort of followup to my other MongoDB question about the torrent indexer.
I'm making an open source torrent indexer (like a mini TPB, in essence), and offer both SQLite and MongoDB for backend, currently.
However, I'm having trouble with the MongoDB part of it. In Sinatra, I get when trying to upload a torrent, or search for one.
In uploading, one needs to tag the torrent — and it fails here. The code for adding tags is as follows:
def add_tag(tag)
if $sqlite
unless tag_exists? tag
$db.execute("insert into #{$tag_table} values ( ? )", tag)
end
id = $db.execute("select oid from #{$tag_table} where tag = ?", tag)
return id[0]
elsif $mongo
unless tag_exists? tag
$tag.insert({:tag => tag})
end
return $tag.find({:tag => tag})[:_id] #this is the line it presumably crashes on
end
end
It reaches line 105 (noted above), and then fails. What's going on? Also, as an FYI this might turn into a few other questions as solutions come in.
Thanks!
EDIT
So instead of returning the tag result with [:_id], I changed the block inside the elsif to:
id = $tag.find({:tag => tag})
puts id.inspect
return id
and still get an error. You can see a demo at http://torrent.hypeno.de and the source at http://github.com/tekknolagi/indexer/
Given that you are doing an insert(), the easiest way to get the id is:
id = $tag.insert({:tag => tag})
id will be a BSON::ObjectId, so you can use appropriate methods depending on the return value you want:
return id # BSON::ObjectId('5017cace1d5710170b000001')
return id.to_s # "5017cace1d5710170b000001"
In your original question you are trying to use the Collection.find() method. This returns a Mongo::Cursor, but you are trying to reference the cursor as a document. You need to iterate over the cursor using each or next, eg:
cursor = $tag.find_one({:tag => tag})
return cursor.next['_id'];
If you want a single document, you should be using Collection.find_one().
For example, you can find and return the _id using:
return $tag.find_one({:tag => tag})['_id']
I think the problem here is [:_id]. I dont know much about Mongo but `$tag.find({:tag => tag}) is probably retutning an array and passing a symbol to the [] array operator is not defined.