trying to get my head around Feedzirra here.
I have it all setup and everything, and can even get results and updates, but something odd is going on.
I came up with the following code:
def initialize(feed_url)
#feed_url = feed_url
#rssObject = Feedzirra::Feed.fetch_and_parse(#feed_url)
end
def update_from_feed_continuously()
#rssObject = Feedzirra::Feed.update(#rssObject)
if #rssObject.updated?
puts #rssObject.new_entries.count
else
puts "nil"
end
end
Right, what I'm doing above, is starting with the big feed, and then only getting updates. I'm sure I must be doing something stupid, as even though I'm able to get the updates, and store them on the same instance variable, after the first time, I'm never able to get those again.
Obviously this happens because I'm overwriting my instance variable with only updates, and lose the full feed object.
I then thought about changing my code to this:
def update_from_feed_continuously()
feed = Feedzirra::Feed.update(#rssObject)
if feed.updated?
puts feed.new_entries.count
else
puts "nil"
end
end
Well, I'm not overwriting anything and that should be the way to go right?
WRONG, this means I'm doomed to always try to get updates to the same static feed object, as although I get the updates on a variable, I'm never actually updating my "static feed object", and newly added items will be appended to my "feed.new_entries" as they in theory are new.
I'm sure I;m missing a step here, but I'd really appreciate if someone could shed me a light on it. I've been going through this code for hours, and can't get to grips with it.
Obviously it should work fine, if I did something like:
if feed.updated?
puts feed.new_entries.count
#rssObject = initialize(#feed_url)
else
Because that would reinitialize my instance variable with a brand new feed object, and the updates would come again.
But that also means that any new update added on that exact moment would be lost, as well as massive overkill, as I'd have to load the thing again.
Thanks in advance!
How to do updates is a bit counterintuitive with the current API. This example shows the best way to do it:
# I'm using Atom here, but it could be anything. You don't need to know ahead of time.
# It will parse out to the correct format when it updates.
feed_to_update = Feedzirra::Parser::Atom.new
feed_to_update.feed_url = some_stored_feed_url
feed_to_update.etag = some_stored_feed_etag
feed_to_update.last_modified = some_stored_feed_last_modified
last_entry = Feedzirra::Parser::AtomEntry.new
last_entry.url = the_url_of_the_last_entry_for_a_feed
feed_to_update.entries = [last_entry]
updated_feed = Feedzirra::Feed.update(feed_to_update)
updated_feed.updated? # => nil if there is nothing new
updated_feed.new_entries # => [] if nothing new otherwise a collection of feedzirra entries
updated_feed.etag # => same as before if nothing new. although could change with comments added to entries.
updated_feed.last_modified # => same as before if nothing new. although could change with comments added to entries.
Basically, you'll have to save off four pieces of data (feed_url,
last_modified, etag, and the url of the most recent entry). Then when you
want to do updates you construct a new feed object and call update on
that.
I think a more obvious solution would be to add :if_modified_since option to fetch_and_parse method of class Feed, see https://github.com/pauldix/feedzirra/blob/master/lib/feedzirra/feed.rb#L116 and https://github.com/pauldix/feedzirra/blob/master/lib/feedzirra/feed.rb#L206
You can reset #rssObject to the updated feed.
feed = Feedzirra::Feed.update(#rssObject)
if feed.updated?
puts feed.new_entries.count
#rssObject = feed
else
puts 'nil'
end
The number of entries in #rssObject will keep growing as new entries are found. So if the first fetch finds 10 entries, and then next finds 10 new entries, #rssObject.entries.size will be 20.
Note that you can do this regardless of whether update finds new entries. If feed.updated? is false, feed will be the original feed object, #rssObject.
Related
I have some code to make API requests through Full contact to resolve domains for a list of company names, and output these in a csv table. I found the code was terminating whenever it hit a response code other than 200 or 202.
I have the following rescue block written:
def get_parse_all
ORG_ARRAY.each do |company_name|
begin
org_info = get_org_info(company_name)
rescue
next
end
parse_org_info(org_info)
end
end
The issue is, I can't figure out how to still include the skipped company names (due to bad response code)in the output. I am getting a list of the successful calls, but I can't tell which ones were skipped and why.
I have tried to puts "error" before next, but it doesn't appear in the output csv. And if I remove next, I get a nil:NilClass (NoMethodError)
I've read some documentation but I am new to this and I'm quite stuck. If someone could point me in the right direction, that would be greatly appreciated!
Thank you in advance for any help :)
In this case, it sounds like what you're looking to do is iterate over an array and transform each element to another value. The result would be another array, and each element within it would either be org_info or an error.
For this, you would use map instead of each. Remember that each does not return the result of the block, e.g. ORG_ARRAY.each do will always return ORG_ARRAY no matter what you do in the block.
def get_parse_all
ORG_ARRAY.map do |company_name|
begin
parse_org_info(get_org_info(company_name))
rescue => e
e
end
end
end
As mentioned in a comment, you should also avoid using "naked rescue" - rescue a more specific error instead of any error at all.
I working on a json file, I think. But Regardless, I'm working with a lot of different hashes and fetching different values and etc. This is
{"notification_rule"=>
{"id"=>"0000000",
"contact_method"=>
{"id"=>"000000",
"address"=>"cod.lew#gmail.com",}
{"notification_rule"=>
{"id"=>"000000"
"contact_method"=>
{"id"=>"PO0JGV7",
"address"=>"cod.lew#gmail.com",}
Essential, this is the type of hash I'm currently working with. With my code:
I wanted to stop duplicates of the same thing in the text file. Because whenever I run this code it brings both the address of both these hashes. And I understand why, because its looping over again, but I thought this code that I added would help resolve that issue:
Final UPDATE
if jdoc["notification_rule"]["contact_method"]["address"].to_s.include?(".com")
numbers.print "Employee Name: "
numbers.puts jdoc["notification_rule"]["contact_method"]["address"].gsub(/#target.com/, '').gsub(/\w+/, &:capitalize)
file_names = ['Employee_Information.txt']
file_names.each do |file_name|
text = File.read(file_name)
lines = text.split("\n")
new_contents = lines.uniq.join("\n")
File.open(file_name, "w") { |file| file.puts new_contents }
end
else
nil
end
This code looks really confused and lacking a specific purpose. Generally Ruby that's this tangled up is on the wrong track, as with Ruby there's usually a simple way of expressing something simple, and testing for duplicated addresses is one of those things that shouldn't be hard.
One of the biggest sources of confusion is the responsibility of a chunk of code. In that example you're not only trying to import data, loop over documents, clean up email addresses, and test for duplicates, but somehow facilitate printing out the results. That's a lot of things going on all at once, and they all have to work perfectly for that chunk of code to be fully operational. There's no way of getting it partially working, and no way of knowing if you're even on the right track.
Always try and break down complex problems into a few simple stages, then chain those stages together as necessary.
Here's how you can define a method to clean up your email addresses:
def address_scrub(address)
address.gsub(/\#target.com/, '').gsub(/\w+/, &:capitalize)
end
Where that can be adjusted as necessary, and presumably tested to ensure it's working correctly, which you can now do indepenedently of the other code.
As for the rest, it looks like this:
require 'set'
# Read in duplicated addresses from a file, clean up with chomp, using a Set
# for fast lookups.
duplicates = Set.new(
File.open("Employee_Information.txt", "r").readlines.map(&:chomp)
)
# Extract addresses from jdoc document array
filtered = jdocs.map do |jdoc|
# Convert to jdoc/address pair
[ jdoc, address_scrub(jdoc["notification_rule"]["contact_method"]["address"]) ]
end.reject do |jdoc, address|
# Remove any that are already in the duplicates list
duplicates.include?(address)
end.map do |jdoc, _|
# Return only the document
jdoc
end
Where that processes jdocs, an array of jdoc structures, and removes duplicates in a series of simple steps.
With the chaining approach you can see what's happening before you add on the next "link", so you can work incrementally towards a solution, adjusting as you go. Any mistakes are fairly easy to catch because you're able to, at any time, inspect the intermediate products of those stages.
I have a decent amount of knowledge about Ruby code, but I do have an issue with my current project. I use Gosu to make 2D games (and once I figure out how, simple 3D games), so I need a resolution.
And this is where my question comes in, why does Ruby keep giving me an error when seeing if the settings file exists? I've been trying to get it working with the file not existing, which keeps giving me the error, "'initialize': No such file or directory # rb_sysopen - ./settings.set (Errno::ENOENT)" which has been annoying me for the last few days. The file gives no issues and actually works as intended when I leave the file created, but I want it to be where if the file gets deleted off of someone's computer, it rebuilds the file and creates a new one using default values.
Here's the area that it keeps crashing at:
settings = []
settings_file_existance = File.file?("settings.set")
if settings_file_existance == true
File.open("settings.set").readlines.each do |line|
settings.push line
end
else
settings_file = File.open("settings.set")
settings_file.write "800"
settings_file.write "\n"
settings_file.write "600"
settings_file.close
end
I have tried looking for fixes on this site, along with many others, but no one so far has been able to help.
You could try this:
settings = []
if File.file?("settings.set")
settings = File.read("settings.set").split("\n")
else
File.open("settings.set", "w") do |file|
file.write "800\n600"
end
end
As a side note, consider that the above code will set settings only if settings.set file exits, otherwise it will remain in an empty array (i.e. []).
If you wish to avoid that, just define settings with the default values, for example:
settings = [800, 600]
Now if settings.set file doesn't exist, then settings will be [800, 600], otherwise it will be overwritten with the values from settings.set.
To avoid writing 800 and 600 twice, you could use settings variable to get the values to be written in the new file, for example:
file.write(settings.join("\n"))
Putting it all together, your code would look like this:
settings = [800, 600]
if File.file?("settings.set")
settings = File.read("settings.set").split("\n")
else
File.open("settings.set", "w") do |file|
file.write(settings.join("\n"))
end
end
In my application, the user must upload a text document, the contents of which are then parsed by the receiving controller action. I've gotten the document to upload successfully, but I'm having trouble reading its contents.
There are several threads on this issue. I've tried more or less everything recommended on these threads, and I'm still unable to resolve the problem.
Here is my code:
file_data = params[:file]
contents = ""
if file_data.respond_to?(:read)
contents = file_data.read
else
if file_data.respond_to?(:path)
File.open(file_data, 'r').each_line do |line|
elts = line.split
#
#
end
end
end
So here are my problems:
file_data doesn't 'respond_to?' either :read or :path. According to some other threads on the topic, if the uploaded file is less than a certain size, it's interpreted as a string and will respond to :read. Otherwise, it should respond to :path. But in my code, it responds to neither.
If I try to take out the if statements and straight away attempt File.open(file_data, 'r'), I get an error saying that the file wasn't found.
Can someone please help me find out what's wrong?
PS, I'm really sorry that this is a redundant question, but I found the other threads unhelpful.
Are you actually storing the file? Because if you are not, of course it can't be found.
First, find out what you're actually getting for file_data by adding debug output of file_data.inspect. It maybe something you don't expect, especially if form isn't set up correctly (i.e. :multipart => true).
Rails should enclose uploaded file in special object providing uniform interface, so that something as simple as this should work:
file_data.read.each_line do |line|
elts = line.split
#
#
end
This is probably simple, but I have spent way too much time trying to figure it out, and am sure someone here will know, so here goes. Please be patient.
Bottom line is that I've got some data that I can't figure out how to loop over.
#Get the data from mongomapper map_reduce
#urls = DistinctUrls.build.find()
puts #urls.count
3
puts #urls.to_json
[{"_id":"http://msn.com","value":3.0},{"_id":"http://yahoo.com","value":12.0},{"_id":"http://google.com","value":2.0}]
#urls.each do |entry|
puts "Here I am" # Never gets printed, not sure why.
puts "url" + entry['_id']
end
What I don't understand is that if I have a count of 3, why it won't enter the loop?
I'm not sure if the mongomapper or map_reduce details matter. I'm putting them here just in case. If it makes sense, I can add the details of the map/reduce if needed.
Thanks for your help.
First you wrote #urls then #url. I think only one of them is correct.
Update: As the documentation says you can iterate over the cursor with each but after the full iteration it will be closed. Maybe this is your case that you've already iterated over it once. Probably the to_json did it.
You can check whether the cursor is closed or not with the following statement:
#urls.closed?
Check this before the iterating part.