My text file contains an email/password list to set up accounts. Once I've used an email and password combo I would like to erase it form the text file.
My text file looks like this:
email,pass
email,pass
etc..
once I've used the e/p combo I would like to delete it from the file:
File.open("yahoo_accounts.txt") do |email|
email.each do |item|
email, password = item.chomp.split(',')
emails << email
passwords << password
emails.zip(passwords) { |name, pass|
browser = Watir::Browser.new :ff
#using the email and pass
File.open("yahoo_accounts.txt", "w") do |out_file|
File.foreach("yahoo_accounts.txt","r") do |line|
out_file.puts line unless line == '#{name},#{pass}'
end
end
browser.close
end
end
The problem occurs when I try to delete them from the file. I get an "browser.rb:382:in `assert_exists': browser was closed (Watir::Exception::Error)",
but that might just be the browser closing.
If all the e/p's are extracted (meaning there is nothing to delete) in the beginning, how can I loop it to keep going, instead of ending in error after the first zip loop?
If the file is not huge, you can read it into an array, modify the array and then write the array to the file, overwriting the previous contents.
Code
def remove_lines(fname, user_name, password)
IO.write(fname, IO.read(fname).gsub(/^#{user_name},#{password}\n/, ''))
end
Example
Let's create some data:
text =<<_
Bubba,boar
Henrietta,vespa
Luigi,pink
Bubba,boar
Luigi,mauve
_
#=> "Bubba,boar\nHenrietta,vespa\nLuigi,pink\nBubba,boar\nLuigi,mauve\n"
and write it to a file:
FName = "tmp"
IO.write(FName, text)
#=> 61
We can confirm what we've written to file:
IO.read(FName)
#=> "Bubba,boar\nHenrietta,vespa\nLuigi,pink\nBubba,boar\nLuigi,mauve\n"
Now remove the lines for user_name/password #=> Bubba/boar:
remove_lines(FName, "Bubba", "boar")
#=> 39
Let's confirm it worked by examining FName:
IO.read(FName)
#=> "Henrietta,vespa\nLuigi,pink\nLuigi,mauve\n"
Postscript
If something goes wrong while writing the array to file, you may lose both the old file and the new file. For that reason, it is often good practice to write the array to a temporary file, then delete the original file, then rename the temporary file to the name of the original file.
You can't write and read to the same file at the same time like that; the first line you write, will clobber the rest of the file so you won't be able to read it anymore.
You should write the changes to a temporary file as you read the lines, then move the temp file to the original file at the end. Something like:
require 'tempfile'
require 'fileutils'
temp_file = Tempfile.new('foo')
begin
File.open("yahoo_accounts.txt", 'r') do |file|
file.each_line do |line|
temp_file.puts line unless line == '#{name},#{pass}'
end
end
temp_file.close
FileUtils.mv(temp_file.path, "yahoo_accounts.txt")
ensure
temp_file.close
temp_file.unlink
end
I am trying to scrape a website and store data in XML using Mechanize and Nokogiri. I didn't set up a Rails project and I am only using Ruby and IRB.
I wrote this method:
def mechanize_club
agent = Mechanize.new
agent.get("http://www.rechercheclub.applipub-fft.fr/rechercheclub/")
form = agent.page.forms.first
form.field_with(:name => 'codeLigue').options[0].select
form.submit
page2 = agent.get('http://www.rechercheclub.applipub-fft.fr/rechercheclub/club.do?codeClub=01670001&millesime=2015')
body = page2.body
html_body = Nokogiri::HTML(body)
codeclub = html_body.search('.form').children("tr:first").children("th:first").to_i
#codeclubs << codeclub
filepath = '/davidgeismar/Documents/codeclubs.xml'
builder = Nokogiri::XML::Builder.new(encoding: 'UTF-8') do |xml|
xml.root {
xml.codeclubs {
#codeclubss.each do |c|
xml.codeclub {
xml.code_ c.code
}
end
}
}
end
puts builder.to_xml
end
My first problem is that I don't know how to test my code.
I call ruby webscraper.rb in my console, the file is treated I think, but it doesn't create an XML file in the specified path.
Then, more specifically I am quite sure this code is wrong as I didn't get a chance to test it.
Basically what I am trying to do is to submit a form several times:
agent = Mechanize.new
agent.get("http://www.rechercheclub.applipub-fft.fr/rechercheclub/")
form = agent.page.forms.first
form.field_with(:name => 'codeLigue').options[0].select
form.submit
I think this code is ok, but I dont want it to only select options[0], I want it to select an option, then scrape all the data I need, then go back to page, then select options[1]... until there are no more options (an iteration I guess).
the file is treated I think, but it doesnt create an xml file in the specified path.
There is nothing in your code that creates a file. You print some output, but don't do anything to open or write a file.
Perhaps you should read the IO and File documentation and review how you are using your filepath variable?
The second problem is that you don't call your method anywhere. Though it's defined and Ruby will see it and parse the method, it has no idea what you want to do with it unless you invoke the method:
def mechanize_club
...
end
mechanize_club()
Im trying to edit a file with a ruby scriopt to add a html tag eg at the beginning of the file and line breaks eg. at the end of each line.
Cannot find a clear example to do this.
Any help will be much appreciated.
Thanks
Here is example code that does what you need(you need to call the modify_file function):
def add_tag(tag, str)
return "<#{tag}>\n#{str}\n</#{tag}>"
end
def modify_file(filename)
content = ""
File.open(filename){|file| content = file.read}
content.gsub(/\n/, "</br>\n")
content = add_tag("html", content)
File.open(filename, "w") {|file| file.write(content)}
end
Here's my code..
require "open-uri"
base_url = "http://en.wikipedia.org/wiki"
(1..5).each do |x|
# sets up the url
full_url = base_url + "/" + x.to_s
# reads the url
read_page = open(full_url).read
# saves the contents to a file and closes it
local_file = "my_copy_of-" + x.to_s + ".html"
file = open(local_file,"w")
file.write(read_page)
file.close
# open a file to store all entrys in
combined_numbers = open("numbers.html", "w")
entrys = open(local_file, "r")
combined_numbers.write(entrys.read)
entrys.close
combined_numbers.close
end
As you can see. It basically scrapes the contents of the wikipedia articles 1 through 5 and then attempts to combine them nto a single file called numbers.html.
It does the first bit right. But when it gets to the second. It only seem's to write in the contents of the fifth article in the loop.
I can't see where im going wrong though. Any help?
You chose the wrong mode when opening your summary file. "w" overwrites existing files while "a" appends to existing files.
So use this to get your code working:
combined_numbers = open("numbers.html", "a")
Otherwise with each pass of the loop the file contents of numbers.html are overwritten with the current article.
Besides I think you should use the contents in read_page to write to numbers.html instead of reading them back in from your freshly written file:
require "open-uri"
(1..5).each do |x|
# set up and read url
url = "http://en.wikipedia.org/wiki/#{x.to_s}"
article = open(url).read
# saves current article to a file
# (only possible with 1.9.x use open too if on 1.8.x)
IO.write("my_copy_of-#{x.to_s}.html", article)
# add current article to summary file
open("numbers.html", "a") do |f|
f.write(article)
end
end
How can I display the output in html? Here is my sample code in ruby:
m = ["a","b", "c"]
m.each do |i|
#html = "<p>"+#{i}+"</p>"
end
I have a separate view file which reads the #html string:
<%= #html.html_safe</p>
My question is, how do i invoke the view file?
First fix typo in your code
#html = "<p>"+#{i}+"</p>"
should read
#html = "<p>#{i}</p>"
Now coming to the point just pass this at the end of your action
erb :<name-of-your-template>
this assumes you are having your erb templates under views directory