Is it possible to write to a file while looping in Ruby? My code looks something like this:
navigator.rb
def launch_process
while obj.present?
return something while something_else
end
end
app.rb # launcher
navigator = Navigator.new(args)
var = navigator.launch_process
$file = File.open("output.csv", "a+")
open($file, 'a+') { |file| file.write(var) } # won't work
The idea is to update the CSV file with data that is being gradually returned by the Navigator Object, launch_process method.
Here's an example :
def launch_process(io)
10.times do |i|
sleep 1
io.puts i
## Uncomment this line if you want to update the file during every iteration
# io.flush
end
end
f = File.open('output.csv','a+')
launch_process(f)
f.close
If you post more code it would be easier to adapt this loop to your example.
Related
I've been trying to scrape data using Nokogiri and HTTParty and can scrape data off a website successfully and print it to the console but I can't work out how to output the data to an xml file in the repo.
Right now the code looks like this:
class Scraper
attr_accessor :parse_page
def initialize
doc = HTTParty.get("https://store.nike.com/gb/en_gb/pw/mens-nikeid-lifestyle-shoes/1k9Z7puZoneZoi3?ref=https%253A%252F%252Fwww.google.com%252F")
#parse_page ||= Nokogiri::HTML(doc)
end
def get_names
item_container.css(".product-display-name").css("p").children.map { |name| name.text }.compact
end
def get_prices
item_container.css(".product-price").css("span.local").children.map { |price| price.text }.compact
end
private
def item_container
parse_page.css(".grid-item-info")
end
scraper = Scraper.new
names = scraper.get_names
prices = scraper.get_prices
(0...prices.size).each do |index|
puts " - - - Index #{index + 1} - - -"
puts "Name: #{names[index]} | Price: #{prices[index]}"
end
end
I've tried changing the .each method to include a File.write() but all it ever does is write the last line of the output into the xml file. I would appreciate any insight as to how to parse the data correctly, I am new to scraping.
I've tried changing the .each method to include a File.write() but all it ever does is write the last line of the output into the xml file.
Is the File.write method inside the each loop? I guess what's happening here is You are overwriting the file on every iteration and that's why you are seeing only the last line.
Try putting the each loop inside the block of the File.open method like:
File.open(yourfile, 'w') do |file|
(0...prices.size).each do |index|
file.write("your text")
end
end
I also recommend reading about the Nokogiri::XML::Builder and then saving it's output to the file.
I want to export sc files from SpaceEngine, then for each file, create a new file with the same name but with the extension .txt.
Here is my code:
require 'fileutils'
require 'IO/console'
puts "Make sure your export folder is clear of everything but the files you want to turn into Object text files."
puts "Starting Process"
i = 0
Dir.foreach('C:\SpaceEngine\export') do |item|
next if item == '.' or item == '..'
i = i + 1
name = File.basename(item, ".*")
current = File.new("#{name}.txt", "w");
current.close
end
sleep 2
I have the latter part already, but I can't get it to read the original files one by one, and then only put certain things from the original into the new file.
# test.sc
# assume this is your test data
this has foo
this does not
this also has foo
this has some other stuff
this is the last line which has foo
blah
blah blah 💩
# filejunk.rb
# you need to write a method that handles the data inside the file you want to
# modify, change, replace etc. but for example
def replace_file_data(filename)
lines = File.readlines
lines.select{|l| l.include?'foo'} #assumes you only want lines with 'foo' in them
end
Dir.glob('C:\SpaceEngine\export\*.sc').each_with_index do |filename, i|
i += 1
name = File.basename(filename, ".*")
current = File.new("#{name}.txt", "w") {|f| f.write replace_file_data(filename) }
current.close
end
I am trying find a certain piece of code in a .rb file, once found I want to cut it from the current file and then paste it into another existing file. So an an example:
file1.rb has the following:
RSpec.describe 'Get Test Data' do
it "should get test data for build" do |example|
log_start_test("#{example.description}")
get_test_data
log_complete_test("#{example.description}")
end
end
I want to find it "should get test data for build" do |example| and then cut this piece of code:
it "should get test data for build" do |example|
log_start_test("#{example.description}")
get_test_data
log_complete_test("#{example.description}")
end
and paste it another file.
So far I have been able to find the desired string using something like this:
File.open("#{Dir.pwd}/spec/api/test_data_search_spec.rb") do |f|
f.each_line do |line|
if line =~ /do |example|/
puts "Found root #{line}"
end
end
end
Just not able to figure out the exact regular expression to find the required block and then how do i do a cut from a file and paste into another one? Any ideas would be great.
regular expressions are not suited to parse code.
you could use method_source as an existing solution to the problem.
Thanks #phoet. That would have worked for specific methods but I was looking more for moving rspec example code block around. But here is what I ended up using as an example:
def move_example_block(example_description, source_file,destination_file)
lines = File.readlines("#{Dir.pwd}/spec/api/#{source_file}_spec.rb")
desired_block = lines.join[/it "should get test data for build" do(.*)end/m]
temp = desired_block.freeze
puts temp
filename = "#{Dir.pwd}/spec/api/#{source_file}_spec.rb"
text = File.read(filename)
puts = text.gsub(/it "#{example_description}" do(.*)end/m, "end")
File.open(filename, "w") { |file| file << puts }
filename = "#{Dir.pwd}/spec/api/#{destination_file}_spec.rb"
text = File.read(filename)
puts = text.gsub(/end/, temp)
File.open(filename, "w") { |file| file << puts }
end
I'm writing a test for one of my classes which has the following constructor:
def initialize(filepath)
#transactions = []
File.open(filepath).each do |line|
next if $. == 1
elements = line.split(/\t/).map { |e| e.strip }
transaction = Transaction.new(elements[0], Integer(1))
#transactions << transaction
end
end
I'd like to test this by using a fake file, not a fixture. So I wrote the following spec:
it "should read a file and create transactions" do
filepath = "path/to/file"
mock_file = double(File)
expect(File).to receive(:open).with(filepath).and_return(mock_file)
expect(mock_file).to receive(:each).with(no_args()).and_yield("phrase\tvalue\n").and_yield("yo\t2\n")
filereader = FileReader.new(filepath)
filereader.transactions.should_not be_nil
end
Unfortunately this fails because I'm relying on $. to equal 1 and increment on every line and for some reason that doesn't happen during the test. How can I ensure that it does?
Global variables make code hard to test. You could use each_with_index:
File.open(filepath) do |file|
file.each_with_index do |line, index|
next if index == 0 # zero based
# ...
end
end
But it looks like you're parsing a CSV file with a header line. Therefore I'd use Ruby's CSV library:
require 'csv'
CSV.foreach(filepath, col_sep: "\t", headers: true, converters: :numeric) do |row|
#transactions << Transaction.new(row['phrase'], row['value'])
end
You can (and should) use IO#each_line together with Enumerable#each_with_index which will look like:
File.open(filepath).each_line.each_with_index do |line, i|
next if i == 1
# …
end
Or you can drop the first line, and work with others:
File.open(filepath).each_line.drop(1).each do |line|
# …
end
If you don't want to mess around with mocking File for each test you can try FakeFS which implements an in memory file system based on StringIO that will clean up automatically after your tests.
This way your test's don't need to change if your implementation changes.
require 'fakefs/spec_helpers'
describe "FileReader" do
include FakeFS::SpecHelpers
def stub_file file, content
FileUtils.mkdir_p File.dirname(file)
File.open( file, 'w' ){|f| f.write( content ); }
end
it "should read a file and create transactions" do
file_path = "path/to/file"
stub_file file_path, "phrase\tvalue\nyo\t2\n"
filereader = FileReader.new(file_path)
expect( filereader.transactions ).to_not be_nil
end
end
Be warned: this is an implementation of most of the file access in Ruby, passing it back onto the original method where possible. If you are doing anything advanced with files you may start running into bugs in the FakeFS implementation. I got stuck with some binary file byte read/write operations which weren't implemented in FakeFS quite how Ruby implemented them.
I am new to ruby and make a lot of mistakes, so I hope people who are experienced in ruby can share a bit of knowledge.
I can't figure out how I can make ruby save the text into a txt file that a method all writes.
class generator
def all
puts "i want to save this text into a txt file"
end
end
new_gen = generator.new
new_gen.all
my_file = File.new("Story.txt", "a+")
my_file.puts("all")
my_file.puts("\n")
my_file.close
I tried everything, but the txt file either has the "all" in it or it's completely blank. Any ideas? I also tried my_file.puts(all) and my_file.puts(new_gen.all).
If you want Generator to do the writing you could pass it an IO object.
class Generator
def initialize(io)
#io = io
end
def all
#io.puts "i want to save this text into a txt file"
end
end
# write to STDOUT
gen = Generator.new(STDOUT)
gen.all
# write to file
File.open("Story.txt", "a+") do |file|
gen = Generator.new(file)
gen.all
end
You method should simply return a string. Puts displays the string, does not return it. So change the class to:
class generator
def all
"i want to save this text into a txt file" # optionally add a return
end
end
new_gen = generator.new
new_gen.all
Then use the last version you tried: my_file.puts(new_gen.all)
Try this:
class Generator
def all
"i want to save this text into a txt file"
end
end
gen = Generator.new
f = File.new("Story.txt", "a+")
f.puts gen.all
f.close