The below with hardcoded file path works
File.open('/cm/Chef/instnst.loc', 'w') do |f2|
# use "\n" for two lines of text
f2.puts "inventory_loc=#{orainsloc}\ninst_group=dbas"
end
But when I change it to read from a JSON parameter, it errors. What am I doing wrong?
File.open('#{node['installParams']['InstallFilesLocation']}/instnst.loc', 'w') do |f2|
# use "\n" for two lines of text
f2.puts "inventory_loc=#{orainsloc}\ninst_group=dbas"
end
#{} formatting has be inside double quotes like "#{node['installParams']['InstallFilesLocation']}/instnst.loc".
Related
I have a file formatted by lines like this (I know it's a terrible format, I didn't write it):
id: 12345 synset: word1,word2
I want to read the entire file and check to see if every line is correct without having to look line by line.
I've looked into File and Regex, but couldn't find what I need. I tried to use File.read to read the entire file all at once, then use m modifier for regex to check multiple lines, but it's not working the way I anticipated (perhaps it's not what I need).
p.s. Ruby newbie :)
Assuming your file always ends with a newline, this should work:
/^(id: \d+ synset: \w+,\w+\n)+$/m
The full ruby:
content = ''
File.open('myfile.txt', 'r') { |f| content = f.read }
puts 'file is valid!' if content =~ /^(id: \d+ synset: \w+,\w+\n)+$/m
You can use this regex to check each line of the file: ^id:\s*\d+\s+synset:\s*(?:\w+,)*\w+$. You can try the following code, but I don't know any Ruby, I just searched and tested a little. It might work.
line_num = 0
text = File.open('file.txt').read
text.each_line do |line|
line_num += 1
if !/^id:\s*\d+\s+synset:\s*(?:\w+,)*\w+$/.match(line)
print "Line #{line_num} is incorrect"
end
end
I have string values which I am writing to csv file in the form of array as -
output = "This is a, ruby output"
CSV.open("output/abc.csv", "a+") do |csv|
csv << [output]
end
When I check my file abc.csv the row added has quotation marks (") at the start and end of the field. How can I get rid of it?
File output as ---
"This is a, ruby output"
So far I've tried tr or slice before saving to csv, but it seems writing is causing it.
If you get rid of the quotes then your output is no longer CSV. The CSV class can be instructed to use a different delimiter and will only quote if that delimiter is included in the input. For example:
require 'csv'
output = "This is a, ruby output"
File.open("output/abc.csv", "a+") do |io|
csv = CSV.new(io, col_sep: '^')
csv << [output, "the end"]
end
Output:
This is a, ruby output^the end
I'm trying to search through a specified string and assign the results to an array.
Opening and writing to "input.txt" and "ms3.txt" files works fine. putting a normal string like reassign << "hello" in works fine its just when i use line.grep and the regex following it prints nothing to the console or the ms3 file it doesn't even throw up any errors
i've also tried a search and replace: reassign << line.gsub(/[abc]/, '£')
Here's the code
# encoding: utf-8
#!/usr/bin/ruby
file = File.open("input.txt", "w+")
reassign = []
file.each_line do |line|
reassign << line.grep(/[abc]/)
end
new_file = File.open("ms3.txt", "w+")
new_file.puts(reassign)
new_file.close
Your code can be streamlined a lot to make it more Ruby-like, and to make it behave better:
# encoding: utf-8
#!/usr/bin/ruby
file = File.open("input.txt", "w+")
reassign = []
file.each_line do |line|
reassign << line.grep(/[abc]/)
end
new_file = File.open("ms3.txt", "w+")
new_file.puts(reassign)
new_file.close
#! lines have to be first, so reverse the encoding and "slash-bang" lines.
open takes a block, which will allow Ruby to automatically close the file when the block exits. This is a very powerful, and smart thing to do, as it keeps your file I/O environment clean. It's possible, and often a problem as in your code, to open files but never close them, which can exhaust all the file handles on a machine if it's done in a loop, causing all apps to fail. Using the block form will avoid this.
The IO class has foreach, which makes it simple to iterate over the lines of a file. Take advantage of it instead of opening the file then using each_line, because it simplifies your code.
Here's how I'd initially write your code:
#!/usr/bin/ruby
# encoding: utf-8
reassign = []
File.foreach("input.txt") do |line|
reassign << line[/[abc]/]
end
File.write("ms3.txt", reassign.join("\n"))
But, after refactoring it I'd end up with:
#!/usr/bin/ruby
# encoding: utf-8
File.open('ms3.txt', 'w') do |fo|
fo.puts File.foreach('input.txt').grep(/[abc/])
end
The open opens the output file using a block to take advantage of automatically closing the file when the block exits.
foreach is an iterator, and normally is used with a block to pass each line read into the block. Instead, I'm letting grep read all the lines found and search for the pattern.
Any lines found by grep that match the pattern are returned as an array to puts which will iterate over them, appending "\n" to the end of each.
fo.puts directs the output of puts to the output file.
end causes the block to exit, which causes open to close the file.
That's untested but looks correct.
There are several issues with your code:
You open "input.txt" with open mode "w+". According to the documentation, this truncates your file to zero length. An empty file doesn't contain any lines and therefore, file.each_line doesn't invoke the block.
If you want to read from the file, use "r", which is the default:
file = File.open("input.txt")
You don't close file. Use the block form which closes the file automatically:
File.open("input.txt") do |file|
# ...
end
line is a String and there's no String#grep method. But since File includes Enumerable, you can use Enumerable#grep instead:
reassign = file.grep(/[abc]/)
A complete example:
File.open("input.txt") do |file|
reassign = file.grep(/[abc]/)
File.open("ms3.txt", "w+") do |new_file|
new_file.puts(reassign)
end
end
I was trying to use the following code to read lines from a file. But when reading a file, the contents are all in one line:
line_num=0
File.open('xxx.txt').each do |line|
print "#{line_num += 1} #{line}"
end
But this file prints each line separately.
I have to use stdin, like ruby my_prog.rb < file.txt, where I can't assume what the line-ending character is that the file uses. How can I handle it?
Ruby does have a method for this:
File.readlines('foo').each do |line|
puts(line)
end
http://ruby-doc.org/core-1.9.3/IO.html#method-c-readlines
File.foreach(filename).with_index do |line, line_num|
puts "#{line_num}: #{line}"
end
This will execute the given block for each line in the file without slurping the entire file into memory. See: IO::foreach.
I believe my answer covers your new concerns about handling any type of line endings since both "\r\n" and "\r" are converted to Linux standard "\n" before parsing the lines.
To support the "\r" EOL character along with the regular "\n", and "\r\n" from Windows, here's what I would do:
line_num=0
text=File.open('xxx.txt').read
text.gsub!(/\r\n?/, "\n")
text.each_line do |line|
print "#{line_num += 1} #{line}"
end
Of course this could be a bad idea on very large files since it means loading the whole file into memory.
Your first file has Mac Classic line endings (that’s "\r" instead of the usual "\n"). Open it with
File.open('foo').each(sep="\r") do |line|
to specify the line endings.
I'm partial to the following approach for files that have headers:
File.open(file, "r") do |fh|
header = fh.readline
# Process the header
while(line = fh.gets) != nil
#do stuff
end
end
This allows you to process a header line (or lines) differently than the content lines.
It is because of the endlines in each lines.
Use the chomp method in ruby to delete the endline '\n' or 'r' at the end.
line_num=0
File.open('xxx.txt').each do |line|
print "#{line_num += 1} #{line.chomp}"
end
how about gets ?
myFile=File.open("paths_to_file","r")
while(line=myFile.gets)
//do stuff with line
end
Don't forget that if you are concerned about reading in a file that might have huge lines that could swamp your RAM during runtime, you can always read the file piece-meal. See "Why slurping a file is bad".
File.open('file_path', 'rb') do |io|
while chunk = io.read(16 * 1024) do
something_with_the chunk
# like stream it across a network
# or write it to another file:
# other_io.write chunk
end
end
What's the best (most efficient) way to parse a tab-delimited file in Ruby?
The Ruby CSV library lets you specify the field delimiter. Ruby 1.9 uses FasterCSV. Something like this would work:
require "csv"
parsed_file = CSV.read("path-to-file.csv", col_sep: "\t")
The rules for TSV are actually a bit different from CSV. The main difference is that CSV has provisions for sticking a comma inside a field and then using quotation characters and escaping quotes inside a field. I wrote a quick example to show how the simple response fails:
require 'csv'
line = 'boogie\ttime\tis "now"'
begin
line = CSV.parse_line(line, col_sep: "\t")
puts "parsed correctly"
rescue CSV::MalformedCSVError
puts "failed to parse line"
end
begin
line = CSV.parse_line(line, col_sep: "\t", quote_char: "Ƃ")
puts "parsed correctly with random quote char"
rescue CSV::MalformedCSVError
puts "failed to parse line with random quote char"
end
#Output:
# failed to parse line
# parsed correctly with random quote char
If you want to use the CSV library you could used a random quote character that you don't expect to see if your file (the example shows this), but you could also use a simpler methodology like the StrictTsv class shown below to get the same effect without having to worry about field quotations.
# The main parse method is mostly borrowed from a tweet by #JEG2
class StrictTsv
attr_reader :filepath
def initialize(filepath)
#filepath = filepath
end
def parse
open(filepath) do |f|
headers = f.gets.strip.split("\t")
f.each do |line|
fields = Hash[headers.zip(line.split("\t"))]
yield fields
end
end
end
end
# Example Usage
tsv = Vendor::StrictTsv.new("your_file.tsv")
tsv.parse do |row|
puts row['named field']
end
The choice of using the CSV library or something more strict just depends on who is sending you the file and whether they are expecting to adhere to the strict TSV standard.
Details about the TSV standard can be found at http://en.wikipedia.org/wiki/Tab-separated_values
There are actually two different kinds of TSV files.
TSV files that are actually CSV files with a delimiter set to Tab. This is something you'll get when you e.g. save an Excel spreadsheet as "UTF-16 Unicode Text". Such files use CSV quoting rules, which means that fields may contain tabs and newlines, as long as they are quoted, and literal double quotes are written twice. The easiest way to parse everything correctly is to use the csv gem:
use 'csv'
parsed = CSV.read("file.tsv", col_sep: "\t")
TSV files conforming to the IANA standard. Tabs and newlines are not allowed as field values, and there is no quoting whatsoever. This is something you will get when you e.g. select a whole Excel spreadsheet and paste it into a text file (beware: it will get messed up if some cells do contain tabs or newlines). Such TSV files can be easily parsed line-by-line with a simple line.rstrip.split("\t", -1) (note -1, which prevents split from removing empty trailing fields). If you want to use the csv gem, simply set quote_char to nil:
use 'csv'
parsed = CSV.read("file.tsv", col_sep: "\t", quote_char: nil)
I like mmmries answer. HOWEVER, I hate the way that ruby strips off any empty values off of the end of a split. It isn't stripping off the newline at the end of the lines, either.
Also, I had a file with potential newlines within a field. So, I rewrote his 'parse' as follows:
def parse
open(filepath) do |f|
headers = f.gets.strip.split("\t")
f.each do |line|
myline=line
while myline.scan(/\t/).count != headers.count-1
myline+=f.gets
end
fields = Hash[headers.zip(myline.chomp.split("\t",headers.count))]
yield fields
end
end
end
This concatenates any lines as necessary to get a full line of data, and always returns the full set of data (without potential nil entries at the end).