I have a text file structured like this:
----------------------------------------
----------------------------------------
All Events Information: 1 : Timestamp: 08.11.2017 01:46:10
Message: DEBUG, Thread ID:12,63645705970991641611 - Request: POST https://some.url.com:8080/api/Customers/
{"json":"someValue"}
For every Message that contains the request URL "/api/Customers" I want to output the following line {"json":"someValue"}. If it doesn't, it should skip the whole block.
Here's what I have so far. It puts the line correctly, but I want to get the following line:
File.open("my_file.log", "r").each_line do |line|
if line.include? "POST https://some.url.com:8080/api/Customers/\r"
p line
end
end
Here is a very simple solution:
def test
show_next_line = false
File.open("/home/gergra/code/test/my_file.log", "r").each_line do |line|
if show_next_line
show_next_line = false
puts line
end
if line.include? "POST https://some.url.com:8080/api/Customers/"
show_next_line = true
end
end
end
Related
I have some logs that need analyzing to check if the logs do not have abnormalities and are incorrect form so to speak.
I have generated a CSV file for it:
"timestamp","source","message
"2021-10-18T09:12:29.000Z","Storage","Storage apache: [18/Oct/2021:09:12:29 +0800] 10.102.141.82 - GET /deviceManager/rest/
"2021-10-18T09:12:29.000Z","Storage","Storage apache: [18/Oct/2021:09:12:29 +0800] 10.102.141.82 - GET /deviceManager/rest/
"2021-10-18T09:12:29.000Z","Storage","Storage apache: [18/Oct/2021:09:12:29 +0800] 10.102.141.82 - GET /deviceManager/rest/
"2021-10-18T09:12:29.000Z","Storage","Storage apache: [18/Oct/2021:09:12:29 +0800] 10.102.141.82 - GET /deviceManager/rest/
I use the CSV gem to parse/read this file and use an RSpec test to expect some value/text/time format etc. I have written the code below. It takes the rows from 8 to 12 for example and I want to expect a text called "Huawei" f.e in those rows.
RSpec.describe "Log parsing" do
it 'returns the source' do
table = CSV.read("Messages_result.csv")
puts arr = table.values_at(8..12)
arr.each do |rows|
expect(rows).to include('Huawei')
end
end
end
The problem I am getting is it always executes the expect for the first line but I want to parse/iterate through the whole CSV file and should as well show me for each line a result. My expect message will change of course but I just want to check first for a basic text like Huawei. Can somebody please show what I am doing wrong since each do should theoretically go through the complete rows and throw an expectation for each?
i want to parse/iterate through the whole csv file and should as well show me for each line a result.
That's not possible, see Add a configuration option to continue on failure.
That said, by modifying a little the code, you can make Rspec check the whole file and display all the rows that fail your expect:
RSpec.describe "Log parsing" do
CSV.foreach("Messages_result.csv", :headers => true) do |row|
it 'returns the source' do
expect(row.to_h.values).to include("Huawei")
end
end
end
outputs:
FF
Failures:
1) Log parsing returns the source
Failure/Error: expect(row.to_h.values).to include("Huawei")
expected ["2021-10-18T09:10:29.000Z", "Storage", "Storage apache: [18/Oct/2021:09:10:29 +0800] 10.102.141.82 - GET /deviceManager/rest/"] to include "Huawei"
# ./Messages_result.rb:10:in `block (3 levels) in <main>'
2) Log parsing returns the source
Failure/Error: expect(row.to_h.values).to include("Huawei")
expected ["2021-10-18T09:11:24.000Z", "Storage", "Storage apache: [18/Oct/2021:09:11:24 +0800] 10.102.141.82 -...../license/feature HTTP/1.1 python-requests/2.21.0 - - application/json - / gzip, deflate 200 49 0"] to include "Huawei"
# ./Messages_result.rb:10:in `block (3 levels) in <main>'
3) ...
If you really want to display a message for each line of the CSV then you don't have any other choice than print it yourself. For example:
# get the output stream that Rspec is currently using
ostream = RSpec.configure { |c| c.output_stream }
# define a few colorization helpers
if ostream.tty?
def red str; "\e[31m#{str}\e[0m"; end
def green str; "\e[32m#{str}\e[0m"; end
else
def red str; str; end
def green str; str; end
end
RSpec.describe "Log parsing" do
it 'returns the source' do
ostream.puts
ostream.puts " -) Log parsing returns the source - details"
expected = "Huawei"
success = true
CSV.foreach("Messages_result.csv", :headers => true) do |row|
values = row.to_h.values
detail = "expected #{values} to include #{expected.inspect}"
ostream.print ' ' * 5
if values.include?(expected)
ostream.puts green("PASSED: #{detail}")
else
ostream.puts red("FAILED: #{detail}")
success = false
end
end
ostream.puts
ostream.flush
expect(success).to be(true)
end
end
I want to add newline character below.
But the result is wrong.
Teach me what is wrong.
test.txt(before)
------------------
2014-09
2014-10
2014-11
------------------
test.txt(after)
------------------
2014-09
2014-10
2014-11
------------------
I make a ruby script below, but the result is wrong.
f = File.open("test.txt","r+")
f.each{|line|
if line.include?("2014-10")
f.puts nil
end
}
f.close
the result
------------------
2014-09
2014-10
014-11
------------------
To solve your problem, the easiest way is to create a new file to output your new text into. To do you'll need to open the input file and the output file and iterate each line of the file check the condition and put desired line into the output file.
Example
require 'fileutils'
File.open("text-output.txt", "w") do |output|
File.foreach("text.txt") do |line|
if line.include?("2014-10")
output.puts line + "\n"
else
output.puts line
end
end
end
FileUtils.mv("text-output.txt", "text.txt")
Easy way
File.write(f = "text.txt", File.read(f).gsub(/2014-10/,"2014-10\n"))
Reading and writing a file at the same time can get messy, same thing with other data structures like arrays. You should build a new file as you go along.
Some notes:
you should use the block form of File.open because it will stop you from forgetting to call f.close
puts nil is the same as puts without arguments
single quotes are preferred over double quotes when you don’t need string interpolation
you should use do ... end instead of { ... } for multi-line blocks
File.open(...).each can be replaced with File.foreach
the intermediate result can be stored in a StringIO object which will respond to puts etc.
Example:
require 'stringio'
file = 'test.txt'
output = StringIO.new
File.foreach(file) do |line|
if line.include? '2014-10'
output.puts
else
output << line
end
end
output.rewind
File.open(file, 'w') do |f|
f.write output.read
end
i read multipe file and i try to get data in yaml file, but i dont know why i get nothing in my yaml file .
Do you have an idea where i can make a mistake ?
a = array.size
i = 0
array.each do |f|
while i < a
puts array[i]
output = File.new('/home/zyriuse/documents/Ruby-On-Rails/script/Api_BK/licence.yml', 'w')
File.readlines(f).each do |line|
output.puts line
output.puts line.to_yaml
#output.puts YAML::dump(line)
end
i += 1
end
end
There's two problems...
You are initializing i to zero too early... when you process the
first file 'f' you process JUST that first file as many times as you
have files in the array, but for all following files i is now always >= a so you're not doing anything with them.
You are doing File.new in every iteration of 'f' so you are wiping out your last iteration.
This might work better...
output = File.new('licence.yml', 'w')
array.each do |f|
puts f
File.readlines(f).each do |line|
output.puts line
output.puts line.to_yaml
end
end
I'm trying to parse the first 5 lines of a remote CSV file. However, when I do, it raises Errno::ENOENT exception, and says:
No such file or directory - [file contents] (with [file contents] being a dump of the CSV contents
Here's my code:
def preview
#csv = []
open('http://example.com/spreadsheet.csv') do |file|
CSV.foreach(file.read, :headers => true) do |row|
n += 1
#csv << row
if n == 5
return #csv
end
end
end
end
The above code is built from what I've seen others use on Stack Overflow, but I can't get it to work.
If I remove the read method from the file, it raises a TypeError exception, saying:
can't convert StringIO into String
Is there something I'm missing?
Foreach expects a filename. Try parse.each
You could manually pass each line to CSV for parsing:
require 'open-uri'
require 'csv'
def preview(file_url)
#csv = []
open(file_url).each_with_index do |line, i|
next if i == 0 #Ignore headers
#csv << CSV.parse(line)
if i == 5
return #csv
end
end
end
puts preview('http://www.ferc.gov/docs-filing/eqr/soft-tools/sample-csv/contract.txt')
Can anyone provide some clues as to why these two pieces of code are not equivalent? My only thought is that the .open in the latter code portion is not in a block and the file handle is left open.
File.open(file) do |io|
io.each_line do |line|
body_string << line
end
end
and this one
File.open(file).each_line {|line| body_string << line}
Thanks.
See IO class's API.
If File.open is given a block , it opens the file, executes the block, then closes the file.
If it is not given a block, it returns an object representing the file (just like File::new), so it might still be opened.
File test.rb:
def test1
body_string = []
[ File.open(ARGV[0]).each_line { |line| body_string << line }, body_string ]
end
def test2
body_string = []
[ File.open(ARGV[0]) do |io|
io.each_line { |line| body_string << line }
end, body_string ]
end
puts(test1.inspect)
puts(test2.inspect)
File f:
hello!
output of ruby test.rb f:
[#<File:f>, ["hello!\n"]]
[#<File:f (closed)>, ["hello!\n"]]
The only difference is that, when File.open is given a block, it automatically closes the file handle.
HTH