How to count paragraphs in a text file - ruby

Using the oliver.txt
write a method called count_paragraphs that counts the number of paragraphs in the text.
In oliver.txt the paragraph delimiter consists of two or more consecutive newline characters, like this: \n\n, \n\n\n, or even \n\n\n\n.
Your method should return either the number of paragraphs or nil.
I have this code but it doesn't work:
def count_paragraphs(some_file)
file_content = open(some_file).read()
count = 0
file_content_split = file_content.split('')
file_content_split.each_index do |index|
count += 1 if file_content_split[index] == "\n" && file_content_split[index + 1] == "\n"
end
return count
end
# test code
p count_paragraphs("oliver.txt")

It's much easier to either count it directly:
file_content.split(/\n\n+/).count
or count the separators and add one:
file_content.scan(/\n\n+/).count + 1

To determine the number of paragraphs there is no need to construct an array and determine its size. One can instead operate on the string directly by creating an enumerator and counting the number of elements it will generate (after some cleaning of the file contents). This can be done with an unconventional (but highly useful) form of the method String#gsub.
Code
def count_paragraphs(fname)
(File.read(fname).gsub(/ +$/,'') << "\n\n").gsub(/\S\n{2,}/).count
end
Examples
First let us construct a text file.
str =<<BITTER_END
Now is the time
for all good
Rubiest to take
a break.
Oh, happy
day.
One for all,
all for one.
Amen!
BITTER_END
# " \n\nNow is the time\nfor all good\nRubiest to take\na break.\n \n \nOh, happy\nday.\n\nOne for all,\nall for one.\n\n \nAmen!\n"
Note the embedded spaces.
FNAME = 'temp'
File.write(FNAME, str)
#=> 128
Now test the method with this file.
count_paragraphs(FNAME)
#=> 4
One more:
count_paragraphs('oliver.txt')
#=> 61
Explanation
The first step is deal with ill-formed text by removing spaces immediately preceding newlines:
File.read(fname).gsub(/ +$/,'')
#=> "\n\nNow is the time\nfor all good\nRubiest to take\na break.\n\n\nOh, happy\nday.\n\nOne for all,\nall for one.\n\n\nAmen!\n"
Next, two newlines are appended so we can identify all paragraphs, including the last, as containing a non-whitespace character followed by two or more newlines.1.
Note that files containing only spaces and newlines are found to contain zero paragraphs.
If the file is known to contain no ill-formed text, the operative line of the method can be simplified to:
(File.read(fname) << "\n\n").gsub(/\S\n{2,}/).count
See Enumerable#count and IO#read. (As File.superclass #=> IO, read is also in instance method of the class File, and seems to be more commonly invoked on that class than on IO.)
Note that String#gsub without a block returns an enumerator (to which Enumerable#count is applied),
Aside: I believe this form of gsub would be more widely used if it merely had a separate name, such as pattern_match. Calling it gsub seems a misnomer, as it has nothing to do with "substitution", "global" or otherwise.
1 I revised my original answer to deal with ill-formed text, and in doing so borrowed #Kimmo's idea of requiring matches to include a non-whitespace character.

How about a loop that memoizes the previous character and a state of being in or outside of a paragraph?
def count_paragraphs(some_file)
paragraphs = 0
in_paragraph = false
previous_char = ""
File.open(some_file).each_char do |char|
if !in_paragraph && char != "\n"
paragraphs += 1
in_paragraph = true
elsif in_paragraph && char == "\n" && previous_char == "\n"
in_paragraph = false
end
previous_char = char
end
paragraphs
rescue
nil
end
This solution does not build any temporary arrays of the full content so you could parse a huge file without it being read into memory. Also, there are no regular expressions.
The rescue was added because of the "Your function should return either the number of paragraphs or nil" which did not give a clear definition of when a nil should be returned. In this case it will be returned if any exception happens, for example if the file isn't found or can't be read, which will raise an exception that will be catched by the rescue.
You don't need an explicit return in Ruby. The return value of the last statement will be used as the method's return value.

Related

index out of string, ruby

Here's a small part of my code, pretty self explanatory, it copies all characters to temp from input and skips spaces.
input = gets.to_s.chomp
temp=String.new
for i in 0..input.length-1
if (input[i]==" ")
next
else
temp[i]=input[i]
end
end
puts "#{temp},END"
gets
However, i tested it with a 'hello world' input, and it should've given me helloworld But i'm getting
8:in '[]=':index 6 out of string(IndexError)
meaning the problem starts while it's skipping the space, for some reason.
keep in mind that i don't get any errors if i put a string that doesn't contain a space
Whenever string manipulation is required, it may be desirable to convert the string to an array of its parts, manipulate those parts and then join them back into a string, but more often as not it is simpler to just operate on the string itself, mainly using methods from the class String. Here you could Kernel#puts the following.
"%s END" % gets.delete(" \n")
#=> "helloworld"
String#delete removes both spaces and the return character ("\n") that Kernel#gets tacks onto the end of the string that is entered. A variant of this is "%s END" % gets.chomp.delete(" ").
Another way would be to puts
"%s END" % gets.gsub(/\s/, '')
#=> "helloworld"
The regular expression /\s/, causes String#gsub to remove all whitespace, which includes both spaces (and tabs) that are entered and the "\n" that gets tacks on to the end of the string.
I guess your error is due to the difference between the string 'hello world' and that you're "rejecting" whitespaces. In such case, for each whitespace in the string being used, the temp will have one less.
You can assign the input[i] when isn't a whitespace to the temp variable in the position temp.size, this way you don't skip indexes.
It could be temp[temp.size] or just modifying temp with +=.
for i in 0...input.size
if input[i] == ' '
next
else
temp[temp.size] = input[i]
end
end
Note you can replace the for loop for each (the Ruby way):
input = 'hello world'
temp = ''
(0...input.size).each do |index|
input[index] == ' ' ? next : temp[temp.size] = input[index]
end
# helloworld
If you want to skip all white spaces from your input and print the output, you can do so with a one-liner:
puts "#{gets.chomp.split.join}, END"
In ruby, you hardly need to write loops using for construct unlike other traditional languages like Java.

Ruby: How to copy a word in a string containing a specific sequence of letters?

I am trying to read in a text file and iterate through every line. If the line contains "_u" then I want to copy that word in that line.
For example:
typedef struct {
reg 1;
reg 2;
} buffer_u;
I want to copy the word buffer_u.
This is what I have so far (everything up to how to copy the word in the string):
f_in = File.open( h_file )
test = h_file.read
text.each_line do |line|
if line.include? "_u"
# copy word
# add to output file
end
end
Thanks in advance for your help!
Don't make it harder than it has to be. If you want to scan a body of text for words that match a criteria, do just that:
text = "
word_u1
something
_u1 foo
bar _u2
another word_u2
typedef struct {
reg 1;
reg 2;
} buffer_u;
"
text.scan(/\w+/).select{ |w| w['_u'] }
# => ["word_u1", "_u1", "_u2", "word_u2", "buffer_u"]
Regex are useful but the more complex ("smarter") they are, they slower they run unless you are very careful to anchor them, as anchors give them hints on where to look. Without those, the engine tries a number of things to determine exactly what you want, and that can really bog down the processing.
I recommend instead simply grabbing the words in the text:
scan(/\w+/)
Then filtering out the ones that match:
select{ |w| w['_u'] }
Using select with a simple sub-string search w['_u'] is extremely fast.
It could probably run faster using split() instead of scan(/\w+/) but you'll have to deal with cleaning up non-word characters.
Note: \w means [a-zA-Z0-9_] so what we generally call a "word" character is actually a "variable" definition for most languages since words generally don't include digits or _.
You can probably reduce your code to:
File.read( h_file ).scan(/\w+/).select{ |w| w['_u'] }
That will return an array of matching words.
Caveat: Using read has scalability issues. If you're concerned about the size of the file being read (which you always should be) then use foreach and iterate over the file line-by-line. You will probably see no change in processing speed.
You can try something like this:
words = []
File.open( h_file ) { |file| file.each_line { |line|
words << line.split.find { |a| a =~ /_u/ }
}}
words.compact!
# => [["buffer_u"]]
puts words
# buffer_u
This regex should catch a word ending with _u
(\w*_u)(?!\w)
The matching group will match a word ending with _u not followed by letters digits or underscores.
If you want _u to appear anywhere in a word use
(\w*_u\w*)
See DEMO here.
This will return all such words in the file, even if there are two or more in a line:
r = /
\w* # match >= 0 word characters
_u # match string
\w* # match >= 0 word characters
/x # extended mode
File.read(fname).scan r
For example:
str = "Cat_u has 9 lives, \n!dog_u has none and \n pig_u_o and cow_u, 3."
fname = 'temp'
File.write(fname, str)
#=> 63
Confirm the file contents:
File.read(fname)
#=> "Cat_u has 9 lives, \n!dog_u has none and \n pig_u_o and cow_u, 3."
Extract strings:
File.read(fname).scan r
#=> ["Cat_u", "dog_u", "pig_u_o", "cow_u"]
It's not difficult to modify this code to return at most one string per line. Simply read the file into an array of lines (or read a line at a time) and execute s = line[r]; arr << s if s for each line, where r is the above regex.

Why doesn't this loop stop?

I'm trying to take the string "xxxyyyzzz" and split it up into an array that groups the same letters. So I want the output to be ["xxx","yyy","zzz"]. I'm not sure why this code keeps on looping. Any suggestions?
def split_up(str)
i = 1
result = []
array = str.split("")
until array == []
if array[i] == array[i-1]
i += 1
else
result << array.shift(i).join("")
end
i = 1
end
result
end
puts split_up("xxxyyyzzz")
The looping is because your until condition never exits. You are incrementing i when the successive characters match, but at the end of the loop you are resetting i to 1.
If you edit this section and add this line:
until array == []
puts i # new line
Then you'll see that i is always 1, and the code keeps printing 1 forever.
Delete the line i = 1 line and you'll get the result you want.
Also, you may be interested in reading about the Ruby string scan method, and pattern matching and capture groups, and using look-ahead and look-behind zero-length assertions, which can match boundaries.
Here is how I would personally accomplish splitting a string at letter boundaries:
"xxxyyyzzz".scan(/(.)(\1*)/).map{|a,b| a+b }
=> ["xxx", "yyy", "zzz"]
The scan method is doing this:
. matches any character e.g. "x", and the parentheses capture this.
\1* matches the previous capture any number of time, e.g. "xx", and the parentheses capture this.
Thus $1 matches the first character "x" and $2 matches all the repeats "xx".
The scan block concatenates the first character and its repeats, so returns "xxx".
As mentioned above, this can be solved using scan like this:
def split_up(string)
repeat_alphabets = /(\w)(\1*)/
string.scan(repeat_alphabets).map do |match|
match[0] << match[1]
end
end
Explanation:
The regular expression matches repeating characters, but due to the construction of the regex matches occur as pairs of the alphabet and remaining repeated instances.
m[0] << m[1] joins the matches to form the required string.
map combines the string into an array and returns the array as it being the last statement.

Ruby - Get file contents with in a separator in an array

I have a file like this:
some content
some oterh
*********************
useful1 text
useful3 text
*********************
some other content
How do I get the content of the file within between two stars line in an array. For example, on processing the above file the content of array should be like this
a=["useful1 text" , "useful2 text"]
A really hack solution is to split the lines on the stars, grab the middle part, and then split that, too:
content.split(/^\*+$/)[1].split(/\s+/).reject(&:empty?)
# => ["useful1","useful3"]
f = File.open('test_doc.txt', 'r')
content = []
f.each_line do |line|
content << line.rstrip unless !!(line =~ /^\*(\*)*\*$/)
end
f.close
The regex pattern /^*(*)*$/ matches strings that contain only asterisks. !!(line =~ /^*(*)*$/) always returns a boolean value. So if the pattern does not match, the string is added to the array.
What about this:
def values_between(array, separator)
array.slice array.index(separator)+1..array.rindex(separator)-1
end
filepath = '/tmp/test.txt'
lines = %w(trash trash separator content content separator trash)
separator = "separator\n"
File.write '/tmp/test.txt', lines.join("\n")
values_between File.readlines('/tmp/test.txt'), "separator\n"
#=> ["content\n", "content\n"]
I'd do it like this:
lines = []
File.foreach('./test.txt') do |li|
lines << li if (li[/^\*{5}/] ... li[/^\*{5}/])
end
lines[1..-2].map(&:strip).select{ |l| l > '' }
# => ["useful1 text", "useful3 text"]
/^\*{5}/ means "A string that starts with and has at least five '*'.
... is one of two uses of .. and ... and, in this use, is commonly called a "flip-flop" operator. It isn't used often in Ruby because most people don't seem to understand it. It's sometimes mistaken for the Range delimiters .. and ....
In this use, Ruby watches for the first test, li[/^\*{5}/] to return true. Once it does, .. or ... will return true until the second condition returns true. In this case we're looking for the same delimiter, so the same test will work, li[/^\*{5}/], and is where the difference between the two versions, .. and ... come into play.
.. will return toggle back to false immediately, whereas ... will wait to look at the next line, which avoids the problem of the first seeing a delimiter and then the second seeing the same line and triggering.
That lets the test assign to lines, which, prior to the [1..-2].map(&:strip).select{ |l| l > '' } looks like:
# => ["*********************\n",
# "\n",
# "useful1 text\n",
# "\n",
# "useful3 text\n",
# "\n",
# "*********************\n"]
[1..-2].map(&:strip).select{ |l| l > '' } cleans that up by slicing the array to remove the first and last elements, strip removes leading and trailing whitespace, effectively getting rid of the trailing newlines and resulting in empty lines and strings containing the desired text. select{ |l| l > '' } picks up the lines that are greater than "empty" lines, i.e., are not empty.
See "When would a Ruby flip-flop be useful?" and its related questions, and "What is a flip-flop operator?" for more information and some background. (Perl programmers use .. and ... often, for just this purpose.)
One warning though: If the file has multiple blocks delimited this way, you'll get the contents of them all. The code I wrote doesn't know how to stop until the end-of-file is reached, so you'll have to figure out how to handle that situation if it could occur.

How do I extract all possible methionine residues to the end from a protein sequence?

I am looking to extract all Methionine residues to the end from a sequence.
In the below sequence:
MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG
Original Amino Acid sequence:
atgtttgaaatcgaagaacatatgaaggattcacaggtggaatacataattggccttcataatatcccattattgaatgcaactatttcagtgaagtgcacaggatttcaaagaactatgaatatgcaaggttgtgctaataaatttatgcaaagacattatgagaatcccctgacgggg
I want to extract from the sequence any M residue to the end, and obtain the following:
- MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG
- MKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG
- MNMQGCANKFMQRHYENPLTG
- MQGCANKFMQRHYENPLTG
- MQRHYENPLTG
With the data I am working with there are cases where there are a lot more "M" residues in the sequence.
The script I currently have is below. This script translates the genomic data first and then works with the amino acid sequences. This does the first two extractions but nothing further.
I have tried to repeat the same scan method after the second scan (See the commented part in the script below) but this just gives me an error:
private method scan called for #<Array:0x7f80884c84b0> No Method Error
I understand I need to make a loop of some kind and have tried, but all in vain. I have also tried matching but I haven't been able to do so - I think that you cannot match overlapping characters a single match method but then again I'm only a beginner...
So here is the script I'm using:
#!/usr/bin/env ruby
require "bio"
def extract_open_reading_frames(input)
file_output = File.new("./output.aa", "w")
input.each_entry do |entry|
i = 1
entry.naseq.translate(1).scan(/M\w*/i) do |orf1|
file_output.puts ">#{entry.definition.to_s} 5\'3\' frame 1:#{i}\n#{orf1}"
i = i + 1
orf1.scan(/.(M\w*)/i) do |orf2|
file_output.puts ">#{entry.definition.to_s} 5\'3\' frame 1:#{i}\n#{orf2}"
i = i + 1
# orf2.scan(/.(M\w*)/i) do |orf3|
# file_output.puts ">#{entry.definition.to_s} 5\'3\' frame 1:#{i}\n#{orf3}"
# i = i + 1
# end
end
end
end
file_output.close
end
biofastafile = Bio::FlatFile.new(Bio::FastaFormat, ARGF)
extract_open_reading_frames(biofastafile)
The script has to be in Ruby since this is part of a much longer script that is in Ruby.
You can do:
str = "MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG"
str.scan(/(?=(M.*))./).flatten
#=> ["MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG", MKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG", "MNMQGCANKFMQRHYENPLTG", "MQGCANKFMQRHYENPLTG", "MQRHYENPLTG"]
This works by capturing loookaheads starting with M and advancing one char at a time.
str = "MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG"
pos = 0
while pos < str.size
if md = str.match(/M.*/, pos)
puts md[0]
pos = md.offset(0)[0] + 1
else
break
end
end
--output:--
MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG
MKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG
MNMQGCANKFMQRHYENPLTG
MQGCANKFMQRHYENPLTG
MQRHYENPLTG
md -- stands for the MatchData object.
match() -- returns nil if there is no match, the second argument is the start position of the search.
md[0] -- is the whole match (md[1] would be the first parenthesized group, etc.).
md.offset(n) -- returns an array containing the beginning and ending position in the string of md[n].
Running the program on the string "MMMM" produces the output:
MMMM
MMM
MM
M
I have also tried matching but I haven't been able to do so - I think
that you cannot match overlapping characters a single match method but
then again I'm only a beginner...
Yes, that's true. String#scan will not find overlapping matches. After scan finds a match, the search continues from the end of the match. Perl has some ways to make regexes back-up, I don't know whether Ruby has those.
Edit:
For Ruby 1.8.7:
str = "MFEIEEHMKDSQVEYIIGLHNIPLLNATISVKCTGFQRTMNMQGCANKFMQRHYENPLTG"
pos = 0
while true
str = str[pos..-1]
if md = str.match(/M.*/)
puts md[0]
pos = md.offset(0)[0] + 1
else
break
end
end

Resources