how do i make my code read random lines 37 different times? - ruby

def pick_random_line
chosen_line = nil
File.foreach("id'sForCascade.txt").each_with_index do |line, id|
chosen_line = line if rand < 1.0/(id+1)
end
return chosen_line
end`enter code here
Hey, i'm trying to make that code pick 37 different lines. So how would I do that i'm stuck and confused.

Assuming you don't want the same line to repeat more than once, I would do it in one line like this:
File.read("test.txt").split("\n").shuffle.first(37)
File.read("test.txt") reads the entire file.
split("\n") splits the file to lines based on the \n delimiter (I assume your file is textual and have lines separated by new line character).
shuffle is a very convenient method of Array that shuffles the lines randomly. You can read about it here:
http://docs.ruby-lang.org/en/2.0.0/Array.html#method-i-shuffle
Finally, first(37) gives you the first 37 lines out of the shuffled array. These are guaranteed to be random from the shuffle operation.

You can do something like this:
input_lines = File.foreach("test.txt").map(&:to_s)
output_lines = []
37.times do
output_lines << input_lines.delete_at(rand(input_lines.length))
end
puts output_lines
This will ensure that you aren't grabbing duplicate lines and you don't need to do any fancy checking.
However, if your file is less than 37 lines this may cause a problem, it also assumes that your file exists.
EDIT:
What is happening is the rand call is now changing the range on which it is called based on the size of the input lines. And since you are deleting at an index when you take the line out, the length shrinks and you do not risk duplicating lines.

If you want to save relatively few lines from a large file, reading the entire file into an array (and then randomly selecting lines) could be costly. It might be better to count the number of lines in the file, randomly select line offsets and then save the lines at those offsets to an array. This approach is no more difficult to implement than the former one, but makes the method more robust, even if the files in the current application are not overly large.1
Suppose your filename were given by FName. Here are three ways to count the numbers of lines in the file:
Count lines, literally
cnt = File.foreach(FName).reduce(0) { |c,_| c+1 }
Use $.
File.foreach(FName) {}
cnt = $.
On Unix-family computers, shell-out to the operating system
cnt = %x{wc -l #{FName}}.split.first.to_ii
The third option is very fast.
Random offsets (base 1) for n lines to be saved could be computed as follows:
lines = (1..cnt).to_a.sample(n).sort
Saving the lines at those offsets to an array is straightforward; for example:
File.foreach(FName).with_object([]) do |line,a|
if lines.first == $.
a << line
lines.shift
break a if lines.empty?
end
end
Note that $. #=> 1 after the first line is first line is read, and $. is incremented by 1 after each successive line is read. (Hence base 1 for line offsets.)
1 Moreover, many programmers, not just Rubiests, are repelled by the idea of amassing large numbers of anything and then discarding all but a few.

Related

Multiple sequence alignment. Convert multi-line format to single-line format?

I have a multiple sequence alignment file in which the lines from the different sequences are interspersed, as in the format outputed by clustal and other popular multiple sequence alignment tools. It looks like this:
TGFb3_human_used_for_docking ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPY
tr|B3KVH9|B3KVH9_HUMAN ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPY
tr|G3UBH9|G3UBH9_LOXAF ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPY
tr|G3WTJ4|G3WTJ4_SARHA ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPY
TGFb3_human_used_for_docking LRSADTTHST-
tr|B3KVH9|B3KVH9_HUMAN LRSADTTHST-
tr|G3UBH9|G3UBH9_LOXAF LRSTDTTHST-
tr|G3WTJ4|G3WTJ4_SARHA LRSADTTHST-
Each line begins with a sequence identifier, and then a sequence of characters (in this case describing the amino acid sequence of a protein). Each sequence is split into several lines, so you see that the first sequence (with ID TGFb3_human_used_for_docking) has two lines. I want to convert this to a format in which each sequence has a single line, like this:
TGFb3_human_used_for_docking ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPYLRSADTTHST-
tr|B3KVH9|B3KVH9_HUMAN ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPYLRSADTTHST-
tr|G3UBH9|G3UBH9_LOXAF ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPYLRSTDTTHST-
tr|G3WTJ4|G3WTJ4_SARHA ALDTNYCFRNLEENCCVRPLYIDFRQDLGWKWVHEPKGYYANFCSGPCPYLRSADTTHST-
(In this particular examples the sequences are almost identical, but in general they aren't!)
How can I convert from multi-line multiple sequence alignment format to single-line?
Looks like you need to write a script of some sort to achieve this. Here's a quick example I wrote in Python. It won't line the white-space up prettily like in your example (if you care about that, you'll have to mess around with formatting), but it gets the rest of the job done
#Create a dictionary to accumulate full sequences
full_sequences = {}
#Loop through original file (replace test.txt with your file name)
#and add each line to the appropriate dictionary entry
with open("test.txt") as infile:
for line in infile:
line = [element.strip() for element in line.split()]
if len(line) < 2:
continue
full_sequences[line[0]] = full_sequences.get(line[0], "") + line[1]
#Now loop through the dictionary and write each entry as a single line
outstr = ""
with open("test.txt", "w") as outfile:
for seq in full_sequences:
outstr += seq + "\t\t" + full_sequences[seq] + "\n"
outfile.write(outstr)

Get random text block from file using bash

what is the simplest way of reading a random block of characters from a text file using bash?
A block is a set of characters which begin with X and end with X, where X is a character sequence, usually it will be "\n\n"
We can assume that file has short lines, less than 200 characters each.
Blocks don't have more than 20 lines.
I have seen threads like get random line, get text from between two tokens, but it's not exacly what I need.
I can write a simple program in C that will read how many blocks are in file, get a random number from a given range and then search for a block with this ID, but there must be an easier way.
Example:
X = "\n\n"
File: (the .'s are not in the file, I used them to make "empty" line at the begginning and end of code)
.
first line
second line and some other text
fourth line
sixth line
seventh line, more textęęę
.
Running the script for first time, output:
fourth line
Running the script for the second time, output:
first line
second line and some other text
Yours faithfully,
user2420535
To get a uniformly random block from a file of blank-line-separated blocks in one pass,
awk -v RS='\n\n' '
BEGIN { srand(); }
rand() < 1.0/NR { s=$0; }
END { print s; }
' file
This is a simple case of Reservoir Sampling.

How do I find the percent complete when parsing a file?

How can I print what percentage of a file I have already parsed. I am parsing a text file, so I use:
file.each_line do
Is there a method like each_with_index that is available to use with strings?
This is how I currently use each_with_index to find percentage complete:
amount = 10000000
file.each_with_index do |line, index|
if index == amount
break
end
print "%.1f%% done" % (index/(amount * 1.0) * 100)
print "\r"
To get the number of lines, you can do a couple different things.
If you are on Linux or Mac OS, take advantage of the underlying OS and ask it how many lines are in the file:
lines_in_file = `wc -l #{ path_to_file_to_read }`
wc is extremely fast, and can tell you about lines, words and characters. -l specifies lines.
If you want to do it in Ruby, you could use File.readlines('/path/to/file/to/read') or File.read('/path/to/file/to/read').lines, but be very careful. Both will read the entire file into memory, and, if that file is bigger than your available RAM you've just beaten your machine to a slow death. So, don't do that.
Instead use something like:
lines_in_file = 0
File.foreach('/path/to/file/to/read') { lines_in_file += 1 }
After running, lines_in_file will hold the number of lines in the file. File.foreach is VERY fast, pretty much equal to using File.readlines and probably faster than File.read().lines, and it only reads a line at a time so you're not filling your RAM.
If you want to know the current line number of the line you just read from a file, you can use Ruby's $..
You're concerned about "percentage of a file" though. A potential problem with this is lines are variable length. Depending on what you are doing with them, the line length could have a big effect on your progress meter. You might want to look at the actual length of the file and keep track of the number of characters consumed by reading each line, so your progress is based on percentage of characters, rather than percentage of lines.
Get all the lines upfront, then display the progress as you perform whatever operation you need on them.
lines = file.readlines
amount = lines.length
lines.each_with_index do |line, index|
if index == amount
break
end
print "%.1f%% done" % (index/(amount * 1.0) * 100)
print "\r"
end
Without having to load the file beforehand, you could employ size and pos methods:
f = open('myfile')
while (line = f.gets)
puts "#{(f.pos*100)/f.size}%\t#{line}"
end
Less lines, less logic and accurate to a byte.
Rather than reading the whole file and loading it in memory (as with read or readlines), I suggest to use File.foreach reading the file as a stream, line by line.
count = 0
File.foreach('your_file') { count += 1 }
idx = 0
File.foreach('your_file') do |line|
puts "#{(idx+1).to_f / count * 100}%"
idx += 1
end

Double "gsub" Variable

Is it possible to use variables in both fields of the gsub method ?
I'm trying to get this piece of code work :
$I = 0
def random_image
$I.to_s
random = rand(1).to_s
logo = File.read('logo-standart.txt')
logo_aleatoire = logo.gsub(/#{$I}/, random)
File.open('logo-standart.txt', "w") {|file| File.puts logo_aleatoire}
$I.to_i
$I += 1
end
Thanks in advance !
filecontents = File.read('logo-standart.txt')
filecontents.gsub!(/\d+/){rand(100)}
File.open("logo-standart.txt","w"){|f| f << filecontents }
The magic line is the second line.
The gsub! function modifies the string in-place, unlike the gsub function, which would return a new string and leave the first string unmodified.
The single parameter that I passed to gsub! is the pattern to match. Here, the goal is to match any string of one or more digits -- this is the number that you're going to replace. There's no need to loop through all of the possible numbers running gsub on each one. You can even match numbers as high as a googol (or higher) without your program taking longer and longer to run.
The block that gsub! takes is evaluated each time the pattern matches to programmatically generate a replacement number. So each time, you get a different random number. This is different from the more usual form of gsub! that takes two parameters -- there the parameter is evaluated once before any pattern matching occurs, and all matches are replaced by the same string.
Note that the way this is structured, you get a new random number for each match. So if the number 307 appears twice, it turns into two different random numbers.
If you wanted to map 307 to the same random number each time, you could do the following:
filecontents = File.read('logo-standart.txt')
randomnumbers = Hash.new{|h,k| h[k]=rand(100)}
filecontents.gsub!(/\d+/){|match| randomnumbers[match]}
File.open("logo-standart.txt","w"){|f| f << filecontents }
Here, randomnumbers is a hash that lets you look up the numbers and find what random number they correspond to. The block passed when constructing the hash tells the hash what to do when it finds a number that it hasn't seen before -- in this case, generate a new random number, and remember what that random number the mapping. So gsub!'s block just asks the hash to map numbers for it, and randomnumbers takes care of generating a new random number when you encounter a new number from the original file.

Using Ruby to find the first previous occurrence of a string

I'm creating some basic work assistance utilities using Ruby. I've hit a problem that I don't really need to solve, but curiosity has the best of me.
What I would like to be able to do is search the contents of a file, starting from a particular line and find the first PREVIOUS occurrence of a string.
For example, if I have the following text saved in a file, I would like to be able to search for "CREATE PROCEDURE" starting at line 4 and have this return/output "CREATE PROCEDURE sp_MERGE_TABLE"
CREATE PROCEDURE sp_MERGE_TABLE
AS
SOME HORRIBLE STATEMENT
HERE
CREATE PROCEDURE sp_SOMETHING_ELSE
AS
A DIFFERENT STATEMENT
HERE
Searching for content isn't a challenge, but specifying a starting line - no idea. And then searching backwards... well...
Any help at all appreciated!
TIA!
I think you have to read file line one by line
then follwing will work
flag=true
if flag && line.include?("CREATE PROCEDURE")
puts line
flag=false
end
If performance isn't a big issue, you could just use a simple loop:
# pseudocode
line_no = 0
while line_no < start_line
read line from file
if content_found in this line
last_seen = line_no # or file offset
end
line_no += 1
end
return last_seen
I'm afraid you will have to work line by line through the file, unless you have some index over it, pointing to the beginnings of the lines. That would make the loop a little bit simpler but working through the file in backwards manner is harder (unless you keep the whole file in memory).
Edit:
I just had a much better idea, but I'm going to include the old solution anyway.
The benefit of searching backwards means you only have to read the first chunk of the file, upto the specified line number. For proximity, you get closer and closer to the start_line, and if you find a match you just forget the old one.. You still read in some redundant data at the beginning, but at least it's O(n)
path = "path/to/file"
start_line = 20
search_string = "findme!"
#assuming file is at least start_line lines long
match_index = nil
f = File.new(path)
start_line.times do |i|
line = f.readline
match_index = i if line.include? search_string
end
puts "Matched #{search_string} on line #{match_index}"
Of course, bear in mind that the size of this file plays an important role in answering your question.
If you wanted to get really serious, you could look into the IO class - it seems like this might be the ultimate solution. Untested, just a thought.
f = File.new(path)
start_line.downto(0) do |i|
f.lineno = i
break if f.gets.include?(search_string)
end
Original:
For an exhaustive solution, you could try something like the following. The downside is you'd need to read the whole file into memory, but it takes into account continuing from the bottom-up if it gets to the top without a match. Untested.
path = "path/to/file"
start_line = 20
search_string = "findme!"
#get lines of the file into an array (chomp optional)
lines = File.readlines(path).map(&:chomp)
#"cut" the deck, as with playing cards, so start_line is first in the array
lines = lines.slice!(start_line..lines.length) + lines
#searching backwards can just be searching a reversed array forwards
lines.reverse!
#search through the reversed-array, for the first occurence
reverse_occurence = nil
lines.each_with_index do |line,index|
if line.include?(search_string)
reverse_occurence = index
break
end
end
#reverse_occurence is now either "nil" for no match, or a reversed-index
#also un-cut the array when calculating the index
if reverse_occurence
occurence = lines.size - reverse_occurence - 1 + start_line
line = lines[reverse_occurence]
puts "Matched #{search_string} on line #{occurence}"
puts line
end
1) Read the entire file into a string.
2) Reverse the file-data string.
3) Reverse the search string.
4) Search forward. Remember to match end-of-line instead of beginning-of-line, and to start from position end-minus-N rather than from N.
Not very fast or efficient, but it's elegant. Or at least clever.

Resources