I'm trying to find out if any of the words in the array keywords:
keywords = Array['hello', 'hi', 'home']
appear in split:
split = work.split(' ')
and for each keyword that appears in split, do something.
I have this code:
for keywords in splitDescription
score += 2
end
The code just gives me 2+ for each word in split, but I only want 2+ for each keywords appearing in split.
Use set intersection on arrays (Array#&):
(splitDescription & keywords).each do |found_keyword|
# something
end
If you know there might be some words repeating and you want to iterate over each occurrence:
splitDescription.select { |word| keywords.include?(word) }.each do |word|
# something
end
Related
I have an array of words. I would like to get the letter count to print next to each word. I get the words to print with a total of all letters to print at the end. Any guidance would be great.
enter image description here
If you are looking to print length of each word, below should work.
a = #[array of words]
a.each {|e| p "#{e} has #{e.length} letters"}
words = [] #array words
words.each {|w| puts "#{w} : #{w.length}"}
other way you can use Gem "words_counted" https://github.com/abitdodgy/words_counted for natural language processor
An anagram group is a group of words such that any one can be converted into any other just by rearranging the letters. For example, "rats", "tars" and "star" are an anagram group.
Now I have an array of words and I am going to find the anagram words
to find this I have written the following code
actually it works for some words like scar and cars, but it doesn't work
for [scar , carts].
temp=[]
words.each do |e|
temp=e.split(//) # make an array of letters
words.each do |z|
if z.match(/#{temp}/) # match to find scar and cars
puts "exp is True"
else
puts "exp is false"
end
end
end
I just think that while [abc] means a or b or c I can separate my words to letters and then look for other cases in the array
Your algorithm is incorrect and inefficient (quadratic time complexity). Why regex?
Here's another idea. Define the signature of a word such that all the letters of a word are sorted. For example, the signature of hello is ehllo.
By this definition, anagrams are words that have the same signature, for example, rats, tars and star all have the signature arst. The code to implement this idea is straight-forward.
Two words are anagrams if they contain the same letters. There are several ways to figure out whether they do, the most obvious one is sorting the letters alphabetically. Then you want to separate the words into groups. Here's an idea:
words = %w[cats scat rats tars star scar cars carts]
words.group_by {|word| word.each_char.sort }.values
# => [['cats', 'scat'], ['rats', 'tars', 'star'], ['scar', 'cars'], ['carts']]
The problem is that /#{e.split(//)}/ here is pretty much nonsensical.
To illustrate this, lets see what happens:
word = 'wtf'
letters = word.split(//) # => ["w", "t", "f"]
regex = /#{letters}/ # => /["w", "t", "f"]/
'"'.match(regex) # => 0
','.match(regex) # => 0
' '.match(regex) # => 0
't'.match(regex) # => 0
What happens is interpolating something in a regex replaces it with the result of its to_s method. And since character sets match a single character in what's inside, you will get a regex that matches " or , or or any of the letters in the original word.
Therefore, I will unfortunately call your solution unsalvageable.
A very easy way to check if two words are anagrams is to sort their characters and see if the result is the same.
The faster way would be:
def is_anagram? w1, w2
w1.chars.sort == w2.chars.sort
end
You could also do something like this I suppose:
def is_anagram? w1, w2
w2 = w2.chars
w1.chars.permutation.to_a.include?(w2)
end
then run it like this:
is_anagram? "rats", "star"
=> true
Note:
This post has been edited as per Cary Swoveland's advice.
words = ['demo', 'none', 'tied', 'evil', 'dome', 'mode', 'live',
'fowl', 'veil', 'wolf', 'diet', 'vile', 'edit', 'tide',
'flow', 'neon']
groups = words.group_by { |word| word.split('').sort }
groups.each { |x, y| p y }
I want my output to search and count the frequency of the words "candy" and "gram", but also the combinations of "candy gram" and "gram candy," in a given text (whole_file.)
I am currently using the following code to display the occurrences of "candy" and "gram," but when I aggregate the combinations within the %w, only the word and frequencies of "candy" and "gram" display. Should I try a different way? thanks so much.
myArray = whole_file.split
stop_words= %w{ candy gram 'candy gram' 'gram candy' }
nonstop_words = myArray - stop_words
key_words = myArray - nonstop_words
frequency = Hash.new (0)
key_words.each { |word| frequency[word] +=1 }
key_words = frequency.sort_by {|x,y| x }
key_words.each { |word, frequency| puts word + ' ' + frequency.to_s }
It sounds like you're after n-grams. You could break the text into combinations of consecutive words in the first place, and then count the occurrences in the resulting array of word groupings. Here's an example:
whole_file = "The big fat man loves a candy gram but each gram of candy isn't necessarily gram candy"
[["candy"], ["gram"], ["candy", "gram"], ["gram", "candy"]].each do |term|
terms = whole_file.split(/\s+/).each_cons(term.length).to_a
puts "#{term.join(" ")} #{terms.count(term)}"
end
EDIT: As was pointed out in the comments below, I wasn't paying close enough attention and was splitting the file on each loop which is obviously not a good idea, especially if it's large. I also hadn't accounted for the fact that the original question may've need to sort by the count, although that wasn't explicitly asked.
whole_file = "The big fat man loves a candy gram but each gram of candy isn't necessarily gram candy"
# This is simplistic. You would need to address punctuation and other characters before
# or at this step.
split_file = whole_file.split(/\s+/)
terms_to_count = [["candy"], ["gram"], ["candy", "gram"], ["gram", "candy"]]
counts = []
terms_to_count.each do |term|
terms = split_file.each_cons(term.length).to_a
counts << [term.join(" "), terms.count(term)]
end
# Seemed like you may need to do sorting too, so here that is:
sorted = counts.sort { |a, b| b[1] <=> a[1] }
sorted.each do |count|
puts "#{count[0]} #{count[1]}"
end
Strip punctuation and convert to lower-case
The first thing you probably want to do is remove all punctuation from the string holding the contents of the file and then convert what's left to lower case, the latter so you don't have worry about counting 'Cat' and 'cat' as the same word. Those two operations can be done in either order.
Changing upper-case letters to lower-case is easy:
text = whole_file.downcase
To remove the punctuation it is probably easier to decide what to keep rather than what to discard. If we only want to keep lower-case letters, you can do this:
text = whole_file.downcase.gsub(/[^a-z]/, '')
That is, substitute an empty string for all characters other than (^) lowercase letters.1
Determine frequency of individual words
If you want to count the number of times text contains the word 'candy', you can use the method String#scan on the string text and then determine the size of the array that is returned:
text.scan(/\bcandy\b/).size
scan returns an array with every occurrence of the string 'candy'; .size returns the size of that array. Here \b ensures 'candy gram' has a word "boundary" at each end, which could be whitespace or the beginning or end of a line or the file. That's to prevent `candycane' from being counted.
A second way is to convert the string text to an array of words, as you have done2:
myArray = text.split
If you don't mind, I'd like to call this:
words = text.split
as I find that more expressive.3
The most direct way to determine the number of times 'candy' appears is to use the method Enumberable#count, like this:
words.count('candy')
You can also use the array difference method, Array#-, as you noted:
words.size - (words - ['candy']).size
If you wish to know the number of times either 'candy' or 'gram' appears, you could of course do the above for each and sum the two counts. Some other ways are:
words.size - (myArray - ['candy', 'gram']).size
words.count { |word| word == 'candy' || word = 'gram' }
words.count { |word| ['candy', 'gram'].include?(word) }
Determine the frequency of all words that appear in the text
Your use of a hash with a default value of zero was a good choice:
def frequency_of_all_words(words)
frequency = Hash.new(0)
words.each { |word| frequency[word] +=1 }
frequency
end
I wrote this as a method to emphasize that words.each... does not return frequency. Often you would see this written more compactly using the method Enumerable#each_with_object, which returns the hash ("object"):
def frequency_of_all_words(words)
words.each_with_object(Hash.new(0)) { |word, h| h[word] +=1 }
end
Once you have the hash frequency you can sort it as you did:
frequency.sort_by {|word, freq| freq }
or
frequency.sort_by(&:last)
which you could write:
frequency.sort_by {|_, freq| freq }
since you aren't using the first block variable. If you wanted the most frequent words first:
frequency.sort_by(&:last).reverse
or
frequency.sort_by {|_, freq| -freq }
All of these will give you an array. If you want to convert it back to a hash (with the largest values first, say):
Hash[frequency.sort_by(&:last).reverse]
or in Ruby 2.0+,
frequency.sort_by(&:last).reverse.to_h
Count the number of times a substring appears
Now let's count the number of times the string 'candy gram' appears. You might think we could use String#scan on the string holding the entire file, as we did earlier4:
text.scan(/\bcandy gram\b/).size
The first problem is that this won't catch 'candy\ngram'; i.e., when the words are separated by a newline character. We could fix that by changing the regex to /\bcandy\sgram\b/. A second problem is that 'candy gram' might have been 'candy. Gram' in the file, in which case you might not want to count it.
A better way is to use the method Enumerable#each_cons on the array words. The easiest way to show you how that works is by example:
words = %w{ check for candy gram here candy gram again }
#=> ["check", "for", "candy", "gram", "here", "candy", "gram", "again"]
enum = words.each_cons(2)
#=> #<Enumerator: ["check", "for", "candy", "gram", "here", "candy",
# "gram", "again"]:each_cons(2)>
enum.to_a
#=> [["check", "for"], ["for", "candy"], ["candy", "gram"],
# ["gram", "here"], ["here", "candy"], ["candy", "gram"],
# ["gram", "again"]]
each_cons(2) returns an enumerator; I've converted it to an array to display its contents.
So we can write
words.each_cons(2).map { |word_pair| word_pair.join(' ') }
#=> ["check for", "for candy", "candy gram", "gram here",
# "here candy", "candy gram", "gram again"]
and lastly:
words.each_cons(2).map { |word_pair|
word_pair.join(' ') }.count { |s| s == 'candy gram' }
#=> 2
1 If you also wanted to keep dashes, for hyphenated words, change the regex to /[^-a-z]/ or /[^a-z-]/.
2 Note from String#split that .split is the same as both .split(' ') and .split(/\s+/)).
3 Also, Ruby's naming convention is to use lower-case letters and underscores ("snake-case") for variables and methods, such as my_array.
I have an array:
array = ["abhor", "rage", "mad"]
and I want to check if a string includes any word in that array - But only that word (not a substring).
string = 'I made a cake.'
count = 0
array.each do |word|
if string.include? word
count += 1
end
end
However, the above does increment count by 1 because it's picking up the word mad from made in my string. How can I search for only mad and ensure made doesn't get counted?
The array intersection operator & is useful here.
Here's two options, depending on how you define "word":
1) If a word is any sequence of non-whitespace characters, then you can do:
array & string.split
In your example, this results in the intersection of array and words in string, which is empty.
2) If a word is any sequence of alphanumeric characters including _, then you can do:
array & string.scan(/\w+/)
For example if array = ["abhor", "rage", "mad", "cake"] then #1 above will be empty (because you have cake. with a period in the string) but will return ['cake'] for method #2.
I would do it like this:
array = ["abhor", "rage", "mad"]
string = 'I made a cake.'
string.split.count{|word| array.include?(word)}
The problem with doing a simple split is that it won't take into account punctuation. What you need is a regex, which is a little more complicated.
array.each do |word|
count += 1 if string.match(/\W#{word}\W/)
end
If you're willing to go down the regex road, \b represent a word boundary. This example, which includes all of the words in your sentence and a few which are fragments, correctly returns 4.
array = ["abhor", "rage", "mad", "I", "made", "a", "cake", "cak"]
string = 'I made a cake.'
count = 0
array.each do |word|
if string =~ /\b#{word}\b/
count += 1
end
end
Try splitting up the words first.
words = string.split
count = 0
words.each do |word|
count += 1 if array.include? word
end
What is the best way to validate a gets input against a very long word list (a list of all the English words available)?
I am currently playing with readlines to manipulate the text, but before there's any manipulation, I would like to first validate the entry against the list.
The simplest way, but by no means the fastest, is to simply search against the word list each time. If the word list is in an array:
if word_list.index word
#manipulate word
end
If, however, you had the word list as a separate file (with each word on a separate line), then we'll use File#foreach to find it:
if File.foreach("word.list") {|x| break x if x.chomp == word}
#manipulate word
end
Note that foreach does not strip off the trailing newline character(s), so we get rid of them with String#chomp.
Here's a simple example using a Set, though Mark Johnson is right,
a bloom filter would be more efficient.
require 'set'
WORD_RE = /\w+/
# Read in the default dictionary (from /usr/share/dict/words),
# and put all the words into a set
WORDS = Set.new(File.read('/usr/share/dict/words').scan(WORD_RE))
# read the input line by line
STDIN.each_line do |line|
# find all the words in the line that aren't contained in our dictionary
unrecognized = line.scan(WORD_RE).find_all { |term| not WORDS.include? term }
# if none were found, the line is valid
if unrecognized.empty?
puts "line is valid"
else # otherwise, the line contains some words not in our dictionary
puts "line is invalid, could not recognize #{unrecognized.inspect}"
end
end
are you reading the list from a file?
can't you have it all in memory?
maybe a finger tree may help you
if not, there's not more than "read a chunk of data from the file and grep into"
Read the word list into memory, and for each word, make an entry into a hash table:
def init_word_tester
#words = {}
File.foreach("word.list") {|word|
#words[word.chomp] = 1
}
end
now you can just check every word against your hash:
def test_word word
return #words[word]
end