I'm finishing my first "real world" program and I'm not sure how much commenting is too much commenting. What exactly is best practice when adding comments? Do I explain every step or not bother with the "how" and focus more on the "why" or is there no such thing as too much commenting?
Realistically if you are working in a team there should be an established amount of commenting/documentation that is expected of you. If you are keeping the code for yourself you can use as little commenting as you want. ALTHOUGH you should try to keep your code clean, readable, and easy enough to understand with a little thought. If you are keeping your code modular by using functions/methods you should probably describe what that function is doing. ex:
# Calls LS on the ROOT directory then prints output
def main():
bashCommand = "ls /"
process = subprocess.Popen(bashCommand.split(), stdout=subprocess.PIPE)
output = process.communicate()[0]
print output
A bad practice would be leaving decent chunks of code commented out (in your end product). Here is an example of bad practice:
def searchPcGame(game):
#need to parse id="search_items"
url = 'http://www.pcgamer.com/search/%s' %game
rv = requests.get(url)
soup=BeautifulSoup(rv.text)
zz = soup.find_all(id="search_items")
titles = [a.get_text() for a in soup.find_all('h3')]
# zzSoup = BeautifulSoup(zz.text)
# foo = []
# foo.append(articlesSoup)
# return titles
return zz
Related
In a blog post about unconditional programming Michael Feathers shows how limiting if statements can be used as a tool for reducing code complexity.
He uses a specific example to illustrate his point. Now, I've been thinking about other specific examples that could help me learn more about unconditional/ifless/forless programming.
For example in this cat clone there is an if..else block:
#!/usr/bin/env ruby
if ARGV.length > 0
ARGV.each do |f|
puts File.read(f)
end
else
puts STDIN.read
end
It turns out ruby has ARGF which makes this program much simpler:
#!/usr/bin/env ruby
puts ARGF.read
I'm wondering if ARGF didn't exist how could the above example be refactored so there is no if..else block?
Also interested in links to other illustrative specific examples.
Technically you can,
inputs = { ARGV => ARGV.map { |f| File.open(f) }, [] => [STDIN] }[ARGV]
inputs.map(&:read).map(&method(:puts))
Though that's code golf and too clever for its own good.
Still, how does it work?
It uses a hash to store two alternatives.
Map ARGV to an array of open files
Map [] to an array with STDIN, effectively overwriting the ARGV entry if it is empty
Access ARGV in the hash, which returns [STDIN] if it is empty
Read all open inputs and print them
Don't write that code though.
As mentioned in my answer to your other question, unconditional programming is not about avoiding if expressions at all costs but about striving for readable and intention revealing code. And sometimes that just means using an if expression.
You can't always get rid of a conditional (maybe with an insane number of classes) and Michael Feathers isn't advocating that. Instead it's sort of a backlash against overuse of conditionals. We've all seen nightmare code that's endless chains of nested if/elsif/else and so has he.
Moreover, people do routinely nest conditionals inside of conditionals. Some of the worst code I've ever seen is a cavernous nightmare of nested conditions with odd bits of work interspersed within them. I suppose that the real problem with control structures is that they are often mixed with the work. I'm sure there's some way that we can see this as a form of single responsibility violation.
Rather than slavishly try to eliminate the condition, you could simplify your code by first creating an array of IO objects from ARGV, and use STDIN if that list is empty.
io = ARGV.map { |f| File.new(f) };
io = [STDIN] if !io.length;
Then your code can do what it likes with io.
While this has strictly the same number of conditionals, it eliminates the if/else block and thus a branch: the code is linear. More importantly, since it separates gathering data from using it, you can put it in a function and reuse it further reducing complexity. Once it's in a function, we can take advantage of early return.
# I don't have a really good name for this, but it's a
# common enough idiom. Perl provides the same feature as <>
def arg_files
return ARGV.map { |f| File.new(f) } if ARGV.length;
return [STDIN];
end
Now that it's in a function, your code to cat all the files or stdin becomes very simple.
arg_files.each { |f| puts f.read }
First, although the principle is good, you have to consider other things that are more importants such as readability and perhaps speed of execution.
That said, you could monkeypatch the String class to add a read method and put STDIN and the arguments in an array and start reading from the beginning until the end of the array minus 1, so stopping before STDIN if there are arguments and go on until -1 (the end) if there are no arguments.
class String
def read
File.read self if File.exist? self
end
end
puts [*ARGV, STDIN][0..ARGV.length-1].map{|a| a.read}
Before someone notices that I still use an if to check if a File exists, you should have used two if's in your example to check this also and if you don't, use a rescue to properly inform the user.
EDIT: if you would use the patch, read about the possible problems at these links
http://blog.jayfields.com/2008/04/alternatives-for-redefining-methods.html
http://www.justinweiss.com/articles/3-ways-to-monkey-patch-without-making-a-mess/
Since the read method isn't part of String the solutions using alias and super are not necessary, if you plan to use a Module, here is how to do that
module ReadString
def read
File.read self if File.exist? self
end
end
class String
include ReadString
end
EDIT: just read about a safe way to monkey patch, for your documentation see https://solidfoundationwebdev.com/blog/posts/writing-clean-monkey-patches-fixing-kaminari-1-0-0-argumenterror-comparison-of-fixnum-with-string-failed?utm_source=rubyweekly&utm_medium=email
I have following recursive function written in Ruby, however I find that the method is running too slowly. I am unsure if this the correct way to do it, so please suggest how to improve the performance of this code. The total file count including the subdirectories is 4,535,347
def start(directory)
Dir.foreach(directory) do |file|
next if file == '.' or file == '..'
full_file_path = "#{directory}/#{file}"
if File.directory?(full_file_path)
start(full_file_path)
elsif File.file?(full_file_path)
extract(full_file_path)
else
raise "Unexpected input type neither file nor folder"
end
end
With 4.5M directories, you might be better off working with a specialized lazy enumerator so as to only process entries you actually need, rather than generating each and every one of those 4.5M lists, returning the entire set and iterating through it in entirety.
Here's the example from the docs:
class Enumerator::Lazy
def filter_map
Lazy.new(self) do |yielder, *values|
result = yield *values
yielder << result if result
end
end
end
(1..Float::INFINITY).lazy.filter_map{|i| i*i if i.even?}.first(5)
http://ruby-doc.org/core-2.1.1/Enumerator/Lazy.html
It's not a very good example, btw: the important part is Lazy.new() rather than the fact that Enumerator::Lazy gets monkey patched. Here's a much better example imho:
What's the best way to return an Enumerator::Lazy when your class doesn't define #each?
Further reading on the topic:
http://patshaughnessy.net/2013/4/3/ruby-2-0-works-hard-so-you-can-be-lazy
Another option you might want to consider is computing the list across multiple threads.
I don't think there's a way to speed up much your start method; it does the correct things of going through your files and processing them as soon as it encounters them. You can probably simplify it with a single Dir.glob do, but it will still be slow. I suspect that this is not were most of the time is spent.
There very well might be a way to speed up your extract method, impossible to know without the code.
The other way to speed this up might be to split the processing to multiple processes. Since reading & writing is probably what is slowing you down, this way would give you hope that the ruby code executes while another process is waiting for the IO.
I'm interested in building a DSL in Ruby for use in parsing microblog updates. Specifically, I thought that I could translate text into a Ruby string in the same way as the Rails gem allows "4.days.ago". I already have regex code that will translate the text
#USER_A: give X points to #USER_B for accomplishing some task
#USER_B: take Y points from #USER_A for not giving me enough points
into something like
Scorekeeper.new.give(x).to("USER_B").for("accomplishing some task").giver("USER_A")
Scorekeeper.new.take(x).from("USER_A").for("not giving me enough points").giver("USER_B")
It's acceptable to me to formalize the syntax of the updates so that only standardized text is provided and parsed, allowing me to smartly process updates. Thus, it seems it's more a question of how to implement the DSL class. I have the following stub class (removed all error checking and replaced some with comments to minimize paste):
class Scorekeeper
attr_accessor :score, :user, :reason, :sender
def give(num)
# Can 'give 4' or can 'give a -5'; ensure 'to' called
self.score = num
self
end
def take(num)
# ensure negative and 'from' called
self.score = num < 0 ? num : num * -1
self
end
def plus
self.score > 0
end
def to (str)
self.user = str
self
end
def from(str)
self.user = str
self
end
def for(str)
self.reason = str
self
end
def giver(str)
self.sender = str
self
end
def command
str = plus ? "giving ##{user} #{score} points" : "taking #{score * -1} points from ##{user}"
"##{sender} is #{str} for #{reason}"
end
end
Running the following commands:
t = eval('Scorekeeper.new.take(4).from("USER_A").for("not giving me enough points").giver("USER_B")')
p t.command
p t.inspect
Yields the expected results:
"#USER_B is taking 4 points from #USER_A for not giving me enough points"
"#<Scorekeeper:0x100152010 #reason=\"not giving me enough points\", #user=\"USER_A\", #score=4, #sender=\"USER_B\">"
So my question is mainly, am I doing anything to shoot myself in the foot by building upon this implementation? Does anyone have any examples for improvement in the DSL class itself or any warnings for me?
BTW, to get the eval string, I'm mostly using sub/gsub and regex, I figured that's the easiest way, but I could be wrong.
Am I understanding you correctly: you want to take a string from a user and cause it to trigger some behavior?
Based on the two examples you listed, you probably can get by with using regular expressions.
For example, to parse this example:
#USER_A: give X points to #USER_B for accomplishing some task
With Ruby:
input = "#abe: give 2 points to #bob for writing clean code"
PATTERN = /^#(.+?): give ([0-9]+) points to #(.+?) for (.+?)$/
input =~ PATTERN
user_a = $~[1] # => "abe"
x = $~[2] # => "2"
user_b = $~[3] # => "bob"
why = $~[4] # => "writing clean code"
But if there is more complexity, at some point you might find it easier and more maintainable to use a real parser. If you want a parser that works well with Ruby, I recommend Treetop: http://treetop.rubyforge.org/
The idea of taking a string and converting it to code to be evaled makes me nervous. Using eval is a big risk and should be avoided if possible. There are other ways to accomplish your goal. I'll be happy to give some ideas if you want.
A question about the DSL you suggest: are you going to use it natively in another part of your application? Or do just plan on using it as part of the process to convert the string into the behavior you want? I'm not sure what is best without knowing more, but you may not need the DSL if you are just parsing the strings.
This echoes some of my thoughts on a tangental project (an old-style text MOO).
I'm not convinced that a compiler-style parser is going to be the best way for the program to deal with the vaguaries of english text. My current thoughts have me splitting up the understanding of english into seperate objects -- so a box understands "open box" but not "press button", etc. -- and then having the objects use some sort of DSL to call centralised code that actually makes things happen.
I'm not sure that you've got to the point where you understand how the DSL is actually going to help you. Maybe you need to look at how the english text gets turned into DSL, first. I'm not saying that you don't need a DSL; you might very well be right.
As for hints as to how to do that? Well, I think if I were you I would be looking for specific verbs. Each verb would "know" what sort of thing it should expect from the text around it. So in your example "to" and "from" would expect a user immediately following.
This isn't especially divergent from the code you've posted here, IMO.
You might get some milage out of looking at the answers to my question. One commenter pointed me to the Interpreter Pattern, which I found especially enlightening: there's a nice Ruby example here.
Building on #David_James' answer, I've come up with a regex-only solution to this since I'm not actually using the DSL anywhere else to build scores and am merely parsing out points to users. I've got two patterns that I'll use to search:
SEARCH_STRING = "#Scorekeeper give a healthy 4 to the great #USER_A for doing something
really cool.Then give the friendly #USER_B a healthy five points for working on this.
Then take seven points from the jerk #USER_C."
PATTERN_A = /\b(give|take)[\s\w]*([+-]?[0-9]|one|two|three|four|five|six|seven|eight|nine|ten)[\s\w]*\b(to|from)[\s\w]*#([a-zA-Z0-9_]*)\b/i
PATTERN_B = /\bgive[\s\w]*#([a-zA-Z0-9_]*)\b[\s\w]*([+-]?[0-9]|one|two|three|four|five|six|seven|eight|nine|ten)/i
SEARCH_STRING.scan(PATTERN_A) # => [["give", "4", "to", "USER_A"],
# ["take", "seven", "from", "USER_C"]]
SEARCH_STRING.scan(PATTERN_B) # => [["USER_B", "five"]]
The regex might be cleaned up a bit, but this allows me to have syntax that allows a few fun adjectives while still pulling the core information using both "name->points" and "points->name" syntaxes. It does not allow me to grab the reason, but that's so complex that for now I'm going to just store the entire update, since the whole update will be related to the context of each score anyway in all but outlier cases. Getting the "giver" username can be done elsewhere as well.
I've written up a description of these expressions as well, in hopes that other people might find that useful (and so that I can go back to it and remember what that long string of gobbledygook means :)
Is there any reason to do use block initialization, like this:
x = Observer.new do
add_event(foo)
some_other_instance_method_on_observer
self.some_attribute = something
end
instead of initializing attributes using the dot operator on an instance variable like this:
x = Observer.new
x.add_event(foo)
x.some_other_instance_method_on_observer
x.some_attribute = something
The only reason here is to have more concise code(you don't have to add the instance name before the method call). By using blocks in general you can write very neat, concise and readable code. Some times you can save your code consumers lots of typing and even code logic.
Here is a traditional case!
file = File.open("some_file.txt","w")
file << "more code"
file.close
Compare it to this nice block alternative:
File.open("some_file.txt","w") { |file| file << "adding new stuff" }
It saved the user from the hassle of having to open and close(personally i keep forgetting this one) the file himself. Instead it made him focus on what he wants.
Try to invest blocks in such situations + when you want to write nice DSLs.
One advantage is that it makes it obvious that those additional actions are initialization actions.
Building on khell's answer, variables created within a block go out of scope outside of the block, which is a good thing if you don't have any more use of it.
If your code is in a method, which returns the object. Compare
def create_observer
Observer.new {|x| x.foo, x.bar = calculate_foo, calculate_bar }
end
def create_observer
x = Observer.new
x.foo = calculate_foo
x.bar = calculate_bar
x
end
So using a block can potentially allow you to write your logic in one line.
Even if you use parallel assignment in the second example, you still need 3 lines.
Also, check out tap here. It has similar benefits.
I'd like to slice and dice large datafiles, up to a gig, in a fairly quick and efficient manner. If I use something like UNIX's "CUT", it's extremely fast, even in a CYGWIN environment.
I've tried developing and benchmarking various Ruby scripts to process these files, and always end up with glacial results.
What would you do in Ruby to make this not so dog slow?
This question reminds me of Tim Bray's Wide Finder project. The fastest way he could read an Apache logfile using Ruby and figure out which articles have been fetched the most was with this script:
counts = {}
counts.default = 0
ARGF.each_line do |line|
if line =~ %r{GET /ongoing/When/\d\d\dx/(\d\d\d\d/\d\d/\d\d/[^ .]+) }
counts[$1] += 1
end
end
keys_by_count = counts.keys.sort { |a, b| counts[b] <=> counts[a] }
keys_by_count[0 .. 9].each do |key|
puts "#{counts[key]}: #{key}"
end
It took this code 7½ seconds of CPU, 13½ seconds elapsed, to process a million and change records, a quarter-gig or so, on last year’s 1.67Ghz PowerBook.
Why not combine them together - using cut to do what it does best and ruby to provide the glue/value add with the results from CUT? you can run shell scripts by putting them in backticks like this:
puts `cut somefile > foo.fil`
# process each line of the output from cut
f = File.new("foo.fil")
f.each{|line|
}
I'm guessing that your Ruby implementations are reading the entire file prior to processing. Unix's cut works by reading things one byte at a time and immediately dumping to an output file. There is of course some buffering involved, but not more than a few KB.
My suggestion: try doing the processing in-place with as little paging or backtracking as possible.
I doubt the problem is that ruby is reading the whole file in memory. Look at the memory and disk usage while running the command to verify.
I'd guess the main reason is because cut is written in C and is only doing one thing, so it has probably be compiled down to the very metal. It's probably not doing a lot more than calling the system calls.
However the ruby version is doing many things at once. Calling a method is much slower in ruby than C function calls.
Remember old age and trechery beat youth and skill in unix: http://ridiculousfish.com/blog/archives/2006/05/30/old-age-and-treachery/