As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
I'm not sure if this is quite the right place, but it seems like a decent place to ask.
My current job involves manual analysis of large data sets (at several levels, each more refined and done by increasingly experienced analysts). About a year ago, I started developing some utilities to track analyst performance by comparing results at earlier levels to final levels. At first, this worked quite well - we used it in-shop as a simple indicator to help focus training efforts and do a better job overall.
Recently though, the results have been taken out of context and used in a way I never intended. It seems management (one person in particular) has started using the results of these tools to directly affect EPR's (enlisted performance reports - \ it's an air force thing, but I assume something similar exists in other areas) and similar paperwork. The problem isn't who is using these results, but how. I've made it clear to everyone that the results are, quite simply, error-prone.
There are numerous unavoidable obstacles to generating this data, which I have worked to minimize with some nifty heuristics and such. Taken in the proper context, they're a useful tool. Out of context however, as they are now being used, they do more harm than good.
The manager(s) in question are taking the results as literal indicators of whether an analyst is performing well or poorly. The results are being averaged and individual scores are being ranked as above (good) or below (bad) average. This is being done with no regard for inherent margins of error and sample bias, with no regard for any sort of proper interpretation. I know of at least one person whose performance rating was marked down for an 'accuracy percentage' less than one percentage point below average (when the typical margin of error from the calculation method alone is around two to three percent).
I'm in the process of writing a formal report on the errors present in the system ("Beginner's Guide to Meaningful Statistical Analysis" included), but all signs point to this having no effect.
Short of deliberately breaking the tools (a route I'd prefer avoiding but am strongly considering under the circumstances), I'm wondering if anyone here has effectively dealt with similar situations before? Any insight into how to approach this would be greatly appreciated.
Update:
Thanks for the responses - plenty of good ideas all around.
If anyone is curious, I'm moving in the direction of 'refine, educate, and take control of interpretation'. I've started rebuilding my tools to try and negate or track error better and automatically generate any numbers and graphs they could want, with included documentation throughout (while hiding away as obscure references the raw data they currently seem so eager to import to the 'magical' excel sheets).
In particular, I'm hopeful that visual representations of error and properly created ranking systems (taking into account error, standard deviations, etc.) will help the situation.
Either modify the output to include error information (so if the error is +/- 5 %, don't output 22%, output 17% - 27%), or educate those whom this is being used against to the error so that they can defend themselves when it is used against them.
Well, you seem to have run afoul of the Law of Unintended Consequences in the context of human behavior.
Unfortunately, once the cat is out of the bag, it's pretty hard to put back in. You have a few options (which are not mutually exclusive, by the way) to consider, including:
Alter the reports so that their data can no longer be abused in the way you describe.
Work with management to help them understand why their use of your data is improper or misleading.
Work with those whose performance is being measured to pressure management to rethink their policy on the matter.
Work with management/analysts to come up with a viable means to measure performance in a way that is fair to everyone.
Break the report in a manner that makes them unusable for any purposes.
Clearly there is a desire on the part of management to get analytics on performance of analysts. Likely there is a real need for this ... and your reports happened to fill a void in the available information. The best option for everyone would be to find a way to effectively and fairly fill this need. There are many possible ways to achieve this - from dropping dense rankings in favor of performance tiers to using time-over-time variance to refine performance measurements.
Now, it's entirely possible that the existing reports you've provided simply cannot be applied in a fair and accurate manner to address this problem. In which case, you should work with your management team to make sure they understand why this is the case - and either redefine the way performance is measured or take the time to develop an appropriate and fair methodology.
One of the strongest means to convince management that their (ab)use of the data in your report is unwise is to remind them of the concept of perverse incentives. It's entirely possible that over time, analysts will modify their behavior in a way that results in higher rankings in performance reports at the cost of real performance or quality of results that are not otherwise captured or expressed. You seem to have a good understanding of your domain - so I would hope that you could provide specific and dramatic examples of such consequences to help make your case.
All you can do is to try and educate the managers as to why what they're doing is incorrect.
Beyond that, you can't stop idiots from being idiotic, and you'll just go mad trying.
I definitely wouldn't "break" code that people are relying on, even if it's not a specific deliverable. That will only cause them to complain about you, a move which may affect your own EPR :-)
I really think the key here is good communication with your managers.
Besides, I like PatrickV's idea. You could also try some other ways to engineer your tool around the problem so that it'll seem silly/be hard to use it as performance measurement - change the name of the statistics to mean something other than "how good programmer X is", make it hard to get data per-person, show error statistics.
You can also try to display the data in another way (this may actually make your managers think you are trying to help them). Show a graph - a several pixels difference in position may be harder to identify than a numeric results (my guess - your managers are using excel and coloring red everything below average). Draw the error margin so it doesn't make sense to obsess over fractions of percentages.
Give the result as a scale - low and high margin that take into account your error information, it is harder to compare.
Edit: Oh yeah, and read about "social interfaces". You can start with's Spolsky's Not Just Usability and Building Communities with Software.
I would echo #paxdiablo's advice, as a first step:
Work on the report on the inherent errors. In fact, make it the introduction to every copy generated.
When you refer to the measurement errors, indicate they are the lower limit of the errors (unless there actually aren't any).
Try to educate the manager(s) in the error of his/her ways.
If possible, discuss the issue with your manager. And perhaps with the offending managers' management, depending on how familiar you are with them you probably limit it to just "express some concerns" and giving a heads-up.
Consult your HR department, or whomever is in charge of fairness in the performance reviews.
Good luck.
The problem is that the code is not yours, it belongs to your company. They really can do whatever they want with it.
I hate to say this, but if you have an issue with the ethics of your company you will have to leave that company.
One thing you could do is implement the comparison yourself. If he really wants to check if somebody is performing significantly less than the rest, it should be tested formally as well.
Now to choose the right test is a bit tricky without knowing the data and the structure, so I can't really advise you on that one. Just take into account that if you do pairwise comparisons, or compare multiple scores against an average, that you run into the multitesting problem. A classic way of correcting is using Bonferroni. If you implement that one, you can be sure that at a certain point, noone will jump out any more. The Bonferroni correction is very conservative. Another option is using Dunn-Sidak, which is supposed to be less conservative.
The correct implementation would be an ANOVA -if the assumptions are met and the data suitable off course- with a post-hoc comparison like a Tukey Honest Significant Difference test. That way at least the uncertainty on the results is taken into account.
If you don't have a clue on which test to use, describe your data in detail on stats.stackexchange.com and ask for help on which test to use.
Cheers
I just wanted to elaborate on the Perverse Incentives answer of LBushkin. I can easily see your problem extending to where analysts will avoid difficult topics for fear of reducing their score. Or maybe they will provide the same answer as earlier stages to avoid hurting a friends score, even if that is not correct. An interesting question is what happens if the later answer is incorrect - you have no truth, just successive analytic opinions - in this case I assume the first answer is marked as "incorrect", right?
Maybe presenting some of these extensions to the manager will help.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
Coming from an IT background, I've been involved with software projects but I'm not a programmer. One of my biggest challenges is that having a lot of experience in IT, people often turn to me to manage projects that include software development. The projects are usually outsourced and there isnt a budget for a full time architect or PM, which leaves me in a position to evaluate the work being performed.
Where I've managed to get through this in the past, I'm (with good reason) uneasy about accepting these responsibilities.
My question is, from a perspective of being technically experienced but not in programming, how can I evaluate whether coding is written well besides just determining if it works or not? Are there methodologies, tips, tricks of the trade, flags, signs, anything that would say - hey this is junk or hey this is pretty damn good?
Great question. Should get some good responses.
Code cleanliness (indented well, file organization, folder structure)
Well commented (not just inline comments, but variables that say what they are, functions that say what they do, etc.)
Small understandable functions/methods (no crazy 300 line methods that do all sorts of things with nested if logic all over the place)
Follows SOLID principles
Is the amount of unit test code similar in size and quality as the code base of the project
Is the interface code separate from the business logic code which in turn should be separate from the infrastructure access code (email, database, web services, file system, etc.)
What does a performance analysis tool think of the code (NDepend, NDoc, NCover, etc.)
There is a lot more to this...but this gets your started.
Code has 2 primary audiences:
The people who use it
The people who develop it
So you neeed 2 simple tests:
Run the code. Can you get it to do the job it is supposed to do?
Read the code. Can you understand the general intentions of the developer?
If you can answer yes to both of these, it is great code.
When reading the code, don't worry that you are not a programmer. If code is well written / documented, even a non-programmer should be able to see guess much of what it is intended to achieve.
BTW: Great question! I wish more non-programmers cared about code quality.
First, set ground rules (that all programmers sign up to) that say what's 'good' and what isn't. Automate tests for those that you can measure (e.g. functions less than a number of lines, McCabe complexity, idioms that your coders find confusing). Then accept that 'good coding' is something you know when you see rather than something you can actually pin down with a set of rules, and allow people to deviate from the standard provided they get agreement from someone with more experience. Similarly, such standards have to be living documents, adapted in the face of feedback.
Code reviews also work well, since not all such 'good style' rules can be automatically determined. Experienced programmers can say what they don't like about inexperienced programmers' code - and you have to get the original authors to change it so that they learn from their mistakes - and inexperienced programmers can say what they find hard to understand about other people's code - and, by being forced to read other people's code, they'll also learn new tricks. Again, this will give you feedback on your standard.
On some of your specific points, complexity and function size work well, as does code coverage during repeatable (unit) testing, but that last point comes with a caveat: unless you're working on something where high quality standards are a necessity (embedded code, as an example, or safety-critical code) 100% code coverage means you're testing the 10% of code paths that are worthwhile to test and the 90% that almost never get coded wrong in the first place. Worthwhile tests are the ones that find bugs and improve maintainability.
I think it's great you're trying to evaluate something that typically isn't evaluated. There have been some good answers above already. You've already shown yourself to be more mature in dealing with software by accepting that since you don't practice development personally, you can't assume that writing software is easy.
Do you know a developer whose work you trust? Perhaps have that person be a part of the evaluation process.
how can I evaluate whether coding is written well
There are various ways/metrics to define 'well'or 'good', for example:
Delivered on time
Delivered quickly
No bugs after delivery
Easy to install
Well documented
Runs quickly
Uses cheap hardware
Uses cheap software
Didn't cost much to write
Easy to administer
Easy to use
Easy to alter (i.e. add new features)
Easy to port to new hardware
...etc...
Of these, programmers tend to value "easy to alter": because, their job is to alter existing software.
Its a difficult one and could be where your non-functional requirements will help you
specify your performance requirements: transactions per second, response time, expected DB records over time,
require the delivery to include outcome from a performance analysis tool
specify the machine the application will be running on, you should not have to upgrade your hardware to run the app
For eyeballing the code and working out whether or not its well written its tougher, the answers from #Andrew & #Chris cover it pretty much... you want code that looks good, is easy to maintain and is performant.
Summary
Use Joel Test.
Why?
Thanks for tough question. I was about to write a long answer on merits of direct and indirect code evaluation, understanding your organisational context, perspective, figuring out a process and setting a criteria for code to be good enough, and then the difference between the code being perfect and just good enough which still might mean “very impressive”. I was about to refer to Steve McConnell’s Code Complete and even suggest delegating code audit to someone impartial you can trust, who is savvy enough business and programming-wise to get a grasp of the context, perspective, apply the criteria sensibly and report results neatly back to you. I was going to recommend looking at parts of UI that are normally out of end-user reach in the same way as one would be judging quality of cleaning by checking for dirt in hard-to-reach places.
Well, and then it struck me: what is the end goal? In most, but very few edge cowboy-coding scenarios, as a result of the audit you’re likely to discover that the code is better than junk, but certainly not damn good, maybe just slightly below the good enough mark. And then what is next? There are probably going to be a few choices:
Changing the supplier.
Insisting on the code being re-factored.
Leaving things as they are and from that point on demanding better code.
Unfortunately, none of the options is ideal or very good either. Having made an investment changing supplier is costly and quite risky: part of the software conceptual integrity will be lost, your company will have to, albeit indirectly, swallow the inevitable cost of the new supplier taking over the development and going through the learning curve (exactly opposite to that most suppliers are going to tell you to try and get their foot in the door). And there is going to be a big risk of missing the original deadlines.
The option of insisting on code re-factoring isn’t perfect either. There is going to be a question of cost and it’s very likely that for various contractual and historical reasons you won’t find yourself in a good negotiation position. In any case re-writing software is likely to affect deadlines and the organisation what couldn’t do the job right the first time is very unlikely to produce much better code on the second attempt. The latter is pertinent to the third option I would be dubious of any company producing a better code without some, often significant, organisational change. Leaving things as they are not good either: a piece of rotten code unless totally isolated is going to eventually poison the rest of the source.
This brings me to the actual conclusion, or in fact two:
Concentrate on picking the right software company in a first place, since going forward options are going to be somewhat constrained.
Make use of IT and management knowledge to pick a company that is focused on attracting and retaining good developers, that creates a working environment and culture fit for production of good quality code instead of relying on the post factum analysis.
It’s needless to expand on the importance of choosing the right company in the first place as opposed to summative evaluation of delivered project; hopefully the point is already made.
Well, how do we know the software company is right? Here I fully subscribe to the philosophy evangelised by Joel Spolsky: quality of software directly depends on quality of people involved which as it has been indicated by several studies can vary by an order of magnitude. And through the workings of free markets developers end up clustered in companies based on how much a particular company cares about attracting and retaining them.
As a general rule of life, best programmers end up working with the best, good with good, average with average and cowboy coders with other cowboy coders. However, there is a caveat. Most companies would have at least one or two very good developers they care about and try their hardest to retain. These devs are always put on a frontline: to fire fight, to lure a customer, to prove the organisation potential and competence. Working amongst more than average colleagues, overstretched between multiple projects, and being treated as royalty, sadly, these star programmers very often loose touch with the reality and become prima donnas who won’t “dirty” their hands with any actual programming work.
Unfortunately, programming talent doesn’t scale and it’s unlikely that the prima donna is going to work on your project past the initial phase designed to lure and lock in you as a customer. At the end the code is going to be produced by a less talented colleague and as a result you’ll get what you’ll get.
The solution is to look for a company there developer talents are more consistent and everyone is at least good enough to produce the right quality of code. And when it comes to choosing such an organisation that’s where Joel Test comes mighty handy. I believe it’s especially suitable for application by someone who has no programming experience but good understanding of IT and management.
The more points company scores on the Joel Test the more it’s likely to attract and retain good developers and most importantly provide them with the conditions to produce quality code. And since most great devs are actually in love with programming all the need is to be teamed up, given good and supportive work environment, a credible goal (or even better incredible) and they’ll start chucking out high quality code. It’s that simple.
Well, the only thing is that company that scores full twelve points on the Joel’s Test is likely to charge more than a sweatshop that scores a mere 3 or 5 (a self-estimated industry average). However, the benefits of having the synergy of efficient operations and bespoke trouble-free software that leverage strategic organisational goals will undoubtedly produce exceptional return on investment and overcome any hurdle rates by far outweighing any project costs. I mean, at the end of the day the company's work will likely be worth the money, every penny of it.
Also hope that someone will find this longish answer worthwhile.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I often find that I do a less than complete work on a feature, especially in the Design phase. I detect several reasons:
I'm over-optimistic
I feel the need to provide quick solutions, so sometimes I fool myself into thinking the design is fool-proof when in fact it's still full of holes, just to get the job done faster. Of course I end up paying dearly later.
I'm aware of this behavior of mine for some time, yet I still find I don't manage to compensate. Have you encountered similar problems? How do you approach solving them?
I use a couple of techniques. The first is a simple paper to-do list. In the morning I write down my tasks for the day. I try to work on a task until I can cross it off. I cross it off only when I'm done to my own satisfaction. My to-do list helps me stay focused. When an interruption comes in, I can consciously choose whether it is important enough to interrupt what I'm doing now.
The second technique I use is to give up on the idea of "done" for a design. Instead, I focus on what I've started calling "successions", where a design goes through predictable stages. Each stage supports the current functionality well and will be succeeded at some point by the next stage. This lets me do a good job, a job I can be proud of, without over-designing.
I have the intuition that there is a small catalog of such successions (like http://www.threeriversinstitute.org/FirstOneThenMany.html) that would cover most of design. In the meantime, I try to remember that "sufficient to the day are the troubles thereof".
I run into this problem a lot.
My solution is a notebook. (The old fashioned paper kind).
I write out how I'm planning on implementing the solution as an bulleted overview list, and then I try and flesh out each point on the list.
Often, during that process, I come across issues I hadn't thought of.
Of course, the 80/20 rule still applies... I still come across things when I'm actually doing the implementation that hadn't occurred to me, but with experience these tend to diminish.
EDIT: If I'm still not sure at the end of this process, I put together a throwaway prototype testbed... It's important to make sure it's throwaway, because otherwise you run the risk of including some nasty hacks in your real codebase.
It's very common to miss edge-cases and detail when you're in the planning phase of a project, especially in the software development field. Please don't feel that this is a personal failing; it's something endemic.
To counter this, many software development methodologies have emerged. Most recently there has been a shift by many development teams to 'agile' methods, where there is a focus on rapid development with little up-front technical design (after all, many complexities are only discovered when you actually begin developing). I'm currently using the Scrum system, which has been excellent in my small team:
http://en.wikipedia.org/wiki/Agile_methods
http://en.wikipedia.org/wiki/Scrum_%28development%29
If you find that your organisation will not accept what they may regard as a radical shift in approach, it may be worth investigating whether they will agree to the development of a prototype system. This means that you could code up a feature to investigate the technologies involved and judge whether it's feasible, without having to commit to full development, a quality bar, testing schedules etc. The prototype should be thrown away once the feasibility has been proved or disproved, then proper development may begin, including all that you've learned in the process.
If your problem is more related to time management, then I'd recommend the Getting Things Done approach (http://en.wikipedia.org/wiki/Getting_things_done). This is pragmatic and simple, concentrating on making you productive without overloading you with information that isn't immediately relevant to your current work. I've found that I get overwhelmed with project/feature ideas at times and it really helps to write everything down and file it for a later time when I have the resources available to work effectively.
I hope this helps and best of luck!
Communication.
The best way to not rush yourself into programming mistakes is communication. Yes, good ol' fashioned accountability. If another person in the office is involved in the process, the better the outcome. If a programmer just takes on the task without any concern for anybody else, then there is a higher possiblity for mistakes.
Accountability Checklist:
How do we support this?
Who needs to know what has changed?
Why are we doing this in the first place?
Will there be anybody who doesn't want this changed?
Will someone else understand how I did this?
How will the user perceive and use this change?
A skepticle comrad is usually good enough to help. Functional Specifications are good, they usually answer all of these thoughts. But, sometimes a conversation with another person can help you with it and you can get changes out the door faster.
I have learned, through years of mistakes (though still making them), that almost anything I want to use repeatedly, or distribute, needs to be designed properly. So getting burned enough times will end your optimism.
When getting pressure from management, I tell them I will have to put in the thought anyway, so I should do it when it's cheap. I think on paper as well, so I can actually prove that I'm doing something and it keeps my fingers on the keyboard, both of which provides a soothing effect to management. ;-)
At the risk of sounding obvious - be pessimistic. I had a few experiences where I thought "that should take a few hours" and it ended up taking a couple days because of all the little things that pop up unexpectedly.
By far the best way I've found to manage things is to (much like Andrew's answer) write out the design and requirements as a starting point. Then I go through and look for weak points in the design, gotchas and additional use cases etc. I try to look at this as a critical exercise - there's no code written yet, so this is the time to be totally ruthless and look for every weak point. Look for error conditions you'll have to handle, and whatever amount of time you think it will take to complete each feature/function, pad that amount by a lot. I've had times where I've doubled my initial estimate and still not been that far off the mark.
It's very hard as a programmer to realistically project debugging time - writing the code is easy to estimate, but debugging that into functioning, valid code is something else entirely. Therefore I find there's no exact science to it but I just pad tasks by a whole bunch, so that I have plenty of breathing room for debugging.
See also Evidence Based Scheduling which is a fascinating concept in scheduling developed by FogCreek for their FogBugz product.
You and the rest of the world.
You need more a more detailed design, more accurate estimate, and the willingness to accept that sometimes the optimal solution is not necessarily the best solution (e.g., you could code some loop in assembler to get optimal performance, but that's going to take a lot longer than just doing
for (i=1; i<=10; i++) {}
). Is the time spent doing it really worth it for an accounting package over a missile system.
I like to designing, but over time I've found that much design up front is a lot like building castles into the sky - it's too much speculation, however well-educated, missing critical feedback from actually implementing and using the design.
So today I'm much more into accepting that while implementing a design I will learn a lot of new stuff about it, and need to feed that learning back into the design. Doing that is a skill that is fun to learn, including the skills to keep a design flexible by keeping it simple, free of duplication and cohesive and decoupled, of changing the design in small, controlled steps (=refactoring), and writing the necessary extensive suite of automated tests that make this kind of changes safe.
This seems to be a much more effective approach to me than getting better at "up front design speculation" - and addtionally it makes me equally well prepared for the inevitable moment when the design needs to be changed due to a simply unforseeable change in the requirements.
Divide, divide, divide. List all the steps that will be required to finish the project, then list all the steps those steps will require to be concluded, and so on until you reach atomic items you are absolutely sure you can finish in a day or less. Add the duration of all these values to arrive at a length of time.
Then double it. Now you have a number that, if depressing, is at least somewhat realistic.
If possible "Sleep on your design" before publishing it. I find after I leave work, I usually think of things I have missed. This usually happens while I am lying in bed before falling asleep or even while showering the next day.
I also find it valuable to have a peer/friend that I trust review what I have before distributing it. Somebody else almost always sees something I didn't think of or miscommunicated.
I like to do as others stated here. Write down in pseudo code what the flow of your app will be. This immediately highlights some detailed areas that may require further attention that where not apparent up front.
Pseudo code is also readable to business users who can verify your approach meets their needs.
Using pseudo code also creates a nice set of methods that could be put to use as an interface in the final solution. Once the pseudo code is fairly tight, look for patterns and review some common GOF patterns. They do not have to be perfect but using them will sheild you from having to rewrite the code later during the revisions that are bound to come along.
Just taking an hour or two write psuedo code, yields some invaluable time saving pieces later on:
1. An object model emerges
2. The program's flow is clearly defined for others
3. It can be used as documentation of your design with some refinement
4. Comments are easier to add and will be clearer for someone else reviewing your code.
Best of luck to you!
I've found that the best way to make sure you've chosen a good design is to make sure that you understand the problem, know the limitations you have, and know what things are must-haves vs. nice-to-haves.
Understanding the problem will involve talking to the people who have the need and keeping them anchored to what needs to get done first instead of how they think it ought to get done. Once you know what actually has to happen, you can go back and talk over requirements about how.
Knowing your limitations may be quite easy: needs to run on the iPhone; has to be a web application; needs to integrate with the already-existing Java code and deployment setup; and so on. It may be quite difficult: you don't know what the potential size of your user base is (hundreds? thousands? millions?); you don't know whether you'll need to localize it (though if you're not sure, assume you will have to).
Must-haves vs, nice-to-haves: this is possibly the most difficult part. Users very often have emotional attachments to "requirements" ("It should look just like Excel") that are not actually part of the "has to happen" stuff. You often have to juggle functionality vs. desires to get an acceptable implementation. You can't always give everyone a pony.
Make sure you write all this down! Even if it evolves along the way, or the design is small, having a "this is what we're planning to do now" guide to refer to when you need ot make a decision about committing resources makes it easier to restrain yourself from implementing a really cool whiz-bang feature instead of a boring must-do.
Since you recognize that you feel the need to provide a quick solution, perhaps it will slow you down to realize that you can probably solve the problem faster and deliver it sooner if you spend more upfront time in design. For instance if you spend 3 hours designing and 30 hours writting code, it probably means that if you spend 6 hours designing you might need to only spend 10 hours writing code. (These are not actual figures just examples). You might try to quantify this for yourself on the next few projects you do. Do a couple where you behave as you normally would and see what ratio of design/codewriting/testing&debugging you actually do. Then on the next project deliberately increase the percentage of time you spend on design phase and see if it does shorten the time needed for the other phases. You will have to try for several projects on this as well to get a true baseline since the projects may be quite different. Do it as a test to see if you can improve your performance on the the other phases and thus deliver a faster product if you spend 20% more time or 50% more time or 100% more time on design.
Remember the later in the process you find the problem with a design the harder (and more time-consuming) it is to fix.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
When do you start to consider a code base to be getting too large and unwieldy?
-when a significant amount of your coding time is devoted to "where do I put this code?"
-when reasoning about side-effects starts to become really hard.
-when there's a significant amount of code that's just "in there", and nobody knows what it does or if it's still running but it's too scary to remove
-when lots of team members spend significant chunks of their time chasing down intermittent bugs caused by some empty string somewhere in the data where it wasn't expected, or something that you think would usually be caught in a well-written application, in some edge case
-when, in considering how to implement a new feature, "complete rewrite" starts to seem like a good answer
-when you dread looking at the mess of code you need to maintain and wish you could find work building something clean and logical instead of dumpster diving through the detritus of someone else's poorly organized thinking
When it's over 100 lines. Joke. This is probably the hardest question to answer, because it's very individual.
But if you structure the application well and use different layers for i.e. interfaces, data, services and front-end you will automaticly get a nice "base"-structure. Then you can dividie each layer into different classes and then inside the classes you point out the appropriet methods for the class.
However, there's not an "x amount of lines per method is bad" but think of it more like this, if there is possibility of replication, split it from the current peice and make it re-usable.
Re-using code is the basics of all good structure.
And splitting up into different layers will help the base to become more and more flexible and modular.
There exist some calculable metrics if that's what you're searching for. Static code analysis tools can help with that:
Here's one list: http://checkstyle.sourceforge.net/config_metrics.html
Other factors can be the time it takes to change/add something.
Other non-calculable factors can be
the risk associated to changes
the level intermingling of features.
if the documentation can keep up with the features / code
if the documentation represent the application.
the level of training needed.
the quantity of repeat instead of reuse.
Ah, the god-program anti-pattern.
When you can't remember at least the
outline of sections of it.
When you have to think about how
changes will affect itself or
dependencies.
When you can't remember all the
things it's dependant on or depend
on it.
When it takes more than a few
minutes(?) to download the source or
compile.
When you have to worry about how to
deploy new versions.
When you encounter classes which are
functionally identical to other
classes elsewhere in the app.
So many possible signs.
I think there are many thoughts to why some code base is too large.
It is hard to remain in a constant naming convention. If classes/methods/atributes can't be named consistently or can't be found consistently, then it's time to reorganize.
When your programmers are surfing the web and going to lunch in order to compile. Keeping compiling/linking time to a minimum is important for management. The last thing you want is a programmer to get distracted by twiddling their thumbs for too long.
When small changes start to affect many MANY other places of code. There is a benefit to consolidation of code, but there is also a cost. If a small change to fix one bug causes a dozen more, and this is commonly happens, then your code base needs to be spread out (versioned libraries) or possibly unconsolidated (yes, duplicate code).
If the learning curve of new programmers to the project is obviously longer than acceptable (usually 90 days), then your code base/training isn't set up right.
..There are many many more, I'm sure. If you think about it from these three perspectives:
Is it hard to support?
Is it hard to change?
Is it hard to learn?
...Then you will have an idea if your code fits the "large and unwieldy" category
For me, code becomes unwieldy when there's been a lot of changes made to the codebase that weren't planned for when the program was initially written or last refactored significantly. At this point, stuff starts to get fitted into the existing codebase in odd places for expediency and you start to get a lot of design artifacts that only make sense if you know the history of the implementation.
Short answer: it depends on the project.
Long answer:
A codebase doesn't have to be large to be unwieldy - spaghetti code can be written from line 1. So, there's not really a magic tripping point from good to bad - it's more of a spectrum of great <---> awful, and it takes daily effort to keep your codebase from heading in the wrong direction. What you generally need is a lead developer that has the ability to review others' code objectively, and keep an eye on the architecture and design of the code as a whole - no one line developer can do that.
When I can't remember what a class does or what other classes it uses off the top of my head. It's really more a function of my cognitive capacity coupled with the code complexity.
I was trying to think of a way of deciding based on how your collegues perceive it to be.
During my first week at a gig a few years ago, I said during a stand-up that I had been tracking a white rabbit around the ContainerManagerBean, the ContainerManagementBean and the ContextManagerBean (it makes me shudder just recalling these words!). At least two of the developers looked at their shoes and I could see them keeping in a snigger.
Right then and there, I knew that this was not a problem with my lack of familiarity with the codebase - all the developers perceived a problem with it.
If over years of development different people code change requests and bug fixes you will sooner or later get parts of code with duplicated functionality, very similar classes, some spaghetti etc.
This is mostly due to the fact that a fix is needed fast and the "new guy" doesn't know the code base. So he happily codes away something which is already there.
But if you have automatic checks in place checking the style, unit test code coverage and similar you can avoid some of it.
A lot of the things that people have identified as indicating problems don't really have to do with the raw size of the codebase, but rather its comprehensibility. How does size relate to comprehensibility? If at all...
I've seen very short programs that are just a mess -- easier to throw away and redo from scratch. I've also seen very large programs whose structure is transparent enough that it is comprehensible even at progressively more detailed views of it. And everything in between...
I think look at this question from the standpoint of an entire codebase is a good one, but it probably pays to work up from the bottom and look first at the comprehensibility of individual classes, to multi-class components, to subsystems, and finally up to an entire system. I would expect the answers at each level of detail to build on each other.
For my money, the simplest benchmark is this: Can you explain the essence of what X does in one sentence? Where X is some granularity of component, and you can assume an understanding of the levels immediately above and below the component.
When you come to need a utility method or class, and have no idea whether someone else has already implemented it or have any idea where to look for one.
Related: when several slightly different implementations of the same functionality exist, because each author was unaware of other authors' work.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
When is it the right time and when is it the wrong time to take the quick and dirty approach versus the proper elegant solution?
This started from the comments on my question: Decode Base64 data in Java.
I wanted a way to do something using only internal Java or something I wrote. The only built in way is to use the sun.* package which is unsupported. So when is it the right time and when is it the wrong time to take that approach?
In my opinion the only "right time" to use a quick and dirty solution (aka hack) is when...
There is a mission-critical bug that needs immediate fixing and the proper solution is significantly complicated to implement.
It's an internal application and some higher ups need a quick feature implemented (hey, neither you nor your customers use it).
In any situation, you have to balance the time that will be spent on solving the problem, and then the time that will be spent in maintaining that solution. If the lifetime of whatever you do is expected to be lengthy, then the extra time spent up front to do things the right way will, presumably, be easier to maintain and will save you in the long run.
However, if there are time constraints, the penalties for going over may far outweigh the maintenance cost. For example, if you MUST get a product out the door, do a quick-and-dirty solution now, and then log it as a bug to get fixed later in a patch, or the next version. That will require you to re-do the work, but like I said, it is all a balancing act, and often driven by cost at it's root.
The right answer is that its never time to do the quick and dirty solution.
But sometimes (when having a big customerbase and lots of programs out there) its better to ship a fix with a Q&D that maybe breaks something, but FOR SURE fix more.
I dont know if I understand the question right but Ill leave it with this.
If there exists a simple but less elegant solution that closely matches the style and techniques used elsewhere in the code base (and consequently, is more easily understood by your colleagues), it may be safer to take that path -- compared to the solution that is perfect from an academic point of view, but difficult to understand if you never happened to read that specific paper.
Similarly, a small change may be considerably safer than a major refactoring effort that the elegant solution would basically require. This is especially relevant in any world that lacks unit tests.
If the project is a "proof of concept" work rather than a production implementation, it may be the right time for a quick and dirty solution that gets the job done of illustrating how easy or hard some particular part is of a system. I'm thinking of where one may want to test out a new ERP or CRM system and just want to see the mechanics of it without spending a ton of time and money to get a more realistic prototype working.
In most other cases, it is a question of trade offs. How many defects can something have that gets released into the wild? Are just no showstoppers bugs OK or does there have to be few only a few minor or cosmetic bugs left. If some project managers or business units want something quick but not necessarily fully working then it the quick and dirty solution may work if those asking understand the risks. In a way this is like the question of how many tests should a doctor give before making a diagnosis: Should you be examined head to toe through a variety of different waves of energy, e.g. X-ray, CT scan, MRI, etc. Or can the doctor just look at what you are doing and know what is wrong like Dr. House does from time to time.
It seems to me that quick and dirty solution is more often necessitated by a lack of planning somewhere moreso than it is because of a sudden change of business rules (not to say that doesn't happen though!). Either someone wasn't given appropriate tools to do their job, someone vastly underestimated the amount of time something will take, or someone got too attached to a feature that is overly complicated. In most of those cases, it's best to push back your release or scale back the features in the release.
In the case that it was necessitated by a business priority that you have no control over (like a legal issue), then you need to get it out the door as quickly as possible. But you should make the fix better for the next release!
It's the right time to use a quick and dirty solution when:
You need the solution to be quick (assuming no professional solution exists that is approximately as quick), and
You don't suffer consequences for it being dirty.
In some ways, that's a flippant answer, but it does cover it.
In your situation, you can't introduce a new library quickly. And you do have a process to mitigate the risk of using experimental code. As long as that process is reliable, and you are confident that production code won't include your quick & dirty solution, then go ahead.
Once my company was struggling to get some code working for a demo at a trade show. They were running out of time, the demo just one day away. I suggested they hard-code some options instead of making them configurable and dynamic.
It worked -- they got the demo running for the next day's show. Then when they got back to the office, they immediately removed the hard-coded part and finished their feature.
If you can rely on replacing the quick & dirty hack being top priority, then quick & dirty can be appropriate. It's just so seldom the case that it's top priority, that using this solution is usually a bad idea.
As noted by tj111 and others, there are times when the risk of an elegant solution are greater than that of the quick and dirty. I.e., you deliver something beautiful, but by that time the customers no longer want it, want something different, or are pissed off.
To be sure though, there is always a long-term cost with doing a quick and dirty solution and it is important to deliver a firm and forceful "by the way" caveat to management. Else the quick and dirty becomes like crack to management who often do not know or care of the difference between the elegant and dirty (especially when it is under the hood). That is until things start to go to hell, and even then the fixitnow stress is offloaded to the hapless (often new) technical team.
So yeah, sometimes you have to do it because of $$ and time considerations, but don't cloak it or forget it and strive and push to re-factor soon when the crisis wanes.
Remember a lot Application get built off of their prototypes.
We always want to think its throw away code. But it will grow!
In my case I work in a research and development position where we have a long process for getting approval when adding external libraries.
If code is marked as experimental it will be cleaned up before moving to production. I needed this for a quick experiment to see if something was even going to make sense for us to work on and needed to get the data processing quickly. So the quick and dirty approach was the right one to get things done asap. If we decide to work on the experiment I will have time to replace the functionality properly.
In the standard case where as kdgregory put it in the comments of one answer:
Bzzt. In a professional environment,
using an unsupported, undocumented
feature is never the correct decision.
And in a corporate environment,
"experiments" become "production code"
with no chance to fix the hacks.
I think he is correct that it is not the appropriate approach when you work in a position where you have even a doubt that it may go into production. The only exception with this is if you absolutely do not have the time for a proper fix but can mark it as being a high priority issue in some visible public place (bug tracker) and will definitely be able to fix it in a future release.
Sometimes a quick and dirty solution is synonymous with an unmaintainable solution. Other times it just means solving the (simple) problem you actually have, not the (more complex) generalization of that problem. I'm a grad student doing research in bioinformatics and I often need to write small programs that do simple things like reformat or summarize a few hundred megs of data. Occasionally I need to write several of these apps per day. If I wrote "proper" solutions to these I'd never get anything done.
Solving the general case (not quick and dirty) would mean making an app that handled errors robustly, was usable by people other than me, was configurable, etc. This would solve lots of problems that I don't have. I'm probably the only one that will ever use these programs. All the configurability I need is a few command line options and the ability to edit the source file as needed. All the error handling I need is displaying a decent error message and exiting if there's a problem.
Of course, sometimes these programs do grow into something more important and sometimes I do end up reusing code from them. Maintainability counts. Even in a quick and dirty solution, magic numbers should not be hard coded, variables should not be named "foo", "bar", and "stuff", and programs should not be written as one giant 400-line main() function. I learned these the hard way.
Sometimes you have to do things in Q&D way. What you need to do in that circumstance is to protect yourself by hiding the dirty implementation behind an interface or an adapter.
To use the cited example, the Sun.* package works for now, but may not sometime in the future. Fine. Then wrap it in an adapter that implements the desired interface. Then, when Sun changes the API (hopefully providing something permanent like Java.BASE64), you can quickly update your implementation to cope with the change.
I do a quick and dirty fix when three conditions are met:
There isn't enough time to do a proper solution
The fix and it's scope are clear and contained (it's easy to come back, know what the fix addressed, can be easily removed and replaced by a proper implementation)
The fix is atomic or has no dependencies
"Quick and Dirty", is often synonymous with "cheap yet actually works", which is generally all you need to get the job done. If anybody criticizes your code, pretty much the best defence is to say "yeah- it was just a quick and dirty solution". That way your coder colleagues can feel superior to you, whilst you pointy haired boss looks on, rubbing his chin in silent approval.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I often find myself fighting overengineering -- the person in charge of designing the software comes up with an architecture that's, way, way overcomplicated.
It's all fine and dandy to have all the esoteric features that users will never know about and get that sense of achievement when you're doing something that all the magazine articles are telling you is the latest, cool thing, but we are going to spend half of our engineering time on this monument to our cleverness, and not, you know, the actual product that our users need and upper management expects to be completed within a reasonable or at least a bounded time frame.
And you'll probably just have to revert back to the simpler solution anyway when you start running out of time, that is, if you get that chance.
We've all heard the refrain: Keep It Simple, Stupid™.
How do you fight with overcomplexity in your team?
One example I've had to work with repeatedly lately is when the decision has been made to go to a fully denormalized database design rather than an RDBMS. "because it's faster!" Fully denormalized databases are really hard to get right, and are only appropriate for really specialized data problems like Flickr or ebay, and which can be extremely expensive in terms of developer time relative to the rest of your development.
Tooth and nail, and sometimes you lose. The problem is that it's always easy to be tempted to build something cool.
Why build something simple and efficient when it can be complicated and wonderful?
Try to remind people of the XP rule of building the simplest thing that can possibly work.
Make sure to bounce any ideas you have off of someone else. Oftentimes, we get so wrapped up in doing things a certain way that it takes another set of eyes to set you right. There've been many times that I've figured out difficult problems by having somebody else there to say "do we really need that?" This helps make my code simpler.
On the point of dealing with people you disagree with, Ward Cunningham has a good point:
It was a turning point in my programming career when I realized that I didn't have to win every argument. I'd be talking about code with someone, and I'd say, "I think the best way to do it is A." And they'd say, "I think the best way to do it is B. I'd say, "Well no, it's really A." And they'd say, "Well, we want to do B." It was a turning point for me when I could say, "Fine. Do B. It's not going to hurt us that much if I'm wrong. It's not going to hurt us that much if I'm right and you do B, because, we can correct mistakes. So lets find out if it's a mistake. ... Usually it turns out to be C. It's a learning experience for both of us. If we decide without the experience, neither of us really learns. Ward won, and somebody else didn't. Or vice versa. It's too much of a battle. Why not say, "Well, let's just code it up and see what happens. If it doesn't work, we'll change it.""
My advice? If you want to do something better, come up with a simple prototype that demonstrates that it's better. Opinions are great, but code talks.
I have seen this formula somewhere:
skill = complexity of problem / complexity of solution http://img39.imageshack.us/img39/1586/whatisskill.png
In other words, it requires skill to create a simple solution to a complex problem. If somebody purposefully designs and takes pride in complex overengineered solutions, then he is unconsciously incompetent.
Personally, what helps me to keep my designs simple, is the TDD cycle. First write a test that specifies what you're trying to reach, and then produce "the simplest thing that could possibly work". And every now and then, reflect on what you have produced, and think about how to make it more simple.
Never build extra flexibility and abstraction layers into the system, until it is required by something that you have now. Changing the code is easy, when you have a good unit test suite, so you can add those abstraction layers later, when the need arises, if it ever arises. Otherwise, "you ain't gonna need it".
Some symptoms of too complex design are when writing tests is complicated. If the tests require a long setup code, maybe you have too many dependencies or in some other way too much complexity. If you run into concurrency bugs, then maybe you should think about how to design the system so that concurrency is restricted to the absolute minimum number of classes. Maybe use a message-passing architecture, such as the Actor model, and make practically every component single-threaded, even though the system as a whole is multi-threaded.
At least for me, the bigger issue is that it's often hard to tell what feature is in there because of its buzzword-friendly, magaziney enterprisey goodness and which is in there because it adds a level of flexibility that will be useful in the future.
It's been shown that people are generally terrible at anticipating future complexity, and side-effects of current decisions. Unfortunately this doesn't always mean simplest is best - in my case, there've been loads of things I thought were too complicated at first and didn't see the value of until much later (er... spring). Also things I thought made sense that turned out to be wildly overcomplicated (EJB1). So I know that my intuition about these things is faulty.
Best bet - any kind of indirection layer should be supported with an argument supporting the value of the flexibility it adds vs. its added dev complexity.
However, people who are dogmatically maintaining a particular db setup on abstract grounds are probably in the "building it because I read that it's the right thing" camp. It might be unrealistic, but some people might be convinced if you build a test version and benchmark, especially if the results show more effort leading to an insignificant performance increase.
It's all fine and dandy to have all
the esoteric features that users will
never know about and...
This would be feature creep, not unnecessarily complicated design. It's different from your example on databases.
One example I've had to work with
repeatedly lately is when the decision
has been made to go to a fully
denormalized database design rather
than an RDBMS. "because it's faster!"
In this case several things may be going on. One of them is, you might be wrong and these people could really know what they are saying because they have worked with very similar examples. Another is they might be wrong, i.e. their design doesn't offer the speed advantages they claim. In this case there could be two different situations: (1) They are giving speed too much weight in their design, or (2) speed is really critical. If speed is indeed so relevant, the team shouldn't rely only in assumptions - they should try different prototypes and evaluate their speed in the critical paths. You don't build an F1 car in one way just "because it's faster", instead you keep trying several alternative design solutions and pick the fastest one which still doesn't increase maintenance costs too much.
Sometimes you can argue it and reach an agreement, sometimes you can't. It's life.
A final word, though. You don't fight complexity. You treat it. You identify the really important things and act accordingly.
I assume you mean "fully denormalized database design rather than a normalized (e.g., third or fourth normal form) model", because a relational model is managed by an RDBMS regardless of how normalized it is.
Can't judge without knowing more about your requirements, your abilities, and those of your teammates.
I fear that your KISS admonition might not work in this case, because one big, denormalized table might be defended as the simplest thing possible.
How does anybody solve these kinds of problems? Communication, persuasion, better data, prototypes of alternative technologies and techniques. This is what makes software development so hard. If there was only one way to do these things, and everyone agreed on them, we truly could script it or get anyone to develop systems and be done with it.
Get some data. Your words might not be enough. If a quick prototype can demonstrate your point, create it.
"Strong opinions, lightly held" should be your motto.
Sometimes a technical point isn't worth alienating your entire team. Sometimes it is. Your call.
You fight someone else's overdesign/feature creep in several ways:
Request feature priority, based on actual user requirements. Mock up features for alpha and beta testers, and ask if they would trade N months of delay for it.
Aggressively refactor to avoid special-casing. Break code into layers or modular components whenever appropriate. Find a balance between "works just fine now" and "easy to extend later".
Notify your management when you disagree with design decisions, prepare to be overruled, and accept the decision. Don't go over anyone's head or sabotage code.
The best way I have found is to relentlessly ask - again and again - 'What is the business problem we are trying to solve' and 'How does this decision help to solve that problem'.
I find that too often folks jump to solutions instead of being crystal clear on what the problem is.
So in your example of how to organize a database, my question would be 'What do we think are the transaction requirements for this project today, next month, next year, five years from now'. It could be that it makes sense to spend a lot of time to get the data model right, it could be a waste of time. You don't know what the parameters are until you get the problem definition clear.
You may suffer from "too many architects in the team" syndrome. One or two people at most should design/architect a system which will be coded by a team of 5 to 10 people. Input is welcome from everyone but architectural decision makers should be few and experienced.
(the numbers are semi-random and could be different depending on other factors as well)
I try to be open when discussing matters. But when i am discussing with someone else between something that seems simple and another one complicated, i get as stubborn as can be. It helps quite a lot, so long as you are very coherent from one decision to another.
Your example isn't really a complicated design, it's a design choice that you don't agree with. Since you're working on the code, you could easily be right because many of these decisions are made by people reading an article in an article and thinking it sounds like a good goal, or the decision could have been made by someone who ran into problems before and was trying to prevent it from happening again.
Personally I've done a lot of stuff the easy way and a lot of it the hard way, and I'm never happy when I choose doing something the easy way over the hard way. Now I've learned tricks like "never pass around a naked collection, always wrap it in a business class".
If I were to explain the reasoning behind that to someone who hadn't been through the same experiences, they wouldn't understand it until they tried comparing the "easy way" to the "hard way" a few times.
The solution should be no more complex than the problem.
The question intertwines itself with the thought of essential complexity. A sort must touch each element, by its essence. How much more complex must it then get, to solve the problem, given the technical constraints existing on it?
Do the people involved have enough time and incentive to find a simple solution? Without care, complexity will increase. If you spend most of your time trying to do the quickest possible bug fix or feature addition then saying "keep it simple" will not be enough.
Ensure that there are some people on the team with war wounds from maintaining large programs, and people with experience of refactoring, and then give them time to sort the software out. If you can arrange that certain features and opportunities are out of scope, that will help people remove unneeded code. If you want a metric, aim to reduce lines of code; but try not to obsess over it. Improve the test coverage; consider eliminating bits of code which are hard to test.
Don't try to do everything at a stretch. Break every problem/task into manageable chunks. Then prioritize, keeping KISS and YAGNI in mind. This will help you focus on building what you need. If you've done it right, you'll have a good core you can add to later, given time, money, resources and inspiration.