Many poeple have online startups in their head that may potentially attracts millions, but most of the time you will only have minimal budget (time and resource) to start with so you want to have it delivered within a year's time. Short after launch, you are bound to perform one or a series of upgrades that may include: code refactor to newer foundation, adding hierarchy(ies) in software architecture or restructure database(s). This cycle of upgrade/refactor continues as:
New features avaiable in latest version of the language(s)/framework(s) you use.
Availability of new components/frameworks/plugins that may potentially improve the product.
Requirement has changes it's direction, existing product wasn't designed to cope with new needs.
With above as prerequisite, I want to take this discussion serious and identify the essence of an upgradable solution for a web application. In the discussion you may talk about any stages of development (initial, early upgrade, incremental upgardes) and cover one of more of the following:
Choice of language(s) for a web application.
Decision for using a framework or not? (Consider the overhead)
Choice of DBMS and its design
Choice of hardware(s) and setups?
Strategy to constant changes in requirements (, which can be a natural of web application)
Strategy/decision toward total redesign
Our company's web solution is on its fourth major generation, having evolved considerably over the past 8 years. The most recent generation introduced a broad variety of constructs to help with exactly this task as it was becoming unwieldy to update the previous generation based on new customer demands. Thus, I spent quite a bit of time in 2009 thinking about exactly this problem.
The single most valuable thing you can do is to employ an Agile approach to building software. In particular, you should maintain an environment in which a new build can be (and is) created daily. While daily builds are only one aspect of Agile, this is the practice that is most important in addressing your question. While this isn't the same thing as upgradeability, per se, it nonetheless introduces a discipline into the process that helps reduce the chance that your code base will become unwieldy (or that you'll become an Architect Astronaut).
As far as frameworks and languages go, there are two primary requirements: that the framework be long-lived and stable and that the environment support a Separation of Concerns. ASP.NET has worked well for me in this regard: it has evolved in a rational manner and without discontinuities that invalidate older code. I use a separate Business Logic Layer to manage SoC but ASP.NET does now support MVC development as well. In contrast, I came to dislike PHP after a few months working with it because it just seemed to encourage messy practices that would endanger future upgrades.
With respect to DBMS selection, any modern RDMS (SQL Server, MySQL, Oracle) would serve you well. Here is the key though: you will need to maintain DDL scripts for managing upgrades. It is just a fact of life. So, how do you make this a tractable process? The single most valuable tool from any third-party developer is my copy of SQL Compare from Red Gate. This process used to be a complete nightmare and a significant drag on my ability to evolve my code until I found this tool. So, the generic recommendation is to use a database for which a tool exists to compare database structures. SQL Server is just very fortunate in this regard.
Hardware is almost a don't care. You can always move to new hardware as long as your development process includes with a reasonable release build process.
Strategy for constant changes in requirements. Again, see Agile. I'd encourage you not to even think of them as "requirements" any more - in the traditional sense of a large document filled with specifications. Agile changes that in important ways. I don't keep a requirements document either except when working on contract for an external, paying customer so that I can be assured of appropriate billing and prevent feature creep. At this point, our internal process is so rapid and fluid that the reports from our feature request/bug management software (FogBugz if you want to know) serves as our documentation when documenting a new release for marketing.
The strategy/decision for total redesign is: don't. If you put a reasonable degree of thought into the process you'll be using, choose mainstream tools, and enforce a Separation of Concerns then nothing short of a complete abandonment of HTTP and RDBMSs should cause a total redesign.
If you are Agile enough that anything can change, you are unlikely to ever be in a position where everything must change.
To get the ball rolling, I'd have thought a language/framework that supports the concept of dependency injection (or Inversion of Control as is seems to be called these days) would be high on the list.
You will find out that RDBMS technology is not easily scalable. All vendors will tell you otherwise yet when you try multiple servers and load-balancing the inherent limitations will show up. Everything else can be beefed up with "bigger iron" and may be more efficient code but Databases cannot be split and distributed easily.
Web applications will hopefully drive the innovation in database technologies and help us break out of the archaic Relational Model mind-set. It is long overdue.
I recommend paying a lot of attention to this weak link right from the start.
Related
I'm a fresh graduate and having trouble which is the best way to create a web project if branching or single web project with many conditions.
FOR EXAMPLE: I have multiple clients with different insight about our system. Some clients want to use certain features while others wont.
SOLUTION 1: Create each branch for each client to separate their project buildup
PROB. ENCOUNTERED: If I have multiple clients who wants the same features it would be very tedious to repeat the programs.
SOLUTION 2: Only have a single project that have many setup conditions to activate feature depending on the clients need
PROB. ENCOUNTERED: This will be slower because each feature that can be unique for each client should have a program condition. It is also much harder to debug in this solution.
Probably the easiest way is to use a plugin style system. If you define it correctly it will make for updating the system easy if there was a bug in one of the features that a client desires. Now you can still ship the program with all the features, but in a config file you could turn features on and off. Now this will require you having a good logging system in place. Both of these systems require a fair amount of research.
Any sort of branching as you suggest is a bad idea. I would recommend that you use Git instead of TFVC in TFS and use Git flow (http://nkdalm.net/Git-Flow) for your branching strategy.
As long as your branches are short lived (hours not days) they can be useful. If you end up with a branch for each customer you will end up with too much technical cruft and ultimately drowning in complexity. This is not a pleasant place to be.
You should invest in, and implement, a good feature toggle system that you can use to enable and disable features for customers. Where you hit an identical algorithm or behaviour that needs to be different per customer you should implement a switch possibly with dependency injection or plugins.
If you use good engineering practices, practice some form of test first, and maintain working code at all times, you should never need more than a single long term branch.
On the software development projects that you have worked on, what has been the approximate cost (expressed as a percentage of total system cost) of system integration? System integration includes integrating with other software, databases, etc.
33.3% because system integration is usually associated with a fair amount of risk that is not as prevalent in other phases of the projects (coding, documentation, etc).
This is a very difficult value to estimate, especially when you are facing integrating with a system that you are not familiar with. The best you can do is track you or your team's past performance on similar projects and use those values to try to estimate how you will perform on new projects.
Generally, system integration will take longer if:
It uses a protocol, database engine, operating system, etc. that you or your team have not yet worked with.
Vendor or community support is lacking or unresponsive.
Official system documentation is not detailed enough or is out of date.
The system does not have large global market share. Such a system will not have a wide user base and a big footprint in online programming Q&A sites such as this one. This may include new, less popular, or highly domain-bound systems.
Between 0 and 99%. I have built systems with no integration at all and systems that were basically just integration of other systems. The nice thing about integration can be that it is easy to estimate. But only when the interface is fully understood. Then it is just a duplication of functionality.
There are some complicating factors, though. They can make it very expensive to impossible:
is the system you have to integrate with well understood (do the programmers who developed it still work there?)
is the system you have to integrate with well-refactored (and has automated unit and acceptance tests)?
single or multiple platform?
are domain experts available?
It depends on the integrated system's importance and other factors.
I've worked in systems with integration in a bunch of web services that were the application's core. If the web services were down, our system was simply useless.
I would list the following variables when trying to evaluate the cost:
How many systems do you integrate and how frequently are they changed?
Do you have documentation to these systems?
Is it a third party component/service that you have no control of?
If you have control over the integrated system, does it use too much "legacy" code, like COBOL; (just an example, at least where I work COBOL programmers are expensive);
Are your employees experienced with the integrated system and with the application itself?
In case of failure of the integrated service, what is the impact on your application?
How much is an employee's hour rate in these scenarios? How many hours they would need to work on these integrated systems? How much money do you have for your project? I can't say it's going to cost X% on your case without knowing these details, specially the last one.
Is WF used at user interface or business layer? If at UI layer, then does someone coding in business layer even need to use or learn it?
Chris offered a great answer, so I'll take a different tack. Workflow is really not the same as logic. Workflows are generally compositions of logic, which makes them a complimentary contributor to the whole. Where a business component nicely encapsulates a single piece of business rules or behavior, a workflow composes multiple business components, rules, and behavior to meet the needs of larger, longer-running business processes.
Workflows can fit anywhere, filling process needs for UI, business, and even data layers. If you do end up using WF, I highly recommend that you train your general development staff, so that there is broad knowledge and understanding. Despite having a visual aspect to it, WF still requires some low-level coding to build up a library of activities and such that workflow creators and maintainers will use.
EDIT:
Its up to you whether to use it or not. Generally, WF is used by larger projects, because WF offers a valuable return on investment. WF is harder to use than simply writing code. There is a fairly steep learning curve, requires a different way of approaching business problems, and requires special skill sets. Providing code to WF adds overhead, as you have to write activities that may be used in workflows, as well as build the workflows themselves.
If you think that using WF for your small project will provide a valuable return on investment, then go for it. You will need to both recoup the initial costs of training, infrastructural implementation, and deployment, as well as providing long-term, tangible cost savings over simply writing code to solve the problems at hand. I would be doubtful that WF will offer a small project such ROI. For larger projects, or multiple integrated projects for a single large company, it is much easier to realize that ROI than it is with smaller projects, which is why workflow platforms like WF are usually only used on that scale.
WF is the interface to the buisness layer.
Workflows are a declarative way of writing programs by linking together pre-defined activities rather than an imperative programming model of writing lines of code. IN addition, you have many graphical tool that allow you to wrtie workflows. shuch as black pearl, K2 and sharepoint designer.
They are very much used and are well worth learning. I started using then in sharepoint and ms crm. I now always look to workflows to solve my common problems.
here are a few links:
mirosoft msdn.microsoft.com/en-us/netframework/default.aspx
Wkik: http://en.wikipedia.org/wiki/Windows_Workflow_Foundation
WF is mostly understood quite imperatively - a sequence of chronological events.
However, if you delve deeper you can combine activities in different ways. A good example are the state-based workflows that are usually also shown when presenting WF. WF allows you to put the Workflow into an idle mode: It's current state is persisted and reloaded once e.g. an external event takes place. As such, workflows can be useful in tracking long-running interactions where the system has to wait e.g. for some external process to end or some user to interact with the system.
I'd say that WF wouldn't be terribly useful in the UI context. They are better suited as a backend module to express processes integrating several services that have a tendency towards changing in time.
From my point of view, though, it is noticable that WF is a version 1 product: There are a number of awkward things that may guide you into code that is hard to maintain, parts of the infrastructure are quite complex to be used, you may find some API inconsistencies here & there.
There is also a guideline out there what you shouldn't do if you ever want to port your workflows to the next generation of WF.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
Let's say you work for a huge company which suddenly decides to do custom in-house software development. Additionally, they want to be able to offer successful developments to their customers as well (if any).
Now you are in charge of it.
What would you see as most important to build a successful software development infrastructure?
flexible to future growth
flexible on used technologies (projects with c, java, .net, web, mobile, ...)
What kind of tools (source control, forge, ...), hardware (virtual, seperate dev & production, ..), processes (guidelines, code reviews, ...), etc.
UPDATE: Please don't answer that you need the right people and the right tools. This is exactly what i am looking for.. What are the right tools and what people of what type would you hire first to join your team? Think of it as you will be the lead of that development.
Set yourself up to pass the Joel Test with at least a score of 10.
I think having the right people is going to be the most important. Nothing else will matter if your programmers stink.
Someone in charge who knows what they're doing.
Obviously, there are lots of factors, but here are the ones I'd say are crucial:
Hire smart people (and pay them what they're worth)
Select good tools appropriate for the kind of development (don't go for cheap tools)
Establish version control system and policies
Establish testing mechanisms and policies
Don't be afraid to outsource the stuff you don't know how to do
Get the best people for the job. If they aren't willing to pay for the best available, or give you a hard time over your personnel budget, you're off to a bad start.
Get the right tools for the job... software, hardware, support contracts from your vendors, etc.
Establish procedure early on for your development life cycle, and make sure that you have the people in place to make use of it. This is everything from how you evaluate Opportunity Assessments to Development, Testing, and post-production support. Make sure you have the people and the tools for each part of the life cycle.
Dont try to be flexible in technologies. First start by focusing on one technology (Java, .NET, whatever...) and then move to other if you need to. You will be able to solve problems using any technologies, but it is very hard to find people good in many technologies.
At the infrastructure level, Source Control is a must. Continuous integration is a plus. Take time to put in place a standard project layout that you will be able to evolve. It make it easier for developers to switch projects. Take time to put in place a good build process (Ant, Maven, in the Java world). Integrate your build process with your IDE so that developers dont have to wait 5 minutes to deploy their project every time they want to test a code change.
I agree with Guillaume: If you want to build a department from scratch, you need to focus. You need to build your team, have everybody grow into their new responsibilities, get to know each other etc. Trying to go into too many directions at once is the direction towards failure.
So, identify the technology you want to develop in. Since the primary goal in your example is in-house development, the in-house requirements will determine your decision. Build your team with that primary goal in mind.
For in-house development, you need at least two people who already know the company and its processes. (Two because one will definitely be ill or on holidays when the first major crisis hits you). On the other hand you need some outsiders, who are not entrenched by the "we have always done it like this" mindset, who can think out of the box. Those should also be at least two people, for the reason stated above. Your job as the team leader is to balance those two groups and integrate them into a team.
For future growth, always think in terms of organic growth.
Do not increase the team size by 200 %, hire one new guy here and another guy (or gal) there. Slowly build your team.
When you take on a new project, always think of expanding your teams expertise. Try something new with every project. That can be a new source repository, an automated daily build process, a new system to write specifications or documentation, or even a different technology (for example Java when you usually develop in .Net, Delphi or C++). Just make certain you never try to make a big leap in an important project. (I once worked for a company who decided to switch from VB 6.0 to .Net for the biggest project they had ever attempted before. They survived. Barely.)
That way your department will slowly but constantly expand its capabilities. Then when the opportunity presents itself to do development for an external customer, you will already have accumulated most of the knowledge you need in order to pull it off.
Oh yes, and smacl is right, too: You need solid QA/QM if you want your department to survive long term.
Start laying out (and follwing) your QA rules from day one. Keep them as short and flexible as possible. Add what you discover to be missing, and throw out what proves to be unnecessary or impractical.
Not sure this is what you wanted to know, but I felt the need to say it ;-)
Develop a strong QA strategy, including acceptance criteria and change control. Preferably keeping it lightweight to suit internal clients. In addition understand how to carry out requirements analysis, expectation management, and resource management.
Put another way, don't just wing it to create crappy solutions that waste more time than they save and are impossible to maintain. Take time to think about what you want and need, how you can achieve it, and what it is going to cost.
I will offer an answer more focused specifically on coding and the developers / architects role in addition to the previous answers on teams, version control, qa etc. which are of course all important.
Many of your decision is very dependant on your specific business and software structure (a single product code base, SOA, many projects etc.) But in general you should always spend significant time up front developing Core Software Infrastrcuture that will pay huge dividends during the SDLC.
Software infrastruture
Coding Naming Conventions Exception
Handling strategies Logging
Strategies Settings and Configuration
Base classes and Helper Classes
General Architecture and Layers
(Presentation, Facade, Domain
Entities, Data Stores etc.)
Design Tools such as UML 2.0
Requirements
Management / End user interaction
There are tons more, but these are certainly some basics to think about. All of the successful projects I have been involved with incorporated decent software infrastructure. I will also note that many of the project that fail have a common theme... lack of a common infrastructure in place. In most cases these failed projects are lead by a non-technical person that think they can simply throw a bunch of ideas at a few programmers and expect them to deliver in a few weeks.
Bottom line, you need to invest some up front planning and prototyping to ensure success in the long term!
Good luck.
Raiford
www.blacksaber.com
The first persons you should hire should be experienced senior level professionals. Then build up from them / with their input. Add the junior people later.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 14 years ago.
Improve this question
I've seen a lot of questions and discussions about build vs. buy, but most stick with the simplistic approach that you can simply do one or the other. Most of the time you have to either buy and integrate or build yourself. Either way you're in for some work.
In the next 30-60 days I NEED to implement a couple managerial projects to keep everyone from ripping their hair out and killing each other. The largest of which is a ticketing system (emails, support requests, self service, etc.).
There is no shortage of options but at the end of the day we'll have to buy whatever we decide to use, add all our clients and their users and make sure we keep things in sync over time. We'll also have to provide a single sign-on and do some design work to make it all look like we built it from scratch.
If we build we get to skip the integration pain points, albeit with a limited (but focused) feature set.
What do you typically analyze while making a decision like this? If it better to have 4-5 systems that do a very specific job well, or one monolithic system that does everything?
You've identified a key issue - when you buy you still have work to do, and potentially lots of it. Having said that my overall leaning every time is towards buy. Writing code is hard, debugging code is much harder - when you buy, you're not just buying the code/application you're buying the fact that it works - the latter is 90% of the benefit.
However, as your needs are pretty common, why not go with open source. This has two stand out benefits.
1) As you have access to the source, you can bend it to your will - ie no need to lash single sign on over the top of an existing system. Tailor the login modules to use your already existing infrastructure, therefore no need to keep things in sync, time savings, clean approach etc etc. Much open source acknowledges the real world by componentising (?) those aspects which are environement specific anyway. They're often DB/Identity agnostic.
2) If you choose wisely you will have a ready band of top tech staff who already understand the system ready to help - the only problem is they don't work for you (yet!).
My advice would be pick one of your easy targets - the ticketing system seems like the one, analyse whats out there that in the open source world that meets most/all of your needs. Evaluate and put out a request on Rent A Coder for any changes that are required. Sit back and await the results, which are hopefully excellent. You've lost a little time, and gained a lot of experience.
Open source does not equal Linux/Unix - lots of good stuff for .Net out there too.
One system is better for the following:
One data repository(i.e. the Database)
Easy way to link each system together, do cross referencing. No need to build intermediate importer/exporters/sync-ers
Allows for single log in. This is very useful in businesses to make sure everyone know where to find the right information. So more "what was the site for the bug tracking again..." Not everyone will use all the tools the majority of the time, and they will forget how to access and even use.
Everything has the same look and feel
Saves on training
Maintenance is cheaper. Everything is the same to update. Admins dont have to specialize in hear separate system.
But... obviously you're stuck with what you buy. Make sure to get a system if you can that you can build your own addins for, to match it to your busienss' model.
Obviously "it depends." My general rule is that if it's internal we buy it and integrate if required. Our corporate sys admin has a support line to someone external to our organization if she has issues and it isn't a huge project burdening our developers.
If it's part of a product I'm shipping, I build it or take bits of source as needed from open source libraries. There's nothing worse than someone else's black box code breaking your product. The fewer the dependencies in a shipping product the better, IMHO.
I'd lean toward buy for a support product like you mention. The good ones offer great integration points to shared authentication systems, user facing theming, and probably a boatload of features your customer service team hasn't realized they want/need yet.
But, what to analyze. The biggest thing for me when it comes to 'managerial' projects like this is opportunity cost. What else could my team be working on that will make our company significantly more money, get us more customers, etc? Of course these projects have some positive impact on the bottom line, but nothing compared to new products, improved products, etc. How long over time, including maintenance, will developers/pm's/testers spend on this managerial project? If you buy, integration points don't change often, but if you build, your customers (in house people) will be asking for new features constantly and you'll be in the position to maintain this project for the rest of your tenure.
Buy? What is this buy of which you speak, stranger?
Seriously, I haven't had to buy a piece of software for my own projects for a long long time. All my development tools are free, all my third-party libraries are free (not GPL). Even my OS is free. I have to pay for Windows for testing purposes but the majority of work uses tools that are cross-platform.
Anything that requires code not immediately available from free tools or libraries, I either write from scratch (all the algorithms are available for free on the web) or use my (huge since I'm so old) snippet library which I've been adding source code to for many years.
It's almost always quicker to buy ("obtain") than build unless the bought stuff is so crappy that integration is a nightmare. This can be mitigated by avoiding the latest whizz-bang stuff from suppliers that have little track record.
The more 'standard' your requirements, the better buying fits (Or to put it another way, don't reinvent the wheel). Conversely, the more unique your requirements the more you might consider building.
You quite rightly point out that even when buying there tends to be some customisation. Bear in mind that any customisation will cost you at each upgrade/patching time. I suggest that if your requirements are close to the business model supported by one of the tools you might buy that you serious consider realigning the business process to the vendors standard. If this is not possible ask if you are buying the correct tool.
I would suggest that if someone suggests building it for cost reasons run screaming. In my experience the cost of buying is well known and the cost of build is well hidden. Remember that you will be making a decision to keep coding for the life of the App (average of 7 years for a business app) but may be considering only the initial development cost when deciding between buy and build.
I have a strong preference for a single monolithic database but sometimes this is not workable. More important is to have a 'single source of the truth'; if you have multiple databases holding like data, pick one as the authoritative source of a given piece of data and have a process to maintain all others in agreement with that source. Preferable this will be automatic.
The monolithic system that does everything is the the Raison d'être for so many Enterprise applications. What I've found, however, is that if you're not willing to pay a buttload of money, you're going to have integration issues.
The 'best' solution is quite subjective, and any answer is as right as it is wrong, but if I were king, I'd probably go with the entrenched open source solution where it fit, and wrap web services around the items that needed to talk to each other. If I were king.
As a tangential point, there are free ticketing systems like RT (et. al.) that you need not worry about buying.