Thanks for all the questions and responses posted on here. This site usually shows up whenever I search for information from google, and in many cases, the answers are usually relevant to the issues I needed solved.
I want to preface my question by stating that I've been programming (.NET, XML, T-SQL, AJAX, etc) for less than 2 years, and I still have a lot to learn; so, pardon my ignorance.
Here's my situation (and question): I'm building a social web application, which I know will have much traffic in a short time; as a result,
What are the basic information that I need to have, in order not to be overwhelmed? It's currently a one-man affair, and here is the hosting specification that I plan to start with: 2GB RAM, 600 HDD, 1000 GB bandwidth, and 2.13GHz Duo Core Processor.
I've read about web-farms, but I've never had an opportunity to use them, so I'm not entirely sure how to phrase this question: how can one split the same application on multiple physical servers? How do you make all the files act as one entity? And since every .net application requires a web.config, how is it split among the various files on these multiple servers?
I've built smaller projects before, but this is the first big project I'm building, and to be frank, I'm a little intimidated. So, I would like to ensure I know what I'm getting into before starting.
Thank you.
Based on your background I assume you are developing in a .Net environment? If so, I highly recommend you take a look at Windows Azure. Developing your app against Azure will allow you to deploy your app in Microsoft's cloud platform. Once deployed you can shrink and grow your resources according to demand without having to deal with the relative hassle of setting up multiple servers in multiple locations and managing it all. This allows you to pay for a "little bit" of server up front and if your app gets popular you can easily pay for "web farm" like power and geographic diversity. It also gives you a decent framework for developing an app that will scale relatively well. That's an 18,000-feet overview. If you can put some more details in your question I'm sure you will get more detailed responses. Best of luck!
Your "social web application" will not have any users if it isn't working and deployed. Don't worry about scaling much until the site actually does something useful and has a few hundred users (or at least a few dozen!). Get it working, find people around you who can help when the going gets tough, and keep at it. Otherwise your concerns about needing to scale will never be warranted.
Related
Openshift pricing model states that you can have 3 gears in the free tier.
Other services normally explain their free tiers in number of "web workers" that you can have.
What is an openshift's gear exactly then? I know that you can install a different programming environment in each gear, but if you install the same one (let's say: ruby) in all your 3 free-tier gears, do you have 3 web-workers running at the same time? (As in: improving scalability and redundancy; are they load-balanced?)
{disclosure: I work as an Developer Evangelist on OpenShift}
We also put a big effort into explaining our pricing in terms developers can understand. Please look at the middle of the pricing page:
https://www.openshift.com/products/pricing
There is a section on how gears work. In that section you can see we can show what we load tested to give you a good starting point for what resources a normal Drupal application should be able to use. I find this easier than how many opteron servers I get.
Remember this is just an estimate, and your mileage may vary, but I think it is easier to understand.
We would love to see other people do load testing with other frameworks or applications and give us their feedback.
Gears can be thought of as equivalents to "web workers" however gears is better described as a unit (collection of resources).
In the OpenShift free tier you only have small gears (512MB memory, 1GB HDD, etc.) So the small gear is a unit of resources that you have for you to use.
https://www.openshift.com/pricing/index.html
If you want an application that scales (using the free tier), it can be done however some find it difficult as DB often get it's own GEAR, which only leaves you one remaining gear for scaling.
I'm one of those classic native programmers that has spent most of his past with .exe's and .jar's. As of the past year I've thrown my self into the world of web frameworks and technologies that seize to impress me. As of the past 1½ month I have fallen In love with Go because of it's strictness, and also how 'stand-alone' it seems to be. So now to the real question...
Go app engine application, why do we need this?
What is the difference and reasoning to choose a wrapped application (framework)?
I assume its purpose is to load some of the communication off the application to the wrapper, but sadly I can't seem to figure out (through documentation and discussion) what the specific purpose behind this modulation is.
Best regards and cyber high fives in your direction!
These really are two different questions.
1. Why GAE?
It's up to you. GAE provides cloud-based hosting that you pay rental to use. It's a bit similar to Amazon Web Services. Your Go app would be uploaded to GAE, where it provides your web service and your users can do lots of wonderful things. Meanwhile you never need to know which actual server is doing the serving at any given time - the app can migrate across their servers dynamically. GAE provides a high uptime and a low effort for you in keeping the server secure, backed up etc. It will also be elastic to cope with surges in load.
You may instead prefer to rent a private server (e.g. at Rackspace) or just a virtual machine. You'd perferably need to be a Linux expert (get lots of help at Serverfault) and you'll have to do the backup, firewall etc all yourself. It may cost (much) less. Or more.
2. Choosing a framework?
The net/http API allows you to write HTTP server code to do pretty well anything you want. But you have to do quite a lot of hard work. At the opposite extreme, frameworks like Revel make rapid server development possible, as long as it does the things you want of it. If you stray into functionality beyond what it offers, you might have to do quite a lot of digging to find out how to extend the framework.
Other interesting toolkits include Gorilla, Gocraft Web and Goji. In terms of complexity, these sit about halfway between Revel and basic net/http.
To answer your second question, here are some pros and cons of using a framework (e.g., revel) vs. something simpler like a toolkit (e.g., gorilla)
In general, the pros of using a framework are:
it provides a lot of sub-packages to handle important web-related sub-tasks like templating, generating data in specified formats like json or xml, query escaping, etc.
it handles boilerplate http handling
it (hopefully) enforces best practices like escaping strings
it helps you manage complexity by enforcing a consistent design pattern in the way you handle requests
Cons of using a framework:
frameworks tend to be "opinionated," meaning you have to buy into their general philosophy and understand their core concepts before you can make use of them; for a lot of frameworks this can be quite a bit of mental overhead
extra layer of abstraction, meaning you're another step removed from what's really going on, and there will be more stuff to understand and debug if something goes wrong
it can be brittle and hard to do something that isn't a standard use case in the framework
future maintainability: most frameworks don't tend to have a super long lifespan. Django and Rails have been around for a long time, but there's a massive graveyard of frameworks that came before them. Hindsight is 20/20, but it's hard to pick the right horse from the outset.
Recommendation
It's hard to make the call upfront. So much depends on the specifics of your problem, but I'd say in the case of Go, opt for the simpler option. Much of the value-add of frameworks in other languages is the fact that they contain useful sub-packages that handle important tasks, but Go already contains a lot of these in its standard library (e.g., encoding/json, net/http, net/url, text/template). I've built a fairly sophisticated web app using just the Gorilla toolkit and the go standard library, and it's been amazingly good, and the best part is, it's incredibly easy to understand what the code does and I can explain it to someone else without requiring them to first read through the massive About page of some third party framework.
If you want to get a sense of how other people use Gorilla, you might try looking at real-world usage examples. Compare that to how people use more sophisticated frameworks and pick whichever you like better.
I have a critical meeting tomorrow and I need to convince my operations manager to let our web team leverage Heroku vs our current solution, Network Solutions.
Network Solutions is pretty locked down. They offer rails, but with a locked set of usable gems and the version of rails is pretty old. I am also not productive in PHP. We do have "large windows" hosting with them, but they are running IIS 6 and I would want to use MVC. I am not saying it is not usable, there are work arounds.. but why bother with the headache? Just switch to Heroku and call it a day.
Does anyone have any experience with pricing on Heroku? We do B2B applications and see very small amounts of traffic, at best we would see 5000 hits a month.
Pushing from GIT eliminates making "small changes" to the live site via ftp, and not following up on the local version. (massive time sink / headache)
Heroku gives EACH project it's own "VPS" essentially, so if one of our apps bears fruit it can scale with out taking out a whole server.
How easy it is to actually scale? Would I be wrong to say that it is easy as a "push of a button?"
It's going to be around for a while right? I am using the argument that Heroku is "pay as you go", and since our apps are very low traffic our costs will be very low. But, I understand that this makes for an un-sustainable business model- like how Google App Engine had to re-evaluate it's payment structure. Would Heroku ever go to a "shared" hosting plan for apps like ours? Just so we are not getting a free ride (I'd like to support the platform).
One of the concerns is getting "another" hosting environment, when we have been trying to consolidate. Unfortunately they chose Network Solutions before I got there.
Thank you in advance
Go at it from this angle. Explain to your boss that should your business require a certain feature that you currently don't have, being able to add gems at will to a Heroku platform will allow the extensibility and give you options for better serving customers and scale with the business's future growth, whereas NS is very limiting, causing bottlenecks and problems that your business will be hurt by, because it will result in lost productivity for you and will not support the architecture you'll need to stay ahead in the future. Talk in business terms that will make sense to your boss, and many times they will be able to understand why you want to do a certain thing. Help it relate to the strategic objectives that you are likely to undertake in the future. Paint a big picture. Draw out some scenarios that make your point in a "how does this affect the bottom line/hassles for him."
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
So, I've got an idea for a website. I can start off using any platform and frameworks I want, but there are almost too many options.
OS Platform:
Windows, *nix
Web Framework:
Rails, ASP.NET, ASP.NET MVC, Django, Zend, Cake, others
Hosting:
EC2, Dedicated Server, Shared Hosting, VPS, App Engine, Azure, others
Persistence:
S3, MySql, PostreSql, Sql Server, SimpleDB, CouchDB, others
How do you avoid decision paralysis and get started?
Firstly, your familiarity with a framework's language should dictate which framework you choose. Don't add the burden of learning another language on top of learning a framework.
Next, have a look at the remaining frameworks. Do they have good documentation? What about the community. (A good community can go a long way to making up any shortcomings of a given technology.) Does the framework solve the problems that you need solved?
Finally, just dive in and try something! Pick the one that makes the most sense to you and start writing code. Don't do too much hand-wringing over your decision. If it becomes obvious that you made the wrong choice, it should be obvious quite early. Learn from what you've accomplished so far and consider restarting with a different technology. (Just don't get several weeks down the road before you make this decision!)
I'm sure you don't like all of those technologies equally. Pick a framework that you like and get to work.
It depends on what your app is going to be doing. A handful of the technologies you listed are direct competitors (like Django vs. Rails), but some are completely different ways to do things (like MySQL vs. S3).
Questions to answer before you begin:
Will the app need to be horizontally partitioned in the near term? If so, using EC2, Google App Engine or Azure would be a good option.
Will your app fit into the constraints of Google App Engine? If so, it requires a lot less hassle on your part than running on bare metal (whether real or virtual).
What's your preferred web framework? If you want an MS framework, you'll need to run on a host that supports that.
What will your persistence and data access patterns look like? This will determine whether to use a database or something more exotic.
If you are running on EC2, the other AWS services are more appealing. Similarly, if you are using GAE, you have only one option for persistence. If you are using Rails, may as well start with MySQL.
In answer to your question of how to reduce the number of options, the answer is to realize that many of the options are related, so you don't have as many choices to make as it first appears.
Some advice that was once given to me is, pick what your friends (or colleagues) are using. Having people around you that you can share ideas and the learning experience with is invaluable.
If you want to learn something new: I'd just go with your gut and get started. If it sucks then switch to something more familiar.
If you don't have much time: Go with what you know and forget about the other options. Just start coding.
Optimize for happiness. Pick the one that you like the most. Or the one that intrigues you the most.
I've worked in Microsoft shops, in Ruby on Rails, and in homegrown shops having Apache, Jetty, even Mason.
All frameworks have their warts, their idiosyncracies that will keep you up until 3 AM, and their "tribal knowledge" vagaries that will be completely unexportable to other frameworks. (The last point is sometimes by design, the whole "platform entrenchment" business strategy)
Listen to what the supporters of the frameworks say about the problems with the other frameworks (Google: X framework vs Y framework). Pick the framework that has the loudest supporters. If they are equally loud, make the decision with a dice roll.
With me it's simple.
I only know MS stack and see no point in "checking out" all of those you mentioned.
No, actually I once tried to use JSF before excluding it from my list permanently.
Use what you are experienced in and where you can be more productive. The objective is to get your site up and running. Go for it.
One of the biggest factors in determining which platform/framework to use is your budget. You have to factor in the cost of licensing, software required to develop/maintain your website and other miscellaneous costs.
I suggest you begin with a scorecard of your own construction. Perhaps you can find different ones on the web, but if you do, modify them to meet YOUR needs. There should be a scorecard for each level in the stack (as you've described). Each scorecard should share some aspects to grade with other scorecards but each will also have their unique aspects.
Once constructed, weight each aspect graded according to your needs.
Once you've chosen the weights, pick the scales for grades.
At this point promise yourself you wont mess with the weights or the scale and then start collecting data on your options for each level in the stack.
You may also want to put a time limit on the collection period.
Make your decision based on the outcome of the scorecard.
The beauty of this approach is that the effort is made in constructing the scorecard, not in circular arguments of options. The effort in making the scorecard is vendor agnostic and focuses on the desired result, not the options. Thus you can avoid paralysis.
One more thing, my best scorecards have included sections addressing the availability of resources and other human related things. Don't make the mistake of just looking at the technology.
good luck.
Go for personal preferences.
One decision at a time:
Firts I would begin with type of language:
Script: PHP, Python,
Serious: Java, .Net
The language will restrict your OS, plattform and will give you hints for the dataabse decission. The database load is also important. And, Do you want logic in the DDBB? how much data?
Last advice. Try combinations well tested. LAMP, WAMP, Windows with SQL Server and .NET.
Evaluate each platform and technology for quality of tools for your needs. For example, if you are cost sensitive, you would value free operating systems and tools higher than costly ones. If you need performance, you would value tools which provide high performance higher than ones that don't.
It entirely depends on your situation. I spent several months evaluating stuff for a new commercial web site last year, and it was very easy to feel paralized. In the end it was talking to several people who'd done similar things, and of course reading a lot of stuff online and from Amazon. I chose Java, since our team had a lot of experience in it, and it has good performance and extensive supporting technologies. Oracle is our database but we used a persistence manager to make it easy to change later on. We used a half-dozen very good libraries to eliminate much of the boring and repetitive coding (Restlet, iBatis, Freemarker, XStream, jQuery, SLF4J). We used Glassfish as our web server.
Yours sounds like a small project with only you to work on it. In that case, pick a complete framework instead of a smorgasbord like we did. Pick something fun to work with, and something with good "return on resume". Look very hard at Ruby on Rails, Django (kind of a Python on Rails), and Groovy on Grails (a Rails-wannabe for the Java world). In your shoes I'd pick Ruby on Rails because there's a large and growing community and a good number of books and tutorials. Plus, Ruby looks like a worthwhile language to learn. For your database, just pick one. These frameworks make it easy to change your mind later. Pick MySQL unless you have another you like better.
And as other posters said, just do it! ;-)
Like others said, pick something you and your employees are familiar with. I highly doubt you are close to being industry ready with all those techs.
OS Platform: Windows, *nix
Shouldn't matter except for Windows licensing costs, and that is probably the least of your expenses.
Web Framework: Rails, ASP.NET, ASP.NET MVC, Django, Zend, Cake, others
Dependent on your favorite language
Hosting: EC2, Dedicated Server, Shared Hosting, VPS, App Engine, Azure, others
You should design your product to be movable, so you can scale among these. If you know for sure you are going big, then just start off with EC2. App Engine is extremely limiting, ex. they don't let you form outbound connections.
Persistence: S3, MySql, PostreSql, Sql Server, SimpleDB, CouchDB, others
You need to do the research yourself whether or not your product requires an RDBMS or a simple key/value store, and what features each of these have.
Just go for it! Your platform choice really is not all that important as long as you make a reasonable choice (Ruby + Rails, Python + Django, PHP + Cake/CodeIgniter). Any of these can be used to build successful sites. If your site really takes off, you'll be able to scale it fine.
I'm having trouble deciding if I want a project of mine to be web-based (as in a web-app), desktop-based (a desktop application), or a desktop application that can sync or connect to the cloud.
I don't know if anyone else would have an interest in this application, and it's only going to be for me, so I'm leaning toward desktop application. If, for some reason, I finish it, release it, and people actually like it, I might see about making it sync to the cloud as well (think v2). But I'm not sure how hard it is to make such a radical change, and I don't want to end up with something good that is useless because I made a poor choice before I even started the project.
Is there any sort of guidance for this? Any rules of thumb or best practices? Any personal experiences?
If the language matters, I'm thinking about Java simply because I'm most comfortable with it, and it would easily allow me to share it with my friends for testing and if I get stuck and need help from someone else in person.
I generally ask a few questions:
Can it even be done on the web? Something I did not too long ago involved an image editing component, and had to be a web app. It involved much pain to get this work, and a desktop app would have been a far better way to go.
Will I need to access it from anywhere? Yeah you could load it up on a thumb drive, but the web is far more feasible in this case.
Will there be multiple users? This could go either way, but "long tail" stuff usually means web.
What tech do you want to use? The latest and greatest WPF based UI? Desktop (yeah yeah, silverlight, let's not go there ok?). The brain dead stupid easy user management of Django or others? Web.
If it were a web app, will you need to worry about common attack vectors like SQL Injection, XSS, etc? A desktop app has its own issues here too, but tend to have less exposure.
How resource intensive is it? Will 10 users kill performance of a web server?
Versioning on the desktop can be a pain, whereas with a webapp everyone is on the same version. This can bite you though, see the New Facebook user pushback.
EDIT:
Cost can be a factor too. A web app with a database backend typically means a web server. If you want to stick with, say, the Microsoft Stack, you'll need licenses for SQL Server which can get pricey. Open source is cheaper, but may not be an option in all cases. "Serving" a desktop app is generally cheaper.
If you release as a web-app, you won't have to port it over. You'll also have access to it wherever you go.
I base my choice on the GUI mostly. If the GUI is going to be complex, and (needs to be fast or will have aspects of it that will take a lot of time to process) then I will go with the Desktop. If it is simple, and will always have small data sets to work with at once, the I will go with the Web.
I have worked on an app that was made as a web app, when clearly it was better suited for the desktop. It was a massive failure. I don't know HOW customers put up with it, cause I certainly wouldn't have used it. The desktop version (which took over 6 months to re-write) blew the web version out of the water.
That being said, I have seen some nice web apps.
All I can suggest are several factors that would be relevant. How you determine the answer and weight for the factor is up to you and other circumstances:
What is your audience? Do you have any control over them?
How complex are the interactions you expect to implement?
Do you require near real-time data updates?
How often do you expect to update the application after the first release?
Do you expect a well-defined set of client platforms, or can you not predict that?
Note that your choices also can include a Java WebStart application, which mitigates some of the disadvantages of a typical desktop application.
I'd say that most applications should be desktop-based. The advantages are faster and more fluid apps.
You should only create a web application if there are obvious benefits from it, like access from everywhere. (If that's necessary for your app.)
A downside of web applications can also be that it is dependent on the developer, if you quit supporting it all your users (if you'll have any) can't use it anymore. Furthermore, there is a chance that users are not willing to store their data online.
Ultimately it depends on what kind of an application you want to write. Even if you create it as a desktop-app, you can later on rewrite it for the web. Often a 2.0 version of software needs almost complete rewriting anyway.
Sometime web can be good and sometime not. We are in a new wave that go in the web but do not forget few things:
GUI in web is more complicated because of multiple browser
People who need to work on your system might not like working the whole day in a browser
Web can be slower for some application (image editing, hard job that require a lot of CPU)
Rapid Gui like Visual Studio for winform are faster than for web
But web has many advantage in the deployement and in the portability. If your system is well structured you could make both or change to one to other later with something build with MVC. Just change your visual and you will be fine.
If this were an application to be used my multiple users, with shared data, you're probably going to want a server anyway. In that case I'd lean towards a web application.
Otherwise you've got the complexity of syncing data between the desktop and a server.
Two important questions not on the list so far:
Will the first version have any features that need lowish-level access to hardware?
Will future versions have any featuers that need lowish-level access to hardware?
It's pretty easy to answer the first one, but giving the second one some thought can save you some headache down the road.
My default choice is to go with a web solution, as it's easier to deploy and generally multi-platform. The only time I go with winforms apps is when there are pressing security, performance, or functionality issues that require it.
Previously you'd have written a desktop application, as tool were better for that and you'd have written it faster. People used to want web apps, but always ended up with desktop.
Nowadays things are different, you can write a webservice just as quickly and easily so there's no reason not to go web-based.
The advantages of web-based are flexibility, scalability and ease of deployment. It won't be as responsive as a desktop app could be, but that's not so much of an issue if you think about your design.