I am learning about containers (mostly Docker) since it is coming to windows. And the benefits seem very similar to IIS.
I work behind a firewall building apps for my company (Line of Business). We have a bunch of VMs that will each host a family of web services. One VM can have 20+ services running on IIS.
In that scenario what does deploying my services via Docker get me that I don't already get using IIS?
NOTE: I am completely new to Docker and only have developer level experience in IIS.
Docker is not a replacement for IIS - it can run an application like IIS within a container (I assume - not sure how this is going to work on Windows).
Docker is more like a replacement for a VM - the biggest difference between a VM and a Docker container is that the Docker container is MUCH lighter than a full VM. The usual claim that you see is that you can run many more Docker containers on a host than VMs (but your mileage may vary - some of the claims out there are bit... overstated).
Basically, the idea is this: a VM is a full virtual machine - a real OS running on top of virtual hardware (that looks real enough to the OS). Thus, you're going to get all the bells & whistles of an OS, including stuff you probably don't need if you're running IIS or another HTTP server.
Docker, on the other hand, just uses the host's OS but employs some useful features of the OS to isolate the processes running in the container from the rest of the host. So you get the isolation of a VM (useful in case something fails or for security) without the overhead of a whole OS.
Now you could run "20+ services" in a single Docker container, but that's not generally recommended. Because Docker containers are so lightweight, you can (and should!) limit them to one service per container. This gives you benefits such as
separation of concerns: your database container is just that - a database. Nothing else. And furthermore, it only handles the data for the application that's using it.
improved security: if you want to set it up this way, your database container can only be accessed from the application that's using that database.
limited stuff installed: your database container should be running MySQL only - no SSH daemon, no web server, no other stuff. Simple & clean, with each container doing exactly one thing.
portability: I can configure my images, pull them to a new host, and start up the container and I will be guaranteed to have the exact same environment on the new host that I have on the old one. This is extremely useful for development.
That's not to say you couldn't set something similar up using VMs - you certainly could - but imagine the overhead of a full VM for each component in your application.
As an example, my major project these days is a web application running on Apache with a MySQL database, a redis server, and three microservices (each a simple independent web application running on Lighttpd). Imagine running six different servers for this application.
Docker containers add support for .NET, SQL Server, and other workloads that would integrate with IIS. You also benefit from docker portability, as you could take your container images and run them on AWS or Azure,as well as privately. And, you get access to a large ecosytem of docker based tools . . . bottomline, the industry is moving to support the Docker API.
To host a web application on IIS in a container, a good starting point is using the latest IIS docker image for applications to be hosted on IIS, or if ASP.NET or WCF is the target platform, using the relevant images from these two platforms, which in turn include IIS:
https://hub.docker.com/r/microsoft/iis/
https://hub.docker.com/r/microsoft/aspnet/
https://hub.docker.com/r/microsoft/wcf/
Related
I want to create another instance of dspace to work on different projects. however I do not know how or if it will conflict with this running.
While it's technically possible, I would advise against this for the following 3 key reasons:
Quite a few configuration aspects of DSpace still count on a Tomcat restart to take effect. If you have two instances running in the same Tomcat, it means you have to bring both of them down when you want to update one of them.
Performance related issues are already far from trivial to debug in DSpace, even if you have only one instance running in one Tomcat. If you run two instances, it is very likely that you will only make this more difficult.
This kind of setup is non-standard. As with all non-standard setups, you will find it much harder to get community support, as very few other people will be in the same boat.
So ... either run two VMs, or just two Tomcat processes on one VM.
If after these warnings, you still want to do it, the basics would be to run all of the webapps you want twice in the tomcat, on different ports. The minimum you would need are 2x XMLUI OR JSPUI and 2x SOLR. It could be possible to run one solr webapp, and keep 2 search, statistics, authority and oai indexes in this one SOLR webapp, but I don't know what the side effects could be.
1) Obviously each instance should be installed to a different set of directories.
2a) Create a separate Context for each instance. That will give them different paths: http://legion.example.com/one/, http://legion.example.com/two/.... I do this on my development workstation all the time.
2b) You can also create separate domains and IP addresses, bind them to multiple Host objects in a single Tomcat configuration: http://one.example.com/, http://two.example.com/.... I have four low-volume production DSpace instances running in one Tomcat instance on a midsized host.
Each DSpace instance needs its own database, but PostgreSQL can host dozens. You should consider creating separate database user accounts for each.
You'll also need separate Handle resolvers for each DSpace, just the same as if each instance was on its own host. When configured for DSpace, the Handle resolver uses the DSpace database instead of its own, so it's specific to a single instance.
Solr ought to be able to serve several sets of cores for several DSpaces, but you'll have to do a fair amount of configuration to keep them distinct and ensure that each DSpace is using its own set. You'll learn a lot more about Solr than you need to know for the captive Solr instance that gets installed with a single DSpace.
But then, you'll also learn a lot more about Tomcat than you need to know for a single DSpace....
If you declare your Contexts in external files ([Tomcat]/config/Catalina/localhost/one.xml etc.) and you have automatic deployment set up right, you can just 'touch' one of the Contexts to restart it without restarting a whole Tomcat. Otherwise you can use the Tomcat Manager webapp to do this. Consider well whether you want to have the Manager running, though, because it is quite powerful and it is exposed on the network. I run such applications on yet another, non-routable address so they can't be reached from Outside.
DSpace is not small, so you will need to ensure that you have enough memory to run several instances and that Tomcat's memory limits are adjusted accordingly. I would suggest also installing a resource monitor such as Psi Probe and glancing at it regularly. The above comments on performance are spot-on.
Learning to make all this work was loads of fun, and took quite some time. On the other hand, for development you may prefer something like https://github.com/DSpace/vagrant-dspace, a packaged virtual machine with DSpace and friends inside.
I'm looking to setup Plastic SCM on a hosted server. Considering an Amazon EC2 instance for this. Any recommendations would be appreciated.
Minimum server specs for good performance
Tips on setup/config
Windows v. Linux
MySQL v. SQL Server v. SQL Express
Thanks!
We have extensively tested Plastic on EC2, in fact it is one of the main environments where we run Plastic SCM tests.
It all depends on the load that the server needs to handle.
Tiny server for occasional pushing and pulling
For instance, the demo server we use to handle the evaluation guide runs on a tiny EC2 instance, with Linux and MySQL and a total RAM of 512Mb. It is good for occasional pushing and pulling but of course not to be used under heavy load.
Big server for extreme load
On the other hand, we use a more powerful server to run 'load tests' with 300 concurrent bot clients doing about 2000 checkins per minute on a big repository. We detail the specs here. Basically, for higher perf:
20GB RAM
2 x Intel Xeon X5570
4 core per processor (2 threads per core) (2.7Ghz) – 16 logical cores – Amazon server running Windows Server 2012 + SQL Server 2012
Central vs distributed development
That being said, remember that if you setup a cloud server your bigger restriction for heavy load won't be the server itself but the network. If you plan to work in a centralized way (your workspaces directly work connected to the cloud server) then network will definitely be a consideration. Every checkin, every create branch, every switch to a new branch will mean connecting to the remote server and chances are that you won't get the same network speed you get on a LAN.
The other option is that you work distributed: you have your own Plastic repositories on the developer machines and you just push/pull to the central server. If that's the case it will work great and the requirements won't be high at all.
Specs for a 15-users team working distributed + Amazon EC2 server
If that's your case I'd go for:
Linux server + MySQL (cheaper than windows and works great)
Make sure you install the server with the packages we provide. We include our own build of Mono that will make wonders. Remember to set up the mono server to run with sgen (the latest Mono Garbage Collector).
Install MySQL (or MariaDB). Follow the instructions we provide here. Remember we do need to configure the max_allowed_packet in MySQL so it allows 10Mb packages (we use 4Mb but set it to 10). Everything is explained on the guide.
Use "user/password" security mode. Remember to configure the permissions so only your team can access :-)
For 15 users a m1.small instance will be more than enough (1.75Gb of RAM and a little bit of CPU).
Configure SSL and remove regular TCP so that your server is always secured. Check this.
We added an option in 5.4 that is able to store all data in an encrypted way, so even if the central repo is hacked in Amazon (unlikely) nobody will access your data.
Clients (I'll assume you're using Windows):
Install both client and server (remember we install a server to handle the local replicas of the repos).
Configure it in UP (user/password) mode.
Push and pull from the remote.
Alternatively you can also configure the SQLite backend (the one I've been using for 4 years now on Windows) which is extremely fast. By default, on Windows, a SQL Server Compact Edition (embedded) will be installed. It is ok too.
Connect to the server using SSL.
Hope it helps :-)
Lots of time, We build a web application in our own personal system, with a local server that comes in packages like WAMP, XAMPP. But, the configuration we have on our system hardly matches 100% with the server where we host that application. This adds up to the debugging complexity a lot. Specially when the server you host is configured to hide the errors. Relying on error logs to debug everything, is a uncomfortable options, which is also not guaranteed to be available.
What are the measures we can take to minimise such differences?
You guys might say, it depends on web server as the configuration might be different from server to server. Even in this case, or shared hosting, I would like to know of the pointers to take, before hosting a application or even start building it.
We use a staging server that is a clone of our deployment server. Running code by the staging server (using version control) is both fast and a reliable way to ensure that the new code/code changes work as expected on the live environment.
That said, we use Ubuntu servers and I've opted to use Ubuntu as my development environment as well, makes stuff so much easier.
I’m hoping some of you with experience using amazon EC2 could offer some advice… of course it’ll be subjective which is fine, I’m pretty sure your guestimate would be better than mine.
I am planning on moving all my client’s websites from shared hosting environments to Amazon EC2. They’re all pretty low traffic sites (the busiest site receives around 50 unique visitors a day). There’s about 8 sites, but I may expand this as I take on more projects and host more sites… current capacity planning is for say 12 sites.
Each site runs on ASP.Net (Umbraco CMS), and requires a SQL Server database.
My thoughts are one of the following:
Setup a Small Instance (1.7gb RAM, 1 EC2 Compute Unit), and run IIS and SQL Server Express on that server.
Setup 2 Micro Instances (613MB Ram each, Up to 2 EC2 Compute Units) – one for IIS, the other for SQL Server.
Which arrangement do you think would work the best for my requirements. I’ve started setting up a Micro instance with Server 2008, SQL Server Express, etc… and finding it not coping with the memory requirements, hence considering expanding. I could always configure on a Small instance, then export the AMI and fire it up in a Micro instance after, and do the same every time any serious changes to the server are required. I guess I could even do all updates etc on a spare Small Spot instance, then switch load that AMI up in a Micro and transfer the IP Address across, so I don’t need to do too much work on the production servers. I figure if I store all my website data files on EBS Volumes, then it should be fairly easy to move hosting between servers with minimal downtime, while never working on a production server.
I’m interested to know what you all think, and what strategies you employ for such activities as upgrades, windows updates, software installations, etc.
And what capacity do you think I’d need for my requirements.
Cheers
Greg
Well, first-up, Server 2008 doesn't play well in the 613MB RAM the Micro instance gives you. It runs, but it's a dog, and it barks louder the more services (IIS, SSE, etc) you layer on top. We using nothing smaller than a Small for Server 2008, and in fact typically do the environment config in a Medium and scale down to Small once the heavy lifting is complete and the OS is ready to use. Server 2003, however, seems to breathe easier on a Micro - but we still do the config on a larger instance and scale down.
We're running low-traffic websites on Server 2003/IIS6 in a Micro, with a Server 2008/SS install on a shared, separate, Small instance. We do also have one Server 2008/IIS7 Micro build running, but only to remind ourselves why we don't use it more widely. ;)
Larger websites run Server 2008/IIS7 in either Small or Medium instances, but almost always still using that shared separate SS instance for database services. We try not to deploy multiple SS installations, since it makes maintenance and backups more complex.
Stashing content and config on EBS Volumes is of course good practice, unless you like rebuilding the entire system whenever an Instance disappears. Snapshotting your Instances periodically is also good practice, since you can spin-up a new Instance from a baseline AMI and swap the snapshot in as a boot Volume for fast recovery in the event of disaster.
Recently I stumbled across mongoDB, couchDB etc.
I am hoping to have a play with this type of database and was wondering how much access to the hosting server one needs to get it running.
If anyone has any knowledge of this, I would love to know whether it can be set up to work when your app is hosted via a 'normal' hosting company.
I use Mongo, and so I'm really only speaking for Mongo, but your typical web hosting environment wouldn't allow you to set up your own database. You'd want root-level (admin) access to the server to set up Mongo. To get that, you'd want something like a VPS or a dedicated server.
However, to just play around with Mongo, I'd recommend downloading the binary for your OS and giving it a run. Their JavaScript shell interface is very easy to use.
Hope that helps!
Tim
Various ways:-
1) There are many free mongodb hosting available. Try DotCloud.com. Many others here http://www.cloudhostingguru.com/mongoDB-server-hosting.php
2) If you are asking specifically about shared hosting, the answer is mostly no. But, if you could run mongoDB somewhere else (like from the above link) and want to connect from your website, it is probably possible if your host allows your own extensions (for php)
3) VPS
How about virtual private server hosting? The host gives you what looks like an entire machine... hard drive, CPU, memory. You get to install whatever you want, since it's your (virtual) machine.
In terms of MongoDB like others have said, you need the ability to install the MongoDB software and run it (normally as a daemon). However, hosted services are just beginning to appear, such as MongoHQ. Perhaps something like this might be appropriate once its out of beta (or if you request an invite).
It appears hosted CouchDB services are also popping up, such as couch.io or Cloudant. I personally have no experience with Couch so I can be less certain than with Mongo, but I'd imagine that again to run it yourself, you'd need to install the software (and thus require root access).
If you don't currently have a VPS or dedicated server (or the cloud-based versions of the aforementioned), perhaps moving your data out to a dedicated hosted service would be an ideal way to go to avoid the pain and expense of changing your hosting setup.
You can host your application and your database in the different hosting servers.
For MongoDB you can use mongohq or mongolab with space 0.5 Gb for free