I was poring over the docs for Open DayLight, and can't seem to wrap my head around what software-defined networking even is. All the media hype, blogs and articles I can find on SDN are riddled with buzzwords that don't mean anything to me as an engineer. So I ask: What (exactly) is SDN? What are some specific uses cases/problems it solves? Is it:
Just making proprietary networking hardware serve network APIs, thus allowing programs to configure them (instead of IT guys using a console or web interface)?; or
Implementing (traditionally proprietary) networking hardware as software; or
Writing software that somehow integrates with virtual networking hardware used by virtualization platforms (vLANs, vSwitches, etc.)?; or
Something else completely?!?
BONUS: How does Open DayLight fit into this equation here?
First of all, you are right, there is not official definition from NIST or some similar standardization body and the fact that its meaning is fuzzy is exploited by marketing people.
The main point of SDN is that it allows to program network functions with APIs.
In the past, networking devices like switches and routers were only configurable using a proprietary interface (be it vendor specific tools or just the CLI on the device) and there were no APIs which allow to configure OSI L2 - L3 aspects like VLANs and routes but also L6 - L7 aspects like load balancing highly dynamically. Btw. In the case of L6 - L7 functions, the term NVF = Network Virtualized Function seems to be established by now.
This is needed especially for multi tenancy capable virtualized IaaS systems. You can create new VPCs and arrange them together at will. To really isolate tenants from each other, you need to have a L2 isolation and so the same dynamics that is offered for VPCs is propagated to the networking for interconnecting them.
Conclusion: It is about your first bullet with the extension, that the APIs must not necessarily be offered by some hardware appliance, it can also be offered by some pure software implementation.
Regarding OpenDaylight:
It is the OpenStack pendant for SDN. They also actively push integration with OpenStack. They say they are an "open, reference framework for programmability and control through an open source SDN and NFV solution". This means it provides (as you say) a façade for the manfold aspects of networking.
They have all the big names as members which probably means they have the power to establish a de-facto standard like OpenStack did. Members benefit in that they can provide plugins, integrations and adaptations for their products so that they seamlessly integrate with OpenDayligh and you only need to care about a single standard API.
SDN is programmable networks. Different SDN solutions provide different functions in their APIs towards the app developer.
There is a good overview of SDN for software developers here:
https://github.com/BRCDcomm/BVC/wiki/SDN-applications
The most common elements for SDN solutions are
North-bound API: A programming interface used by an application/script to monitor, manage and control the network topology and packet flows within the network.
Network elements: Switching or routing network elements that enforce the rules provided by the application via the north-bound API. These elements may be physical (Cisco, Brocade, Tallac, etc) or virtual (Open VSwitch, Brocade Vyatta vrouter, Cisco 1000, etc) or a combination.
Controller-based solutions have a clustered architectural element (the 'controller') that provides the north-bound api towards applications and an extensible set of south-bound APIs to which network devices connect. Some controllers available today are OpenDaylight, Open Network Operating System (ONOS), Juniper Open Contrail, Brocade Vyatta Controller (ODL distribution), HP VAN Controller and more.
Best rules of thumb to understand an SDN offering:
Read its north-bound API - this tells you what you will be able to monitor, manage and control in your network.
Find out which south-bound APIs it supports - this tells you which switches/routers it might work with.
Some SDN use cases/applications:
DevOps/Admin automation - Applications and scripts that make a network admin or DevOps life easier through automation. OpenStack Neutron is a common example.
Security - HP provides 'Network Protector' that learns the topology of the network and then monitors activity providing alerts and/or remediation of non-compliant behaviors.
Network optimization
Brocade offers 'Traffic Manager' that monitors network utilization and modifies traffic flows in real time to optimize quality based on defined policies.
HP provides 'HP Network Optimizer' that provides an end-to-end voice optimized path for enterprise Microsoft Lync users.
Lyatiss provisions AWS networks in realtime to meet application needs.
Monitoring classroom time-on-task - Elbrys provides an application that provides a teacher with a dashboard to monitor student's time-on-task in real time and cause redirects of individual students to web pages of their choosing. (Disclaimer: I work for Elbrys Networks)
OpenDaylight project proposals page - https://wiki.opendaylight.org/view/Project_Proposals:Main
The concept of SDN is very simple. SDN decouples control-plane (i.e. decision making) from data-plane (the actual forwarding actions) and provides API between them (e.g. OpenFlow API).
Image source: https://www.commsbusiness.co.uk/features/software-defined-networking-sdn-explained/
With SDN architecture, network engineers no longer have to learn proprietary CLI commands for different vendors. They can focus on developing logically centralized control programs to make network global decisions and send it down to network switches (data-plane). Dumped network switches (data-plane) received controller rules/decisions and process network packets accordingly if no decision found they ask the controller.
For example: In SDN architecture routing algorithms developed as a program in the controller, it collects all required metadata (e.g. switches, ports, host connections, links, speed, etc) from the network then make a routing decision for each switch in the network. While in a conventional network, a routing algorithm is implemented in a distributed fashion in all switches (i.e. generally each switch has its own intelligence and makes its own routing decision).
SDN explained by Nick Feamster
Here is a good paper that illustrates the road map to SDN
Related
I'm trying to implement a communication system among a variety of devices connected through WiFi.
A Desktop ( Mac / Win / Linux ) serves as a server, whereas mobile phones ( Android / iPhone / Blackberry ), say 50 in number, will be clients.
There should be a client-server as well as client-client 2-way communication.
In client-server communication, I need to access a database in the server.
While surfing about this, I came across ZeroMQ as a high-performance asynchronous messaging library and a better solution for complex Distributed communication system.
Note:
Yeah, I am completely new to this communication and networks, but try learning that. ( Guess the fact is well reflected in the clarity of question :P )
EDIT:
if ZeroMQ seems not to be a good option, suggest me some other means of achieving this.
Yes, ZeroMQ is a great and powerful tool
This does not mean it is the best tool to use for any particular project.
Many other facts matter more, than the built-in code and service-archetypes.
Project's potential for creeping scope, moving sands in diversity of target devices, respective O/S versions, patches, EoL-maintenance/unsupported orphanages
Project plan / vs the Teams' already accrued { ZeroMQ and other-tools } craftmanship
Scaling of the services - from 5, 50, 500, 5000+
Service robustness / { service & transaction }-self-healing strategies
Service risks associated with an absence of any version-{ -control- | -enforcement- }-policy in loosely coupled or even un-controlled domain
Service risks from (non-)-{ -stable | -available } language bindings or wrapper mediators.
One will always learn a lot, once opening the ZeroMQ perspective
There are many points of view, that will help one to better design even non-distributed services. A Zero-copy design rule, a Zero-sharing for performance targets, (almost) Zero-latency, (almost) Zero-overheads for (almost) linear-scaling -- these are just few principles, one may benefit from, if learning ZeroMQ from its ground-Zero-roots.
The best next step I may direct one to, feel free to read ZeroMQ posts here for FF-reading and do not miss to download the great must-read book from Pieter HINTJENS: "Code Connected, Volume 1".
After having ZeroMQ views understood, Nanomsg or any other tool available may give one some additional views ( and one will be then mature and ready also assess the risks / costs to be paid on such grounds ).
We have a set of Microservices collaborating with each other in the eco system. We used to have occasional problems where one or more of these Microservices would go down accidentally. Thankfully, we have some monitoring built around which would realize this and take corrective action.
Now, we would like to have redundancy built around each of those Microservices. I'm thinking more like a master / slave approach where a slave is always on stand by and when the master goes off, the slave picks it up.
Should we consider using any framework that we could use as service registry, where we register each of those Microservices and allow them to be controlled? Any other suggestions on how to achieve the kind of master / slave architecture with the Microservices that would enable us to have failover redundancy?
I thought about this for a couple of minutes and this is what I currently think is the best method, based on experience.
There are a couple of problems you will face with availability. First is always having at least one endpoint up. This is easy enough to do by installing on multiple servers. In the enterprise space, you would use a name for the endpoint and then have it resolve to multiple servers (virtual or hardware). You would also load balance it.
The second is registry. This is a very easy problem with API management software. The really good software in this space is not cheap, so this is not a weekend hobbyist type of software. But there are open source API Management solutions out there. As I work in the Enterprise space, I am very familiar with options like Apigee, CA, Mashery, etc. so I cannot recommend an open source option and feel good about myself.
You could build your own registry, if you desire. Just be careful how you design it, as a "registry of all interface points" leads to a service that becomes more tightly coupled.
We have a traditional VB application which are used for Organization operations. Now we are building a Hybrid application developed by using HTML5,CSS and Javascript which is targeted on Google Chromium desktop container. Now we are planning to provide a way to exchange large data like employees records between both of these 2 applications. Now my specific question is
What are the different ways to achieve communication between Chromium desktop container and VB application to exchange large chunks of data?
Sounds a bit painful no matter what.
Chrome Apps Architecture
All external processes are isolated from the app.
This would seem to suggest the obvious course is to use cloud data services, whether on public or private clouds.
I suspect that for political as well as practical reasons no cloud vendor goes to the trouble to provide VB/VBA-friendly APIs for their services. Mainly nobody wants to deal with support issues from the teeming hordes of casual coders the VB community is saddled with.
The VB6 community hasn't stepped up and taken care of this themselves either.
If you can limp along with the burdens of ".Net Inter Clop" (the usual MS answer) that might be a way to exploit existing API implementations.
Otherwise you might roll your own cloud. I see a few obvious services you'd want to implement in your cloud with lightweight APIs easily implemented in both of your development ecosystems:
Bulk Storage. I suggest WebDAV, which IIS supports. If you eschew the locking features then WebDAV API implementations are pretty easy in both JS and VB. Or buy (or scrounge open source) implementations of a more complete WebDAV client library.
DBMS. Pick any, implement a simple REST-like XML over HTTP API. Relatively easy to implement.
Push Notifications. I'd write a custom service accepting long-duration TCP connections from all clients, and with protocols and workflow à la Amazon SNS or Google Cloud Messaging. Such a service would be generally light in resource consumption but you'd probably want a dedicated box with OS tweaks to support a large number of active TCP connections.
Maybe optionally a message queue service?
Nothing novel here, these are all well established patterns.
All of the tools to do that are pretty off-the-shelf whether you want your cloud servers to be based on Windows, Linux, or generically Java anywhere.
Most of the effort will probably go into developing a consistent authentication model, access control model, and of course an integrated administration interface, monitoring, and logging to help keep operating overhead low and uptime high. Well, that and developer docs and training.
Ok, still a lot of work. Too bad there isn't a "cloud in the box" with the API libraries you'd need that you can buy off the shelf today.
Or perhaps I'm missing something obvious?
I am trying to develop a publish/subscribe system.
To this end, I have read some papers and articles regarding it.
And they all talk about Messaging service as an integral part of publish/subscribe system.
My question is, can I develop a publish subscribe system without using MOM like JMS?
Or am I missing or oversimplifying things?
I do not think you are oversimplifying things. There are stand-alone products available that provide advanced functionality based on publish/subscribe, without being part of a larger MOM system.
One of them is a group of products implementing the Data Distribution Service (DDS) specification, as standardized by the Object Management Group (OMG). Check out this Wikipedia entry for a very brief introduction and list of references.
DDS supports many advanced data management features like a strong-typed and content aware databus, distributed state management and historical data access. Its rich set of Quality of Service settings allows to off-load a lot of the complexity from your applications to the middleware. This is all based on the publish/subscribe paradigm.
If you would tell more about your application, then I might be able to point you to similar use cases using this technology -- if you are interested.
It depends what you mean by "MOM". If you think MOM = JMS then yes, there are plenty of pub/sub applications which are not JMS servers (off the top of my head): 0MQ, TIBCO Rendezvous and the many AMQP implementations around.
I guess my definition of MOM is an infrastructure for reliably getting a message from one system to another in an asynchronous manner. Pub/sub is a feature on top of the message transport which allows a message to be distributed to multiple other systems. Once you get beyond the point of opening a socket and stuffing a bunch of bytes down it, I would argue you are in the realm of MOM.
So, no you don't need JMS to do pub/sub....there are plenty of open-source and closed-source alternatives out there. Which one depends on your requirements and skills.
You can look at multicast that provides one to many communication. Multicast does not require MOM, instead it requires multicast enabled IP network. Usually the network routers take care of creating copies of message and delivering messages to destinations.
Where would I go to learn write code that had to be very, very secure and that DOES expose external services (running on a standard Windows or Linux OS). Knowing what services can and cannot be safely exposed would be part of the issue. Note that I am not looking for a favorite choice between Linux and Windows, as the choice is not likely to be mine to make in any given case. However the level of security needs to be military grade.
I almost feel embarressed giving this as a for instance, but how would I know whether or not I could use, say, WCF, in such a setting.
High security is a difficult concept as it generally involves way more than just the code you wrote.
Basically every layer of the OSI model has to be taken into consideration. Things like, preventing capture of the data stream (or it being rerouted) between the end points (quantum cryptography).
At the higher levels, you have things like various things like
Physical security of the devices (all endpoints if possible).
Hardening the OS (e.g: closing ports, turning off unused services, using kerberos, VPN tunnels, and leveraging white lists of machines allowed to connect, etc);
Encrypting the data at rest (file encryption), in transmission (SSL), and in memory (column/table encryption).
Ensuring and enforcing proper authentication and authorization at every level (in app, in sql, etc).
Log EVERYTHING. At a minimal it should answer "who/what/when/where/how"
Along with the logging, Actively Monitor it. aka: intrusion detection.
Then we can move on to other things like looking at other attack vectors like sql injection, xss, internal / disgruntled employees, etc.
And once you've done all of that be prepared when a hacker gets away with everything they want simply by social engineering.
In short, the best tact to take in order to secure any computer related application is to listen to the ethos of Fox Mulder, and Trust No One. Another favorite of mine that applies is: It's only paranoia if they aren't after you.
You could use formal methods to (sort-of) prove the critical parts of your software. A tool like Frama-C (free, LGPL license, targetting embedded systems) could be relevant (at least if your software is critical, embedded, written in C).
But military grade don't mean much. Your client will (and should) define exactly the standards to respect. For instance, critical [civilian] aircraft software needs to follow something like DO-178C (or its predecessor, DO-178B). Different industries have different standards similar to that. (both railways and medical industries have their own standards, which might be different in North America than in Europe).
If your system (& client) is less demanding (i.e. no billion dollars or hundreds lives threatened by bugs) you could consider customizing your compiler or using some other tool. For example, GCC is customizable thru plugins or thru MELT extensions.
Don't forget that software reliability has a big price (that means a big cost for you, hence for your client).
Well, the question of where can be answered simply. Not in school. I suggest to create a learning path for yourself. Pick a technology that you like and learn it inside out. A basic book to get you started should suffice, however the rest of the stuff you learn as you go, or via the documentation of that technology.
For instance - learning under .NET (Microsoft) involves a basic A-Press text-book (i suggest Pro C# and The .NET 4.0 Platform). Thereafter searching through the .NET Framework Reference on MSDN will give you the rest.
If you are looking for WCF reference, I suggest the (MCTS Exam 70-503, Microsoft .NET Framework 3.5 Windows Communication Foundation) and MSDN.
Just keep in mind that not a single technology will achieve what you are looking for. For example: WCF co-mingles with WF (Windows Workflow Foundation), as well as SQL Data Services and Entity Framework. Being exposed to multiple technologies will definitely broaden your vision.
===============================================================================
WCF is a beast in this regard. Here are the advantages over some other means of communication:
Messages (data) passed between end points can be secured via message-level security (encryption). The transport channel chosen can also be secured at protocol level via transport layer security (encryption).
End points themselves can authorize and impersonate clients (client level security). You can implement end-to-end service tracing, health monitoring & performance counters, message logging, as well as forward and backward compatibility with newer/older clients (via graceful degradation of the message format, provided in WCF). If you chose to do so, you can even implement routing as fail-safe for your communications channel. WCF also supports transactions (ACID), concurrency, as well as a per-instance throttling, giving you the most flexibility in writing secure/robust military grade code.
In retrospect the security and flexibility of WCF are astonishing. A similiar technology (if not the same) is the WS-Security spec. It is part of the WS-* specifications for web services and deals with Xml signature and Xml encryption to provide secure communications channel between two end points.
The disadvantages of WS-* however is that it is a one-way means of communication. WCF can facilitate 2 way communication. A client can send a request to a server, but also a server can send requests to the client. WS-* dictates that a client can only send and receive responses to the server, but not vice versa.
I am not a WCF developer so i thought the highlights might provoke you into doing your own research. "There are hundreds of ways to skin an animal, neither of them is wrong..."