SNMP is generally used to monitor the health of components in network.
For SDN [Software Defined network], is it desirable to use SNMP . I am having doubt like is it better to use some other protocol like NETCONFIG
In general SNMP can be used for configuration of a device, however personally I will not stretch it too far specifically when network configuration operation potentially spans across multiple devices and as a result will have higher order transaction requirements.
RFC3512 provides good perspective around configuration using SNMP. Reading through the RFC it will become apparent that within a device transaction relies on how well the MIBs (the objects used via SNMP for performing configuration changes) are designed and implemented. For configuration spanning across multiple devices the device transaction alone will not suffice, if rolling back the configuration is a requirement (this depending on the nature of service/use-case being addressed by your SDN controller). I would recommend reading the Transaction Control in MIB Objects further to understand the requirements on the protocol and eventually the capabilities of the MIB modules that one will be using for configuration.
Netconf was created with configuration of devices in mind and it offers various capabilities that are of use in this regard. These are covered in detail in the IETF standard for Netconf Protocol RFC under Capabilities section. The capabilities such as Candidate Configuration, Validate Configuration, Confirmed Commit, Rollback on Error and other such are specified in the standard which shall further aid in orchestration of a transaction across multiple device.
Related
Does Marklogic supports backpressure or allow to send data in chunks that is reactive approach ?
'Reactive' is a fairly new term describing a particular incarnation of old concepts common in server and database technologies, but fairly new to modern client and middle-tier programming.
I am assuming the question is prompted by the need/desire to work within an existing 'Reactive' framework (such as vert.x or Rx/Java). For that question, the answer is 'no' - there is not an 'official' API which integrates directly with these frameworks to my knowledge. There are community APIs which I have not personally used, an example is https://github.com/etourdot/vertx-marklogic (reactive, vert.x marklogic API).
MarkLogic is a 'reactive' design internally in that it implements the functionality the modern 'reactive' term is used to describe -- but does not expose any standard 'reactive APIs' for this (there are very few standards in this area). Code running within MarkLogic server (xquery,javascript) implicitly benefits from this - although there is not an explicit backpressure API, a side effect of single threaded blocking IO (from the app perspective) is that the equivalent of 'back pressure' is implemented by implicit flow control of the IO APIS - you cannot over drive a properly configured ML server on a single thread doing blocking IO. Connections to an overloaded server will take longer and eventually time out ('backpressure' :)
Similarly, (most of) the external APIs (REST, XCC) are also blocking, single threaded.
The server core manages rate control via a variety of methods such as actively managing the TCP connection queue size, keep alive times, numbers of active threads etc.
In general the server does a very good job at this without explicit low level programming needed, balancing the latency across all clients. If this needs improving, the administration guides have good direction on how to tune the various parameters so the system behaves well on its own.
If you want to implement a per-connection client aware 'reactive' API you will need to implement it yourself. This can be done using the same techniques used for other blocking IO APis -- i.e. either use multiple threads or non-blocking IO. Some of the ML SDK's have provision for non-blocking IO or control over timeouts which can be used to implement a 'reactive' API.
Similarly, code running in the server itself (XQuery or JavaScript) can implement 'reactive' type behaviour by making use of the task queue -- as exposed by the xdmp:spawn-xxx apis. This is done in many libraries to manage bulk ingest. Care must be taken to carefully control the amount of concurrency as you can easily overload the server by spawning too many concurrent requests. Managing state is a bit tricky as there is a interaction/opposition between the transaction model and task creation -- the former generally presenting an idempotent view of data that can be incongruous with the concept of 'current' wrt asynchronous tasks.
I was poring over the docs for Open DayLight, and can't seem to wrap my head around what software-defined networking even is. All the media hype, blogs and articles I can find on SDN are riddled with buzzwords that don't mean anything to me as an engineer. So I ask: What (exactly) is SDN? What are some specific uses cases/problems it solves? Is it:
Just making proprietary networking hardware serve network APIs, thus allowing programs to configure them (instead of IT guys using a console or web interface)?; or
Implementing (traditionally proprietary) networking hardware as software; or
Writing software that somehow integrates with virtual networking hardware used by virtualization platforms (vLANs, vSwitches, etc.)?; or
Something else completely?!?
BONUS: How does Open DayLight fit into this equation here?
First of all, you are right, there is not official definition from NIST or some similar standardization body and the fact that its meaning is fuzzy is exploited by marketing people.
The main point of SDN is that it allows to program network functions with APIs.
In the past, networking devices like switches and routers were only configurable using a proprietary interface (be it vendor specific tools or just the CLI on the device) and there were no APIs which allow to configure OSI L2 - L3 aspects like VLANs and routes but also L6 - L7 aspects like load balancing highly dynamically. Btw. In the case of L6 - L7 functions, the term NVF = Network Virtualized Function seems to be established by now.
This is needed especially for multi tenancy capable virtualized IaaS systems. You can create new VPCs and arrange them together at will. To really isolate tenants from each other, you need to have a L2 isolation and so the same dynamics that is offered for VPCs is propagated to the networking for interconnecting them.
Conclusion: It is about your first bullet with the extension, that the APIs must not necessarily be offered by some hardware appliance, it can also be offered by some pure software implementation.
Regarding OpenDaylight:
It is the OpenStack pendant for SDN. They also actively push integration with OpenStack. They say they are an "open, reference framework for programmability and control through an open source SDN and NFV solution". This means it provides (as you say) a façade for the manfold aspects of networking.
They have all the big names as members which probably means they have the power to establish a de-facto standard like OpenStack did. Members benefit in that they can provide plugins, integrations and adaptations for their products so that they seamlessly integrate with OpenDayligh and you only need to care about a single standard API.
SDN is programmable networks. Different SDN solutions provide different functions in their APIs towards the app developer.
There is a good overview of SDN for software developers here:
https://github.com/BRCDcomm/BVC/wiki/SDN-applications
The most common elements for SDN solutions are
North-bound API: A programming interface used by an application/script to monitor, manage and control the network topology and packet flows within the network.
Network elements: Switching or routing network elements that enforce the rules provided by the application via the north-bound API. These elements may be physical (Cisco, Brocade, Tallac, etc) or virtual (Open VSwitch, Brocade Vyatta vrouter, Cisco 1000, etc) or a combination.
Controller-based solutions have a clustered architectural element (the 'controller') that provides the north-bound api towards applications and an extensible set of south-bound APIs to which network devices connect. Some controllers available today are OpenDaylight, Open Network Operating System (ONOS), Juniper Open Contrail, Brocade Vyatta Controller (ODL distribution), HP VAN Controller and more.
Best rules of thumb to understand an SDN offering:
Read its north-bound API - this tells you what you will be able to monitor, manage and control in your network.
Find out which south-bound APIs it supports - this tells you which switches/routers it might work with.
Some SDN use cases/applications:
DevOps/Admin automation - Applications and scripts that make a network admin or DevOps life easier through automation. OpenStack Neutron is a common example.
Security - HP provides 'Network Protector' that learns the topology of the network and then monitors activity providing alerts and/or remediation of non-compliant behaviors.
Network optimization
Brocade offers 'Traffic Manager' that monitors network utilization and modifies traffic flows in real time to optimize quality based on defined policies.
HP provides 'HP Network Optimizer' that provides an end-to-end voice optimized path for enterprise Microsoft Lync users.
Lyatiss provisions AWS networks in realtime to meet application needs.
Monitoring classroom time-on-task - Elbrys provides an application that provides a teacher with a dashboard to monitor student's time-on-task in real time and cause redirects of individual students to web pages of their choosing. (Disclaimer: I work for Elbrys Networks)
OpenDaylight project proposals page - https://wiki.opendaylight.org/view/Project_Proposals:Main
The concept of SDN is very simple. SDN decouples control-plane (i.e. decision making) from data-plane (the actual forwarding actions) and provides API between them (e.g. OpenFlow API).
Image source: https://www.commsbusiness.co.uk/features/software-defined-networking-sdn-explained/
With SDN architecture, network engineers no longer have to learn proprietary CLI commands for different vendors. They can focus on developing logically centralized control programs to make network global decisions and send it down to network switches (data-plane). Dumped network switches (data-plane) received controller rules/decisions and process network packets accordingly if no decision found they ask the controller.
For example: In SDN architecture routing algorithms developed as a program in the controller, it collects all required metadata (e.g. switches, ports, host connections, links, speed, etc) from the network then make a routing decision for each switch in the network. While in a conventional network, a routing algorithm is implemented in a distributed fashion in all switches (i.e. generally each switch has its own intelligence and makes its own routing decision).
SDN explained by Nick Feamster
Here is a good paper that illustrates the road map to SDN
What is the Major difference in SNMP over DTLS and SNMPv3 USM model, which one to prefer.
TLS and DTLS make use of the Transport Security Model (TSM) security model, defined in RFC5591 which was created as an alternative to the USM security model
The difference is what is the problem each of them wants to resolve.
SNMP v3 USM enhances the SNMP message format itself to add proper integrity check and encryption, so that it can be transferred on probably unsafe wires.
SNMP over DTLS uses another approach that it tries to build safe channels on unsafe wires, and allows the messages to be transferred safely.
You can prefer either of them to achieve significant security improvements over the original SNMP, but please analyze your scenarios carefully and put the following into consideration,
What are the devices in your whole solution?
Whether they support both models? Which one of them only supports one of the model?
Then you can make a decision on which model to choose. As far as I know SNMP v3 USM is more widely supported, but I might be wrong.
I am trying to develop a publish/subscribe system.
To this end, I have read some papers and articles regarding it.
And they all talk about Messaging service as an integral part of publish/subscribe system.
My question is, can I develop a publish subscribe system without using MOM like JMS?
Or am I missing or oversimplifying things?
I do not think you are oversimplifying things. There are stand-alone products available that provide advanced functionality based on publish/subscribe, without being part of a larger MOM system.
One of them is a group of products implementing the Data Distribution Service (DDS) specification, as standardized by the Object Management Group (OMG). Check out this Wikipedia entry for a very brief introduction and list of references.
DDS supports many advanced data management features like a strong-typed and content aware databus, distributed state management and historical data access. Its rich set of Quality of Service settings allows to off-load a lot of the complexity from your applications to the middleware. This is all based on the publish/subscribe paradigm.
If you would tell more about your application, then I might be able to point you to similar use cases using this technology -- if you are interested.
It depends what you mean by "MOM". If you think MOM = JMS then yes, there are plenty of pub/sub applications which are not JMS servers (off the top of my head): 0MQ, TIBCO Rendezvous and the many AMQP implementations around.
I guess my definition of MOM is an infrastructure for reliably getting a message from one system to another in an asynchronous manner. Pub/sub is a feature on top of the message transport which allows a message to be distributed to multiple other systems. Once you get beyond the point of opening a socket and stuffing a bunch of bytes down it, I would argue you are in the realm of MOM.
So, no you don't need JMS to do pub/sub....there are plenty of open-source and closed-source alternatives out there. Which one depends on your requirements and skills.
You can look at multicast that provides one to many communication. Multicast does not require MOM, instead it requires multicast enabled IP network. Usually the network routers take care of creating copies of message and delivering messages to destinations.
I was browsing for an open source messaging software and after some good bit of research I came across these three products. I've taken these out for a preliminary test drive, having had them handle messages for queues and topics, and from what I've read all three of these products are good picks for an Open Source messaging solution for most companies. What I was wondering was what are the advantages that these products may have over one another? What I'm particularly interested in is messaging throughput, including persistent messaging throughput, security, scalability, reliability, support, routing capabilities, administrative options such as metrics and monitoring, and generally just how well each program runs in a large business environment.
Check out http://queues.io/
From their site:
The goal is to create a quality list of queues with a collection of articles, blog posts, slides, and videos about them. After reading the linked articles, you should have a good idea about: the pros and cons of each queue, a basic understanding of how the queue works, and what each queue is trying to achieve. Basically, you should have all the information you need to decide which queue will best fit your needs.
'messaging' covers a lot of options - and there must be at least a dozen different types of technologies that could be the right answer - having built many production messaging environments, using a variety of technologies/approaches, having a better understanding your requirements would help.
are you needing subject-based subscriptions? do you need multicast delivery? do you need dynamic subscribers/listeners? would your listeners be requerying for best sources even after finding an acceptable publisher/feed?
do you need guaranteed delivery? delivery confirmation? is you publisher storing any undelivered messages, or do you need the messaging system to do that for you automagically? how often does your feed data go stale - e.g. email-ish alerts can be store-and-forward but real-time pricing data is only valid for a short interval (and then probably needs to go away rather than cause confusion)
how volatile is your network topology? are your subscribers (or publishers) expecting to live at a fixed address? or are they mobile devices? could they appear to you over more complex internetwork topologies requiring registration and possibly imposing routing restrictions? if so any idea the frequency of these topology changes?
do you only need a java interface? are any of your subscribers to be integrated into windows components (like feeds into excel)?
if you're only interested in experience comparing the similar products you named then perhaps you have already thought through these topics.
as to products, in my experience Tibco is still the leader in throughput and scalability, especially in a real-time environment. ibm MQ would be next, especially in a store-and-forward architecture. with both of those products you get a level of support on which you can justify betting a fundamental part of your business systems. there's a reason both of those have been around for a couple of decades.
another often overlooked option is Tuxedo - it provides not only messaging but a proven transactional capability that remains unparalleled. Oracle continue to be committed to this product and, again, the level of support available is second to none.
i love open sourced solutions and am always glad to find production quality software for free - but if you are creating a fundamental part of your business infrastructure then an active community still might not indicate whether a particular voluntary project is the best bet.
my 2c worth. hope it helps.
First, I am no expert in this, but maybe I can give you some thought hints.
ActiveMQ and Qpid are both under the Apache umbrella and are message queues. But Qpid is an implementation of the AMQP specification.
AMQP is a protocol specification, on the wire level, so messages can be exchanged with other AMQP message queues (e.g RabbitMQ).
ActiveMQ and HornetQ are queues that you can use with a JMS API. The Java Message Service is a specification on an API level.
But you have the option to access Qpid via a JMS API, too.
I think performance is a secondary thought. To have an active community is more important.
http://x-aeon.com/wp/2013/04/10/a-quick-message-queue-benchmark-activemq-rabbitmq-hornetq-qpid-apollo/
Benchmark includes some performance numbers for you to decide, with both persistent and transient results.