I've been reading RFC3414 about User-based Security Model (USM) for version 3 of the Simple Network Management Protocol (SNMPv3) , and it says :
There are at least two threats that an SNMP Security Model need not
protect against. The security protocols defined in this memo do not
provide protection against:
Denial of Service This SNMP Security Model does not attempt to
address the broad range of attacks by which service on behalf of
authorized users is denied. Indeed, such denial-of-service attacks
are in many cases indistinguishable from the type of network
failures with which any viable network management protocol must
cope as a matter of course.
Traffic Analysis This SNMP Security Model does not attempt to
address traffic analysis attacks. Indeed, many traffic patterns
are predictable - devices may be managed on a regular basis by a
relatively small number of management applications - and therefore
there is no significant advantage afforded by protecting against
traffic analysis.
I didn't quite understand how SNMP need not protect against them. Can someone explain this clearly please ? Thank you very much.
Related
If protocols built with Polybase base their own rewards on DB data, could a bad DB indexer not abuse its storage power?
I just wanted to understand more on the Polybase incentive structure and how you guys make sure that there is no bad actor in the network? If there is a bad guy, how do you deal with it?
Using zk-snarks means that most clients on the network can easily validate an action end-to-end and refuse to accept an invalid one. Therefore, you would get slashed, and still not achieve your objective.
The root hash/proof on the blockchain is the most important, as it validates all other proofs, and that inherits its security from the L1 (e.g. Near or Ethereum).
The slashing is mostly used to prevent people from spamming the network with false claims.
Source: Polybase whitepaper (in the "Economics" section).
Note: I'm a founder of Polybase.
For microservices, the common design pattern used is API-Gateway. I am a bit confused about its implementation and implications. My questions/concerns are as follows:
Why other patterns for microservices are not generally discussed? If they are, then did I miss them out?
If we deploy a gateway server, isn't it a bottleneck?
Isn't the gateway server vulnerable to crashes/failures due to excessive requests at a single point? I believe that the load would be enormous at this point (and keeping in mind that Netflix is doing something like this). Correct me if I am wrong in understanding.
Stream/download/upload data (like files, videos, images) will also be passing through the gateway server with other middleware services?
Why can't we use the proxy pattern instead of Gateway?
From my understanding, in an ideal environment, a gateway server would be entertaining the requests from clients and responding back after the Microservices has performed the due task.
Additionally, I was looking at Spring Cloud Gateway. It seems to be something that I am looking for in a gateway server but the routing functionality of it confuses me if it's just a routing (redirect) service and the microservice would be directly responsible for the response to the client.
The gateway pattern is used to provide a single interface to a bunch of different microservices. If you have multiple microservices providing data for your API, you don't want to expose all of these to your clients. Much better for them to have just a single point of entry, without having to think about which service to poll for which data. It's also nice to be able to centralise common processing such as authentication. Like any design pattern, it can be applied very nicely to some solutions and doesn't work well for others.
If throughput becomes an issue, the gateway is very scalable. You can just add more gateways and load balance them
There are some subtle differences between proxy pattern and API gateway pattern. I recommend this article for a pretty straightforward explanation
https://blog.akana.com/api-proxy-or-gateway/
In the area of microservices the API-Gateway is a proven Pattern. It has several advantages e.g:
It encapsulate several edge functionalities (like authentication, authorization, routing, monitoring, ...)
It hides all your microservices and controls the access to them (I don't think, you want that your clients should be able to access your microservices directly).
It may encapsulate the communication protocols requested by your microservices (sometimes the service may have a mixture of protocols internally which even are only allowed within a firewall).
An API-Gateway may also provide "API composition" (orchestrating the calls to several services an merge their results to one). It s not recommended, to implement a such composition in a microservice.
and so on
Implementing all these feature in a proxy is not trivial. There is a couple of API-Gateways which provide all these functionalities and more like the Netflix-Zuul, Spring-Gateway or the Akana Gateway.
Furthermore, in order to avoid your API-Gateway from being a bottleneck you may :
Scale your API-Gateway and load balance it (as mentioned above by Arran_Duff)
Your API-Gateway should not provide a single one-size-fits-all API for all your clients. Doing so you will, in the case of huge request amount (or large files to down/up load) for sure encounter the problems you mentioned in questions 3 and 4. Therefore in order to mitigate a such situation your Gateway e.g may provide each client with a client specific API (a API-Gateway instance serves only a certain client type or business area..). This is exactly what Netflix has done to resolve this problem (see https://medium.com/netflix-techblog/embracing-the-differences-inside-the-netflix-api-redesign-15fd8b3dc49d)
1.Why other patterns for microservices are not generally discussed? If they are, then did I miss them out?
There are many microservice pattern under different categories such as database , service etc .This is a very good article https://microservices.io/patterns/index.html
2.If we deploy a gateway server, isn't it a bottleneck?
Yes to some extent .Q3's answers image will answer this.
3.Isn't the gateway server vulnerable to crashes/failures due to excessive requests at a single point? I believe that the load would be enormous at this point (and keeping in mind that Netflix is doing something like this). Correct me if I am wrong in understanding.
4.Stream/download/upload data (like files, videos, images) will also be passing through the gateway server with other middleware services?
Why can't we use the proxy pattern instead of Gateway?
The use case for an API Proxy versus an API Gateway depends on what kinds of capabilities you require and where you are in the API Lifecycle. If you already have an existing API that doesn’t require the advanced capabilities that an API Gateway can offer than an API Proxy would be a recommended route.
You can save valuable engineering bandwidth because proxies are much easier to maintain and you won’t suffer any negligible performance loss. If you need specific capabilities that a proxy doesn’t offer you could also develop an in-house layer to accommodate your use case. If you are earlier in the API lifecycle or need the extra features that an API Gateway can provide, then investing in one would pay dividends.
I was poring over the docs for Open DayLight, and can't seem to wrap my head around what software-defined networking even is. All the media hype, blogs and articles I can find on SDN are riddled with buzzwords that don't mean anything to me as an engineer. So I ask: What (exactly) is SDN? What are some specific uses cases/problems it solves? Is it:
Just making proprietary networking hardware serve network APIs, thus allowing programs to configure them (instead of IT guys using a console or web interface)?; or
Implementing (traditionally proprietary) networking hardware as software; or
Writing software that somehow integrates with virtual networking hardware used by virtualization platforms (vLANs, vSwitches, etc.)?; or
Something else completely?!?
BONUS: How does Open DayLight fit into this equation here?
First of all, you are right, there is not official definition from NIST or some similar standardization body and the fact that its meaning is fuzzy is exploited by marketing people.
The main point of SDN is that it allows to program network functions with APIs.
In the past, networking devices like switches and routers were only configurable using a proprietary interface (be it vendor specific tools or just the CLI on the device) and there were no APIs which allow to configure OSI L2 - L3 aspects like VLANs and routes but also L6 - L7 aspects like load balancing highly dynamically. Btw. In the case of L6 - L7 functions, the term NVF = Network Virtualized Function seems to be established by now.
This is needed especially for multi tenancy capable virtualized IaaS systems. You can create new VPCs and arrange them together at will. To really isolate tenants from each other, you need to have a L2 isolation and so the same dynamics that is offered for VPCs is propagated to the networking for interconnecting them.
Conclusion: It is about your first bullet with the extension, that the APIs must not necessarily be offered by some hardware appliance, it can also be offered by some pure software implementation.
Regarding OpenDaylight:
It is the OpenStack pendant for SDN. They also actively push integration with OpenStack. They say they are an "open, reference framework for programmability and control through an open source SDN and NFV solution". This means it provides (as you say) a façade for the manfold aspects of networking.
They have all the big names as members which probably means they have the power to establish a de-facto standard like OpenStack did. Members benefit in that they can provide plugins, integrations and adaptations for their products so that they seamlessly integrate with OpenDayligh and you only need to care about a single standard API.
SDN is programmable networks. Different SDN solutions provide different functions in their APIs towards the app developer.
There is a good overview of SDN for software developers here:
https://github.com/BRCDcomm/BVC/wiki/SDN-applications
The most common elements for SDN solutions are
North-bound API: A programming interface used by an application/script to monitor, manage and control the network topology and packet flows within the network.
Network elements: Switching or routing network elements that enforce the rules provided by the application via the north-bound API. These elements may be physical (Cisco, Brocade, Tallac, etc) or virtual (Open VSwitch, Brocade Vyatta vrouter, Cisco 1000, etc) or a combination.
Controller-based solutions have a clustered architectural element (the 'controller') that provides the north-bound api towards applications and an extensible set of south-bound APIs to which network devices connect. Some controllers available today are OpenDaylight, Open Network Operating System (ONOS), Juniper Open Contrail, Brocade Vyatta Controller (ODL distribution), HP VAN Controller and more.
Best rules of thumb to understand an SDN offering:
Read its north-bound API - this tells you what you will be able to monitor, manage and control in your network.
Find out which south-bound APIs it supports - this tells you which switches/routers it might work with.
Some SDN use cases/applications:
DevOps/Admin automation - Applications and scripts that make a network admin or DevOps life easier through automation. OpenStack Neutron is a common example.
Security - HP provides 'Network Protector' that learns the topology of the network and then monitors activity providing alerts and/or remediation of non-compliant behaviors.
Network optimization
Brocade offers 'Traffic Manager' that monitors network utilization and modifies traffic flows in real time to optimize quality based on defined policies.
HP provides 'HP Network Optimizer' that provides an end-to-end voice optimized path for enterprise Microsoft Lync users.
Lyatiss provisions AWS networks in realtime to meet application needs.
Monitoring classroom time-on-task - Elbrys provides an application that provides a teacher with a dashboard to monitor student's time-on-task in real time and cause redirects of individual students to web pages of their choosing. (Disclaimer: I work for Elbrys Networks)
OpenDaylight project proposals page - https://wiki.opendaylight.org/view/Project_Proposals:Main
The concept of SDN is very simple. SDN decouples control-plane (i.e. decision making) from data-plane (the actual forwarding actions) and provides API between them (e.g. OpenFlow API).
Image source: https://www.commsbusiness.co.uk/features/software-defined-networking-sdn-explained/
With SDN architecture, network engineers no longer have to learn proprietary CLI commands for different vendors. They can focus on developing logically centralized control programs to make network global decisions and send it down to network switches (data-plane). Dumped network switches (data-plane) received controller rules/decisions and process network packets accordingly if no decision found they ask the controller.
For example: In SDN architecture routing algorithms developed as a program in the controller, it collects all required metadata (e.g. switches, ports, host connections, links, speed, etc) from the network then make a routing decision for each switch in the network. While in a conventional network, a routing algorithm is implemented in a distributed fashion in all switches (i.e. generally each switch has its own intelligence and makes its own routing decision).
SDN explained by Nick Feamster
Here is a good paper that illustrates the road map to SDN
I'm designing a RESTful Service Oriented Architecture web application to make it scale as good as possible and put different kind of services on different machines (separating resource intensive operations from other services).
I also want users to be able to access their data to make their own applications.
I'm not sure if I have to design these services to be opened to the world, so it's just a matter of make them listen on a web domain (like AWS) or create another service to handle API requests.
It makes sense to me to have secure opened webservices, but it does add a lot of complexity to the architecture itself because each service becomes a client that has to be recognized (trust) by other services in the same suite, just as well as I have to recognize 3rd party applications trying to access their own data.
Is this a right SOA approach? What I want to be sure is that I'm not mixing wrong concepts designing a wrong service oriented architecture.
All services have crud interfaces so they could be queried using REST principles.
Depending on the nature of your system, it may be viable to have unsecured webservices, so they can all talk to each other without the security overheads. To make the services available to 3rd parties, you could then use a Service Perimeter Guard as the only mechanism for accessing the services externally and apply security at this layer. This has the benefit of providing consistent security across all of your services, however if the perimeter is compromised then access to all of the services is obtained.
This approach may not be viable for all services. For instance information that is considered "personal-in-confidence" (e.g., employee data such as home addresses, emergency contact details, health data, etc), will need to be secured so that unauthorised staff cannot access it.
Regarding your comment of putting different services on different machines, this will result in under-utilised resources on some machines and possibly over-utilised resources on others. To avoid this, deploy all services to all machines and use a load-balancer. This will provide more optimal resource usage and simplify deployments (e.g., using Chef or Puppet) as all of the nodes are the same. As the resource usage increases, you can then simply add more nodes. Similarly if the resource usage is low, you can remove nodes.
Regarding your last sentence, there is a whole lot more to REST than CRUD (such as HATEOAS).
Where would I go to learn write code that had to be very, very secure and that DOES expose external services (running on a standard Windows or Linux OS). Knowing what services can and cannot be safely exposed would be part of the issue. Note that I am not looking for a favorite choice between Linux and Windows, as the choice is not likely to be mine to make in any given case. However the level of security needs to be military grade.
I almost feel embarressed giving this as a for instance, but how would I know whether or not I could use, say, WCF, in such a setting.
High security is a difficult concept as it generally involves way more than just the code you wrote.
Basically every layer of the OSI model has to be taken into consideration. Things like, preventing capture of the data stream (or it being rerouted) between the end points (quantum cryptography).
At the higher levels, you have things like various things like
Physical security of the devices (all endpoints if possible).
Hardening the OS (e.g: closing ports, turning off unused services, using kerberos, VPN tunnels, and leveraging white lists of machines allowed to connect, etc);
Encrypting the data at rest (file encryption), in transmission (SSL), and in memory (column/table encryption).
Ensuring and enforcing proper authentication and authorization at every level (in app, in sql, etc).
Log EVERYTHING. At a minimal it should answer "who/what/when/where/how"
Along with the logging, Actively Monitor it. aka: intrusion detection.
Then we can move on to other things like looking at other attack vectors like sql injection, xss, internal / disgruntled employees, etc.
And once you've done all of that be prepared when a hacker gets away with everything they want simply by social engineering.
In short, the best tact to take in order to secure any computer related application is to listen to the ethos of Fox Mulder, and Trust No One. Another favorite of mine that applies is: It's only paranoia if they aren't after you.
You could use formal methods to (sort-of) prove the critical parts of your software. A tool like Frama-C (free, LGPL license, targetting embedded systems) could be relevant (at least if your software is critical, embedded, written in C).
But military grade don't mean much. Your client will (and should) define exactly the standards to respect. For instance, critical [civilian] aircraft software needs to follow something like DO-178C (or its predecessor, DO-178B). Different industries have different standards similar to that. (both railways and medical industries have their own standards, which might be different in North America than in Europe).
If your system (& client) is less demanding (i.e. no billion dollars or hundreds lives threatened by bugs) you could consider customizing your compiler or using some other tool. For example, GCC is customizable thru plugins or thru MELT extensions.
Don't forget that software reliability has a big price (that means a big cost for you, hence for your client).
Well, the question of where can be answered simply. Not in school. I suggest to create a learning path for yourself. Pick a technology that you like and learn it inside out. A basic book to get you started should suffice, however the rest of the stuff you learn as you go, or via the documentation of that technology.
For instance - learning under .NET (Microsoft) involves a basic A-Press text-book (i suggest Pro C# and The .NET 4.0 Platform). Thereafter searching through the .NET Framework Reference on MSDN will give you the rest.
If you are looking for WCF reference, I suggest the (MCTS Exam 70-503, Microsoft .NET Framework 3.5 Windows Communication Foundation) and MSDN.
Just keep in mind that not a single technology will achieve what you are looking for. For example: WCF co-mingles with WF (Windows Workflow Foundation), as well as SQL Data Services and Entity Framework. Being exposed to multiple technologies will definitely broaden your vision.
===============================================================================
WCF is a beast in this regard. Here are the advantages over some other means of communication:
Messages (data) passed between end points can be secured via message-level security (encryption). The transport channel chosen can also be secured at protocol level via transport layer security (encryption).
End points themselves can authorize and impersonate clients (client level security). You can implement end-to-end service tracing, health monitoring & performance counters, message logging, as well as forward and backward compatibility with newer/older clients (via graceful degradation of the message format, provided in WCF). If you chose to do so, you can even implement routing as fail-safe for your communications channel. WCF also supports transactions (ACID), concurrency, as well as a per-instance throttling, giving you the most flexibility in writing secure/robust military grade code.
In retrospect the security and flexibility of WCF are astonishing. A similiar technology (if not the same) is the WS-Security spec. It is part of the WS-* specifications for web services and deals with Xml signature and Xml encryption to provide secure communications channel between two end points.
The disadvantages of WS-* however is that it is a one-way means of communication. WCF can facilitate 2 way communication. A client can send a request to a server, but also a server can send requests to the client. WS-* dictates that a client can only send and receive responses to the server, but not vice versa.
I am not a WCF developer so i thought the highlights might provoke you into doing your own research. "There are hundreds of ways to skin an animal, neither of them is wrong..."