I am trying to introduce Pact in our company. However the consumer calls APIs using providers SDKs and the host-port is dynamically determined using Kubernetes. I am new to all this backend technology so trying to understand how do we deal with this since it will be impossible to get host/port into pom.xml if its dynamic?
It depends whether you're talking about port of the mock service in the consumer tests, or the port of the provider in the verification step.
In the consumer tests, is it possible to provide a test implementation of the part of the SDK that looks up the port? Perhaps you could contact the provider team to see they could supply one that would allow you to set a known port?
In regards to the provider, you would typically run the verification step against a locally running provider in the CI build, not against one deployed into a live environment, so a known port should be able to be used.
Related
I have an application separated in various OSGI bundles which run on a single Apache Karaf instance. However, I want to migrate to a microservice framework because
Apache Karaf is pretty tough to set up due its dependency mechanism and
I want to be able to bring the application later to the cloud (AWS, GCloud, whatever)
I did some research, had a look at various frameworks and concluded that Quarkus might be the right choice due to its container-based approach, the performance and possible cloud integration opportunities.
Now, I am struggeling at one point and I didn't find a solution so far, but maybe I also might have a misunderstanding here: my plan is to migrate almost every OSGI bundle of my application into a separate microservice. In that way, I would be able to scale horizontally only the services for which this is necessary and I could also update/deploy them separately without having to restart the whole application. Thus, I assume that every service needs to run in a separate Quarkus instance. However, Quarkus does not not seem to support this out of the box?!? Instead I would need to create a separate configuration for each Quarkus instance.
Is this really the way to go? How can the services discover each other? And is there a way that a service A can communicate with a service B not only via REST calls but also use objects of classes and methods of service B incorporating a dependency to service B for service A?
Thanks a lot for any ideas on this!
I think you are mixing some points between microservices and osgi-based applications. With microservices you usually have a independent process running each microservice which can be deployed in the same o other machines. Because of that you can scale as you said and gain benefits. But the communication model is not process to process. It has to use a different approach and its highly recommended that you use a standard integration mechanism, you can use REST, you can use Json RPC, SOAP, or queues or topics to use a event-driven communication. By this mechanisms you invoke the 'other' service operations as you do in osgi, but you are just using a different interface, instead of a local invocation you do a remote invocation.
Service discovery is something that you can do with just Virtual IP's accessing other services through a common dns name and a load balancer, or using kubernetes DNS, if you go for kubernetes as platform. You could use also a central configuration service or let each service register itself in a central registry. There are already plenty different flavours of solutions to tackle this complexity.
Also more importantly, you will have to be aware of your new complexities, but some you already have.
Contract versioning and design
Synchronous or asynchronous communication between services.
How to deal with security in the boundary of the services / Do i even need security in most of my services or i just need information about the user identity.
Increased maintenance cost and redundant side code for common features (here quarkus helps you a lot with its extensions and also you have microprofile compatibility).
...
Deciding to go with microservices is not an easy decision and not one that should be taken in a single step. My recommendation is that you analyse your application domain and try to check if your design is ok to go with microservices (in terms of separation of concenrs and model cohesion) and extract small parts of your osgi platform into microservices, otherwise you mostly will be force to make changes in your service interfaces which would be more difficult to do due to the service to service contract dependency than change a method and some invocations.
I have a functioning application using Spring Boot, Rabbit MQ & MySQL DB locally. I'm curious, how I can upload this app to the AWS Environment and get it working seamlessly.
The only part where I'm lost is how to get RabbitMQ in the cloud? Any suggestions?
I see three options for your needs :
Use AmazonMQ managed service. This uses ActiveMQ under the hood, and supports the AMQP protocol (so you can continue to use the RabbitMQ client). Here's an article on how to do it : https://aws.amazon.com/blogs/compute/migrating-from-rabbitmq-to-amazon-mq/.
Use a third-party managed service (such as CloudAMQP). This is similar to the first option, but you can choose a RabbitMQ provider if you wish.
Install RabbitMQ on an EC2 instance and manage it yourself. This is the most flexible option, but it will require more effort on your part and it will probably cost more. I would recommend this option only if you have special requirements that are not met by using a hosted service.
In all cases, I would also recommend to use a messaging library such as Spring Messaging or Apache Camel to isolate your code from your messaging implementation. This will reduce the boilerplate code you need for messaging and allows you to focus on your application logic.
I'm building an event-driven microservice architecture, which is supposed to be Cloud agnostic (as much as possible). Since this is initially going in GCP and I don't want to spend a long time in configurations and all that, I was going to use GCP's Pub/Sub directly for the event queue and would take care of other Cloud implementations later, but then I came across Spring Cloud Dataflow, which seemed nice because these are Spring Boot microservices and I needed a way to orchestrate them.
Does Spring Cloud Dataflow support Pub Sub as it's event queue?
Would it make my life easier in terms of configuration and setup going that path, rather than choosing a non native broker?
It'd be useful first to unpack the Spring Cloud Stream's "binder abstraction" because it is using this framework, you'd have a portable event-driven streaming application, which can run locally in your laptop or any cloud of your choice against the desired message broker.
Learn more about the binder-abstraction here. Here are all the available binder implementations of choice. Google PubSub is an option, and it is maintained by Google here.
Now, let's talk about Spring Cloud Data Flow (SCDF). Once when you have built the streaming applications, you could use SCDF to design+create a data pipeline made of such applications. There's the option to mix and reuse the collection of utility applications that we build, maintain, and release as well. The utility applications can be packaged with Google PubSub or other binders. More details here.
When you deploy the data pipeline, SCDF will resolve and download the individual applications to deploy them natively on platforms like Kubernetes or Cloud Foundry. We have users doing the same in a variety of cloud infrastructure (VMs, Bare-metal, EC2, Rackspace, etc.), including DIY platforms, too.
While also automating the deployment of the applications, SCDF will automate the configuration setup based on naming conventions derived from stream/task and application names as a combination. So, when the apps bootstrap, they would have automatically received the connection configurations (from SCDF) and as well the destination/topic to connect to along with the other metadata to reason through a collection of apps as a "stream" or a "task/batch" data pipeline. This allows you to monitor and manage the pipelines centrally.
Lastly, there's the native ability in SCDF to rolling-upgrade/rolling-downgrade 1 or many applications in a data pipeline without impacting the upstream or downstream consumers in production. More details here. There's a webinar recording (demo starts at ~41.25) on how to do with CI/CD automation.
In terraform we get a state file, and CloudFormation also has a notion of a working state. How does go cloud handle the state, do we have to create it ourselves?
For more info on Go Cloud
https://github.com/google/go-cloud
https://godoc.org/github.com/google/go-cloud
Terraform wants to solve the problem of managing and provisioning Cloud services.
Go Cloud wants to solve the problem of using Cloud services in application code.
So, they work well together. For example, the Go Cloud sample guestbook app (https://github.com/google/go-cloud/tree/master/samples/guestbook) uses Terraform to provision the resources needed to run the app on various Cloud providers; the application code in the sample has a small amount of provider-specific setup code, but the application logic itself is provider-agnostic.
go-cloud:
The Go Cloud Project is an initiative that will allow application developers to seamlessly deploy cloud applications on any combination of cloud providers. It does this by providing stable, idiomatic interfaces for common uses like storage and databases. Think database/sql for cloud products.
Terraform:
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.
So with go-cloud you could create a tool like terraform that, for now can provide generic APIs for:
Unstructured binary (blob) storage
Variables that change at runtime (configuration)
Connecting to MySQL databases
Server startup and diagnostics: request logging, tracing, and health checking
My organization is using Websphere Application Server with RAD. My unit is developing Web Services that take data from a consumer, and often pass the data through other Web Services.
Currently, we have two ways of defining time-out's for data to be returned from called services:
Using the Spring framework
Websphere profile configuration
If we need to update the values using Spring we need to re-deploy our service. If we define them as JVM properties we need to do a JVM configuration change and 'restart/recycle/stop and start' the application.
My co-worker had it suggested to him that there may be some better solutions to this problem. I'm wondering if this is true, and if so what they are?
edit:
One option we might be considering is "Application Policy Sets". We'd like to know if this is a good alternative method:
can we define Spring to leverage policy sets
can we define multiple policy sets for an application when timeout values for different services called by our service are different?
Does this console change take affect at run-time, or do we need to recycle?
I suggest you use JAX-WS policy sets and bindings on WAS level, because it allows you to:
define policy set for each web service separately
in policy set define web service parameters, from HTTP transport parameters to WS-Security policies such as UsernameToken, digital signature and encryption
web services remain unchanged, the code remains the same
there is only one place where you change it: in WAS console, application is not changed
The only drawback here is that web services should be generated from RAD (you can use top-down or bottom-up approach).
Please see more details in this excellent 3-part DeveloperWorks tutorial