This is going to be a broad topic so please bear with me. So I build a microservices app started as a hobby and now in the few months I put into it I made something useful. So, far I used STS (spring tool) with maven and Eureka client.
I have an independent microservices talking to each other and a UI microservice that present the results to you. It's a single page app I will lay down the structure (GitHub repo also same) :
--my-app
--my-microservice-discovery
--my-microservice-domain (jdk12)
--my-microservice-searcher
--my-microservice-orchestrator
--my-microservice-ui
--my-transferobjects (common jar not microservice)
--pom.xml (my main module pom)
So, this is what I am dealing with in GitHub I made a single repository of my-app containing all these spring boot projects, in IDE everything works, now comes the deployment part on some cloud provider. I choose Heroku as I had some experience with it in the past but also because I cannot make this work with Google (although they have an attractive starter scheme going on right now). Connecting to a private GitHub repo from Google Cloud build is pain, but Heroku does this with style.
I decided to go command line because that's how I have to deal on the cloud and that's where all hell broke loose, I got lots of dependencies issues between the version of JDKs managed well by IDE but not defined correctly for maven yet.
I am still managed to make my local build success (command line) but I had to do some hard-code configuration to fix jdk12 for a my-microservice-domain pom like below and similar fix for my-transfer objects but because of the Lombok issue no idea why I have to provide jdk8 specifically for this project.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>12</source>
<target>12</target>
<executable>/home/Downloads/jdk-12</executable>
</configuration>
</plugin>
</plugins>
</build>
My questions are as follows :
Do we have a toolchain example to handle this different JDK compilation issue plus do we always have to provide the JDK local installation path?
How to deploy this on Heroku, how it will know where is my jdk12 is, or just defining the source/target version in pom will do the trick on the cloud, also Heroku supports jdk12 or google cloud?
Is it a good idea to have multi-repo or single repo deployment I can still change but I like it this way.
I get it that I need to create docker images for each of my microservices, but someone has a tutorial to do so locally first, or some GitHub repo so that I can look some examples.
Once I add all those docer files in each microservice is this sufficient to deploy its production level? I read a lot about APIGateway, load balancer .. where they fit into my architecture.
I also use localhost everywhere for EurekaServer and Eureka/Feign client properties will it also work the same way on the cloud and Eureka server will be able to find all my services as it does locally with no config changes needed on cloud?
What is better Google cloud or Heroku, google cloud seems a bit of a hassle for now.
These are my worries please advise.
Ok I am going to answer my own question I did bit of reading and what #cool suggested above and I ended up going multiple repository way and I achieved what was needed.
I also chosen Heroku simply because the ease of things there felt like my local environment and such a simple app like mine on latest binaries has no problem whatsoever.
I followed few links for setting up my Eureka server and client and Procfile and some environment variable direcrlt from your Apps settting page from dashboard.
Needless to say I also maintained multiple profiles (dev,prod,test) and for ui I use vaadin hence some additional step needed in your UI app pom.xml for production.
I am bit of concerned with the way my services were terminated of no activity on heroku plus right now the issue with services discovery among each other. So my eureka server reports > all instances but they cannot contact each other.
Right now I am busy with other things [viz. fixing bugs] as launch is a month later. I will soon post a question about this ;).
Related
Is there any way by chance to make spring config server to clone only specific path instead of whole git repo? When I'm running it on production I may not want my entire code base repository to be cloned to some location as it is always a risk. I've done my research and couldn't figure out ways to clone only specific paths of git repo.
One solution is to host the configurations in a seperate repo. But that would defeat the purpose of one code many deploys suggestion of 12factor.net.
Another possibility is to copy the properties into config server's classpath and then to use native profile to load them. But this would defeat the purpose of Spring ConfigServer.
Also kindly do clarify What would be the best way to run spring config server on a production?
You should be separating your code base from your configuration settings.
Repo 1: Your App Code
Repo 2: Your Spring Cloud Config Server Code
Repo 3: Your configuration settings
When you're deploying, you're deploying the code. Not the configuration. A configuration change should not require a new deployment. A new deployment might require an updated configuration change, but the two are generally decoupled.
Configuration can be dynamically updated while code is running. This is completely separate from updating code and deploying the app. That's the whole point of externalized configuration.
TL;DR: You're looking at "one code many deploys" wrong. Configuration is externalized.
I took this straight from 12factor.net
Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.
Even though you are no longer storing them as constants but still in the same repository is not enough separation from the code. Use a separate repository for configurations.
I am having three modules in my Maven project:
parent
rest-api # Spring REST API
web-client # AngularJS web client
application # Project to bundle it all up for a standalone
I am not sure if I have here an "elegant" solution so please hit me with a stick if that is complete garbage, but this is how it works - or how it's supposed to work:
rest-api
Module rest-api does simply offer the REST API and other core functionality - basically it is pure server code. It is a jar artifact.
web-client
To separate client and server code I am having the module web-client. It is a war project that hold the client webapp.
application
This module is supposed to glue it all up. It depends on rest-api and web-client. It does two important things:
It's pom.xml uses the spring-boot-maven-plugin to build a standalone runnable jar - my ultimate goal
It provides the main(String args[]) method that starts the #SpringBootApplication with SpringApplication.run(EasyModelAccessServer.class, args);
What I am currently able to is tell Eclipse to run this in a servlet container. The server boots up and I my two resources, the rest-api and the web-client working. Everything is fine so far.
The issue
What is not fully working is the standalone. If I package up the whole thing and run the server:
$ path/to/application: mvn clean package
$ path/to/application: java -jar target/application.jar
Only the REST API will work. The reason is because the web-client is not added or introduced as a web app to the Spring built-in Tomcat.
The question
is how I can make this work. There are two options which come to my mind:
Somehow sneak in the web-client.war file into the application.jar such that it is available as a resource and programmatically call tomcat.addWebapp("/web-client", "path/to/web-client.war") (or something like that) to load the additional service
Hope that the spring-boot-maven-plugin or another Spring Maven plug-in can do that for me and find somebody that links me to an example.
I've tried it with 1. but I didn't succeed to move web-client.war into application.jar but I am also not sure if I should actually do that.
FAQ
Q: "Why do you separate everything instead of merge all those modules into a server module where the Spring Maven plug-in would do everything for you out of the box?"
A: I really want to separate the client code from the server code. I could however merge web-client into application but last time I tried that I had 10 other issues why this did not work out so I decided to keep it that way and that it actually shouldn't be so hard to load an additional server resource.
Q: "Can I take a look at the project?"
A: Yes, you can. Just take a look: https://github.com/silentsnooc/easy-model-access Please forgive me that I am currently using tabs instead of whitespaces - I am going to change that as soon as I got everything up and running.
Apologies for the vagueness of this question.
Is it possible or, how can a ninjaframework application be deployed on Openshift.
Basically, I have a Ninjaframework application running locally in superdev mode. I can even build a war file. However, when I push it to an Openshift git repository it simply doesn't work. Specifically, I get a vague "Oops. That's an internal server error and all we know." page when I try to access the site.
TBH, I'm not surprised. I need to somehow specify to Openshift how to build the application. I'm hoping this can be done by some Maven configurations etc. but I haven't the faintest idea how?
Any help or pointers very much appreciated.
Yep, cool, thanks guys. I'm afraid i was so clueless on how to do this that logs would only tell me what I already knew; that simply git committing a NinjaFramework application configured to build as a fat jar (by default per the archetype) was never going to work in a Tomcat application server. Apologies if that didn't come across in the question.
I expected some elaborate Maven stuff would be required but luckily I was wrong. It turns out that a simple Maven change was all that was required to have the application deployable in OpenShift (Tomcat 7).
I've outlined the steps here in tutorial form: http://outbottle.com/deploying-a-ninjaframework-application-on-openshift/
I'm building "read-only" webservice (Google Cloud Endpoints as backend for an Android App) so I created a project using maven:
mvn archetype:generate -Dappengine-version=1.9.10 -Dfilter=com.google.appengine.archetypes:
and selecting archetype hello-endpoints-archetype to have some sample code to work on.
This works well and my app is correctly calling the service as expected (and the service is correctly supplying the data in return).
Now I have to implement an "update" service to periodically (4 to 6 times a dya) update the data supplied to the app. So I added a servlet to my project to be called by cron. Trouble is: one of the library used during this update uses multi-threads which cause an AccessControlException to be thrown because apparently multi-thread is only allowed in backend modules...
But after having read dozens of pages on google app engine, I still don't know how to "break" my application into modules so that particular servlet would be run as a backend module while the already existing servlet keep working as they do. So far, all I got was that I should use an EAR application composed of several WAR modules, but I don't even know if my current application is an EAR or not...
I'm using Eclipse Luna, maven 3.2.1 (embeded with Eclipse), google app engine 1.9.10, writing in Java
Could anyone please help me by explaining the directory structure and/or configuration files I have to look at, modify and/or add?
Thanks for any help provided!
You can find an example of multi-modules project here.
However, note that even in backend modules the threading is limited to 50 threads, as stated here.
More of a standard practice questions:
Is there any difference in deploying an app as EAR vs WAR? How do you decide? (I know WAR is just a web application may or may not have Java EE features like messaging)
Lets say I have a Spring MVC application stack with Hibernate (MySQL DB), should this be deployed as a War or EAR?
When do we need to worry about JBoss deployment descriptors, if I am not using EJBs. (Just Spring MVC). Lets assume I have JMS as well. Do we need to configure/update/create any other JBoss related config files?
When we package our application EAR/WAR, it include EVERYTHING that we need for our app. Is there a scenario where we need to keep some config / xml files outside of this archive in a specified JBoss folder?
Is it common practice to deploy directly from Eclipse or better to use Ant, etc? Advantage / Disadvantage?
Obviously, I am a newbie :-). Trying to understand this.
1.
This is not always an easy decision, but for beginners and for small projects I would say it's nearly always a WAR. The reason for using an EAR is mainly to isolate a business layer from a UI/Web layer. See this question for more details: How can one isolate logical layers of an Java EE application
2.
I might be mistaken but I think that Spring people typically prefer WARs.
3.
JBoss (vendor) specific deployment descriptors are mostly needed to configure so-called "administered objects" and security. Sometimes they can be used for extra features that are not covered by the Java EE specification (e.g. setting the web root for a WAR). Administered objects are typically data sources (connection to a database) and JMS destinations (queues and topics).
In the traditional Java EE approach these have to be created as far away from the code as possible, which typically means a system admin would create them inside the target AS using some kind of GUI or admin console. In this setup, you as developer would throw a WAR with "unresolved dependencies" over the wall, and a system admin (or "deployer") would then spend days figuring out what those unresolved dependencies should be.
If the communication is relatively good between developers and deployers, the WAR or EAR might be thrown over the wall together with a readme-file, that at least gives some insight into which resources are needed. Depending on the organization the development team might not get any access or feedback about how those "unresolved dependencies" have been resolved. E.g. a data source with a max of 5 connections may have been created, but this may be insufficient if some code does say 10 parallel queries. Without the development team knowing the exact data source configuration, some classes of runtime problems and performance issues may be relatively hard to solve.
To mitigate these problems, some vendors, for some artifacts, offer the developer to create those "unresolved dependencies" instead using proprietary deployment descriptors which are then embedded in the WAR or EAR. For simple local JMS destinations this is then in most cases the end of it, but for data sources there is a little bit more to it. Namely, there has to be a mechanism to switch between data sources for different stages such as Dev, Beta, QA, Production etc. Additionally, it's rarely a good idea to have production passwords in the source code.
If you have a simple app that you want to try out locally, stages and production passwords are not a concern. If you deploy for a (large) company it is.
In Java EE 6 you can define a data source using a standard descriptor (web.xml, ejb-jar.xml or application.xml), and in Java EE 7 you can do the same for JMS destinations. There is no standard way to configure those based on stage, but there is a glimmer of hope that Java EE 8 will address this (see e.g. JAVAEE_SPEC-19). Vendors are not universally happy with those standardized methods, and their main documentation will almost always extensibly tell you how to do those things using their proprietary tools and descriptors, and if you're lucky as a small note tell you there's a standardized way (and then sometimes downplay that or scare you by saying it's not recommended to be used in production).
4.
See answer to 3 mostly. One option to solve the problem of how to switch between stages and keep production passwords out of the WAR/EAR, is to have the full definition of said data source inside the AS (inside JBoss in your case). Every AS installation is tied to a specific server in this setup. If data sources need to be updated, removed or new ones added, you have to communicate with your operations team (if any). As said, depending on your organization this can be anything between trivial and practically impossible.
5.
When developing you most often use your IDE to do a deployment. For production you would never do that. For production you may build with Ant (or Maven) and deploy via something like Jenkins, or e.g Chef.
Check here : .war vs .ear file
If you read the preceeding response, you'd guess that "WAR" it is.
Deployment descriptor are needed to manage the modules of JBoss, if you don't have any conflict or don't need any tweaking, you won't need any deployment descriptor.
You may need to play with some JBoss file if you want to add modules to JBoss, or configure datasources, etc. Read the JBoss documentation for more info.
You can deploy from eclipse during your development phase, but as your other environments (qualification, production, test, etc) should be separeted from your developing one and that they won't have any eclipse installed on them, you should get used to manage your server from the command line and drop your war's in the right directories.
It's a short answer, but I hope it will help.
Read JBoss documentation for more info.