I am not sure if the title of the question is correct, but please read the question.
I have been working on C/C++ for most part of my work life (close to 11 years). we only had C/C++ source/header files and all dependencies were managed by Makefiles. things were simple and manageable.
for the last 1.5 years i have shifted to Java domain. and i feel extremely irritated that most difficult aspect of working with anything new is the dependency managers. e.g. maven, leiningen, builder, sbt, etc etc etc.
whenever i download anything new from the open source world, there is a significant amount of time to be spent to just to setup the compilation, build, run environment. that too when i am using eclipse. why can't all the dependencies be placed along with the software to be downloaded?? why the tools like maven,leiningen, etc must make a separate internet connection to download the dependencies. i know that maven forms a local repository and should be able to find the dependency locally as it downloads whole internet anyway, but why is this model used. I am behind a firewall and not everything is accessible, and the tools fail to download dependencies. i am sure the same situation is there in most work environments.
recently i started with clojure, and boy it has been a pain to get eclipse configured for clojure. leiningen is supposed to be some magic which must be used with any clojure development. sometimes it feels learning leiningen is more important than learning concepts of clojure. i downloaded so called 'standalone' jar file for leiningen as 'self-install' was not working for me. but it fooled me. as soon as i run 'lein' command it is making an internet connection and trying to download somethings. WHY? it wont even print the help menu without connecting with the internet. WHY? there is no way i can fulfill its demands without bypassing my internet firewall, as i dont know, and no one can tell me what all things this guy wants. there is simply no other way.
And every one seems to be inventing their own. Java had ant which was simple, and went to Maven, some project uses Ruby based Builder, Clojure has leiningen, Scala has sbt. Go has something else. WHY? Why we need this added complexity in a world already full of complexity. why cant there be just one tool.
All you experts in Java technology please excuse my rant. I am sure this question will be downvoted and closed as from someone who is not trying hard enough to understand the things. But please believe me i have spend enough hours battling with this unnecessary complexity.
I just want to know how others get around this, or am i the only unfortunate one facing these issues.
I guess this question cannot accept an answer. I humbly can provide you with elements, hopefully they will help you get some perspective on the problem.
There are mainly 2 problems I identify with Java build systems:
some of them are declarative while others are using scripts
the fragmentation of the Java tools for building and exercising control is tied to people and Java stewardship of the space, not so much the technological choices.
Maven is the paramount of a method of defining your build using a formal grammar in a standard manner. Your pom.xml file contains a lot more than just your build : it is the identity of your artifacts, the project metadata, the modules and the plugins brought in. It treats with particular attention of the declaration of the dependencies and repositories.
Maven is declarative.
For a certain population of programmers, this is great, and they don't create new projects very often. It works well over time, it consolidates the build nicely.
Ant is a different system where you define tasks that will execute, chained in a particular order. All the definitions are made using XML and in effect, you are writing scripts and declaring how they will be stitched together.
Buildr (full disclosure: I am a committer there) is a build system which was created off the frustration of dealing with the inefficiency of the declarative approach for cases where the build needed to do additional steps and complex testing and the rigidity of using XML for a build. It is script-based, enforcing conventions over configuration (expecting a few good defaults, but letting you drive if you need to change things).
I am not familiar with Gradle and SBT but I think they extend and build on this approach, from what I heard.
So this gives you I hope a better picture of the landscape in terms of build tools.
The reason why no standard build tool emerged is probably tied to the fact Sun didn't push one with Java. Eventually, I think they adopted Ant (I have some most JSR jars being built with it). There also has been some products built in this space over extending some of those build systems ; there is always going to be a huge difference between people being paid to maintain code rather than doing it on the side.
And well, people argue. Build systems are a great way to start a flame war. We have a hard time agreeing on a standard though some of the common elements are now settling well around the Maven artifacts.
As for the need to download the Internet over and over again, it's a rather long story but here are a few things that may trigger the need for an unnecessary download:
any of the dependencies using SNAPSHOT will try to get the latest snapshot. This is a great scheme but it takes its toll. You might depend on something that depends on a snapshot, and get a download because of that.
Maven doesn't redownload the artifacts but sometimes checks md5. This is easy to fix, just use the -O option from the command line.
Tools like Buildr were built around fixing this issue once for all. First off, you only download what you said you would. Second, no connection is made again unless you asked for it. By default, Buildr doesn't play the transitive dependencies game though you can ask for it, but you have to do it explicitly.
I hope this was informative and that your journey in Java land becomes less painful going forward.
Related
I have been given a project to work on, and I know that in order to make efficient progress I need a good mental model of how things work together. I am supposed to be using an IDE of my choice, as well as Maven and Apache Tomcat to compile and build the project. I can do that, but the program is not fully functional, and because I don't understand how all the software works together, debugging is difficult. I have spent a lot of time researching how these things work, and what they do, but I am still not understanding. I know that Maven handles the dependencies, but I do not really understand what that means. I am confused by what Tomcat technically is, but I understand that it is needed to run dynamic programs. Beyond that, I do not understand how it works, or what it does. I also do not understand what happens between the IDE to the Tomcat server, as in, what is connecting them and how do I get my changes to affect what I see in my browser? Thank you!
Very short.
Maven builds your code by compiling the sources you write into byte code which the Java Virtual Machine can run, and include code written by others that your code needs. This is a good thing.
Tomcat has your compiled code (which needs to be in the form of a webapp) deployed to it, after which it can be run if appropriately invoked.
How the compiled code gets from your machine after Maven builds it, into Tomcat varies depending on the need of the team doing it, but during development this is often facilitated by your IDE - this typically also makes it easier to debug.
If at all possible, I would suggest finding a local mentor which can help you getting started because there is a lot of moving parts your need to have a basic understanding of.
I am developing a Java framework/API to solve a problem at a client. The code/idea is my property (not the client's). I think it might be useful for others, so I would like to publish it as a open source project.
By publishing I mean bringing it out in the open - making it available as a Maven project.
I can think of conforming to Maven structure, proper documentation/example usage available on a web site, and unit tests, maybe some code coverage threshold.
But does it have to be run by some committee? Do I have to present it to somebody? What steps do I need to take to eventually have it available as a Maven dependency?
There's no committee or approval process that I know of. All you have to do is put your code into a public Github repo. This is how open source software works.
Per Kapep's excellent suggestion below, you have to choose a license as well. Apache, Creative Commons, Gnu, MIT - these are a few of your choices. Know what they mean before you decide.
Your problem begins on that day - you'll have to make others aware of it and see if it's adopted by others. If it's good, you'll have the nice problems of dealing with a user base and having others change your code. If not, it'll languish in the repo.
What are the current downsides of Gradle? I have been researching different build tools but I haven't seen anything that seems to point out problems with Gradle as of October 2014. I have seen things a bit dated saying that Gradle users are on the bleeding edge. Is that still true or has Gradle reached decent point of maturity? (As far as I know, in terms of ide integration it might be more mature than others). Searching "why not use gradle" doesn't really help and "problems with gradle" shows people getting help (a plus). Most of what I have read were build tool comparisons and newer ones didn't list any flaws of Gradle.
Having not really used gradle except with libgdx projects I can't confirm issues presented in old comparisons still exist, but it seems like they don't.
The one thing I have seen that might be a problem is that it is "slow". If slowness is really an issue please explain how slow and what are the impacts.
Another somewhat reasonable downside is that people need to learn it to use it. To a person knowing how to use 0 tools this isn't really a problem and for others, it seems that Gradle is well documented and easy enough to learn.
I understand there are complications with switching and making sure the build still fully works so I am not asking what are the downsides of switching to gradle specifically but more generally why not use it now in a new project?
Looking for non opinionated reasons/problems.
I am currently using Gradle as part of Android Studio, and its working decently. For me I had a HUGE headache getting started because the default settings of Gradle requires internet connection for the initial build (I assume it was for updates) and it fails to build if it can't make communication with the internet the first time. (so there are some Firewall issue you might need to work around) So the only 'Con' I have to add is that Firewalls can be a problem with initial set up of Gradle.
There has been some discussion in abandoning our CI system (Hudson FWIW) due to the fact that our projects are somewhat segmented. Without revealing too much, you can think of each project as similar to a web site project: it has dependencies, its own unit tests, etc.
It seems like one of the major benefits of CI is to make sure that each component of a project works together, but aside from project inheritance most of our projects are standalone and unit tested fairly well.
Given what I have explained here (the oddity in our project organization); can anyone explain any benefits of CI for segmented\modular\many projects?
So far as I can tell, this is the only good reason I've found:
“Bugs are also cumulative. The more bugs you have, the harder it is to remove each one. This is partly because you get bug interactions, where failures show as the result of multiple faults - making each fault harder to find. It's also psychological - people have less energy to find and get rid of bugs when there are many of them - a phenomenon that the Pragmatic Programmers call the Broken Windows syndrome.”
From here: http://martinfowler.com/articles/continuousIntegration.html#BenefitsOfContinuousIntegration
I would use Hudson for the following reasons:
Ensuring that your projects build/compile properly.
Building jobs dependent on the build success of other jobs.
Ensuring that your code adheres to agreed-upon coding standards.
Running unit tests.
Notifying development team of any issues found.
If the number of projects steadily increases, you will find the need to be able to manage each one effectively, especially considering the above reasons for doing so.
In your situation, you can benefit from CI in (at least) these two ways:
You can let the CI server run certain larger test suites automatically after each subversion/... check-in. Especially those which test the interaction of different modules, hence the name continuous integration. This takes away the maintenance work and waiting time from the developers when they consider a check-in. Some CI (e.g. Hudson) also can be configured to automatically build modules when a depending module is build. This way you can let it automatically test if depending modules are compatible with the new version of the changed one.
You can let the CI server publish the new artifacts to the repository of a dependency resolver (e.g., Ivy, Maven). This way, the various modules can automatically download the latest (stable) revisions of the modules they depend on. Combine this point with the previous one and imagine the possibilities (!!!).
Right now a project I'm working on has reached a level of complexity that requires more than a few steps (actually its become arcane!) to produce a complete/usable product. And unfortunately we didn't start out with a Continuos Integration mindset, so as you can imagine its kind of painful at times, and at others I can easily waste half a day trying to get a clean/tested build.
Anyways as any HUGE project it consists of many components in many different languages (not only enterprise style Java or C# for example), as well as many graphical, and textual resources. Now the problem is that when I look for Continuos Integration, I always find best practices and techniques that assume one is starting a new project, from the ground up. However this isn't a new project, so I was wondering what are some good resources to proactively start migrating from Arcane Integration towards Continuos Integration :)
Thanks in advance!
Here it is in two simple (hah) steps.
Go for the repeatable build:
Use source control, get all code checked in.
Establish and document all tools used to build (mainly, which compiler version). Have a repeatable deployment and set up process for these tools.
Establish and document clearly any resources which are necessary to build, but are not checked in (third party installations, service packs, etc). Have a repeatable deployment and set up process for these dependencies.
Before commiting to source control, developers must
update their working copy
successfully build
run and pass automated tests
These steps can be done 1 at a time, sort of a path to follow. You'll get benefits at each stage. For example, if you aren't using source control at all, just getting the code into source control (without anything else) is a big step forward. Also, if there are no automated tests, then developers can't run them - but they can still get the prior commits and get the compiler to check their work.
If you can do all of these, you'll get to a nice sane place.
The goals are repeatable build processes and developers that are plugged in to how their changes affect the build and other developers.
Then you can reap the bonuses by establishing higher compliance:
Developers establish a frequent commit habit. Code that is in the working copy should never be more than 1 day old.
Automated build process monitors source control for check-ins and gets the results to a place where the users can accept them (such as a test environment, a preview website, or even simply placing an .exe where the user can find it).
The same way you eat an elephant (one bite at a time) ;-) Continuous integration requires an automated build. Start with that. Automate the building of each piece. Ant or NAnt is a great way to do this. Have each component's construction be a NAnt task. Then your entire system build can aggregate those individual tasks.
From there, you can add tasks for deployment, for unit testing, etc. If you want to use a CI technology, you can wire it up to your NAnt build.
I would start by first writing down all the steps it takes you to do the build and test manually. After that you at least have a guide for doing it the old way, and writing things down gives you the chance to look at it as a complete process.
Then look for parts to script.
Ideally you want to trigger a build and test from a code commit and only rebuild and retest the changed parts, with perhaps a full build and test nightly or weekly. You'll need log files or database entries and reports on the build success or lack of it.
You'll want to search out and evaluate pre-built products and open-source build-your-own kits. You can certainly write all the scripting and reporting yourself, but it will take a while and you'll probably end up with a just barely good enough reporting system since your job is coding the product, not coding the build system. :-)
I would guess that migrating isn't really an option--Half-ass solutions will only make it worse.
My approach would be to take one creative engineer who understands the build process, sit him down and say "Fix this". Give him a week or two.
The end goal would be a process that runs beginning to end with a single make command.
I also recommend an automated "Setup" procedure where you simply do a checkout and run a batch file from a network share to install and build all your tools. The amount of time this will save overall is staggering if you bring in new programmers. Most projects take one to three days to get set up on a new computer--and it's always the "new" programmer who doesn't know what's going on doing the installs on his own system...
In short: Incrementally
Choose a framework that will work across the diverse range of projects.
One by one, add components to the framework.
If you are not familiar with the framework, tackle a couple of the easier components first, to reduce risk of screwing up.
If you do understand the framework, tackle some of the more difficult and/or commonly built components first, so your team (and management) will appreciate the benefits early, and support the effort more.
Be sure to have a plan to include all of your components, because that's when the full benefit will be realized.
Bring your team with you; make sure you have consensus that this is going to be valuable, or people won't maintain it as the components change.