Task throws error=7: Argument list too long - shell

Question

I have an Ant's build.xml file which executes with no problems on my machine (Ubuntu), but throws the following error:
/var/lib/hudson/workspace/myproject/build.xml:254: Error running /var/lib/hudson/tools/java_6/bin/javac compiler
at org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:525)
(...)
Caused by: java.io.IOException: Cannot run program "/var/lib/hudson/tools/java_6/bin/javac": java.io.IOException: error=7, Argument list too long
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at java.lang.Runtime.exec(Runtime.java:593)
at org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:862)
at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:481)
at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:495)
at org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:522)
... 19 more
Caused by: java.io.IOException: java.io.IOException: error=7, Argument list too long
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 24 more
The argument list is quite big in fact contains all the jar files from WEB-INF/lib which is 231650 characters long!
Any suggestions how to fix it?

Answer 1

With a command that long you are likely running in to ARG_MAX at the shell.
This will report a good estimate of the available length,
expr getconf ARG_MAX - env|wc -c - env|wc -l * 4 - 2048
A nice article about command arg lists and length can be found here

Answer 2

Run ant -d. This will produce capacious amounts of output. However, it will also show your entire compile line which may help you understand why it is so long.
Are you using Jenkins/Hudson and that's where the error occurs?
Try the following:
Disable the build.
Log into your build server, AS YOUR JENKINS USER and find the workdir directory where Jenkins/Hudson is attempting the build.
You may have to change $PATH or set $JAVA_HOME to point to the JDK that Hudson/Jenkins is using.
Now, run ant -d  <target> just as Jenkins/Hudson would. Pipe this output through tee into a file. Now, take a look and see what Hudson/Jenkins is doing and why javac has too many arguments.

Answer 3

Use apply for your fileset in your build.xml, e.g.
<?xml version="1.0" encoding="UTF-8"?>
<project default="build">
<fileset id="myfiles" dir="${basedir}">
<include name="**/*.java"/>
<exclude name="**/Resources/**"/>
<modified>
<param name="cache.cachefile" value="${basedir}/cache.${project}.fileset.myfiles.properties"/>
</modified>
</fileset>
<target name="execute-some-command">
<apply executable="javac" dir="${basedir}" failonerror="true">
<fileset refid="myfiles"/>
</apply>
</target>
</project>
By default, the command will be executed once for every file.
If you need to use parallel to run the command only once, then use maxparallel to limit the amount of parallelism by passing at most this many sourcefiles at once (e.g. set to 1000 to pass a thousand files per run). For example:
<apply executable="javac" parallel="true" maxparallel="1000" dir="${basedir}">
<fileset refid="myfiles"/>
</apply>
To see how many files you've got in total, check the content of cache file (look for cache.cachefile in above example).