I have an Ant script called from Maven to perform some tasks. One such task is to read a text file linewise and use its content to perform text replacements on a huge set of files. The text file will be in the following format on each line:
name#URL#Title
Ant task for this is as follows:
<target name="update-urls" depends="load-ant-contrib">
<loadfile property="file" srcfile="${mapping.file}"/>
<foreach param="file.entry" list="${file}" delimiter="${line.separator}" target="update-template"></foreach>
</target>
<!--Get the name,url and title of each line-->
<target name="update-template">
<propertyregex property="name"
input="${file.entry}"
regexp="(.*)#(.*)#(.*)$"
select="\1"/>
<propertyregex property="url"
input="${file.entry}"
regexp="(.*)#(.*)#(.*)$"
select="\2"/>
<propertyregex property="title"
input="${file.entry}"
regexp="(.*)#(.*)#(.*)$"
select="\3"/>
<echo>${template.name}</echo>
<!--Use title and url to match and replace the URL with the URL in text file-->
<replaceregexp byline="true"
match="(<title>${title} *</title>.*)${url} *"
replace="\1../modified-path/${name}.zip"
flags="g"
encoding="utf-8"
>
<!--Following are the files on which replacement has to happen-->
<fileset dir="${huge.set.of.files}/items">
<include name="**/info.xml"/>
</fileset>
</replaceregexp>
</target>
Since this needs to work on huge set of files, the above solution is not very efficient. I have coded a mutithreaded solution for it in Java which takes up multiple regex expressions(name, value pairs) at once.I've integrated it into my maven project as a mojo.
I'm having trouble with two things:
Framing a data-type to contain a list of name, url, title triplets in Ant.
If I'm successful with step 1., how to return this data structure to POM from where the Ant task got called and decode this data structure to be able to access each element's name, url and title separately..
Can someone guide me on how to go about this..
In my project I have a json file I use for configuration that I have git set to ignore. When the repository is first cloned, the configuration file that is part of the project and that is copied to the output directory doesn't exist. I've gotten this to work using tasks in the 'BeforeBuild' target in the project that will copy the sample file to the actual config file if it doesn't exist.
<Target Name="BeforeBuild">
<ItemGroup>
<MySourceFiles Include="Configuration.sample.json" />
</ItemGroup>
<ItemGroup>
<MyDestinationFiles Include="Configuration.json" />
</ItemGroup>
<Message Importance="high" Condition="!Exists('#(MyDestinationFiles)')"
Text="Copying #(MySourceFiles) to #(MyDestinationFiles)" />
<Copy Condition="!Exists('#(MyDestinationFiles)')"
SourceFiles="#(MySourceFiles)"
DestinationFiles="#(MyDestinationFiles)" />
</Target>
So if I build the project, then delete the configuration file and do a build, nothing happens because no changes have been made that would change the outputs I think. Is there a way to change the project file so that a build will be flagged as necessary? It shouldn't come up very often and I can always do a 'Clean' or 'Rebuild' manually, but it's nagging at me since I'm just starting to learn MSBuild files.
From the documentation on a Target's Outputs attribute:
The files that form outputs into this target. Multiple files are
separated by semicolons. The timestamps of the files will be compared
with the timestamps of files in Inputs to determine whether the Target
is up to date
So if you add the paths to the outputfiles created by your Beforebuild target to it's Outputs attribute, at the start of every build msbuild will check if those files exist and if not it will start a build because now the project is considered to not be up-to-date anymore. In practice use:
<Target Name="BeforeBuild" Outputs="#(MyDestinationFiles)">
I have a set of customer-specific configuration files that are located in a single Release folder at the time of the build. The file names are something like:
CustomerA_InstanceConfigurationX.config
CustomerA_InstanceConfigurationY.config
CustomerB_InstanceConfigurationX.config
CustomerB_InstanceConfigurationY.config
... etc.
During the build, I want to copy the customer-specific configuration files into a customer-specific Binaries folder:
$(BuildDirectory)\Binaries\Installers\CustomerA\ProductName\
$(BuildDirectory)\Binaries\Installers\CustomerB\ProductName\
So CustomerA_InstanceConfigurationX.config and CustomerA_InstanceConfigurationY.config would go into go into $(BuildDirectory)\Binaries\Installers\CustomerA\ProductName\ and so on.
How can I set the SourceFiles and the DestinationFolder properties to make this happen?
I have the list of Customers as a meta of an Instance property and set the SourceFiles and DestinationFiles around it:
<ItemGroup>
<ConfigFilesToCopy Include="$(BuildDirectory)\stage\InstallerDev\%(Instance.Customer)\Setup\bin\Release\%(Instance.Customer)_*.*" />
<DestionationsForConfigFiles Include="$(BuildDirectory)\Binaries\Installers\%(Instance.Customer)\InstallerDev\" />
</ItemGroup>
<Copy SourceFiles="#(ConfigFilesToCopy)" DestinationFolder="%(DestionationsForConfigFiles.FullPath)" />
That just copies all the customer .config files into all the customer-specific Binaries folder though.
Using the message task, %(Instance.Customer) outputs:
CustomerA
CustomerB
CustomerC.
%(ConfigFilesToCopy.Identity) outputs:
"C:\Builds\AgentX\YYY\INSTALLER_DEV Build\stage\InstallerDev\CustomerA\Setup\bin\Release\CustomerA_InstanceX.config"
"C:\Builds\AgentX\YYY\INSTALLER_DEV Build\stage\InstallerDev\CustomerA\Setup\bin\Release\CustomerA_InstanceY.config"
"C:\Builds\AgentX\YYY\INSTALLER_DEV Build\stage\InstallerDev\CustomerB\Setup\bin\Release\CustomerB_InstanceX.config"
"C:\Builds\AgentX\YYY\INSTALLER_DEV Build\stage\InstallerDev\CustomerB\Setup\bin\Release\CustomerB_InstanceY.config"
Etc.
%(DestionationsForConfigFiles.Identity) outputs:
C:\Builds\Agent8\Five0\INSTALLER_DEV Build\Binaries\Installers\CustomerA\InstallerDev\
C:\Builds\Agent8\Five0\INSTALLER_DEV Build\Binaries\Installers\CustomerB\InstallerDev\
C:\Builds\Agent8\Five0\INSTALLER_DEV Build\Binaries\Installers\CustomerC\InstallerDev\
Etc.
If someone could offer some help on achieving this or had a alternative approach for it, that'd be great. (E.g., I could re-organize the customer-specific configuration files into a customer-specific folders or something.) Thanks a lot in advance!
[** Update Note **: For now, I hard-coded each customer name into a ConfigFilesToCopy list item as well as a DestinationsForConfigFiles item.
<ConfigFilesToCopy Include="$(BuildDirectory)\stage\InstallerDev\CustomerA\Setup\bin\Release\CustomerA_*.*">
<DestionationsForConfigFiles>$(BuildDirectory)\Binaries\Installers\CustomerA\InstallerDev\</DestionationsForConfigFiles>
</ConfigFilesToCopy>
<ConfigFilesToCopy Include="$(BuildDirectory)\stage\InstallerDev\CustomerB\Setup\bin\Release\CustomerB_*.*">
<DestionationsForConfigFiles>$(BuildDirectory)\Binaries\Installers\CustomerB\InstallerDev\</DestionationsForConfigFiles>
</ConfigFilesToCopy>
This works, but I am basically wondering if it's possible to do the same thing without explicitly using the customer name so that I don't have to maintain this list every time we add a new customer.
Make the destination metadata of the files (I changed the paths so it better fits in SO's code box, but the idea is the same):
<Target Name="Copy">
<ItemGroup>
<ConfigFilesToCopy Include="$(SomeDir)\%(Instance.Customer)\%(Instance.Customer)_*.*">
<DestionationsForConfigFiles>$(SomeDir)\Binaries\%(Instance.Customer)</DestionationsForConfigFiles>
</ConfigFilesToCopy>
</ItemGroup>
<Message Text="Source=%(ConfigFilesToCopy.Identity) Dest=%(ConfigFilesToCopy.DestionationsForConfigFiles)" />
</Target>
In Ant, we use patternset to include or exclude some set of file using a pattern such as
<unzip src="${tomcat_src}/tools-src.zip"
dest="${tools.home}">
<patternset>
<include name="**/*.java"/>
<exclude name="**/Test*.java"/>
</patternset>
</unzip>
Is Ant capable of taking this patternset from an external file say txt or xml?
Seeing around the Ant the wiki does not mention of such usuage, but i am thinking otherwise.
Consider using includesfile/excludesfile or includes/excludes attributes of patternset.
In case of includes/excludes, you can use values of properties stored in your normal property file.
I'm fairly new to ant, and I've seen uncle Bob's "extract until you drop" episode.
As a result I try to define ant-targets as small as possibly possible, so you can see exactly the essence of the target, and no more. For more details, you have to refer to sub-targets.
Whether that's good or bad style is a different debate (or a flame-war maybe).
Therefore, I was creating a build script that, in pseudo-code, would look like this:
build =
compile
instrument if coverage
The coverage task is split into subtargets, too:
coverage:
create-coverage-dirs
call-cobertura
EDIT- I want to express that coverage sub-targets should not be run.
But... I'm having a hard time expressing this 'cleanly' in ant-ese.
Assuming that I can use the depends attribute to indicate ... inter-target dependencies, I got to something like this:
<target name="build" depends="compile, coverage"/>
<target name="compile"> .... </target>
<target name="coverage" depends="
create-coverage-dirs,
taskdef-cobertura"
if="build.with.coverage">
<cobertura-instrument ...> ... </cobertura-instrument>
</target>
<target name="create-coverage-dirs">
...
</target>
<target name="taskdef-cobertura">
...
</target>
Whow this looked nice!
Only it seemed that, when executing, the coverage task was duefully omitted, but it's sub-tasks were still executed when build.with.coverage was false!
>ant -v compile
Build sequence for target(s) `build' is
[compile, create-coverage-dirs, taskdef- cobertura, coverage, build]
Complete build sequence is
[compile, create-coverage-dirs, taskdef-cobertura, coverage, build, ]
I can put an if attribute in every coverage sub-task, but that doesn't seem clean to me.
So here's the question:
Is my ant-ese a horrible dialect? Am I 'making ant into make'?
Should if be used this way, or is there an if-and-recurse kind-of attribute?
Repeat after me: Ant is not a programming language. In fact, write it down 100 times on the blackboard.
Ant is not a programming language, so don't think of it as such. It is a build dependency matrix.
It's difficult for programmers to wrap their heads around that idea. They want to tell Ant each step and when it should be done. They want loops, if statements. They'll resort to having a build.sh script to call various targets in Ant because you can't easily program Ant.
In Ant, you specify discrete tasks, and which tasks depend upon other tasks, and let Ant handle where and when things get executed.
What I am saying is that you don't normally split tasks into sub-tasks and then try calling <ant> or <subant> on them.
Have discrete tasks, but then let each task know what other tasks they depend upon. Also remember that there is no true order in Ant. When you list the depends= tasks, there is no guarantee which order they'll be executed in.
Standard Ant Style (which means the way I do it (aka The Right Way), and not the way my colleague does it (aka The Wrong Way)), normally states to define tasks at the top of the properties file and not in any target. Here's a basic outline on how I structure my build.xml:
<project name=...>
<!-- Build Properties File -->
<property name="build.properties.file"
value="${basedir}/build.properties"/>
<property file="${build.properties.file"/>
<!-- Base Java Properties -->
<property name="..." value="..."/>
<taskdef/>
<taskdef/>
<!-- Javac properties -->
<property name="javac..." value="..."/>
<task/>
<task/>
</project>
This creates an interesting hierarchy. If you have a file called build.properties, it will override the properties as defined in the build.xml script. For example, you have:
<property name="copy.verbose" value="false"/>
<copy todir="${target}"
verbose="${copy.verbose}">
<fileset dir="${source}"/>
</copy>
You can turn on the verbose copy by merely setting copy.verbose = true in your build.properties file. And, you can specify a different build properties file by merely specifying this on the command line:
$ ant -Dbuild.properties.file="my.build.properties"
(Yes, yes, I know there's a -propertycommand line parameter for ant)
I normally set the various values in the build.xml to the assumed defaults, but anyone can change them by creating a build.properties file. And, since all the base properties are at the beginning, they're easy to find.
Tasks are defined in this non-target space too. That way, I can easily find the definition since they're in the same place in each build.xml, and I know I can use a task without worrying whether the task defining target has been hit.
Now, to your question:
Define your tasks (and don't have a tar defining task, or you'll drive yourself crazy). Then, define the dependencies on each of those tasks. Developers can select the targets they want to hit. For example:
<project>
<description>
yadda, yadda, yadda
</description>
<taskdef name="cobertura"/>
<target name="compile"
description="Compile the code"/>
<!-- Do you have to compile code before you run Cobertura?-->
<target name="coverage"
description="Calculate test coverage"
depends="compile">
<mkdir dir="${coverage.dir}"/>
<cobertura-instrument/>
</target>
<project>
If you want to compile your code, but not run any tests, you execute ant with the compile target. If you want to run tests, you execute ant with a coverage target. There's no need for the depends= parameter.
Also notice the description= parameter and the <description> task. That's because if you do this:
$ ant -p
Ant will show what's in the <description> task, all targets with a description parameter, and that description. This way, developers know what targets to use for what tasks.
By the way, I also recommend doing things the right way (aka doing it the way I do it) and name your targets after the Maven lifecycle goals. Why? Because it was a good way to standardize on the names of targets. Developers know that clean will remove all built artifacts, and compile will run the <javac> task, and that test will run the junit tests. Thus, you should use the goals in the Cobertura plugin: cobertura.
Edit
my problem is: I regard 'coverage' as related to 'optimized' and 'debug', i.e. a build flavor. That's where my difficulty lies: for Java, coverage results in an an extra intermediate target in the compile step.
I'm looking at the Corburta page, and there's no real change in the <javac> task (which is part of the compile target.
Instead, you run Corburtura on the already built .class files, and then run your <junit> task. The big change is in your <junit> task which must now include references to your Corburtura jars, and to your instrumented classes.
I imagine you could have a corburturatarget or what ever you want to call it. This target runs the instrumented JUnit tests. This is the target you want developers to hit, and should contain a description that it runs instrumented tests.
Of course, you can't run the instrumented Junit tests without first instrumenting them. Thus, your corburtura target will depend upon another instrument.tests target. This target is internal. People who run your build.xml don't normally say "instrument tests" without running those tests. Thus, this target has no description.
Of course, the instrument.tests target depends upon having .class files to instrument, so it will have a dependency upon the compile target that runs the <javac> task:
<target name="instrument.classes"
depends="compile">
<coburtura-instrument/>
</target>
<target name="corburtura"
depends="instrument.classes"
description="Runs the JUnit tests instrumented with Corburtura">
<junit/>
</target>
The only problem is that you're specifying your <junit> target twice: Once when instrumented, and once for normal testing. This might be a minor issue. If you update how your JUnit tests run, you have to do it in two places.
If you want to solve this issue, you can use <macrodef> to define a JUnit test running Macro. I used what was on the Corbertura page to help with the outline. Completely non-tested and probably full of syntax errors:
<target name="instrument.tests"
depends="compile">
<corburtura-instrument/>
</target>
<target name="corburtura"
depends="instrument.tests"
description="Instrument and run the JUnit tests">
<run.junit.test fork.flag="true">
<systemproperty.addition>
<sysproperty key="net.sourceforge.corbertura.datafile"
file="${basedir}/cobertura.ser" />
</systemproperty.addition>
<pre.classpath>
<classpath location="${instrumented.dir}" />
</pre.classpath>
<post.classpath>
<classpath refid="cobertura_classpath" />
</post.classpath>
</run.junit.test>
</target>
<target name="test"
description="Runs the Junit tests without any instrumentation">
<run.junit.test/>
</target>
<macrodef name="run.junit.test">
<attribute name="fork.flag" default="false"/>
<element name="sysproperty.addition" optional="yes"/>
<element name="pre.classpath" optional="yes"/>
<element name="post.classpath" optional="yes"/>
<sequential>
<junit fork="#{fork.flag}" dir="${basedir}" failureProperty="test.failed">
<systemproperty.addtion/>
<pre.classpath/>
<classpath location="${classes.dir}" />
<post.classpath/>
<formatter type="xml" />
<test name="${testcase}" todir="${reports.xml.dir}" if="testcase" />
<batchtest todir="${reports.xml.dir}" unless="testcase">
<fileset dir="${src.dir}">
<include name="**/*Test.java" />
</fileset>
</batchtest>
</junit>
</sequential>
</macrodef>
I would not use a property at all in this case, but rely solely on depends (which seems more natural to me for this task):
<target name="build" depends="compile, coverage"/>
<target name="compile"> ...
<target name="coverage"
depends="compile, instrument,
create-coverage-dirs, taskdef-cobertura"> ...
The if attribute tests if the property exists, not if it is true or false. If you don't want to run the coverage target then don't define the property build.with.coverage.
As of Ant 1.8.0 you can use property expansion to resplver property as a boolean:
<target name="coverage" depends="
create-coverage-dirs,
taskdef-cobertura"
if="${build.with.coverage}">