Bug 48754

Summary: Features to make it easy to create hierarchical / maintainable builds
Product: Ant Reporter: Jon Cox <jcox-dated-1266352898.9c8d3f>
Component: CoreAssignee: Ant Notifications List <notifications>
Status: NEW ---    
Severity: enhancement    
Priority: P2    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: All   

Description Jon Cox 2010-02-16 22:18:26 UTC
Suppose you have a master build file and a bunch of subdirectories
that represent sub-projects, each with their own build.xml file.
Further, suppose you wish to avoid all use of recursive tags such as 
<subant>, because recursive builds inherently break the dependency DAG,
and therefore require hand-maintenance of dependency relationships
(c.f.: "Recursive Make Considered Harmful"). Instead, it would be nice
if you could rely entirely upon the <import> tag, and have a predictable
static dependencies within a single DAG, and be confident that the
meaning of the names in the depends="..." attribute can be understood
locally.  In other words, you don't want the addition of seemingly 
unrelated sub-projects to change the dependency logic of "innocent bystander"
sub-projects.  Currently, Ant has some major problems with that,
but fortunately it's rather easy to correct.

Aside:  I was able to create a workaround without modifying Ant's source,
but it imposes some constraints upon each build.xml that should really be unnecessary.  

Ok, consider the following:

In the file system, we might have something like this:
 
 <basedir>/build.xml   (imports project/xxx/build.xml and project/yyy/build.xml)
 <basedir>/project/xxx/build.xml           (imports ../../project/yyy.build.xml)
 <basedir>/project/yyy/build.xml       


Within project/yyy/build.xml let's say we see the following snippet:

  <target name="moo">               ... </target>
  <target name="cow" depends="moo"> ... </target>

It's tempting to think that cow in project yyy depends on moo in project yyy,
but if you're running from the top-level build.xml file, 'moo'
could easily be hijacked if *either* the top-level build file happened
to define a target named 'moo' or  project/xxx/build.xml happened to define
a target named 'moo'.  Why?  Because names are not resolved according to 
their lexical scope in Ant, but  rather in terms of the runtime order
in which they were encountered.  If the master build.xml file imported
project/xxx/build.xml before project/yyy/build.xml then within 
project/yyy/build.xml 'moo'  really means xxx.moo,  not yyy.moo.

Now consider the chaos of a master build with N sub-projects where N 
starts to get large, and sub-projects have interdependencies....  This 
means the only safe way you can currently refer to the target 'moo' 
in project yyy is by giving the full/ugly name  (i.e:  depends="yyy.moo").
That's really unfortunate.  It gets worse though:   because Ant does not
create top-level  <projectName>.<targetName> aliases for targets 
that are in the build.xml file given on the command line,  you've now
lost the ability to invoke Ant conveniently from the project/yyy directory.
Ant will say "Huh? what's the yyy.moo target?  Never heard of it!"

The seriousness of this problem becomes apparent as soon as 
each sub-project wants to claim natural sounding target names
like "compile".


The solution to this problem is rather simple:

(1) 
Immediately prior to the topological dependency sort of targets
re-write all target names mentioned in depends="..." 
by converting them into absolute form ( <projectName>.<targetName>).
Thus our original example is transformed  **as if** we'd written:

  <target name="moo">               ... </target>
  <target name="cow" depends="yyy.moo"> ... </target>


(2) Ensure that Ant creates top-level <projectName>.<targetName> aliases
for all targets, not just targets discovered via <import>.


Together, (1) and (2) allow you to have natural target names in each
build.xml file, and run Ant from either the master build file 
(in which case you could say  'ant yyy.compile')  or from the 
project/yyy/  directory (where you could just say 'ant compile').

Incidentally, I've made an number of other enhancements 
that allow you to avoid mentioning the name of the current
target (following the 'do not repeat yourself' principle).
I've got a rather nice setup now, and would be happy to share
it and/or discuss some of this stuff with folks who are already 
contributors to this project.  

Feel free to drop me an email.


     Cheers,
     -Jon


Aside:
When are people typically on ##ant -- nobody seems to be logged on
whenever I check.
Comment 1 Stefan Bodewig 2010-02-17 13:34:33 UTC
What you describe sounds a lot like <include> of Ant 1.8.0, please take a look at it, it might be all you need.

WRT #ant - I didn't even know it existed.  It's likely that there are not too many IRC users around Ant.
Comment 2 Jon Cox 2010-02-26 18:16:19 UTC
(In reply to comment #1)
> What you describe sounds a lot like <include> of Ant 1.8.0, please take a look
> at it, it might be all you need.
> 
> WRT #ant - I didn't even know it existed.  It's likely that there are not too
> many IRC users around Ant.


The new <include> tag looks promising -- I'll check it out.

Here are some other things I did that would be nice to have in "standard ant":

I created some magical properties that are static with respect to 
the context in which they're evaluated (but implemented dynamically).
The net effect is that I've faked a static "scoped" definition for 
the following magical properties.  

Props related to targets/projects/tasks:
  *  ${this.target}              the current target name (e.g.: 'javac')
  *  ${this.target.description}  target description text (possibly undefined) 
  *  ${this.project}             the current project name (e.g.: 'myproj')
  *  ${this.task}                the current task name (e.g.: 'javac-project')

Props related to current build file dir:
  *  ${this.dir}           full dir parent pathname containing this build file
  *  ${this.dir.name}      leaf name of parent dir containing this build file

Props related to current build file name:
  *  ${this.file}          full pathname of this build file
  *  ${this.file.name}     leaf name of this build file (typically, "build.xml")
  *  ${this.file.basename} basename of this build file (typically, "build").


This ends up being enormously useful because it means you can
cut/paste snippets of ant without modification.  Critically, 
because the interpolation of ${this.target}  and ${this.project}
is done immediately prior to the topological sort of targets,
all dependencies remain entirely *static*  the other this.XXX
variables are updated via a BuildListener  

Consider the following snippet from a sub-project in my build.
I can have the exact same blob of XML in project "moo" and "cow":

    <target name="javac-test"
            depends="javac"
            description="Compile Java 'test' classes in this subproject">
        <javac-project
            name="${this.project}" 
            src="test"/>
    </target>


Now my <javac-project> macro is fed  name="moo"  
when the blob of XML above is in the moo project,
and name="cow" when it's in the cow project!

Therefore, because I never end up repeating names
of things that can be known statically prior to 
the topological sort, my XML ends up being a lot
more reusable / cut-and-paste-able.  Extremely nice!


Let me know what you think.

            Cheers,
            -Jon


Side note:  it would be highly desirable if Ant gave the build listener 
a special event signifying that the topological dependency sort is
is about to happen... I had to kludge around that one.  Another nasty
bit was that the "dependencies" field in Target is private, yet there's
no way to remove dependencies (just add them).  That means that if you
want to do a  custom fixup of dependencies, you've got to resort to
introspection and the ugly field.setAccessible(true) trick.  It works,
but it's icky.