This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 121962 - [60cat] Cyclic dependencies between projects cause incorrect error markings
Summary: [60cat] Cyclic dependencies between projects cause incorrect error markings
Status: NEW
Alias: None
Product: java
Classification: Unclassified
Component: Source (show other bugs)
Version: 6.x
Hardware: PC Linux
: P2 blocker (vote)
Assignee: Rastislav Komara
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-11-15 08:12 UTC by gugrim
Modified: 2009-02-03 10:57 UTC (History)
3 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Test projects illustrating cyclic dependency problem. (4.68 KB, application/x-gzip)
2007-11-17 12:11 UTC, gugrim
Details
Revised example showing that separation of interfaces makes the system work (6.13 KB, application/x-compressed)
2007-11-17 15:45 UTC, Jesse Glick
Details
Modified test case showing error markings in code. (4.92 KB, text/plain)
2007-11-18 17:14 UTC, gugrim
Details

Note You need to log in before you can comment on or make changes to this bug.
Description gugrim 2007-11-15 08:12:07 UTC
[ BUILD # : 200711081200 ]
[ JDK VERSION : 1.6.0_02 ]

If two projects reference each others output jar files, the metadata
repository update process will mark a lot of code as errors although
the code compiles fine.

Supporting cyclic dependencies is IMO important for two reasons:

1) NB 5 had no problem with this so if you have cyclic dependencies
that you can't easily get rid of you will not want to update to NB 6.

2) With very large systems consisting of many projects there is a
good chance that there will eventually be cyclic dependencies.

I'm currently working in a very large project that have almost 200
subprojects, each producing an EAR. The subprojects represent
services that call each other. These services has evolved over the
years and there are now quite a few cyclic dependencies. Nobody likes
them but without them the project structure would be even more
complex. To handle the compilation, when two interdependent projects
are promoted to test or production, our build system ensures that one
of them is always compiled against the other project's previous
version. 

This client will not be able to upgrade to NB6 before this issue is
resolved.
Comment 1 Jan Lahoda 2007-11-15 11:49:24 UTC
Well, 5.5 kept project artifacts on the error checker's classpath, which caused that the errors were not reported in the
case of circular dependencies (as long as the projects were built). Please note that this means that sometimes errors
were not reported correctly: imagine class A, which is already compiled inside the output jar, and delete this class -
the usages of it will not be marked as errors, but clean build is not going to work. I am also not sure whether
refactoring worked (or can work) with "no problem" in presence of circular dependencies. There is a "secret" command
line switch in NB6.0 to get behavior similar to 5.5 - "-J-DCacheClassPath.keepJars=true", but it may have negative side
effects (like the one described above).

From what I can tell, the circular dependencies are not supported by the IDE, and the scope of changes necessary to
support circular dependencies (not only initial scan, but also refactoring, etc.) is unclear to me. Moreover, the
circular dependencies between source roots with different classpaths seems very obscure to me. One idea that might work
wrt initial scan would be to re-compile the source roots several (3?) times, simulating "bootstrap" process of the projects.

BTW: having projects A and B with circular dependencies, if you build A against an old B build, and then B, then the
result may not work at all, IMO.
Comment 2 Jesse Glick 2007-11-15 15:14:40 UTC
WONTFIX from my perspective. The IDE does not generally support cyclic dependencies between separate source roots. You
can include several source roots in parallel in a single project, in which case they must share a classpath and are
considered equivalent to a single source root.

Teasing apart cyclic dependencies into a directed acyclic graph will be generally beneficial and is not necessarily that
hard, especially if you use a tool such as JDepend or Lattix which is designed for this purpose. The amount of work is
probably proportional to the size of the minimum graph "cut" needed to break all cycles.
Comment 3 gugrim 2007-11-16 07:21:06 UTC
In the very large project I've referred to, each project (freeform) produces an EAR and a library jar containing
compiled remote interfaces. Projects, i e deployed J2EE applications, can call each other freely and thereby cause
cyclic dependencies, but only on interface level. A clean rebuild of the entire system is no problem since the interface
jars for all projects can be compiled and built in one phase and then all the projects can be fully built in any order.

When opening two such interdependent projects in NetBeans, each project will in its classpath have the other projects
remote interface jar. A clean rebuild of any or both projects is no problem.

This kind of setup is IMO not "wrong". It's not even unclean. If the entire system was significantly smaller we would
use the same logical architecture but with local beans within a single EAR in a single project. Only JBoss would have a
problem with that kind of project since it can't handle cyclic dependencies between session beans within a single EAR. 

Since we have 800 entity beans and almost 200 session beans we have chosen a more modular physical architecture and are
very happy with NetBeans 5 and I hope there is a way to make this work with NB6 without us having to change 200 projects.
Comment 4 Jesse Glick 2007-11-16 11:46:52 UTC
I think I see what you mean. In other words, there are no cycles if you ignore project containership, but there are
cycles when you group sources and outputs into projects. Such a setup should work, I think - DEFECT if it does not, but
could you attach a self-contained minimal test case (preferably using Java SE code only, for simplicity) we can play
with to be sure we are talking about the same thing? Otherwise we can go round and round discussing different things and
nothing will happen.
Comment 5 gugrim 2007-11-17 12:11:07 UTC
Created attachment 53157 [details]
Test projects illustrating cyclic dependency problem.
Comment 6 gugrim 2007-11-17 12:25:54 UTC
I've just attached a file containing two very simple projects, ServiceA and ServiceB. Both have an mock-up bean
implementation and an interface for it. The services call each other using the interfaces.

None of the projects can be compiled from scratch since they need client jars produced by the other project. However,
both projects have the targets jar-client that compile only the interfaces and create client jars. 

To build both projects from scratch you first invoke jar-client on one of them (or both if you prefer) then build the
other, then build the first.

To see the problem, ensure that both projects are opened in NetBeans and are successfully compiled. Close NetBeans,
delete the index and open NetBeans. After a while you will get a red bage on one of them.

In this simplified case it is very easy to get rid of it, In a real world project it is not so easy since you would have
java class in the badged project that refer to the badged class. They will also show error markings.

As I've explained earlier, this is not a contrived case, albeit very simplified. For us it is actually a showstopper
unless there is some easy way to work around it.

Hope this test case is helpful! I'm also taking the liberty to remove the INCOMPLETE keyword.
Comment 7 Jesse Glick 2007-11-17 15:44:22 UTC
After opening the built projects in a new IDE instance, I get an error mark just on ServiceBBean.java but this
disappears after switching tabs - the well-known out-of-date error badge problem, I guess. Find Usages on
ServiceA.methodInA shows nothing for some reason, though FU on ServiceB.methodInB correctly shows ServiceABean and Go to
Declaration works in all cases.

Whatever problems there are I think derive from the fact that you _do_ have a real cyclic dependency at the Java source
root level, ignoring project containership. If you properly separate interface from implementation, as in the example I
will attach, the build scripts are potentially simpler (no excludes="...", no special bootstrapping) and the IDE seems
to load and work with the projects just fine, built or unbuilt. Impl classes can refer to whichever interfaces they
like, but no other impl classes; interface classes could refer to other interface classes so long as there are no
cross-source-root cycles introduced in this way. Such a system also prevents accidental dependencies from being
introduced, as they are not offered in code completion and would be rejected by the Ant build. You could separate the
interfaces into their own projects or leave each in the same project as its implementation; it should not matter.

So I would still recommend WONTFIX. I don't think we have any plans to support cyclic dependencies between supposedly
independent source roots.
Comment 8 Jesse Glick 2007-11-17 15:45:14 UTC
Created attachment 53158 [details]
Revised example showing that separation of interfaces makes the system work
Comment 9 gugrim 2007-11-17 17:13:41 UTC
I know that there are ways to get around the problem by splitting the source roots in such a way as you suggest, but the
problem is that this is not the way my client's 200 projects are structured. They use EJB2 with XDoclet generated
interfaces. Each project has two source roots, one primary and one for XDoclet generated sources that are not under
version control. The service bean implementation is also an XDoclet generated facade for service modules. Neither source
root can therefore be built from a clean slate when there are cyclic dependencies between projects, wich is why the
output library contents are based on package names, not source roots. This works very well with Ant and with NetBeans 5.

Apparently NetBeans 6 can't handle this kind of setup and I suspect that if my client is forced to choose between
changing all the projects or staying with NetBeans 5, they will choose the latter.
Comment 10 Jesse Glick 2007-11-17 19:52:15 UTC
Well again I can't reproduce any blocker issue in NB 6 with the sample project pair. I build them from the command line
then start a new NB instance with both projects open.

ServiceBBean.java has the temporary error badge in the Projects tab and editor, which goes away upon making some edits
or switching tabs etc.; the status box in the corner of the editor window is nonetheless green, and there are no lines
marked as erroneous (unless you introduce an actual error!). After restarting the IDE the error badge is still gone.
(You can also use Tools > Options > Java Code > Tasklist > [ ] Enable Error Badges on Files if this is a persistent
problem.)

The main problem I see is that Find Usages only works in certain directions, and similarly Refactor > Rename etc. may
not find all occurrences reliably. I tried various tweaks to <classpath> and <built-to> without success.
Comment 11 gugrim 2007-11-18 17:14:40 UTC
Created attachment 53171 [details]
Modified test case showing error markings in code.
Comment 12 gugrim 2007-11-18 17:28:16 UTC
I've now modified the test a little to illustrate the red markings in the code, indicating that the code has errors.

Although both projects compile fine, provided you build the client jars first, the assignment on line 10 in OtherBean in
ServiceB shows an "incompatible type" error. Imagine opening a project with 50 files showing red markings all over,
which is what happened to me in a real project.

There are no cyclic dependencies on Java source level, only between source roots and thereby between projects. I still
think that NetBeans should be able to handle this, if for no other reason than because it used to.

I haven't yet tried with a cyclic setup like this using standard NetBeans EJB module and application projects but since
these projects do not place remote interfaces in different source roots from the bean implementation I suspect that
there will be trouble.
Comment 13 Jesse Glick 2007-11-18 21:59:57 UTC
[BTW please take care to select the correct content type when attaching files - you used text/plain for a .tar.gz file!]

I built the new 6.0.1 projects and opened in a fresh userdir on 071114 using JDK 6 on Ubuntu 7.10. As before, there was
a transient error badge on the editor tab for sa.ejb.OtherBean, which disappeared when I switched tabs away and back.
After that, there were no error marks to be seen, whether on editor badges, the Projects tab, or as error annotations in
the editor.
Comment 14 gugrim 2007-11-19 07:10:22 UTC
Sorry about the mime type. When I start NetBeans with these two projects open, it is ServiceB that is gets badged, not
ServiceA, and sb.ejb.OtherBean gets error markings.
Comment 15 gugrim 2007-11-19 07:13:37 UTC
Just a thought. I built both projects, closed NetBeans, deleted the index and started NetBeans again. Perhaps that is
why we see different prohjects getting badged.
Comment 16 Jesse Glick 2007-11-19 16:43:36 UTC
I think it's arbitrary which project gets the transient error badge. In my case I build the projects, then start NB and
open the projects in one command using e.g.

ant -f Cyclic601/ServiceA/build.xml jar-client
ant -f Cyclic601/ServiceB/build.xml
ant -f Cyclic601/ServiceA/build.xml
.../bin/netbeans --userdir .../new-directory Cyclic601/Service{A,B}
Comment 17 gugrim 2007-11-20 07:09:26 UTC
OK, for me it is consistently ServiceB. I'll change ServiceA so that it is identical to ServiceB. That should show the
error marking, which is what this issue is all about, not incorrect badging even if the cause is the same.
Comment 18 Jan Lahoda 2007-11-21 17:16:26 UTC
Sorry for late answer. IMO, all the visual artifacts (error badges on files/projects, error underlining in the editor,
etc.) only manifest the fact that the Java caches are not correct. The reason is that with two interdependent source
roots A and B (parsed in this order), classes from B cannot be resolved while parsing A (please note that each source
root is currently parsed only once). The actual outcomes may differ depending on where exactly is the cycle broken by
the sorting algorithm.

The usecase described here is not, IMO, a true cyclic dependency - there are two "virtual" source roots under each
"physical" source root, and the dependencies between these "virtual" source roots are acyclic. I do not think it is
realistic to (truly) support real cyclic dependencies between source roots (and I would not say that NB5.5 supported it
- it did not show the errors in the editor because of implementation of the error checker in NB5.5, but no other feature
understood the dependency, AFAICT).

What could be theoretically supported are these "virtual" source roots - through excludes, etc. But this would require
changes on many places, IMO, not only the Java support.
Comment 19 gugrim 2007-11-21 17:36:49 UTC
Yes, what we are talking about is not a true cyclic dependency down to the class level. That would be very difficult to
compile from scratch and I don't expect NetBeans ever to support it. Can't see any good reason not to get rid of it if
it ocurred.

As soon as I have some time I'll try it out with two interdependent NetBeans standard EJB module projects since I
suspect that they will also cause problems even if they are interdependent only on interface level.
Comment 20 Jesse Glick 2007-11-21 20:37:11 UTC
Right, virtual roots - meaning a physical root paired with an include/exclude pattern as introduced in 6.0 - could in
principle be treated independently by the Java infrastructure. But it was decided not to do so for the foreseeable
future due to the complexity and risk of such a system. Therefore, as in all previous releases, all **/*.java beneath a
root folder are considered a unit. If certain features happened to work despite cycles at the physical root level in NB
5.x then this was probably due to accident rather than design.
Comment 21 Rastislav Komara 2009-02-03 10:57:14 UTC
Overtake.