This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 126570

Summary: Support API for handling process and file streams
Product: serverplugins Reporter: Petr Hejl <phejl>
Component: CodeAssignee: apireviews <apireviews>
Status: RESOLVED DUPLICATE    
Severity: blocker CC: mkleint, mkrauskopf, phejl, tor
Priority: P2 Keywords: API, API_REVIEW_FAST
Version: 6.x   
Hardware: All   
OS: All   
Issue Type: ENHANCEMENT Exception Reporter:
Attachments: support api patch

Description Petr Hejl 2008-02-03 18:19:32 UTC
Every server plugin needs to handle process streams or logs somehow (usually printing it somewhere). This involves
additional threads, synchronization, starting, stopping etc. Unfortunately server plugin usually handles this in error
prone way. Many bugs has been already fixed (usually synchronization issues).

This support api should solve such needs.
Comment 1 Petr Hejl 2008-02-06 15:55:46 UTC
Created attachment 56173 [details]
support api patch
Comment 2 Petr Hejl 2008-02-06 15:57:32 UTC
Entry point is ReaderManager.

PrintingLineProcessor, ProxyLineProcessor, WaitingLineProcessor are just useful implementations of LineProcessor.
Comment 3 Jaroslav Tulach 2008-02-08 23:58:59 UTC
Y01 Consider public final class FileInput { ... public interface Provider { FileInput getFileInput(); } }
Y02 Hide those useful implementations of line processors and replace them with factory methods
Y03 Write tests. What is your code coverage?
Y04 I guess you can remove ReaderManager.Pair if you have public static ReaderManager empty(); public ReaderManager  
enhance(LineReader lineReader, LineProcessor lineProcessor). One would then use 
empty().enhance(...).enhance(...).enhance(...) to register few pairs into a reader manager.
Comment 4 Jesse Glick 2008-02-09 00:16:26 UTC
I seem to recall that Tor was interested in defining some API for managing external processes and capturing their output
in a form friendly to the Output Window. Not sure if this API would be related at all.
Comment 5 Torbjorn Norbye 2008-02-09 00:32:38 UTC
Yes - we needed something like this for Ruby - the code now lives in ruby/platform/ and somebody also forked it and made
a copy in languages/execution/ which I think is used for the Scala console etc.

One thing I remember added which I didn't see from a very simple read-through of the attachment is the ability to
rewrite the output.  Specifically, one kind of line processor is able to recognize a particular line, and when it does,
it can replace (or augment) the output by inserting both plain text lines and action-lines.

For example, when we recognize an error message from a rails server that indicates there was a port conflict, we rewrite
the message in a clearer user message and add an action link the user can press to restart the server on a different
port.  Or, when you run a rake task and the rake task aborts, we recognize the error message which says to re-run the
task with a certain flag to get verbose output - we add a link that performs this task - and so on. (This is the
OutputRecognizer.ActionText class).

We also have code for not just processing output, but actually launching processes too - building up a descriptor of the
process to be executed (arguments, environment, etc.), adding a completion listener, and so on. This integrated with the
output window such that you can get Stop and Restart buttons (which just reuses the execution descriptor to relaunch the
process and so on).  I think it would be very useful to include this in some sort of execution library as well since
there's quite a bit of code. To do the Stop/Restart bit I went and studied and copied code from the ant integration
module for example.

Martin has been working on this code most recently so he may have more to say about it.
Comment 6 Petr Hejl 2008-02-09 09:19:16 UTC
Y01: Ok, I'll look at it.
Y02: I was thinking about it. It is doable just for PrintingLineProcessor, ProxyLineProcessor, not for
WaitingLineProcessor as it defines await() method.
Y03: There are 22 test covering the output code already. I just didn't want to "pollute" api diff with it. BTW the whole
code including tests is in repository in server module (non public packages *.output).
Y04: Nice idea. Will do.

Comment 7 Petr Hejl 2008-02-09 09:29:36 UTC
> One thing I remember added which I didn't see from a very simple read-through of the attachment is the ability to
> rewrite the output.  Specifically, one kind of line processor is able to recognize a particular line, and when it does,
> it can replace (or augment) the output by inserting both plain text lines and action-lines.

Well you can write a LineProcessor you need. I didn't want to place any TechnologyXYProcessor into such generic module
(thats the reason why there is no stack trace linking for example - it is not common to all different servers).

> We also have code for not just processing output, but actually launching processes too - building up a descriptor of the
> process to be executed (arguments, environment, etc.), adding a completion listener, and so on. This integrated with the
> output window such that you can get Stop and Restart buttons (which just reuses the execution descriptor to relaunch the
> process and so on).  I think it would be very useful to include this in some sort of execution library as well since
> there's quite a bit of code. To do the Stop/Restart bit I went and studied and copied code from the ant integration
> module for example.

I considered this to be next step (built upon this api) - if it would be doable in generic way (without some technology
specific code). This should be implemented as a possibility - to not to mandate the implementor to use process creation
when he just want to process a stream (file, network, process...).


I'll look at your code next week.

Thanks,
P.
Comment 8 Milos Kleint 2008-02-09 12:27:00 UTC
I got the same usecases as Tor for Maven support. I'm either parsing external process output (in case the command line
maven is used for execution)  or feed the output processors through events (in case of embedded maven execution - in
same VM).

Additionally I have support for defining some processors only for certain sections of the output (mainly performance
feature)

mk1: I don't understand the workflow in the LineReader:
+     * Single line can be sent to the {@link LineProcessor} multiple times.
+     * Such mutiple processing is always separated by call of the method
+     * {@link LineProcessor#reset()} (however it is not guaranteed that
+     * {@link LineProcessor#reset()} is called only in this case).

Why is a single line being sent/processsed multiple times?

mk2: just for consideration here's how I have the api defined in maven support.
OutputProcessor.java which is a rough equivalent of LineProcessor
    /**
     * for each line of output, the registered processor will get a chance to
     * add highlighting etc.
     */
    void processLine(String line, OutputVisitor visitor);

where OutputVisitor has methods that get set by the processors. At the end, the infrastructure prints the line (or a
replacement or nothing) with a hyperlink possibly etc.
OutputVisitor is a final class passed to the Processors so it's reasonably extensible.
for full overview of the API, see:
http://fisheye.codehaus.org/browse/mevenide/trunk/mevenide2/netbeans/nb-project/src/main/java/org/codehaus/mevenide/netbeans/api/output

Comment 9 Petr Hejl 2008-02-09 15:30:13 UTC
mk1: seems to be bug in javadoc

Reset is intended to notify some processor change (like log switching) - original motivation was to emphasise that in
rare cases the same line can appear (for example processor waiting for "started" or something like that should consider
that.

I agree that javadoc is confusing and I will fix it.
Comment 10 Petr Hejl 2008-02-09 15:41:18 UTC
Seems like a missing api for many modules - maybe it deserves wider discussion and different module (other than server).
However this could be impossible to do for 6.1.

Or we can place temporarily solution somewhere and move it to seperate module for 7.0 (I'm not sure about all available
possibilities of the api publishing/placement).
Comment 11 Petr Hejl 2008-02-11 18:37:11 UTC
The api is imo bit more generic version of OutputForwarder, OutputProcessor and OutputRecognizer of the Ruby execution
support. Will investigate further.
Comment 12 Petr Hejl 2008-04-08 08:56:39 UTC
*** Issue 114089 has been marked as a duplicate of this issue. ***
Comment 13 Petr Hejl 2008-06-20 13:09:58 UTC
Will be part of extexecution module - issue 136929. 

*** This issue has been marked as a duplicate of 136929 ***