Bug 59033 - Parallel Download : Rework Parser classes hierarchy to allow pluging parsers for different mime types
Summary: Parallel Download : Rework Parser classes hierarchy to allow pluging parsers ...
Status: RESOLVED FIXED
Alias: None
Product: JMeter
Classification: Unclassified
Component: HTTP (show other bugs)
Version: 2.13
Hardware: All All
: P2 enhancement (vote)
Target Milestone: ---
Assignee: JMeter issues mailing list
URL:
Keywords: PatchAvailable
Depends on:
Blocks: 59140
  Show dependency tree
 
Reported: 2016-02-19 14:12 UTC by Philippe Mouawad
Modified: 2016-03-18 11:23 UTC (History)
1 user (show)



Attachments
Part 1 of patch (28.14 KB, patch)
2016-02-19 23:31 UTC, Philippe Mouawad
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Mouawad 2016-02-19 14:12:17 UTC
In order to make parallel download more realistic we should add parsing of CSS files.

For example, if you try parallel download with Tomcat 8 Homepage:
- JMeter will download:
*http://localhost:8080/tomcat.png
*http://localhost:8080/tomcat.css

- Browser will add to those 5 coming from parsing of tomcat.css:
*http://localhost:8080/bg-nav.png
*http://localhost:8080/asf-logo.png
*http://localhost:8080/bg-upper.png
*http://localhost:8080/bg-button.png
*http://localhost:8080/bg-middle.png
Comment 1 Philippe Mouawad 2016-02-19 16:01:12 UTC
This should not be that complex to implement:

We need to handle @Import and background(-image), parsing url()

In terms of architecture, it is in fact nearly here as:
1/ we can use in fact HTMLParser implementation, maybe by the way rename it to ExternalResourcesParser.
2/ HTTPSamplerBase#parsersForType can be filled in with "text/css", new parser implementation
Comment 2 Philippe Mouawad 2016-02-19 23:31:56 UTC
Created attachment 33573 [details]
Part 1 of patch

Hello,
Find attached a patch that:
- Introduces new interfaces to make extraction of links more generic, not strictly HTML 
- makes some cleanup in code

Pay attention to the fact that:
- static initializer in HttpSamplerBase does not defaults anymore for html, that's why in jmeter.properties, htmlParser.className and htmlParser.types are uncommented
- Junit test has slightly been changed


This is a breaking change but as we are releasing a 3.0, it should be OK.
Regards
Philippe
Comment 3 Philippe Mouawad 2016-02-24 12:28:37 UTC
Hi,
Can I commit this patch ?
Thanks
Comment 4 Philippe Mouawad 2016-03-07 21:23:13 UTC
Author: pmouawad
Date: Mon Mar  7 21:22:17 2016
New Revision: 1733983

URL: http://svn.apache.org/viewvc?rev=1733983&view=rev
Log:
Bug 59033 - Parallel Download : Rework Parser classes hierarchy to allow pluging parsers for different mime types
Bugzilla Id: 59033

Added:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/BaseParser.java   (with props)
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/LinkExtractorParseException.java   (with props)
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/LinkExtractorParser.java   (with props)
Modified:
    jmeter/trunk/bin/jmeter.properties
    jmeter/trunk/bin/testfiles/jmeter-batch.properties
    jmeter/trunk/bin/testfiles/jmetertest.properties
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/HTMLParseError.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/HTMLParseException.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/HTMLParser.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/JTidyHTMLParser.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/JsoupBasedHtmlParser.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/LagartoBasedHtmlParser.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/RegexpHTMLParser.java
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/sampler/HTTPSamplerBase.java
    jmeter/trunk/test/src/org/apache/jmeter/protocol/http/parser/TestHTMLParser.java
    jmeter/trunk/xdocs/changes.xml
Comment 5 Felix Schumacher 2016-03-18 11:19:13 UTC
Date: Fri Mar 18 10:34:47 2016
New Revision: 1735568

URL: http://svn.apache.org/viewvc?rev=1735568&view=rev
Log:
Followup to r1733983: Rename private constants to uppercase letter words.

Bugzilla Id: 59033

Modified:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/BaseParser.java
Comment 6 Felix Schumacher 2016-03-18 11:23:29 UTC
Date: Fri Mar 18 11:17:05 2016
New Revision: 1735574

URL: http://svn.apache.org/viewvc?rev=1735574&view=rev
Log:
Followup to r1733983

Ensure, that only one instance of a LinkExtractorParser gets cached, even if two
threads are creating one at the same time.

Bugzilla Id: 59033

Modified:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/BaseParser.java

Date: Fri Mar 18 11:17:05 2016
New Revision: 1735574

URL: http://svn.apache.org/viewvc?rev=1735574&view=rev
Log:
Followup to r1733983

Ensure, that only one instance of a LinkExtractorParser gets cached, even if two
threads are creating one at the same time.

Bugzilla Id: 59033

Modified:
    jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/parser/BaseParser.java