Bug 11609 - refresh of scrape pages doesn't work
Summary: refresh of scrape pages doesn't work
Alias: None
Product: Taglibs
Classification: Unclassified
Component: Scrape Taglib (show other bugs)
Version: unspecified
Hardware: PC All
: P3 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
Depends on:
Reported: 2002-08-10 22:15 UTC by Stefan Arnold
Modified: 2005-03-20 17:06 UTC (History)
0 users


Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Arnold 2002-08-10 22:15:40 UTC
I'm currently tried to use this taglibs, which covers all my needs. But I found 
out, that this tags doesn't refresh after the time which those scrapes will be 
So I had a deeper look into the source and found the problem in the PageData 
The method "scrapePage" connects at the first call to the server and fetches 
the data correct. But this doesn't work at the second time, when the cache time 
is elapsed. The reasen for this is because the variable scraping will then not 
be set to false.
You see my change at the end which works fine.
I hope this will help you

	public void scrapePage(String url, long time, PageContext pc)
		throws JspException {
		long currenttime = new Date().getTime(); // get the current time

		// check to see if a scrape is needed
		if (((currenttime - lastscrape) > time) || newflag || 
changeflag) {
			if (!scraping.booleanValue()) {
				// if a scrape is in progress wait until scrape 
is finished
				synchronized (scraping) {
					if ((page == null) || !page.isAlive()) {
						// create thread page if it 
doesn't exist check for a
						// proxy connection
						try {
							if (pport != -1 && 
pserver != null)
								page = new Page
(url, this, pc, pport, pserver, auth);
							//page = new Page(url, 
this, pc, pport, pserver, ssl, auth);
								page = new Page
(url, this, pc);
							//page = new Page(url, 
this, pc, ssl);
						} catch (MalformedURLException 
mue) {
().log("PageData.scrapePage(): " + mue.getMessage());
					if ((((currenttime - lastscrape) > 
time) || newflag || changeflag)
						&& page != null) {
						// set scraping flag 
						scraping = new Boolean(true);
		if (scraping.booleanValue() && (page != null) && !page.isAlive
()) {
			scraping = new Boolean(false); // done scraping reset 

		if ((newflag || changeflag) && (page != null)) {
			try {
				page.join(); // wait for scrape to finish
				changeflag = false; // reset changeflag
			} catch (InterruptedException ie) {
				// exception shouldn't happen if it does log it 
to the server
					"PageData.scrapePage(): Page thread 
interrupted " + ie.toString());
		// check to see if a MalformedPatternException was thrown if so 
		// JspException for JSP page builder
		if (exception) {
			exception = false; // reset exception flag
			throw new JspException(exceptiontext);