This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 64433 - Improve invalid encoding handling
Summary: Improve invalid encoding handling
Status: VERIFIED FIXED
Alias: None
Product: serverplugins
Classification: Unclassified
Component: Code (show other bugs)
Version: 5.x
Hardware: All All
: P2 blocker (vote)
Assignee: Petr Pisl
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-09-15 14:15 UTC by Martin Schovanek
Modified: 2006-05-16 12:37 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Stack trace (3.16 KB, text/plain)
2005-09-22 14:24 UTC, Pavel Fiala
Details
Stack trace when sniffing mime type on an XML file w/ invalid encoding. (5.61 KB, text/plain)
2005-10-15 19:24 UTC, _ pcw
Details
NPE in XmlDataObject$InfoParser.waitFinished (20.10 KB, text/plain)
2005-11-09 07:50 UTC, _ pcw
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Schovanek 2005-09-15 14:15:24 UTC
[#200509141800, jdk1.4.2]

I was trying following scenaraio on various XML based web documets. The scenario
tipically leads to several exceptions and erasing tested document. 

steps to reproduce:
-------------------
1) open a document in XML (text based) view
2) change encoding in the XML header to a invalid,
e.g. change:
<?xml version="1.0" encoding="UTF-8"?>
to:
<?xml version="1.0" encoding="UTF-999"?>
3) save the document
ERROR: see results for particular document types.

RESULTS:

- web.xml, ejb-jar.xml, sun-ejb-jar.xml, sun-app.xml
 throws a IO exceptions and asks for reloading of external chages and after it
 erase whole document, no undo is possible

- sun-web.xml
 throws a IOException but the docume is not erased

- faces-config.xml
 throws: MissingResourceException: ... key TEXT_SAVE_AS_UTF

- .xml, .jspx, .tld, struts-config.xml
 work fine, they show warning message about wrong encoding; comment out the
 invalid XML header, insert valid XML header with UTF-8 encoding and save the
 document

I would recommend to all XML based document types handle invalid encoding like
the .xml does.

If I forget a document type please check it and attach result to this issue.
Comment 2 Pavel Fiala 2005-09-22 14:24:09 UTC
Created attachment 25069 [details]
Stack trace
Comment 3 Pavel Fiala 2005-09-22 14:28:59 UTC
I think it should be fixed in 
org.netbeans.modules.xml.text.syntax.UniKit.write(OutputStream, Document, int, int)
Comment 4 Marek Fukala 2005-09-29 12:25:36 UTC
There isn't any problem with the UniKit.write(...). It is correct that the
method throws UnsupportedEncodingException. The calling code has to catch it and
handle properly (like save cookie of XMLDataObject does as Martin mentioned).
For more info how to handle it see  

org.netbeans.modules.xml.core.text.TextEditorSupport.saveDocument() from
xml/text-edit module.
Comment 7 Pavel Fiala 2005-10-12 15:32:31 UTC
The fix involves only ejb-jar.xml and web.xml
Comment 8 _ pcw 2005-10-15 19:20:41 UTC
This comment applies to the SJSAS configuration files (sun-web.xml, etc.) though
it also seems to apply to any XML file (I reproduced it w/ wscompile config
files as well.)

Suppose you have a web project w/ sun-web.xml that has invalid encoding and you
open the project.  "java.io.UnsupportedEncodingException: UTF-999" (full trace
attached below) is thrown when the mimetype is being sniffed.

It seems to me that this should be handled by the mime resolver code.

Comment 9 _ pcw 2005-10-15 19:24:04 UTC
Created attachment 25995 [details]
Stack trace when sniffing mime type on an XML file w/ invalid encoding.
Comment 10 Petr Jiricka 2005-10-25 14:51:51 UTC
Yes, this looks like something deep in the openide/core code. Reassigning to
openide/filesystems.
Comment 11 rmatous 2005-10-26 17:23:59 UTC
While resolving mimetype org.netbeans.core.filesystems.DefaultParser.parse 
behaves exactly the same way as if there was whatever other parsing problem -
which means: catches this UnsupportedEncodingException, calls EM.notify with
severity INFORMATIONAL and return null. Finally text/xml is returned anyway
because in FileUtil is still this mimetype hardcoded for extension .xml. 

Is there anything wrong on this behaviour ? If so, please reassign back to me. 

Reassigned back to serverplugins.
Comment 12 Petr Jiricka 2005-11-08 15:06:18 UTC
So what is the remaining issue? I believe the only remaining area is
sun-web.xml, reassigning to Peter.
Comment 13 Petr Pisl 2005-11-08 18:00:56 UTC
I did small check with this results:

web.xml, ejb-jar.xml - the warning message is displayed "The declared encoding
XXX is not uspported, Changed it to UTF-8?" If you user push button NO, new
message is displayed: "The declared encoding XXXX is not supported. The data
will be treated as UTF-8. After this the file is saved in UTF-8.

faces-config.xml - still Missing resource exception

sun-app.xml, sun-ejb-jar.xml - still UnsupportedEncodingException and in the
same time the question File xxxx was modified externally. Reload it?

jsp document - There is message: "The XXXX character set that is used in
yyyy.jspx is not a valid character set for JSP pages. Do you want to save the
file using the UTF-8 character set?". The file is saved in utf-8 encoding and
nothing is changed in the file. When you open this file, then the ide ask user
"The XXXX character set that is used in yyyy.jspx is not a valid character set
for JSP pages. Do you want to load the file using the UTF-8 character set?"

Struts config.xml, *.tld, *.xml files which are handled only with the xml data
loader - display question "The declared encoding XXX is not supported, Changed
it to UTF-8?" If choose no, then the file is not saved, if choose yes, then file
is saved in utf-8,  the old prolog is commented out (<!-- was: <?xml
version="1.0" encoding="XXXXX"?> -->) and the new prolog with utf-8 encoding is
inserted into the file.

At first we need to solve the exceptions during saving faces-config.xml and
sun-*.xml. I'm going to fix the issue for faces-config.xml, which I fired as
separate issue #68300.

Peter, should you fix the exceptions during saving sun-*.xml files?

In the future, we should try to have same behavior, as Martin wrote, for all
.xml files in these cases. But I am not sure, that we are able to do it in the
NB 5.0 timeframe.
Comment 14 _ pcw 2005-11-08 21:12:18 UTC
I am working on the exceptions when saving sun-xxx files from XML text right
now, will have a fix soon.  I somehow missed seeing these when I had this bug
earlier or I would have already fixed them :(
Comment 15 _ pcw 2005-11-09 07:30:36 UTC
I checked a fix that resolves the problem when saving a file w/ invalid encoding
that was open in the XML text editor for the sun-xxx files.

http://serverplugins.netbeans.org/source/browse/serverplugins/sun/sunddui/src/org/netbeans/modules/j2ee/sun/share/config/ConfigDataObject.java?r1=1.10&r2=1.11
Comment 16 _ pcw 2005-11-09 07:49:46 UTC
Here's another incarnation of this bug:

Create a web app w/ sjsas as the target server.  Open sun-web.xml as text to
confirm UTF8 encoding (and existence of this file), then close it.

Now in an external editor, open this sun-web.xml and change the encoding to
UTF-999 (anything illegal), then save.

Go back to the ide and select the sun-web.xml node with the mouse.  At this
point, I get an invocation target exception with the following attached massive
stack trace (includes some of the INFORMATIONAL exceptions commented on above,
but the problem seems to be a NPE in XmlDataObject.InfoParser.waitFinished().)

I see no evidence that the SJSAS plugin is involved here, or can catch any of
these.  I can dig into XmlDataObject to see if indeed this is our fault, or
perhaps someone who knows this area better can either tell me it's not, or point
me in the right direction.
Comment 17 _ pcw 2005-11-09 07:50:46 UTC
Created attachment 26730 [details]
NPE in XmlDataObject$InfoParser.waitFinished
Comment 18 _ ludo 2005-11-21 17:14:27 UTC
So what's next?
Comment 19 Petr Jiricka 2005-11-21 17:33:46 UTC
Petr Pisl promised to look at this.
Comment 20 Petr Pisl 2005-12-05 12:37:45 UTC
The problem with NPE which was mentioned by PCW, is not in the xml module but in
the open ide/loaders, where they try to parse xml files. It happens for every
xml files and only if user modified the file externally. So I close this issue
and fire new one against open ide - loaders #69876
Comment 21 Martin Schovanek 2006-05-16 12:37:57 UTC
Verified in #200605160200 build.