Bug 53983

Summary:	UnsupportedEncodingException: ansi
Product:	POI	Reporter:	emergency.shower
Component:	HSMF	Assignee:	POI Developers List <dev>
Status:	RESOLVED FIXED
Severity:	normal
Priority:	P2
Version:	3.8-FINAL
Target Milestone:	---
Hardware:	PC
OS:	All

Description emergency.shower 2012-10-09 08:28:27 UTC

Please see https://issues.apache.org/jira/browse/TIKA-1003.

Tika cannot decode the "ansi" encoding.

The mail contains the headers

 Content-Type: text/plain; charset="ansi"
 Content-Transfer-Encoding: 7bit


org.apache.tika.exception.TikaException: Unexpected RuntimeException from org.apache.tika.parser.microsoft.OfficeParser@14945
        at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:244)
        at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
        at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
        at org.apache.tika.Tika.parseToString(Tika.java:380)
        at de.uplanet.lucy.server.docplug.tika.TikaDocPlug.prepare(Unknown Source)
        at de.uplanet.lucy.server.lucene.directory.DirectoryIndexManager.b(Unknown Source)
        at de.uplanet.lucy.server.lucene.directory.DirectoryIndexManager.a(Unknown Source)
        at de.uplanet.lucy.server.lucene.directory.DirectoryIndexManager.a(Unknown Source)
        at de.uplanet.lucy.server.lucene.directory.DirectoryIndexManager.createIndex(Unknown Source)
        at de.uplanet.lucy.server.lucene.directory.LuceneDirectoryJob.doWork(Unknown Source)
        at de.uplanet.lucy.server.scheduler.AbstractJob.execute(Unknown Source)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: java.lang.RuntimeException: Encoding not found - ansi
        at org.apache.poi.hsmf.datatypes.StringChunk.parseAs7BitData(StringChunk.java:155)
        at org.apache.poi.hsmf.datatypes.StringChunk.parseString(StringChunk.java:86)
        at org.apache.poi.hsmf.datatypes.StringChunk.set7BitEncoding(StringChunk.java:74)
        at org.apache.poi.hsmf.MAPIMessage.set7BitEncoding(MAPIMessage.java:421)
        at org.apache.poi.hsmf.MAPIMessage.guess7BitEncoding(MAPIMessage.java:380)
        at org.apache.tika.parser.microsoft.OutlookExtractor.parse(OutlookExtractor.java:80)
        at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:210)
        at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:161)
        at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
        ... 12 more
Caused by: java.io.UnsupportedEncodingException: ansi
        at java.lang.StringCoding.decode(StringCoding.java:170)
        at java.lang.String.<init>(String.java:443)
        at java.lang.String.<init>(String.java:515)
        at org.apache.poi.hsmf.datatypes.StringChunk.parseAs7BitData(StringChunk.java:153)
        ... 20 more

Comment 1 Nick Burch 2012-10-09 09:25:07 UTC

Do you know what's the closest JVM-supported encoding we should be mapping
this onto?

Comment 2 Yegor Kozlov 2012-10-09 11:58:13 UTC

I think it should be Windows-1252, this name is often used  as a synonym for ANSI.

See http://en.wikipedia.org/wiki/Windows-1252

(In reply to comment #1)
> Do you know what's the closest JVM-supported encoding we should be mapping
> this onto?

Comment 3 emergency.shower 2012-10-09 12:01:37 UTC

IMO there's no such thing like a closest encoding since it may depend on the language of the encoded text.

http://en.wikipedia.org/wiki/Code_page#Windows_.28ANSI.29_code_pages

Nowever, personally I would prefer Windows-1252, or ISO-8859-1 ;-)

Comment 4 Nick Burch 2014-05-08 15:17:55 UTC

Logic added in r1593306 to treat "ansi" as an alias for windows-1252 when decoding HSMF strings