lumis.util

Class CharsetUtil

java.lang.Object

lumis.util.CharsetUtil

public class CharsetUtil
extends Object

Utility class to guess the encoding of a given text file.

Unicode files encoded in UTF-16 (low or big endian) or UTF-8 files with a Byte Order Marker are correctly discovered. For UTF-8 files with no BOM, if the buffer is wide enough, the charset should also be discovered.

A byte buffer of 4KB is usually sufficient to be able to guess the encoding.

Usage:

 // guess the encoding
 Charset guessedCharset = CharsetToolkit.guessEncoding(file, 4096);

 // create a reader with the correct charset
 CharsetToolkit toolkit = new CharsetToolkit(file);
 BufferedReader reader = toolkit.getReader();

 // read the file content
 String line;
 while ((line = br.readLine())!= null)
 {
 System.out.println(line);
 }
 

author Guillaume Laforge

Since:

4.0.0

Version:

$Revision: 10458 $ $Date: 2009-06-02 18:49:09 +0000 (Tue, 02 Jun 2009) $

Constructor Summary

Constructors

Constructor and Description
CharsetUtil(File file)
CharsetUtil(InputStream inputStream) Creates a CharsetUtil for guessing from an input stream.

Method Summary

Modifier and Type	Method and Description
Charset	getCharset()
Charset	getDefaultCharset() Retrieves the default Charset
boolean	getEnforce8Bit() Gets the enforce8Bit flag, in case we do not want to ever get a US-ASCII encoding.
boolean	hasUTF16BEBom() Has a Byte Order Marker for UTF-16 Big Endian (utf-16 and ucs-2).
boolean	hasUTF16LEBom() Has a Byte Order Marker for UTF-16 Low Endian (ucs-2le, ucs-4le, and ucs-16le).
boolean	hasUTF8Bom() Has a Byte Order Marker for UTF-8 (Used by Microsoft's Notepad and other editors).
void	init(InputStream inputStream) Initializes this CharsetUtil for guessing from an input stream.
void	setDefaultCharset(Charset defaultCharset) Defines the default Charset used in case the buffer represents an 8-bit Charset.
void	setEnforce8Bit(boolean enforce) If US-ASCII is recognized, enforce to return the default encoding, rather than US-ASCII.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CharsetUtil

public CharsetUtil(File file)
            throws IOException

Parameters:

file - of which we want to know the encoding.

Throws:

IOException

CharsetUtil

public CharsetUtil(InputStream inputStream)
            throws IOException

Creates a CharsetUtil for guessing from an input stream.

Parameters:

inputStream - the input stream.

Throws:

IOException

Since:

4.2.2

Method Detail

init

public void init(InputStream inputStream)
          throws IOException

Initializes this CharsetUtil for guessing from an input stream.

Parameters:

inputStream - the input stream.

Throws:

IOException

Since:

4.2.2

setDefaultCharset

public void setDefaultCharset(Charset defaultCharset)

Defines the default Charset used in case the buffer represents an 8-bit Charset.

Parameters:

defaultCharset - the default Charset to be returned by guessEncoding() if an 8-bit Charset is encountered.

getCharset

public Charset getCharset()

setEnforce8Bit

public void setEnforce8Bit(boolean enforce)

If US-ASCII is recognized, enforce to return the default encoding, rather than US-ASCII. It might be a file without any special character in the range 128-255, but that may be or become a file encoded with the default charset rather than US-ASCII.

Parameters:

enforce - a boolean specifying the use or not of US-ASCII.

getEnforce8Bit

public boolean getEnforce8Bit()

Gets the enforce8Bit flag, in case we do not want to ever get a US-ASCII encoding.

Returns:

a boolean representing the flag of use of US-ASCII.

getDefaultCharset

public Charset getDefaultCharset()

Retrieves the default Charset

Returns:

hasUTF8Bom

public boolean hasUTF8Bom()

Has a Byte Order Marker for UTF-8 (Used by Microsoft's Notepad and other editors).

Returns:

true if the buffer has a BOM for UTF8.

hasUTF16LEBom

public boolean hasUTF16LEBom()

Has a Byte Order Marker for UTF-16 Low Endian (ucs-2le, ucs-4le, and ucs-16le).

Returns:

true if the buffer has a BOM for UTF-16 Low Endian.

hasUTF16BEBom

public boolean hasUTF16BEBom()

Has a Byte Order Marker for UTF-16 Big Endian (utf-16 and ucs-2).

Returns:

true if the buffer has a BOM for UTF-16 Big Endian.