Package lumis.util

Class CharsetUtil

  • public class CharsetUtil
extends java.lang.Object

    Utility class to guess the encoding of a given text file.

    Unicode files encoded in UTF-16 (low or big endian) or UTF-8 files with a Byte Order Marker are correctly discovered. For UTF-8 files with no BOM, if the buffer is wide enough, the charset should also be discovered.

    A byte buffer of 4KB is usually sufficient to be able to guess the encoding.

    Usage:

     // guess the encoding
 Charset guessedCharset = CharsetToolkit.guessEncoding(file, 4096);

 // create a reader with the correct charset
 CharsetToolkit toolkit = new CharsetToolkit(file);
 BufferedReader reader = toolkit.getReader();

 // read the file content
 String line;
 while ((line = br.readLine())!= null)
 {
 System.out.println(line);
 }
    author Guillaume Laforge
    Since:
    4.0.0
    Version:
    $Revision: 10458 $ $Date: 2009-06-02 15:49:09 -0300 (Tue, 02 Jun 2009) $

    • Constructor Summary

      Constructors 
      Constructor Description
      CharsetUtil​(java.io.File file)  
      CharsetUtil​(java.io.InputStream inputStream)
      Creates a CharsetUtil for guessing from an input stream.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.nio.charset.Charset getCharset()  
      java.nio.charset.Charset getDefaultCharset()
      Retrieves the default Charset
      boolean getEnforce8Bit()
      Gets the enforce8Bit flag, in case we do not want to ever get a US-ASCII encoding.
      boolean hasUTF16BEBom()
      Has a Byte Order Marker for UTF-16 Big Endian (utf-16 and ucs-2).
      boolean hasUTF16LEBom()
      Has a Byte Order Marker for UTF-16 Low Endian (ucs-2le, ucs-4le, and ucs-16le).
      boolean hasUTF8Bom()
      Has a Byte Order Marker for UTF-8 (Used by Microsoft's Notepad and other editors).
      void init​(java.io.InputStream inputStream)
      Initializes this CharsetUtil for guessing from an input stream.
      void setDefaultCharset​(java.nio.charset.Charset defaultCharset)
      Defines the default Charset used in case the buffer represents an 8-bit Charset.
      void setEnforce8Bit​(boolean enforce)
      If US-ASCII is recognized, enforce to return the default encoding, rather than US-ASCII.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • CharsetUtil

        public CharsetUtil​(java.io.File file)
            throws java.io.IOException
        Parameters:
        file - of which we want to know the encoding.
        Throws:
        java.io.IOException

      • CharsetUtil

        public CharsetUtil​(java.io.InputStream inputStream)
            throws java.io.IOException
        Creates a CharsetUtil for guessing from an input stream.
        Parameters:
        inputStream - the input stream.
        Throws:
        java.io.IOException
        Since:
        4.2.2

    • Method Detail

      • init

        public void init​(java.io.InputStream inputStream)
          throws java.io.IOException
        Initializes this CharsetUtil for guessing from an input stream.
        Parameters:
        inputStream - the input stream.
        Throws:
        java.io.IOException
        Since:
        4.2.2

      • setDefaultCharset

        public void setDefaultCharset​(java.nio.charset.Charset defaultCharset)
        Defines the default Charset used in case the buffer represents an 8-bit Charset.
        Parameters:
        defaultCharset - the default Charset to be returned by guessEncoding() if an 8-bit Charset is encountered.

      • getCharset

        public java.nio.charset.Charset getCharset()

      • setEnforce8Bit

        public void setEnforce8Bit​(boolean enforce)
        If US-ASCII is recognized, enforce to return the default encoding, rather than US-ASCII. It might be a file without any special character in the range 128-255, but that may be or become a file encoded with the default charset rather than US-ASCII.
        Parameters:
        enforce - a boolean specifying the use or not of US-ASCII.

      • getEnforce8Bit

        public boolean getEnforce8Bit()
        Gets the enforce8Bit flag, in case we do not want to ever get a US-ASCII encoding.
        Returns:
        a boolean representing the flag of use of US-ASCII.

      • getDefaultCharset

        public java.nio.charset.Charset getDefaultCharset()
        Retrieves the default Charset
        Returns:

      • hasUTF8Bom

        public boolean hasUTF8Bom()
        Has a Byte Order Marker for UTF-8 (Used by Microsoft's Notepad and other editors).
        Returns:
        true if the buffer has a BOM for UTF8.

      • hasUTF16LEBom

        public boolean hasUTF16LEBom()
        Has a Byte Order Marker for UTF-16 Low Endian (ucs-2le, ucs-4le, and ucs-16le).
        Returns:
        true if the buffer has a BOM for UTF-16 Low Endian.

      • hasUTF16BEBom

        public boolean hasUTF16BEBom()
        Has a Byte Order Marker for UTF-16 Big Endian (utf-16 and ucs-2).
        Returns:
        true if the buffer has a BOM for UTF-16 Big Endian.