com.nhncorp.neptune.common.io
Class NText

java.lang.Object
  com.nhncorp.neptune.common.io.NText

All Implemented Interfaces:

NWritable, NWritableComparable, java.lang.Comparable, org.apache.hadoop.io.Writable, org.apache.hadoop.io.WritableComparable

Direct Known Subclasses:

AbstractKey

public class NText
extends java.lang.Object
implements NWritableComparable, org.apache.hadoop.io.WritableComparable

This class stores text using standard UTF8 encoding. It provides methods to serialize, deserialize, and compare texts at byte level. The type of length is integer and is serialized using zero-compressed format.

In addition, it provides methods for string traversal without converting the byte array to a string.

Also includes utilities for serializing/deserialing a string, coding/decoding a string, checking if a byte array contains valid UTF8 code, calculating the length of an encoded string.

Nested Class Summary

static class

NText.Comparator A WritableComparator optimized for Text keys.

Constructor Summary

NText()

NText(byte[] utf8)
Construct from a byte array.

NText(NText utf8)
Construct from another text.

NText(java.lang.String string)
Construct from a string.

Method Summary

static int	bytesToCodePoint(java.nio.ByteBuffer bytes) Returns the next code point at the current position in the buffer.
int	charAt(int position) Returns the Unicode Scalar Value (32-bit integer value) for the character at position.
int	compareTo(java.lang.Object o) Compare two Texts bytewise using standard UTF8 ordering.
static java.lang.String	decode(byte[] utf8) Converts the provided byte array to a String using the UTF-8 encoding.
static java.lang.String	decode(byte[] utf8, int start, int length)
static java.lang.String	decode(byte[] utf8, int start, int length, boolean replace) Converts the provided byte array to a String using the UTF-8 encoding.
static java.nio.ByteBuffer	encode(java.lang.String string) Converts the provided String to bytes using the UTF-8 encoding.
static java.nio.ByteBuffer	encode(java.lang.String string, boolean replace) Converts the provided String to bytes using the UTF-8 encoding.
boolean	equals(java.lang.Object o) Returns true iff o is a Text with the same contents.
int	find(java.lang.String what)
int	find(java.lang.String what, int start) Finds any occurence of what in the backing buffer, starting as position start.
int	getAllocatedSize() 레퍼런스 영역 등 모든 오버헤드를 포함한 JVM내에서 실제 차지하고 있는 메모리
byte[]	getBytes() Retuns the raw bytes.
int	getLength() Returns the number of bytes in the byte array
int	hashCode() hash function
void	readFields(java.io.DataInput in) deserialize
static java.lang.String	readString(java.io.DataInput in) Read a UTF8 encoded string from in
void	set(byte[] utf8) Set to a utf8 byte array
void	set(byte[] utf8, int start, int len) Set the Text to range of bytes
void	set(NText other) copy a text.
void	set(java.lang.String string) Set to contain the contents of a string.
static void	skip(java.io.DataInput in) Skips over one Text in the input.
java.lang.String	toString() Convert text back to string
static int	utf8Length(java.lang.String string) For the given string, returns the number of UTF-8 bytes required to encode the string.
static void	validateUTF8(byte[] utf8) Check if a byte array contains valid utf-8
static void	validateUTF8(byte[] utf8, int start, int len) Check to see if a byte array is valid utf-8
void	write(java.io.DataOutput out) serialize write this object to out length uses zero-compressed encoding
static int	writeString(java.io.DataOutput out, java.lang.String s) Write a UTF8 encoded string to out

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

NText

public NText()

NText

public NText(java.lang.String string)

Construct from a string.

NText

public NText(NText utf8)

Construct from another text.

NText

public NText(byte[] utf8)

Construct from a byte array.

Method Detail

getBytes

public byte[] getBytes()

Retuns the raw bytes.

getLength

public int getLength()

Returns the number of bytes in the byte array

charAt

public int charAt(int position)

Returns the Unicode Scalar Value (32-bit integer value) for the character at position. Note that this method avoids using the converter or doing String instatiation

Returns:

the Unicode scalar value at position or -1 if the position is invalid or points to a trailing byte

getAllocatedSize

public int getAllocatedSize()

레퍼런스 영역 등 모든 오버헤드를 포함한 JVM내에서 실제 차지하고 있는 메모리

Returns:

find

public int find(java.lang.String what)

find

public int find(java.lang.String what,
                int start)

Finds any occurence of what in the backing buffer, starting as position start. The starting position is measured in bytes and the return value is in terms of byte position in the buffer. The backing buffer is not converted to a string for this operation.

Returns:

byte position of the first occurence of the search string in the UTF-8 buffer or -1 if not found

set

public void set(java.lang.String string)

Set to contain the contents of a string.

set

public void set(byte[] utf8)

Set to a utf8 byte array

set

public void set(NText other)

copy a text.

set

public void set(byte[] utf8,
                int start,
                int len)

Set the Text to range of bytes

Parameters:

utf8 - the data to copy from

start - the first position of the new string

len - the number of bytes of the new string

toString

public java.lang.String toString()

Convert text back to string

Overrides:

toString in class java.lang.Object

See Also:

Object.toString()

readFields

public void readFields(java.io.DataInput in)
                throws java.io.IOException

deserialize

Specified by:

readFields in interface NWritable

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

java.io.IOException

skip

public static void skip(java.io.DataInput in)
                 throws java.io.IOException

Skips over one Text in the input.

Throws:

java.io.IOException

write

public void write(java.io.DataOutput out)
           throws java.io.IOException

serialize write this object to out length uses zero-compressed encoding

Specified by:

write in interface NWritable

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

java.io.IOException

See Also:

NWritable.write(DataOutput)

compareTo

public int compareTo(java.lang.Object o)

Compare two Texts bytewise using standard UTF8 ordering.

Specified by:

compareTo in interface java.lang.Comparable

equals

public boolean equals(java.lang.Object o)

Returns true iff o is a Text with the same contents.

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

hash function

Overrides:

hashCode in class java.lang.Object

decode

public static java.lang.String decode(byte[] utf8)
                               throws java.nio.charset.CharacterCodingException

Converts the provided byte array to a String using the UTF-8 encoding. If the input is malformed, replace by a default value.

Throws:

java.nio.charset.CharacterCodingException

decode

public static java.lang.String decode(byte[] utf8,
                                      int start,
                                      int length)
                               throws java.nio.charset.CharacterCodingException

Throws:

java.nio.charset.CharacterCodingException

decode

public static java.lang.String decode(byte[] utf8,
                                      int start,
                                      int length,
                                      boolean replace)
                               throws java.nio.charset.CharacterCodingException

Converts the provided byte array to a String using the UTF-8 encoding. If replace is true, then malformed input is replaced with the substitution character, which is U+FFFD. Otherwise the method throws a MalformedInputException.

Throws:

java.nio.charset.CharacterCodingException

encode

public static java.nio.ByteBuffer encode(java.lang.String string)
                                  throws java.nio.charset.CharacterCodingException

Converts the provided String to bytes using the UTF-8 encoding. If the input is malformed, invalid chars are replaced by a default value.

Returns:

ByteBuffer: bytes stores at ByteBuffer.array() and length is ByteBuffer.limit()

Throws:

java.nio.charset.CharacterCodingException

encode

public static java.nio.ByteBuffer encode(java.lang.String string,
                                         boolean replace)
                                  throws java.nio.charset.CharacterCodingException

Converts the provided String to bytes using the UTF-8 encoding. If replace is true, then malformed input is replaced with the substitution character, which is U+FFFD. Otherwise the method throws a MalformedInputException.

Returns:

ByteBuffer: bytes stores at ByteBuffer.array() and length is ByteBuffer.limit()

Throws:

java.nio.charset.CharacterCodingException

readString

public static java.lang.String readString(java.io.DataInput in)
                                   throws java.io.IOException

Read a UTF8 encoded string from in

Throws:

java.io.IOException

writeString

public static int writeString(java.io.DataOutput out,
                              java.lang.String s)
                       throws java.io.IOException

Write a UTF8 encoded string to out

Throws:

java.io.IOException

validateUTF8

public static void validateUTF8(byte[] utf8)
                         throws java.nio.charset.MalformedInputException

Check if a byte array contains valid utf-8

Parameters:

utf8 - byte array

Throws:

java.nio.charset.MalformedInputException - if the byte array contains invalid utf-8

validateUTF8

public static void validateUTF8(byte[] utf8,
                                int start,
                                int len)
                         throws java.nio.charset.MalformedInputException

Check to see if a byte array is valid utf-8

Parameters:

utf8 - the array of bytes

start - the offset of the first byte in the array

len - the length of the byte sequence

Throws:

java.nio.charset.MalformedInputException - if the byte array contains invalid bytes

bytesToCodePoint

public static int bytesToCodePoint(java.nio.ByteBuffer bytes)

Returns the next code point at the current position in the buffer. The buffer's position will be incremented. Any mark set on this buffer will be changed by this method!

utf8Length

public static int utf8Length(java.lang.String string)

For the given string, returns the number of UTF-8 bytes required to encode the string.

Parameters:

string - text to encode

Returns:

number of UTF-8 bytes required to encode