com.datasalt.pangool.tuplemr.mapred.lib.input
Class NullableCSVTokenizer

java.lang.Object
  extended by com.datasalt.pangool.tuplemr.mapred.lib.input.NullableCSVTokenizer
All Implemented Interfaces:
com.googlecode.jcsv.reader.CSVTokenizer

public class NullableCSVTokenizer
extends Object
implements com.googlecode.jcsv.reader.CSVTokenizer

A custom CSVTokenizer policy that detects and provides null values if needed. Null values can happen in two ways: 1) If strict quoting mode is enabled, anything that is not quoted means null. 2) If a null string is provided, if it is found without quotes then it is null.

The null string provided can be null which means there will be no null string at all.


Constructor Summary
NullableCSVTokenizer(char escapeCharacter, boolean strictQuotes, String nullString)
          Escaping wasn't handled by the provided JSCV tokenizer so we add it here.
 
Method Summary
 int getMaxFieldSize()
           
 void setMaxFieldSize(int maxRecordSize)
           
 List<String> tokenizeLine(String line, com.googlecode.jcsv.CSVStrategy strategy, BufferedReader reader)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

NullableCSVTokenizer

public NullableCSVTokenizer(char escapeCharacter,
                            boolean strictQuotes,
                            String nullString)
Escaping wasn't handled by the provided JSCV tokenizer so we add it here.

Method Detail

tokenizeLine

public List<String> tokenizeLine(String line,
                                 com.googlecode.jcsv.CSVStrategy strategy,
                                 BufferedReader reader)
                          throws IOException
Specified by:
tokenizeLine in interface com.googlecode.jcsv.reader.CSVTokenizer
Throws:
IOException

getMaxFieldSize

public int getMaxFieldSize()

setMaxFieldSize

public void setMaxFieldSize(int maxRecordSize)


Copyright © –2014 Datasalt Systems S.L.. All rights reserved.