com.datasalt.pangool.tuplemr.mapred
Class SortComparator

java.lang.Object
  extended by com.datasalt.pangool.tuplemr.mapred.SortComparator
All Implemented Interfaces:
Comparator<ITuple>, org.apache.hadoop.conf.Configurable, org.apache.hadoop.io.RawComparator<ITuple>
Direct Known Subclasses:
GroupComparator

public class SortComparator
extends Object
implements org.apache.hadoop.io.RawComparator<ITuple>, org.apache.hadoop.conf.Configurable

Tuple-based MapRed job binary comparator. It decodes the binary serialization performed by TupleSerialization.

Used to group tuples according to TupleMRConfigBuilder.setOrderBy(com.datasalt.pangool.tuplemr.OrderBy)


Field Summary
protected  org.apache.hadoop.conf.Configuration conf
           
protected  boolean isMultipleSources
           
protected  com.datasalt.pangool.tuplemr.mapred.SortComparator.Nulls nulls
           
protected  com.datasalt.pangool.tuplemr.mapred.SortComparator.Offsets offsets
           
protected  SerializerComparator serializerComparator
           
protected  SerializationInfo serInfo
           
protected  TupleMRConfig tupleMRConf
           
 
Constructor Summary
SortComparator()
           
 
Method Summary
protected  int compare(byte[] b1, int s1, byte[] b2, int s2, Schema schema, Criteria criteria, com.datasalt.pangool.tuplemr.mapred.SortComparator.Offsets o, com.datasalt.pangool.tuplemr.mapred.SortComparator.Nulls n)
           
 int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2)
           
 int compare(ITuple w1, ITuple w2)
          Never called in MapRed jobs.
 int compare(Schema schema, Criteria c, ITuple w1, int[] index1, ITuple w2, int[] index2, org.apache.hadoop.io.serializer.Serializer[] serializers)
           
protected  int compareMultipleSources(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2)
           
static int compareObjects(Object element1, Object element2)
           
 int compareObjects(Object elem1, Object elem2, org.apache.hadoop.io.RawComparator comparator, Schema.Field.Type type, org.apache.hadoop.io.serializer.Serializer serializer)
          Compares two objects.
 org.apache.hadoop.conf.Configuration getConf()
           
 TupleMRConfig getConfig()
           
 int nullCompare(Object o1, Object o2, Criteria.SortElement se)
           
 void setConf(org.apache.hadoop.conf.Configuration conf)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface java.util.Comparator
equals
 

Field Detail

conf

protected org.apache.hadoop.conf.Configuration conf

tupleMRConf

protected TupleMRConfig tupleMRConf

serInfo

protected SerializationInfo serInfo

serializerComparator

protected final SerializerComparator serializerComparator

offsets

protected com.datasalt.pangool.tuplemr.mapred.SortComparator.Offsets offsets

nulls

protected com.datasalt.pangool.tuplemr.mapred.SortComparator.Nulls nulls

isMultipleSources

protected boolean isMultipleSources
Constructor Detail

SortComparator

public SortComparator()
Method Detail

getConfig

public TupleMRConfig getConfig()

compare

public int compare(ITuple w1,
                   ITuple w2)
Never called in MapRed jobs. Just for completion and test purposes

Specified by:
compare in interface Comparator<ITuple>

compare

public int compare(Schema schema,
                   Criteria c,
                   ITuple w1,
                   int[] index1,
                   ITuple w2,
                   int[] index2,
                   org.apache.hadoop.io.serializer.Serializer[] serializers)

compareObjects

public int compareObjects(Object elem1,
                          Object elem2,
                          org.apache.hadoop.io.RawComparator comparator,
                          Schema.Field.Type type,
                          org.apache.hadoop.io.serializer.Serializer serializer)
Compares two objects. Uses the given custom comparator if present. If the type is Schema.Field.Type.OBJECT and no raw comparator is present, then a serializer comparator is used.


compareObjects

public static int compareObjects(Object element1,
                                 Object element2)

nullCompare

public int nullCompare(Object o1,
                       Object o2,
                       Criteria.SortElement se)

compare

public int compare(byte[] b1,
                   int s1,
                   int l1,
                   byte[] b2,
                   int s2,
                   int l2)
Specified by:
compare in interface org.apache.hadoop.io.RawComparator<ITuple>

compareMultipleSources

protected int compareMultipleSources(byte[] b1,
                                     int s1,
                                     int l1,
                                     byte[] b2,
                                     int s2,
                                     int l2)
                              throws IOException
Throws:
IOException

compare

protected int compare(byte[] b1,
                      int s1,
                      byte[] b2,
                      int s2,
                      Schema schema,
                      Criteria criteria,
                      com.datasalt.pangool.tuplemr.mapred.SortComparator.Offsets o,
                      com.datasalt.pangool.tuplemr.mapred.SortComparator.Nulls n)
               throws IOException
Throws:
IOException

getConf

public org.apache.hadoop.conf.Configuration getConf()
Specified by:
getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)
Specified by:
setConf in interface org.apache.hadoop.conf.Configurable


Copyright © –2014 Datasalt Systems S.L.. All rights reserved.