|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectcom.datasalt.pangool.tuplemr.TupleMRConfigBuilder
public class TupleMRConfigBuilder
ConfigBuilder creates TupleMRConfig
immutable instances.
TupleMRConfig
Constructor Summary | |
---|---|
TupleMRConfigBuilder()
|
Method Summary | |
---|---|
void |
addIntermediateSchema(Schema schema)
Adds a Map-output schema. |
TupleMRConfig |
buildConf()
Creates a brand new and immutable TupleMRConfig instance. |
static void |
initializeComparators(org.apache.hadoop.conf.Configuration conf,
TupleMRConfig groupConfig)
Initializes the custom comparator instances inside the given config criterias, calling the Configurable.setConf(Configuration) method. |
void |
setCustomPartitionFields(String... fields)
Sets the fields used to partition the tuples emmited by TupleMapper
. |
void |
setFieldAliases(String schemaName,
Aliases aliases)
Permits to set aliases, or alternate names,to fields that belong to intermediate schema's. |
void |
setGroupByFields(String... groupByFields)
Defines the fields used to group tuples by. |
void |
setOrderBy(OrderBy ordering)
Sets the criteria to sort the tuples by. |
void |
setRollupFrom(String rollupFrom)
|
void |
setSpecificOrderBy(String schemaName,
OrderBy ordering)
Sets how tuples from the specific schemaName will be sorted after being sorted by commonOrderBy and schemaOrder |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public TupleMRConfigBuilder()
Method Detail |
---|
public void addIntermediateSchema(Schema schema) throws TupleMRException
TupleMRException
public void setGroupByFields(String... groupByFields) throws TupleMRException
TupleReducer.reduce(com.datasalt.pangool.io.ITuple, java.lang.Iterable, com.datasalt.pangool.tuplemr.TupleReducer.TupleMRContext, com.datasalt.pangool.tuplemr.TupleReducer.Collector)
call.When multiple schemas are set then the groupBy fields are used to perform co-grouping among tuples with different schemas. The groupBy fields specified in this method in a multi-source scenario must be present in every intermediate schema defined.
A field that's named differently among the intermediate schemas must be aliased in order to be used in the groupBy. For that purpose, use
setFieldAliases(String, Aliases)
.
TupleMRException
public void setRollupFrom(String rollupFrom) throws TupleMRException
TupleMRException
public void setCustomPartitionFields(String... fields) throws TupleMRException
TupleMapper
. The default implementation performs a partial hashing over the group-by
fields.
TupleMRException
TupleHashPartitioner
public void setFieldAliases(String schemaName, Aliases aliases) throws TupleMRException
b.addIntermediateSchema(new Schema("schema1", Fields.parse("my_url:string, my_id:int")
b.addIntermediateSchema(new Schema("schema2",Fields.parse("site:string,visits:int")
b.setFieldAliases("schema1",new Aliases().add("url","my_url"));
b.setFieldAliases("schema2",new Aliases().add("url","site"));
b.setGroupByFields("url");
schemaName
- The schema the fields to be aliased belong to.aliases
- An Aliases
instance that contains pairs of (alias, referenced_field) pairs.
TupleMRException
public void setOrderBy(OrderBy ordering) throws TupleMRException
TupleMRException
OrderBy
public void setSpecificOrderBy(String schemaName, OrderBy ordering) throws TupleMRException
TupleMRException
public TupleMRConfig buildConf() throws TupleMRException
TupleMRConfig
instance.
TupleMRException
public static void initializeComparators(org.apache.hadoop.conf.Configuration conf, TupleMRConfig groupConfig)
Configurable.setConf(Configuration)
method.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |