org.apache.spark.sql.parquet

Class CatalystGroupConverter

Object

parquet.io.api.Converter

parquet.io.api.GroupConverter

org.apache.spark.sql.parquet.CatalystConverter

org.apache.spark.sql.parquet.CatalystGroupConverter

Direct Known Subclasses:

CatalystStructConverter

public class CatalystGroupConverter
extends CatalystConverter

A parquet.io.api.GroupConverter that is able to convert a Parquet record to a Row object.

Constructor Summary

Constructors

Constructor and Description
CatalystGroupConverter(org.apache.spark.sql.catalyst.expressions.Attribute[] attributes) This constructor is used for the root converter only!
CatalystGroupConverter(org.apache.spark.sql.catalyst.types.StructField[] schema, int index, CatalystConverter parent)
CatalystGroupConverter(org.apache.spark.sql.catalyst.types.StructField[] schema, int index, CatalystConverter parent, scala.collection.mutable.ArrayBuffer<Object> current, scala.collection.mutable.ArrayBuffer<org.apache.spark.sql.catalyst.expressions.Row> buffer)

Method Summary

Methods

Modifier and Type	Method and Description
void	end()
parquet.io.api.Converter	getConverter(int fieldIndex)
org.apache.spark.sql.catalyst.expressions.Row	getCurrentRecord() Should only be called in the root (group) converter!
int	size() The number of fields this group has
void	start()

Methods inherited from class org.apache.spark.sql.parquet.CatalystConverter

ARRAY_CONTAINS_NULL_BAG_SCHEMA_NAME, ARRAY_ELEMENTS_SCHEMA_NAME, MAP_KEY_SCHEMA_NAME, MAP_SCHEMA_NAME, MAP_VALUE_SCHEMA_NAME

Methods inherited from class parquet.io.api.GroupConverter

asGroupConverter, isPrimitive

Methods inherited from class parquet.io.api.Converter

asPrimitiveConverter

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CatalystGroupConverter

public CatalystGroupConverter(org.apache.spark.sql.catalyst.types.StructField[] schema,
                      int index,
                      CatalystConverter parent,
                      scala.collection.mutable.ArrayBuffer<Object> current,
                      scala.collection.mutable.ArrayBuffer<org.apache.spark.sql.catalyst.expressions.Row> buffer)

CatalystGroupConverter

public CatalystGroupConverter(org.apache.spark.sql.catalyst.types.StructField[] schema,
                      int index,
                      CatalystConverter parent)

CatalystGroupConverter

public CatalystGroupConverter(org.apache.spark.sql.catalyst.expressions.Attribute[] attributes)

This constructor is used for the root converter only!

Method Detail

size

public int size()

Description copied from class: CatalystConverter

The number of fields this group has

getCurrentRecord

public org.apache.spark.sql.catalyst.expressions.Row getCurrentRecord()

Description copied from class: CatalystConverter

Should only be called in the root (group) converter!

Overrides:

getCurrentRecord in class CatalystConverter

Returns:

getConverter

public parquet.io.api.Converter getConverter(int fieldIndex)

Specified by:

getConverter in class parquet.io.api.GroupConverter

start

public void start()

Specified by:

start in class parquet.io.api.GroupConverter

end

public void end()

Specified by:

end in class parquet.io.api.GroupConverter