org.apache.spark.sql.parquet

Class CatalystArrayContainsNullConverter

Object

parquet.io.api.Converter

parquet.io.api.GroupConverter

org.apache.spark.sql.parquet.CatalystConverter

org.apache.spark.sql.parquet.CatalystArrayContainsNullConverter

public class CatalystArrayContainsNullConverter
extends CatalystConverter

A parquet.io.api.GroupConverter that converts a single-element groups that match the characteristics of an array contains null (see ParquetTypesConverter) into an ArrayType.

Constructor Summary

Constructors

Constructor and Description
CatalystArrayContainsNullConverter(org.apache.spark.sql.catalyst.types.DataType elementType, int index, CatalystConverter parent)
CatalystArrayContainsNullConverter(org.apache.spark.sql.catalyst.types.DataType elementType, int index, CatalystConverter parent, scala.collection.mutable.Buffer<Object> buffer)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.spark.sql.catalyst.types.DataType	elementType()
void	end()
parquet.io.api.Converter	getConverter(int fieldIndex)
int	index() The index of this converter in the parent
int	size() The number of fields this group has
void	start()

Methods inherited from class org.apache.spark.sql.parquet.CatalystConverter

ARRAY_CONTAINS_NULL_BAG_SCHEMA_NAME, ARRAY_ELEMENTS_SCHEMA_NAME, getCurrentRecord, MAP_KEY_SCHEMA_NAME, MAP_SCHEMA_NAME, MAP_VALUE_SCHEMA_NAME

Methods inherited from class parquet.io.api.GroupConverter

asGroupConverter, isPrimitive

Methods inherited from class parquet.io.api.Converter

asPrimitiveConverter

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CatalystArrayContainsNullConverter

public CatalystArrayContainsNullConverter(org.apache.spark.sql.catalyst.types.DataType elementType,
                                  int index,
                                  CatalystConverter parent,
                                  scala.collection.mutable.Buffer<Object> buffer)

CatalystArrayContainsNullConverter

public CatalystArrayContainsNullConverter(org.apache.spark.sql.catalyst.types.DataType elementType,
                                  int index,
                                  CatalystConverter parent)

Method Detail

elementType

public org.apache.spark.sql.catalyst.types.DataType elementType()

index

public int index()

Description copied from class: CatalystConverter

The index of this converter in the parent

getConverter

public parquet.io.api.Converter getConverter(int fieldIndex)

Specified by:

getConverter in class parquet.io.api.GroupConverter

size

public int size()

Description copied from class: CatalystConverter

The number of fields this group has

start

public void start()

Specified by:

start in class parquet.io.api.GroupConverter

end

public void end()

Specified by:

end in class parquet.io.api.GroupConverter