org.apache.spark.sql.avro

Class functions

Object

org.apache.spark.sql.avro.functions

public class functions
extends Object

Constructor Summary

Constructors

Constructor and Description
functions()

Method Summary

Modifier and Type	Method and Description
static Column	from_avro(Column data, String jsonFormatSchema) Converts a binary column of avro format into its corresponding catalyst value.
static Column	from_avro(Column data, String jsonFormatSchema, java.util.Map<String,String> options) Converts a binary column of Avro format into its corresponding catalyst value.
static Column	to_avro(Column data) Converts a column into binary of avro format.
static Column	to_avro(Column data, String jsonFormatSchema) Converts a column into binary of avro format.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

functions

public functions()

Method Detail

from_avro

public static Column from_avro(Column data,
                               String jsonFormatSchema)

Converts a binary column of avro format into its corresponding catalyst value. The specified schema must match the read data, otherwise the behavior is undefined: it may fail or return arbitrary result.

Parameters:

data - the binary column.

jsonFormatSchema - the avro schema in JSON string format.

Returns:

(undocumented)

Since:

3.0.0

from_avro

public static Column from_avro(Column data,
                               String jsonFormatSchema,
                               java.util.Map<String,String> options)

Converts a binary column of Avro format into its corresponding catalyst value. The specified schema must match actual schema of the read data, otherwise the behavior is undefined: it may fail or return arbitrary result. To deserialize the data with a compatible and evolved schema, the expected Avro schema can be set via the option avroSchema.

Parameters:

data - the binary column.

jsonFormatSchema - the avro schema in JSON string format.

options - options to control how the Avro record is parsed.

Returns:

(undocumented)

Since:

3.0.0

to_avro

public static Column to_avro(Column data)

Converts a column into binary of avro format.

Parameters:

data - the data column.

Returns:

(undocumented)

Since:

3.0.0

to_avro

public static Column to_avro(Column data,
                             String jsonFormatSchema)

Converts a column into binary of avro format.

Parameters:

data - the data column.

jsonFormatSchema - user-specified output avro schema in JSON string format.

Returns:

(undocumented)

Since:

3.0.0