Package org.apache.spark.sql.util

Class MapperRowCounter

Object

org.apache.spark.util.AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

org.apache.spark.sql.util.MapperRowCounter

All Implemented Interfaces:

Serializable

public class MapperRowCounter
extends AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

An AccumulatorV2 counter for collecting a list of (mapper index, row count).

Since:

3.4.0

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
MapperRowCounter()

Method Summary

Modifier and Type	Method	Description
void	add(Long v)	Takes the inputs and accumulates.
MapperRowCounter	copy()	Creates a new copy of this accumulator.
MapperRowCounter	copyAndReset()	Creates a new copy of this accumulator, which is zero value.
boolean	isZero()	Returns false if this accumulator has had any values added to it or the sum is non-zero.
void	merge(AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>> other)	Merges another same-type accumulator into this one and update its state, i.e.
void	reset()	Resets this accumulator, which is zero value.
void	setPartitionId(Integer id)
List<scala.Tuple2<Integer,Long>>	value()	Defines the current value of this accumulator

Methods inherited from class org.apache.spark.util.AccumulatorV2

id, isRegistered, name, toString

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

MapperRowCounter

public MapperRowCounter()

Method Details

add

public void add(Long v)

Description copied from class: AccumulatorV2

Takes the inputs and accumulates.

Specified by:

add in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

Parameters:

v - (undocumented)

copy

public MapperRowCounter copy()

Description copied from class: AccumulatorV2

Creates a new copy of this accumulator.

Specified by:

copy in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

Returns:

(undocumented)

copyAndReset

public MapperRowCounter copyAndReset()

Description copied from class: AccumulatorV2

Creates a new copy of this accumulator, which is zero value. i.e. call isZero on the copy must return true.

Overrides:

copyAndReset in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

Returns:

(undocumented)

isZero

public boolean isZero()

Returns false if this accumulator has had any values added to it or the sum is non-zero.

Specified by:

isZero in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

Returns:

(undocumented)

merge

public void merge(AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>> other)

Description copied from class: AccumulatorV2

Merges another same-type accumulator into this one and update its state, i.e. this should be merge-in-place.

Specified by:

merge in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

Parameters:

other - (undocumented)

reset

public void reset()

Description copied from class: AccumulatorV2

Resets this accumulator, which is zero value. i.e. call isZero must return true.

Specified by:

reset in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

setPartitionId

public void setPartitionId(Integer id)

value

public List<scala.Tuple2<Integer,Long>> value()

Description copied from class: AccumulatorV2

Defines the current value of this accumulator

Specified by:

value in class AccumulatorV2<Long,List<scala.Tuple2<Integer,Long>>>

Returns:

(undocumented)