package components
- Alphabetic
- Public
- All
Type Members
-
case class
CommitTracked[Value](value: Value, commit: Committable) extends Tracked[Value] with HasCommittable[Value] with Product with Serializable
A Tracked wrapper using a
Committable
with an unknown source partition.A Tracked wrapper using a
Committable
with an unknown source partition.- Value
The type of the object contained by the tracking wrapper.
- value
The wrapped object.
- commit
Commit information of the source message.
-
trait
HasCommittable[Value] extends Tracked[Value]
A Tracked type which uses a
Committable
as tracking information.A Tracked type which uses a
Committable
as tracking information.- Value
The type of the object contained by the tracking wrapper.
-
trait
HasCommittableSinks[T[X] <: HasCommittable[X]] extends TrackedSink[T] with TrackedMultiSink[T]
A factory object creating Sinks for
HasCommittable
types.A factory object creating Sinks for
HasCommittable
types.- T
The tracked wrapper type of the objects consumed by the Sink.
-
trait
HasPartition[Value] extends Tracked[Value]
A Tracked wrapper superclass for Tracked types which know the partition of their inputs.
A Tracked wrapper superclass for Tracked types which know the partition of their inputs. NOTE: THESE OBJECTS CANNOT BE FLATTENED FROM SEPARATE PARTITIONS!
- Value
The type of the object contained by the tracking wrapper.
-
case class
MockTracked[Value](value: Value, part: Int, consuming: Long) extends Tracked[Value] with Product with Serializable
Mock Tracked wrapper for testing, matches the requirements of the Commitable trackers.
-
case class
MockTransaction[Value](value: Value, part: Int, minOffset: Long, maxOffset: Long) extends Tracked[Value] with HasPartition[Value] with Product with Serializable
Mock Tracked wrapper for testing, matches the requirements of the Transactional trackers.
-
case class
PartTracked[Value](value: Value, commit: Committable, part: Int) extends Tracked[Value] with HasCommittable[Value] with HasPartition[Value] with Product with Serializable
A Tracked wrapper using a
Committable
with an known source partition.A Tracked wrapper using a
Committable
with an known source partition.- Value
The type of the object contained by the tracking wrapper.
- value
The wrapped object.
- commit
Commit information of the source message.
- part
The partition of the source message.
- case class PartialResponse(pii: Option[PiInstance[ObjectId]], returns: Seq[CallResult], errors: Seq[PiFailure[ObjectId]]) extends Product with Serializable
-
class
SequenceResponseBuilder[T[X] <: Tracked[X]] extends AnyRef
Object that aggregates information required to properly respond to a sequence of PiiUpdate and SequenceRequest messages in a combined SequenceRequest topic.
Object that aggregates information required to properly respond to a sequence of PiiUpdate and SequenceRequest messages in a combined SequenceRequest topic.
Example Sequence and Responses: older... ...newer [ S1 S1 U1|U1 S2 S1 U2|S1 U2 S2 U1|S2 U1 U2|U1 U2 U3 S3 ] pii history topic [ R1 | R1 R2 | R1 R2 | R1 R2 | R1 R2 R3 ] reduce request topic
-
abstract
class
Tracked[Value] extends AnyRef
Superclass wrapper around objects processed from objects in Kafka topics.
Superclass wrapper around objects processed from objects in Kafka topics. The wrapper contains the information necessary to update to consume the input messages when the Tracked wrapper is eventually "Produced" to an "output" Kafka topic.
- Value
The type of the object contained by the tracking wrapper.
-
trait
TrackedMultiSink[T[X] <: Tracked[X]] extends AnyRef
Superclass of factory objects capable of creating new Kafka Sinks for Tracked collection types.
Superclass of factory objects capable of creating new Kafka Sinks for Tracked collection types.
- T
The tracked wrapper type of the objects consumed by the Sink.
-
trait
TrackedSink[T[X] <: Tracked[X]] extends AnyRef
Superclass of factory objects capable of creating new Kafka Sinks for Tracked types.
Superclass of factory objects capable of creating new Kafka Sinks for Tracked types.
- T
The tracked wrapper type of the objects consumed by the Sink.
-
trait
TrackedSource[T[X] <: Tracked[X]] extends AnyRef
Superclass of objects capable of creating new Kafka Sources of Tracked types.
Superclass of objects capable of creating new Kafka Sources of Tracked types.
- T
The tracked wrapper type of the objects emitted by the Source.
-
case class
Transaction[Value](value: Value, partOffset: PartitionOffset) extends Tracked[Value] with HasPartition[Value] with Product with Serializable
A Tracked wrapper type using the
Transactional
Kafka interface.A Tracked wrapper type using the
Transactional
Kafka interface.Transactional
commits ensure the topic offset updates and message production happen atomicly.This is necessary for exactly-once message semantics.
- Value
The type of the object contained by the tracking wrapper.
- value
The wrapped object.
- partOffset
Transaction information.
-
case class
Untracked[Value](value: Value) extends Tracked[Value] with Product with Serializable
Untracked objects which satisfy this interface, messages are consumed immediately at their Source.
Untracked objects which satisfy this interface, messages are consumed immediately at their Source.
- Value
The type of the object contained by the tracking wrapper.
- value
The wrapped object.
Value Members
- object CommitTracked extends TrackedSource[CommitTracked] with HasCommittableSinks[CommitTracked] with Serializable
-
object
KafkaConnectors
High-Level Kafka Interface: Responsible for correctly wrapping StatelessComponents into RunnableGraphs to execute off of a Kafka cluster.
High-Level Kafka Interface: Responsible for correctly wrapping StatelessComponents into RunnableGraphs to execute off of a Kafka cluster. Makes use of a KafkaExecutorSettings instance for the correct construction of Kafka Producers and Consumers.
To fulfill the Executor interface, the following connectors must run on a kafka cluster:
- EITHER: At least one
indyReducer
&indySequencer
OR: At least onseqReducer
- At least one
indyAtomicExecutor
- OPTIONAL: Either a
specificResultListener
oruniqueResultListener
if the results of each PiInstance execution need responding to. -
object
KafkaWrapperFlows
Mid-Level Kafka Interface: Defines the akka sources, flows, and sinks, which the high-level interface (KafkaConnectors) builds executable wrappers for the StatelessComponents with.
- object MockTracked extends Serializable
- object MockTransaction extends Serializable
- object PartTracked extends TrackedSource[PartTracked] with HasCommittableSinks[PartTracked] with Serializable
-
object
Tracked
Tracked type helper functions, using implicits to redirect to the correct
TrackedSource
s,TrackedSink
s, andTrackedMultiSink
s. - object Transaction extends TrackedSource[Transaction] with TrackedSink[Transaction] with TrackedMultiSink[Transaction] with Serializable
- object Untracked extends TrackedSource[Untracked] with TrackedSink[Untracked] with Serializable