Packages

package components

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. case class CommitTracked[Value](value: Value, commit: Committable) extends Tracked[Value] with HasCommittable[Value] with Product with Serializable

    A Tracked wrapper using a Committable with an unknown source partition.

    A Tracked wrapper using a Committable with an unknown source partition.

    Value

    The type of the object contained by the tracking wrapper.

    value

    The wrapped object.

    commit

    Commit information of the source message.

  2. trait HasCommittable[Value] extends Tracked[Value]

    A Tracked type which uses a Committable as tracking information.

    A Tracked type which uses a Committable as tracking information.

    Value

    The type of the object contained by the tracking wrapper.

  3. trait HasCommittableSinks[T[X] <: HasCommittable[X]] extends TrackedSink[T] with TrackedMultiSink[T]

    A factory object creating Sinks for HasCommittable types.

    A factory object creating Sinks for HasCommittable types.

    T

    The tracked wrapper type of the objects consumed by the Sink.

  4. trait HasPartition[Value] extends Tracked[Value]

    A Tracked wrapper superclass for Tracked types which know the partition of their inputs.

    A Tracked wrapper superclass for Tracked types which know the partition of their inputs. NOTE: THESE OBJECTS CANNOT BE FLATTENED FROM SEPARATE PARTITIONS!

    Value

    The type of the object contained by the tracking wrapper.

  5. case class MockTracked[Value](value: Value, part: Int, consuming: Long) extends Tracked[Value] with Product with Serializable

    Mock Tracked wrapper for testing, matches the requirements of the Commitable trackers.

  6. case class MockTransaction[Value](value: Value, part: Int, minOffset: Long, maxOffset: Long) extends Tracked[Value] with HasPartition[Value] with Product with Serializable

    Mock Tracked wrapper for testing, matches the requirements of the Transactional trackers.

  7. case class PartTracked[Value](value: Value, commit: Committable, part: Int) extends Tracked[Value] with HasCommittable[Value] with HasPartition[Value] with Product with Serializable

    A Tracked wrapper using a Committable with an known source partition.

    A Tracked wrapper using a Committable with an known source partition.

    Value

    The type of the object contained by the tracking wrapper.

    value

    The wrapped object.

    commit

    Commit information of the source message.

    part

    The partition of the source message.

  8. case class PartialResponse(pii: Option[PiInstance[ObjectId]], returns: Seq[CallResult], errors: Seq[PiFailure[ObjectId]]) extends Product with Serializable
  9. class SequenceResponseBuilder[T[X] <: Tracked[X]] extends AnyRef

    Object that aggregates information required to properly respond to a sequence of PiiUpdate and SequenceRequest messages in a combined SequenceRequest topic.

    Object that aggregates information required to properly respond to a sequence of PiiUpdate and SequenceRequest messages in a combined SequenceRequest topic.

    Example Sequence and Responses: older... ...newer [ S1 S1 U1|U1 S2 S1 U2|S1 U2 S2 U1|S2 U1 U2|U1 U2 U3 S3 ] pii history topic [ R1 | R1 R2 | R1 R2 | R1 R2 | R1 R2 R3 ] reduce request topic

  10. abstract class Tracked[Value] extends AnyRef

    Superclass wrapper around objects processed from objects in Kafka topics.

    Superclass wrapper around objects processed from objects in Kafka topics. The wrapper contains the information necessary to update to consume the input messages when the Tracked wrapper is eventually "Produced" to an "output" Kafka topic.

    Value

    The type of the object contained by the tracking wrapper.

  11. trait TrackedMultiSink[T[X] <: Tracked[X]] extends AnyRef

    Superclass of factory objects capable of creating new Kafka Sinks for Tracked collection types.

    Superclass of factory objects capable of creating new Kafka Sinks for Tracked collection types.

    T

    The tracked wrapper type of the objects consumed by the Sink.

  12. trait TrackedSink[T[X] <: Tracked[X]] extends AnyRef

    Superclass of factory objects capable of creating new Kafka Sinks for Tracked types.

    Superclass of factory objects capable of creating new Kafka Sinks for Tracked types.

    T

    The tracked wrapper type of the objects consumed by the Sink.

  13. trait TrackedSource[T[X] <: Tracked[X]] extends AnyRef

    Superclass of objects capable of creating new Kafka Sources of Tracked types.

    Superclass of objects capable of creating new Kafka Sources of Tracked types.

    T

    The tracked wrapper type of the objects emitted by the Source.

  14. case class Transaction[Value](value: Value, partOffset: PartitionOffset) extends Tracked[Value] with HasPartition[Value] with Product with Serializable

    A Tracked wrapper type using the Transactional Kafka interface.

    A Tracked wrapper type using the Transactional Kafka interface. Transactional commits ensure the topic offset updates and message production happen atomicly.

    This is necessary for exactly-once message semantics.

    Value

    The type of the object contained by the tracking wrapper.

    value

    The wrapped object.

    partOffset

    Transaction information.

  15. case class Untracked[Value](value: Value) extends Tracked[Value] with Product with Serializable

    Untracked objects which satisfy this interface, messages are consumed immediately at their Source.

    Untracked objects which satisfy this interface, messages are consumed immediately at their Source.

    Value

    The type of the object contained by the tracking wrapper.

    value

    The wrapped object.

Value Members

  1. object CommitTracked extends TrackedSource[CommitTracked] with HasCommittableSinks[CommitTracked] with Serializable
  2. object KafkaConnectors

    High-Level Kafka Interface: Responsible for correctly wrapping StatelessComponents into RunnableGraphs to execute off of a Kafka cluster.

    High-Level Kafka Interface: Responsible for correctly wrapping StatelessComponents into RunnableGraphs to execute off of a Kafka cluster. Makes use of a KafkaExecutorSettings instance for the correct construction of Kafka Producers and Consumers.

    To fulfill the Executor interface, the following connectors must run on a kafka cluster:

    - EITHER: At least one indyReducer & indySequencer OR: At least on seqReducer

    - At least one indyAtomicExecutor

    - OPTIONAL: Either a specificResultListener or uniqueResultListener if the results of each PiInstance execution need responding to.

  3. object KafkaWrapperFlows

    Mid-Level Kafka Interface: Defines the akka sources, flows, and sinks, which the high-level interface (KafkaConnectors) builds executable wrappers for the StatelessComponents with.

  4. object MockTracked extends Serializable
  5. object MockTransaction extends Serializable
  6. object PartTracked extends TrackedSource[PartTracked] with HasCommittableSinks[PartTracked] with Serializable
  7. object Tracked

    Tracked type helper functions, using implicits to redirect to the correct TrackedSources, TrackedSinks, and TrackedMultiSinks.

  8. object Transaction extends TrackedSource[Transaction] with TrackedSink[Transaction] with TrackedMultiSink[Transaction] with Serializable
  9. object Untracked extends TrackedSource[Untracked] with TrackedSink[Untracked] with Serializable

Ungrouped