object KafkaConnectors

High-Level Kafka Interface: Responsible for correctly wrapping StatelessComponents into RunnableGraphs to execute off of a Kafka cluster. Makes use of a KafkaExecutorSettings instance for the correct construction of Kafka Producers and Consumers.

To fulfill the Executor interface, the following connectors must run on a kafka cluster:

- EITHER: At least one indyReducer & indySequencer OR: At least on seqReducer

- At least one indyAtomicExecutor

- OPTIONAL: Either a specificResultListener or uniqueResultListener if the results of each PiInstance execution need responding to.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. KafkaConnectors
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. type DrainControl = DrainingControl[Done]
  2. type Environment = KafkaExecutorEnvironment
  3. type Settings = KafkaExecutorSettings

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. def drainAndShutdown(controls: DrainControl*)(implicit env: Environment): Future[Done]
  7. def drainAndShutdownAll(controls: Seq[DrainControl])(implicit env: Environment): Future[Done]
  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def indyAtomicExecutor(exec: AtomicExecutor, threadsPerPart: Int = 1)(implicit env: Environment): DrainControl

    Run a AtomicExecutor off of the Assignment topic.

    Run a AtomicExecutor off of the Assignment topic.

    exec

    AtomicExecutor component responsible for evaluating assignments and responding with SequenceRequests.

    threadsPerPart

    Parallelism of each Consumer topic partition.

    returns

    Control object for the running process.

  14. def indyReducer(red: Reducer)(implicit env: Environment): DrainControl

    Run an independent reducer off of a ReduceRequest topic.

    Run an independent reducer off of a ReduceRequest topic. This allows a reducer to create responses to each ReduceRequest individually by using the ReduceRequest topic offset to track each messages consumption.

    red

    Reducer module responsible for generating responses for ReduceRequests.

    returns

    Control object for the running process.

  15. def indySequencer(implicit env: Environment): DrainControl

    Run an independent sequencer off of a PiiHistory topic, outputting sequenced ReduceRequests into a separate ReduceRequest topic.

    Run an independent sequencer off of a PiiHistory topic, outputting sequenced ReduceRequests into a separate ReduceRequest topic.

    returns

    Control object for the running process.

  16. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  17. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. def sendMessages(msgs: AnyMsg*)(implicit env: Environment): Future[Done]

    Creates a temporary Kafka Producer capable of sending a single message.

    Creates a temporary Kafka Producer capable of sending a single message.

    msgs

    Messages to send.

  21. def shutdown(controls: Control*)(implicit env: Environment): Future[Done]
  22. def shutdownAll(controls: Seq[Control])(implicit env: Environment): Future[Done]
  23. def specificResultListener(groupId: String)(resl: ResultListener)(implicit env: Environment): DrainControl

    Restart a terminated ResultListener group, join an existing group, or start a ResultListener with a specific group id.

    Restart a terminated ResultListener group, join an existing group, or start a ResultListener with a specific group id. Useful as PiiResult messages might need to be visible to multiple ResultListeners.

    returns

    Control object for the running process.

  24. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  25. def toString(): String
    Definition Classes
    AnyRef → Any
  26. def uniqueResultListener(resl: ResultListener)(implicit env: Environment): DrainControl

    Override source to give handler a uniqueGroupId so it each KafkaEventHandler component can listen to all events on the cluster.

    Override source to give handler a uniqueGroupId so it each KafkaEventHandler component can listen to all events on the cluster. The ResultListener is responsible for ignoring the irrelevant messages appropriately.

    returns

    Control object for the running process.

  27. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped