Question

Where does the need for at-least-once and at-most-once semantics come from? Why can’t we have exactly-once semantics? – Justify.

Answer 1

at-most-once semantics

It make sure that for each message using this delivery message is delivered zero or one times; In general it means that messages may be lost. There may be time which message that are delivered later affects the system.

at-least-once semantics

It make sure that for each message using this delivery tries multiple attempts at delivering it, which make sure that at least one succeeds; again, in more casual terms this means that messages may be sent again but not lost. Make sure that information gets send no mattere what time it takes.

Cons of exactly-once semantics:

1)Potentially need huge amounts of storage and infrastructure to support.

2)Need for performance overhead for every event at every operator.

Events will be replayed/retransmitted when failures occur (implementing at-least-once), and through state rollback or event deduplication, operators essentially become idempotent when updating internally managed state.

It really means that distinct updates to the state of an operator that is managed by the stream processing engine are only reflected once. "Exactly-once" by no means guarantees that processing of an event, i.e. execution of arbitrary user-defined logic, will happen only once.