1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

//
// Copyright (c) Dell Inc., or its subsidiaries. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//

//! The Event API for writing and reading data.
//!
//! An Event is represented as a set of bytes within a Stream.
//! For example, an Event could be as simple as a small number of bytes containing
//! a temperature reading from an IoT sensor composed of a timestamp, a metric identifier and a value.
//! An Event could also be a web log data associated with a user click on a website.
//! Applications make sense of Events using their own serializers and deserializers,
//! allowing them to read and write objects in Pravega similarly to reading and writing objects from files.
//!
//! ## [EventWriter]
//! [EventWriter] writes events to the tail of Pravega streams.
//! It has exactly once guarantee that an Event will not missing or duplicating in Pravega
//! in case of a connection failure.
//! Each async write call will return a Result type containing an [oneshot].
//! Application can await on this [oneshot] to make sure that an Event has been persisted in Pravega.
//! If an Event is confirmed to be successfully persisted, any previous Events
//! are also guaranteed to be persisted.
//!
//! ## [EventReader]
//! [EventReader] reads Events from a Pravega Stream.
//! An [EventReader] must belong to a [ReaderGroup]. The [EventReader] read call returns a slice of segment data
//! and application can call its iterator API to get the next Event. After finishing the slice, reader
//! drops the slice and will try to get another segment slice from the stream.
//!
//! ## [ReaderGroup]
//! [ReaderGroup] a named collection of [EventReader].
//! A Reader Group perform parallel reads from a given Stream.
//! It is guaranteed that each Event published to a Stream is sent to exactly one Reader within the Reader Group.
//! There could be one or more [EventReader] in the Reader Group and there could be many different
//! Reader Groups simultaneously reading from any given Stream.
//! A Reader Group can be considered as a "composite Reader" or "distributed Reader",
//! that allows a distributed application to read and process Stream data in parallel.
//! A large amount of Stream data can be consumed by a coordinated group of Readers in a Reader Group.
//! For example, a collection of Flink tasks processing Stream data in parallel using Reader Group.
//!
//! ## [TransactionalEventWriter]
//! [TransactionalEventWriter] provides a way to execute [Transaction] in Pravega.
//! The idea of a Transaction is that a Writer can "batch" up a bunch of Events and commit them as a unit into a Stream.
//! This is useful, for example, in Flink jobs, using Pravega as a sink.
//! The Flink job can continuously produce results for some data processing and use the Transaction
//! to durably accumulate the results of the processing.
//! For example, at the end of some sort of time window, the Flink job can commit the Transaction
//! and therefore make the results of the processing available for downstream processing,
//! or in the case of an error, the Transaction is aborted and the results disappear.
//! See more [details].
//!
//! [EventWriter]: crate::event::writer::EventWriter
//! [oneshot]: https://docs.rs/tokio/1.5.0/tokio/sync/oneshot/index.html
//! [TransactionalEventWriter]: crate::event::transactional_writer::TransactionalEventWriter
//! [Transaction]: crate::event::transactional_writer::Transaction
//! [details]: https://pravega.io/docs/nightly/pravega-concepts/#transactions
//! [EventReader]: crate::event::reader::EventReader
//! [ReaderGroup]: crate::event::reader_group::ReaderGroup
//!
pub mod reader;
#[doc(inline)]
pub use reader::EventReader;

pub mod reader_group;
#[doc(inline)]
pub use reader_group::ReaderGroup;

pub mod transactional_writer;
#[doc(inline)]
pub use transactional_writer::Transaction;
#[doc(inline)]
pub use transactional_writer::TransactionalEventWriter;

pub mod writer;
#[doc(inline)]
pub use writer::EventWriter;

pub mod reader_group_state;