libmonad_event or the
Rust package monad-exec-events.
This page provides an overview of the basic concepts used in both the C
and Rust APIs.
Event rings vs. execution events
Although the real-time data system and its SDK are often called “execution events,” there are two different parts of the SDK:-
Event ring API - “event ring” is the name of a shared memory data
structure and the API for reading and writing to it. Event rings are a
general purpose, IPC broadcast utility for publishing events to any number
of reading processes. The event ring API works with unstructured I/O: like
the UNIX
read(2)andwrite(2)file I/O system calls, the event ring API sees all data as raw byte arrays -
Execution event definitions - the actual “execution events” are the
standardized binary formats that the execution daemons writes to represent
particular EVM actions. It can be thought of as a protocol, a schema, or a
serialization format. Continuing the analogy, if the event ring API is like
the UNIX
read(2)andwrite(2)file APIs, then “execution events” are like a “file format” that defines what a particular file contains
monad-event-ring
and monad-exec-events.
The C SDK is a single library, but the header files for the two different
parts live in different directories: the event ring headers live in the
category/core/event subdirectory, and the execution event files live in
category/execution/ethereum.
Event ring basics
What is an event?
Events are made up of two components:- The event descriptor is a fixed-size (currently 64 byte) object describing the common fields of an event that has happened. It contains the event’s type, a sequence number, a timestamp, and some internal book-keeping information
- The event payload is a variably-sized piece of extra data about the event, which is specific to the event type. For example, a “transaction log” event describes a single EVM log record emitted by a transaction. While the descriptor tells us the event’s type (i.e., that it is “log event”), the payload tells us all the details: the contract address, the log topics, and the log data. Some of the fields in the event descriptor not already mentioned are used to communicate where in shared memory the payload bytes are located, and the payload’s length
Remember that at the event ring API level, an event payload is just an
unstructured byte buffer; the reader must know the format of what they are
reading, and interpret it accordingly
Where do events live?
When an event occurs, an event descriptor is written into a ring buffer that lives in a shared memory segment. This ring buffer is the “event descriptor array” in the diagram below. Event payloads are stored in a different array (in a separate shared memory segment) called the “payload buffer.”- Event descriptors are fixed-size and event payloads are variably-sized
- An event descriptor refers / “points to” the location of its payload
- Event descriptors and payloads live in different contiguous arrays of shared memory
- It supports broadcast semantics: multiple readers may read from the event ring simultaneously, and each reader maintains its own iterator position within the ring
- As in typical broadcast protocols, the writer is not aware of the readers — events are written regardless of whether anyone is reading them or not. Because the writer does not even know what the readers are doing, it cannot wait for a reader if it is slow. Readers must iterate through events quickly, or events will be lost: descriptor and payload memory can be overwritten by later events. Conceptually the event sequence is a queue (it has FIFO semantics) but is it called a ring to emphasize its overwrite-upon-overflow semantics
- A sequence number is included in the event descriptor to detect gaps (missing events due to slow readers), and a similar strategy is used to detect when payload buffer contents are overwritten
Execution event basics
As mentioned, the event ring API works with unstructured I/O. This API does not understand what the data means: it knows about bytes, but does not know anything about blocks, transactions, etc. When working with a particular event ring, the reader interprets the bytes assuming they have some known format. The primary format used with the SDK are the execution events: the binary format that records what is happening in the EVM in real time, during the execution of proposed blocks on the Monad blockchain. There are a few other kinds of formats (called “content types”), but they are only used internally by Category Labs, mostly for performance profiling. For the remainder of the overview, we’ll look at an example execution event.Example: the “transaction start” event
One particularly important kind of event is the “start of transaction header” event, which is recorded shortly after a new transaction is decoded by the EVM. It contains most of the transaction information (encoded as a C structure) as its event payload. The payload structure is defined inexec_event_ctypes.h
as:
monad_c_eth_txn_header structure contains most of the interesting
information — it is defined in eth_ctypes.h as follows:
T_n and T_c) are references
to variable names in the
Ethereum Yellow Paper.
The type monad_c_uint256_ne (“native endian”) is a 256-bit integer that is
stored as a uint64_t[4] in the
limb format
used by most “big integer” libraries that have good performance.
If you are using the Rust SDK,
struct types with the same names (and the same
binary layouts, courtesy of a #[repr(C)] attribute) are generated by
bindgen when the monad-exec-events
package is built. The defining characteristic of the execution event payloads
is that they rely on the “natural” interoperability of simple C data
structures across programming languages.Most popular programming languages have a defined foreign function interface
for working with C code, and this usually also entails some way to “naturally”
work with C structure types. Although C’s data representation is not portable,
these objects live in shared memory, therefore both the reader and writer must
be on the same host, and must follow the same C ABI.Variable-length trailing arrays and subsequent events
For a particular “start of transaction” event, most of the event payload will be the low-level byte representation of astruct monad_exec_txn_header_start
value. Almost always, there will be some extra data in the payload byte array
following this structure:
-
The transaction’s variably-sized
databyte array, whose length is specified by thedata_lengthfield, is also part of the event payload and immediately follows thestruct monad_exec_txn_header_startobject -
If this is an EIP-4844 transaction, a
blob_versioned_hashesarray will immediately follow thedataarray
_length fields are
listed in the fixed-size structure.
The EIP-2930 and EIP-7702 lists are also variable-length items in a
transaction, but they are not recorded in the payload of the “start of
transaction header” event.
Instead of being recording in trailing arrays, a unique event will be recorded
for each EIP-2930 access list entry and each EIP-7702 authorization tuple. The
number of these events is published in the “start of transaction header”
event payload (see the access_list_count and auth_list_count fields), so
that the reader will know how many more events to expect.
Execution event properties in the descriptor
So far we’ve talked about the payload for a “start of transaction” event, but the common properties of the event are recorded directly in the event descriptor. Most importantly, these include the numeric code that identifies the type of event, so we know we’re supposed to interpret the unstructured payload bytes as astruct monad_exec_txn_header_start in the first place.
An event descriptor is defined this way:
event_type field will be set
to the value of the C enumeration constant MONAD_EXEC_TXN_HEADER_START, a
value of type enum monad_exec_event_type. This tells the user that it is
appropriate to cast the const uint8_t * pointing to the start of the event
payload to a const struct monad_event_txn_header_start * (or to perform
the corresponding unsafe cast in Rust).
All the C enumeration constants start with a MONAD_EXEC_ prefix, but
typically the documentation refers to event types without the prefix, e.g.,
TXN_HEADER_START.
Note that the transaction number is not included in the payload structure.
Because of their importance in the blockchain protocol, transaction numbers
are encoded directly in the event descriptor. At a low level (i.e., in the
C API) this information is encoded in the context_ext[1] field. In the Rust
API — which is a bit more user friendly — it is decoded and presented as the
structure field ExecEventRingFlowInfo::txn_idx.1
The potential presence of subsequent events with EIP-2930 and EIP-7702
information is why TXN_HEADER_START is called the start of the transaction
header. A corresponding event called TXN_HEADER_END is emitted after all the
transaction header information has been seen. TXN_HEADER_END has no payload,
and only serves to announce that all events related to the transaction input
have been recorded. Such an event is called a “marker event” in the
documentation.
Finally, the reason it is called a “header” in the first place, is that there
are many more events related to transactions. The various “transaction header”
events only describe all the inputs that were in the block. Most of the events
describe transaction outputs: the logs, the call frames, the state changes,
and the receipt.
Example in-memory layout
The following diagram illustrates everything explained above about a transaction header’s variable-length trailing arrays, related subsequent events, and its terminating marker event. This example transaction has two accounts in its EIP-2930 access list, and no EIP-7702 entries. Each address in an EIP-2930 list records a separateTXN_ACCESS_LIST_ENTRY event, with a
variable-length trailing array of potentially-accessed storage keys.
Patterns in execution event serialization
Why are EIP-2930 entries recorded as separateTXN_ACCESS_LIST_ENTRY events
instead of as variable-length trailing arrays in TXN_HEADER_START? Because
there are two levels of variable-length information involved. There are a
variable number of EIP-2930 accounts, and then for each account, a
variable-length number of associated storage keys.
The event serialization protocol tries to be very simple: the only time a
variable length trailing array will be recorded is when the array element
type is fixed size. In particular, data that is shaped like a
“jagged array”
is not permitted in an event payload.
Whenever there are multiple dimensions to the variability, they are “factored
out” by using more distinct events. The trade-off is between fewer events with
a more complex encoding, vs. more events which “unfold” the data into a
“flatter” shape. The latter choice fits better with the “zero-copy C ABI”
data model.2
Technically, the EIP-7702 authorization list could be represented as a
variable-length trailing array, since the authorization tuples are fixed-size.
However, as a design decision, variable-length trailing arrays are only allowed
to have simple element types like u8 or uint256, and there cannot be too
many of them.
The decoding logic of VLT arrays tends to be error prone; it looks confusing
because it is harder to “see” in the code exactly what the serialization rules
are. “Unfolding” the data into more events is more self-documenting: distinct
typed objects are created rather than relying on implicit parsing rules for
reinterpreting unstructured trailing data.
Consequently, VLT arrays are only used when their use seems “obvious”, e.g.,
the storage key arrays in each EIP-2930 access list entry.
Footnotes
- This encoding and the rationale for storing it in the descriptor is described elsewhere in the documentation, in the section describing flow tags). ↩
-
The natural C encoding of jagged arrays requires an array of pointers.
This can’t work in a shared memory structure unless we explicitly control
the address in virtual address space where the event ring file is mapped,
e.g., with
MAP_FIXED. ↩