Besu : Modularity Implementation Approach

Created by Justin Florentine, last modified by Matt Nelson (Deactivated) on Aug 02, 2023

Context

Making Besu more modular requires thinking about the dimensions along which those modules are divided up and separated from each other. Those separations can be categorized into two types: areas of related business logic, and areas of related software function. The former is the domain of "stuff Ethereum needs" and the later is the domain of "stuff good Java software needs". When we modularize Besu, we are looking to produce reusable, composable software components within the business logic domain.  

Bounded Contexts / Business Logic / Blockchain Domain are all synonyms for the type of modules we want to create to serve the pillars of our 2022 Vision. We will know we are doing this right when users can easily create a Besu that combines different modules into the client they need.

Cross Cutting Concerns are a little bit different. These should be invisible to the users, but very helpful to the developers. Any Bounded Context may depend on any mix of Cross Cutting Concerns. This part is ok to be a web of dependencies, they are often external projects/libraries we have selected for use all throughout Besu.


Business LogicCross Cutting Concerns
  1. Consensus
    1. Proof of Work
    2. External (or none?): Proof of Stake
      1. driven by Engine API
    3. Clique
    4. IBFT
    5. QBFT
  2. P2P
    1. ETH/66 and prior
    2. DevP2P
  3. Execution
    1. EVM
    2. Tracing
  4. Transaction Management
    1. Tessera integration
    2. MEV
    3. public and private transaction support
    4. Proof of work tx validation
  5. Synchronizing
    1. Full sync
    2. Fast sync
    3. Snap sync
    4. Checkpoint sync
    5. Backwards sync
  6. Storage
    1. World State
      1. Forest
      2. Bonsai
      3. Snapshot (RocksDB specific)
      4. Verkle
    2. Blockchain 
    3. Key-Value specific implementations under each.
  1. Cryptography
    1. Elliptic Curves
    2. Signatures
    3. Hashing
    4. native implementations
  2. Serialization
    1. RLP
    2. JSON
    3. GraphQL(ish. it's a lot broader than just serialization)
  3. APIs
    1. RPC
      1. HTTP
      2. Websockets
    2. GraphQL
    3. IPC
  4. Inversion of Control
    1. Dagger
    2. Spring
  5. Observability
    1. Logging
    2. Metrics
    3. Debugging extras
  6. Configuration
    1. PicoCLI
    2. Genesis state vs. named networks
  7. Builds
    1. Static code analysis 
    2. Use-case specific distributions
    3. test automation
      1. Unit
      2. Integration
      3. System
      4. Fuzz


Goals

  1. Pick relevant modules for abstraction against the goals outlined in our Modularity Design. A good consideration is the rule of threes: we should only abstract a module when we have a need for it in three contexts.
  2. Begin work on one abstraction 
    1. Document approach 
    2. Design implementation of interface or inversion of control
    3. Review software engineering practices with the working group
    4. Review code changes and implementation details with working group
  3. Define success criteria for remaining modules
  4. Template design work from one slice and share learnings
  5. Determine remaining modules (what needs new abstraction, against our goals) 
  6. Create working group plan and discuss division of work

Proof of Concept:

Introduce Inversion of Control to implement a vertical slice of functionality. We need to find a feature that touches on a few cross-cutting concerns, but just one Bounded Context.

Nominees:

  1. Transaction validation stack - touches each consensus mechanism, any L2 network, MEV, block building, and more. Good candidate. 
  2. MetricsSystem - Each time we need to expose a metric, we need to get this MetricsSystem from upper Layers and transfer it from constructor to constructor. 
  3. Configuration management - 
  4. Jumpdest caching. Only relevant to the EVM, but requires caching, configuration, Observability, and hashing. Plan would be to provide each of these as a dependency.
  5. PoA consensus mechanisms. Impact TBD.
  6. Transaction pool. Impact TBD.
  7. Merge Context. What if the Merge Coordinator and related classes could be a dagger module?
  8. Protocol Schedule. Impact TBD.

Once the question of "will dependency injection make for cleaner composition of modules into an application" is answered, we can then start to incrementally adopt it within each bounded context, and across the cross-cutting concerns.