Besu : Mainnet activity log H1 2022

In-Progress Reports

Maintainer github handle: shemnon

Maintainer org: Hedera

Query for mainnet label on closed PRs: https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Ashemnon+label%3Amainnet

Details on mainnet contributions

• EVM performance - tripled Gas Per Second through a number of enhancements
• EVM Compliance - updated code to handle nonce limits
• Reference Tests - bright ref tests to current revision
• Consulted for shadowfork consensus break
• Preliminary design discussions with Karim to validate checkpointing in sync.

Maintainer Github handle: gezero

Maintener org: ConsenSys

Query for closed PRs: https://github.com/hyperledger/besu/pulls?q=is%3Apr+author%3Agezero+is%3Aclosed+

Details on mainnet contributions:

• Bacwards Sync:
  • Backwards Sync is now adding blocks to BadBlocksManager
  • Backward Sync should remember recent finalized blocks
  • Improving backwards sync
  • Batching backward sync
  • Backward sync
• Full sync:
  • FullSync Future should stop when total terminal difficulty is reached
• Fast sync:
  • Clear previous version fast sync runs data
  • When downloading the world state, persist children before parents
• Miscelanious fixes:
  • findSyncTarget always called with Optional.empty
  • Return GOQUORUM_PRIVATE_STORAGE where it was before i broke it

Maintainer GitHub handle: daniellehrner

Maintainer org: ConsenSys

Query for mainnet label on closed issues: https://github.com/hyperledger/besu/issues?q=is%3Aissue+is%3Aclosed+assignee%3Adaniellehrner+label%3Amainnet+

Details on mainnet contributions

• Update reference tests to the latest version and include test for gray glacier
• Gray glacier hard fork
• fcU: Remove check if new finalized is descendant from the previous one
• EngineForkchoiceUpdated: Verify if payload attributes are valid
• Subcommand blocks import throws exception
• Acceptance test for execution engine apis
• MERGE: Rename field random to prevRandao
• MERGE: Rename opcode RANDOM to PREVRANDAO
• MERGE: Add Kiln test vectors
• MERGE: Implement engine_exchangeTransitionConfigurationV1 API
• MERGE: extend semantics of executePayload / newPayload method
• Change payloadId of engine_forkchoiceUpdatedV1 API to fixed 8 bytes

Maintainer GitHub handle: fab-10

Maintainer org: ConsenSys

Query for mainnet label on closed PRs: https://github.com/hyperledger/besu/pulls?q=is%3Apr+author%3Afab-10+is%3Aclosed+label%3Amainnet

Details on mainnet contributions

• Feature: "The Merge" switch to PoS
  
  • Extend block parameter methods to accept 'finalized' and 'safe' as block tags
  • Do not require a minimum block height when downloading headers or blocks
  • Allow to backward sync to request headers back to last finalized block if present or genesis
  • Replace deprecated INVALID_TERMINAL_BLOCK with INVALID lvh 0x0
  • Stop backward sync if genesis block has been reached
  • Fix sync terminal condition, that should stop on difficulty >= TTD
  • Make pandas thread safe
  • Fix: getMixHashOrPrevRandao to return the value present in the block header
  • Stop the BlockPropagationManager when it receives the TTD reached event
  • Fast sync with merge
  • Persist last finalized block hash, prerequisite for The Merge
  • Extend validateAndProcessBlock to return an error message in case of …
  • refactor MergeCoordinator executeBlock to return validation reason
  • Persist last finalized block hash
• Feature: transactions synchronization improvements
  • Skip seen transactions
  • Improve eth/66 support
  • Tune transaction synchronization parameter to adapt to mainnet traffic
• Feature: JSON RPC response streaming
  • Improve closing behavior of JsonResponseStreamer, and make clear it i…
  • Response streaming: stop on IO error and remove queue management
• Code improvements
  • Introduce RocksDbSegmentIdentifier to avoid changing the storage plug…
  • Improve code readability
• Fix: block propagation and synchronization
  • Refactor to async retrieve blocks, and change peer when retrying to get a block
  • Do not exit ChainDownloader in case of a generic CancellationException
• Various
  • Log initial sync stats every 10 secs at max to avoid spamming logs
  • Disable RocksDB TTL compactions
  • Filter Netty native lib errors likewise the pure Java implementation

Features for "The Merge" that I ported on the main branch, but have been originally written by others developers from ConsenSys on the merge branch.

Code for the following PRs was written by Gary Schulte Justin Florentine Daniel Lehrner Jiri Peinlich (Deactivated)

• Execution specific RPC endpoint
• Add components to handle the transition from PoW to PoS
• Add The Merge core components
• Add header validation rules needed to validate The Merge blocks
• Add PostMergeContext that will stop syncing after the switch to PoS
• Rename MergeOptions to MergeConfigOptions to follow the standard
• New MutableBlockchain method rewindToBlock by hash prerequisite for T…
• Merge: extend block creation and mining to support The Merge
• Merge: port of backward sync
• Merge: add terminal block number and hash to genesis config options

Maintainer GitHub handle: macfarla

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Amacfarla+label%3Amainnet

Details on mainnet contributions

• Trace APIs - implement the remaining endpoints - worked with frankisawesome, mark-terry and pinges on these 
  • trace_callMany #2886 
  • trace_rawTransaction #2887 
  • trace_get #2889
• bugfix
  • Prevent connecting to self enode 
  • Fixed bug with contract address supplied to `debug_accountAt`
  • Fixed bug where GraphQL "any" topic does not return any results
  • Handle null pointer caused by empty json
• CI improvements
  • LGTM config file preventing out of memory errors 
  • CodeQL config file preventing out of memory errors
  • Add trivy docker scan
  • Separate CI steps for spotless and DCO

Maintainer GitHub handle: pinges

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+author%3Apinges+is%3Aclosed+label%3Amainnet

Details on mainnet contributions

• Trace APIs - implement the remaining endpoints - worked with frankisawesome, mark-terry and macfarla on these 
  • trace_callMany #2886 
  • trace_rawTransaction #2887 
  • trace_get #2889
• bugfix
  • Fix Null pointer exception in VertxPeerDiscoveryController

Maintainer GitHub handle: mark-terry

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Amark-terry+label%3Amainnet

Details on mainnet contributions

• Trace APIs - implement the remaining endpoints - worked with frankisawesome, pinges and macfarla on these 
  • trace_callMany #2886 
  • trace_rawTransaction #2887 
  • trace_get #2889

Maintainer GitHub handle: frankisawesome

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Afrankisawesome+label%3Amainnet

Details on mainnet contributions

• Trace APIs - implement the remaining endpoints - worked with pinges, mark-terry and macfarla on these 
  • trace_callMany #2886 
  • trace_rawTransaction #2887 
  • trace_get #2889
• Allow optional API methods with no authentication
• Add uPNP only for P2P NAT method
• bugfix
  • Fixed Null pointer exception in DeFramer

Maintainer GitHub handle: taccatisid

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Ataccatisid+label%3Amainnet

Details on mainnet contributions

• code improvement
  • Refactor timer from BesuRunner into TransactionPoolEvictionService

Maintainer GitHub handle: wcgcyx

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Awcgcyx+label%3Amainnet

Details on mainnet contributions

• code improvement
  • Make warnings consistent between CLI, ENV and TOML 

Maintainer GitHub handle: siladu

Maintainer org: ConsenSys

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Asiladu+label%3Amainnet

Details on mainnet contributions

• code improvement
  • Ensure port clash is reported when engine-rpc-port clashes with default websocket port

Maintainer GitHub handle: matkt

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Amatkt+is%3Amerged

Maintainer org: ConsenSys

Details on mainnet contributions

•  Fix mainnet launcher issue
  • https://github.com/hyperledger/besu/pull/3352
• Snapsync implementation
  • https://github.com/hyperledger/besu/pull/3572
  • https://github.com/hyperledger/besu/pull/3601
  • https://github.com/hyperledger/besu/pull/3625
  • https://github.com/hyperledger/besu/pull/3656
  • https://github.com/hyperledger/besu/pull/3703
  • https://github.com/hyperledger/besu/pull/3710
  • https://github.com/hyperledger/besu/pull/3754
  • https://github.com/hyperledger/besu/pull/3761
  • https://github.com/hyperledger/besu/pull/3773
  • https://github.com/hyperledger/besu/pull/3920
• Checkpoint sync implementation
  • https://github.com/hyperledger/besu/pull/3849
• Fix ethstats issue
  • https://github.com/hyperledger/besu/pull/3729
• Fix issues on bonsai
  • https://github.com/hyperledger/besu/pull/3712 (bonsai issue with fastsync)
  • https://github.com/hyperledger/besu/pull/3645 (nullpointer on bonsai)
  • https://github.com/hyperledger/besu/pull/3684 (bonsai storage modification to work with snapsync)
• Candidate fix for the issue "Invalid column family specified in write batch"
  • https://github.com/hyperledger/besu/pull/3940

Maintainer GitHub handle: jflo

Query for mainnet label on closed PRs. https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Ajflo+is%3Amerged+updated%3A%3E%3D2022-01-01

Maintainer org: ConsenSys

Details on mainnet contributions

• JWT and Websockets, mostly on engine api
  • https://github.com/hyperledger/besu/pull/4039
  • https://github.com/hyperledger/besu/pull/4019
  • https://github.com/hyperledger/besu/pull/3657
  • https://github.com/hyperledger/besu/pull/3508
  • https://github.com/hyperledger/besu/pull/3493
  • https://github.com/hyperledger/besu/pull/3834
• Engine API
  • https://github.com/hyperledger/besu/pull/3350
• Merge related
  • https://github.com/hyperledger/besu/pull/3976
  • https://github.com/hyperledger/besu/pull/3944
  • https://github.com/hyperledger/besu/pull/3941
  • https://github.com/hyperledger/besu/pull/3903
  • https://github.com/hyperledger/besu/pull/3757
  • https://github.com/hyperledger/besu/pull/3726
  • https://github.com/hyperledger/besu/pull/3623
  • https://github.com/hyperledger/besu/pull/3548
  • https://github.com/hyperledger/besu/pull/3224
• Bugfixes and releases
  • https://github.com/hyperledger/besu/pull/4032
  • https://github.com/hyperledger/besu/pull/3979
  • https://github.com/hyperledger/besu/pull/3978
  • https://github.com/hyperledger/besu/pull/3965
  • https://github.com/hyperledger/besu/pull/3937
  • https://github.com/hyperledger/besu/pull/3861
  • https://github.com/hyperledger/besu/pull/3844
  • https://github.com/hyperledger/besu/pull/3840
  • https://github.com/hyperledger/besu/pull/3657
  • https://github.com/hyperledger/besu/pull/3305
  • https://github.com/hyperledger/besu/pull/3258

Maintainer GitHub handle: ahamlat

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+author%3Aahamlat

Maintainer org: ConsenSys

Details on mainnet contributions, largely discovery and profiling for besu performance cases.

• performance tuning mainnet use cases:
  • 
  • rocksdb performance and memory consumption:
    • 
  •  fast sync optimizations:
    • 
  • peering investigations
    • 

Maintainer GitHub handle: garyschulte

Query for mainnet label on closed PRs https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed+author%3Agaryschulte+is%3Amerged+

Maintainer org: ConsenSys

Details on mainnet contributions

• Support for arm64 native and docker images:
  • https://github.com/hyperledger/besu-native/pull/61
  • https://github.com/hyperledger/besu/pull/3904 
  • https://github.com/hyperledger/besu/pull/3848
  • https://github.com/hyperledger/besu/pull/3820
  • https://github.com/hyperledger/besu/pull/3816
• Merge engine api and merge transition implementation:
  • https://github.com/hyperledger/besu/pull/3248
  • https://github.com/hyperledger/besu/pull/3302
  • https://github.com/hyperledger/besu/pull/3341
  • https://github.com/hyperledger/besu/pull/3379
  • https://github.com/hyperledger/besu/pull/3393
  • https://github.com/hyperledger/besu/pull/3437
  • https://github.com/hyperledger/besu/pull/3447
  • https://github.com/hyperledger/besu/pull/3486
  • https://github.com/hyperledger/besu/pull/3501
  • https://github.com/hyperledger/besu/pull/3547
  • https://github.com/hyperledger/besu/pull/3554
  • https://github.com/hyperledger/besu/pull/3565
  • https://github.com/hyperledger/besu/pull/3569
  • https://github.com/hyperledger/besu/pull/3575
  • https://github.com/hyperledger/besu/pull/3607
  • https://github.com/hyperledger/besu/pull/3615
  • https://github.com/hyperledger/besu/pull/3696
  • https://github.com/hyperledger/besu/pull/3728
  • https://github.com/hyperledger/besu/pull/3748
  • https://github.com/hyperledger/besu/pull/3871
  • https://github.com/hyperledger/besu/pull/3875
  • https://github.com/hyperledger/besu/pull/3889
  • https://github.com/hyperledger/besu/pull/3894
  • https://github.com/hyperledger/besu/pull/3913
• Bonsai support
  • https://github.com/hyperledger/besu/pull/3707
  • https://github.com/hyperledger/besu/pull/3720
  • https://github.com/hyperledger/besu/pull/3734
  • https://github.com/hyperledger/besu/pull/3793
  • https://github.com/hyperledger/besu/pull/3930
• General UX
  • https://github.com/hyperledger/besu/pull/3907
  • https://github.com/hyperledger/besu/pull/3936
  • https://github.com/hyperledger/besu/pull/3958

Maintainer GitHub handle: diega

Query for mainnet label on closed PRs: https://github.com/hyperledger/besu/pulls?q=is%3Apr+is%3Aclosed++label%3Amainnet+author%3Adiega+created%3A2022-01-01..2022-06-30+

Maintainer org: Ethereum Classic Cooperative

Details on mainnet contributions:

• Code improvements
  • SLF4J migration
  • Junit5 migrations (here, here and here)
  • New errorprone rule
• Features
  • Add IPC transport
• Bug fixes
  • Flacky tests fix
  • Fix CORS regression