Skip to main content

blob-archiver-rs

This is a Rust implementation of the Beacon Chain blob archiver

The Blob Archiver is a service to archive and allow querying of all historical blobs from the beacon chain. It consists of two components:

  • Archiver - Tracks the beacon chain and writes blobs to a storage backend
  • API - Implements the blob sidecars API, which allows clients to retrieve blobs from the storage backend

Storage​

There are currently two supported storage options:

  • On-disk storage - Blobs are written to disk in a directory
  • S3 storage - Blobs are written to an S3 bucket (or compatible service)

You can control which storage backend is used by setting the STORAGE_TYPE to either file or s3.

The s3 backend will also work with (for example) Google Cloud Storage buckets (instructions here).

Development​

# Run the tests
cargo test --workspace --all-features --all-targets --locked

# Lint the project
cargo clippy --workspace --all-targets --all-features -- -D warnings

# Build the project
cargo build --workspace --all-targets --all-features

Run Locally​

To run the project locally, you should first copy .env.template to .env and then modify the environment variables to your beacon client and storage backend of choice. Then you can run the project with:

docker compose up

Get blobs​

After successfully starting the service, you can use the following command to obtain the blob:

  • get blob by block id from api service:
# also there is support other type of block id, like: finalized,justified.
curl -X 'GET' 'http://localhost:8000/eth/v1/beacon/blob_sidecars/head' -H 'accept: application/json'
  • get blob by slot number from api service:
curl -X 'GET' 'http://localhost:8000/eth/v1/beacon/blob_sidecars/7111008' -H 'accept: application/json'

Storage Dashboard​

MinIO has started the dashboard, allowing you to view the status of blob storage. By default, you can access it directly at:

http://localhost:9999

Options​

verbose​

--verbose=<VERBOSE>
--verbose=2

log_dir​

--log_dir=<LOG_DIR>
--log_dir=/var/log/blob-archiver

log_rotation​

--log_rotation=<LOG_ROTATION>
--log_rotation=DAILY

beacon_endpoint​

--beacon_endpoint=<BEACON_ENDPOINT>
--beacon_endpoint=http://localhost:5052

beacon_client_timeout​

--beacon_client_timeout=<BEACON_CLIENT_TIMEOUT>
--beacon_client_timeout=10

poll_interval​

--poll_interval=<POLL_INTERVAL>
--poll_interval=6

listen_addr​

--listen_addr=<LISTEN_ADDR>
--listen_addr=0.0.0.0:8000

origin_block​

--origin_block=<ORIGIN_BLOCK>
--origin_block="0x0"

storage_type​

--storage_type=<STORAGE_TYPE>
--storage_type="s3"

s3_endpoint​

--s3_endpoint=<S3_ENDPOINT>
--s3_endpoint="http://localhost:9000"

s3_bucket​

--s3_bucket=<S3_BUCKET>
--s3_bucket="blobs"

s3_path​

--s3_path=<S3_PATH>
--s3_path=/blobs

s3_compress​

--s3_compress=<S3_COMPRESS>
--s3_compress=false

fs_dir​

--fs_dir=<FS_DIR>
--fs_dir=/blobs