Backfilling Messages in Lotus

This tutorial guides you through the process of backfilling messages in Lotus for versions v1.25.1 or higher, ensuring your Lotus node is up-to-date with the blockchain's history.

Overview

This tutorial guides you through the process of backfilling messages in Lotus for versions v1.25.1 or higher, ensuring your Lotus node is up-to-date with the blockchain’s history.

Prerequisites

  • Lotus version 1.25.1 or higher installed.
  • Basic understanding of Lotus configuration and command-line tools.

Configuration Settings

Update Lotus config.toml: Ensure your config.toml file includes the following settings:

EnableEthRPC = true
EthTxHashMappingLifetimeDays = 0
DisableRealTimeFilterAPI = false
DisableHistoricFilterAPI = false
EnableMsgIndex = true

Backfill Process

Step 1: Verify or Adjust Configuration

Ensure the config.toml settings are correctly set as described above.

Step 2: Initiate Backfill

Run the backfill command via lotus-shed. For a comprehensive backfill, use a high epoch number:

lotus-shed indexes backfill-msgindex --epochs=9999999

In the output of the backfill command, the nrRowsAffected metric indicates how many messages have been backfilled. A value of 0 means no new messages were backfilled, as they already exist in the database.

By default, backfill-msgindex starts from the current head and processes back through the chain for the specified number of epochs. If necessary, you can stop the backfill process and restart it from an earlier point in the chain by using the --from option to ensure thorough coverage. This can be verified if nrRowsAffected remains 0, indicating no missed messages.

Step 3: Check Database Index Size

Use ls -l /data/node/sqlite/ to inspect the size of your indexes, indicating the backfill’s extent.

Step 4: Additional Backfill Steps

Repeat the backfill process for transaction hashes and events using similar commands, ensuring comprehensive coverage of all blockchain data.

lotus-shed indexes backfill-txhash --epochs=9999999
lotus-shed indexes backfill-events --epochs=9999999

Performance Considerations

Backfilling can be resource-intensive. Consider hardware requirements and possibly setting up additional nodes for load balancing.

Conclusion

Following these steps ensures your Lotus node is fully synchronized with the network’s history, enhancing performance and reliability.

GitHub issues and pull requests related to backfilling and performance improvements.

Edit this page on GitHub