This guide explains the advanced configuration options available for optimizing the sealing pipeline on a lotus-miner
This section controls some of the behavior around sector sealing:
[Sealing] # Upper bound on how many sectors can be waiting for more deals to be packed in it before it begins sealing at any given time. # If the miner is accepting multiple deals in parallel, up to MaxWaitDealsSectors of new sectors will be created. # If more than MaxWaitDealsSectors deals are accepted in parallel, only MaxWaitDealsSectors deals will be processed in parallel # Note that setting this number too high in relation to deal ingestion rate may result in poor sector packing efficiency # 0 = no limit # # type: uint64 # env var: LOTUS_SEALING_MAXWAITDEALSSECTORS #MaxWaitDealsSectors = 2 # Upper bound on how many sectors can be sealing+upgrading at the same time when creating new CC sectors (0 = unlimited) # # type: uint64 # env var: LOTUS_SEALING_MAXSEALINGSECTORS #MaxSealingSectors = 0 # Upper bound on how many sectors can be sealing+upgrading at the same time when creating new sectors with deals (0 = unlimited) # # type: uint64 # env var: LOTUS_SEALING_MAXSEALINGSECTORSFORDEALS #MaxSealingSectorsForDeals = 0 # Prefer creating new sectors even if there are sectors Available for upgrading. # This setting combined with MaxUpgradingSectors set to a value higher than MaxSealingSectorsForDeals makes it # possible to use fast sector upgrades to handle high volumes of storage deals, while still using the simple sealing # flow when the volume of storage deals is lower. # # type: bool # env var: LOTUS_SEALING_PREFERNEWSECTORSFORDEALS #PreferNewSectorsForDeals = false # Upper bound on how many sectors can be sealing+upgrading at the same time when upgrading CC sectors with deals (0 = MaxSealingSectorsForDeals) # # type: uint64 # env var: LOTUS_SEALING_MAXUPGRADINGSECTORS #MaxUpgradingSectors = 0 # When set to a non-zero value, minimum number of epochs until sector expiration required for sectors to be considered # for upgrades (0 = DealMinDuration = 180 days = 518400 epochs) # # Note that if all deals waiting in the input queue have lifetimes longer than this value, upgrade sectors will be # required to have expiration of at least the soonest-ending deal # # type: uint64 # env var: LOTUS_SEALING_MINUPGRADESECTOREXPIRATION #MinUpgradeSectorExpiration = 0 # DEPRECATED: Target expiration is no longer used # # type: uint64 # env var: LOTUS_SEALING_MINTARGETUPGRADESECTOREXPIRATION #MinTargetUpgradeSectorExpiration = 0 # CommittedCapacitySectorLifetime is the duration a Committed Capacity (CC) sector will # live before it must be extended or converted into sector containing deals before it is # terminated. Value must be between 180-1278 days (1278 in nv21, 540 before nv21). # # type: Duration # env var: LOTUS_SEALING_COMMITTEDCAPACITYSECTORLIFETIME #CommittedCapacitySectorLifetime = "12960h0m0s" # Period of time that a newly created sector will wait for more deals to be packed in to before it starts to seal. # Sectors which are fully filled will start sealing immediately # # type: Duration # env var: LOTUS_SEALING_WAITDEALSDELAY #WaitDealsDelay = "6h0m0s" # Whether to keep unsealed copies of deal data regardless of whether the client requested that. This lets the miner # avoid the relatively high cost of unsealing the data later, at the cost of more storage space # # type: bool # env var: LOTUS_SEALING_ALWAYSKEEPUNSEALEDCOPY #AlwaysKeepUnsealedCopy = true # Run sector finalization before submitting sector proof to the chain # # type: bool # env var: LOTUS_SEALING_FINALIZEEARLY #FinalizeEarly = false # Whether new sectors are created to pack incoming deals # When this is set to false no new sectors will be created for sealing incoming deals # This is useful for forcing all deals to be assigned as snap deals to sectors marked for upgrade # # type: bool # env var: LOTUS_SEALING_MAKENEWSECTORFORDEALS #MakeNewSectorForDeals = true # After sealing CC sectors, make them available for upgrading with deals # # type: bool # env var: LOTUS_SEALING_MAKECCSECTORSAVAILABLE #MakeCCSectorsAvailable = false # Whether to use available miner balance for sector collateral instead of sending it with each message # # type: bool # env var: LOTUS_SEALING_COLLATERALFROMMINERBALANCE #CollateralFromMinerBalance = false # Minimum available balance to keep in the miner actor before sending it with messages # # type: types.FIL # env var: LOTUS_SEALING_AVAILABLEBALANCEBUFFER #AvailableBalanceBuffer = "0 FIL" # Don't send collateral with messages even if there is no available balance in the miner actor # # type: bool # env var: LOTUS_SEALING_DISABLECOLLATERALFALLBACK #DisableCollateralFallback = false # maximum precommit batch size - batches will be sent immediately above this size # # type: int # env var: LOTUS_SEALING_MAXPRECOMMITBATCH #MaxPreCommitBatch = 256 # how long to wait before submitting a batch after crossing the minimum batch size # # type: Duration # env var: LOTUS_SEALING_PRECOMMITBATCHWAIT #PreCommitBatchWait = "24h0m0s" # time buffer for forceful batch submission before sectors/deal in batch would start expiring # # type: Duration # env var: LOTUS_SEALING_PRECOMMITBATCHSLACK #PreCommitBatchSlack = "3h0m0s" # enable / disable commit aggregation (takes effect after nv13) # # type: bool # env var: LOTUS_SEALING_AGGREGATECOMMITS #AggregateCommits = true # minimum batched commit size - batches above this size will eventually be sent on a timeout # # type: int # env var: LOTUS_SEALING_MINCOMMITBATCH #MinCommitBatch = 4 # maximum batched commit size - batches will be sent immediately above this size # # type: int # env var: LOTUS_SEALING_MAXCOMMITBATCH #MaxCommitBatch = 819 # how long to wait before submitting a batch after crossing the minimum batch size # # type: Duration # env var: LOTUS_SEALING_COMMITBATCHWAIT #CommitBatchWait = "24h0m0s" # time buffer for forceful batch submission before sectors/deals in batch would start expiring # # type: Duration # env var: LOTUS_SEALING_COMMITBATCHSLACK #CommitBatchSlack = "1h0m0s" # network BaseFee below which to stop doing precommit batching, instead # sending precommit messages to the chain individually. When the basefee is # below this threshold, precommit messages will get sent out immediately. # # type: types.FIL # env var: LOTUS_SEALING_BATCHPRECOMMITABOVEBASEFEE #BatchPreCommitAboveBaseFee = "0.00000000032 FIL" # network BaseFee below which to stop doing commit aggregation, instead # submitting proofs to the chain individually # # type: types.FIL # env var: LOTUS_SEALING_AGGREGATEABOVEBASEFEE #AggregateAboveBaseFee = "0.00000000032 FIL" # When submitting several sector prove commit messages simultaneously, this option allows you to # stagger the number of prove commits submitted per epoch # This is done because gas estimates for ProveCommits are non deterministic and increasing as a large # number of sectors get committed within the same epoch resulting in occasionally failed msgs. # Submitting a smaller number of prove commits per epoch would reduce the possibility of failed msgs # # type: uint64 # env var: LOTUS_SEALING_MAXSECTORPROVECOMMITSSUBMITTEDPEREPOCH #MaxSectorProveCommitsSubmittedPerEpoch = 20 # type: uint64 # env var: LOTUS_SEALING_TERMINATEBATCHMAX #TerminateBatchMax = 100 # type: uint64 # env var: LOTUS_SEALING_TERMINATEBATCHMIN #TerminateBatchMin = 1 # type: Duration # env var: LOTUS_SEALING_TERMINATEBATCHWAIT #TerminateBatchWait = "5m0s" # UseSyntheticPoRep, when set to true, will reduce the amount of cache data held on disk after the completion of PreCommit2 to 11GiB. # # type: bool # env var: LOTUS_SEALING_USESYNTHETICPOREP #UseSyntheticPoRep = false
PreCommitSectorsBatch introduced by FIP-0008 supports a miner to pre-commit a number of sectors at once.
BatchPreCommit is set to false, pre-commitments will be sent to the chain via
PreCommitSector messages once they are ready. If
BatchPreCommit is set to true, lotus will batch pre-commitments until any of
PreCommitBatchSlack is hit:
MaxPreCommitBatchis the maximum amount of sectors' pre-commitments to batch in one
PreCommitSectorsBatchmessage. According to FIP-0008, this values is up to 256.
PreCommitBatchWaitis how long to wait before submitting the current batch. Note: the ticket of pre-commitment has an expiration of approximately 31.5 hours, one sector’s pre-commit ticket expires WILL cause the whole message to fail. Therefore, we recommend miners to set this value lower than 30 hours.
PreCommitBatchSlackis the time buffer to forcefully submit the current batch before any of the sector’s pre-commit ticket or a deal will expire. For example, if this value is set to 1 hour, which is 120 epochs, then a
PreCommitSectorsBatchmessage will be submitted for the existing batch 120 epochs before the earliest epoch among precommits' tickets and deal’s start epochs in this batch. We recommend you to set a longer slack to prevent message failures due to expirations.
To check the list of the sectors pre-commitments that are in the batching queue, run:
./lotus-miner sectors batching precommit
This will output the sector IDs:
14 15 16
To ignore the configuration and force push the current batch, run:
./lotus-miner sectors batching precommit --publish-now=true
Then in the output, the message CID of the
PreCommitSectorsBatch message and the sector number of the sectors' pre-commitments that are being submitted is listed:
Batch 0: Message: bafy2bzacecgihnlvbsqu7yksco3vs5tzk3ublbcnkedlofr6nhbq55k5ye3ci Sectors: 14 OK 15 OK 16 OK
ProveCommitAggregate introduced by FIP-0013 supports a miner to prove-commit a number of sectors at ones.
AggregateCommits is set to false, prove-commitments will be sent to the chain via
ProveCommitSector messages once they are ready. If
AggregateCommits is set to true, lotus will aggregate and batch pre-commitments until any of
CommitBatchSlack is hit:
MaxCommitBatchis the maximum amount of sectors' prove-commitments to batch in one
ProveCommitAggregatemessage. According to FIP-0013, this value is up to 819.
CommitBatchWaitis how long to wait before submitting the current batch after crossing
MinCommitBatch. Note: a prove-commitment must be submitted within 30 days after the pre-commit has landed on-chain. It is recommended to set this value lower than 30 days to prevent collateral loss.
CommitBatchSlackis the time buffer to forcefully submit the current batch before any of the sector’s pre-commitment or a deal will expire. For example, if this value is set to 1 hour, which is 120 epochs, then a
ProveCommitAggregatemessage will be submitted for the existing batch 120 epochs before the earliest epoch among precommits' expirations and deals' start epochs in this batch. We recommend you to set a longer slack to prevent message failures due to deal expirations or loss of collateral
AggregateAboveBaseFeeis the network base fee to start aggregating proofs. When the network base fee is lower than this value, the prove commits will be submitted individually via
ProveCommitSector. According to the Batch Incentive Alignment introduced in FIP-0013, we recommend you to set this value to 0.15 nanoFIL to avoid unexpected aggregation fee in burn.
MinCommitBatch is the minimum amount of sectors' prove-commitment to be batched in one
ProveCommitAggregate message. According to FIP-0013, this value cannot be less than 4, which is the cross-over point where prove-commit aggregation wins out on single prove-commit gas costs. If any of
CommitBatchSlack is hit by the amount of prove-commit is the batching queue is less than
MinCommitBatch, then prove-commitments in this batch will be proceeded individually via
To check the list of the sectors prove-commitments that are in the batching queue, run:
./lotus-miner sectors batching commit
This will output something like:
10 11 12 13 14 15 16 17
To ignore the configuration and force push the current batch, run:
./lotus-miner sectors batching commit --publish-now=true
Then in the output, the message CID of the
ProveCommitAggregate message and the sector number of the sectors' prove-commitments that are being submitted is listed:
Batch 0: Message: bafy2bzacedtmykgf5g4evdvapacpmo4l32ewu5l7yxqkzjh3h6fhev7v7qoys Sectors: 15 OK 17 OK 12 OK 10 OK 11 OK 13 OK 16 OK 14 OK
If the sectors in the queue are less than
MinCommitBatch, then individual
ProveCommitSector messages will be sent for each sector:
Batch 0: Message: bafy2bzacedpwysxdsg2ft3hfbwn6ayyaanivfwkx4inav3zm34hwmmwgsljkk Sectors: 18 OK Batch 1: Message: bafy2bzacedrx7l34ckaue7hm2ubousl3djuigyu2xw4xzywgkhttxecsm5ba2 Sectors: 19 OK
The available units are:
"ms": int64(Millisecond), "s": int64(Second), "m": int64(Minute), "h": int64(Hour),
For example, if you want to set the sector lifecycle to 180 days, you can multiply 180 days by 24 hours per day to get 4320 hours and set this value to
UseSyntheticPoRep configuration option (applicable after the nv21 upgrade), when set to true, enables the Synthetic Proof-of-Replication (Synthetic-PoRep) protocol. This protocol significantly reduces the storage overhead of the standard PoRep protocol from 352 GiB to approximately 11 GiB for 32 GiB sectors, and from 704 GiB to approximately 11 GiB for 64 GiB sectors. This is achieved by trading the storage of PoRep overhead data for synthetic challenge proofs, resulting in a net savings of up to 98% in PoRep storage consumption.
[Sealing] # UseSyntheticPoRep, when set to true, reduces PoRep storage consumption by trading overhead data for synthetic challenge proofs. # # type: bool # env var: LOTUS_SEALING_USESYNTHETICPOREP #UseSyntheticPoRep = false
The storage section controls whether the
lotus-miner can perform certain sealing actions. Depending on the setup and the use of additional seal workers, you may want to modify some of the options.
[Storage] # Upper bound on how many sectors can be fetched in parallel by the storage system at a time ParallelFetchLimit = 10 # Sealing steps that the miner can perform itself. Sometimes we have a dedicated seal worker to do them and do not want the miner to commit any resources for this. AllowAddPiece = true AllowPreCommit1 = true AllowPreCommit2 = true AllowCommit = true AllowUnseal = true AllowReplicaUpdate = true AllowProveReplicaUpdate2 = true AllowRegenSectorKey = true
Worker assigning logic
The storage section includes a worker assigning logic. It allows you to specify a assigning logic that suits your setup.
utilizationis the default assigning logic, and tries to assign tasks to lotus-workers with the lowest utilization.
spreadassigning logic tries to assign tasks to as many distinct workers as possible.
# Assigner specifies the worker assigner to use when scheduling tasks. # "utilization" (default) - assign tasks to workers with lowest utilization. # "spread" - assign tasks to as many distinct workers as possible. # # type: string # env var: LOTUS_STORAGE_ASSIGNER #Assigner = "utilization"
You can also choose to use some experimental assigner logics:
experiment-spread-qcount- this logic is similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states.
experiment-spread-tasks- this logic is similar to the spread assigner, but counts running tasks on a per-task-type basis
experiment-spread-tasks-qcount- this logic similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis.
experiment-random- In each schedule loop this assinger logic figures a set of all workers which can handle the task and then picks a random one.
Disallow remote finalize
If you do not want
Finalize tasks to be run by remote workers, you can set the
DissallowRemoteFinalize option to true. If set to true, all finalize tasks will be run on workers with local access to both the long-term storage and the sealing path containing the sector.
DisallowRemoteFinalize when set to true will force all Finalize tasks to run on workers with local access to both long-term storage and the sealing path containing the sector. If you see stuck Finalize tasks after enabling this setting, check 'lotus-miner sealing sched-diag' and 'lotus-miner storage find [sector num]' # type: bool # env var: LOTUS_STORAGE_DISALLOWREMOTEFINALIZE #DisallowRemoteFinalize = true
The fees section allows to set limits to the gas consumption for the different messages that are submitted to the chain by the miner:
[Fees] # Maximum fees to pay MaxPreCommitGasFee = "0.025 FIL" MaxCommitGasFee = "0.05 FIL" MaxTerminateGasFee = "0.5 FIL" # This is a high-value operation, so the default fee is higher. MaxWindowPoStGasFee = "5 FIL" MaxPublishDealsFee = "0.05 FIL" MaxMarketBalanceAddFee = "0.007 FIL" [Fees.MaxPreCommitBatchGasFee] Base = "0 FIL" PerSector = "0.02 FIL" [Fees.MaxCommitBatchGasFee] Base = "0 FIL" PerSector = "0.03 FIL"
Depending on the network congestion, the base fee for a message may grow or decrease. Your gas limits will have to be larger than the base fee for the messages to be included. A very large max fee can, however, result in the quick burning of funds when the base fee is very high, as the miner automatically submits messages during normal operation, so be careful about this. It is also necessary to have more funds available than any max fee set, even if the actual fee will be far less than the max fee set.
Set the maximum cost you are willing to pay for onboarding per sector in
MaxCommitBatchGasFee.PerSector to avoid unexpected high costs.
MaxCommitBatchGasFee.PerSectoris enough to aggregate proofs for 6 sectors. Adjust this respectively according to your operation. If the value is too low, the message may wait in the mempool for a long while. If you don’t have enough funds, the message will not be sent.
Make new CC sector available for snap-deal
MakeCCSectorsAvailable makes all the new CC sectors available to be upgraded with snap-deals. When this boolean is set to
true, all pledged “CC” sectors from that point onwards will be converted to “Available” state after sealing. This enables sealing the incoming storage deals more quickly into these “Available” sectors compared to creating a new sector for the deals.
Disabling new sector for deal
MakeNewSectorForDeals is set to
lotus-miner will create new sectors for incoming deals. This option can set to
false to ensure that all new deals are sealed as snap-deals into CC sectors. This can help reduce the sealing time for the new deals as long as CC sectors are ready for the snap-deals.