Filecoin protocol opportunities from PL Cryptonet

[anorth]

The PL Cryptonet team devotes a lot of time to Filecoin core protocol improvement: identification of problems and opportunities, protocol concepts, analysis, detailed design, implementation, and community engagement. We are often exploring many more improvements than those described in current FIPs. We focus on immediate projects in order to ship them, at the expense of time spent communicating farther off proposals, because we’re limited by both our own resources and network upgrade opportunities. But this can lead observers to conclude that there isn’t a huge number of improvements just waiting in the wings for the resources and opportunity to deliver them.

This is a snapshot of the problems, opportunities, and solutions we’re exploring today. This isn’t the roadmap for Filecoin, but one team’s version based on present knowledge. We know other teams are pursuing other improvements and that’s great. This list is limited to core L1 protocol improvements, but both Cryptonet and other teams are working on higher level protocols and applications too. This list only includes items in which we have high confidence, and omits some problems in earlier stages of investigation.

This also isn’t a priority list or schedule for development. It’s a list of what’s available. I haven’t given any timelines for the project here as they’re subject to constrained resources and governance processes. In an ideal (to us) situation, we could realise most of these in year or so, but limits on our own resources and the network upgrade cadence make that quite unlikely. We’re also quite conscious that some changes cause work and disruption to participants. We need to consult with you all to ensure the long-term gains are worth the immediate costs.

We think these opportunities add up to huge improvements in utility, efficiency, and robustness for the Filecoin network. Any support to help us deliver these improvements to the network is greatly appreciated.

Cryptonet’s Filecoin protocol opportunities

Direct FIL+
Remove mandatory use of the (expensive, limited) built-in market actor from data onboarding. Support FIL+ onboarding with no additional deal costs. Stepping stone to efficient and flexible market contracts, data DAOs etc. See FIP discussions #719 and #730.

Data onboarding notifications on-chain (user markets)
Synchronous notifications to smart contracts when data is committed in a sector (think ActivateDeals to any contract). Supports user-programmed storage applications like markets and data DAOs. These can implement features like deal extension, renewal, transfer, renegotiation etc. Storage applications can still use FIL+ too. Allows on-the-fly deal creation at commitment time (skip PublishStorageDeals). No FIP discussion yet, some details are in the Direct FIL+ technical design.

Scalable FIL+
Support for very large (≫ sector) FIL+ allocations in a single transaction with ~constant on-chain costs, using vector commitments. User markets can use a similar scheme for arbitrary-size deals. See #708.

SP control over Window PoST schedule
Allow SPs to reschedule the recurring Window PoSt to a time of day that suits their operations, allows for maintenance window, reduces staffing hours, etc. See #735 (thanks to the team at IPFSForce for spearheading this one).

Replace batch balancer with storage fees
The batch balancer incentivises wasting chain computation and makes storage operations extra-expensive when the network is congested. Replacing this with explicit storage fees can encourage proof aggregation and make costs more predictable, while preserving network burn in case execution gas becomes abundant. See #557 and #587.

Gas lanes
Shield critical storage operations from volatility in gas prices caused by user behaviour. Retain demand-based network burn with storage fees. See #679 (thanks to PL CryptoEconLab for leading this one).

Snap Deal aggregation and/or optimistic verification
Aggregated verification for proofs of replica update. Support cheap utilisation of CC sectors at scale. No public discussion for aggregation yet, see #645 for optimistic verification.

Repeated Snap Deals
Support for sector data to be replaced more than once.

Non-interactive PoRep
Get rid of the pre-commit step entirely with one-shot sector onboarding (using a larger proof). Removes operational risk of pre-commit deposits. Simplifies sealing-as-a-service workflows. See #603.

Cron safety
Reduce or remove risks to chain validation time and economic distortions caused by use of cron. Remove cron from the built-in market actor. Remove subsidised single-sector onboarding. Move Window PoST deadline maintenance to explicit SP action. Ensure gas is an effective constraint on ~all execution. See #638 and #689 (thanks to the Lotus team who are thought leaders here).

PoRep safety
Provide a clear recovery and transition mechanism in case a flaw is ever found in PoRep. Allow long sector commitments without compromising recovery ability. See #727.

Remove termination penalty bias
Remove protocol bias to terminate specific sectors without regard for their network value, caused by variable termination penalty inputs. Simplify calculations to make penalty clear and predictable. Possibly reduce penalty for CC sectors vs data. See #691.

Remove pledge bias
Remove protocol bias to terminate specific sectors without regard to their network value, caused by variable sector initial pledge requirements. See #734.

Flexible sector & data commitments
Reduce risks of long commitments while making sector extension the default behaviour, rather than a costly intervention. Maintain commitments to data that applications can rely on, while giving SPs flexibility over which sector stores which data. See early notes.

Provable data deletion
A mechanism for SPs to prove that data has been removed from a sector, and protocol hooks for clients and smart contracts to use that.

Sector transfer
Support for transferring sectors between SPs. Supports on-chain markets for sealing-as-a-service. Needs appropriate restrictions for data clients’ consent.

On-chain data commitment
Support for verifying a sector’s data commitment (CommD) from smart contracts.

Multicalls & delegated execution
Support for multiple actor message in a single transaction (with one signature). Support for delegating execution to another actor, so gas can be paid on behalf. See #738.

Account abstraction
Support smart contracts as message originators. Supports smart-contract wallets. See #380 (thanks to the FVM team for taking point here).

Pledge shortfall
Allow reduced initial pledge requirements when tokens are scarce, in return for protocol fees. See #666 (thanks to PL CryptoEconLab for collaboration on this one).

[luckyparadise]

Thank you so much @anorth for this. My plan is that eventually, we can have a more formal way of sharing workstreams and priorities. Again let us know how we can help you further communicate or land these improvements. I know there are a lot of constraints as to the prioritization or timeliness of these improvements, but I guess it could still be helpful from your vantage position to take wild guesses as to the ideal times these improvements should (dare I say) land and even levels of importance. It will also greatly help us in pre-scoping network upgrades or at least supporting in rallying consensus from various teams on what should or should not be prioritized for any given upgrade. Otherwise, great job sharing these!