Immunefi Matching Bug Bounty Proposal

[BraveNewDeFi]

Overview

In July 2021, Immunefi submitted a grant application to request funding and create a bug bounty matching program to incentivize whitehat disclosures for critical threats and prevent claimable events.

Now that the mutual has matched a $200,000 bounty for a critical vulnerability disclosure, members need to determine if the program should continue, receive additional funding, and if any adjustments should be made to the terms of the program.

In a previous Request for Comment (RFC) post, I provided a review of the program, outlined the benefits, and sought out comments from the community. Members will find the proposal below and an overview of the suggestions offered during the review period.

Proposed Changes to the Program

While the program started with a maximum cap on payouts at $200k, there is a cost effective benefit to incentivize critical disclosures for protocols with a certain level of active cover within the mutual. The proposed changes are included below and expanded upon further in the next section.

• Increasing maximum cap on total payouts from $200k to $600k.

• Whitelist any project with greater than $2m in active cover.

• Adjust matching amount from $1 in matching for every $1 offered as a critical bug bounty (1:1) to $0.50 in matching for every $1 offered as a critical bug bounty (0.5:1) to incentivize projects to offer larger bug bounties.

Proposal to Renew Program, Increase Budget

This proposal seeks a renewal of the Immunefi Matching Bug Bounty Program with the following requirements and terms:

1. Projects with an active bug bounty program on Immunefi.

2. Provide matching for bug bounties with a critical threat level rating.

3. Cap maximum total payouts at $600k but allow matching up to $600k for projects with greater than $8m in active cover; for projects with active cover between $2m and $8m, the matching bounty will be capped at $200k per bounty payout.

4. Matching ratio will adjust from $1 in matching for every $1 offered as a critical bug bounty (1:1) to $0.50 in matching for every $1 offered as a critical bug bounty (0.5:1)–this will create a greater incentive for projects to increase the size of their critical bounty payouts, so long as there is demand for cover on Nexus Mutual.

Matching bug bounty payouts deliver cost effective value to members when the matching payout is less than potential claim payouts on a certain percentage of a project’s active cover amount.

Not all loss events lead to a 100% claim rate. While data on past claim events is thin, members can assume a matching bug bounty is most cost effective when estimated as a percentage of 10% to 40% of the active cover amount for a project.

After taking into consideration Hugh and Mitchell’s respective suggestions, I’m proposing we:

• Cap maximum payouts at $200k (or the remaining funding) for projects with active coverage in the $2m to $8m range; and
• Cap maximum payouts at $600k (or the remaining funding) for projects with greater than $8m in active coverage

A comparison of matching bounty payouts as stated above can be found in the Active Cover (as of 24 March 2022) Google Sheet; the comparison is between Columns M-to-R, which show the requirements above, and Columns E-to-K, which display matching bounties capped at $200k.

The terms above present the most cost effective benefit for members of the mutual; however, it does pose the challenge of communicating which projects are eligible based on the active coverage qualification. We would need to work with Immunefi to determine the best way to display/communicate the projects that are eligible for a matching payout; this will ensure we can provide a greater incentive for whitehats to review code for listed projects.

TL;DR: Proposal to renew would seek $600k in total funding to incentivize whitehat hackers in Immunefi’s community to review code for listed protocols participating in the program. Bug bounties create a marketplace for blockchain security analysts’ attention; the mutual benefits when a matching bounty is less costly than paying 10% to 40% (or greater) of current liabilities (active cover) made possible by exploitable flaws in design/code.

Immunefi Matching Bug Bounty Program Review

Members can read the review in the Immunefi Matching Bug Bounty Program Review | Request for Comment (RFC) forum post.

Hugh’s Comments

@hugh shared his thoughts on the potential cost savings–if members assume a 10%-40% claim rate on active cover (i.e., current liabilities). While data is thin, past loss events give us an indication that not all active cover policies are being utilized, which means the claim rate would likely be less than 100% of active cover on each platform.

Given this, members would benefit by adjusting the program to ensure protocols/platforms have enough active coverage to justify a matching bug bounty payout. Hugh suggested adjusting the requirements for the program as follows:

• Protocols with active cover greater than $2m and an active bug bounty program on Immunefi

• Matching would remain at the 1:1 matching rate, up to a maximum matching bounty of $200k per bounty payout

• Maximum of $600k total in matching payouts (i.e., 3 total matching bounty payouts) before the program is reviewed and assessed for renewal

See the Active Cover (as of 24 March 2022) | Hugh’s Suggestions Google Sheet, which outlines the cost effective rate for protocols with a Matching Bug Bounty Program per Hugh’s comments.

Mitchell’s Comments

@Mitchell_Amador also weighed in and highlighted the benefits of the program for both Immunefi and Nexus Mutual.

He also suggested expanding the program, with the following goals to drive more value for the mutual:

• Encourage projects to launch bug bounty programs on Immunefi. Driving more projects to launch their bug bounty on Immunefi makes them eligible for the program and allows them to benefit from a community of 10,000+ blockchain security analysts, hackers who can review their code for flaws.

• Make projects with significant liabilities eligible for the program. Projects with the largest amount of active cover pose the greatest risk to the mutual. Making projects eligible is a low cost and provides a cost effective use of funds.

• Incentivizing projects to increase their bounties and make payouts more competitive within the market. More projects are trending toward a $1m+ critical bounty standard; adjusting the program to give protocols an incentive to increase their critical bug bounty payout, while ensuring a matching bounty is still cost effective for the mutual works for all parties involved.

Mitchell also suggested changing the matching ratio from 1:1 payouts to 0.5:1 payouts to give projects a greater incentive to offer higher payouts. He also offered the perspective that capping maximum payouts at $200k per bounty doesn’t serve the mutual’s interests.

Instead, he suggested matching 0.5:1 up to the $600k cap on total payouts. For example: paying out $525k for a critical vulnerability disclosure on Anchor Protocol would represent 7.42% of $7,080,225.97 (10% of active cover) and 1.85% of $28,320,903.86 (40% of active cover) respectively.

To quote Mitchell:

As to Hugh’s suggestion on capping the amount before another review, that seems wise to me, although I would suggest removing the cap per critical, since that does not serve the Mutual’s interests in driving disclosures (which is driven by bounty size). Pegging the maximum per critical to the total pool (as we did before) seems wiser, as it helps drive up bounty sizes (which does most of the work in incentivizing hackers).

Matching payouts are already derisked by their necessary review and sign-off by the Nexus Mutual team. If further de-risking is desired, I’d suggest cutting the matching rate (from 1:1 to 0.5:1 or something similar), rather than capping reward size, to encourage projects to assume more cost burden to receive these incentives.

About Immunefi

Immunefi is the leading bug bounty and security services platform for DeFi, which features the world’s largest bounties. Immunefi guards over $100 billion in users’ funds across projects like Nexus Mutual, Chainlink, SushiSwap, PancakeSwap, Bancor, Cream Finance, Compound, Alchemix, Synthetix, and others. The company has paid out the most significant bug bounties in the software industry, amounting to over $10 million, and has pioneered the scaling DeFi bug bounties standard.

More than 300 protocols and products have been launched on Immunefi, as the de facto home of Web3 bug bounty programs, for an aggregated pool of $121m in critical bug bounties. Their community has 10,000+ registered blockchain security analysts and hackers on their platform. In aggregate, Immunefi disclosures prevented more than $23 billion in probable exploits.

Members can review past bug reports, war rooms, and whitehacks on Immunefi’s Medium.

Conclusion

The partnership with Immunefi provides a benefit for both of our communities, while serving our mission to protect more users on-chain. A use of funds from the DAO treasury (a.k.a., the Community Fund) should always deliver value or preserve value for the mutual. Granting $600k from the treasury would strengthen our partnership with a first-in-class blockchain security organization, while creating incentives for blockchain security analysts and hackers to review code for listed projects in order to discover flawed code that presents a critical threat before it can be exploited by a greyhat or blackhat; this preserves value for all NXM holders and active Risk Assessors.

After seven (7) days of review and discussion, this proposal will transition to a Snapshot vote, where the following choices will be presented to members of the mutual:

• Option A: Renew the program; allocate $600k in total payouts; 0.5-to-1 matching payouts up to $600k for projects with active cover >$8m; 0.5-to-1 matching bounty payout up to $200k for projects with active cover between $2m to $8m.

• Option B: Renew the program; allocate $600k in total payouts; 1-to-1 matching payouts up to $200k for projects with active cover >$2m

• Option C: Do not renew the program; deny request for further funding

[Melvin]

I don’t understand why the matching funding is being compared to “10% of liability”.

If cover is being sold for about 1% of liability (taking into account the wNXM discount), then assuming a “10% of liability” loss is flawed by a factor of 10.

Also, if NexusMutual is willing to pay out “up to 10% of liability”, then it is spending 10x premium collected, on the bug bounty.

I don’t see how this is adding value?

[BraveNewDeFi]

Happy to expand on the points above, @Melvin.

If a critical vulnerability exists in one of our listed protocols where members of the mutual are underwriting large amounts of cover, an exploit that leads to a claim event could result in claim payouts that cost significantly more than the proposed matching bounty payouts.

The 10% to 40% of liabilities represents an estimated figure of the total cost of claims, if a loss event were to occur and resulted in payouts for those holding active cover.

The below are several examples:

Enzyme v3

• Active cover noted in the Google Sheet: $55,471,404.70
• 10% of that amount: $5,547,140.47
• 40% of that amount: $22,188,561.88
• Premiums earned at the 1% premium you noted: $554,714.047
• Proposed matching bounty amount: $200,000

Alpha Homora v2

• Active cover noted in the Google Sheet: $21,819,035.83
• 10% of that amount: $2,181,903.58
• 40% of that amount: $8,727,614.33
• Premiums earned at the 1% premium you noted: $218,190.35
• Proposed matching bounty amount: $250,000

SushiSwap

• Active cover noted in the Google Sheet: $12,636,944.59
• 10% of that amount: $1,263,694.459
• 40% of that amount: $5,054,777.83
• Premiums earned at the 1% premium you noted: $126,369.44
• Proposed matching bounty amount: $600,000

Reflexer

• Active cover noted in the Google Sheet: $4,230,769.66
• 10% of that amount: $423,076.966
• 40% of that amount: $1,692,307.86
• Premiums earned at the 1% premium you noted: $42,307.69
• Proposed matching bounty amount: $200,000

For the above examples, all instances of a matching bounty payout at the 10% of active cover amount is more cost effective than if members were to pay claims at 10% of the active cover. Not all examples net the mutual a profit when the matching bounty is compared to premiums, but the important note is that this program minimizes capital outflows by incentivising whitehats to review code for flaws to prevent loss events from happening.

The matching bounty program with Immunefi is a preventative measure to catch flaws before they can be exploited and retain value in the capital pool that would otherwise be used for claim payments in the event an exploit were to occur.

[Melvin]

Thank you for the fast update .

@BraveNewDeFi

Not all examples net the mutual a profit when the matching bounty is compared to premiums

So in these 4 examples, you agree that it would not make sense to bounty-match on Reflexer and Homora. (if it does make sense because it reduces risk, then it is implying the premium collected to reduce that risk was too low).

In the other 2 examples, I still question the rationale. It is not as if the whitehats will uncover 100% of the flaws to reduce the risk to 0%. (and if this were true, NexusMutual would not be required).
Thus a bounty of $200,000 to uncover only some of the risks, may still not be worthwhile versus $554,000 in premium collected. Will the bugs discovered really reduce the risks during that period by over one third?

[BraveNewDeFi]

I summarized @Hugh and @Mitchell_Amador’s comments from the Request for Comment version of this proposal, and after taking their comments into review, I created this proposal.

Don’t agree with you on the points you’ve made in the first graph. We’re assuming three main points when we’re talking about risk:

1. There may be flawed code and/or critical vulnerabilities in listed protocols
2. There may be claim payouts if an exploit were to occur (dependent on claims filing, cause of exploit)
3. Members will continue to buy cover from the mutual

In your analysis, I’m taking away an “all things held equal” approach in your review of the data. The Google Sheet analysis I provided was to give insight into the total value of a matching payout when compared to payouts according to 10%, 40%, and 100% of active cover.

The program allows the mutual to incentivise greater review of listed protocol code, but it doesn’t necessarily mean there will be a matching bounty payout within the near term. It’s a preventative measure to catch exploitable flaws before they can lead to a loss event.

And I do agree that risk can never be brought to zero, in any market or area in life. Risk will always remain but programs like the one we’ve started with Immunefi can minimize the mutual’s exposure to risk and strengthen relationships with listed protocols. I do see this program as a net benefit to the mutual.

As far as quantifying the impact of potential risks: the program offers matching bounty payouts for critical vulnerability disclosures, which are the sort of flaws that lead to major loss events. In theory, yes, it would significantly reduce the mutual’s exposure to risk but it cannot reduce it to zero.

Either a whitehat catches a critical vuln. before it’s exploited or a blackhat does and exploits the flaw. If the bounty is large enough, grayhats are more likely to report than exploit. I’ll defer to @Mitchell_Amador on the instance of critical vuln. disclosures though Immunefi.

For a non-hypothetical example: let’s take the Yearn critical vuln. disclosure, where we paid out $200k to the whitehat who reported the flaw.

• $200k in matching bounty payout, brought the total payout to $400k when Yearn’s original $200k is taken into account
• 2021 Historical Premiums across Yearn cover products: $936,330.11
• Matching bounty payout equivalent to 21.36% of 2021 Yearn premiums

Paying out 10% of current liabilities on Yearn (all vaults) Protocol Cover would cost $868,670.60, or 92.77% of 2021 historical premiums earned for Yearn Finance.

How do you think we could structure the program to be beneficial for the mutual? Or do you take the overall view that this program is not worth funding?

[Hugh]

@Melvin

So in these 4 examples, you agree that it would not make sense to bounty-match on Reflexer and Homora. (if it does make sense because it reduces risk, then it is implying the premium collected to reduce that risk was too low).

This isn’t the correct comparison to make.

Premiums charged is irrelevant to answer this question as we should be comparing:

Probability of Hack without bounty x Expected Payout - Probability of Hack with Bounty x Expected Payout VS Probability of Bounty Payout x Bounty Amount

We don’t really know a lot of the terms here so we have to make some guesses. But the lower the absolute value of the payouts (directly linked to cover exposed), the less beneficial the program. So it makes sense to exclude protocols with relatively low cover amounts. What should the threshold be? That’s hard to work out, but some threshold makes sense, and it’s irrelevant what the premium is charged.

Note: Premium price should be linked to probability of hack, but premium sufficiency (or not) is a separate issue.

[schorgie30]

Thanks for the comprehensive explanation on the proposal here @BraveNewDeFi! Another big benefit here is the non-financial benefit this program could have if it does in fact help catch flaws before they are exploited. It’s a net positive in regards to Nexus positioning if less major claims are paid out I would think - I can see some counter arguments to that but think it’s directionally correct.

[Mitchell_Amador]

Thank you for preparing such a comprehensive review @BraveNewDeFi, the Nexus Mutual community continues to impress with its professionalism.

Option A seems like the superior route, as it better incentivizes pro-security behavior, which reduces risk of claim events. Both options are good, but Option A will be much more effective in achieving its aim.