Immunefi Matching Bug Bounty Program Review | Request for Comment (RFC)

[BraveNewDeFi]

Overview

In July 2021, Immunefi submitted a Community Fund grant application to request funding and create a bug bounty matching program to incentivize whitehat disclosures and prevent claimable events. Below is an excerpt from the proposal:

Immunefi, DeFi’s leading bug bounty service protecting over 100+ DeFi projects and $25 billion in user funds, proposes a bug bounty matching program to prevent claim events on Nexus Mutual covered partners, thereby directly strengthening Nexus Mutual core business.

Immunefi will handle all operations and logistics (bug bounty program setup, quality monitoring, project support, and payments), with Nexus Mutual’s role being to approve and provide matching payouts to qualifying critical bug reports as they come in.

The Nexus Mutant community voted in favor of allocating up to 2500 NXM to a rewards pool for the purpose of providing 1:1 matching payouts, with a maximum of $200,000 per valid critical bug disclosed by whitehats through Immunefi. The program launched in September 2021, and the first matching bug bounty was recently paid out to an anonymous whitehat who disclosed a critical vulnerability within Yearn’s smart contract system. Members can read Immunefi’s announcement of the $200,000 matching bounty the mutual paid out to the whitehat for more information.

Now that the mutual has matched a $200,000 bounty for a critical vulnerability (“critical vuln.”) disclosure, members need to review the program, determine if it should continue and receive additional funding, and if any adjustments should be made to the terms of the program.

About Immunefi

Immunefi is the leading bug bounty and security services platform for DeFi, which features the world’s largest bounties. Immunefi guards over $100 billion in users’ funds across projects like Nexus Mutual, Chainlink, SushiSwap, PancakeSwap, Bancor, Cream Finance, Compound, Alchemix, Synthetix, and others. The company has paid out the most significant bug bounties in the software industry, amounting to over $10 million, and has pioneered the scaling DeFi bug bounties standard.

Members can review past bug reports, war rooms, and whitehacks on Immunefi’s Medium.

Benefits of the Program

Using the Yearn critical vuln. as an example:

Existing Bounty: $200,000

Matching Bounty: $200,000

Increase to Incentive: 100%

Yearn Active Cover: 11,988,443.41 DAI | 4,706.17 ETH
* Includes Yield Token Cover products + Yearn Protocol Cover policies

In this instance, the matching bounty represents 1.67% of existing liabilities. If we were to only include active cover for Yearn Protocol Cover (8,807,214.84 DAI), then the matching bounty represents 2.27% of existing liabilities.

Providing a matching bounty to incentivize a disclosure is between 97.73% to 98.33% more cost effective than paying all claims for Yearn Protocol Cover at the moment.

Claims prevention can deliver a tremendous amount of value for members, while furthering our goal to protect more users in DeFi.

This program also strengthens the value proposition of being listed on Nexus Mutual. By virtue of being listed, protocols receive increased security protection through the Immunefi Matching Bug Bounty Program.

Initial Protocols, Terms of Program

Below are the initial protocols selected for the Bug Bounty Matching Program:

• Alpha Finance | Critical Vulnerability Payout of $750,000
• BadgerDAO | Critical Vulnerability Payout of $750,000
• Bancor | Critical Vulnerability Payout of $100,000
• Compound | Critical Vulnerability Payout of $50,000
• Pool Together | Critical Vulnerability Payout of $25,000
• Sushiswap | Critical Vulnerability Payout of $1,250,000
• Synthetix | Critical Vulnerability Payout of $200,000
• Vesper Finance | Critical Vulnerability Payout of $200,000
• Yearn Finance | Critical Vulnerability Payout of $200,000

Immunefi outlines the process of this program in their announcement.

The way the program works is a straightforward, two-part process:

1. Any successful critical bug report (per Immunefi criteria) on an approved project is subsequently reviewed by the Nexus core team

2. If exploitation of the critical vulnerability would have resulted in a payout, the Nexus core team agrees to provide a 1:1 matching payout up to $200,000

Review of Protocols, Terms of Program

If members of the mutual would like to renew the program and continue funding, then the community would need to discuss the following:

• How much funding should be allocated to the matching bounty program?
  • Initial allocation of up to 2500 NXM ($200k)
• Should any of the protocols selected for the Matching Bug Bounty Program be changed?

I’ve included a Google Sheets with a breakdown of Platform/Product liabilities (i.e., active cover), liabilities vs. potential matching payout, liabilities vs. max. bounty payout, etc.

• Immunefi Matching Bounty Analysis

See the legend here, which includes my classifications noted by color in the Google Sheets breakdown.

Any protocol noted as “green” has a ratio that indicates a matching payout would be more cost effective than paying out all active cover policies in the event of a major loss. Worth considering as well: a protocol like THORChain would require the mutual to pay out ~75% of active covers to be cost effective. Protocols I believe to have unfavorable liabilities vs. potential matching payout ratios are denoted as “red,” as included in legend above.

The two protocols included in existing program that are no longer cost effective are:

• Synthetix
• PoolTogether V3

I would suggest that these protocols are replaced with other protocols; any two (or more) protocols denoted with green would be good candidates for the program, based on my analysis.

Call for Comments and Review

Before creating a formal proposal, I’d like to get feedback from members of the mutual regarding the matching bug bounty program.

I am in support of continuing the program and allocating funding for it, but I want to open this review up for wider comment/discussion on exact terms (e.g., total funding, matching amount, protocols added/removed, etc.).

Please review, comment below, and share your thoughts!

Members can signal their support/non-support using the poll below.

• Yes, we should continue the Immunefi Matching Program
• No, we should not continue the Immunefi Matching Program

[schorgie30]

This program makes a lot of sense and would love to see Notional included! A win for the protocol that receives additional security via increased bounty payouts as well as members of the mutual as they opt for smaller bounty payouts rather than larger payouts if there is an exploit.

[Hugh]

Thanks @BraveNewDeFi for bringing this up and the analysis.

I’d just like to make one point on the cost savings. We can’t really assume a critical bug will lead to paying out the full coverage exposure Nexus has at the time. For example, the Yearn bug applied to one pool (or a subset of pools) and not the full protocol. That means anyone who bought coverage for assets in a different pool wouldn’t suffer a loss, and therefore a claim wouldn’t be paid. There are also instances where a critical bug doesn’t lead to complete financial loss, only partial, so therefore claim payments are also lower than the full coverage exposure.

So we have to make assumptions about claim payment ratios given a critical bug exists. This analysis is very hard to do right now as there isn’t really enough data, but I would use a rough rule of thumb around the 25% mark. I don’t have much to justify this, but it feels like it should be less than 50% based on the hacks we’ve seen so far (very few are full losses like CREAM), but it’s likely still material. I’d guess the true answer is in the 10% - 40% range.

In addition we’d like the program to be clearly positive, so we’d expect active coverage to be substantially above $800k if we’re matching $200k per critical.

On this basis I think we should extend the program to the following:

• Protocols that have active cover > $2m and an active bug bounty program on Immunefi
• 1:1 matched payout up to $200k maximum per critical
• a maximum of $600k to be paid out in total, so 3 max payments, before the programme is reviewed again.

Looking forward to hearing others views.

[BraveNewDeFi]

I’ll update the sheet to show the cost savings with a potential 10% to 40% range. These are all good points, @Hugh

Edit: Analysis Google Sheet has been updated. I added a break where Hugh suggested adding the >$2m in coverage, and this matches the view the data bears out.

I agree that adding criteria that allows for more protocols to participate is the best route. I can add this to our weekly review of cover sales in Mutant Meetups, should we proceed with this view and the proposal is approved through governance.

Would be a good weekly review on our social channels to announce our active cover amount and which protocols are eligible for the matching bounty program. Would like to hear @Mitchell_Amador’s thoughts on this approach (i.e., having an active cover cuttoff in lieu of whitelisting protocols for the program).

[nexudm]

Is there any good source of data to indicate the correlation between the size of a bug bounty and reporting vs exploitation? I am a little surprised that Yearns bounty is so small relative to their TVL, Gnosis also.

In cases where the matching amount of the existing bounty is 40% or less, I would imagine the additional funds would make negligible difference to the likelihood of investigation and reporting of bugs and by extension would not result in fewer claims.

On the other hand there is something positive to be said for bug bounty matching as a tool for marketing, though I think Immunefi generally flies under the radar apart from devs and vulnerability analysts.

Perhaps the provision or matching of a bug bounty could be negotiated with protocols directly on an individual basis in exchange for lightly directing their users towards the mutual for cover.

As it stands I believe this is a promising initiative to revisit in the future when it can be scaled up to meaningful amounts but is currently not cost effective with the limited funds available.

[GordonGekko]

This programme is beneficial in that it encourages dialogue with protocols about the size/availability of bounties, and prevention is usually going to be better than cure. In order to qualify for matching bounties protocols/Immunify should be asked to comply with our current risk management standards, whatever they may be. This programme and sponsoring investigative reporting on protocol abuse like Rekt articles are the best form of marketing.

[hETHLedger]

I think this program is definitely worth continuing and should this be brought to a vote, I will be voting yes.

[Mitchell_Amador]

Thanks for the thorough update @BraveNewDeFi! Some further updates on where we’ve gone with Immunefi:

• 300+ protocols and products launched on Immunefi, as the de facto home of Web3 bug bounty programs, for an aggregated pool of $121m in critical bug bounties
• 10,000+ blockchain security analysts and hackers (as registered on our application), making Immunefi the largest security community in crypto by at least an order of magnitude
• In aggregate, Immunefi disclosures prevented more than $23 billion in probable exploits. A pretty superb outcome, validating the efficacy of large bug bounties as the most effective tool per dollar spent in the arsenal of blockchain security teams.

And it’s still early days for us. This year we’re looking at dramatically expanding our hacker community to prevent more hacks before they happen.

But to address @nexudm’s question, the only good source of data is our own report dataset, which is presently private (looking forward to changing that as we execute on our roadmap). But I can answer that question: there is a strong correlation between the size of a bug bounty and disclosure of critical vulnerabilities.

The best whitehat hackers are near uniformly attracted to working on the largest bounties, in hopes of securing a life changing reward. They compare the available bounty against all other bounties, and opt for hacking on projects that they both like and offer the greatest rewards. For whitehats, the TVL itself is less important, since they do not intend to exploit.

So a significant matching boost to bounty programs of covered projects will indeed attract more disclosures, as it will attract more top hacker attention to the exclusion of other programs. The key to driving best results would be to make those boosts such that they make the covered bounty programs competitive relative to the entire market of programs.

Of course, the above does not apply to grey and blackhats; for them large bounty size is by far the most important thing, and is compared against TVL.

As to how to think about expanding the program, there are three clear determinants for driving value to Nexus Mutual:

1. Driving projects to launch bug bounty programs to Immunefi; it’s clear that our security community delivers value in a way nothing else can, but that value is contingent on hosting a program with us. The reality is that our community tends not trust non-Immunefi programs, due to poor treatment endemic to the industry. Driving projects to Immunefi is extremely low cost.
2. Making eligible for matching all protocols with significant liabilities; these are the major risk factors to the mutual, and their coverage is the place to start. Make projects eligible for matching is low cost.
3. Driving their bounties up to the point where they are competitive with the wider market; this will drive whitehat disclosures in a way nothing else will. At this point, strong projects typically have $1m+ bounties. Driving bounties up can be low cost or not, depending on how the matching program is structured.

#1 and #2 are low hanging fruit and easy to do; #3 can also be, if it begins with encouraging projects to boost their bug bounty sizes (which they have an incentive to do anyways), as that costs the Mutual almost nothing. If the Mutual is driving protocols to Immunefi and encouraging them to launch large bounties, most of the heavy lifting financially is done by the projects themselves (as it should be).

I would suggest restructuring the matching program to optimize for persuading projects to launch their own large bounties on Immunefi.

For example, instead of 1:1 matching, the Mutual could suggest matching only after a protocol’s bounty has reached a certain bounty floor, to incentivize making covered protocol’s bounties more competitive with market rates.

As to Hugh’s suggestion on capping the amount before another review, that seems wise to me, although I would suggest removing the cap per critical, since that does not serve the Mutual’s interests in driving disclosures (which is driven by bounty size). Pegging the maximum per critical to the total pool (as we did before) seems wiser, as it helps drive up bounty sizes (which does most of the work in incentivizing hackers).

Matching payouts are already derisked by their necessary review and sign-off by the Nexus Mutual team. If further de-risking is desired, I’d suggest cutting the matching rate (from 1:1 to 0.5:1 or something similar), rather than capping reward size, to encourage projects to assume more cost burden to receive these incentives.

[Hugh]

Thanks @Mitchell_Amador - this is very helpful context.

How about @BraveNewDeFi you and I (and anyone else that would like to) quickly discuss structure and we can come back here with a concrete proposal for further comments before finalising what we should vote on. Seems like the most efficient use of everyone’s time.

[BraveNewDeFi]

Sounds great, @Hugh. I’ll discuss this on tomorrow’s call, as well, so we can let people know how to contribute there.