Improving Claims Assessment and Policy Wording

[Chris_Ease]

After having dealt with the claims process ourselves, our team at Ease has some feedback on how it could be improved for the end user.

Summary of Events:

We recently submitted a claim regarding loss from an Ease product. We wrote up a thorough summary of what occurred, announced it publicly, calculated exactly what the cost would be to reimburse users, and submitted our Nexus claim.

Hugh was initially the only one to respond and he mentioned his views on the situation. We responded to his views to facilitate open discussion on our claim, and then discussion was silent, with 100% of the vote currently in favor of approval.

On Friday, March 24th 6:33 PM UTC, an anonymous individual funded a wallet through Binance with wNXM and immediately staked 10,000 against claim ID v2#9, for Ease protocol cover. (Source). Why we find this concerning:

1. They chose to vote at the end of the work week at the very end of the initial voting period.
2. This user provided no contributions into the discussion of this claim. It’s surprising for a large holder, who has never voted on a claim before, to vote ‘no’ on a relatively minor claim, with a very large stake and without any stated reasoning behind it.

Improving Claims Assessment

After this experience from the claimant side, we believe there are some improvements to be made for the sake of transparency. Here are our current thoughts

1. Require reasoning on assessment vote:

Claims discussion is an important step in the assessment process. There is a large burden on the policyholder to provide evidence to support their claim submission. There should be a similar burden on the assessor to provide reasoning when they vote on a claim. Require all assessors to provide reasoning behind their approval or denial when submitting an on-chain vote. Assessors including direct references to policy wording in their reason would be strongly encouraged.

This would accomplish a few things. First, it would allow a claimant to either, be able to give a counter argument, or shore up any record keeping issues for their 2nd claim submission. Second, it would also allow assessors to acknowledge when they utilize discretion for a claim that might not be 100% valid according to the policy wording. And third, it could make it easier for the advisory board to identify fraudulent votes if a voter inputs clear dummy information.

2. Limit of voting power:

This one is pretty straightforward. No one user should have too much influence on the process. There still should be some level of prevention so the assessment process requires multiple users to vote. One idea would be to require X times the USD value of the claim in votes on the policy for it to pass. And then, users can only vote an amount equal to Y, which is a fraction of the total required vote.

One example: A policy claim for $100K requires $400k of NXM in total votes, but no one can vote more than $100k of NXM.

3. Clearer UI for Assessment Votes:

Showing values of Yay/Nay votes rather than percentages for increased transparency. Also a UI that shows voting history, what address voted when, how much, link to txn hash, etc.

A timer showing the cool down period countdown after voting ends would be nice too, in addition to detailed statuses (approved/denied; claimed, with tx id).

4. Separate Assessor and Underwriter wallets

For the sake of reducing conflict of interest it would be a good step to have any wallet that is underwriting unable to stake on a vote for claims assessment and vice-versa.

For example, if a user has 100k NXM total, and underwrites with 50k of it, and then uses the remaining 50k NXM to stake for voting, a clear conflict of interest is created.

Improving Policy Wording

1. A transparent list of what code is covered by protocol cover:

A source of division among us and assessors was whether arNXM was covered by an Ease claim, as arNXM was originally an armor brand. However, we rebranded all Armor products under Ease branding 27 days before Ease was even listed as a covered protocol. So our belief was it fell under the definition of Designated protocol (see below). It might be best for claims moving forward if there is less ambiguous wording about what code falls under the scope of a protocol’s cover policies.

2. Stricter Definition for “loss of funds”:

Another division was whether the loss of wNXM backing value for our claim constituted loss of funds. We believed it did, as a user withdrawing wNXM had an irrecoverable loss of .024 wNXM for each arNXM withdrawn post-exploit. But the argument against it was “no value was taken from the protocol due to an excess mint of tokens”. The below wording should be amended in some way if reduction of backing value isn’t supposed to fall under policy wording.

These were some of the thoughts our team had after dealing with the claims process ourselves. We would love to hear feedback, as I’m sure Nexus best knows what is possible to be realistically implemented into the system.

[BraveNewDeFi]

Hey @Chris_Ease! Thank you for sharing the Ease team’s experience during their recent claim submission.

Claims Assessment

You make a lot of great points, and I agree that the Claims Assessment UI can be improved to provide greater transparency and to give claim filers more information. On your points:

@Chris_Ease

Require reasoning on assessment vote:

Every member who assesses a claim actually does provide their rationale when they submit responses to the following questions.

Once an assessor fills this out and proceeds to cast their vote, this information is captured and stored on IPFS. You can see the the IPFS hash for the assessment in question you referred to above.

I’ve shared my feedback on how to improve the Claims Assessment UX with the engineering team, which is based on your feedback and my view that the current design is geared toward assessors and not for the benefit of claims filers.

Proposed Changes

On the Assessment page, there would be more granular detail on each vote that is cast, where anyone can see:

1. The assessors address, which can be used to view any potential staking exposure and potential conflicts of interest.
2. The assessor’s voting power, so anyone can understand how much voting power has been cast.
3. Rationale, which would link to the IPFS hash with the assessor’s reasoning.
4. The result of their vote: Approve or Deny

All of the information is captured on-chain, but it would improve transparency and give the person who filed the claim (and anyone else) the ability to understand why people have or have not voted to approve a claim. This would also provide a clear view if obvious fraud has occurred.

I think this addresses points 1 and 3 in your post.

@Chris_Ease

Limit of voting power:

I’m not sure that limiting voting power would benefit the claims process. If fraud has occurred, the Advisory Board can step in, burn a malicious assessor’s stake, and reverse the claim vote. There is protection against fraud within the system. If someone comes in and casts a large vote to deny, the vote is extended by up to 24 hours, so no one can come in at the end of an assessment and vote to deny. This design choice coupled with the Advisory Board’s power prevents fraud from impacting legitimate claims.

In past loss events, we’ve seen an active group, albiet a smaller group, participate in the assessment process. Sometimes assessors with a large amount of voting power vote on claims and push them to approval when an actual loss has occurred. This cuts both ways, as lower participation and limiting voting power can make it easier for someone to sway a vote with less voting power.

Personally, I believe the current claims process safeguards against fraud, but I’ll defer to other members on this point.

@Chris_Ease

Separate Assessor and Underwriter wallets

I don’t think this solves the issue either, as someone could have two separate wallets with their NXM balance split across the two, and it would be effectively more difficult to determine if someone has a conflict of interest in a vote.

No one can vote in claims assessment with capital that is being used to underwrite cover, so each member is required to stake separate NXM as a claims assessor to participate. If the assessor’s address is displayed in the UI and anyone can check to see if they have underwriting exposure to the calim in question, this discourages fraudulent behavior and also makes it easier to detect fraud.

And someone still has to risk losing their NXM if they vote to deny a legitimate claim or if they vote to push through an illegitimate claim. Members can only have one (1) whitelisted membership address, and that design makes it easier to detect conflicts of interest.

We’ve also seen people participate in assessment and act honestly, even when the member voting to approve a claim has NXM staked against the protocol or custodian that suffered a loss. I am one of those people: I had NXM staked against CREAM when the exploit happened in October 2021. I voted to approve several claims then, even though it resulted in some of my NXM being burned.

This is true for other members, as well, who have contributed honestly even though it resulted in some of the NXM being burnt. I don’t think we should discourage honest actors from participating in the assessment process. Instead, each address puts their reputation in the mutual at risk, as well as the NXM they stake to act as an assessor.

@Chris_Ease

On Friday, March 24th 6:33 PM UTC, an anonymous individual funded a wallet through Binance with wNXM and immediately staked 10,000 against claim ID v2#9, for Ease protocol cover. (Source). Why we find this concerning:

1. They chose to vote at the end of the work week at the very end of the initial voting period.
2. This user provided no contributions into the discussion of this claim. It’s surprising for a large holder, who has never voted on a claim before, to vote ‘no’ on a relatively minor claim, with a very large stake and without any stated reasoning behind it.

On this point: on-chain participation happens 24/7. No one can control when others decide to vote on claims, and no one can force someone to post their rationale on Discord. However, the rationale stored on IPFS can be added to the UI.

For anyone who isn’t familiar with the claims assessment process, you can refer to the entry in the Nexus Mutual documentation.

Cover Wording

I definitely agree that Protocol Cover needs to be revised. I have a variety of notes on potential changes to Protocol Cover, and once I’ve collected them, I’ll share my thoughts on the forum as well.

As you know, any changes to Protocol Cover wording would require an RFC then a formal Nexus Mutual Protocol Improvement Proposal, which would be subject to an on-chain vote before any changes can be implemented.

Again, you raise a lot of valid points, and I look forward to improving the assessment UI to improve transparency and make it easier for claims filers to understand the reasoning behind asssessors’ votes.

[Chris_Ease]

Thank you for such a thorough reply!

@BraveNewDeFi

• The assessors address, which can be used to view any potential staking exposure and potential conflicts of interest.
• The assessor’s voting power, so anyone can understand how much voting power has been cast.
• Rationale, which would link to the IPFS hash with the assessor’s reasoning.
• The result of their vote: Approve or Deny

I agree, these suggested UI changes would help immensely for points 1 and 3! Specifically, the IPFS hash was something I was not aware about. Having direct access to all this info in the interface would be very beneficial for assessor voting transparency.

One suggestion I would add: Rather than only a link to the IPFS hash, provide the full text questions with the assessor’s answers to reduce the chance of miscommunication. My reason being: the way the IPFS hash has the questions worded, vs the wording the assessor sees in the claims process are quite different.

The image above shows the questions that an assessor answers on Protocol Cover claims (Also of note: I couldn’t find any documentation about the specific questions asked to the assessor, luckily we had an image of them taken from when our claim was live).

Question 1, the assessor is asked “Has the protocol lost or made user’s fund irrecoverable while the cover was active”?

But in the IPFS hash it uses the wording “occuredDuringCoverPeriod”.

Using our claim as an example, there was debate that whether the excess minting of tokens counted as “irrecoverable funds”. So in this case, the assessor said “No”, the funds were not made irrecoverable while cover was active.

Using only info derived from the IPFS hash, it could be interpreted as “no event occurred during the cover period”. Without access to the full questions, a claimant could interpret the assessor as not voting in good faith. Because, from the claimants PoV they clearly provided proof of an event occurring, but the assessor is ignoring it. However, from the assessors PoV, they are only saying “No” to the funds being made irrecoverable, not to whether an event occurred.

Regarding points 2 and 4

@BraveNewDeFi

There is protection against fraud within the system. If someone comes in and casts a large vote to deny, the vote is extended by up to 24 hours, so no one can come in at the end of an assessment and vote to deny. This design choice coupled with the Advisory Board’s power prevents fraud from impacting legitimate claims

@BraveNewDeFi

Instead, each address puts their reputation in the mutual at risk, as well as the NXM they stake to act as an assessor.

I think if there was more transparency of what an investigation into fraudulent voting looks like I could be swayed to your view. Currently, the documentation outlines the process as such:

While the idea is sound in theory. In practice, this description leaves a lot of questions unanswered:

1. What triggers a review by the Advisory board? Is the responsibility on the claimant to call for a review, or is each vote already reviewed for possible fraud by the board or another Nexus team member?
2. Can the assessor defend their claim if accused of fraud? What does that process look like.
3. Are the advisory board’s finding’s published to the discord or forums when they find evidence of fraud?
4. The cooldown period is only 24 hours, does that mean the Advisory board only has that window to investigate and submit the merkle-tree root hash? Given 8 hours of that might be sleep, or if the cooldown begins on the weekend, the board might not be available for enough of that time to conduct a proper review.
5. What if evidence of fraud is discovered after the cooldown period? Can the claimant appeal (if they already used both their claim submissions)? Can the fraudster still be punished?

I believe if the fraud investigation process was more defined, I would find it less necessary to have any voting power or wallet limitations in the system.

[BraveNewDeFi]

@Chris_Ease

One suggestion I would add: Rather than only a link to the IPFS hash, provide the full text questions with the assessor’s answers to reduce the chance of miscommunication. My reason being: the way the IPFS hash has the questions worded, vs the wording the assessor sees in the claims process are quite different.

This can definitely be done, and I can pass this along to the engineering team, so they can include this in the UI that anyone can see per the changes I outlined in my response.

@Chris_Ease

What triggers a review by the Advisory board? Is the responsibility on the claimant to call for a review, or is each vote already reviewed for possible fraud by the board or another Nexus team member?

Whenever there’s a claim event, I always prepare a Claims Tracker with the eligible covers that may be filed, where the qualifying factor is a cover being active when a loss occurs. I also create posts in Discord and ensure everyone in the community–all members and the team–are aware whenever a new claim is filed. And I am always monitoring active claims, as for the vast majority of claims, the members who filed DM me or ask on Discord for regular updates. I do my best to keep people up to date, and I know there are an active group of members that keep a close eye on each claim that gets filed, including the other teams within the DAO and with the Foundation.

With the way the claims assessment process is designed, there are a few important things to note:

@Chris_Ease

What triggers a review by the Advisory board? Is the responsibility on the claimant to call for a review, or is each vote already reviewed for possible fraud by the board or another Nexus team member?

The cool-down period is only 24-hours, yes, but this is after a vote closes. The minimum time a claim is open for review is three (3) days. If someone comes along near the end of a vote, like the assessor you pointed out did, the voting time gets extended by up to 24 hours. Each subsequent vote will add more time to the voting period. This prevents someone from coming in last minute and swaying a vote one way or the other against the leading consensus right before a vote closes.

From the docs:

There is a 24-hour silent period, where no votes should be casted before an assessment vote closes.

If a vote is submitted during the last 24 hours of the vote, the voting period is extended with an amount of time proportional to the voter’s stake, with 24 hours representing the maximum time increase.

This design feature prevents “rush attacks,” where someone tries to overturn a claim outcome by submitting a vote at the last minute that moves the majority outcome with no time to appeal the vote.

This means that there’s more than 24 hours to identify and determine that fraud has occurred. It’s between 2-3 days, if not more depending on voting activity, when you add the cool-down time and the remaining time on a claim vote.

If it’s clear that someone is denying a legitimate claim or trying to push through an illegitimate claim, the Advisory Board will take action. This happened previously for a Kraken claim, where the member who filed the claim also voted to approve the claim when no loss had occured on Kraken. See a screenshot of the announcement, where the AB voted to burn this assessors stake for voting fraudulently.

@Chris_Ease

Can the assessor defend their claim if accused of fraud? What does that process look like.

The Advisory Board has the power to burn assessor’s stakes if there’s reason to believe they voted fraudulently. The AB can ask for feedback from members, but this decision is solely in the hands of the AB.

Someone who votes fraudulently could make their case on Discord or reach out to the AB, but if someone votes to deny, provides no contribution to the discussion, and their rationale clearly conflicts with the facts of the loss event, the AB still has the power to burn their stake.

@Chris_Ease

Are the advisory board’s finding’s published to the discord or forums when they find evidence of fraud?

In the past when this happened, the AB did ask for members’ feedback before escalating this to an AB vote, so yes, whenever fraud is detected, the AB will make an announcement before taking action.

@Chris_Ease

The cooldown period is only 24 hours, does that mean the Advisory board only has that window to investigate and submit the merkle-tree root hash? Given 8 hours of that might be sleep, or if the cooldown begins on the weekend, the board might not be available for enough of that time to conduct a proper review.

As I shared in the comment above, there’s more time for the AB to act. In reality, the AB needs to act within that 2-3 day window if a vote outcome needs to be reversed and the assessor’s stake burned.

However, if someone votes fraudulently but other members step in and vote to approve with enough voting weight to defeat the fraudulent assessor, then there’s a longer timeframe to take action.

As it states in the docs:

Claims assessment stakes are locked for 90 days after a member’s last vote was cast.

A claims assessor has to lock their NXM for 90 days, so a decision can be made within that timeframe if fraud is detected and the AB votes to burn that assessor’s stake.

@Chris_Ease

What if evidence of fraud is discovered after the cooldown period? Can the claimant appeal (if they already used both their claim submissions)? Can the fraudster still be punished?

A member can always file another claim. Previously, there was a two-claim limit in place, but with V2, members can file as many claims as they like, so long as they provide an ETH deposit as part of the claims filing process. From the docs:

Claim deposits are required to prevent people from spamming the claims process with multiple claims, as there is no limit to the number of times a claim can be submitted. As long as a cover holder provides a claim deposit, they can file a claim as many times as preferred.

If a claim is denied when it should be approved, the claimant can always file another claim. Just the same, an assessor who commits fraud can be punished within the 90-day window while their staked NXM is locked in claims assesssment.

All good questions, @Chris_Ease. Happy to answer any other questions you have, as well.