Wormhole Incident Report — 02/02/22

On Feb 2, 2022, an attacker exploited a signature verification vulnerability in the Wormhole network to mint 120k Wormhole-wrapped Ether on Solana. These tokens were not backed by Ether deposits on the Ethereum side of the Portal bridge. The attacker then bridged 93,750 of these tokens to Ethereum, withdrawing the unwrapped Ether from the contract.

To prevent further exploits, Wormhole node operators temporarily stopped relaying messages from on-chain contracts, then upgraded the contract to fix the vulnerability. Jump Crypto has recapitalized the contract to ensure that all Wormhole-wrapped Ether on every chain is fully backed. The Wormhole network is back online and fully operational as of 13:29 UTC, Feb 3, 2022. The total duration of the incident was approximately 16 hours.

A $10,000,000 reward is offered for any information leading to the arrest and conviction of those responsible for the hack of Wormhole on February 2, 2022, or the recovery of the stolen assets. The $10,000,000 whitehat offer remains open for the timely return of the funds. You can reach out to bounty@wormholenetwork.com.

Background

Wormhole is a decentralized, cross-chain message passing protocol. It enables applications to send messages from one chain to another. The network is operated by a decentralized group of nineteen Guardians who sign each transmitted message to attest to its authenticity. The protocol uses a multi-party signature system where a message is treated as authentic if ⅔+ of the Guardians have signed it.

Portal is a token bridge constructed on top of the Wormhole network. Portal enables users to deposit funds into a contract on a source chain, then mint a Wormhole-wrapped version of the token on a destination chain. The minting function requires a Wormhole-authenticated message from the source chain contract. This check ensures that Wormhole-wrapped tokens are backed 1:1 by tokens in the source chain contract.

The Guardians are also responsible for governing the Wormhole network. Upgrades to the protocol and contracts require a ⅔+ supermajority vote of Guardians.

Timeline and response

18:24 UTC — An unidentified attacker exploited a vulnerability in the Solana-side Wormhole contract and tricked it into minting 120,000 uncollateralized Wormhole-wrapped ETH (weETH). The attacker then sent 93,750 weETH back to Ethereum, redeeming it for native ETH (1, 2, 3), and swapped the remaining weETH into SOL on Solana.

19:07 UTC — Wormhole network contributors noticed the discrepancy in outstanding funds during a routine check and immediately investigated.

19:10–19:20 UTC — The contributors identified the vulnerability, declared an incident, and established a “war room” conference call. The call included Wormhole contributors from Jump Crypto, the auditing company Neodyme, representatives from major stakeholders, and other trusted external researchers. In addition, open lines of communication were established with the Wormhole Guardians running the network.

19:33 UTC — The contributors alerted the Guardians to the possible exploit and asked them to stop relaying messages to pause any further token transfers across the network.

19:40 UTC — Token transfers paused when a superminority of six nodes stopped relaying (~37% of the voting power on the network). Over the course of the next hour, a supermajority was achieved.

19:40 UTC — Two teams started working in parallel to close the vulnerability focusing on two different approaches. The first approach would upgrade the contract using the in-protocol governance mechanism. The second approach would be to exploit the same vulnerability as the attacker to complete the upgrade, fixing the vulnerability.

19:55 UTC — The bug was conclusively verified by internal peer review and multiple third-party auditing companies. The fix was committed to a temporary private fork of the Wormhole repository (it is now published to the public repository). The team circulated a contract upgrade governance proposal for Guardians to vote on. To expedite the process, simulations and testing of the fix were conducted in parallel. Since the network was down, the votes could not be immediately submitted. Therefore, a workaround was tested that would allow Guardians to submit governance votes without enabling token transfers.

20:14 UTC — The plan was agreed to by the Guardians on how to safely bring their nodes back online and submit their governance votes.

20:15 UTC — An attempt was made to communicate with the attacker, offering a whitehat agreement of $10million for returning the stolen funds.

The hacker has not responded at the time of writing.

20:42 UTC — The Wormhole Twitter account alerted the community of the exploit via a tweet and informed partners and other projects. Further status updates were provided as the situation evolved.

22:07 UTC — The Neodyme team, which had been working on the fallback plan to upgrade using the exploit, succeeded in building and testing a working prototype. The contributors decided to move forward with this plan because it would likely be the fastest solution to close the vulnerability.

22:25 UTC — A second tweet was sent by the Wormhole Twitter account informing users that 120k ETH had been stolen, but the funds would be replenished. Additional communications were sent to partners and projects.

00:32 UTC — In close coordination with the other teams, and after further improvements, Neodyme successfully ran the upgrade to close the vulnerability.

00:32 UTC — The vulnerability was fixed. At this point, the attacker could no longer create new fraudulent Wormhole messages. However, as part of the original exploit, the attacker had already prepared several additional Wormhole messages that could still be used to steal further funds. The team continued working to address this problem, and maintained frequent communications with Wormhole network community members.

03:42 UTC The team introduced safeguards for the Solana-side Portal (token bridge) to blacklist the fraudulent Wormhole messages to prevent the attacker from stealing additional funds. Developing this fix required extensive testing. The upgrade governance process was proposed to apply this fix.

05:53 UTC — Governance reached ⅔+ consensus and the upgrade was executed on Solana. Ongoing verification and chain forensics work by multiple teams were done to verify the upgrade.

11:27 UTC — Consensus was reached on starting the network back up.

13:08 UTC — Jump Crypto replenished the contract with 120k ETH, restoring full collateral.

13:29 UTC — The Wormhole network and its Portal bridge were brought back online.

13:39 UTC — A tweet was sent from the official Wormhole account to inform the community that the network was back online.

18:13 UTC — Jump Crypto confirmed that it provided the Ether to refill the contract.

Vulnerability

The root cause of the exploit was a bug in the signature verification code of the core Wormhole contract on Solana. This bug allowed the attacker to forge a message from the Guardians to mint Wormhole-wrapped Ether.

On Solana, Portal’s token minting program uses the verify_signatures function to validate the source chain message before minting Wormhole-wrapped tokens. This process relies on information contained in the instruction sysvar account, which is trusted because it is populated by the Solana runtime.

Users can specify arbitrary input accounts when calling a function on Solana. Each program is responsible for validating that the provided accounts are the ones they expected. The issue in the exploitable version was that the verify_signatures function did not assert that the user provided account was the special instruction sysvar.

An attacker could craft an account and populate it with data to make it look like the instruction sysvar account. This fake instruction sysvar could then be passed to Wormhole’s verify_signatures function to fool it into thinking that the signatures had been successfully verified. Any arbitrary Wormhole message with Solana as the destination chain could be signed by an attacker, including messages to mint wrapped Wormhole tokens on Solana.

The vulnerability was fixed by adding the missing check:

Addressing rumors

In the aftermath of the attack, rumors surfaced on social media that Wormhole contributors had been aware of the vulnerability weeks prior to the attack. These speculations were sparked by a recent commit in the Wormhole repository.

The commit was a routine upgrade to the latest version of the Solana toolchain (1.9.4). This version of the toolchain deprecated the load_current_index and load_instruction_at methods in favor of checked variants. These checked variants include the missing check of the instruction sysvar account that was the source of the vulnerability.

While the commit did fix the vulnerability, it was a coincidental byproduct of the toolchain upgrade. While obvious in hindsight, the security implications were not apparent at the time. The commit was pushed to the public git repository on 13 Jan, 2022, where it was set to be audited before the next release. The attacker may have noticed the vulnerability because this commit was public, or they may have felt forced to execute the attack once the fix was merged.

During the incident, the team developed a fix on top of the most recent contract release, predating the toolchain upgrade. This approach minimized the surface area of the change. The deployed fix performs the same check on the instruction sysvar account, but was developed independently. The connection between the fix and the toolchain upgrade only became clear in the aftermath of the exploit.

Looking forward

The Wormhole community remains committed to the security of the bridge and all participants.

Audits

Before Wormhole launched in Summer 2021, its code was reviewed by Neodyme, one of the most accomplished Solana auditing firms, with the report published in January 2022. Subsequently, three firms were scheduled before the incident to perform comprehensive and ongoing audits of the Wormhole code base:

  • Kudelski kicked off an audit in the 3rd week of January 2022.
  • Neodyme has recently been contracted to perform ongoing audits to secure Wormhole’s roadmap and network additions.
  • Trail of Bits has been contracted to perform two audits in 2022.

Further auditing engagements are expected to be announced soon.

Security Roadmap

There are several items on our roadmap which further strengthen the security of cross-chain messaging and bridging, including

  • Accounting mechanism to isolate risks to individual chains
  • Dynamic risk management
  • Ongoing monitoring and early detection of incidents.

Bug Bounty on Immunefi

In December 2021, the community began the process of launching a formal bug bounty program on Immunefi. The program will launch in mid-February and feature a maximum bounty of $3.5M — believed to be the highest in the industry.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Wormhole

Cross-chain interoperability protocol connecting high value blockchains