WTF News (Week 19)

Hello and welcome to issue number nine of WTF News, our biweekly status update.

Over the last two weeks, we ran a handful of wallets through their paces, and today we can reveal the results! Spoiler alert, but there were more issues in our tests than in the best scoring wallets.

First, the regular status update!

Call to ActionπŸ”—

CI/Build PipelinesπŸ”—

What good is a testing framework if it isn't used? Get in touch if you want to integrate WTF into your wallet's testing pipeline. We'll even build the automation ourselves!

Testing the TestsπŸ”—

We want to expand our tests for WTF, to catch issues with the framework itself. To that end, if you work on a wallet that works offline (aside from the RPC provider) and want to try out WTF, get in touch with us!

Code ChangesπŸ”—

Bug FixesπŸ”—

Upstream IssuesπŸ”—

Test ReportπŸ”—


Wallet Test Framework – f999503092a10f2b409b5cc733db065c124e56e5

Not a statement of a wallet’s quality or correctness.

Wallet Version Connection Passing Tests Pass % Platforms Browser Notes
Brave Wallet EIP-3085 61 100% Browser Brv 1.57.42
Taho 0.46.1 EIP-3085 61 100% Extension Chr 115.0.5790.170 Had to impersonate
Frame 0.6.7 EIP-3085 60 98% Desktop Ff 102.14.0esr (64-bit) Missing signTransaction
Enkrypt 1.24.0 Manual 59 97% Extension Ff 102.14.0esr (64-bit) Attempted EIP-3085, but popup doesn’t show RPC url
Coinbase 3.30.2 EIP-3085 48 79% Extension Chr 115.0.5790.170 Missing hex prefix in logs
Metamask 10.34.0 EIP-3085 48 79% Extension Ff 102.14.0esr (64-bit)
OKX 2.58.1 EIP-3085 18 30% Extension Chr 115.0.5790.170 Tests stopped running somewhere around getFilterChanges
TokenPocket 1.1.19 EIP-3085 15 25% Extension Chr 115.0.5790.170 Tests stopped running somewhere around getFilterChanges
Argent X 5.7.2 Can’t Connect 0 0% Extension Chr 115.0.5790.170 No window.ethereum
Bitkeep 1.4.8 Can’t Connect 0 0% Extension Chr 115.0.5790.170 Attempted EIP-3085, and Manual, but got system error
Sequence 2.0.4 Can’t Connect 0 0% Extension Chr 115.0.5790.170 Cannot add custom chains
Exodus 23.8.14 Can’t Connect 0 0% Extension Chr 115.0.5790.170 Cannot add custom chains

The astute reader may notice that there are in fact 62 tests in the repository at that commit. Our eth_getProof test is broken, and we are excluding it from this report.


Our intent is not to claim that certain wallets are better than others, and a low score here is as likely to indicate that our tests have problems as it is to indicate bugs in a wallet. We're very interested in improving scores across the board, so we're happy to accept contributions to fix tests and just as happy to help debug.

Wallet SelectionπŸ”—

Very little of this report is scientific/rigorous, and the wallet selection process was no exception. We chose a few wallets we're familiar with, and a small sampling of random wallets from the Find an Ethereum Wallet list.

Because Wallet Test Framework only supports window.ethereum so far, we're limited to wallets that provide it; that pretty much excludes mobile wallets for the time being. Our architecture requires connecting to a custom chain, so wallets that don't support custom chains cannot be tested (labelled with Can't Connect in the table.)

For future reports, get in touch if you're interested in having your results published (or if you'd prefer we didn't mention you.)


Congratulations to Brave and Taho for getting perfect scores! Considering this was our first time trying them out with the Framework (we've been using Frame to develop), this was quite surprising.


This is a contentious endpoint that we've discussed previously. Very few wallets support eth_signTransaction, mostly by choice.

Interestingly, this was Frame's only failure: instead of signing in the wallet, Frame passes these requests through to the RPC provider.


There are only two hard things in Computer Science: cache invalidation and naming things. Phil Karlton

Caching is hard. We've been bitten by it. In a space where you have to pay for every API request, but your users expect a free wallet experience, caching is a necessary evil.

We suspect, but haven't verified for all wallets, that many of the test failures are caused by stale values cached by wallets.

In the case of eth_call, we have verified that MetaMask specifies an old block number, causing a call into a freshly deployed contract to fail. This is unlikely to cause major issues on a real network with many seconds between blocks, but the tests mine blocks much more frequently, triggering the issue. Regardless, we're following up with their team to see what we can do.

Post-merge EndpointsπŸ”—

Some of the newer endpoints like eth_maxPriorityFeePerGas aren't yet supported in all wallets. Expect these tests to start passing as they are implemented.

Stalled TestsπŸ”—

OKX and TokenPocket both failed somewhere around the eth_getFilterChanges suites, and didn't continue to the rest of the tests. It's likely that they would've passed more tests had they continued.

We haven't debugged why the tests stalled. It could be as simple as adding a timeout to WTF, or something more involved. We'll be looking into this before the next report.

Future PlansπŸ”—

We'll continue adding tests, and we'll try to run through this report somewhat regularly. That said, this is entirely a manual process. Once wallets start running WTF as part of their continuous integration processes, we can make some pretty graphs, include more wallets, and make reports more often.

In the meantime, join us on Discord!