Game Boy Advance Test Suite

[endrift]

In the interest of furthering the state of Game Boy Advance emulation, I've been writing a test suite that tests various aspects of hardware. At the moment, it only tests timing operations, but way more will be coming in the future. Source code is available on GitHub.


Memory tests
Each test contains 44 configurations. The current version contains 1552 tests total.

Scores:

1. mGBA (0.6 beta 1): 1552 passes
   
2. mGBA (0.5.0): 1552 passes
   
3. higan v103: 1552 passes
   
4. NO$GBA 2.8e (BIOS): 1394
   
5. VBA-M 2.0.0 beta 3: 1375 passes
   
6. VGBA 5.5: 1255 passes
   
7. NO$GBA 2.8e: emulation crashes
   

I/O read tests
The current version contains 123 tests.

1. higan v103: 123 passes
   
2. mGBA (0.6 beta 1): 116 passes
   
3. mGBA (0.5.0): 104 passes
   
4. NO$GBA 2.8e: 100 passes
   
5. VBA-M 2.0.0 beta 3: 100 passes
   
6. VGBA 5.5: 10 passes
   

Timing tests
Each test contains 20 (or 10 if only an ARM version is supported) configurations:

• ROM 4/2 waitstates, no prefetch
  
• ROM 4/2 waitstates, prefetch
  
• ROM 3/2 waitstates, no prefetch
  
• ROM 3/2 waitstates, prefetch
  
• ROM 4/1 waitstates, no prefetch
  
• ROM 4/1 waitstates, prefetch
  
• ROM 3/1 waitstates, no prefetch
  
• ROM 3/1 waitstates, prefetch
  
• WRAM
  
• IWRAM
  

The current version contains 1560 tests total.

Scores:

1. mGBA (master, BIOS): 1474 passes
   
2. mGBA (master): 1434 passes
   
3. higan v103: 1300 passes
   
4. mGBA (0.5.2, BIOS): 1176 passes
   
5. mGBA (0.5.2): 1136 passes
   
6. NO$GBA 2.8e (BIOS): 988 passes
   
7. NO$GBA 2.8e: 968 passes
   
8. VBA-M 2.0.0 beta 3: 678 passes
   
9. VGBA 5.5: 4 passes
   

Timer count-up tests
Each test contains 36 configurations. The current version contains 936 tests total.

Scores

1. mGBA (master): 784 passes
   
2. higan v103: 469 passes
   
3. VBA-M 2.0.0 beta 3 (BIOS): 440 passes
   
4. mGBA (0.5.2): 356 passes
   
5. mGBA (0.6 beta 1): 344 passes
   
6. NO$GBA 2.8e (BIOS): 172 passes
   

Carry instruction tests
Each test contains 3 configurations. The current version contains 93 tests total.

Scores:

1. mGBA (0.5.0): 93 passes
   
2. higan v103: 93 passes
   
3. NO$GBA 2.8b: 93 passes
   
4. VBA-M 2.0.0 beta 3: 93 passes
   
5. VGBA 5.5: 93 passes
   

BIOS math tests
Each test contains 5 cases: the outputs in registers r0 through r3 and the program state flags. Note that for most of these tests, r0 is the only important case, although some games may rely on the other values. The current version contains 625 tests total. These tests only apply to emulators with HLE BIOS.

Scores:

1. mGBA (0.6 beta 1): 625 passes
   
2. VBA-M 2.0.0 beta 3: 425 passes
   
3. NO$GBA 2.8e: emulator crashes
   
4. mGBA (0.5.2): emulator crashes
   
5. VGBA 5.5: emulator crashes
   

DMA tests
The current version contains 1256 tests total.

Scores:

1. mGBA (master): 1232 passes
   
2. mGBA (0.6.1): 1112 passes
   
3. NO$GBA 2.8f: 1060 passes
   
4. VBA-M 2.0.1: 1032 passes
   
5. VGBA 5.5: 998 passes
   

Downloads

• v0-r51
  
• v0-r46
  
• v0-r44
  
• v0-r42
  
• v0-r40
  
• v0-r38
  
• v0-r35
  
• v0-r33
  
• v0-r29
  
• v0-r23
  
• v0-r22
  
• v0-r20
  
• v0-r19
  
• v0-r16
  
• v0-r13
  
• v0-r11
  
• v0-r10
  

[driscoll42]

I've been running some Accuracy tests of Android emulators, so far testing GB/GBC, NES, SNES, and GENS/MD over on reddit which has seen quite a bit of enjoyment by the community there. By far the number one requested console I test next has been GBA. As of yet I've only found ARMWrestler and CPU Test by Deadbody as suitable test ROMs. I didn't consider that enough testing to truly make a good test suite out of but your tests here look extremely promising.

Basically what I'd like to ask you is would you consider your tests plus the two CPU tests above as sufficient to make a general statement about accuracy? Or should I hold off until there are further tests I find/you expand your test suite?

Android GB/GBC Emulator Accuracy Testing Results
Android NES Emulator Accuracy Testing Results
Android SNES Emulator Accuracy Testing Results
Android GENS Emulator Accuracy Testing Results

[endrift]

Oh my god no, this test suite is woefully inadequate at the moment. I hope to improve it, but it's nowhere near done yet. Also, it's worth noting that four of the tests in armwrestler (all of the ldm writeback tests) fail on actual hardware, so that test suite isn't that great. CPU Test by Deadbody is extremely minimal, too.

[driscoll42]

Ah damn, well glad I asked. I'll certainly be keeping an eye on your test suite for whenever you judge it adequete. Thanks for your info on the other tests, I know I'm grasping a bit at straws here but I would love to do some kind of real testing.

[Spikestuff]

Oh my god no, this test suite is woefully inadequate at the moment.

Mappy VM is a great example of this being the case.

That's very cute! :D A bug is causing it to show as if it passed 100%. I looked at the source; no prefetch emulation.

post on twitter and byuu's investigation

[MoochMcGee]

MAME does REALLY bad. Like, worse than most other GBA emulators with a 129/1260

[endrift]

New version out with some memory loading tests. mGBA is top again, as one might come to expect by now.

[Spikestuff]

Strongest Emulator is gone forever.

[endrift]

RIP in peace. There will be a new build coming tonight with a fuckton more tests, and then double that if I end up doing the store tests today too.

[Spikestuff]

Based Mappy

Edit: Q: Can I use r26 to make a list?
A: No.

Edit: Q: Can I use r29 to make a list?
A: No.

Edit: Q: Can I use r33 to make a list?
A: No.

Edit: Q: Can I use r35 to make a list?
A: Yes, you, no wait, no... iGBA