Tests a wide variety of N64 features, from common to hardware quirks. Written in Rust. Executes quickly.

Overview

n64-systemtest

Tests a wide variety of N64 features, from common to hardware quirks. Written in Rust. Executes quickly.

n64-systemtest is a test rom that is useful for emulator developers or anyone who is interested in understanding how the N64 really works. Design goals of this test rom:

  1. Productivity 1: n64-systemtest itself decides whether it failed or succeeded. No need to compare images,
  2. Productivity 2: Writing new tests is quick and easy,
  3. Readability: Tests should be easy to understand and provide good error messages that make it clear what's broken,
  4. Speed: Everything should run quickly so that the test rom can be used for regression tests,
  5. Open source Rust: Everything that is used to produce the final rom is open-source, written in Rust.

Status

n64-systemtest tests common but also some of the more exotic features of the MIPS CPU:

  • MFC0/DMFC0/MTC0/DMTC0: Some registers (e.g. EntryHi, BadVAddr) are expected to be 64 bit
  • LLD/LD/SC/SCD
  • TRAP instructions, BREAK, SYSCALL
  • TLB
  • Exceptions: Overflow (ADD, DADD etc), unaligned memory access (e.g. LW)
  • Reading from and writing to ROM

How to build

n64-systemtest can be built on Windows, mac or Linux (including within WSL). The steps are pretty much the same.

  1. Get Rust: https://www.rust-lang.org/tools/install
  2. Get the source, e.g. through git checkout https://github.com/lemmy-64/n64-systemtest.git
  3. If this is your only rust-on-n64 project, simply run make install-dependencies (If you need cargo-n64 for other projects, install manually as needed. Notice that cargo-n64 official is on an older Rust unstable than n64-systemtest).
  4. Run make n64 to build the test rom.

Please note: N64 roms require a bootcode called IPL3. This bootcode is expected to setup hardware and copy the rom into memory. n64-systemtest comes with its own IPL3, which will NOT run on hardware. Once there is a community built open-source IPL3, we'll switch to that. If you'd like to use your own IPL3, please update the path in the Makefile at the very top.

How to run

Run the rom in your emulator of choice. Expect one of three things:

  1. The rom says something like "Done! Tests: 262. Failed: 0". If this is your emulator: Congratulations, you are done.
  2. The rom says something like "Done! Tests: 262. Failed: 1" OR the screen is full of error messages. This means that issues were found because the emulator isn't perfect. Hopefully, the error messages are clear enough to indicate what's broken.
  3. An empty screen: The emulator didn't make it to the end. See troubleshooting.

Troubleshooting

n64-systemtest runs A LOT of tests. If things are very broken, it can be hard to figure out how make any progress. There are two ways to make progress in such a situation:

ISViewer

All output that is printed on screen is also printed to memory mapped registers. For debugging, it is very valuable to hook this up and e.g. print to the console. To do that, simply provide the following two things:

  • 0xB3FF0020 until 0xB3FF0220: A buffer that can be written to using SB
  • 0xB3FF0014: A SW-writable length register. When written to, print the contents of the buffer

Disable tests

While running all tests is nice once a majority passes, it can be a pain for bringup. tests/testlist.rs contains the list of all tests. Simply comment out some or all as needed.

acknowledgment

This project was inspired by Peter Lemon's excellent N64 Bare Metal tests: https://github.com/PeterLemon/N64/ Furthermore, it wouldn't have been possible without the excellent cargo-n64, which brought Rust to the N64: https://github.com/rust-console/cargo-n64

Comments
  • Add masking and behavior tests for some COP0 registers

    Add masking and behavior tests for some COP0 registers

    This PR implements masking and behavior tests for the following COP0 registers:

    • Random
    • Wired
    • Status
    • Config
    • Processor Revision Id
    • Unused (7, 21, 22, 23, 24, 25, and 31)
    • Parity Error
    • Cache Error

    I've included rustdocs for each test to explain any finer details of what behavior is being tested. Below are some extra notes regarding the design of this PR.

    Random

    While it is possible to test only the masking by naively checking if Random is some value less than 64, this could produce a false-positive case. An emulator could allow writes to the lower 6 bits, but this is incorrect as the entire register is read-only. You also can't perform a simple "read, write, read, compare equality" test because the value is supposed to change on every instruction.

    I'm planning to try to address this later in a different PR, to have different levels of scrutiny (not every emulator cares about being extremely accurate).

    Also note, despite the name, Random is definitely not random in any way.

    Status and Config

    These regs, especially Config, have some constant 0 and 1 bits. So the test writes the existing value combined with whatever the opposite of these constant bits are. Then checks if the readback has these constant bits. Just writing all 1's or all 0's (like other masking tests) could produce false-negatives.

    Processor Revision Id

    I provided a test to the homebrew community that prints out their system's PRId register. All tested systems, including one PAL, produced 0xB22. It's safe to assume, until and exception is found, that this is likely what all n64 consoles will provide.

    Unused

    The COP0 has several "unused" registers which aren't completely unusable. Writes to any of these registers, won't actually store anything, but the written value is held in some kind of internal "latch", and can be read back at a later time. Only writes to any COP0 register appears to change the "latched" value. Other instructions that store data to regular GPR's do not affect it.

    Also, all unused registers are effectively interchangeable. Writing some value to Unused7, will match the readback from Unused31, etc. Writes to any used register (whether read-only or not), will match the readback from any Unused register.

    Parity and Cache Error

    According to VR4300 docs, these registers only exist for "compatibility reasons" but also supposedly aren't used by the CPU. They do still exist though, so they do not behave like the Unused registers. The lower 8-bits of Parity are also write-able.

    opened by bigbass1997 2
  • LB followed by LUI seems to be weird

    LB followed by LUI seems to be weird

    Found this in an unrelated test:

    LB $1, 0($2) LUI $2, 0x8000

    The LUI seems to change the memory address that the LB is loading from. Investigate this deeper. Does this happen with just LUI or also other instructions?

    opened by lemmy-64 1
  • Simplify and improve build process

    Simplify and improve build process

    Previously, we always needed to use some roundabout way of compiling the project, using some external tool or secondary project. From my experience using Rust for other embedded hardware, I've learned how to set up this project, such that much of what nust64 (and cargo-n64) would normally do for us, can be done directly by Cargo.


    By adding a field in rust-toolchain.toml, Cargo will automatically install the extra toolchain component (rust-src) if it is missing. Similarly, Cargo (via rustup) will also install the listed toolchain version automatically as well. Cargo/rustup will actually do this if you try to run most, if not all, cargo/rustup commands within the project directory.

    By setting default = ["default_tests"] in Cargo.toml, the default_tests feature flag will automatically be set by default, unless --no-default-features is specified. I think it's safe to say that this is the typical expected behavior for most developers. We can still explore alternative ways to organize tests in the future.

    Most of the magic though happens in .cargo/config.toml. The build target and other flags that normally would be specified with environment variables and CLI arguments, can be expressed here instead. Not only does this eliminate the need for nust64/cargo-n64 handling the compilation step, it gives the project developer full control over the build target and linker script.

    However, this will still result in an ELF file. In order to create an N64 ROM, we must still use some external program. Fortunately, Cargo also lets you specify a "runner". A runner is a program which Cargo will automatically run when compilation completes, while also including the path to the ELF file. nust64 will be this runner for now. If anyone wants to build/use a different runner, then they can freely do so. (I cannot identify any way to have cargo automatically install the runner for the user. Using a build.rs script doesn't work for this as far as I could tell. But the install process is very simple, and specified in the README.)

    In the future, if anything needs to be done pre-compilation, it should be possible to use build.rs (https://doc.rust-lang.org/cargo/reference/build-scripts.html)


    Please note: Because the compilation side is handled entirely by cargo instead of nust64, the builder/ project is no longer necessary. And so I've deleted it, and moved all of the n64-systemtest files back to the root directory. Fortunately, this time git actually understands what I did lol, and just shows them as being file renames.


    Advantages:

    • The project developer, gains far more control over the build process. That means: They can use any linker script they want, and reconfigure the build target.
    • The Cargo workspace with the "builder" project is no longer necessary, thus reducing repository complexity/size, and improving compile speeds (as it's only compiling n64-systemtest, and not builder+nust64+dependencies).
    • Compiling in debug mode (aka the 'dev' profile) is now available. Although, due to some build target issue(s), opt-level=0 cannot be used.
    • Rust projects like this one, are much less dependent on external software (cargo-n64/nust64).

    Disadvantages:

    • The runner program must be installed by anyone wishing to compile the project (cargo install nust64)
    • The runner can only be one program. So people who want an emulator/etc to run automatically after building, must make their own script (e.g. .sh/.bat). This might be solvable by adding some extra functionality to nust64, or perhaps we can provide example shell scripts and commented-out runner configurations.

    ~~PR will remain a draft until Lemmy completes his FPU test additions~~

    opened by bigbass1997 1
  • Failed to build

    Failed to build

    I'm trying to build on Fedora 36. I get this response:

    [[email protected] n64-systemtest]$ cargo run --release
       Compiling proc-macro2 v1.0.39
       Compiling unicode-ident v1.0.1
       Compiling syn v1.0.96
       Compiling serde_derive v1.0.137
       Compiling serde v1.0.137
       Compiling autocfg v1.1.0
       Compiling crc32fast v1.3.2
       Compiling libc v0.2.126
       Compiling memchr v2.5.0
       Compiling serde_json v1.0.81
       Compiling adler v1.0.2
       Compiling semver v1.0.10
       Compiling cfg-if v1.0.0
       Compiling camino v1.0.9
       Compiling ryu v1.0.10
       Compiling hashbrown v0.12.1
       Compiling os_str_bytes v6.1.0
       Compiling itoa v1.0.2
       Compiling once_cell v1.12.0
       Compiling textwrap v0.15.0
       Compiling termcolor v1.1.3
       Compiling strsim v0.10.0
       Compiling crc-catalog v1.1.1
       Compiling bitflags v1.3.2
       Compiling bytes v1.1.0
       Compiling dunce v1.0.2
       Compiling miniz_oxide v0.5.3
       Compiling indexmap v1.9.0
       Compiling clap_lex v0.2.2
       Compiling crc v2.1.0
       Compiling flate2 v1.0.24
       Compiling quote v1.0.18
       Compiling atty v0.2.14
       Compiling object v0.28.4
       Compiling clap v3.2.5
       Compiling cargo-platform v0.1.2
       Compiling toml v0.5.9
       Compiling cargo_metadata v0.14.2
       Compiling nust64 v0.1.3
       Compiling builder v0.1.0 (/var/home/loganmc10/git_stuff/n64-systemtest/builder)
        Finished release [optimized] target(s) in 29.34s
         Running `target/release/builder`
    thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /var/home/loganmc10/.cargo/registry/src/github.com-1ecc6299db9ec823/nust64-0.1.3/src/elf.rs:188:10
    
    opened by loganmc10 1
  • Figure out exception vectors in TLB tests

    Figure out exception vectors in TLB tests

    In the TLB tests, some exceptions are expected at 80000180, others at 80000080. Figure out why this is needed and maybe design some specific test cases around that

    opened by lemmy-64 1
  • [fpu] expand underflow tests

    [fpu] expand underflow tests

    This should cover all instructions that can produce an inexact denormal on the host FPU. Previously, this was tested only for DIV.S, and in CTV.S.D (though only for positive denormals).

    Technically, the final result is identical to the case in which an exact denormal is calculated by the host (either an unimplemented exception, or a flush), but for the emulator they are likely different codepaths as the control flags generated by the host FPU are likely different.

    opened by rasky 0
  • Add integer division special case tests

    Add integer division special case tests

    This PR adds tests for integer division by zero and dividing the minimum value by -1 for all four opcodes (DIV/DIVU/DDIV/DDIVU), checking both the quotient and remainder for correctness.

    Note that emulators not explicitly supporting the min / -1 case tend to crash with an arithmetic overflow exception on x86, which makes diagnosing the issue a bit more tricky unless you are aware of this test.

    opened by sp1187 0
  • Builder feature parity with old makefile

    Builder feature parity with old makefile

    Small PR to implement a few options that were missing compared to the old makefile.

    • Added a message upon successfully building and saving the rom, which tells the user where the rom was saved to.
    • Added arguments for running the finished rom with usb64 and unfloader.

    The makefile appeared to have a way to start some undefined emulator, but I didn't port this because not every emulator's CLI interface works the same way.

    There was also a command to build the mini-ipl3 source. But since it's highly unlikely a non-contributor would need to edit that, and the compiled binary is included in the repo, adding this command doesn't seem necessary. Plus, it's very possible the user doesn't have bass installed on their PATH or at all.

    There is a problem with discoverability of what feature flags are available. But this should be resolved in later PR(s) that rework test selection. (For now, people can look at n64-systemtest's Cargo.toml, even though it isn't user-friendly.)

    opened by bigbass1997 0
  • Migrate to nust64

    Migrate to nust64

    Currently, a fork of cargo-n64 in combination with a makefile, is used for building this project into an n64 rom.

    There are several problems with cargo-n64 including:

    1. Unmaintained for over a year.
    2. Relies on an old version of Rust nightly (which is partly why a fork is used).
    3. Arbitrarily limits the size of the resulting binary for no apparent reason (again, partly why a fork is used).
    4. Overly-complicated codebase which makes modifications difficult.
    5. Cannot be used as a crate, necessitating a makefile for ease-of-use.

    I've dissected cargo-n64 to learn exactly what it's doing that makes this whole process work, and reworked everything into a new crate: nust64. In short, the build process boils down to two steps: running a cargo build command with special arguments (e.g. custom architecture target) which produces an ELF binary, and extracting/combining the necessary parts of the ELF into a rom.


    Migrating to nust64 fixes all of the above problems. However, in order to use it, this project's structure has to change a little. In this PR, you'll see in the Cargo.toml, it now uses a Cargo/Rust feature called Workspaces. Nearly everything from the current repo, has been moved into its own directory /n64-systemtest/, and a new directory /builder/ has been created. Each workspace "member" is its own separate project, but they can still be run/built/accessed from other members and from the root directory.

    Due to how the root Cargo.toml is set up, when you run cargo build or cargo run in the root directory, cargo will only perform the respective command on the "builder" project. You can think of the "builder" project, as a makefile, except that it is a full Rust program.


    This PR only handles the migration to nust64. It doesn't change the n64-systemtest code at all (besides moving it). It doesn't add any new functionality. It does update the README to reflect the changed build instructions. There is potential for additional functionality to be added later, but this PR is only focused on the migration. (The n64-systemtest code is up-to-date as of 5fb0574a24bfd2a05e80b27d3e6e93d1811294f5)

    Similar to the make n64 command, running cargo run or cargo run --released will by default include --features default_tests and default to the mini-ipl3 binary, unless otherwise specified.


    If you run into any problems building this PR into a working rom, I suggest running cargo clean once to clear out any outdated junk. Otherwise, let me know any other problems or questions you have.

    opened by bigbass1997 0
  • Add additional rustdocs

    Add additional rustdocs

    Adds some docs to commonly used methods and structures.

    While this isn't a crate, rustdocs are still helpful for new developers who are learning the codebase (such as myself). IDE's and editors that support Rust, typically have some way of rendering these docs. Otherwise devs can run cargo doc to generate the Rust standard web-based docs (/target/doc/n64_systemtest/index.html).

    opened by bigbass1997 0
  • Bump Rust nightly to 2022-03-27

    Bump Rust nightly to 2022-03-27

    The following features no longer have to be declared: const_fn_trait_bound, asm, global_asm

    Unfortunately, a few new features are needed now: asm_const, asm_experimental_arch (as we're on MIPS) and asm_sym.

    Stil a win as asm! itself got stabilized and we just need subfeatures.

    opened by lemmy-64 0
Owner
null
Tests + experiments that aim to answer questions of optimisation + what's possible w/ Rust.

rust-experiments Resources TheDevMethod: Rust YT tutorials Introduction to Rust - Part 15: Futures How to use Mutex and Atomically Reference Counted A

DeGatchi 1 Jun 17, 2022
Rust library for hardware accelerated drawing of 2D shapes, images, and text, with an easy to use API.

Speedy2D Hardware-accelerated drawing of shapes, images, and text, with an easy to use API. Speedy2D aims to be: The simplest Rust API for creating a

null 215 Nov 17, 2022
A Rust-like Hardware Description Language transpiled to Verilog

Introduction This projects attempts to create a Rust-like hardware description language. Note that this has nothing to do with Rust itself, it just ha

Benjamin Stürz 4 Sep 1, 2022
Checks Crusader Kings 3 user mod files for common mistakes and warns about them.

ck3-tiger Pounces on bugs. Checks Crusader Kings 3 user mod files for common mistakes and warns about them. For example: missing localizations, or usi

Richard Braakman 7 Oct 26, 2022
Quickly set up a `probe-run` + `defmt` + `flip-link` embedded project

app-template Quickly set up a probe-run + defmt + flip-link embedded project Dependencies 1. flip-link: $ cargo install flip-link 2. probe-run: $ # ma

Knurling 194 Nov 27, 2022
Simple autoclicker written in Rust, to learn the Rust language.

RClicker is an autoclicker written in Rust, written to learn more about the Rust programming language. RClicker was was written by me to learn more ab

null 7 Nov 15, 2022
Rust programs written entirely in Rust

mustang Programs written entirely in Rust Mustang is a system for building programs built entirely in Rust, meaning they do not depend on any part of

Dan Gohman 544 Nov 19, 2022
clone of grep cli written in Rust. From Chapter 12 of the Rust Programming Language book

minigrep is a clone of the grep cli in rust Minigrep will find a query string in a file. To test it out, clone the project and run cargo run body poem

Raunak Singh 1 Dec 14, 2021
The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language.

rcc The Rust Compiler Collection is a collection of compilers for various languages, written with The Rust Programming Language. Compilers Language Co

null 2 Jan 17, 2022
Game Boy Emulator written in Rust, as a way to fully grasp the Rust programming language

Flan's Game Boy Emulator Game Boy Emulator written in Rust, as a way to get hands-on with the Rust programming language, and creating a proper project

Flan 2 Jul 29, 2022
Nixt is an interpreted programming language written in Rust

Nixt Nixt is an interpreted lisp inspired programming language written in Rust Index About Examples Installation Build About Nixt goal is to provide a

Wafelack 17 Jul 18, 2022
Fegeya Elitebuild, small, powerful build system. Written in Rust.

Fegeya Elitebuild Small, powerful, work-in-progress build system. Written in Rust. Features: No functions (all are built-ins) All variables are global

Ferhat Geçdoğan 25 Nov 9, 2022
Raytracer tutorial for PPCA 2021, written in Rust.

Pseudo Photograph Company of ACM 工科和ACM的朋友们都已结课!看看这些了不起的艺术品: 工科 ACM ACM伪摄影公司,简称PPCA,于2021年成立 ?? 这个项目的主要工作是使用Rust语言实现一个光线追踪渲染器。以这个形式,你能通过学习一门新的(而且漂亮的)语

null 112 Nov 14, 2022
An i386 operation system written in pure rust for fun and no profit.

OrustS An i386 operation system written in pure rust (for fun and no profit). This operation system is under active developing. Checklist implement a

M4tsuri 10 Aug 12, 2022
A tool to make grocery lists written in Rust

grusterylist: makes grocery lists, written in Rust grusterylist uses and can add to local libraries of user-added recipes and grocery items to put tog

null 3 Jun 17, 2022
DWARF packaging utility, written in Rust, supporting GNU extension and DWARF 5 package formats.

thorin thorin is an DWARF packaging utility for creating DWARF packages (*.dwp files) out of input DWARF objects (*.dwo files; or *.o files with .dwo

The Rust Programming Language 19 Nov 16, 2022
a super fast scientific calculator with dimensional analysis support written in Rust 🦀

larvae a super fast scientific calculator with dimensional analysis support written in Rust ?? ?? heavily inspired from insect Usage: Command mode: $

John McAvoy 2 Jan 10, 2022
A complicated eso-lang written in Rust that doesn't limit your creativity!

Documentation for mott Introduction mott (from french "mot" = word) is (maybe) the first programming language, that doesn't limit your creativity to p

Lars 1 Feb 1, 2022
ppfetch written in Rust.

ppfetch ppfetch, but written in Rust. ?? About The ppfetch was created to be a simple, basic, and posix fetch. Now, it's a simple, fast and written in

Redson dos Santos Silva 0 Oct 27, 2022