install-mac: Adding target linker to /Users/shea/.cargo/config...

install-env: Installing xargo.

error: Couldn't load Cargo configuration

Caused by: could not parse TOML configuration in /Users/shea/.cargo/config

Caused by: could not parse input as TOML /Users/shea/.cargo/config:1:4-2:1 expected =, but found

make: *** [env] Error 101

Something I changed between 1f9b6cd2d67f6b0c0004b8fd1ff5678c0338be2b and 19d09785340e1d49ba70100c1766aa3fe59db9d1 seems to be causing a double fault (or triple fault?).

I need to figure out where the fault is occurring and which commit caused it before I can pinpoint the cause of the issue. I'm assuming that the interrupt handling refactor in commits 621f40dd6e6731092043f4b3826f44dbb8f51235 and ec14c269f4ea9140ba5983a26d09464266345ddb, or the changes to DTable::get_ptr() are probably what's causing the problem.

It'd be nice if we could use Homebrew to install more of the macOS dev dependencies (like x86_64-pc-elf binutils and gcc, and objconv), so that they can be installed from the Brewfile rather than forcing everyone who wants to build SOS to run the install scripts.

This PR adds mtools to the list of build dependencies on Linux and adds it to the packages installed on Arch/Manjaro and Debian/Ubuntu. Closes #52.

Thanks @rachlmac for fixing this issue.

If we know the location of ISR functions in the compiled binary statically (which we do), then it seems to me that we should be able to create interrupt gates for them statically as well, at compile-time. This way, we could have our IDT already populated with kernel ISRs at compile-time, rather than having to fill it with Gate::absent() and then loop through a different array of ISRs and insert them into the IDT as part of the init process, like we currently do.

Our current interrupt handling architecture certainly works, but it is a bit of a mess. We have an assembly language file that makes an array of interrupt handler functions, but they all just call the same Rust function which then matches the interrupt ID and dispatches again. This is a mess.

The current approach has way too many moving parts, with ASM interrupt handlers getting dispatched to by the IDT but then all just jumping to the same God function (handle_interrupt) in Rust-land, which dispatches again based on the interrupt ID. It also doesn't help us to achieve our goal of using files ending in .asm only in the boot sequence.

Also, even though we have separate handle_interrupt and handle_cpu_exception functions, and separate macros in the assembly file for generating ASM handlers for exceptions and normal interrupts, we still dispatch CPU exceptions through handle_interrupt. Even the name handle_interrupt is a problem: a function with a name that broad is probably a God function that needs to be broken apart.

It seems to me we have several options. We could keep the current architecture, but try to do away with the assembly handlers by just sticking handle_interrupt and handle_cpu_exception directly into the IDT. In @phil-opp's recent blog post on interrupt handling, he just writes a separate Rust handler function for each interrupt and sticks them into his IDT. This might be a better way to go - it's simpler and gets rid of the handle_interrupt God function.

This PR adds a working implementation of paging and remapping the kernel into the high address space on x86_64! 🍾

Note that interrupts had to be re-disabled temporarily because of issues from #78 that I'm still looking into.

This PR also includes a number of changes to the elf and alloc crates that were necessary to get paging working correctly. It doesn't finish the whole "allocator rewrite" project, but it brings us further along.

Paging issues:

• Closes #18
• Closes #19
• Closes #20

Allocator issues:

• Closes #64
• Closes #65
• Closes #66
• Closes #72

Also makes some progress on issues #31, #73, #76, and #84.

This PR adds the pure-Rust boot sequence for x86_64.

This closes #17, closes #67, closes #68, closes #69, and closes #70.

There is now a page fault when we try to set up the kernel heap, but this is likely because we are trying to create the heap before we've remapped the kernel into the higher address space. Since this is already slated to be more or less completely rewritten anyway (see #73 and so on), I think we can go ahead and merge this.

Rather than representing a pointer to an allocation as *mut u8, can we represent it as an Option<Unique<u8>>?

Using Unique might allow the compiler to do more reasoning about an allocation result...

build errors:

   Compiling memory v0.0.1 (file:///Users/eliza/Code/sos-kernel/memory)
error[E0407]: method `step` is not a member of trait `core::iter::Step`
   --> memory/src/macros.rs:229:13
    |
229 | /             fn step(&self, by: &Self) -> Option<Self> {
230 | |                 self.number
231 | |                     .step(&by.number)
232 | |                     .map(|s| $ty { number: s})
233 | |             }
    | |_____________^ not a member of trait `core::iter::Step`
    |
   ::: memory/src/arch/x86_64/mod.rs
    |
34  |       #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Page!(PAddr) )]
    |  ___________________________________________________________-
35  | |     pub struct PhysicalPage { pub number: u64 }
36  | | }
37  | | impl fmt::Debug for PhysicalPage {
...   |

error[E0407]: method `steps_between_by_one` is not a member of trait `core::iter::Step`
   --> memory/src/macros.rs:247:13
    |
247 | /             fn steps_between_by_one(start: &$ty, end: &$ty) -> Option<usize> {
248 | |                 use ::core::iter::Step;
249 | |                 <<$addr as Addr>::Repr as Step>::steps_between( &start.number
250 | |                                                               , &end.number
251 | |                                                               , &1
252 | |                                                               )
253 | |             }
    | |_____________^ not a member of trait `core::iter::Step`
    |
   ::: memory/src/arch/x86_64/mod.rs
    |
34  |       #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Page!(PAddr) )]
    |  ___________________________________________________________-
35  | |     pub struct PhysicalPage { pub number: u64 }
36  | | }
37  | | impl fmt::Debug for PhysicalPage {
...   |

error[E0407]: method `is_negative` is not a member of trait `core::iter::Step`
   --> memory/src/macros.rs:263:23
    |
263 |               #[inline] fn is_negative(&self) -> bool { unimplemented!(); }
    |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not a member of trait `core::iter::Step`
    |
   ::: memory/src/arch/x86_64/mod.rs
    |
34  |       #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Page!(PAddr) )]
    |  ___________________________________________________________-
35  | |     pub struct PhysicalPage { pub number: u64 }
36  | | }
37  | | impl fmt::Debug for PhysicalPage {
...   |

error[E0407]: method `step` is not a member of trait `core::iter::Step`
   --> memory/src/macros.rs:229:13
    |
229 | /             fn step(&self, by: &Self) -> Option<Self> {
230 | |                 self.number
231 | |                     .step(&by.number)
232 | |                     .map(|s| $ty { number: s})
233 | |             }
    | |_____________^ not a member of trait `core::iter::Step`
    |
   ::: memory/src/lib.rs
    |
160 |       #[derive( Copy, Clone, Eq, Ord, PartialEq, PartialOrd, Page!(VAddr) )]
    |  ____________________________________________________________-
161 | |     pub struct VirtualPage { pub number: usize }
162 | | }
163 | |
...   |
222 | |
223 | |     /// Remove `n` pages from the beginning of this `PageRange`
    | |___________________________________- in this macro invocation

error[E0407]: method `steps_between_by_one` is not a member of trait `core::iter::Step`
   --> memory/src/macros.rs:247:13
    |
247 | /             fn steps_between_by_one(start: &$ty, end: &$ty) -> Option<usize> {
248 | |                 use ::core::iter::Step;
249 | |                 <<$addr as Addr>::Repr as Step>::steps_between( &start.number
250 | |                                                               , &end.number
251 | |                                                               , &1
252 | |                                                               )
253 | |             }
    | |_____________^ not a member of trait `core::iter::Step`
    |
   ::: memory/src/lib.rs
    |
160 |       #[derive( Copy, Clone, Eq, Ord, PartialEq, PartialOrd, Page!(VAddr) )]
    |  ____________________________________________________________-
161 | |     pub struct VirtualPage { pub number: usize }
162 | | }
163 | |
...   |
222 | |
223 | |     /// Remove `n` pages from the beginning of this `PageRange`
    | |___________________________________- in this macro invocation

error[E0407]: method `is_negative` is not a member of trait `core::iter::Step`
   --> memory/src/macros.rs:263:23
    |
263 |               #[inline] fn is_negative(&self) -> bool { unimplemented!(); }
    |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not a member of trait `core::iter::Step`
    |
   ::: memory/src/lib.rs
    |
160 |       #[derive( Copy, Clone, Eq, Ord, PartialEq, PartialOrd, Page!(VAddr) )]
    |  ____________________________________________________________-
161 | |     pub struct VirtualPage { pub number: usize }
162 | | }
163 | |
...   |
222 | |
223 | |     /// Remove `n` pages from the beginning of this `PageRange`
    | |___________________________________- in this macro invocation

error[E0050]: method `steps_between` has 3 parameters but the declaration in trait `core::iter::Step::steps_between` has 2
   --> memory/src/macros.rs:237:58
    |
237 |               fn steps_between(start: &$ty, end: &$ty, by: &$ty)
    |                                                            ^^^^ expected 2 parameters, found 3
    |
   ::: memory/src/arch/x86_64/mod.rs
    |
34  |       #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Page!(PAddr) )]
    |  ___________________________________________________________-
35  | |     pub struct PhysicalPage { pub number: u64 }
36  | | }
37  | | impl fmt::Debug for PhysicalPage {
...   |
    |
    = note: `steps_between` from trait: `fn(&Self, &Self) -> core::option::Option<usize>`

error[E0046]: not all trait items implemented, missing: `add_usize`
   --> memory/src/macros.rs:227:9
    |
227 |           impl ::core::iter::Step for $ty {
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `add_usize` in implementation
    |
   ::: memory/src/arch/x86_64/mod.rs
    |
34  |       #[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Page!(PAddr) )]
    |  ___________________________________________________________-
35  | |     pub struct PhysicalPage { pub number: u64 }
36  | | }
37  | | impl fmt::Debug for PhysicalPage {
...   |
    |
    = note: `add_usize` from trait: `fn(&Self, usize) -> core::option::Option<Self>`

error[E0050]: method `steps_between` has 3 parameters but the declaration in trait `core::iter::Step::steps_between` has 2
   --> memory/src/macros.rs:237:58
    |
237 |               fn steps_between(start: &$ty, end: &$ty, by: &$ty)
    |                                                            ^^^^ expected 2 parameters, found 3
    |
   ::: memory/src/lib.rs
    |
160 |       #[derive( Copy, Clone, Eq, Ord, PartialEq, PartialOrd, Page!(VAddr) )]
    |  ____________________________________________________________-
161 | |     pub struct VirtualPage { pub number: usize }
162 | | }
163 | |
...   |
222 | |
223 | |     /// Remove `n` pages from the beginning of this `PageRange`
    | |___________________________________- in this macro invocation
    |
    = note: `steps_between` from trait: `fn(&Self, &Self) -> core::option::Option<usize>`

error[E0046]: not all trait items implemented, missing: `add_usize`
   --> memory/src/macros.rs:227:9
    |
227 |           impl ::core::iter::Step for $ty {
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing `add_usize` in implementation
    |
   ::: memory/src/lib.rs
    |
160 |       #[derive( Copy, Clone, Eq, Ord, PartialEq, PartialOrd, Page!(VAddr) )]
    |  ____________________________________________________________-
161 | |     pub struct VirtualPage { pub number: usize }
162 | | }
163 | |
...   |
222 | |
223 | |     /// Remove `n` pages from the beginning of this `PageRange`
    | |___________________________________- in this macro invocation
    |
    = note: `add_usize` from trait: `fn(&Self, usize) -> core::option::Option<Self>`

error: aborting due to 10 previous errors

error: Could not compile `memory`.

Currently, we tend to format both virtual (VAddr) and physical (PAddr) addresses using the "{:#p}" format specifier for pointers. This works fine, but makes telling whether an address is physical or virtual difficult when reading through logs, et cetera.

It'd be nice if we could have a consistent notation for these. I suggest possibly replacing "0xff00" with "Pxff00 for physical addresses and Vxff00 for virtual addresses, or appending a P/V to the end of the hex address, like 0xff00_p.

This can be hard-coded to the hello-world test elf for now, but eventually we'll want to generalise this for including userspace stuff in the OS distribution.

sos-os/kernel now has a Chat Room on Gitter

@hawkw has just created a chat room. You can visit it here: https://gitter.im/sos-os/kernel.

This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.

Happy chatting.

PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

┌─╼[~/Projects/sos-kernel] [master*]
└────╼ make kernel
   Compiling vga v0.1.0 (file:///Users/shea/Projects/sos-kernel/vga)
error[E0277]: the trait bound `core::ptr::NonNull<[[Char; 80]; 25]>: core::marker::Send` is not satisfied in `Terminal`
  --> vga/src/lib.rs:91:1
   |
91 | / pub static CONSOLE: Mutex<Terminal>
92 | |     = Mutex::new(unsafe { Terminal::new(
93 | |          Palette::new(Color::LightGrey, Color::Black )
94 | |        , 0xB8000
95 | |     )});
   | |________^ `core::ptr::NonNull<[[Char; 80]; 25]>` cannot be sent between threads safely
   |
   = help: within `Terminal`, the trait `core::marker::Send` is not implemented for `core::ptr::NonNull<[[Char; 80]; 25]>`
   = note: required because it appears within the type `Terminal`
   = note: required because of the requirements on the impl of `core::marker::Sync` for `spin::Mutex<Terminal>`
   = note: shared static variables must have a type that implements `Sync`

error: aborting due to previous error

error: Could not compile `vga`.

To learn more, run the command again with --verbose.
make: *** [target/x86_64-sos-kernel-gnu/debug/sos_kernel] Error 101