This project is mirrored from Pull mirroring updated .
  1. 13 Aug, 2019 1 commit
  2. 03 Aug, 2019 1 commit
  3. 16 Jul, 2019 1 commit
  4. 15 Jul, 2019 1 commit
  5. 13 Jul, 2019 2 commits
  6. 07 Jul, 2019 1 commit
  7. 06 Jul, 2019 1 commit
  8. 25 Jun, 2019 1 commit
  9. 16 Jun, 2019 2 commits
  10. 10 Jun, 2019 1 commit
  11. 02 Jun, 2019 1 commit
  12. 04 May, 2019 2 commits
  13. 16 Jan, 2019 1 commit
  14. 26 Jul, 2018 1 commit
    • Colin Finck's avatar
      Port the Memory Manager to AArch64, with full support for 4-level Paging, 4... · a711534f
      Colin Finck authored
      Port the Memory Manager to AArch64, with full support for 4-level Paging, 4 KiB, 2 MiB, and 1 GiB Pages, and Execute-Disable!
      * Make PageTableEntryFlags architecture-independent by adding builder pattern methods.
        Instead of providing PageTableEntryFlags::EXECUTE_DISABLE, one simply uses the execute_disable() method now.
        The different implementations of each method for AArch64 and x86_64 map to the respective architecture flags.
      * Make a boolean execute_disable the only additional parameter for mm::allocate() to make it architecture-independent.
      * Remove the do_ipi parameter from paging functions.
        It was x86_64-specific and there is nothing wrong with always doing an IPI when necessary and application processors have been booted.
      * Make the LEVEL/MAP_LEVEL constant ascending instead of descending for the AArch64 implementation to match with the Page Table names (L0, L1, etc.).
  15. 24 Jul, 2018 1 commit
    • Colin Finck's avatar
      The first real AArch64 bringup commit for HermitCore-rs · a9671ff5
      Colin Finck authored
      * Split the CMake files into an architecture-independent and an architecture-dependent part.
        This overhaul of the build system also removes the custom "module system", which doesn't make much sense for a Rust kernel and doesn't work well with such a split configuration.
      * Add an aarch64-unknown-hermit-kernel.json target for Xargo.
      * Implement basic IRQ and serial port functions for AArch64 to get a first output.
      * Copy the 4-level paging from x86_64 to AArch64 and remove the parts relying on the "x86" crate.
        While this still needs some work to get the names and flags right, 4-level paging should generally work on AArch64 with the same concepts that are used for x86_64.
      * Comment out and stub out many functions for AArch64 to let is somewhat compile.
      * Redefine core_id as a CPU number that is guaranteed to be sequential to make it architecture-independent.
        For x86_64, this number is now translated to a Local APIC ID in the "apic" module only.
      * Add a per-architecture TaskStacks structure, which contains "stack" and "ist" on x86_64 and only "stack" on AArch64.
      * Add a per-architecture network_adapter_init function to initialize RTL8139 only for x86_64.
      * Get rid of the top-level "arch" directory and put the reasonable architecture-dependent include files into /include/hermit/<ARCH>, all prefixed with "arch_".
      * Make the inclusion of some crates dependent on the target architecture.
      * Rename get_number_of_processors to get_processor_count and make it return a usize.
  16. 29 Jun, 2018 1 commit
    • Colin Finck's avatar
      Overhaul the timing framework to improve the global timer resolution to 1... · aee76165
      Colin Finck authored
      Overhaul the timing framework to improve the global timer resolution to 1 microsecond and simplify the code, also for a later AArch64 port.
      * Rename the udelay() syscall to sys_usleep() for consistence and accept a u64 parameter.
        udelay() with an u32 parameter is still provided to serve the existing customers, but should eventually vanish.
      * Use time values in 1 microsecond granularity and u64 everywhere instead of simulating a 10ms timer with the CPU Time-Stamp Counter.
        This guarantees maximum precision for the best timing function we currently provide (sys_usleep).
        It also simplifies the code, because we can simply add microseconds to the timer tick count.
      * Rewrite update_timer_ticks() as get_timer_ticks().
        It simply divides get_timestamp() by get_frequency() now to simulate a 1 microsecond timer with the CPU Time-Stamp Counter.
        This requires no per-core variables and is much more accurate.
      * Calibrate the Local APIC Timer for 1 microsecond resolution.
        This reduces the maximum timeout to 34 seconds on a Intel Xeon E5-2650 v3 @ 2.30GHz, but for longer timeouts, the one-shot timer would simply
        fire multiple times.
      * Detect CPU support for the TSC-Deadline Mode of the Local APIC Timer and use it.
        This one is easier to program than the legacy One-Shot Mode, even more accurate, and has no maximum timeout.
  17. 06 Jun, 2018 1 commit
  18. 30 May, 2018 1 commit
    • Colin Finck's avatar
      Implement graceful shutdown. · b684516e
      Colin Finck authored
      * Call acpi::poweroff() from arch::x86_64::processor::shutdown().
      * Store the exit code of the last task that finished.
      * Add a sys_shutdown() syscall implemented per SysCallInterface.
        The generic implementation just calls arch::processor::shutdown(), whereas the Proxy and Uhyve SyscallInterfaces send SysExit
        commands over their communication channels.
        This supersedes the per-SyscallInterface sys_exit() call, which would wrongly shut down the entire system when the first task exited.
      * Add a sys_lwip_register_tcpip_task() syscall to register the Task ID of the lwIP TCP/IP thread.
        This enables us to shut down the system when only the lwIP TCP/IP thread is left.
      * Make TaskId u32 to be consistent with the Tid type of the syscalls.
  19. 13 Apr, 2018 1 commit
  20. 04 Apr, 2018 1 commit
  21. 30 Mar, 2018 1 commit
  22. 29 Mar, 2018 1 commit
  23. 22 Mar, 2018 1 commit
  24. 16 Feb, 2018 1 commit
    • Colin Finck's avatar
      Remove the locks around current_task and finished_tasks. Check if we need to... · d9ed73bd
      Colin Finck authored
      Remove the locks around current_task and finished_tasks. Check if we need to wake up a CPU using the new is_halted variable (locked together with ready_queue).
      This reduces the average cycles for a getpid call from 147 to 28 on my ThinkPad X61 (Core 2 Duo T7300).
      Basically every syscall benefits from this change.
  25. 12 Feb, 2018 1 commit
    • Colin Finck's avatar
      Fix race conditions and synchronization. · f6de65d8
      Colin Finck authored
      * Previously, the scheduler switched to the idle task, reenabled interrupts, and later halted the CPU with a HLT instruction,
        all without any atomics. This resulted in Wakeup interrupts being received *BEFORE* going into the halt state, and then not
        waking up again.
        To solve this, I'm now using the newly implemented irq::enable_and_wait() to perform the "sti; hlt" sequence, which is
        atomic on x86 according to
      * This and other accesses to current_task can also happen concurrently, so guard it with an RwLock for minimum overhead.
  26. 07 Feb, 2018 1 commit
    • Colin Finck's avatar
      Get rid of the .percore section, MAX_CORES, and megabytes of statically allocated data. · 108d9c9d
      Colin Finck authored
      - Introduce a PerCoreVariables structure statically allocated for the Boot Processor and dynamically allocated when booting each Application Processor.
        Its address is stored in the GS register.
        All fields are of type PerCoreVariable<T>, which implements get() and set() functions for easy per-CPU access to the encapsulated variables.
      - Store the only mutable pointer to the current CPU's scheduler in PerCoreVariables.
        Introduce a function core_scheduler() for easy access. This saves us from doing a BTreeMap search every time we want to access the current CPU's scheduler and should improve performance.
        The SCHEDULERS BTreeMap now contains an immutable reference to each CPU scheduler. This way, we can be sure that only spinlock-guarded fields (like ready_queue) of a different CPU's scheduler can be modified.
      - Use a larger DEFAULT_STACK_SIZE for tasks and the KERNEL_STACK_SIZE only for booting (and later the Idle task).
        Due to Rust's missing support for code generic over different array sizes, I couldn't adapt the KernelStack struct for this and handle stacks as mere pointer addresses now.
      - Dynamically allocate the GDT and TSS to get rid of the MAX_CORES dependency.
      - Introduce an extra_flags parameter in mm::allocate to allow allocating executable and non-executable memory.
        Allocated stacks are now executable to possibly support dynamically generated code on the stack (see
      - Slim down boot.asm and entry.asm to not do any initialization twice that is later done in
        Enable EFER_NXE in Assembler though in order to allow early access to NX-protected memory.
      - Generate a next to the config.asm containing the configured stack sizes. Finally, you can adjust the stack sizes centrally in the CMake configuration again.
        We no longer need a config.h, so remove all references to it.
      - Get rid of and
        Replace the current_task_lwip_errno dummy by sys_lwip_get_errno() and sys_lwip_set_errno() syscalls and move the kputchar() into a proper sys_putchar() syscall.
  27. 05 Feb, 2018 1 commit
    • Colin Finck's avatar
      Implement everything required for Round-Robin Scheduling and pthread-embedded support. · 092825bf
      Colin Finck authored
      * Rework get_next_task(), reschedule(), and schedule() into a cleaner scheduler() function.
        This one can be called from every longer call an application makes into the kernel (like sys_msleep).
        It also does Round-Robin Scheduling when a task has already run for at least one tick and another task of the same priority is available.
      * Implement sys_yield(), calling the scheduler in sys_msleep(), and an exported udelay() that is different from processor::udelay().
        The processor::udelay() version is guaranteed to always do busy-waiting and not call into the scheduler.
        The exported udelay() syscall blocks the current task for longer waits and calls into the scheduler.
        It is used by the Go runtime of GCC and should better be called sys_usleep() for consistence. But this requires a breaking change in our GCC that would make it incompatible with the HermitCore C version.
      * Rework scheduler priorities. Reduce the number to four, with 0 now being IDLE_PRIO and 3 being HIGH_PRIO again.
        This reduces memory usage for PriorityTaskQueues.
        We now also need processor::msb instead of processor::lsb. Implement that in a more Rust-friendly way (returning Option<u64>).
      * Use a spin::RwLock for Task::heap to handle concurrent modifications of the shared task heap from multiple (cloned) tasks.
      * Implement a RecursiveMutex for pthread-embedded. This one grants access when a mutex has not been acquired or acquired by the same task.
        As we know that the scheduler() function of a per-core scheduler is never called from a different CPU and never in an interrupt, we can guard the internal state of the RecursiveMutex with a simple Spinlock without risking deadlocks.
      * Give more unique names to the opaque structures for internal HermitCore-rs synchronization primitives.
        This prevents name clashes in pthread-embedded and also makes their HermitCore association more obvious.
      We can now run thr_hello with any number of CPUs.
      The multithreaded Go application "pi" also outputs a result now, but still doesn't finish all threads properly.
  28. 29 Jan, 2018 1 commit
    • Colin Finck's avatar
      Reenable SMP to run the scheduler on all cores and implement timing functions... · 7a9007c6
      Colin Finck authored
      Reenable SMP to run the scheduler on all cores and implement timing functions as well as waking up a task after a specified timeout.
      * All functions internally use the absolute wakeup_time (timer_ticks value of the CPU) to specify timeouts.
        Exported functions often expect a relative value in ms, which is then calculated to an absolute wakeup_time.
      * Implement a single queue blocked_tasks with all blocked tasks sorted by wakeup time. Tasks waiting indefinitely are simply put at the end.
        It is based on a doubly-linked list, so elements can also be removed from the middle.
        This, and guarding blocked_tasks with a SpinlockIrqSave, allows waking up timed tasks prematurely (e.g. when a semaphore has been released) without risking race conditions where semaphore release and timeout occur simultaneously.
      * Calibrate the Local APIC Timer to have a counter value for a single tick of the internal 100 Hz tick counter.
        It is then used as a one-shot timer for the next task that shall be woken up at a specific wakeup_time.
        In contrast to the C version of HermitCore, the maximum counter divisor of 128 is chosen, because it provides enough accuracy for timing in the range of milliseconds, yet allows for much longer timeouts.
      * Implement WakeupReason, so a task knows whether it has been woken up manually or by a timeout.
      * All this is used to implement sys_sem_timedwait() and related syscalls.
      * Ignore overflows of RDTSC/RDTSCP and always calculate diff = get_timestamp() - last_rdtsc.
        Even on a 4 GHz CPU, an overflow can only happen every 146 years. You are safe if you reboot your OS every 100 years :)
      * Fix a horrible bug: Use the address of the boot stack rather than its garbage value! Interestingly, this worked well so far..
        Nevertheless, HermitCore-rs currently needs a 32K stack to boot up. Needs further investigation!
      * Introduce the concept of "CPU Numbers" (sequential numbers to count the enabled CPUs).
        They often match with Core IDs, but don't necessarily need to, e.g. when a CPU Core has been disabled.
        get_core_id_for_cpu_number now translates between both.
      * Implement the Wakeup IPI to wake up an initialized CPU when its scheduler loop is in a HALT state.
      All of this gets "thr_hello" to work on machines with >= 3 CPUs and the Go application "server" to get much further.
      Work is still needed to enable round-robin scheduling on machines with more tasks than CPUs.
  29. 25 Jan, 2018 2 commits
  30. 24 Jan, 2018 1 commit
  31. 23 Jan, 2018 1 commit
    • Colin Finck's avatar
      Turn the HermitCore Rust version into a real OS that can run the C "Hello World" app! :) · d118a26d
      Colin Finck authored
      * Implement a scheduler. This one is based on the eduOS-rs scheduler, but was rewritten to include the following benefits:
         - Multi-core support
         - Task heap support
         - Using Safe Rust Rc<RefCell<Task>> items instead of unsafe Shared<Task> items
         - Reusing the boot stack for the idle task instead of allocating a new one and copying it over in assembly
         - Task arguments
         - Saving/restoring the fs register (for future TLS support) as part of the task state
         - Less duplicated code
         - Future FPU state support
      * Replace the single by a "syscalls" module that groups all syscalls by category:
         - Implement sys_getpid(), sys_getprio(), sys_exit(), sys_sbrk(), and partly sys_write().
         - Fail gracefully in sys_open(), sys_close(), sys_kill(), and sys_signal().
         - Add a sys_spawn() syscall to spawn a new task.
           LwIP now uses it for adding its kernel thread.
         - Add official syscalls for Semaphores and Spinlocks to provide a clean interface to these OS features.
           LwIP and pthread have previously used inlined versions from the HermitCore C version, which resulted in dependencies to internal OS functions.
         - Add a sys_rand() syscall to generate a random 32-bit number using a CPU random number generator if available.
           Otherwise, a Park-Miller-Lehmer Minimal Standard Generator is used. This moves random number logic from LwIP back to the OS.
      * Implement the page fault handler to allocate the Task Heap with Large Pages (2 MiB) on demand.
      * Implement an allocate_aligned() API for the FreeList to allocate a memory block aligned to a desired boundary.
        This is now used when allocating physical memory for a large page.
        You should however still prefer allocate() if BasePageSize alignment is sufficient, because this one doesn't need a filled node pool.
      * Increase the boot stack to 16 KB. 8 KB is no longer enough for the Rust boot process.
      * Remove more headers of the HermitCore C version that are no longer used.
      * Improve readability by checking alignments with the modulo operator.