# x86_64 architecture documentation

This section documents coreboot's x86_64 support. When enabled,
every coreboot stage is built for x86_64, contrary to UEFI's implementation
that only runs some stages in x86_64.
On UEFI the PEI phase, which is x86_32, brings up DRAM and installs
page tables for the x86_64 DXE and BDS phases.

## Toolchain requirements for x86_64 support
* The compiler must support generating code for the *large memory model*
  (-mcmodel=large). It's supported since GCC 4.4.

  Page tables can be used to provide security benefits, such as by marking
  memory as non-executable or removing it entirely. This could be useful
  for SMM to mark regular DRAM as NX.

  The large memory model causes the compiler to emit 64bit addressing
  instructions, which increases code size. In theory, this is roughly
  made up for by the faster execution of the x86_64 code.

* All x86 coreboot stages and payloads must be loaded below 4GiB in
  physical memory. When jumping to the payload coreboot will drop from
  long mode back to protected mode to keep compatibility with these payloads.

## Comparison to UEFI
On UEFI the SEC and PEI phases (similar to coreboot's bootblock and romstage)
are run in x86_32 mode. The following (guessed) reasons are likely:
* There's no need for x86_64 as memory hasn't been trained yet. The whole 4GiB
  address space, including CAR, memory mapped SPI flash and PCI BARs, are
  accessible in x86_32.
* When the EFI specification was written compilers did not support
  *large memory model*, required in CAR when using a 1:1 page mapping
* Code is 20% bigger in x86_64 due to *large memory model* where pointers and
  function calls always use 8 byte addressing. However flash size was very
  limited, compared to today's flash chips, when the EFI spec was written.

## Current software constraints for x86_64 support
The following constraints are coreboot limitations as it was intended to run in
protected mode only. The code still assumes 32bit pointers in some places and thus:
* The high dword of pointers must always be zero.
* All memory returned by malloc must be below 4GiB in physical memory.
* All code that is to be run must be below 4GiB in physical memory.
* CBMEM must reside below 4GiB in physical memory.

Any software within coreboot must not access memory resources above 4GiB until
end of BS_DEV_RESOURCES in ramstage. Only at that point the full memory map is
known and identity mapped.

## Supported boards
On the supported boards you can enable x86_64 compilation by setting the
Kconfig `USE_X86_64_SUPPORT`. This config option is enabled if the SOC/CPU
selects `HAVE_X86_64_SUPPORT`.

## Protected mode wrappers
On some platforms binary blobs are run to initialize parts of the hardware.
When these binary blobs have been compiled for x86_32, then coreboot must
switch to protected mode in order to call and run the blobs. Once the invoked
blobs finish running, coreboot needs to switch back to long mode.

Since every BLOB is different a SoC must be enabled to support x86_64 mode
by providing the correct wrapper around the x86_32 BLOBs.

## TODO
* Support more platforms
* Fix running VGA Option ROMs
* Fix running MRC.bin (Sandy Bridge / Haswell boards)
* Identity map memory above 4GiB in ramstage
* Fine grained page tables for SMM:
   * Must not have execute and write permissions for the same page.
   * Must only allow TSEG pages to be marked as executable.
   * Must reside in SMRAM.
   * Must be placed together with SMM rmodule.
* Support 64bit PCI BARs above 4GiB
* Jump to compatible payloads in long mode

## Porting other boards
* Fix compilation errors
* Test how well CAR works with x86_64 and paging
* Improve mode switches
* Test libgfxinit / VGA Option ROMs / FSP

## Known bugs on real hardware

According to Intel x86_64 mode hasn't been validated in CAR environments.
However, coreboot developers working on x86_64 support have tried this on
various Intel platforms, and so far haven't found any issues with CAR when
running in x86_64 mode.

## Known bugs on KVM enabled QEMU

The `x86_64` reference code runs fine in QEMU's soft-cpu, but has serious issues
when using KVM mode on some machines. This is due to various mechanisms trying
to accelerate the code execution.

Known issues in QEMU:
* After entering long mode, the FPU doesn't work anymore, including accessing
  MMX registers. It works fine before entering long mode. It works fine when
  switching back to protected mode. Other registers, like SSE registers, are
  working fine.
* Reading from virtual memory, when the page tables are stored in ROM, causes
  the MMU to abort the "page table walking" mechanism when the lower address
  bits of the virtual address to be translated have a specific pattern.
  Instead of loading the correct physical page, the one containing the
  page tables in ROM will be loaded and used, which breaks code and data as
  the page table doesn't contain the expected data. This in turn leads to
  undefined behaviour whenever the 'wrong' address is being read.
* Disabling paging in compatibility mode crashes the CPU.
* Returning from long mode to compatibility mode crashes the CPU.
* Entering long mode crashes on AMD host platforms.