The Training-Mode Boot Flow
fstart coreboot firmware hardening FSPModern PCs run a lot of proprietary firmware on their main CPUs. In the
past, it was sometimes possible to have all the initialisation steps in
open-source, for instance with the coreboot firmware framework. At some
point however, the initialisation process became more complex, and less
documented. Then, alas, open-source firmware fell behind.
A huge part of this complexity comes with the DRAM training. Modern DDR
DRAM needs very precise timings, and the time a signal takes can differ
from hardware unit to hardware unit. Hence, special algorithms are used
to detect working timings at runtime. These timings are often stored in
the firmware flash, and are picked up on subsequent boots. In coreboot,
this is called MRC cache, named after Intel's Memory Reference Code.
On AMD systems, the DRAM training and potentially more of the initiali-
sation is done by firmware on an auxiliary processor. When the main x86
CPU comes out of reset, DRAM is already available. And thanks to AMD's
openSIL effort, everything running on the x86 cores can be open-source
again. The separation of the more complex, more proprietary steps from
the initialisation that runs on the x86 seems to pay off. Let's explore
a different separation that runs on a single processor and utilises the
idea of cached training data as a barrier, the Training-Mode Boot Flow.
Open-source software is prevailing, because of its flexibility, transparency and trustworthiness. In firmware, OTOH, we still have proprietary components, often because of IP concerns. Mixing proprietary and open-source firmware components on a single boot path has several disadvantages.
- Separation of responsibilities is often unclear.
- Proprietary parts need to be fully trusted, even though they don’t share the transparency of open-source parts.
- Proprietary parts often can’t be modernised, e.g. switching open-source parts to a memory-safe language alone doesn’t provide the usual guarantees.
- ABIs are required, lower flexibility, and are rarely vendor agnostic.
To name a few. The Training-Mode Boot Flow (TBF) offers separate boot paths, where the regular boot mode can be fully open-source, transparent, and security hardened. The regular mode is assisted by the training mode when necessary. However, the initial bootblock decides which mode will run. Switching between the modes is only possible with a system reset.
(bootblock)
+——————————————————————————————————————————————————————+
| Boot-Mode Arbitrator |
+——————————————————————————————————————————————————————+
|| ||
|| ____________ ||
\/ / `, \/
+————————————————+ |`————————————'| +————————————————+
| Training Mode | | | | Regular Boot |
|----------------| |`————————————'| |----------------|
| train DRAM |<-------| static DT |------->|. |
| | | |`————————————'| | :- merge DTs |
| v | .--->| dynamic DT |------->|' | |
| other prep. | / `————————————' | v |
| `--------|-' (flash) | configure hw |
| | | | |
+————————————————+ | v |
|| | write tables |
/\ reset || | | |
\\_________// | v |
`---------' | |
+————————————————+
||
||
\/
+————————————————+
| Boot-Loader |
|----------------|
! !
: :
Both modes are configured by a static devicetree in flash. Training results are communicated via a dynamic devicetree that resides in flash as well, much like an MRC cache. The Flattened Device Tree format could be used here, but other, firmware-framework specific formats would work as well.
Training Mode
The purpose of the training mode is only to train DRAM and perform other, complex, proprietary tasks. It is configured by a static devicetree inside the boot flash. The results of the training mode are written into the flash as the dynamic devicetree. It ends immediately after this, with a cold reset.
This doesn’t only separate the complex algorithms, but can also serve as a security barrier: If a memory-safe language is used for the regular boot mode, parsing of the dynamic DT should be safe, even if the training mode is compromised. Further checks can be performed on devicetree values before they are deployed to the hardware. If possible, it should be enforced that the training mode never continues the boot process.
Limiting the training-mode to this special purpose also reduces the attack surface. It only has to read the static DT from flash, where it could even be read only, and communicate sparsely with potentially untrusted hardware (e.g. DRAM SPDs).
Regular Boot Mode
The regular boot mode starts by reading the static and the dynamic devicetrees from flash and merges them together. From then on, hardware configuration will get all information from a single devicetree, so the code doesn’t need to know where the information came from.
Otherwise the regular boot mode can follow usual firmware frameworks in architecture, for instance Fstart or coreboot. Except that they wouldn’t have to contain code for the chores anymore that the training mode handles. This also means that firmware binaries can become cleaner: No more repetitive checking if we are on a training / normal / fast path. There would only be a single path (assuming there is no suspend-to-RAM).
Boot-Mode Arbitrator
The boot-mode arbitrator is the first component that runs after reset. By default it runs the regular boot mode. Whenever the regular boot mode detects that it requires (re-)training, it can request this from the arbitrator (for instance via a CMOS flag) and reset the system. The arbitrator would normally handle the request by simply running the training mode. It can also implement a counter for rate-limiting the training attempts, to avoid flash wear in error cases.
The arbitrator is also responsible for early security and serves as a trust anchor. Ideally, it would use a hardware root-of-trust directly (e.g. burned-in hash of a verification key). For measured boot, the arbitrator can be the S-RTM.
As it wouldn’t contain any proprietary technology and needs to be trusted, the arbitrator should be implemented as open-source, in a memory safe language. Because of its limited scope and expected size, it can also be considered for formal verification.
Training-Mode Enforcement
Assuming we can enforce that the training mode always resets the system and cannot boot into an OS for instance, an attacker who exploits a flaw in the training-mode code couldn’t pretend an untampered system. Such an enforcement would most likely require changes in silicon or other firmware of an SoC. For instance, by adding a watchdog that cannot be reset and is set up by the boot-mode arbitrator.
Other measures that could be investigated:
- Globally disabling DMA during training mode.
- Flash read protection: Limiting read access to configuration data and the training mode’s own code would also limit an attacker’s options.
- The boot-mode arbitrator could probably implement a hypervisor.
Conclusion
Compared to a monolithic boot flow where the binaries serve multiple purposes, the Training-Mode Boot Flow brings several advantages:
- Hardening can focus on a much lighter regular boot mode.
- Reference code doesn’t need to be re-written in one go, e.g. to transition to a modern programming language. Non-proprietary things can be handled by the regular boot mode from the beginning. No invasive changes to current reference code would be necessary: It could be wrapped to produce the dynamic devicetree.
- Responsibilities are clearly separated. The regular boot mode configures the hardware with information from the combined devicetree. The training mode runs proprietary or complex algorithms and provides additional devicetree data.
- Using a merged devicetree in the regular boot mode allows to change where specific devicetree information comes from, without any need for changes in the regular boot mode. For instance hard-coded vs. trained DRAM configuration.
Overall, the Training-Mode Boot Flow can provide a trustworthy boot experience, without the need to trust all the components involved. It allows to protect IP, without the usual high costs to integrate everything into a single boot process.