This patch introduces a new build option named COLD_BOOT_SINGLE_CPU,
which allows platforms that only release a single CPU out of reset to
slightly optimise their cold boot code, both in terms of code size
and performance.
COLD_BOOT_SINGLE_CPU defaults to 0, which assumes that the platform
may release several CPUs out of reset. In this case, the cold reset
code needs to coordinate all CPUs via the usual primary/secondary
CPU distinction.
If a platform guarantees that only a single CPU will ever be released
out of reset, there is no need to arbitrate execution ; the notion of
primary and secondary CPUs itself no longer exists. Such platforms
may set COLD_BOOT_SINGLE_CPU to 1 in order to compile out the
primary/secondary CPU identification in the cold reset code.
All ARM standard platforms can release several CPUs out of reset
so they use COLD_BOOT_SINGLE_CPU=0. However, on CSS platforms like
Juno, bringing up more than one CPU at reset should only be attempted
when booting an EL3 payload, as it is not fully supported in the
normal boot flow.
For platforms using COLD_BOOT_SINGLE_CPU=1, the following 2 platform
APIs become optional:
- plat_secondary_cold_boot_setup();
- plat_is_my_cpu_primary().
The Porting Guide has been updated to reflect that.
User Guide updated as well.
Change-Id: Ic5b474e61b7aec1377d1e0b6925d17dfc376c46b
On the ARMv8 architecture, cache maintenance operations by set/way on the last
level of integrated cache do not affect the system cache. This means that such a
flush or clean operation could result in the data being pushed out to the system
cache rather than main memory. Another CPU could access this data before it
enables its data cache or MMU. Such accesses could be serviced from the main
memory instead of the system cache. If the data in the sysem cache has not yet
been flushed or evicted to main memory then there could be a loss of
coherency. The only mechanism to guarantee that the main memory will be updated
is to use cache maintenance operations to the PoC by MVA(See section D3.4.11
(System level caches) of ARMv8-A Reference Manual (Issue A.g/ARM DDI0487A.G).
This patch removes the reliance of Trusted Firmware on the flush by set/way
operation to ensure visibility of data in the main memory. Cache maintenance
operations by MVA are now used instead. The following are the broad category of
changes:
1. The RW areas of BL2/BL31/BL32 are invalidated by MVA before the C runtime is
initialised. This ensures that any stale cache lines at any level of cache
are removed.
2. Updates to global data in runtime firmware (BL31) by the primary CPU are made
visible to secondary CPUs using a cache clean operation by MVA.
3. Cache maintenance by set/way operations are only used prior to power down.
NOTE: NON-UPSTREAM TRUSTED FIRMWARE CODE SHOULD MAKE EQUIVALENT CHANGES IN
ORDER TO FUNCTION CORRECTLY ON PLATFORMS WITH SUPPORT FOR SYSTEM CACHES.
FixesARM-software/tf-issues#205
Change-Id: I64f1b398de0432813a0e0881d70f8337681f6e9a
This patch introduces a new platform build option, called
PROGRAMMABLE_RESET_ADDRESS, which tells whether the platform has
a programmable or fixed reset vector address.
If the reset vector address is fixed then the code relies on the
platform_get_entrypoint() mailbox mechanism to figure out where
it is supposed to jump. On the other hand, if it is programmable
then it is assumed that the platform code will program directly
the right address into the RVBAR register (instead of using the
mailbox redirection) so the mailbox is ignored in this case.
Change-Id: If59c3b11fb1f692976e1d8b96c7e2da0ebfba308
The attempt to run the CPU reset code as soon as possible after reset
results in highly complex conditional code relating to the
RESET_TO_BL31 option.
This patch relaxes this requirement a little. In the BL1, BL3-1 and
PSCI entrypoints code, the sequence of operations is now as follows:
1) Detect whether it is a cold or warm boot;
2) For cold boot, detect whether it is the primary or a secondary
CPU. This is needed to handle multiple CPUs entering cold reset
simultaneously;
3) Run the CPU init code.
This patch also abstracts the EL3 registers initialisation done by
the BL1, BL3-1 and PSCI entrypoints into common code.
This improves code re-use and consolidates the code flows for
different types of systems.
NOTE: THE FUNCTION plat_secondary_cold_boot() IS NOW EXPECTED TO
NEVER RETURN. THIS PATCH FORCES PLATFORM PORTS THAT RELIED ON THE
FORMER RETRY LOOP AT THE CALL SITE TO MODIFY THEIR IMPLEMENTATION.
OTHERWISE, SECONDARY CPUS WILL PANIC.
Change-Id: If5ecd74d75bee700b1bd718d23d7556b8f863546
In order for the symbol table in the ELF file to contain the size of
functions written in assembly, it is necessary to report it to the
assembler using the .size directive.
To fulfil the above requirements, this patch introduces an 'endfunc'
macro which contains the .endfunc and .size directives. It also adds
a .func directive to the 'func' assembler macro.
The .func/.endfunc have been used so the assembler can fail if
endfunc is omitted.
FixesARM-Software/tf-issues#295
Change-Id: If8cb331b03d7f38fe7e3694d4de26f1075b278fc
Signed-off-by: Kévin Petit <kevin.petit@arm.com>
The cpu-ops pointer was initialized before enabling the data cache in the cold
and warm boot paths. This required a DCIVAC cache maintenance operation to
invalidate any stale cache lines resident in other cpus.
This patch moves this initialization to the bl31_arch_setup() function
which is always called after the data cache and MMU has been enabled.
This change removes the need:
1. for the DCIVAC cache maintenance operation.
2. to initialise the CPU ops upon resumption from a PSCI CPU_SUSPEND
call since memory contents are always preserved in this case.
Change-Id: Ibb2fa2f7460d1a1f1e721242025e382734c204c6
This patch adds support to call the reset_handler() function in BL3-1 in the
cold and warm boot paths when another Boot ROM reset_handler() has already run.
This means the BL1 and BL3-1 versions of the CPU and platform specific reset
handlers may execute different code to each other. This enables a developer to
perform additional actions or undo actions already performed during the first
call of the reset handlers e.g. apply additional errata workarounds.
Typically, the reset handler will be first called from the BL1 Boot ROM. Any
additional functionality can be added to the reset handler when it is called
from BL3-1 resident in RW memory. The constant FIRST_RESET_HANDLER_CALL is used
to identify whether this is the first version of the reset handler code to be
executed or an overridden version of the code.
The Cortex-A57 errata workarounds are applied only if they have not already been
applied.
FixesARM-software/tf-issue#275
Change-Id: Id295f106e4fda23d6736debdade2ac7f2a9a9053
This patch extends the build option `USE_COHERENT_MEMORY` to
conditionally remove coherent memory from the memory maps of
all boot loader stages. The patch also adds necessary
documentation for coherent memory removal in firmware-design,
porting and user guides.
FixesARM-Software/tf-issues#106
Change-Id: I260e8768c6a5c2efc402f5804a80657d8ce38773
This patch gathers miscellaneous minor fixes to the documentation, and comments
in the source code.
Change-Id: I631e3dda5abafa2d90f464edaee069a1e58b751b
Co-Authored-By: Soby Mathew <soby.mathew@arm.com>
Co-Authored-By: Dan Handley <dan.handley@arm.com>
This patch adds CPU core and cluster power down sequences to the CPU specific
operations framework introduced in a earlier patch. Cortex-A53, Cortex-A57 and
generic AEM sequences have been added. The latter is suitable for the
Foundation and Base AEM FVPs. A pointer to each CPU's operations structure is
saved in the per-cpu data so that it can be easily accessed during power down
seqeunces.
An optional platform API has been introduced to allow a platform to disable the
Accelerator Coherency Port (ACP) during a cluster power down sequence. The weak
definition of this function (plat_disable_acp()) does not take any action. It
should be overriden with a strong definition if the ACP is present on a
platform.
Change-Id: I8d09bd40d2f528a28d2d3f19b77101178778685d
This patch introduces a framework which will allow CPUs to perform
implementation defined actions after a CPU reset, during a CPU or cluster power
down, and when a crash occurs. CPU specific reset handlers have been implemented
in this patch. Other handlers will be implemented in subsequent patches.
Also moved cpu_helpers.S to the new directory lib/cpus/aarch64/.
Change-Id: I1ca1bade4d101d11a898fb30fea2669f9b37b956
This patch disables routing of external aborts from lower exception levels to
EL3 and ensures that a SError interrupt generated as a result of execution in
EL3 is taken locally instead of a lower exception level.
The SError interrupt is enabled in the TSP code only when the operation has not
been directly initiated by the normal world. This is to prevent the possibility
of an asynchronous external abort which originated in normal world from being
taken when execution is in S-EL1.
FixesARM-software/tf-issues#153
Change-Id: I157b996c75996d12fd86d27e98bc73dd8bce6cd5
The purpose of platform_is_primary_cpu() is to determine after reset
(BL1 or BL3-1 with reset handler) if the current CPU must follow the
cold boot path (primary CPU), or wait in a safe state (secondary CPU)
until the primary CPU has finished the system initialization.
This patch removes redundant calls to platform_is_primary_cpu() in
subsequent bootloader entrypoints since the reset handler already
guarantees that code is executed exclusively on the primary CPU.
Additionally, this patch removes the weak definition of
platform_is_primary_cpu(), so the implementation of this function
becomes mandatory. Removing the weak symbol avoids other
bootloaders accidentally picking up an invalid definition in case the
porting layer makes the real function available only to BL1.
The define PRIMARY_CPU is no longer mandatory in the platform porting
because platform_is_primary_cpu() hides the implementation details
(for instance, there may be platforms that report the primary CPU in
a system register). The primary CPU definition in FVP has been moved
to fvp_def.h.
The porting guide has been updated accordingly.
FixesARM-software/tf-issues#219
Change-Id: If675a1de8e8d25122b7fef147cb238d939f90b5e
This patch reworks the crash reporting mechanism to further
optimise the stack and code size. The reporting makes use
of assembly console functions to avoid calling C Runtime
to report the CPU state. The crash buffer requirement is
reduced to 64 bytes with this implementation. The crash
buffer is now part of per-cpu data which makes retrieving
the crash buffer trivial.
Also now panic() will use crash reporting if
invoked from BL3-1.
FixesARM-software/tf-issues#199
Change-Id: I79d27a4524583d723483165dc40801f45e627da5
This patch reworks the manner in which the M,A, C, SA, I, WXN & EE bits of
SCTLR_EL3 & SCTLR_EL1 are managed. The EE bit is cleared immediately after reset
in EL3. The I, A and SA bits are set next in EL3 and immediately upon entry in
S-EL1. These bits are no longer managed in the blX_arch_setup() functions. They
do not have to be saved and restored either. The M, WXN and optionally the C
bit are set in the enable_mmu_elX() function. This is done during both the warm
and cold boot paths.
FixesARM-software/tf-issues#226
Change-Id: Ie894d1a07b8697c116960d858cd138c50bc7a069
This patch reworks the cold boot path across the BL1, BL2, BL3-1 and BL3-2 boot
loader stages to not use stacks allocated in coherent memory for early platform
setup and enabling the MMU. Stacks allocated in normal memory are used instead.
Attributes for stack memory change from nGnRnE when the MMU is disabled to
Normal WBWA Inner-shareable when the MMU and data cache are enabled. It is
possible for the CPU to read stale stack memory after the MMU is enabled from
another CPUs cache. Hence, it is unsafe to turn on the MMU and data cache while
using normal stacks when multiple CPUs are a part of the same coherency
domain. It is safe to do so in the cold boot path as only the primary cpu
executes it. The secondary cpus are in a quiescent state.
This patch does not remove the allocation of coherent stack memory. That is done
in a subsequent patch.
Change-Id: I12c80b7c7ab23506d425c5b3a8a7de693498f830
This patch reworks FVP specific code responsible for determining
the entry point information for BL3-2 and BL3-3 stages when BL3-1
is configured as the reset handler.
Change-Id: Ia661ff0a6a44c7aabb0b6c1684b2e8d3642d11ec
The crash reporting support and early initialisation of the
cpu_data allow the runtime_exception vectors to be used from
the start in BL3-1, removing the need for the additional
early_exception vectors and 2KB of code from BL3-1.
Change-Id: I5f8997dabbaafd8935a7455910b7db174a25d871
This patch prepares the per-cpu pointer cache for wider use by:
* renaming the structure to cpu_data and placing in new header
* providing accessors for this CPU, or other CPUs
* splitting the initialization of the TPIDR pointer from the
initialization of the cpu_data content
* moving the crash stack initialization to a crash stack function
* setting the TPIDR pointer very early during boot
Change-Id: Icef9004ff88f8eb241d48c14be3158087d7e49a3
This patch adds a common handler for FIQ and IRQ exceptions in the
BL3-1 runtime exception vector table. This function determines the
interrupt type and calls its handler. A crash is reported if an
inconsistency in the interrupt management framework is detected. In
the event of a spurious interrupt, execution resumes from the
instruction where the interrupt was generated.
This patch also removes 'cm_macros.S' as its contents have been moved
to 'runtime_exceptions.S'
Change-Id: I3c85ecf8eaf43a3fac429b119ed0bd706d2e2093
This change adds optional reset vector support to BL3-1
which means BL3-1 entry point can detect cold/warm boot,
initialise primary cpu, set up cci and mail box.
When using BL3-1 as a reset vector it is assumed that
the BL3-1 platform code can determine the location of
the BL3-2 images, or load them as there are no parameters
that can be passed to BL3-1 at reset.
It also fixes the incorrect initialisation of mailbox
registers on the FVP platform
This feature can be enabled by building the code with
make variable RESET_TO_BL31 set as 1
FixesARM-software/TF-issues#133FixesARM-software/TF-issues#20
Change-Id: I4e23939b1c518614b899f549f1e8d412538ee570
This patch is based on spec published at
https://github.com/ARM-software/tf-issues/issues/133
It rearranges the bl31_args struct into
bl31_params and bl31_plat_params which provide the
information needed for Trusted firmware and platform
specific data via x0 and x1
On the FVP platform BL3-1 params and BL3-1 plat params
and its constituents are stored at the start of TZDRAM.
The information about memory availability and size for
BL3-1, BL3-2 and BL3-3 is moved into platform specific data.
Change-Id: I8b32057a3d0dd3968ea26c2541a0714177820da9
This patch reworks the handover interface from: BL1 to BL2 and
BL2 to BL3-1. It removes the raise_el(), change_el(), drop_el()
and run_image() functions as they catered for code paths that were
never exercised.
BL1 calls bl1_run_bl2() to jump into BL2 instead of doing the same
by calling run_image(). Similarly, BL2 issues the SMC to transfer
execution to BL3-1 through BL1 directly. Only x0 and x1 are used
to pass arguments to BL31. These arguments and parameters for
running BL3-1 are passed through a reference to a
'el_change_info_t' structure. They were being passed value in
general purpose registers earlier.
Change-Id: Id4fd019a19a9595de063766d4a66295a2c9307e1
Previously exception handlers in BL3-1, X19-X29 were not saved
and restored on every SMC/trap into EL3. Instead these registers
were 'saved as needed' as a side effect of the A64 ABI used by the C
compiler.
That approach failed when world switching but was not visible
with the TSP/TSPD code because the TSP is 64-bit, did not
clobber these registers when running and did not support pre-emption
by normal world interrupts. These scenarios showed
that the values in these registers can be passed through a world
switch, which broke the normal and trusted world assumptions
about these registers being preserved.
The Ideal solution saves and restores these registers when a
world switch occurs - but that type of implementation is more complex.
So this patch always saves and restores these registers on entry and
exit of EL3.
FixesARM-software/tf-issues#141
Change-Id: I9a727167bbc594454e81cf78a97ca899dfb11c27
Instead of using the system register helper functions to read
or write system registers, assembler coded functions should
use MRS/MSR instructions. This results in faster and more
compact code.
This change replaces all usage of the helper functions with
direct register accesses.
Change-Id: I791d5f11f257010bb3e6a72c6c5ab8779f1982b3
The current code does not always use data and instruction
barriers as required by the architecture and frequently uses
barriers excessively due to their inclusion in all of the
write_*() helper functions.
Barriers should be used explicitly in assembler or C code
when modifying processor state that requires the barriers in
order to enable review of correctness of the code.
This patch removes the barriers from the helper functions and
introduces them as necessary elsewhere in the code.
PORTING NOTE: check any port of Trusted Firmware for use of
system register helper functions for reliance on the previous
barrier behaviour and add explicit barriers as necessary.
FixesARM-software/tf-issues#92
Change-Id: Ie63e187404ff10e0bdcb39292dd9066cb84c53bf
Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.
Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.
FixesARM-software/tf-issues#31
Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e
Make codebase consistent in its use of #include "" syntax for
user includes and #include <> syntax for system includes.
FixesARM-software/tf-issues#65
Change-Id: If2f7c4885173b1fd05ac2cde5f1c8a07000c7a33
This extends the --gc-sections behaviour to the many assembler
support functions in the firmware images by placing each function
into its own code section. This is achieved by creating a 'func'
macro used to declare each function label.
FixesARM-software/tf-issues#80
Change-Id: I301937b630add292d2dec6d2561a7fcfa6fec690
This patch adds the following support to the BL3-1 stage:
1. BL3-1 allows runtime services to specify and determine the security
state of the next image after BL3-1. This has been done by adding
the `bl31_set_next_image_type()` & `bl31_get_next_image_type()`
apis. The default security state is non-secure. The platform api
`bl31_get_next_image_info()` has been modified to let the platform
decide which is the next image in the desired security state.
2. BL3-1 exports the `bl31_prepare_next_image_entry()` function to
program entry into the target security state. It uses the apis
introduced in 1. to do so.
3. BL3-1 reads the information populated by BL2 about the BL3-2 image
into its internal data structures.
4. BL3-1 introduces a weakly defined reference `bl32_init()` to allow
initialisation of a BL3-2 image. A runtime service like the Secure
payload dispatcher will define this function if present.
Change-Id: Icc46dcdb9e475ce6575dd3f9a5dc7a48a83d21d1
This patch reworks BL2 to BL3-1 hand over interface by introducing a
composite structure (bl31_args) that holds the superset of information
that needs to be passed from BL2 to BL3-1.
- The extents of secure memory available to BL3-1
- The extents of memory available to BL3-2 (not yet implemented) and
BL3-3
- Information to execute BL3-2 (not yet implemented) and BL3-3 images
This patch also introduces a new platform API (bl2_get_bl31_args_ptr)
that needs to be implemented by the platform code to export reference to
bl31_args structure which has been allocated in platform-defined memory.
The platform will initialize the extents of memory available to BL3-3
during early platform setup in bl31_args structure. This obviates the
need for bl2_get_ns_mem_layout platform API.
BL2 calls the bl2_get_bl31_args_ptr function to get a reference to
bl31_args structure. It uses the 'bl33_meminfo' field of this structure
to load the BL3-3 image. It sets the entry point information for the
BL3-3 image in the 'bl33_image_info' field of this structure. The
reference to this structure is passed to the BL3-1 image.
Also fixes issue ARM-software/tf-issues#25
Change-Id: Ic36426196dd5ebf89e60ff42643bed01b3500517
This patch uses the reworked exception handling support to handle
runtime service requests through SMCs following the SMC calling
convention. This is a giant commit since all the changes are
inter-related. It does the following:
1. Replace the old exception handling mechanism with the new one
2. Enforce that SP_EL0 is used C runtime stacks.
3. Ensures that the cold and warm boot paths use the 'cpu_context'
structure to program an ERET into the next lower EL.
4. Ensures that SP_EL3 always points to the next 'cpu_context'
structure prior to an ERET into the next lower EL
5. Introduces a PSCI SMC handler which completes the use of PSCI as a
runtime service
Change-Id: I661797f834c0803d2c674d20f504df1b04c2b852
Co-authored-by: Achin Gupta <achin.gupta@arm.com>
This patch ensures that VBAR_EL3 points to the simple stack-less
'early_exceptions' when the C runtime stack is not correctly setup to
use the more complex 'runtime_exceptions'. It is initialised to
'runtime_exceptions' once this is done.
This patch also moves all exception vectors into a '.vectors' section
and modifies linker scripts to place all such sections together. This
will minimize space wastage from alignment restrictions.
Change-Id: I8c3e596ea3412c8bd582af9e8d622bb1cb2e049d
Ctags seem to have a problem with generating tags for assembler symbols
when a comment immediately follows an assembly label.
This patch inserts a single space character between the label
definition and the following comments to help ctags.
The patch is generated by the command:
git ls-files -- \*.S | xargs sed -i 's/^\([^:]\+\):;/\1: ;/1'
Change-Id: If7a3c9d0f51207ea033cc8b8e1b34acaa0926475
Traps when accessing architectural features are disabled by clearing bits
in CPTR_EL3 during early boot, including accesses to floating point
registers. The value of this register was previously undetermined, causing
unwanted traps to EL3. Future EL3 code (for example, context save/restore
code) may use floating point registers, although they are not used by current
code.
Also, the '-mgeneral-regs-only' flag is enabled in the GCC settings to
prevent generation of code that uses floating point registers.
Change-Id: I9a03675f6387bbbee81a6f2c9ccf81150db03747
ns_entry_info used to be a per-cpu array. This is a waste of space
because it is only accessed by the primary CPU on the cold boot path.
This patch reduces ns_entry_info to a single-cpu area.
Change-Id: I647c70c4e76069560f1aaad37a1d5910f56fba4c
- Add instructions for contributing to ARM Trusted Firmware.
- Update copyright text in all files to acknowledge contributors.
Change-Id: I9311aac81b00c6c167d2f8c889aea403b84450e5
This patch makes sure the C runtime environment is properly
initialised before executing any C code.
- Zero-initialise NOBITS sections (e.g. the bss section).
- Relocate BL1 data from ROM to RAM.
Change-Id: I0da81b417b2f0d1f7ef667cc5131b1e47e22571f
- Check at link-time that bootloader images will fit in memory
at run time and that they won't overlap each other.
- Remove text and rodata orphan sections.
- Define new linker symbols to remove the need for platform setup
code to know the order of sections.
- Reduce the size of the raw binary images by cutting some sections
out of the disk image and allocating them at load time, whenever
possible.
- Rework alignment constraints on sections.
- Remove unused linker symbols.
- Homogenize linker symbols names across all BLs.
- Add some comments in the linker scripts.
Change-Id: I47a328af0ccc7c8ab47fcc0dc6e7dd26160610b9
blx_plat_arch_setup() should only perform platform-specific
architectural setup, e.g. enabling the MMU. This patch moves
generic architectural setup code out of blx_plat_arch_setup().
Change-Id: I4ccf56b8c4a2fa84909817779a2d97a14aaafab6
Sandrine Bailleux
Achin Gupta
Kévin Petit
Vikram Kanigiri
Yatharth Kochar
Soby Mathew
Juan Castillo
Andrew Thoelke
Dan Handley
Jeenu Viswambharan
Harry Liebel