This patch adds CPU core and cluster power down sequences to the CPU specific
operations framework introduced in a earlier patch. Cortex-A53, Cortex-A57 and
generic AEM sequences have been added. The latter is suitable for the
Foundation and Base AEM FVPs. A pointer to each CPU's operations structure is
saved in the per-cpu data so that it can be easily accessed during power down
seqeunces.
An optional platform API has been introduced to allow a platform to disable the
Accelerator Coherency Port (ACP) during a cluster power down sequence. The weak
definition of this function (plat_disable_acp()) does not take any action. It
should be overriden with a strong definition if the ACP is present on a
platform.
Change-Id: I8d09bd40d2f528a28d2d3f19b77101178778685d
This patch implements the following cleanups in PSCI generic code:
1. It reworks the affinity level specific handlers in the PSCI implementation
such that.
a. Usage of the 'rc' local variable is restricted to only where it is
absolutely needed
b. 'plat_state' local variable is defined only when a direct invocation of
plat_get_phys_state() does not suffice.
c. If a platform handler is not registered then the level specific handler
returns early.
2. It limits the use of the mpidr_aff_map_nodes_t typedef to declaration of
arrays of the type instead of using it in function prototypes as well.
3. It removes dangling declarations of __psci_cpu_off() and
__psci_cpu_suspend(). The definitions of these functions were removed in
earlier patches.
Change-Id: I51e851967c148be9c2eeda3a3c41878f7b4d6978
This patch adds APIs to find, save and retrieve the highest affinity level which
will enter or exit from the physical OFF state during a PSCI power management
operation. The level is stored in per-cpu data.
It then reworks the PSCI implementation to perform cache maintenance only
when the handler for the highest affinity level to enter/exit the OFF state is
called.
For example. during a CPU_SUSPEND operation, state management is done prior to
calling the affinity level specific handlers. The highest affinity level which
will be turned off is determined using the psci_find_max_phys_off_afflvl()
API. This level is saved using the psci_set_max_phys_off_afflvl() API. In the
code that does generic handling for each level, prior to performing cache
maintenance it is first determined if the current affinity level matches the
value returned by psci_get_max_phys_off_afflvl(). Cache maintenance is done if
the values match.
This change allows the last CPU in a cluster to perform cache maintenance
independently. Earlier, cache maintenance was started in the level 0 handler and
finished in the level 1 handler. This change in approach will facilitate
implementation of tf-issues#98.
Change-Id: I57233f0a27b3ddd6ddca6deb6a88b234525b0ae6
This patch pulls out state management from the affinity level specific handlers
into the top level functions specific to the operation
i.e. psci_afflvl_suspend(), psci_afflvl_on() etc.
In the power down path this patch will allow an affinity instance at level X to
determine the state that an affinity instance at level X+1 will enter before the
level specific handlers are called. This will be useful to determine whether a
CPU is the last in the cluster during a suspend/off request and so on.
Similarly, in the power up path this patch will allow an affinity instance at
level X to determine the state that an affinity instance at level X+1 has
emerged from, even after the level specific handlers have been called. This will
be useful in determining whether a CPU is the first in the cluster during a
on/resume request and so on.
As before, while powering down, state is updated before the level specific
handlers are invoked so that they can perform actions based upon their target
state. While powering up, state is updated after the level specific handlers have
been invoked so that they can perform actions based upon the state they emerged
from.
Change-Id: I40fe64cb61bb096c66f88f6d493a1931243cfd37
This patch adds a structure defined by the PSCI service to the per-CPU data
array. The structure is used to save the 'power_state' parameter specified
during a 'cpu_suspend' call on the current CPU. This parameter was being saved
in the cpu node in the PSCI topology tree earlier.
The existing API to return the state id specified during a PSCI CPU_SUSPEND call
i.e. psci_get_suspend_stateid(mpidr) has been renamed to
psci_get_suspend_stateid_by_mpidr(mpidr). The new psci_get_suspend_stateid() API
returns the state id of the current cpu.
The psci_get_suspend_afflvl() API has been changed to return the target affinity
level of the current CPU. This was specified using the 'mpidr' parameter in the
old implementation.
The behaviour of the get_power_on_target_afflvl() has been tweaked such that
traversal of the PSCI topology tree to locate the affinity instance node for the
current CPU is done only in the debug build as it is an expensive operation.
Change-Id: Iaad49db75abda471f6a82d697ee6e0df554c4caf
This patch adds support for SYSTEM_OFF and SYSTEM_RESET PSCI
operations. A platform should export handlers to complete the
requested operation. The FVP port exports fvp_system_off() and
fvp_system_reset() as an example.
If the SPD provides a power management hook for system off and
system reset, then the SPD is notified about the corresponding
operation so it can do some bookkeeping. The TSPD exports
tspd_system_off() and tspd_system_reset() for that purpose.
Versatile Express shutdown and reset methods have been removed
from the FDT as new PSCI sys_poweroff and sys_reset services
have been added. For those kernels that do not support yet these
PSCI services (i.e. GICv3 kernel), the original dtsi files have
been renamed to *-no_psci.dtsi.
FixesARM-software/tf-issues#218
Change-Id: Ic8a3bf801db979099ab7029162af041c4e8330c8
* Move TSP platform porting functions to new file:
include/bl32/tsp/platform_tsp.h.
* Create new TSP_IRQ_SEC_PHY_TIMER definition for use by the generic
TSP interrupt handling code, instead of depending on the FVP
specific definition IRQ_SEC_PHY_TIMER.
* Rename TSP platform porting functions from bl32_* to tsp_*, and
definitions from BL32_* to TSP_*.
* Update generic TSP code to use new platform porting function names
and definitions.
* Update FVP port accordingly and move all TSP source files to:
plat/fvp/tsp/.
* Update porting guide with above changes.
Note: THIS CHANGE REQUIRES ALL PLATFORM PORTS OF THE TSP TO
BE UPDATED
FixesARM-software/tf-issues#167
Change-Id: Ic0ff8caf72aebb378d378193d2f017599fc6b78f
This patch disables routing of external aborts from lower exception levels to
EL3 and ensures that a SError interrupt generated as a result of execution in
EL3 is taken locally instead of a lower exception level.
The SError interrupt is enabled in the TSP code only when the operation has not
been directly initiated by the normal world. This is to prevent the possibility
of an asynchronous external abort which originated in normal world from being
taken when execution is in S-EL1.
FixesARM-software/tf-issues#153
Change-Id: I157b996c75996d12fd86d27e98bc73dd8bce6cd5
This patch adds support for BL3-2 initialization by asynchronous
method where BL3-1 transfers control to BL3-2 using world switch.
After BL3-2 initialization, it transfers control to BL3-3 via SPD
service handler. The SPD service handler initializes the CPU context
to BL3-3 entrypoint depending on the return function indentifier from
TSP initialization.
FixesARM-software/TF-issues#184
Change-Id: I7b135c2ceeb356d3bb5b6a287932e96ac67c7a34
There is no mechanism which allows the TSPD to specify what SPSR to
use when entering BL3-2 instead of BL3-3. This patch divides the
responsibility between tspd_setup() and tspd_init() for initializing
the TSPD and TSP to support the alternate BL3-2 initialization flow
where BL3-1 handsover control to BL3-2 instead of BL3-3.
SPSR generated by TSPD for TSP is preserved due the new division of
labour which fixes#174.
This patch also moves the cpu_context initialization code from
tspd_setup() to tspd_init() immediately before entering the TSP.
Instead tspd_setup() updates the BL3-2 entrypoint info structure
with the state required for initializing the TSP later.
Fixes ARM-software/TF-issues#174
Change-Id: Ida0a8a48d466c71d5b07b8c7f2af169b73f96940
This patch further optimizes the EL3 register state stored in
cpu_context. The 2 registers which are removed from cpu_context are:
* cntfrq_el0 is the system timer register which is writable
only in EL3 and it can be programmed during cold/warm boot. Hence
it need not be saved to cpu_context.
* cptr_el3 controls access to Trace, Floating-point, and Advanced
SIMD functionality and it is programmed every time during cold
and warm boot. The current BL3-1 implementation does not need to
modify the access controls during normal execution and hence
they are expected to remain static.
FixesARM-software/tf-issues#197
Change-Id: I599ceee3b73a7dcfd37069fd41b60e3d397a7b18
Assert a valid security state using the macro sec_state_is_valid().
Replace assert() with panic() in those cases that might arise
because of runtime errors and not programming errors.
Replace panic() with assert() in those cases that might arise
because of programming errors.
FixesARM-software/tf-issues#96
Change-Id: I51e9ef0439fd5ff5e0edfef49050b69804bf14d5
This patch reworks the manner in which the M,A, C, SA, I, WXN & EE bits of
SCTLR_EL3 & SCTLR_EL1 are managed. The EE bit is cleared immediately after reset
in EL3. The I, A and SA bits are set next in EL3 and immediately upon entry in
S-EL1. These bits are no longer managed in the blX_arch_setup() functions. They
do not have to be saved and restored either. The M, WXN and optionally the C
bit are set in the enable_mmu_elX() function. This is done during both the warm
and cold boot paths.
FixesARM-software/tf-issues#226
Change-Id: Ie894d1a07b8697c116960d858cd138c50bc7a069
This patch removes the allocation of memory for coherent stacks, associated
accessor function and some dead code which called the accessor function. It also
updates the porting guide to remove the concept and the motivation behind using
stacks allocated in coherent memory.
FixesARM-software/tf-issues#198
Change-Id: I00ff9a04f693a03df3627ba39727e3497263fc38
This patch uses stacks allocated in normal memory to enable the MMU early in the
warm boot path thus removing the dependency on stacks allocated in coherent
memory. Necessary cache and stack maintenance is performed when a cpu is being
powered down and up. This avoids any coherency issues that can arise from
reading speculatively fetched stale stack memory from another CPUs cache. These
changes affect the warm boot path in both BL3-1 and BL3-2.
The EL3 system registers responsible for preserving the MMU state are not saved
and restored any longer. Static values are used to program these system
registers when a cpu is powered on or resumed from suspend.
Change-Id: I8357e2eb5eb6c5f448492c5094b82b8927603784
This patch adds a 'flags' parameter to each exception level specific function
responsible for enabling the MMU. At present only a single flag which indicates
whether the data cache should also be enabled is implemented. Subsequent patches
will use this flag when enabling the MMU in the warm boot paths.
Change-Id: I0eafae1e678c9ecc604e680851093f1680e9cefa
Many of the interfaces internal to PSCI pass the current CPU
MPIDR_EL1 value from function to function. This is not required,
and with inline access to the system registers is less efficient
than requiring the code to read that register whenever required.
This patch remove the mpidr parameter from the affected interfaces
and reduces code in FVP BL3-1 size by 160 bytes.
Change-Id: I16120a7c6944de37232016d7e109976540775602
The bakery lock code currently expects the calling code to pass
the MPIDR_EL1 of the current CPU.
This is not always done correctly. Also the change to provide
inline access to system registers makes it more efficient for the
bakery lock code to obtain the MPIDR_EL1 directly.
This change removes the mpidr parameter from the bakery lock
interface, and results in a code reduction of 160 bytes for the
ARM FVP port.
FixesARM-software/tf-issues#213
Change-Id: I7ec7bd117bcc9794a0d948990fcf3336a367d543
The array of affinity nodes is currently allocated for 32 entries
with the PSCI_NUM_AFFS value defined in psci.h. This is not enough
for large systems, and will substantially over allocate the array
for small systems.
This patch introduces an optional platform definition
PLATFORM_NUM_AFFS to platform_def.h. If defined this value is
used for PSCI_NUM_AFFS, otherwise a value of two times the number
of CPU cores is used.
The FVP port defines PLATFORM_NUM_AFFS to be 10 which saves
nearly 1.5KB of memory.
FixesARM-software/tf-issues#192
Change-Id: I68e30ac950de88cfbd02982ba882a18fb69c1445
psci_suspend_context is an array of cache-line aligned structures
containing the single power_state integer per cpu. This array is
the only structure indexed by the aff_map_node.data integer.
This patch saves 2KB of BL3-1 memory by placing the CPU
power_state value directly in the aff_map_node structure. As a
result, this value is now never cached and the cache clean when
writing the value is no longer required.
FixesARM-software/tf-issues#195
Change-Id: Ib4c70c8f79eed295ea541e7827977a588a19ef9b
Consolidate all BL3-1 CPU context initialization for cold boot, PSCI
and SPDs into two functions:
* The first uses entry_point_info to initialize the relevant
cpu_context for first entry into a lower exception level on a CPU
* The second populates the EL1 and EL2 system registers as needed
from the cpu_context to ensure correct entry into the lower EL
This patch alters the way that BL3-1 determines which exception level
is used when first entering EL1 or EL2 during cold boot - this is now
fully determined by the SPSR value in the entry_point_info for BL3-3,
as set up by the platform code in BL2 (or otherwise provided to BL3-1).
In the situation that EL1 (or svc mode) is selected for a processor
that supports EL2, the context management code will now configure all
essential EL2 register state to ensure correct execution of EL1. This
allows the platform code to run non-secure EL1 payloads directly
without requiring a small EL2 stub or OS loader.
Change-Id: If9fbb2417e82d2226e47568203d5a369f39d3b0f
The crash reporting support and early initialisation of the
cpu_data allow the runtime_exception vectors to be used from
the start in BL3-1, removing the need for the additional
early_exception vectors and 2KB of code from BL3-1.
Change-Id: I5f8997dabbaafd8935a7455910b7db174a25d871
This patch prepares the per-cpu pointer cache for wider use by:
* renaming the structure to cpu_data and placing in new header
* providing accessors for this CPU, or other CPUs
* splitting the initialization of the TPIDR pointer from the
initialization of the cpu_data content
* moving the crash stack initialization to a crash stack function
* setting the TPIDR pointer very early during boot
Change-Id: Icef9004ff88f8eb241d48c14be3158087d7e49a3
All callers of cm_get_context() pass the calling CPU MPIDR to the
function. Providing a specialised version for the current
CPU results in a reduction in code size and better readability.
The current function has been renamed to cm_get_context_by_mpidr()
and the existing name is now used for the current-CPU version.
The same treatment has been done to cm_set_context(), although
only both forms are used at present in the PSCI and TSPD code.
Change-Id: I91cb0c2f7bfcb950a045dbd9ff7595751c0c0ffb
The SMC handler for PSCI was not correctly handling calls from
secure states, or from AArch32.
This patch completes the handler implementation to correctly
detect secure callers and to clear the top bits in parameters from
AArch32 callers.
The patch also reorganises the switch statement to separate SMC64 and
SMC32 function IDs which allows the compiler to generate much smaller
code for the function.
Change-Id: I36b1ac81fb14253d257255d0477771d54fab0d11
This patch fixes the compilation issue for trusted firmware when the
IMF_READ_INTERRUPT_ID is enabled.
Change-Id: I94ab613b9bc96a7c1935796c674dc42246aaafee
Rename the ic_* platform porting functions to plat_ic_* to be
consistent with the other functions in platform.h. Also rename
bl31_get_next_image_info() to bl31_plat_get_next_image_ep_info()
and remove the duplicate declaration in bl31.h.
Change-Id: I4851842069d3cff14c0a468daacc0a891a7ede84
Previously, the enable_mmu_elX() functions were implicitly part of
the platform porting layer since they were included by generic
code. These functions have been placed behind 2 new platform
functions, bl31_plat_enable_mmu() and bl32_plat_enable_mmu().
These are weakly defined so that they can be optionally overridden
by platform ports.
Also, the enable_mmu_elX() functions have been moved to
lib/aarch64/xlat_tables.c for optional re-use by platform ports.
These functions are tightly coupled with the translation table
initialization code.
FixesARM-software/tf-issues#152
Change-Id: I0a2251ce76acfa3c27541f832a9efaa49135cc1c
Previously, platform.h contained many declarations and definitions
used for different purposes. This file has been split so that:
* Platform definitions used by common code that must be defined
by the platform are now in platform_def.h. The exact include
path is exported through $PLAT_INCLUDES in the platform makefile.
* Platform definitions specific to the FVP platform are now in
/plat/fvp/fvp_def.h.
* Platform API declarations specific to the FVP platform are now
in /plat/fvp/fvp_private.h.
* The remaining platform API declarations that must be ported by
each platform are still in platform.h but this file has been
moved to /include/plat/common since this can be shared by all
platforms.
Change-Id: Ieb3bb22fbab3ee8027413c6b39a783534aee474a
The TSP has a number of entrypoints used by the TSP on different
occasions. These were provided to the TSPD as a table of function
pointers, and required the TSPD to read the entry in the table,
which is in TSP memory, in order to program the exception return
address.
Ideally, the TSPD has no access to the TSP memory.
This patch changes the table of function pointers into a vector
table of single instruction entrypoints. This allows the TSPD to
calculate the entrypoint address instead of read it.
FixesARM-software/tf-issues#160
Change-Id: Iec6e055d537ade78a45799fbc6f43765a4725ad3
Implements support for Non Secure Interrupts preempting the
Standard SMC call in EL1. Whenever an IRQ is trapped in the
Secure world we securely handover to the Normal world
to process the interrupt. The normal world then issues
"resume" smc call to resume the previous interrupted SMC call.
FixesARM-software/tf-issues#105
Change-Id: I72b760617dee27438754cdfc9fe9bcf4cc024858
This patch adds support in the TSPD for registering a handler for
S-EL1 interrupts. This handler ferries the interrupts generated in the
non-secure state to the TSP at 'tsp_fiq_entry'. Support has been added
to the smc handler to resume execution in the non-secure state once
interrupt handling has been completed by the TSP.
There is also support for resuming execution in the normal world if
the TSP receives a EL3 interrupt. This code is currently unused.
Change-Id: I816732595a2635e299572965179f11aa0bf93b69
This patch adds support in the TSP to program the secure physical
generic timer to generate a EL-1 interrupt every half second. It also
adds support for maintaining the timer state across power management
operations. The TSPD ensures that S-EL1 can access the timer by
programming the SCR_EL3.ST bit.
This patch does not actually enable the timer. This will be done in a
subsequent patch once the complete framework for handling S-EL1
interrupts is in place.
Change-Id: I1b3985cfb50262f60824be3a51c6314ce90571bc
This patch adds a common handler for FIQ and IRQ exceptions in the
BL3-1 runtime exception vector table. This function determines the
interrupt type and calls its handler. A crash is reported if an
inconsistency in the interrupt management framework is detected. In
the event of a spurious interrupt, execution resumes from the
instruction where the interrupt was generated.
This patch also removes 'cm_macros.S' as its contents have been moved
to 'runtime_exceptions.S'
Change-Id: I3c85ecf8eaf43a3fac429b119ed0bd706d2e2093
This patch adds an API to write to any bit in the SCR_EL3 member of
the 'cpu_context' structure of the current CPU for a specified
security state. This API will be used in subsequent patches which
introduce interrupt management in EL3 to specify the interrupt routing
model when execution is not in EL3.
It also renames the cm_set_el3_elr() function to cm_set_elr_el3()
which is more in line with the system register name being targeted by
the API.
Change-Id: I310fa7d8f827ad3f350325eca2fb28cb350a85ed
This patch lays the foundation for using the per-cpu 'state' field in
the 'tsp_context' structure for other flags apart from the power state
of the TSP.
It allocates 2 bits for the power state, introduces the necessary
macros to manipulate the power state in the 'state' field and
accordingly reworks all use of the TSP_STATE_* states.
It also allocates a flag bit to determine if the TSP is handling a
standard SMC. If this flag is set then the TSP was interrupted due to
non-secure or EL3 interupt depending upon the chosen routing
model. Macros to get, set and clear this flag have been added as
well. This flag will be used by subsequent patches.
Change-Id: Ic6ee80bd5895812c83b35189cf2c3be70a9024a6
The issues addressed in this patch are:
1. Remove meminfo_t from the common interfaces in BL3-x,
expecting that platform code will find a suitable mechanism
to determine the memory extents in these images and provide
it to the BL3-x images.
2. Remove meminfo_t and bl31_plat_params_t from all FVP BL3-x
code as the images use link-time information to determine
memory extents.
meminfo_t is still used by common interface in BL1/BL2 for
loading images
Change-Id: I4e825ebf6f515b59d84dc2bdddf6edbf15e2d60f
This patch is based on spec published at
https://github.com/ARM-software/tf-issues/issues/133
It rearranges the bl31_args struct into
bl31_params and bl31_plat_params which provide the
information needed for Trusted firmware and platform
specific data via x0 and x1
On the FVP platform BL3-1 params and BL3-1 plat params
and its constituents are stored at the start of TZDRAM.
The information about memory availability and size for
BL3-1, BL3-2 and BL3-3 is moved into platform specific data.
Change-Id: I8b32057a3d0dd3968ea26c2541a0714177820da9
This patch introduces macros (SPSR_64 and SPSR_32) to
create a SPSR for both aarch32 and aarch64 execution
states. These macros allow the user to set fields
in the SPSR depending upon its format.
The make_spsr() function which did not allow
manipulation of all the fields in the aarch32 SPSR
has been replaced by these new macros.
Change-Id: I9425dda0923e8d5f03d03ddb8fa0e28392c4c61e
This patch implements the register reporting when unhandled exceptions are
taken in BL3-1. Unhandled exceptions will result in a dump of registers
to the console, before halting execution by that CPU. The Crash Stack,
previously called the Exception Stack, is used for this activity.
This stack is used to preserve the CPU context and runtime stack
contents for debugging and analysis.
This also introduces the per_cpu_ptr_cache, referenced by tpidr_el3,
to provide easy access to some of BL3-1 per-cpu data structures.
Initially, this is used to provide a pointer to the Crash stack.
panic() now prints the the error file and line number in Debug mode
and prints the PC value in release mode.
The Exception Stack is renamed to Crash Stack with this patch.
The original intention of exception stack is no longer valid
since we intend to support several valid exceptions like IRQ
and FIQ in the trusted firmware context. This stack is now
utilized for dumping and reporting the system state when a
crash happens and hence the rename.
FixesARM-software/tf-issues#79 Improve reporting of unhandled exception
Change-Id: I260791dc05536b78547412d147193cdccae7811a
This patch fixes the broken support for entry into standby states
introduced under commit-id 'd118f9f864' (tf-issues#94). Upon exit from
the platform defined standby state instead of returning to the caller
of the SMC, execution would get stuck in the wfi instruction meant for
entering a power down state. This patch ensures that exit from a
standby state and entry into a power down state do not interfere with
each other.
FixesARM-software/tf-issues#154
Change-Id: I56e5df353368e44d6eefc94ffedefe21929f5cfe
Instead of having a single version of the MMU setup functions for all
bootloader images that can execute either in EL3 or in EL1, provide
separate functions for EL1 and EL3. Each bootloader image can then
call the appropriate version of these functions. The aim is to reduce
the amount of code compiled in each BL image by embedding only what's
needed (e.g. BL1 to embed only EL3 variants).
Change-Id: Ib86831d5450cf778ae78c9c1f7553fe91274c2fa
Previously exception handlers in BL3-1, X19-X29 were not saved
and restored on every SMC/trap into EL3. Instead these registers
were 'saved as needed' as a side effect of the A64 ABI used by the C
compiler.
That approach failed when world switching but was not visible
with the TSP/TSPD code because the TSP is 64-bit, did not
clobber these registers when running and did not support pre-emption
by normal world interrupts. These scenarios showed
that the values in these registers can be passed through a world
switch, which broke the normal and trusted world assumptions
about these registers being preserved.
The Ideal solution saves and restores these registers when a
world switch occurs - but that type of implementation is more complex.
So this patch always saves and restores these registers on entry and
exit of EL3.
FixesARM-software/tf-issues#141
Change-Id: I9a727167bbc594454e81cf78a97ca899dfb11c27
Instead of using the system register helper functions to read
or write system registers, assembler coded functions should
use MRS/MSR instructions. This results in faster and more
compact code.
This change replaces all usage of the helper functions with
direct register accesses.
Change-Id: I791d5f11f257010bb3e6a72c6c5ab8779f1982b3
The current code does not always use data and instruction
barriers as required by the architecture and frequently uses
barriers excessively due to their inclusion in all of the
write_*() helper functions.
Barriers should be used explicitly in assembler or C code
when modifying processor state that requires the barriers in
order to enable review of correctness of the code.
This patch removes the barriers from the helper functions and
introduces them as necessary elsewhere in the code.
PORTING NOTE: check any port of Trusted Firmware for use of
system register helper functions for reliance on the previous
barrier behaviour and add explicit barriers as necessary.
FixesARM-software/tf-issues#92
Change-Id: Ie63e187404ff10e0bdcb39292dd9066cb84c53bf
Jens Wiklander
Soby Mathew
Achin Gupta
Juan Castillo
Dan Handley
Vikram Kanigiri
Andrew Thoelke
Sandrine Bailleux