This patch fixes a crash due to corruption of cpu_ops
data structure. During the secondary CPU boot, after the
cpu_ops has been initialized in the per cpu-data, the
dcache lines need to invalidated so that the update in
memory can be seen later on when the dcaches are turned ON.
Also, after initializing the psci per cpu data, the dcache
lines are flushed so that they are written back to memory
and dirty dcache lines are avoided.
FixesARM-Software/tf-issues#271
Change-Id: Ia90f55e9882690ead61226eea5a5a9146d35f313
This patch fixes a bug in the bakery lock implementation where a data
synchronisation barrier instruction is not issued before sending an event as
mandated by the ARMv8 ARM. This can cause a event to be signalled before the
related memory accesses have completed resulting in erroneous execution.
FixesARM-software/tf-issues#272
Change-Id: I5ce02bf70afb001d967b9fa4c3f77442931d5349
This patch optimizes the Cortex-A57 cluster power down sequence by not
flushing the Level1 data cache. The L1 data cache and the L2 unified
cache are inclusive. A flush of the L2 by set/way flushes any dirty
lines from the L1 as well. This is a known safe deviation from the
Cortex-A57 TRM defined power down sequence. This optimization can be
enabled by the platform through the 'SKIP_A57_L1_FLUSH_PWR_DWN' build
flag. Each Cortex-A57 based platform must make its own decision on
whether to use the optimization.
This patch also renames the cpu-errata-workarounds.md to
cpu-specific-build-macros.md as this facilitates documentation
of both CPU Specific errata and CPU Specific Optimization
build macros.
Change-Id: I299b9fe79e9a7e08e8a0dffb7d345f9a00a71480
This the patch replaces the DSB SY with DSB ISH
after disabling L2 prefetches during the Cortex-A57
power down sequence.
Change-Id: I048d12d830c1b974b161224eff079fb9f8ecf52d
Prior to this patch, the errata workarounds were applied for any version
of the CPU in the release build and in the debug build an assert
failure resulted when the revision did not match. This patch applies
errata workarounds in the Cortex-A57 reset handler only if the 'variant'
and 'revision' fields read from the MIDR_EL1 match. In the debug build,
a warning message is printed for each errata workaround which is not
applied.
The patch modifies the register usage in 'reset_handler` so
as to adhere to ARM procedure calling standards.
FixesARM-software/tf-issues#242
Change-Id: I51b1f876474599db885afa03346e38a476f84c29
This patch adds level specific cache maintenance functions
to cache_helpers.S. The new functions 'dcsw_op_levelx',
where '1 <= x <= 3', allow to perform cache maintenance by
set/way for that particular level of cache. With this patch,
functions to support cache maintenance upto level 3 have
been implemented since it is the highest cache level for
most ARM SoCs.
These functions are now utilized in CPU specific power down
sequences to implement them as mandated by processor specific
technical reference manual.
Change-Id: Icd90ce6b51cff5a12863bcda01b93601417fd45c
This patch adds workarounds for selected errata which affect the Cortex-A57 r0p0
part. Each workaround has a build time flag which should be used by the platform
port to enable or disable the corresponding workaround. The workarounds are
disabled by default. An assertion is raised if the platform enables a workaround
which does not match the CPU revision at runtime.
Change-Id: I9ae96b01c6ff733d04dc733bd4e67dbf77b29fb0
This patch adds handlers for dumping Cortex-A57 and Cortex-A53 specific register
state to the CPU specific operations framework. The contents of CPUECTLR_EL1 are
dumped currently.
Change-Id: I63d3dbfc4ac52fef5e25a8cf6b937c6f0975c8ab
This patch adds CPU core and cluster power down sequences to the CPU specific
operations framework introduced in a earlier patch. Cortex-A53, Cortex-A57 and
generic AEM sequences have been added. The latter is suitable for the
Foundation and Base AEM FVPs. A pointer to each CPU's operations structure is
saved in the per-cpu data so that it can be easily accessed during power down
seqeunces.
An optional platform API has been introduced to allow a platform to disable the
Accelerator Coherency Port (ACP) during a cluster power down sequence. The weak
definition of this function (plat_disable_acp()) does not take any action. It
should be overriden with a strong definition if the ACP is present on a
platform.
Change-Id: I8d09bd40d2f528a28d2d3f19b77101178778685d
This patch adds an optional platform API (plat_reset_handler) which allows the
platform to perform any actions immediately after a cold or warm reset
e.g. implement errata workarounds. The function is called with MMU and caches
turned off. This API is weakly defined and does nothing by default but can be
overriden by a platform with a strong definition.
Change-Id: Ib0acdccbd24bc756528a8bd647df21e8d59707ff
This patch introduces a framework which will allow CPUs to perform
implementation defined actions after a CPU reset, during a CPU or cluster power
down, and when a crash occurs. CPU specific reset handlers have been implemented
in this patch. Other handlers will be implemented in subsequent patches.
Also moved cpu_helpers.S to the new directory lib/cpus/aarch64/.
Change-Id: I1ca1bade4d101d11a898fb30fea2669f9b37b956
Move the remaining IO storage source file (io_storage.c) from the
lib to the drivers directory. This requires that platform ports
explicitly add this file to the list of source files.
Also move the IO header files to a new sub-directory, include/io.
Change-Id: I862b1252a796b3bcac0d93e50b11e7fb2ded93d6
The intent of io_init() was to allow platform ports to provide
a data object (io_plat_data_t) to the IO storage framework to
allocate into. The abstraction was incomplete because io_plat_data_t
uses a platform defined constant and the IO storage framework
internally allocates other arrays using platform defined constants.
This change simplifies the implementation by instantiating the
supporting objects in the IO storage framework itself. There is now
no need for the platform to call io_init().
The FVP port has been updated accordingly.
THIS CHANGE REQUIRES ALL PLATFORM PORTS THAT USE THE IO STORAGE
FRAMEWORK TO BE UDPATED.
Change-Id: Ib48ac334de9e538064734334c773f8b43df3a7dc
Fix the following issues with the console log output:
* Make sure the welcome string is the first thing in the log output
(during normal boot).
* Prefix each message with the BL image name so it's clear which
BL the output is coming from.
* Ensure all output is wrapped in one of the log output macros so it can
be easily compiled out if necessary. Change some of the INFO() messages
to VERBOSE(), especially in the TSP.
* Create some extra NOTICE() and INFO() messages during cold boot.
* Remove all usage of \r in log output.
FixesARM-software/tf-issues#231
Change-Id: Ib24f7acb36ce64bbba549f204b9cde2dbb46c8a3
The patch implements a macro ASM_ASSERT() which can
be invoked from assembly code. When assertion happens,
file name and line number of the check is written
to the crash console.
FixesARM-software/tf-issues#95
Change-Id: I6f905a068e1c0fa4f746d723f18df60daaa00a86
This patch reworks the manner in which the M,A, C, SA, I, WXN & EE bits of
SCTLR_EL3 & SCTLR_EL1 are managed. The EE bit is cleared immediately after reset
in EL3. The I, A and SA bits are set next in EL3 and immediately upon entry in
S-EL1. These bits are no longer managed in the blX_arch_setup() functions. They
do not have to be saved and restored either. The M, WXN and optionally the C
bit are set in the enable_mmu_elX() function. This is done during both the warm
and cold boot paths.
FixesARM-software/tf-issues#226
Change-Id: Ie894d1a07b8697c116960d858cd138c50bc7a069
This patch implements a "tf_printf" which supports only the commonly
used format specifiers in Trusted Firmware, which uses a lot less
stack space than the stdlib printf function.
FixesARM-software/tf-issues#116
Change-Id: I7dfa1944f4c1e634b3e2d571f49afe02d109a351
This patch adds a 'flags' parameter to each exception level specific function
responsible for enabling the MMU. At present only a single flag which indicates
whether the data cache should also be enabled is implemented. Subsequent patches
will use this flag when enabling the MMU in the warm boot paths.
Change-Id: I0eafae1e678c9ecc604e680851093f1680e9cefa
Currently the TCR bits are hardcoded in xlat_tables.c. In order to
map higher physical address into low virtual address, the TCR bits
need to be configured accordingly.
This patch is to save the max VA and PA and calculate the TCR.PS/IPS
and t0sz bits in init_xlat_tables function.
Change-Id: Ia7a58e5372b20200153057d457f4be5ddbb7dae4
Making the simple mmio_read_*() and mmio_write_*() functions inline
saves 360 bytes of code in FVP release build.
FixesARM-software/tf-issues#210
Change-Id: I65134f9069f3b2d8821d882daaa5fdfe16355e2f
The bakery lock code currently expects the calling code to pass
the MPIDR_EL1 of the current CPU.
This is not always done correctly. Also the change to provide
inline access to system registers makes it more efficient for the
bakery lock code to obtain the MPIDR_EL1 directly.
This change removes the mpidr parameter from the bakery lock
interface, and results in a code reduction of 160 bytes for the
ARM FVP port.
FixesARM-software/tf-issues#213
Change-Id: I7ec7bd117bcc9794a0d948990fcf3336a367d543
Replace the current out-of-line assembler implementations of
the system register and system instruction operations with
inline assembler.
This enables better compiler optimisation and code generation
when accessing system registers.
FixesARM-software/tf-issues#91
Change-Id: I149af3a94e1e5e5140a3e44b9abfc37ba2324476
Current ATF uses a direct physical-to-virtual mapping, that is, a physical
address is mapped to the same address in the virtual space. For example,
physical address 0x8000_0000 is mapped to 0x8000_0000 virtual. This
approach works fine for FVP as all its physical addresses fall into 0 to
4GB range. But for other platform where all I/O addresses are 48-bit long,
If we follow the same direct mapping, we would need virtual address range
from 0 to 0x8fff_ffff_ffff, which is about 144TB. This requires a
significant amount of memory for MMU tables and it is not necessary to use
that much virtual space in ATF.
The patch is to enable mapping a physical address range to an arbitrary
virtual address range (instead of flat mapping)
Changed "base" to "base_va" and added "base_pa" in mmap_region_t and
modified functions such as mmap_add_region and init_xlation_table etc.
FixesARM-software/tf-issues#158
Previously, the enable_mmu_elX() functions were implicitly part of
the platform porting layer since they were included by generic
code. These functions have been placed behind 2 new platform
functions, bl31_plat_enable_mmu() and bl32_plat_enable_mmu().
These are weakly defined so that they can be optionally overridden
by platform ports.
Also, the enable_mmu_elX() functions have been moved to
lib/aarch64/xlat_tables.c for optional re-use by platform ports.
These functions are tightly coupled with the translation table
initialization code.
FixesARM-software/tf-issues#152
Change-Id: I0a2251ce76acfa3c27541f832a9efaa49135cc1c
Previously, platform.h contained many declarations and definitions
used for different purposes. This file has been split so that:
* Platform definitions used by common code that must be defined
by the platform are now in platform_def.h. The exact include
path is exported through $PLAT_INCLUDES in the platform makefile.
* Platform definitions specific to the FVP platform are now in
/plat/fvp/fvp_def.h.
* Platform API declarations specific to the FVP platform are now
in /plat/fvp/fvp_private.h.
* The remaining platform API declarations that must be ported by
each platform are still in platform.h but this file has been
moved to /include/plat/common since this can be shared by all
platforms.
Change-Id: Ieb3bb22fbab3ee8027413c6b39a783534aee474a
This patch adds support in the TSP to program the secure physical
generic timer to generate a EL-1 interrupt every half second. It also
adds support for maintaining the timer state across power management
operations. The TSPD ensures that S-EL1 can access the timer by
programming the SCR_EL3.ST bit.
This patch does not actually enable the timer. This will be done in a
subsequent patch once the complete framework for handling S-EL1
interrupts is in place.
Change-Id: I1b3985cfb50262f60824be3a51c6314ce90571bc
Addresses were declared as "unsigned int" in drivers/arm/peripherals/pl011/pl011.h and in function init_xlation_table. Changed to use "unsigned long" instead
FixesARM-software/tf-issues#156
This patch implements the register reporting when unhandled exceptions are
taken in BL3-1. Unhandled exceptions will result in a dump of registers
to the console, before halting execution by that CPU. The Crash Stack,
previously called the Exception Stack, is used for this activity.
This stack is used to preserve the CPU context and runtime stack
contents for debugging and analysis.
This also introduces the per_cpu_ptr_cache, referenced by tpidr_el3,
to provide easy access to some of BL3-1 per-cpu data structures.
Initially, this is used to provide a pointer to the Crash stack.
panic() now prints the the error file and line number in Debug mode
and prints the PC value in release mode.
The Exception Stack is renamed to Crash Stack with this patch.
The original intention of exception stack is no longer valid
since we intend to support several valid exceptions like IRQ
and FIQ in the trusted firmware context. This stack is now
utilized for dumping and reporting the system state when a
crash happens and hence the rename.
FixesARM-software/tf-issues#79 Improve reporting of unhandled exception
Change-Id: I260791dc05536b78547412d147193cdccae7811a
The data cache clean and invalidate operations dcsw_op_all()
and dcsw_op_loius() were implemented to invoke a DSB and ISB
barrier for every set/way operation. This adds a substantial
performance penalty to an already expensive operation.
These functions have been reworked to provide an optimised
implementation derived from the code in section D3.4 of the
ARMv8 ARM. The helper macro setup_dcsw_op_args has been moved
and reworked alongside the implementation.
FixesARM-software/tf-issues#146
Change-Id: Icd5df57816a83f0a842fce935320a369f7465c7f
There are a small number of non-EL specific helper functions
which are no longer used, and also some unusable helper
functions for non-existant registers.
This change removes all of these functions.
Change-Id: Idd656cef3b59cf5c46fe2be4029d72288b649c24
Instead of using the system register helper functions to read
or write system registers, assembler coded functions should
use MRS/MSR instructions. This results in faster and more
compact code.
This change replaces all usage of the helper functions with
direct register accesses.
Change-Id: I791d5f11f257010bb3e6a72c6c5ab8779f1982b3
disable_mmu() cannot work as a C function as there is no control
over data accesses generated by the compiler between disabling and
cleaning the data cache. This results in reading stale data from
main memory.
As assembler version is provided for EL3, and a variant that also
disables the instruction cache which is now used by the BL1
exception handling function.
FixesARM-software/tf-issues#147
Change-Id: I0cf394d2579a125a23c2f2989c2e92ace6ddb1a6
The current code does not always use data and instruction
barriers as required by the architecture and frequently uses
barriers excessively due to their inclusion in all of the
write_*() helper functions.
Barriers should be used explicitly in assembler or C code
when modifying processor state that requires the barriers in
order to enable review of correctness of the code.
This patch removes the barriers from the helper functions and
introduces them as necessary elsewhere in the code.
PORTING NOTE: check any port of Trusted Firmware for use of
system register helper functions for reliance on the previous
barrier behaviour and add explicit barriers as necessary.
FixesARM-software/tf-issues#92
Change-Id: Ie63e187404ff10e0bdcb39292dd9066cb84c53bf
Update code base to remove variables from the .data section,
mainly by using const static data where possible and adding
the const specifier as required. Most changes are to the IO
subsystem, including the framework APIs. The FVP power
management code is also affected.
Delay initialization of the global static variable,
next_image_type in bl31_main.c, until it is realy needed.
Doing this moves the variable from the .data to the .bss
section.
Also review the IO interface for inconsistencies, using
uintptr_t where possible instead of void *. Remove the
io_handle and io_dev_handle typedefs, which were
unnecessary, replacing instances with uintptr_t.
FixesARM-software/tf-issues#107.
Change-Id: I085a62197c82410b566e4698e5590063563ed304
Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.
Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.
FixesARM-software/tf-issues#31
Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e
Add tag names to all unnamed structs in header files. This
allows forward declaration of structs, which is necessary to
reduce header file nesting (to be implemented in a subsequent
commit).
Also change the typedef names across the codebase to use the _t
suffix to be more conformant with the Linux coding style. The
coding style actually prefers us not to use typedefs at all but
this is considered a step too far for Trusted Firmware.
Also change the IO framework structs defintions to use typedef'd
structs to be consistent with the rest of the codebase.
Change-Id: I722b2c86fc0d92e4da3b15e5cab20373dd26786f
Make codebase consistent in its use of #include "" syntax for
user includes and #include <> syntax for system includes.
FixesARM-software/tf-issues#65
Change-Id: If2f7c4885173b1fd05ac2cde5f1c8a07000c7a33
Move almost all system include files to a logical sub-directory
under ./include. The only remaining system include directories
not under ./include are specific to the platform. Move the
corresponding source files to match the include directory
structure.
Also remove pm.h as it is no longer used.
Change-Id: Ie5ea6368ec5fad459f3e8a802ad129135527f0b3
The TZC-400 performs security checks on transactions to memory or
peripherals. Separate regions can be created in the address space each
with individual security settings.
Limitations:
This driver does not currently support raising an interrupt on access
violation.
Change-Id: Idf8ed64b4d8d218fc9b6f9d75acdb2cd441d2449
This extends the --gc-sections behaviour to the many assembler
support functions in the firmware images by placing each function
into its own code section. This is achieved by creating a 'func'
macro used to declare each function label.
FixesARM-software/tf-issues#80
Change-Id: I301937b630add292d2dec6d2561a7fcfa6fec690
Each ARM Trusted Firmware image should know in which EL it is running
and it should use the corresponding register directly instead of reading
currentEL and knowing which asm register to read/write
Change-Id: Ief35630190b6f07c8fbb7ba6cb20db308f002945
Current implementation of Bakery Lock does tight-loop waiting upon lock
contention.
This commit reworks the implementation to use WFE instruction for
waiting, and SEV to signal lock availability. It also adds the rationale
for choosing Bakery Locks instead of exclusion primitives, and more
comments for the lock algorithm.
FixesARM-software/tf-issue#67
Change-Id: Ie351d3dbb27ec8e64dbc9507c84af07bd385a7df
Co-authored-by: Vikram Kanigiri <vikram.kanigiri@arm.com>
bakery_lock_release() expects an mpidr as the first argument however
bakery_lock_release() is calling it with the 'entry' argument it has
calculated. Rather than fixing this to pass the mpidr value it would be
much more efficient to just replace the call with
assert(bakery->owner == entry)
As this leaves no remaining users of bakery_lock_held(), we might as
well delete it.
FixesARM-software/tf-issues#27
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Fixes issues #10:
https://github.com/ARM-software/tf-issues/issues/10
This patch changes all/most variables of type int to be size_t or long
to fix the sizing and alignment problems found when building with the
newer toolchains such as Linaro GCC 13.12 or later.
Change-Id: Idc9d48eb2ff9b8c5bbd5b227e6907263d1ea188b
Signed-off-by: Ryan Harkin <ryan.harkin@linaro.org>
This patch factors out the ARM FVP specific code to create MMU
translation tables so that it is possible for a boot loader stage to
create a different set of tables instead of using the default ones.
The default translation tables are created with the assumption that
the calling boot loader stage executes out of secure SRAM. This might
not be true for the BL3_2 stage in the future.
A boot loader stage can define the `fill_xlation_tables()` function as
per its requirements. It returns a reference to the level 1
translation table which is used by the common platform code to setup
the TTBR_EL3.
This patch is a temporary solution before a larger rework of
translation table creation logic is introduced.
Change-Id: I09a075d5da16822ee32a411a9dbe284718fb4ff6
This patch uses the reworked exception handling support to handle
runtime service requests through SMCs following the SMC calling
convention. This is a giant commit since all the changes are
inter-related. It does the following:
1. Replace the old exception handling mechanism with the new one
2. Enforce that SP_EL0 is used C runtime stacks.
3. Ensures that the cold and warm boot paths use the 'cpu_context'
structure to program an ERET into the next lower EL.
4. Ensures that SP_EL3 always points to the next 'cpu_context'
structure prior to an ERET into the next lower EL
5. Introduces a PSCI SMC handler which completes the use of PSCI as a
runtime service
Change-Id: I661797f834c0803d2c674d20f504df1b04c2b852
Co-authored-by: Achin Gupta <achin.gupta@arm.com>
The modified implementation uses the IO abstraction rather than
making direct semi-hosting calls. The semi-hosting driver is now
registered for the FVP platform during initialisation of each boot
stage where it is used. Additionally, the FVP platform includes a
straightforward implementation of 'plat_get_image_source' which
provides a generic means for the 'load_image' function to determine
how to access the image data.
Change-Id: Ia34457b471dbee990c7b3c79de7aee4ceea51aa6
This is intended primarily for use as a storage abstraction.
It allows operations such as image-loading to be implemented
in a platform-independent fashion. Each platform registers
a set of IO drivers during initialisation. The platform must
also provide a function that will return a device and a specifier
that can be used to access specified content.
Clients of the API will primarily use device and entity handles.
The term "entity" is deliberately vague, to allow for different
representations of content accessed using different types of
specifier, but will often be interpreted as a "file" where the
specifier will normally be its path.
This commit builds, but is intended to be paired with a sample
implementation of "load_image" using a semi-hosting driver on FVP.
Change-Id: Id3b52f1c0eb9ce76b44b99fc6b6460803668cc86
Soby Mathew
Achin Gupta
Dan Handley
Lin Ma
Andrew Thoelke
Harry Liebel
Vikram Kanigiri
Jeenu Viswambharan
Jon Medhurst
Ryan Harkin
James Morrissey