bakery_lock_normal.c uses the raw register accessor, read_sctlr(_el3)
to check whether the dcache is enabled.
Using is_dcache_enabled() is cleaner, and a good abstraction for
the library code like this.
A problem is is_dcache_enabled() is declared in the local header,
lib/xlat_tables_v2/xlat_tables_private.h
I searched for a good place to declare this helper. Moving it to
arch_helpers.h, closed to cache operation helpers, looks good enough
to me.
I also changed the type of 'is_cached' to bool for consistency,
and to avoid MISRA warnings.
Change-Id: I9b016f67bc8eade25c316aa9c0db0fa4cd375b79
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
ARM has a weak memory ordering model. This means that without
explicit barriers, memory accesses can be observed differently
than program order. In this case, the cache invalidate instruction
can be observed after the subsequent read to address.
To solve this, a DMB instruction is required between the cache
invalidate and the read. This ensures that the cache invalidate
completes before all memory accesses in program order after the DMB.
This patch updates the 'read_cache_op' macro to issue a DMB after
the cache invalidate instruction to fix this anomaly.
Change-Id: Iac9a90d228c57ba8bcdca7e409ea6719546ab441
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
bakery_lock_get() uses DMB LD after lock acquisition and
bakery_lock_release() uses DMB ST before releasing the lock. This is
insufficient in both cases. With just DMB LD, stores in the critical
section can be reordered before the DMB LD which could mean writes in
the critical section completing before the lock has been acquired
successfully. Similarly, with just DMB ST, a load in the critical section
could be reordered after the the DMB ST. DMB is the least expensive
barrier that can provide the required ordering.
Signed-off-by: Raghu Krishnamurthy <raghu.ncstate@icloud.com>
Change-Id: Ieb74cbf5b76b09e1789331b71f37f7c660221b0e
NOTE: AARCH32/AARCH64 macros are now deprecated in favor of __aarch64__.
All common C compilers pre-define the same macros to signal which
architecture the code is being compiled for: __arm__ for AArch32 (or
earlier versions) and __aarch64__ for AArch64. There's no need for TF-A
to define its own custom macros for this. In order to unify code with
the export headers (which use __aarch64__ to avoid another dependency),
let's deprecate the AARCH32 and AARCH64 macros and switch the code base
over to the pre-defined standard macro. (Since it is somewhat
unintuitive that __arm__ only means AArch32, let's standardize on only
using __aarch64__.)
Change-Id: Ic77de4b052297d77f38fc95f95f65a8ee70cf200
Signed-off-by: Julius Werner <jwerner@chromium.org>
This patch introduces explicit linker variables to mark the start and
end of the per-cpu bakery lock section to help bakery_lock_normal.c
calculate the size of the section. This patch removes the previously
used '__PERCPU_BAKERY_LOCK_SIZE__' linker variable to make the code
uniform across GNU linker and ARM linker.
Change-Id: Ie0c51702cbc0fe8a2076005344a1fcebb48e7cca
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Enforce full include path for includes. Deprecate old paths.
The following folders inside include/lib have been left unchanged:
- include/lib/cpus/${ARCH}
- include/lib/el3_runtime/${ARCH}
The reason for this change is that having a global namespace for
includes isn't a good idea. It defeats one of the advantages of having
folders and it introduces problems that are sometimes subtle (because
you may not know the header you are actually including if there are two
of them).
For example, this patch had to be created because two headers were
called the same way: e0ea0928d5 ("Fix gpio includes of mt8173 platform
to avoid collision."). More recently, this patch has had similar
problems: 46f9b2c3a2 ("drivers: add tzc380 support").
This problem was introduced in commit 4ecca33988 ("Move include and
source files to logical locations"). At that time, there weren't too
many headers so it wasn't a real issue. However, time has shown that
this creates problems.
Platforms that want to preserve the way they include headers may add the
removed paths to PLAT_INCLUDES, but this is discouraged.
Change-Id: I39dc53ed98f9e297a5966e723d1936d6ccf2fc8f
Signed-off-by: Antonio Nino Diaz <antonio.ninodiaz@arm.com>
With the current implementation, it's possible for a contender to
observe accesses in the Critical Section before acquiring or releasing
the lock. Insert fencing in the locking and release codes to prevent any
reorder.
FixesARM-software/tf-issues#609
Change-Id: I773b82aa41dd544a2d3dbacb9a4b42c9eb767bbb
Signed-off-by: Jeenu Viswambharan <jeenu.viswambharan@arm.com>
Use a _ prefix for macro arguments to prevent that argument from
hiding variables of the same name in the outer scope
Rule 5.3: An identifier declared in an inner scope shall not
hide an identifier declared in an outer scope
Fixed For:
make PLAT=fvp USE_COHERENT_MEM=0
Change-Id: If50c583d3b63799ee6852626b15be00c0f6b10a0
Signed-off-by: Daniel Boulby <daniel.boulby@arm.com>
Use a _ prefix for Macro arguments to prevent that argument from
hiding variables of the same name in the outer scope
Rule 5.3: An identifier declared in an inner scope shall not
hide an identifier declared in an outer scope
Fixed For:
make LOG_LEVEL=50 PLAT=fvp
Change-Id: I67b6b05cbad4aeca65ce52981b4679b340604708
Signed-off-by: Daniel Boulby <daniel.boulby@arm.com>
Void pointers have been used to access linker symbols, by declaring an
extern pointer, then taking the address of it. This limits symbols
values to aligned pointer values. To remove this restriction an
IMPORT_SYM macro has been introduced, which declares it as a char
pointer and casts it to the required type.
Change-Id: I89877fc3b13ed311817bb8ba79d4872b89bfd3b0
Signed-off-by: Joel Hutton <Joel.Hutton@Arm.com>
To make software license auditing simpler, use SPDX[0] license
identifiers instead of duplicating the license text in every file.
NOTE: Files that have been imported by FreeBSD have not been modified.
[0]: https://spdx.org/
Change-Id: I80a00e1f641b8cc075ca5a95b10607ed9ed8761a
Signed-off-by: dp-arm <dimitris.papastamos@arm.com>
This patch fixes a compilation issue with bakery locks when
PSCI library is compiled with USE_COHERENT_MEM = 0 build option.
Change-Id: Ic7f6cf9f2bb37f8a946eafbee9cbc3bf0dc7e900
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
This patch fixes an issue in the normal memory bakery lock
implementation. During assertion of lock status, there is a possibility
that the assertion could fail. This is because the previous update done
to the lock status by the owning CPU when not participating in cache
coherency could result in stale data in the cache due to cache maintenance
operations not propagating to all the caches. This patch fixes this issue
by doing an extra read cache maintenance operation prior to the assertion.
FixesARM-software/tf-issues#402
Change-Id: I0f38a7c52476a4f58e17ebe0141d256d198be88d
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
This patch reworks type usage in generic code, drivers and ARM platform files
to make it more portable. The major changes done with respect to
type usage are as listed below:
* Use uintptr_t for storing address instead of uint64_t or unsigned long.
* Review usage of unsigned long as it can no longer be assumed to be 64 bit.
* Use u_register_t for register values whose width varies depending on
whether AArch64 or AArch32.
* Use generic C types where-ever possible.
In addition to the above changes, this patch also modifies format specifiers
in print invocations so that they are AArch64/AArch32 agnostic. Only files
related to upcoming feature development have been reworked.
Change-Id: I9f8c78347c5a52ba7027ff389791f1dad63ee5f8
This patch unifies the bakery lock api's across coherent and normal
memory implementation of locks by using same data type `bakery_lock_t`
and similar arguments to functions.
A separate section `bakery_lock` has been created and used to allocate
memory for bakery locks using `DEFINE_BAKERY_LOCK`. When locks are
allocated in normal memory, each lock for a core has to spread
across multiple cache lines. By using the total size allocated in a
separate cache line for a single core at compile time, the memory for
other core locks is allocated at link time by multiplying the single
core locks size with (PLATFORM_CORE_COUNT - 1). The normal memory lock
algorithm now uses lock address instead of the `id` in the per_cpu_data.
For locks allocated in coherent memory, it moves locks from
tzfw_coherent_memory to bakery_lock section.
The bakery locks are allocated as part of bss or in coherent memory
depending on usage of coherent memory. Both these regions are
initialised to zero as part of run_time_init before locks are used.
Hence, bakery_lock_init() is made an empty function as the lock memory
is already initialised to zero.
The above design lead to the removal of psci bakery locks from
non_cpu_power_pd_node to psci_locks.
NOTE: THE BAKERY LOCK API WHEN USE_COHERENT_MEM IS NOT SET HAS CHANGED.
THIS IS A BREAKING CHANGE FOR ALL PLATFORM PORTS THAT ALLOCATE BAKERY
LOCKS IN NORMAL MEMORY.
Change-Id: Ic3751c0066b8032dcbf9d88f1d4dc73d15f61d8b
This patch migrates the rest of Trusted Firmware excluding Secure Payload and
the dispatchers to the new platform and context management API. The per-cpu
data framework APIs which took MPIDRs as their arguments are deleted and only
the ones which take core index as parameter are retained.
Change-Id: I839d05ad995df34d2163a1cfed6baa768a5a595d
This patch removes the `owner` field in bakery_lock_t structure which
is the data structure used in the bakery lock implementation that uses
coherent memory. The assertions to protect against recursive lock
acquisition were based on the 'owner' field. They are now done based
on the bakery lock ticket number. These assertions are also added
to the bakery lock implementation that uses normal memory as well.
Change-Id: If4850a00dffd3977e218c0f0a8d145808f36b470
This patch optimizes the data structure used with the bakery lock
implementation for coherent memory to save memory and minimize memory
accesses. These optimizations were already part of the bakery lock
implementation for normal memory and this patch now implements
it for the coherent memory implementation as well. Also
included in the patch is a cleanup to use the do-while loop while
waiting for other contenders to finish choosing their tickets.
Change-Id: Iedb305473133dc8f12126726d8329b67888b70f1
This patch moves the bakery locks out of coherent memory to normal memory.
This implies that the lock information needs to be placed on a separate cache
line for each cpu. Hence the bakery_lock_info_t structure is allocated in the
per-cpu data so as to minimize memory wastage. A similar platform per-cpu
data is introduced for the platform locks.
As a result of the above changes, the bakery lock api is completely changed.
Earlier, a reference to the lock structure was passed to the lock implementation.
Now a unique-id (essentially an index into the per-cpu data array) and an offset
into the per-cpu data for bakery_info_t needs to be passed to the lock
implementation.
Change-Id: I1e76216277448713c6c98b4c2de4fb54198b39e0
This patch is an optimization in the bakery_lock_get() function
which removes the wfe() when waiting for other contenders to choose
their ticket i.e when their `entering` flag is set. Since the time
taken to execute bakery_get_ticket() by other contenders is bounded,
this wait is a bounded time wait. Hence the removal of wfe() and the
corresponding sev() and dsb() in bakery_get_ticket() may result
in better time performance during lock acquisition.
Change-Id: I141bb21294226b54cb6e89e7cac0175c553afd8d
This patch fixes a bug in the bakery lock implementation where a data
synchronisation barrier instruction is not issued before sending an event as
mandated by the ARMv8 ARM. This can cause a event to be signalled before the
related memory accesses have completed resulting in erroneous execution.
FixesARM-software/tf-issues#272
Change-Id: I5ce02bf70afb001d967b9fa4c3f77442931d5349
The bakery lock code currently expects the calling code to pass
the MPIDR_EL1 of the current CPU.
This is not always done correctly. Also the change to provide
inline access to system registers makes it more efficient for the
bakery lock code to obtain the MPIDR_EL1 directly.
This change removes the mpidr parameter from the bakery lock
interface, and results in a code reduction of 160 bytes for the
ARM FVP port.
FixesARM-software/tf-issues#213
Change-Id: I7ec7bd117bcc9794a0d948990fcf3336a367d543
Previously, platform.h contained many declarations and definitions
used for different purposes. This file has been split so that:
* Platform definitions used by common code that must be defined
by the platform are now in platform_def.h. The exact include
path is exported through $PLAT_INCLUDES in the platform makefile.
* Platform definitions specific to the FVP platform are now in
/plat/fvp/fvp_def.h.
* Platform API declarations specific to the FVP platform are now
in /plat/fvp/fvp_private.h.
* The remaining platform API declarations that must be ported by
each platform are still in platform.h but this file has been
moved to /include/plat/common since this can be shared by all
platforms.
Change-Id: Ieb3bb22fbab3ee8027413c6b39a783534aee474a
Reduce the number of header files included from other header
files as much as possible without splitting the files. Use forward
declarations where possible. This allows removal of some unnecessary
"#ifndef __ASSEMBLY__" statements.
Also, review the .c and .S files for which header files really need
including and reorder the #include statements alphabetically.
FixesARM-software/tf-issues#31
Change-Id: Iec92fb976334c77453e010b60bcf56f3be72bd3e
Add tag names to all unnamed structs in header files. This
allows forward declaration of structs, which is necessary to
reduce header file nesting (to be implemented in a subsequent
commit).
Also change the typedef names across the codebase to use the _t
suffix to be more conformant with the Linux coding style. The
coding style actually prefers us not to use typedefs at all but
this is considered a step too far for Trusted Firmware.
Also change the IO framework structs defintions to use typedef'd
structs to be consistent with the rest of the codebase.
Change-Id: I722b2c86fc0d92e4da3b15e5cab20373dd26786f
Move almost all system include files to a logical sub-directory
under ./include. The only remaining system include directories
not under ./include are specific to the platform. Move the
corresponding source files to match the include directory
structure.
Also remove pm.h as it is no longer used.
Change-Id: Ie5ea6368ec5fad459f3e8a802ad129135527f0b3
Masahiro Yamada
Varun Wadekar
Raghu Krishnamurthy
Julius Werner
Antonio Nino Diaz
Jeenu Viswambharan
Daniel Boulby
Joel Hutton
dp-arm
Soby Mathew
Andrew Thoelke
Achin Gupta
Dan Handley