* changes:
feat(tc): enable workaround for CVE-2026-0995
fix(security): workaround for C1-Pro/CME CVE-2026-0995
feat(psci): add psci_cpu_off_start event
feat(smccc): add support for CPU Service calls
This SME erratum in C1-Pro means memory accesses from the CME unit can
remain outstanding after another CPU issues TLBI+DSB. This means SME
can access memory after it has been re-allocated, potentially
overwriting the new owner's data. With pKVM, this could allow the host
access to guest memory if the SME accesses survived the page being
donated to HYP and allocated to the guest.
The workaround is for all affected CPUs to issue DSB locally whenever
another CPU does TLB maintenance. The local DSB completes all
outstanding accesses.
Linux and pKVM share a security state in the GIC, meaning pKVM would
be exposed to interrupt blackouts caused by linux. It is difficult
for the non secure world to avoid races when an SGI is sent to a CPU
that is about go offline and can no longer take the interrupt.
(this would violate the PSCI rules for CPU_OFF calls).
Implement the workaround in EL3 using an SMC in the 'CPU vendor' space.
The workaround uses atomic_inc_return on a global counter to order
parallel callers. This gives each caller a deadline.
Secure SGI are sent to the affected C1 Pro CPUs causing them to run
the workaround, and update their local counter from the global counter.
The CPU that issued the SMC then waits for each SGI'd CPU to update
to at least the deadline from its call.
An SGI being sent can race with an SMC to PSCI CPU_OFF. To avoid
this SGI preventing the CPU from enterring WFI to power off, the
workaround is run pre-emptively and the SGI is masked at the GIC
redistributor.
This mitigation is coordinated with corresponding Operating System
updates for CVE-2026-0995. Both EL3 (TF-A) and the OS must include
their respective fixes to ensure complete mitigation. For example,
the Linux kernel implements a complementary workaround that must
be deployed alongside this TF-A update.
Linux commit:
https://lore.kernel.org/all/20260302165801.3014607-1-catalin.marinas@arm.com/
Ref: https://developer.arm.com/documentation/111823/latest/
Change-Id: Ie969354ad0693fe172d921953b87cfbf4a39ea8e
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>
Signed-off-by: Manish Pandey <manish.pandey2@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
Together with the psci_cpu_on_finish event this is useful to track the
online state of cores.
We cannot trigger the event much later, as then the CPU will be partly
off already (left coherency, for instance).
Change-Id: I072647ece6847b11af1d0e3a0686f5dfd1f2ea58
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Govindraj Raja <govindraj.raja@arm.com>
Newer compilers accept C style integer literal suffixes (like `1ULL`) in
assembly code. Unfortunately, this seems to be a recent development and
older compilers do not. Convert uses of these suffixes to use the
helpers from utils_def_exp.h that will expand correctly.
Change-Id: I5a6e4a52e3c9c85b964fd9fc88548af68cc9998e
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
The pattern for is_feat_xyz_supported() is to not use any #if directives
around it and instead to rely on the compiler to do the right thing an
compile the redundant branches away. This does require a bit of dancing
to appease the compiler - there needs to be an empty function when
FEAT_XYZ is 0 to prevent linker errors.
Change-Id: I4e5eeec7c47d0d1a3dba45757d3d820d380d8e36
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Cortex-A57 erratum 817171 is a Cat B erratum that applies to revisions
r0p0, r0p1. It is fixed in r1p0.
Set L2ACTLR_EL1[26] to 1'b1 to disable L2 regional clock gating.
SDEN documentation:
https://developer.arm.com/documentation/epm049219/latest
Change-Id: I3cf8e70a6abcab8f51098fb22a1d383e123a53d6
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-X1 erratum 2779479 is a Cat B erratum that applies to revisions
r0p0, r1p0, r1p1, r1p2. It is still open.
The erratum can be avoided by setting CPUACTLR3_EL1[47]. Setting this
chicken bit might have a small impact on power and negligible impact
on performance.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401782/latest
Change-Id: I4926f7054be48b08f02f0a4de66114b4d51a5738
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-X1 erratum 1515634 is a Cat B erratum that applies to revision
r0p0. It is fixed in r1p0.
Set CPUACTLR_EL1[11] to one, which flushes the L0 Macro-op cache for
all context synchronization events.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401782/latest
Change-Id: I39bfe27c8dfe575994323aeedf0ed73d1e83745d
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-X1 erratum 1492189 is a Cat B erratum that applies to revision
r0p0. It is fixed in r1p0.
The workaround is to set CPUACTLR5_EL1[8] to 1'b1. The workaround
might result in a small increase in core power consumption.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401782/latest
Change-Id: I288f88f092ee05c15cefb2e764663f4d17fc10a5
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-X4 erratum 2646977 is a Cat B erratum that applies to revision
r0p0. It is fixed in r0p1.
This erratum can be avoided by setting CPUACTLR5_EL1[56:55] to 0b01.
SDEN documentation:
https://developer.arm.com/documentation/109148/latest
Change-Id: Ica7e339280aa97c7d9f6fd8100bd463e4dd978ac
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A510 erratum 1975068 is a Cat B erratum that applies to
revision r0p0. It is fixed in r0p1.
In some systems, software can avoid using Non-shareable mappings.
Where that is not possible, software can set IMP_CMPXECTLR_EL1[9:8] =
0b11. This disables early forwarding of L2 hardware prefetches to
subsequent requests, and may incur a small but not negligible
performance impact.
SDEN documentation:
https://developer.arm.com/documentation/SDEN1873351/latest
Change-Id: I3ac6cbf43a0bbb798b5e39ee1030376afc1b125a
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
* changes:
feat(cpufeat): constrain RAS_TRAP_NS_ERR_REC_ACCESS on ENABLE_FEAT_RAS
fix(build): set defaults to feature flags before platform.mk
refactor(cpufeat): unify FEAT_IDTE3's definitions with arch.h
refactor(el3-runtime): generalise sysreg trapping
refactor(el3-runtime): use contexted SCR_EL3 instead of the register
build: rename default_ones to set_ones
This patch introduces five assembly macros that collapse multiple
bit-manipulation operations on the same system register into a
single mrs/msr pair:
sysreg_lazy_start _reg -- read register into x1
sysreg_lazy_set _bit -- ORR bit into x1 (any 64-bit mask)
sysreg_lazy_clear _bit -- BIC bit from x1 (any 64-bit mask)
sysreg_lazy_insert _src, _lsb, _width -- BFI into x1
sysreg_lazy_commit _reg -- write x1 back to register
Each sysreg_bit_set / sysreg_bit_clear / sysreg_bitfield_insert call
issues its own mrs+msr pair. When several of those target the same
register the reads and writes are redundant. The lazy helpers
replace N reads and N writes with one read and one write.
x1 holds the accumulated register value between start and commit.
x0 is used as a scratch register by sysreg_lazy_set,
sysreg_lazy_clear, and sysreg_lazy_insert. mov_imm is used for bit
values to support arbitrary 64-bit masks, consistent with the
existing hand-written mrs/mov_imm/orr/msr patterns in CPU files.
Change-Id: Iaaf0e4bd7ba85c69d9063b012a9066b3ba40b58e
Signed-off-by: Varun Wadekar <vwadekar@nvidia.com>
Cortex-A77 erratum 1515815 is a Cat B erratum that applies to
revisions r0p0, r1p0. It is fixed in r1p1.
Set CPUACTLR_EL1[11] to 1 so that the L0 Macro-op cache is flushed for
all context synchronization events, ensuring that only a single
instruction is executed before a software step or halt step exception
is taken.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1152370/latest
Change-Id: I1e6faf5a699734f9a5be848807e9c3fa5110d569
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A77 erratum 1253791 is a Cat B erratum that applies to revision
r0p0. It is fixed in r1p0.
This erratum can be avoided by setting CPUACTLR3_EL1[10] to 1, which
prevents parallel execution of divide and square root instructions.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1152370/latest
Change-Id: I76895d167a477246ff5bc6c87237fb4f9724c547
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A77 erratum 1220737 is a Cat B erratum that applies to revision
r0p0. It is fixed in r1p0.
This erratum can be avoided by setting CPUECTLR_EL1[25:24] to 0b11,
which disables write streaming to the L2. This will have an impact on
performance for streaming workloads.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1152370/latest
Change-Id: Iad21fad2b774234b1df808a4074eb3aabc01f2f3
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A78AE erratum 2779481 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2. It is fixed in r0p3.
The erratum can be avoided by setting CPUACTLR3_EL1[47].
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1707912/latest
Change-Id: If45cd8efe24768aaa0d31f56b3b297ba1c10980f
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A78AE erratum 2743229 is a Cat B erratum that applies to
revisions r0p0, r0p1, r0p2. It is fixed in r0p3.
This erratum can be avoided by setting CPUACTLR5_EL1[56:55] to 0b01.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1707912/latest
Change-Id: Ic9a60a695eb00574c25490376337a4ad09b9b2c7
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A715 erratum 2292761 is a Cat B erratum that applies to
revision r0p0. It is fixed in r1p0.
This erratum can be avoided by setting CPUACTLR4_EL1[13] to 1. Using
this workaround has no performance impact.
SDEN documentation:
https://developer.arm.com/documentation/SDEN2148827/latest
Change-Id: Ie2bddb8535a0070da1a58a7753ad3a95c5005646
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Cortex-A78 erratum 1479939 is a Cat B erratum that applies to
revision r0p0. It is fixed in r1p0.
This erratum can be avoided by setting CPUACTLR_EL1[13] to 1 to
disable a performance feature. This should be done before enabling the
MMU.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401784/latest
Change-Id: I553697b5d34da00298526ee0988f52dea8e9e93f
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Use the same naming template and put in arch.h to allow for reuse.
Change-Id: I91a28b5f3e75537422d45c2147cb711625f18282
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
On a first look, the system register trapping code is quite
straightforward - match the register and call a handler. But looking a
bit more closely, with the intention of adding a new one, it isn't -
matching is based on opaque magic numbers and handlers have a lot of
duplication.
This patch tries to resolve both of these by hoisting common
functionality up towards common code and using S3 encodings for the
register matching. It also moves things around a bit to make them more
reusable in future.
Change-Id: Ia69289bfb16615312cc7adcc5cc3e319174b1bf0
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Neoverse V1 erratum 1619807 is a Cat B erratum that applies to
revision r0p0. It is fixed in r1p0.
Set CPUACTLR_EL1 bit 11 to 1 so that all context synchronization
events flush the L0 Macro-op cache, ensuring that when software step
or halt step is enabled the core takes the exception after the
intended single instruction rather than after multiple instructions
from the L0 Macro-op cache.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401781/latest
Change-Id: Ie9595ccbcba04892ebfbfffc067bc2fe1b5a1e6e
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
Neoverse V1 erratum 1542436 is a Cat B erratum that applies to
revision r0p0. It is fixed in r1p0.
Enable the architectural workaround by setting CPUACTLR4_EL1 bit 14 to
1 during boot so that SVE MOVPRFX-prefixed integer multiply
instructions cannot corrupt their scalable vector destination
register, accepting a slight performance impact on SVE prefixing with
MOVPRFX.
SDEN documentation:
https://developer.arm.com/documentation/SDEN-1401781/latest
Change-Id: Ia386b8d4fc7ec8491cc8b68fce4027d4f3c6b843
Signed-off-by: Xialin Liu <xialin.liu@arm.com>
RME architectural requirements are now handled under the feature
detection option ENABLE_FEAT_RME. However, the existing ENABLE_RME build
option performs RMM-specific tasks such as GPT setup, loading the RMM,
and enabling RMMD support.
Since ENABLE_RME now only controls RMM-related functionality, rename it
to ENABLE_RMM to better reflect its purpose and avoid confusion with
ENABLE_FEAT_RME.
For backward compatibility, setting the legacy ENABLE_RME=1 (until it is
deprecated) will automatically enable both ENABLE_FEAT_RME and
ENABLE_RMM.
Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>
Change-Id: Iac945bdffe5002161bf1161b81a5aa7abec68192
ENABLE_RME currently controls multiple, distinct aspects of RME support,
including forcing BL2 to EL3, ROOT world page table setup, GPT
initialization, and full RMM loading and handling.
While full CCA support requires all of these steps, some systems running
on FEAT_RME-capable cores do not need or want an RMM. However, such
systems still require TF-A page table entries to set the .NSE bit so
that TF-A accesses are correctly attributed to the ROOT world,
otherwise, enabling the MMU may cause the system to hang.
To address this, a new build option, ENABLE_FEAT_RME, is introduced. It
handles only the .NSE PTE setup and ignores the rest of the RME/RMM
initialization. ENABLE_FEAT_RME follows the ENABLE_FEAT_* convention and
supports values 0–2, with 2 enabling runtime detection.
Full RME functionality remains gated by ENABLE_RME, which now implicitly
enables ENABLE_FEAT_RME, allowing TF-A to run safely on FEAT_RME systems
without requiring an RMM.
Change-Id: I8391652842ff2e62a73b61829c6250c3805d4a4e
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
* changes:
docs(cpus): explain why the ARCH_WORKAROUND_3 pseudo-erratum is needed
fix(cpus): return ERRATA_MISSING when errata not found
style(smccc): group the ARCH_WORKAROUND_Xs together
style(cpus): add spaces around the CVE-2022-23960 on Neoverse V2
refactor(cpus): clean up FEAT_CSV2 checkers
This is the first FIRME service patch that adds support for basic ABIs
for retrieving the FIRME version, features, and GPI_SET.
This adds a new generic granule transition function that replaces
the existing delegate/undelegate APIs that GPI_SET uses. It also
updates TRP to use GPI_SET when FIRME is supported.
FIRME spec is here, note that it is ALPHA2 quality so further changes
are to be expected:
https://developer.arm.com/documentation/den0149
Change-Id: I57b8ad7e87a0679e15c8247f8457f91f3254dedb
Signed-off-by: John Powell <john.powell@arm.com>
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
The source buffer is read-only for memcpy_s; align the signature with
standard memcpy and allow callers to pass pointers to const data.
Change-Id: Ic785c9d962f4eaf3b870c8461440d52f0dfa1503
Signed-off-by: yaozhm <yaozhongmin@xiaomi.com>
The psci_svc_cpu_data member of cpu_data must be accessed from early
entrypoint code, where the MMU/caching are off, as well as the normal
runtime, where the MMU/caching are on. As a result its accesses cannot
be guaranteed to be coherent and so we must issue CMOs ourselves.
Unfortunately, all CMOs operate on whole cache lines rather than
arbitrarily sized chunks of memory. So all of our CMOs with a size of
sizeof(psci_svc_cpu_data) get rounded up to the nearest cache line.
Since struct psci_cpu_data is declared as aligned to a cache line this
means that whatever lies on the latter parts of its cache line will get
affected too.
Up until the per-cpu framework, this was seemingly fine -
psci_svc_cpu_data was at the end of the cpu_data structure on most
configurations (as PAuth and EL3 exception handling are rarely enabled)
and due to it being a cache line aligned array it would be guaranteed to
sit on a cache line by itself. On configurations where it wasn't last,
it either wasn't a problem due to the access patterns of the other
members or they weren't in cache at the time of the CMOs.
Since the per-cpu framework the above is no longer true. The cpu_data
structure is no longer an array but rather an ordinary member of the
per-cpu region and since we do not enforce any ordering, anything could
be placed after it. When that happens the CMOs have a high chance of
affecting live data and usually leading to a crash.
This patch fixes the problem by asserting that struct psci_cpu_data will
sit alone on a cache line and the CMOs that we do will not have any
unexpected side effects.
The psci_cpu_data_t type alias is also removed to reduce ambiguity and
have a definitive type name for this.
Change-Id: I05cd5f720fea818fcd12fd47422be3e778aa7316
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
The helper xlat_change_mem_attributes_ctx considers only
MT_RO/MT_RW, MT_EXECUTE/MT_EXECUTE_NEVER and MT_USER/MT_PRIVILEGED
attributes, so exclude security state attributes MT_SECURE/NS/ROOT/REALM
extracted from NS and NSE (RME enabled case) bits by
xlat_get_mem_attributes_ctx.
Signed-off-by: Arunachalam Ganapathy <arunachalam.ganapathy@arm.com>
Change-Id: Ic92ed0850886bbb9c4532276b76847a8c426bc23
The GICv3 is architectured to solely manage interrupts targeted to
Normal and Secure world. It doesn't manage interrupts targeting the
more recently introduced Realm world. Hence the new RMMv2.0
specification mandates that EL3 should not save and restore
the GIC registers on a world switch. This change is not backward
compatible with RMMv1.x ABI.
Note the change in implementation of cm_el2_sysregs_context_save()
and cm_el2_sysregs_context_restore() API as GIC state is not
managed by these APIs anymore.
Add new build flag RMM_V1_COMPAT to support backward compatibility
with RMMv1.x. This flag is currently enabled by default.
This patch is a reworked version of the original patch at:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/45658
NOTE: If RMM_V1_COMPAT is not enabled, then RMM_EL3_IFC_VERSION
is bumped to 1.0 which makes it incompatible with an RMM supporting
0.x.
Change-Id: If4c53b85ef31091c254b383ed7b32c39124f0dbb
Signed-off-by: Shruti Gupta <shruti.gupta@arm.com>
MISRA violation C2012-4.10: Precautions shall be taken in order to
prevent the contents of a header file being included more than once.
Used include gaurds #ifndef to fix this violation.
Change-Id: Icbb6321007b768f580d681612dd11541fc4f9fe0
Signed-off-by: Suraj Kakade <suraj.hanumantkakade@amd.com>
This corrects the MISRA violation C2012-7.2:
A “u” or “U” suffix shall be applied to all integer
constants that are represented in an unsigned type.
Suffix "ULL" is added to unsigned integers to fix
this violation.
Signed-off-by: Suraj Kakade <suraj.hanumantkakade@amd.com>
Change-Id: I5398ff9fd5008cc0d98f822e48bf243cdbf5b083
This reverts commit c84cf19308299de9ad68c340a4c4744a0fe2f18a.
Reason: Linux Boot in Realm test fails with this patch because the
NS sets up ich_hcr_el2 which is not expected by RMMv1.0. The GICv3
context switch changes will need to wait till RMM migrates to
RMMv2.0.
Change-Id: I39185bd08f35bc0836d2ef199c5d930d62ee23d2
Signed-off-by: Soby Mathew <soby.mathew@arm.com>
The GICv3 is architectured to solely manage interrupts targeted to
Normal and Secure world. It doesn't manage interrupts targeting the
more recently introduced Realm world. Hence the new RMMv2.0
specification mandates that EL3 should not save and restore
the GIC registers on a world switch. This change is backward
compatible with RMMv1.x ABI.
NOTE: Change in implementation of cm_el2_sysregs_context_save()
and cm_el2_sysregs_context_restore() API as GIC state is not
managed by these APIs anymore.
Change-Id: I24d7fa26503ffad9d9fede21d8449f481e32984e
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
Signed-off-by: Shruti Gupta <shruti.gupta@arm.com>
There are 2 cases in which an erratum will not be found in the list:
a) there is no workaround implemented
b) there is a workaround implemented but it has not been compiled in
Neither case implies that the erratum does not apply - for option a) it
could mean that the erratum is newer than TF-A's awareness and in option
b) it could mean that the flag was forgotten to be set.
Unfortunately, this can't be done in isolation and must be accompanied
by untangling the complicated relationship between CVE identifiers and
the return codes to ensure everything remains the same. First, make
the CVE_2017_5715 and CVE_2022_23960 relationship in the WA_3 SMC call
explicit instead of relying on the checker functions. Then, add semantic
defines for the return values of the workarounds as 0, 1, and -1 are
ambiguous and confusing. This allows the application of a consistent
return pattern.
Change-Id: Ibfae2cd06212dc59b4730a6dca6e9aee1f341609
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
MISRA violation C2012-10.4:
Both operands of an operator in which the usual arithmetic
conversions are performed shall have the same essential type
category.
Change-Id: I193b49035f3870f823370a70d5cc5aef87756467
Signed-off-by: Suraj Kakade <suraj.hanumantkakade@amd.com>
This feature only needs MDCR_EL3.EnSTEPOP to be written and mdstepop_el1
to be context switched when the next EL is EL1.
Change-Id: I70e2a488f4e50da4b181a00648c4f608e1da451c
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>
* changes:
feat(qemu): disable fpregs traps for QEMU in BL31
feat(crypto): enable the runtime instrumentation for crypto extension
feat(crypto): enable access to SIMD crypto in BL1 and BL2
feat(crypto): enable floating point register traps in EL3
feat(crypto): build flag for SIMD crypto extensions for v8+ platform
refactor(build): add a default filter list for lib cflags
* changes:
fix(libfdt): resolve misra 10.3 violations
feat(lib): use C/assembler for HI/LO macros
fix(libfdt): adding missing curly braces
fix(libfdt): fix misra 14.4 and 15.6 violations
fix(libfdt): typecast operands to match data type
