This blog aims to introduce the concept of Trusted Execution Environment (TEE) and how end users can leverage open source software to safely deploy applications that require handling confidential information.
Trusted Execution Environment (TEE) Overview
What is a TEE? What are the benefits of having a TEE?
TEE provides an isolated environment to ensure code/data integrity and confidentiality. Typical embedded systems running Linux or Android are exposed to a large number of security vulnerabilities in both the kernel and user space packages. Vulnerabilities can allow an attacker to gain access to sensitive information and/or insert malware. TEE adds an additional layer of security where code/data running on the TEE can not be accessed/tampered from the normal world OS (eg: Linux/Android). The software running on TEE (secure world) typically involves a tiny security oriented operating system (eg: OP-TEE OS) along with trusted applications. The trusted applications are meant to handle confidential information such as credit card PINs, private keys, customer data, DRM protected media, etc. and provide services to the normal world OS to make use of the confidential information without compromising it.
How is TEE implemented?
TEE requires both software and hardware (built into the processor) support.
On the hardware side, ARM based processors achieve TEE using TrustZone technology. TrustZone enables a single physical processor core to execute code safely and efficiently from both the normal world (Rich OS like Linux/Android) and the secure world (Security OS like OP-TEE). This allows high performance security software to run alongside the normal world operating environment. TrustZone implements a ‘state’ based memory and IO protection. i.e. when the processor is running in the secure state/context (secure world), it has a different view of the system and has access to memory/peripherals that normally can not be accessed from a non-secure state/context (normal world). The two virtual processors context switch via monitor mode when changing the currently running virtual processor.
Simplified hardware view of ARM TrustZone security
On the software side, there is a normal world OS (eg: Linux, Android etc) and a secure world OS (eg: OP-TEE, Trusty, QSEE, SierraTEE, etc.) both running in privileged mode. Similarly, there are user applications in normal world and trusted user applications in secure world both running in user mode. The secure world trusted applications/OS are meant to provide security related services to the normal world user applications.
Simplified software view of ARM TrustZone security
Choosing your secure world OS
Global Platform, a nonprofit organization, has developed TEE API and Framework specifications to standardize TEE and avoid fragmentation. There are various specifications available for TEE Client, Core, etc. that specify interactions between a Trusted Application and Secure World OS, a Trusted Application with another Trusted Application, a Client Application with a Trusted Application, and so on. This allows end users to switch a secure OS without having to change their Trusted Applications. When choosing your secure world OS, there are both commercial/proprietary and open source options. The choice of OS depends on the required features and level of support needed, but, it would be prudent to choose one that is compliant with Global Platform TEE specifications.
Open-source Portable TEE (OP-TEE)
OP-TEE is a open source implementation of TEE. OP-TEE comprises of secure world OS (optee_os), normal world client (optee_client), test suite (optee_test/xtest) and Linux driver. The OS and client have a BSD 2-clause license and are Global Platform compliant. There are 28+ platforms/processors supported and actively maintained by Linaro. Below is a software architecture diagram of the OP-TEE. The product development team is responsible for developing a client application (CA) running on Linux and a trusted application (TA) running on OP-TEE. The CA uses the TEE client API to talk to the TA and avail secure services from it. The CA and TA used shared memory to pass data between each other.
OP-TEE software Architecture (Image Credit: Linaro)
Getting started with OP-TEE
Assuming the SoC you are using is already supported by OP-TEE OS, below are the options for building the software:
- Minimal OP-TEE build system: To help jump start running OP-TEE, some board maintainers (eg: Raspberry Pi 3 and TI AM43xx/AM57xx) provide a minimal build system that generates all the required images (bootloaders, OP-TEE OS, Linux kernel, OP-TEE client and a minimal RFS).
- Yocto Project build system: Include the meta-optee layer in your build and add the relevant packages to local.conf
- Manual: Cross-compile and build the various parts
Note: Various features in OP-TEE OS can be enabled/disabled by providing CFG_
For a custom board with a different memory size, you might need to adjust the secure (CFG_TEE_RAM_START, CFG_TA_RAM_START) and shared (CFG_SHMEM_START) memory locations in platform configuration file. Some platforms also allow the code to run from internal SRAM (CFG_WITH_PAGER).
Example OP-TEE memory map
In case the SoC you are using is not supported, contact your SoC vendor or DIY add support reaching out to OP-TEE maintainers, or seek commercial support (Timesys, WindRiver, …). You also have an option of testing out OP-TEE using a QEMU.
Bootloader support and boot flow
Ideally, the OP-TEE binary must be loaded as early as possible in the boot process (vulnerabilities in bootloaders running prior to OP-TEE could potentially compromise sensitive data). In a typical Linux boot (without TEE), the ROM bootloader loads/executes a 1st stage bootloader (eg: SPL, MLO, SBL1, FSBL) which then executes a 2nd stage bootloader (eg: U-Boot, LittleKernel) which executes the Linux kernel, all from a secure world context. On an ARMv7 based processor, the typical boot flow with TEE is: SPL loads OP-TEE and U-Boot, jumps to OP-TEE and once OP-TEE is done with initialization, it switches to non-secure context and jumps to U-Boot. The OP-TEE code will continue to reside in memory to provide secure services to the Linux kernel. On an ARMv8 based processor, the TEE boot flow involves an additional step of SPL loading ARM Trusted firmware, along with OP-TEE and U-Boot. SPL jumps to arm trusted firmware which later hands control to OP-TEE which in-turn jumps to U-Boot in non-secure context. On an ARMv8 platform, ARM Trusted firmware provides the monitor code to manage the switch between secure and non-secure world, whereas it is built-in to OP-TEE for ARMv7 platforms.
Note: OP-TEE needs to know the jump address to the non secure world (eg: u-boot or kernel load address). This can be provided either at build time (CFG_NS_ENTRY_ADDR) or SPL needs to configure ARM register (r1) with the load address before jumping to OP-TEE.
Different boot flows when using a TEE
A Linux kernel driver for OP-TEE is available in kernel 4.12 or higher. If you are running a older version of kernel, then you will need to backport patches. You will also need to enable the driver following below steps:
- Set CONFIG_OPTEE=y in your kernel config
- Add device tree nodes for OP-TEE as shown here
- Alternatively compiling OP-TEE OS with CFG_DT=y can modify the dtb at runtime to add the required nodes
Troubleshooting OP-TEE issues
Once you have bootloader, Linux and OP-TEE OS running, you may potentially run into issues on your custom board. Below is a guide to understand and work around some common issues:
- Imprecise aborts: These are typically seen when a secure peripheral and/or memory is being accessed by the normal world OS. It is recommended to review the permissions of peripheral/memory regions.
- Memory map conflicts: The load address and runtime address of OP-TEE OS and kernel/dtb/ramfs need to be reviewed carefully, otherwise it could lead to issues like the kernel trying to relocate itself to the OP-TEE runtime region, etc.
- Clock setup: Linux kernel disables clocks to any unused peripherals but the peripheral might be used by the secure world OS which Linux is not aware of. One way to work around this is to include “clk_ignore_unused” as part of Linux kernel bootargs.
- Resource conflict: If a peripheral is being accessed by both normal and secure world code, there is no locking mechanism which could lead to conflicts. In the absence of a locking mechanism, it is recommended that only one of the world owns access to the resource and that it provide services to the other world to interact with the resource.
Trusted Applications (TA)
There are many tutorials available for trusted application development, hence this blog will concentrate on a high-level overview. For getting started with writing a CA and TA, refer to the hello world application.
There are 2 types of trusted apps: Dynamic and Pseudo/Static Apps
- Dynamic Apps reside in the normal world file system (RFS) and gets loaded into the secure world user space at runtime when a Linux client application wants to use it. The TAs are signed and OP_TEE OS verifies the signature before executing.
- Psuedo/Static Apps are typically built as part of the OP-TEE kernel and run in kernel mode. These apps mostly deal with providing services that involve controlling hardware which is difficult to implement in Dynamic App that runs in user space.
OP-TEE OS provides secure data storage facility for TAs. The data is stored either on the Linux filesystem (/data/tee) in encrypted/authenticated (AES-GCM) manner or on an eMMC RPMB partition.
Real world example
TEE can be used to reduce the hardware cost by replacing a dedicated security chip (key storage or crypto authentication) with a software solution.
To access the hardware features, many of the security chips provide an OpenSSL engine interface to user space applications. A similar model can be used by developing a TEE client app as part of OpenSSL engine to interface with the TA, thus minimizing any changes to user space applications. The Trusted Application will need to implement handlers for key management and crypto operations. OP-TEE OS includes libtomcrypt which provides various symmetric/asymmetric/elliptic curve cryptography functions. So the TA is mostly responsible for input validation and calling the appropriate OP-TEE core APIs. Below is an example architecture to achieve the same.
Example architecture for TEE protected cryptographic operations
There are system/platform level security considerations that need to be reviewed before deploying a TEE. Some of the common ones are covered below:
- JTAG: On a production unit, disable JTAG or limit JTAG to non-secure access only to ensure secure world data cannot be accessed/modified over JTAG.
- Secure boot: In order to ensure only authenticated code runs on the device, secure boot and chain of trust must be established. Refer to this blog for more details.
- Permission review: Some OP-TEE platforms do not explicitly set the permissions for non-secure world and defaults to allowing access to all peripherals/memory. Further, the registers configuring these permissions must also be explicitly set as secure access only in order to prevent normal world code from changing the permissions. Once the permissions are set appropriately, use devmem2/memtool from Linux user space to access secure memory regions, and verify a bus error is reported.
- Review use of keys: Secure storage depends on unique hardware key being provided to OP-TEE OS. Refer to this documentation to ensure it is setup correctly for your platform.
Using a TEE is a relatively inexpensive method to add an additional security layer to your device. The availability of open source software to leverage TEE makes deploying trusted applications easy. Apart from the added security benefits, it also has the potential to lower the hardware cost on platforms using a dedicated security chip/co-processor.
Akshay Bhat is a Security Architect at Timesys. Akshay’s experience with embedded systems spans a broad range of industries with a focus on board bring-up, driver development and software security. Akshay received his MS in Electrical Engineering from NYU Polytechnic University.
Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.