TVRAVNF: an efficient low-cost TEE-based virtual remote attestation scheme for virtual network functions

Abstract

With the continuous advancement of virtualization technology and the widespread adoption of 5G networks, the application of the Network Function Virtualization (NFV) architecture has become increasingly popular and prevalent. While the NFV architecture brings a lot of advantages, it also introduces security challenges, including the effective and efficient verification of the integrity of deployed Virtual Network Functions (VNFs) and ensuring the secure operation of VNFs. To address the challenge of efficiently conducting virtual remote attestation for VNFs and establishing trust in virtualized environments like NFV architecture, we propose TVRAVNF, which is a highly efficient and low-cost TEE-based virtual remote attestation scheme for VNFs. The scheme we proposed ensures the security and effectiveness of the virtual remote attestation process by leveraging TEE. Furthermore, we introduces a novel local attestation mechanism, which not only reduces the overall overhead of the virtual remote attestation process but also shortens the attestation interval to mitigate Time-Of-Check-Time-Of-Use attacks, thereby enhancing overall security. We conduct experiments to validate the overhead of the TVRAVNF scheme and compare its performance with that of a typical remote attestation process within a maximum unattested time interval. The experimental results demonstrate that, by employing the local attestation mechanism, our solution achieves nearly an 80% significant performance improvement with a relatively small time overhead for small to medium-sized files. This further substantiates the significant advantages of our approach in both security and efficiency.

Introduction

Since the proposal of the Network Function Virtualization architecture (Yang et al. 2020; Bonfim et al. 2019) in 2012 to address issues such as poor flexibility, low scalability, and high hardware cost in traditional network architectures, NFV technology has been increasingly applied in various scenarios such as the 5G core network (Yousaf et al. 2017; Barakabitze et al. 2020), Internet of Vehicles (Li et al. 2021; Zhuang et al. 2019), and Internet of things (Alam et al. 2020; Lv and Xiu 2019). The core principle of NFV technology involves virtualizing network functions (Zhang 2019; Laghrissi and Taleb 2018), such as firewalls, routers, and load balancers, and running them in the form of Virtualized Network Functions (VNFs) on general-purpose hardware. This decouples the network functions from the hardware form in traditional network architectures. The resulting virtualization implementation provides programmability and flexibility to network functions, enabling users to deploy, configure, and scale network functions according to their specific needs.

Despite the significant progress made by the NFV architecture in enhancing network performance and flexibility, there are still important challenges that urgently need to be addressed in practical deployment (Gonzalez et al. 2018; Lal et al. 2017b). One of these challenges is how to effectively ensure the security and integrity of critical VNFs in the NFV environment. Traditional network security solutions have struggled to adapt to the dynamic and highly variable deployment of network functions in virtualized environments. Additionally, the number of VNFs deployed simultaneously within a timeframe is typically substantial. This implies that verifying the integrity of a large-scale deployment of VNFs often incurs significant overhead. Therefore, there is a pressing need for an efficient, low-cost solution to validate the security and integrity of VNFs.

In response to numerous security threats faced by VNFs, such as unauthorized access, data tampering, and leakage, researchers have proposed various solutions. Lal et al. (2017a) propose the deployment of vVPNs to safeguard the security of communication traffic between VNFs. Shih et al. (2016) introduce the S-NFV scheme, which utilizes SGX to securely isolate the state of NFV applications, protecting them from tampering. Paladi and Karlsson (2017) leverage SGX along with the Linux Integrity Measurement Architecture to preserve the confidentiality of VNF authentication credentials. While researchers have offered diverse solutions to safeguard VNF security, existing approaches often address only specific aspects of VNF security and lack a systematic approach to verify VNF integrity comprehensively. Furthermore, these solutions often rely on designated hardware technologies such as SGX, making them less conducive to universal deployment across heterogeneous environments. Therefore, a generic, technology-agnostic systematic approach to VNF protection represents a promising direction.

In this paper, we propose an efficient and low-cost Trusted Execution Environment (TEE) based virtual remote attestation scheme for measuring the integrity of VNFs, termed TVRAVNF. Unlike conventional virtual remote attestation approaches, TVRAVNF is tailored to the NFV architecture, enabling more effective and virtual remote attestation of the flexible and dynamic nature of VNFs to ensure their integrity and enhance the overall security of the NFV environment. Additionally, TVRAVNF introduces a Local Attestation mechanism, significantly reducing the overhead associated with remote attestation cycles. This enhancement improves attestation efficiency, mitigates risks arising from extended attestation intervals, and facilitates quicker detection of potential security threats. Therefore, TVRAVNF is a specialized, efficient, and resource-friendly virtual remote attestation scheme designed specifically for NFV architectures, making a substantial contribution to enhancing the overall security of NFV environments.

Contribution

In this paper, we have made the following significant contributions:

• 1. We introduce a novel architecture, specifically tailored for NFV environments, known as TVRAVNF, to address challenges such as the lack of universality and high overhead associated with VNF attestation. Through the utilization of our proposed architecture, not only can the VNF attestation challenge be efficiently addressed, but it also safeguards critical data during VNF runtime from potential tampering by attackers.

• 2. The TVRAVNF scheme we propose is an open approach that is hardware-agnostic, supporting various mainstream TEE environments. Our design effectively avoids vendor lock-in, ensuring flexibility and compatibility with different TEE environments.

• 3. Our TVRAVNF scehme not only exhibits strong security measures but also ensures overall efficiency and low cost by introducing the Local Attestation mechanism. This dual focus on security and efficiency enhances the effectiveness of the scheme.

• 4. Through extensive experiments, we meticulously evaluate the overhead, efficiency, and security performance of the TVRAVNF scheme. The results validate that our approach enhances NFV security without imposing unnecessary burdens on overall performance.

Roadmap

The subsequent sections of the paper follow a structured progression. Section 2 provides a comprehensive overview of the relevant background and related work, setting the context for the proposed TVRAVNF scheme. Section 3 delves into the intricacies of the TVRAVNF scheme, elucidating its complete process, architecture, and specific implementation details. Following this, Sect. 4 showcases the deployment of experiments, offering insights into the measured overhead and performance of TVRAVNF. Section 5 engages in an in-depth analysis and discussion of the security performance of the TVRAVNF scheme. Finally, Sect. 6 serves as the conclusion, summarizing the key findings and contributions of the entire paper.

Background and related work

In the following, we introduce the virtualization of TEE technology and the limitations of the existing virtual remote attestation scheme typically used in today’s virtualized environments.

Trusted execution environment

The Trusted Execution Environment technology is dedicated to providing a secure and trustworthy execution environment (Lee et al. 2020). Its core principles involve key features such as hardware isolation, authentication mechanisms, a trusted computing environment, and secure storage. At the hardware level, TEE relies on specialized security processors, such as ARM TrustZone (Pinto and Santos 2019), AMD SEV-SNP (Sev-Snp 2020) or Intel SGX (Zheng et al. 2021), ensuring physical isolation from the regular execution environment through hardware isolation techniques. On the software front, TEE requires support for runtime environments, APIs, and drivers.

With the emergence of virtualization technology and the continuous evolution of cloud computing environments and NFV architectures, the demand for TEE usage in virtualization environments is steadily increasing. Consequently, numerous virtualization solutions for TEEs (Li et al. 2019) have been continually proposed. Hua et al. (2017) virtualizes ARM TrustZone as vTZ, securely providing a virtualized TEE for each customer virtual machine using existing hardware. In Zhao et al. (2022), virtualization of Intel SGX enclaves is achieved on an AMD SEV virtual machine, providing the virtual machine with security protection equivalent to that of hardware-based SGX. Additionally, various schemes, including Zhao et al. (2019) and Xia et al. (2021), have successfully implemented the deployment and utilization of TEE in virtualization environments.

Virtual remote attestation

Remote Attestation (Ambrosin et al. 2020; Aman et al. 2020; Dushku et al. 2020) technology generates evidence by measuring the software and hardware states of a remote system, ensuring the integrity and trustworthiness of the system. The basic process includes measurement, attestation generation, attestation transmission, and attestation verification, often relying on hardware-based trusted roots such as TPM, TCM, etc. In distributed systems, cloud computing environments, and NFV architecture environments, traditional RA schemes no longer suffice for the complex and dynamic virtualization landscape. Therefore, VRA schemes have been proposed to address the remote attestation challenges in virtualized environments. Narayanan et al. (2023) designs and implements a vTPM with SEV-SNP hardware isolation as a virtual trusted root, realizing a stateless virtual remote attestation scheme through it. This VRA scheme can resist attacks on the state of the vTPM and provides robust support for virtualized environments. However, it requires specific AMD secure hardware and TPM hardware. Kucab et al. (2021) proposes a VRA scheme for virtual machines based on Intel SGX. This scheme has a small TCB and can dynamically adjust attestation strategies, offering good flexibility. Its drawback is the dependence on security processors that support Intel SGX technology, introducing hardware dependencies. Apart from VRA schemes based on specific confidential computing technologies, there are also numerous VRA schemes tailored for cloud environments. Some focus on large-scale scenarios with tens of thousands of virtual machines, such as Cheng et al. (2021). Others specifically target verifying the runtime integrity of virtual machines, as exemplified by Ozga et al. (2021). Nevertheless, these approaches did not take into account how to address the challenges specific to NFV scenarios. Conventional VRA schemes typically focus on verifying the integrity of the VM itself, often overlooking or considering only a limited number of software programs running within the VM. Such VRA schemes often fail to effectively address the challenge of verifying the integrity of VNFs in NFV environments. VM-based VRA schemes offer coarse granularity for VNFs, merely ensuring the security of the environment in which VNFs operate. They cannot withstand attacks from malicious VNFs and are not suitable for addressing the attestaion challenges of numerous and rapidly evolving VNFs. Therefore, the development of a VRA scheme tailored specifically for NFV scenarios is a worthwhile research direction.

Despite the progress made in VRA technology, there are still unresolved challenges, including dependencies on specific technologies and designated hardware, the lack of specialized solutions for specific scenarios such as NFV environments, and the excessive overhead introduced by virtual remote attestation schemes. Addressing these challenges will be the focus of our research.

TVRAVNF

In this section, we will provide a detailed exposition of the overall architecture of the TVRAVNF scheme, the threat model, the specific implementation and process of virtual remote attestation, as well as the local attestation mechanism.

The TVRAVNF architecture

The TVRAVNF architecture is a lightweight design that avoids complex components and redundant functionalities. The Fig. 1 illustrates the overall architecture of TVRAVNF deployed in the NFV environment. The key core component of the TVRAVNF architecture is a secure processor with TEE capabilities. Therefore, our proposed architecture can be easily deployed in NFV environments.

The TVRAVNF architecture creates a TEE within virtual instances (typically virtual machines) by deploying a secure processor at the NFVI layer. Once the TEE is established, the results of cryptographic operations, such as encryption, decryption, signing, and hashing, performed by the attestation program running in the TEE, are considered secure and trustworthy. This ensures the authenticity and validity of the attestation results obtained in subsequent Virtual Remote Attestation and Local Attestation processes.

Fig. 1

The architecture of TVRAVNF

Furthermore, the TVRAVNF architecture leverages the secure isolation features of TEE to store critical sensitive data required during the runtime of certain VNFs. Examples of such data include dynamic routing tables for virtual routers, protection rules for virtual firewalls, and network configuration information for virtual load balancers. Storing this data in the TEE prevents malicious tampering by attackers. Consequently, our architecture not only safeguards the integrity of the VNF itself but also prevents unauthorized changes to critical sensitive data during VNF runtime, providing comprehensive protection for the security integrity of VNFs.

Threat model

In practical deployment environments, NFV architecture often faces a myriad of attacks. Malicious attackers may compromise various layers of the NFV architecture, including the hardware layer, virtualization layer, MANO layer, and even the virtualized instances running VNFs. Faced with attackers wielding significant capabilities, the components we can genuinely trust are minimal. Therefore, as illustrated in the Fig. 1, in the TVRAVNF architecture, we only trust the secure processor and the TEE it creates. As long as the attacker cannot breach the security processor, the TEE remains secure, and the data stored within it, along with the VNF and VM remote attestation results obtained through the TEE, are secure and trustworthy. It is essential to note that we do not consider defending against side-channel attacks and physical attacks on the secure processor, as these fall within the purview of hardware manufacturers. Nevertheless, existing solutions (Schwarz et al. 2019, 2020; Shih et al. 2017) can be employed to mitigate such attacks.

Fig. 2

Virtual remote attestation

Remote attestation phase

As a lightweight scheme, the virtual remote attestation process of TVRAVNF is not overly complex. As depicted in Fig. 2, the virtual remote attestation process of TVRAVNF is divided into two main phases: TEE remote attestation and VNF remote attestation.

In the first phase, remote attestation for the TEE is conducted. Depending on the specific implementation of the security processor and TEE, there may be some details that differ. However, verification can be performed by following the corresponding attestation services provided by the hardware processor manufacturer. It is noteworthy that the TVRAVNF scheme is designed to be universal and independent of specified hardware. Therefore, users can employ the TVRAVNF scheme with any security processor, as long as it possesses the capability to provide a TEE. This design choice enhances scalability and adaptability, mitigating cost concerns associated with Vendor lock-in. Once the remote attestation for the TEE is completed, we can be confident that the TEE is created and operated by a genuine security processor. Consequently, storing data and performing remote attestation operations within the TEE is deemed secure.

After verifying the trustworthiness of the TEE, we proceed to the second phase, which involves remote attestation for the VNF. Following the flow depicted in Fig. 2, the Attestation Server initiates a remote attestation request through a secure channel established via TLS with the attestation program deployed in the TEE. Upon receiving the remote attestation request, the verification program in the TEE primarily needs to gather evidence from both the VM and VNFs. For the VNFs, we hash all VNFs running in the VM and combine these hashes with critical operational data hashes stored in the TEE, such as the routing table for a virtual router or protection rules for a virtual firewall. This combined information is then signed and encrypted as evidence for the integrity of the VNFs. Additionally, besides validating the integrity of the VNFs themselves, we also need to ensure the integrity of the VM in which the VNFs operate. Even if the VNFs are secure, compromised control over the VM can jeopardize the overall system security. To verify the VM, we typically collect hashes of critical components such as the VM kernel, important programs, image snapshots, security patches, vulnerability patch records, configuration files, and library files. By verifying the integrity of these crucial parts of the VM, we ensure that the VNFs operate in a secure environment. After gathering this evidence, a secure channel is established via TLS to transmit the evidence to the Attestation Server for evaluation. Once the evaluation is completed, the Attestation Server returns the remote attestation result to the TEE, concluding a round of virtual remote attestation.

Following the virtual remote attestation process of TVRAVNF, the verifier can use the attestation results to identify which virtual machines and the corresponding running VNFs are trustworthy and which ones are not. This determination serves as the basis for subsequent actions and decisions.

Local attestation mechanism

In comparison to other conventional virtual remote attestation solutions, TVRAVNF holds a significant advantage by incorporating a local attestation mechanism. Ordinary virtual remote attestation schemes often struggle to strike a balance between efficiency and security. Attempts to enhance efficiency and reduce overhead in typical approaches involve decreasing the frequency of remote attestations, thereby elongating the time intervals between two successive remote attestations. However, this approach grants attackers a larger window for potential exploits, providing attackers with opportunities for Time-Of-Check-Time-Of-Use (TOCTOU) attacks (De Oliveira Nunes et al. 2021), compromising the overall security. Therefore, achieving a harmonious balance between efficiency and security is challenging for conventional virtual remote attestation solutions.

Fig. 3

Local attestation mechanism

The TVRAVNF scheme introduces a local attestation approach, thereby not only enhancing attestation efficiency and reducing overall overhead but also shortening the attestation intervals. This serves to prevent TOCTOU attacks, thereby boosting security. In TVRAVNF, the designed local attestation is not the conventional attestation conducted in opposition to remote attestation; instead, it involves the Verifier locally attesting the VNFs and VM. This is achieved by storing the hashed evidence of the verified VNFs and VM in the TEE during the remote attestation phase.

The key distinction between the local attestation process and the remote attestation process lies in the fact that the measured hash value does not need to be compared with a hash value from a remote attestation server. Instead, it is compared only with the locally stored hash value in the TEE. This approach saves on numerous overheads, including encryption, signing, and encrypted network communication. Therefore, compared to the remote attestation process, the local attestation process incurs minimal overhead. Moreover, since the local attestation process is brief and entails minimal time expenditure, it allows for multiple local attestations within a short timeframe without imposing a substantial burden on the system. Consequently, for attacks that require intervals within a certain time frame and aim to restore the VNF to its original state after the attack, such as TOCTOU attacks, the intervals between two consecutive local attestations are too short to complete such attacks. Thus, the TVRAVNF solution effectively defends against TOCTOU attacks, a challenge that traditional virtual remote attestation approaches struggle to address.

Figure 3 illustrates the detailed process of the local attestation. As depicted, when the previous virtual remote attestation is successful, the TEE stores the previously measured critical hashes of VNF and VM as Local Hash. Subsequently, after a short user-defined time interval, the local attestation process begins. The local attestation process primarily involves two steps: first, collecting the attestation evidence for the VNF, and second, hashing the attestation evidence and comparing it with the locally stored hash. Upon completion of the local attestation, if the attestation fails, trust is naturally revoked for the VNF or VM, and further measures are taken to restore system trust. If the attestation succeeds, it is checked whether it has exceeded the predefined maximum time interval without remote attestation. If not exceeded, it continues to wait for the time interval for the next local attestation; otherwise, it directly requests the next remote attestation process. In summary, local attestation primarily serves to expedite the detection of attacks, reduce the attackers’ time window, and lower overall costs. However, it cannot completely replace the role of remote attestation, and the attestation server ultimately relies on remote attestation results. Therefore, local attestation mainly complements remote attestation.

With the introduction of the local attestation mechanism, users can flexibly meet their varying security needs by adjusting the maximum time interval without remote attestation and the local attestation time interval, all while maintaining efficiency and low consumption.

Evaluation

Currently, Trusted Execution Environment technologies have reached a high level of maturity, with widespread usage. The most prevalent TEE technologies include ARM TrustZone, Intel SGX, and AMD SEV, etc. Although theoretically, TrustZone technology could also support the deployment of our TVRAVNF, it is primarily designed for ARM architecture. Given that the application scenarios of TVRAVNF are primarily targeted at Network Function Virtualization architecture, which commonly relies on x86 architecture, we have opted for Intel SGX technology in our experiments to establish a TEE environment for validating the effectiveness and evaluating the performance of our TVRAVNF scheme. Furthermore, besides selecting SGX as the specific hardware testing environment, to demonstrate the generality of the TVRAVNF scheme across various TEE environments, we also implemented a simplified version of a generic TEE environment to assess the usability and security of our approach.

In this section, we will comprehensively delve into the implementation details of TVRAVNF, considering various aspects such as overhead and performance metrics.

Implementation

We implemented a prototype of our scheme in the SGX environment using approximately 2600 lines of C (LoC) code and around 200 lines of Python code. Among these, approximately 250 LoC were dedicated to implementing relevant ecall functions and other functionalities executed within the Enclave. Additionally, about 700 LoC were allocated for implementing OCall functions and the Attestation Client functionality. Around 1200 LoC were utilized for realizing SGX enclave remote attestation, while approximately 450 LoC were devoted to implementing a non-SGX remote attestation scheme for comparative purposes. An additional 200 lines of Python code were employed to implement the Attestation Server functionality. The code deployment structure related to the enclave is illustrated in Fig. 4.

In the generic TEE scenario, we employed approximately 1200 lines of Python code to simulate a simplified TEE environment with basic TEE functionalities, including secure environment, secure communication interface, secure storage, secure computation, and secure authentication and authorization. Within this environment, we evaluated the effectiveness of our scheme and its resilience against various types of attacks.

We conducted our experiments on a DELL OptiPlex 5050 machine. This machine is equipped with an Intel Core i7-7700 4-core processor running at a frequency of 3.60 GHz, and it has a memory capacity of 16GiB. The physical machine, serving as the host, operates on Ubuntu 18.04.6 LTS with a Linux kernel version of 5.13. The virtual machine with SGX support, deployed using Qemu version 2.11, maintains consistency with the host machine’s operating system and kernel. We deployed the Attestation Server on a separate DELL OptiPlex 7000 server running Ubuntu 20.04, equipped with an i7-12700 processor and 16GiB of memory. These two systems were interconnected via a local area network (LAN). The average network latency between the two machines was measured at 1.528 ms.

Fig. 4

Code deployment structure

To minimize the randomness in measurement data and enhance the reliability of the results, unless otherwise specified, we conducted each experiment 50 times and computed the average. The final average value obtained from these repetitions was used as the measurement result.

Overhead

Fig. 5

Average TVRAVNF remote attestation time for 100 files

Fig. 6

Average TVRAVNF local attestation time for 100 files

We primarily considered three scenarios to evaluate the overhead of TVRAVNF: the Physical Machine scenario, the Physical SGX scenario, and the Virtual SGX scenario. In the Physical Machine scenario, we conducted experiments by directly deploying the attestation scheme on the physical host machine. In this scenario, there is no virtualization or the overhead introduced by SGX; thus, only the attestation process overhead is present. Consequently, we utilized this scenario as the baseline for comparing performance overhead. The Physical SGX scenario involves deploying the attestation scheme on the physical host machine with the introduction of the SGX environment. This scenario primarily reflects the performance overhead introduced by incorporating SGX technology. The Virtual SGX scenario is designed to closely mimic a real deployment environment. In this scenario, we deployed the SGX enclave within a virtual machine and executed our TVRAVNF scheme within the enclave. By comparing the overhead in this scenario with the two aforementioned scenarios, we can assess the magnitude of the overhead associated with TVRAVNF.

In each of these three scenarios, we employed the TVRAVNF scheme to measure the time required for remote attestation and local attestation for three sets of files, each containing 100 VNF’s files of sizes 1 KB, 1 MB, and 1 GB. By comparing the measured times, we can calculate the specific overhead of the scheme. It is important to note that in real-world scenarios, VNF sizes typically range in the megabyte (MB) level. Therefore, the measurement time for a 1 MB size is closer to real-world conditions. Additionally, we selected 1 KB and 1 GB for testing to showcase the performance of the proposed solution in extreme scenarios.

Remote attestation overhead

Initially, we measured the remote attestation overhead of the TVRAVNF scheme. As depicted in Fig. 5, it illustrates the time taken by the TVRAVNF scheme to perform remote attestation for one hundred VNF program files. In Fig. 5a, for small files with an average size of 1 KB, the remote attestation by TVRAVNF does not impose significant overhead. In the Physical SGX scenario, it only adds approximately 7.3% overhead compared to the bare physical machine, and in the Virtual SGX scenario, it introduces only about 15.7% overhead. The added time of 0.39s in practical scenarios is imperceptible and acceptable. Similarly, for files with an average size of 1 MB, as shown in Fig. 5b, the total time required for attestation is less than 4 s, and the overhead compared to the Physical Bare-Metal scenario is less than 10.5%. For larger files, as illustrated in Fig. 5c with an average size of 1 GB, the added overhead for remote attestation is approximately 11.81%, which is not significantly impactful on the overall duration. In summary, employing the TVRAVNF scheme for remote attestation introduces a roughly ten to fifteen percent overhead. However, these additional costs are well justified by the enhanced security performance and assurance they provide, making them worthwhile in comparison.

Fig. 7

Enclave initialization overhead

Fig. 8

TEE attestation time

Local attestation overhead

The distinctive advantage of the TVRAVNF scheme lies in its local attestation mechanism. The presence of local attestation further reduces the overall overhead introduced by TVRAVNF’s remote attestation. Comparing Figs. 5a, b with Figs. 6a, b, it is evident that the time overhead for local attestation is significantly lower than that for remote attestation. This reduction is especially pronounced for small files of 1 KB, where the local attestation time is reduced by an order of magnitude, shifting from the second-level range to the millisecond level. Although there is a 174% increase in overhead when using the TVRAVNF scheme in a virtual environment for 1 KB small files, in reality, it is only a 1.53-millisecond increase in overhead. For larger files, such as 1 GB, the overhead increases by approximately 10.5%. In comparison to remote attestation, considerable time overhead can be saved with local attestation.

VSGX fixed overhead

In addition to the overhead introduced by the remote and local attestation processes discussed earlier, the virtualization of the VSGX solution itself also incurs some fixed overhead. This includes the enclave initialization time (Fig. 7) and the time for remote attestation of the SGX enclave (Fig. 8). In the VSGX environment, local attestation does not require remote attestation of the enclave but still incurs enclave initialization overhead. Remote attestation, on the other hand, involves both enclave initialization and remote attestation. As shown in the box plot in Figs. 7 and  8, in the VSGX environment, the average enclave initialization time is approximately 2.1 ms, and the time for remote attestation of the SGX enclave is around 6 s. Therefore, to minimize overhead from this perspective, it is natural to reduce the frequency of remote attestation and increase the frequency of local attestation, aligning with the design direction of the TVRAVNF scheme.

Fig. 9

The attestation time comparation between common RA scheme and TVRAVNF

Performance of TVRAVNF

To compare the performance of the conventional remote attestation approach with the TVRAVNF scheme, we designed an experiment as follows: Assuming a defined maximum time interval without remote attestation, a total of 10 attestation processes are required. Please note that we choose 10 iterations here solely for experimental measurement purposes. In practice, user should adjust the ratio between LA and RA appropriately based on the actual requirements, ensuring that the overall TVRAVNF process meets the desired security level. For the conventional remote attestation approach, all 10 processes need to be completed as full remote attestation flows. In contrast, for our proposed TVRAVNF scheme, only the first attestation process is a full remote attestation, and the subsequent 8 processes only require local attestation. The final attestation process is conducted remotely again when the maximum time interval without remote attestation is reached. In total, this results in 2 remote attestation processes and 8 local attestation processes to complete the entire attestation cycle. We conducted experiments in the Virtual SGX environment with 100 files of sizes 1 KB, 1 MB, and 1 GB, respectively. The experimental results are presented in Fig. 9.

From the results, it is evident that for small files of 1 KB and 1 MB, the TVRAVNF scheme can reduce overhead by nearly 80%. For larger files, such as 1 GB, it saves nearly 100 s. Therefore, the TVRAVNF scheme exhibits significant performance advantages compared to conventional remote attestation approaches, enhancing both security and efficiency by saving time and communication overhead. Moreover, since the maximum time interval without remote attestation can be flexibly adjusted by users based on their security requirements, the performance advantages of the TVRAVNF scheme can be further enhanced, providing substantial room for improvement.

Security

To assess the security of TVRAVNF, we tested its ability to defend against various attacks alongside the traditional VRA solution targeting VMs, within the deployed generic TEE simulation environment.

TOCTOU attacks

To evaluate the effectiveness of different solutions against TOCTOU attacks, we set up the following scenario to demonstrate the defense capabilities of each solution: Within a time interval (for simplicity, we chose 60 s), attackers initiate TOCTOU attacks randomly with durations of 5 s, 10 s, and 15 s. We measure the detection success rate of each solution against TOCTOU attacks of different durations and assess their overhead to demonstrate the advantages of each approach. The size of the VNF is set to 1 MB. TVRAVNF performs remote attestation every 60 s and local attestation every 5 and 10 s, denoted as TVRAVNF-5 and TVRAVNF-10, respectively. VRA solutions are denoted as VRA-10, VRA-20, and VRA-30, performing attestation every 10, 20, and 30 s, respectively. By comparing these three solutions with our TVRAVNF solution, we can visually analyze their effectiveness. Table 1 records the experimental results, showing the success rates and time overhead of each solution in defending against TOCTOU attacks of various durations.

Table 1 Comparison of success rates and overheads of various scheme in mitigating TOCTOU attacks

Table 2 Comparison of TVRAVNF and VM-VRA defense capacities against various attacks and their overheads

From the table, it is evident that compared to conventional VRA solutions, TVRAVNF can resist TOCTOU attacks with high intensity at a low overhead cost. For conventional solutions, increasing the success rate of defending against high-intensity TOCTOU attacks with short durations requires a higher frequency of remote attestation, leading to significant overhead. In contrast, TVRAVNF offers flexibility in enhancing security without introducing considerable overhead by adjusting the frequency of local attestation based on security requirements. This effectively addresses the limitations of conventional solutions. Additionally, due to the minimal overhead of TVRAVNF’s local attestation mechanism, which operates at the microsecond level, it can effectively defend against TOCTOU attacks with durations at the microsecond level.

Other attacks

In addition to TOCTOU attacks, we also conducted experiments on the simulated TEE environment to evaluate the defense capabilities of TVRAVNF and conventional VRA solutions against runtime data tampering,VM migration attacks and program code injection attacks. We have compared and documented the defense capabilities of the scheme in Table 2.

In the TVRAVNF scheme, critical runtime data of VNFs is stored in the TEE. Without compromising the TEE, attackers are unable to access or modify the runtime data of VNFs, thus enabling defense against runtime data tampering attacks. On the other hand, conventional VM-based VRA scheme typically only validate the integrity of VNF software, offering no isolation or protection capabilities. Therefore, they can only defend against program code injection attacks while being unable to validate or protect against runtime data tampering. Furthermore, the remote attestation process of TVRAVNF includes a step to verify the availability of the TEE, effectively identifying whether the system is operating in a secure and trusted environment. This capability prevents attackers from migrating virtual machines to unauthorized or compromised high-risk environments for further exploitation. Ordinary VM-VRA schemes lack such mechanisms and thus cannot counter such attacks. Additionally, as indicated in the previous section on overhead from experiments, TVRAVNF demonstrates significant advantages over conventional VM-VRA scheme in terms of overhead.

In summary, the conducted security experiments and their results underscore the outstanding security performance of the TVRAVNF scheme.

Security analysis

This section will analyze the security performance of the TVRAVNF scheme, highlighting its strengths and potential shortcomings.

Local attestation mechanism

The introduction of Local Attestation in the TVRAVNF scheme not only effectively reduces the overall process overhead but also avoids introducing new security vulnerabilities. The acquisition and synchronization of local attestation results are secure and do not leak sensitive information. The hashes required for local attestation do not need to be sent by the remote attestation server; they are simply retained from the previous remote attestation process, where the client collected evidence locally and hashed the results. In the remote attestation process, once the attestation server verifies that the received evidence hash is correct, it returns a successful attestation message to the client. Therefore, if the remote attestation succeeds, it implies that the evidence hash collected locally is also correct, and the client stores it for later use in local attestation. Additionally, updating the evidence hash for local attestation is convenient and quick. If a related VNF undergoes an update, the local attestation, relying on the previous remote attestation result, will fail to pass. In this case, according to the scheme, an immediate remote attestation will be triggered, as the remote server is aware of the update and will verify the evidence hash using the new information. Upon successful verification, the client will replace the previously stored local attestation evidence with the results of this measurement. This completes the synchronization between remote attestation and local attestation results. Moreover, both the evidence collection and hashing operations in both the remote and local attestation processes are executed within a TEE and remain protected within it. Therefore, no sensitive information is leaked during these processes, effectively ensuring the overall security of the scheme.

Maximum time interval without remote attestation

The introduction of the maximum time interval without remote attestation in the TVRAVNF scheme aims not only to reduce overall overhead but also to enhance the security performance of the scheme. As indicated by the earlier experimental results, the overhead incurred by Local Attestation is minimal. Therefore, within the maximum time interval without remote attestation, it is possible to increase the frequency of Local Attestation, thereby further reducing the time gap between two consecutive executions compared to other RA schemes. This can mitigate the risk of attackers exploiting the longer intervals between two remote attestation processes to compromise VNFs and engage in malicious activities based on timing differences.

Hence, the TVRAVNF scheme can enhance its overall security without introducing significant overhead by adjusting the frequency of Local Attestation. Furthermore, since the maximum time interval without remote attestation is dynamically adjustable, users can tailor this interval differently for various VNFs based on distinct security requirements. For VNFs with high-security demands, the maximum time interval can be shortened, while for those with lower requirements, it can be extended. Besides manual adjustments, the maximum time interval can also automatically adapt based on factors such as multiple remote attestation results within a time window. These aspects provide opportunities for further refinement in future work.

Security isolation

In the architecture of the TVRAVNF scheme, we can store the original hashes required for local attestation, as well as configuration files needed during VNF runtime, within the TEE. This approach ensures the correctness and authenticity of local attestation while safeguarding the undisturbed operation of VNFs. Taking examples like virtual routers and virtual firewalls, we can place files such as routing tables and firewall filtering rules in a TEE that is isolated from other areas. When VNFs need to access or modify these files, they can do so through TEE interfaces. In this scenario, even if an attacker possesses high privileges, they cannot alter these critical configuration files.

However, the TVRAVNF architecture does not guarantee complete immunity of VNFs from attacks, as VNFs do not run within the TEE. The decision not to run VNFs within the TEE is primarily due to the limited memory support typically offered by TEEs. If a VNF program requires a substantial amount of memory, TEEs might not be able to support it. Additionally, the associated overhead is a significant concern. Therefore, the TVRAVNF scheme primarily serves to verify the integrity of VNFs and protect specific data, acknowledging that complete isolation may not be achievable.

Conclusion

In this paper, we have successfully proposed and implemented an efficient, lightweight, and versatile virtual remote attestation scheme, TVRAVNF, designed to support NFV architectures. TVRAVNF reduces the overhead of performing virtual remote attestation on VNFs by introducing TEE and a local attestation mechanism, thereby enhancing the overall security performance of NFV architectures. The experimental results, obtained from measurements of overhead and verification performance, demonstrate the outstanding performance of the TVRAVNF scheme. The findings substantiate its low overhead and superior performance compared to conventional Remote Attestation approaches. The introduction of TVRAVNF addresses the critical challenge of efficiently and effectively verifying the security integrity of VNFs in NFV architectures. This significantly enhances the security and stability of scenarios based on NFV architectures, such as 5G networks, cloud computing and data centers.

In future work, we aim to deploy the TVRAVNF scheme in diverse real-world scenarios to validate its practical effectiveness. Based on the outcomes, we will continuously refine and optimize our scheme. Additionally, future research endeavors could explore ways to enhance the automatic dynamic adjustment mechanism for the maximum time interval without remote attestation. This exploration is essential for achieving a balanced point between security and efficiency within the overall scheme.