Attack based on data: a novel perspective to attack sensitive points directly

Abstract

Adversarial attack for time-series classification model is widely explored and many attack methods are proposed. But there is not a method of attack based on the data itself. In this paper, we innovatively proposed a black-box sparse attack method based on data location. Our method directly attack the sensitive points in the time-series data according to statistical features extract from the dataset. At first, we have validated the transferability of sensitive points among DNNs with different structures. Secondly, we use the statistical features extract from the dataset and the sensitive rate of each point as the training set to train the predictive model. Then, predicting the sensitive rate of test set by predictive model. Finally, perturbing according to the sensitive rate. The attack is limited by constraining the L0 norm to achieve one-point attack. We conduct experiments on several datasets to validate the effectiveness of this method.

Introduction

Time Series Classification (TSC) problems are encountered in various real world data mining tasks such as security (Tan et al. 2017; Tobiyama et al. 2016), transportation (Nguyen et al. 2018), health care (Abdelfattah et al. 2018; Ma et al. 2018; Fawaz et al. 2018), etc. Under the background of big data, deep learning technology has developed rapidly and has been widely used in various fields such as computer vision and autonomous vehicles. In recent years, researchers have begun to apply deep learning techniques to TSC problems (Ismail Fawaz et al. 2019; Fawaz et al. 2019).

However, due to the wide applicability of DNNs, they pose a threat that cannot be ignored, so researchers began to study the vulnerability of DNNs. Adversarial attacks in the field of TSC are capable of leading neural networks to misclassify adversarial time series data while remaining difficult for humans to detect (Moosavi-Dezfooli et al. 2016; Papernot et al. 2016a). There are several types of attack methods that have been developed to perturb the input data in order to mislead the model’s output, such as Gradient-based attacks (Goodfellow et al. 2014; Szegedy et al. 2014), Optimization-based attacks (Carlini and Wagner 2017), Transferability attacks (Papernot et al. 2017), Decision-Based attacks (Brendel et al. 2017), etc. In recent years, thanks to the increasingly in-depth research conducted by scholars, more and more effective adversarial attack methods and theories have been proposed.

Fig. 1

Our method is based on data, by extracting the statistical features of the data to find the points which are suitable for the attack. Unlike genetic algorithms which require cyclic query, our method only requires a few times of visiting the target model or even a single visit

Fig. 2

One-pixel attacks on TSC successfully fooled ResNet architecture

Among the many attack methods, one-pixel attack has received widespread attention as an extreme attack method. One-pixel attack was presented by Su et al. (2019) as a special black-box sparse attack, due to its particular attack pattern, which only perturbs a single pixel of an image to mislead the classifier, different from other attacks that may change hundreds of pixels (Wang et al. 2020). Notwithstanding its efficacy with small datasets, the one-pixel attack method encounters difficulties in larger datasets due to the significant computational demands imposed by the requirement for thousands of iterations, thus constraining its practical application.

Inspired by the attack pattern of the one-pixel attack, we propose a black-box sparse attack method based on data which can find the sensitive locations directly.

Compared with the one-pixel attack proposed by Su et al. (2019), the adversarial attack method proposed in this paper is not based on query but based on statistical features of the datasets, which can substantially decrease the number of queries and attack the sensitive points directly. We believe that our approach presents a novel perspective to the field of adversarial attacks. The difference between the method presented in this article and the traditional methods are illustrated in Fig. 1.

In brief, the main contributions of this paper are summarized as follows.

• This article proposes a black-box attack method based on data and directly attack sensitive points, similar to the one-pixel attack, which only needs to perturb a very small number of time points, or even only one point.

• We have conducted experiments under various parameters of the method proposed in this paper. The experiments prove that adversarial attacks can be based not only on classification models but also on the statistical features of the dataset.

Related works

The security problem of DNN has become a critical topic since Szegedy et al. (2014) proposed adversarial examples for image recognition. Many researchers proposed attack methods from different perspectives. In the field of black-box attack, the methods are mainly based on substitute models, genetic algorithms, and zeroth-order optimization. For instance, Papernot et al. (2016b) proposed that utilizes the transferability property of the substitute model to approximate the target model. Wei et al. (2021) manipulated the image attributes to perform the black-box attacks with a genetic algorithm. Szegedy et al. (2014) revealed the sensitivity to well-tuned artificial perturbation which can be crafted by several gradient-based algorithms using back-propagation for obtaining gradient information. Specifically, the gradient-based method was proposed by Goodfellow et al. (2015), the attackers perturb the image in the direction of the gradient of the loss function with respect to the input image thus reducing the classification accuracy. Lin et al. (2020) propose a black-box technique: Black-box Momentum Iterative Fast Gradient Sign Method (BMI-FGSM) to test the robustness of DNN models.

Recently, new attack methods are proposed. Hu et al. (2022) propose Adversarial Texture (AdvTexture) which hides from person detectors by covering clothes. Yang et al. (2022) propose a gradient-free black-box method called TSadv to attack DNNs with local perturbations. He et al. (2022) introduce a generator architecture to alleviate the overfitting issue. Kahla et al. (2022) proposed Boundary-Repelling Model Inversion (BREP-MI) to invert private training data using only the target model’s predicted labels. Luo et al. (2022) proposed a novel algorithm that attacks semantic similarity on feature representations.

one-pixel attack Su et al. (2019) showed how neural networks can be fooled by altering the value of just a single input pixel. As shown in Fig. 2, unlike most attack methods,one-pixel attack only needs to perturb one point. By constraining the $\textbf{L0}$ norm, they enforced a limit on the number of pixels that were allowed to be perturbed. One-pixel attack using differential evolution (Storn and Price 1997) which is an optimization algorithm that can be used to search for the minimum or maximum of a function. The researchers encode the perturbation into an array (candidate solution), each candidate solution contains a fixed number of perturbations and each perturbation is a tuple holding five elements: x-y coordinates and RGB value of the perturbation.

$$\begin{aligned} \begin{aligned} x_{i}(g+1)=x_{r1}(g)+&F(x_{r2}(g)-x_{r3}(g)),\\ r1\ne r&2\ne r3, \end{aligned} \end{aligned}$$

(1)

Once generated, each candidate solution competes with their parents according to the index of the population and the winner survive for next iteration.

In summary, each black-box attack method has its own strengths and weaknesses, and the choice of attack method depends on the specific target model and the attacker’s resources and goals.

Among all the methods mentioned above, none of them is data-based, while our approach is a huge innovation in the field of adversarial attacks.

Methodology

In this section, initially, we provide a description of the problem. Subsequently, we introduce the dataset and the main diagram illustration of the experiment. Finally, we describe the attack methodology in detail.

Fig. 3

In Part A, in the section of the perturbation parameters transformability experiment, the perturbation parameters generated by the ResNet architecture are input into the Inception and Encoder architectures. To be specific, we have done the test for each pair of architectures, the result show as Fig. 6. In the step of training predictive models, global statistical features and classified statistical features are used to train predictive models respectively. In Part B, sensitive curve is used as a visual representation of sensitive rate

Problem description

We assume that the attacker has access to the target model in a black-box setting. The information, such as architecture, parameters is not accessible. Then, we assume part of the dataset (including both time series and labels) is public. Let f represent the target model.

For a time series it can be represented as ${\textbf{x}} = (x_1,\ldots , x_n)$, each element is a numeric value at the corresponding time step. If the original time series ${\textbf{x}}$ belongs to class ori, the output probability of f is, therefore, ${f}_{ori}(x)$. While the perturbation can be represented as $\delta (x)$. The magnitude of perturbation $\epsilon$ is controlled via a factor $\beta$. $\epsilon$ is defined as

$$\begin{aligned} \epsilon =\beta \cdot \left( \dfrac{1}{T}\sum \limits _{i=1}^T(x_i^{max}-x_i^{min})\right) , \end{aligned}$$

(2)

where ${x}_{i}^{max}$ and ${x}_{i}^{min}$ separately represent the maximum and minimum value of a time series of the total T items.

Our goal is to find the solution $\delta (x)$ for the following problem:

$$\begin{aligned} \begin{aligned} {f}_{ori}({\textbf{x}}+\delta ({\textbf{x}})) \quad <\quad {f}_{other}({\textbf{x}}+\mathbf {\delta (x)}) \\ \end{aligned} \end{aligned}$$

(3)

Subject to $\delta$(${\textbf{x}}$) is limited in the shapelet interval l

$$\begin{aligned} \begin{aligned} \Vert \delta ({\textbf{x}})&\Vert _0 = 1\\ \Vert \delta ({\textbf{x}})&\Vert _{\infty }\le \epsilon , \end{aligned} \end{aligned}$$

(4)

where $\Vert \delta ({\textbf{x}})\Vert _0$ and $\Vert \delta ({\textbf{x}})\Vert _{\infty }$ represent the $L_0$ and $L_{\infty }$ norms of $\delta ({{\textbf {x}}})$, respectively. That means we want to add perturbations to minimize the confidence score of class ori.

In order to visit the target model as little as possible, our method uses predictive model to predict the coordinates of sensitive locations in the test set. By using predictive model we can attack the sensitive locations directly.

The main diagram illustration

Before the formal experiment begins, we take the result of one-pixel attack based on differential evolution as the baseline for comparison.

The whole experimental process is shown in Fig. 3. Since it’s not available for the architecture of the target model under the black-box attack state, we have to use the public part of the datasets to prepare the models with multiple architectures as substitute models.

To test the transformability of perturbation parameters between different CNNs, we designed the perturbation parameters transformability experiment.

There are six architecture using in this experiment, respectively MLP,CNN,ResNet,Inception,Encoder and FCN. The code for the above architectures is referenced in the Ismail Fawaz et al. (2019)’s open source project.

We obtained the sensitive rate of the typical datasets through the global search algorithm used on multiple substitute models.

Then, we extract the statistical feature of the public part of the typical datasets.

In the step of training predictive model, the extracted statistical feature and sensitive rate are used to train predictive models.

As shown in Fig. 4, after obtaining the predictive models, the sensitive rate of the test set are obtained by predictive models. Then, complete one-point attack by perturbing the sensitive points.

Table 1 Typical datasets from UCR time series classification archive (POC:PhalangesOutlinesCorrect)

Data preparation and predictive models training

This section introduce the datasets mentioned in this paper and how to extract and use the training set to train the prediction model.

In order to fully demonstrate the effect of the experiment, this experiment selected several typical datasets from the number of classes, length, size of datasets and other aspects as demonstration examples(as shown in Table 1). These datasets are from the UCR/UEA archive (Chen et al. 2015).

There are two data necessary for the training of predictive models, one is the sensitive rate of each point, and the other is the statistical feature. We will introduce them in turn.

The method of one-point attack is to only perturb one point in a time series resulting misclassification of some well-performing models. The perturbation parameters are consist of the parameter records the coordinates of points and the perturbation that can lead to misclassification.

Fig. 4

Take the $\textbf{35}$ th time point of ECG200 dataset as an example. The values at the 35th time point of each time series are spliced into a one-dimensional sequence, and the average value, standard deviation, number of peaks and valleys, skewness and kurtosis of the sequence are obtained through feature extraction. In the subsequent process, the sensitive rate of the $\textbf{35}$ th time point will be predicted through these features

Due to the limitation of black-box state, the attacker is unaware of the model’s architecture. Therefore, it is necessary to consider the impact of the substitute models’ architectures on the results before the training of substitute model. To address this, we design an experiment to test the transformability of perturbation parameters. That is, testing the attack success rate (ASR) of perturbation parameters which extract from the model A on other models. Specifically, the ASR of perturbation parameters extract from the model A on model B can be represented as

$$\begin{aligned} \begin{aligned} \text {ASR}_\text {B}^\text {A}=\dfrac{\sum _{i=0}^n\bigg \lceil f_{\max ^*\ne o ri}^B({\textbf{x}}+\delta _A({\textbf{x}}))\bigg \rceil }{n} \end{aligned} \end{aligned}$$

(5)

where $f^{B}_{max*\ne ori}$ represents the highest probability among all classes output by model B when the time series not belong original class. Otherwise, it output is 0. $\delta _{A}\mathbf ({\textbf{x}})$ represents the perturbation parameters generated by model A.

Sensitive rate

To quantitatively evaluate the vulnerability of a point, we define a degree of difficulty of causing the classifier to produce wrong results after a point is attacked as sensitive rate. The points with high sensitive rate are called sensitive points which means it’s sensitive to attack.

The sensitive rate of a point is related to the value of dividing the minimum perturbation that causes models to misjudge ($|\delta (\mathbf {x_i})_{j}^v\big |_{\text {min}}^f$) by the sum of the perturbation applied at that point (${\sum _{i=\text {min}}^{\text {max}}\delta (\mathbf {x_i})_{j}^v}$), as shown in equation 6. The higher the sensitive rate of a point, the easier it is to be attacked, and vice versa.

$$\begin{aligned} \text {Sensitive Rate}_j=\sum _{i=0}^n\bigg (1-\frac{\big |\delta (\mathbf {x_i})_{j}^v \big |_{\text {min}}^f}{\sum _{i=\text {min}}^{\text {max}}\delta (\mathbf {x_i})_{j}^v}\bigg )\, \end{aligned}$$

(6)

Fig. 5

The figure of using global search algorithm on Adiac dataset ($exp=1.5$, $eps=1$). Only when the perturbation is greater than or equal to $|\delta (\mathbf {x_i})_{j}^v\big |_{\text {min}}^f$ can the model misjudge, the minimum perturbation above is a $\delta ({\textbf{x}})_2$ or $\delta ({\textbf{x}})_3$

Perturbation parameters are obtained by global search algorithm which continuously perturbs the value of time series data and record the perturbation parameters when perturbing successfully, as shown in Fig. 5. For a time series ${\textbf{x}}$, its solution space of global search algorithm can be represented as

$$\begin{aligned} \begin{aligned} \text {(coordinate,value)} \in \{(x&,y) \mid x \in [0,n], y \in [{x}^{min}_{exp},{x}^{max}_{exp}]\},\\ {x}^{min}_{exp} = ({x}^{min} -&{x}^{mid}) \times exp + {x}^{mid},\\ {x}^{max}_{exp} = ({x}^{max} -&{x}^{mid}) \times exp + {x}^{mid} \end{aligned} \end{aligned}$$

(7)

For the purpose of obtaining more perturbation parameters, we expand the solution space of the global search algorithm in terms of parameter setting, which can improve the generalization ability of the attack. Specifically, we increase the parameter $\textbf{exp}$ from ${\textbf{1}}$ to $\mathbf {1.5}$.

In our method, there are two statistical features used to train the predictive model. One is global statistical feature, and the other is classified statistical feature.

Global statistical feature

For a coordinate, the values of all time series data in this dataset at this coordinate are spliced into a row of sequences. Calculate the average value, standard deviation, number of peaks and valleys, skewness and kurtosis of this group of sequences, and splice these five values as the statistical feature of this point. This statistical feature is called the global statistical feature of this dataset. By this method, we can get the global statistical features of each point of this dataset.

$$\begin{aligned} \begin{aligned} \begin{aligned} featur&e_{i}^{global} = [\phi (E_i(X))], \\ \phi (x) = [mean(x),&std(x),wave(x),skew(x),kurt(x)], \\ E_i(X)&= [X_{0,i},X_{1,i},\ldots ,X_{n,i}] \end{aligned} \end{aligned} \end{aligned}$$

(8)

where $feature_{i}^{global}$ represents the statistical feature of ${\textbf{i}}$ th point, mean is the average value of the input sequence, std is the standard deviation of the input sequence, wave is the number of peaks and valleys of the input sequence, skew is the skewness of the input sequence, kurt is the kurtosis of the input sequence, $E_i(X)$ is a sequence consisting of the values of the ${\textbf{i}}$ th point of each time series.

Classified statistical feature

Separate the time series in the dataset by class, calculate the statistical feature sequence of a point in the time series of each class according to the method of calculating the global statistical feature, then splice the statistical feature sequence of each class to form a classified statistical feature matrix with width ${\textbf{C}} \times {\textbf{5}}$, height ${\textbf{n}}$, where ${\textbf{C}}$ is the number of classes and n is the number of time series.

$$\begin{aligned} \begin{aligned} \begin{aligned} feature_{i}^{classes} = [\phi (E_i(X^0)),\ldots ,\phi (E_i(&X^m))], \\ \phi (x) = [mean(x),std(x),wave(x),skew(x)&,kurt(x)],\\f(X^t) = t, E_i(X^t) = [X^t_{0,i},X^t_{1,i},\ldots ,X^t_{n,i}],&t\in [0,m] \end{aligned} \end{aligned} \end{aligned}$$

(9)

$feature_{i}^{classes}$ represents the classified statistical feature of the ${\textbf{i}}$ th point, f is the classifier, t is the category of time series, m is the number of categories, $X^t$ is the time series of the t category.

One-point attack with predict model

Input the features extracted from the testing set into the trained predictive model to predict the sensitive rate of each point in the testing set. One-point attack can be completed by perturbing sensitive points with high sensitive rate.

In detail, the predicted results of the predictive model are recorded in a array of $\mathbf {key-value}$ $\textbf{pairs}$ named $\textbf{SensitivePoints}$. Each key-value pair consists of a index and a sensitive rate. The key-value pairs are arranged from largest to smallest according to the sensitive rate.

According to the sequence of key-value pairs, all time series in the testing set are modified at a single point according to the current key-value pair. If the point where the index is located is on the wave crest, subtract a perturbation. If the point where the index is located is in wave valley, add a perturbation. The reason for this is in accordance with the Frequency Principle (F-Principle).

The Frequency Principle (F-Principle) was proposed by Xu et al. (2019), which reveals the information that deep neural networks (DNNs) attend to during the classification process. The F-Principle suggests that DNNs often fit target functions from low to high frequencies.

Owing to trained DNNs focus on the high frequency information of data, in order to mislead DNNs, we try to modify the frequency information of the original time series data by perturbing the wave crests and wave valleys.

The value of perturbation is positive, which depends on the span of the dataset, and is generally 30% of the difference between the maximum value of the original dataset and the minimum value.

Due to the key-value pairs are sorted according to the sensitive rate, the higher the key-value pair is, the more vulnerable the point is, and the lower the key-value pair is, the more robust the point pair is.

In order to ensure the efficiency of one-point attack, the experiment in this article only uses the first five key-value pairs for one-point attack.

Experiment results and analyse

Fig. 6

The result of perturbation parameters transformability experiment. Specifically, the second line represents the ASR that the perturbation parameters generated by the FCN architecture used to attack the ResNet, MLP, Encoder, Inception, CNN architectures. In particular, the ASR of attacking the ResNet architecture using the perturbation parameters generated by other architectures is very low

Fig. 7

Aligned figure of class activation diagram and sensitive curve. The coordinates with high sensitive rate in the time series data are usually located at wave crests and wave valleys

Fig. 8

CAM of a class of Adiac dataset on each architecture. Only the highlighted positions of the ResNet architecture are very concentrated and distributed in a point shape, while the highlighted positions of the activation diagrams of other architectures are distributed in a strip shape. (Take Adiac dataset as example)

Fig. 9

Sensitive curves of several datasets were plotted, which showed that the vulnerability levels of these datasets varied at different time points. Attacking at locations with high sensitive rates can result in higher attack success rates

Fig. 10

Accuracy of each architecture under different factors ($\beta$) for perturbing. The accuracy of architectures decreases as the $\beta$ increases. (The dashed line represents the accuracy of the architectures after being attacked by one-point attack based on evolutionary algorithm ($\beta$ $=$ 1))

Perturbation parameters transformbility experiment

The result of perturbation parameters transformability experiment is shown in Fig. 6, where the y-axis represents the architectures that generating perturbation parameters, and the x-axis represents the architectures for testing. The values of the heatmap represent the ASR of the perturbation parameters generated by the y-axis architectures attack on the x-axis architectures.

From Fig. 6, in most cases, the perturbation parameters generated by different architectures is transformability. However, the ASR of attacking the ResNet architecture using perturbation parameters generated by other architectures are very low, which shows that only a small part of the perturbation parameters generated by the other five architectures can make the ResNet architecture misclassification. In contrast, the ASR in the first row of the heatmap are generally high, which indicates that most perturbation parameters generated by the ResNet architecture can lead to the misclassification of the other architectures. This shows from the side that the robustness of ResNet model is significantly higher than the other five models.

The analyse of data preparation

Figure 7 shows the aligned figures of class activation diagram and sensitive curve. The Fig. 8 shows the class activation diagram of each architecture in a certain class on the Adiac dataset. The Fig. 9 shows the sensitive curve of some datasets (both figures are obtained through ResNet architecture). For CAM, the highlighted part of the class activation diagram is the vulnerable part. The data plotted in Fig. 7,8,9 is generated from perturbed data produced by the global search algorithm.

Through the analysis of Fig. 8, we can further understand the reason of the special phenomenon in Fig. 6.

Only the highlighted positions of the ResNet architecture are very concentrated and distributed in a point shape, while the highlighted positions of the activation diagrams of other architectures are distributed in a strip shape. This shows that to increase the ASR of the ResNet architecture, it is necessary to focus on attacking its highlighted location. Because the highlighted area of ResNet is very small and concentrated, which leads to better robustness of ResNet compared with the other five architectures. Besides, the global search algorithm proves that the number of successful attacks on ResNet architecture is the lowest among all architectures.

Figure 7 shows the coordinates with high sensitive rate in the time series data are usually located at the peaks or troughs. If at a certain location in a dataset, the more common wave crests and wave valleys appear, the more sensitive to perturbation. This phenomenon can be understood as if the wave form of this position vulnerable they are similar in most time series, then a perturbation is added to this position, which results in the value form of this position exceeding the value range under normal conditions and leads to misjudgment of classifier. What’s more, it can also be explained by the theory of F-Principle. For DNNS, as iteration steps of training increase, the high-frequencies are fitted, the locations of the wave crests and wave valleys are usually the location with high-frequency information.

Table 2 shows the proportion of the perturbation parameters whose values are within the range of the original time series data values in the perturbation parameters obtained using the global search algorithm $(exp=1.5)$. The table shows that the perturb in the original value range only accounts for a small part of the total perturbation parameters, in the field of adversarial attack, properly expanding the value range of perturbation can effectively improve the ASR.

Table 2 The proportion (%) of the perturbation parameters whose values are within the range of the original time series data values in the perturbation parameters obtained using the global search algorithm $(exp=1.5)$

The result of one-point attack

Figure 10 shows that the accuracy of each architecture under different perturbation magnitudes. It can be observed from the figure that the accuracy of architectures decreases as the magnitude of the perturbation increases. The accuracy of the CNNs under the one-point attack based data is basically lower than that of the one-point attack with DE (The perturbation amplitude of one-point attack based on evolutionary algorithm is 1).

Table 3 shows that the size of the dataset and the quantity of perturbed points that impact the effect of the attack. For instance, when the parameter proportion is set to 0.3, then there are 30% samples of the original dataset used to obtain key-value pairs. When the parameter proportion is fixed, the accuracy decreases as the number of perturbation points increases. It is intuitive that when the number of perturbation points increases the success rate of the attack should rightfully increase, which also means that the accuracy of the model on the counter sample decreases. When the number of perturbed points is fixed, the accuracy decreases as parameter proportion decreases. This phenomenon illustrate that the statistical features obatined from the larger size of dataset have more information of the original dataset which is more beneficial for the attacker.

Table 3 This table illustrates the accuracy of deep learning models under varying sizes of datasets and quantities of perturbed points

Discussion and conclusion

We propose a new black-box attack based on data and attack according to the high-frequency component of time series.

Compared to the evolutionary algorithm-based attack method, we are able to generalise to a wider range of application scenarios. Our approach has a higher ASR under the same perturbation magnitude constraints. The method of one-point attack can be used in extreme environments with minimal modification of the original data. This attack has the following features compared to other black-box attacks:

• Based on data.

• Fewer submissions.

• Attack the sensitive points directly

However, our approach does not achieve good performance on some datasets, such as ECG200 and DistalPhalanxTW, which may be because my method only focuses on statistical features, while the information related to sensitivity in these datasets is also contained in the frequency domain and time domain. In future experiments, we plan to increase the number and types of features. We believe that increasing the number and types offeatures can improve the accuracy of predictions and enable the development of more data-based adversarial attack methods.

Moreover, with the progress of explainable artificial intelligence, we hope we can rely on the work of previous researchers, proposing more theoretical work content. Additionally, it would be interesting to investigate the use of more sophisticated substitute models, such as generative models or deep neural networks, to improve the transferability of the attack across different target models.

In addition to theoretical research, we also hope to find practical applications for this achievement in the real world. The attack could be extended to consider other types of time series data beyond those studied in this work, such as images, audio or video. Overall, we believe that our work lays the foundation for further research into the data based black-box attack method.