Using an Attention-Based Neural Network to Classify Imagined Lower-Limb Movements

The success of DL has greatly advanced the development of EEG decoding in MI-based BCIs [10,17]. Inspired by the FBCSP, Schirrmeister et al. [35] first proposed the following two well-known EEG decoding models: shallow ConvNet, which is specifically designed for MI-EEG signal classification; and deep ConvNet, which has a general-purpose architecture suitable for different feature types. Both models achieve competitive classification performance compared with the FBCSP algorithm on various datasets. Mane et al. [36] proposed an advanced MI decoding algorithm named the filter-bank convolutional network (FBCNet), which combines the neurophysiological priors of the FBCSP with CNNs. FBCNet provides multiview EEG representations of spatial features of different frequency bands and temporal features, resulting in remarkable classification performance across four public MI datasets. Lawhern et al. [37] developed a compact CNN (EEGNet) for EEG-based BCIs, which incorporated depthwise and separable convolutions into the network. EEGNet can learn frequency and spatial features intra- and interchannels, being a general EEG decoding model for different BCI paradigms, such as MI, the P300, error-related negativity responses, and movement-related cortical potentials (MRCPs). Ingolfsson et al. [38] proposed an EEG-TCNet model for MI classification. The model combined EEGNet with a temporal convolutional network (TCN) to supplement and utilize EEG features from the temporal domain. TCNet achieved outstanding accuracy while requiring few training parameters. Based on the TCNet, Altaheri et al. [6] designed an attention-based TCN (ATCNet), with multihead self-attention (MSA) embedded as intermediate layers in the TCNet. ATCNet obtained average classification accuracies (ACAs) of 85.38% and 70.97% for subject-dependent and subject-independent experiments, respectively, on the BCIC IV 2a dataset, representing one of the highest performances to date from comprehensive aspects. Liu et al. [5] proposed a CNN-based network with temporal and channel attention mechanisms for MI-EEG classification, demonstrating that the salience level features of EEG signals from time slices and electrode channels could be captured for further classification. Luo et al. [23] proposed a shallow mirror transformer (SMT) for MI-EEG classification; the framework was similar to Conformer [17] but had only one MSA layer. SMT obtained ACAs of 70.41% and 67.28% for the within-subject and cross-subject experiments, respectively, on the BCIC IV 2a dataset.

In general, incorporating an attention mechanism into a CNN has proven to be effective for MI-EEG classification by addressing ‘what’, ‘where, ‘when’, and ‘which’ aspects require attention, namely, the channel mechanism, spatial mechanism, temporal mechanism, and branch mechanism, respectively. Although current studies have yielded notable results in classifying MI-EEG signals, there is still room for improving the decoding performance of MI-based BCIs [6].

Recognizing lower-limb motion intention (LLMI) is highly important in BCI-based medical and nonmedical applications, such as stroke rehabilitation and motion auxiliary applications [1]. MI directly reflects LLMIs with spontaneous EEG generation. Some studies have focused on decoding upper-limb MI to map predefined LLMIs for BCI applications, such as using the decoding results of MI of left- or right-hand movement to indirectly represent LLMIs [39,40,41,42]. However, decoding lower-limb MI directly is more acceptable and user-friendly during LLMI recognition. Several attempts have been made to explicitly decode lower-limb MI. The commonly used lower-limb MI tasks are left- and right-leg movements, foot movements, stepping, walking, standing, sitting, and standing up [27,30,33,40,43,44,45,46,47,48,49]. Bulea et al. [43] proposed a Gaussian mixture model to classify three MI tasks—stand-up, sit-down, and quiet states—from EEG signals, achieving an ACA greater than 70.00%. Jeong et al. [46] proposed a subject-dependent and section-wise spectral filtering method to extract MRCP features, with a binary regularized LDA to classify walking intention and rest state, which obtained an ACA of 86.00%. Peng et al. [27] utilized a particle swarm optimization-based SVM to classify the MI of the leg lifting and rest state based on multidomain fusion features. Ma et al. [30] designed a compound-limbs EEG paradigm that integrated upper-limb swing with lower-limb stepping to decode stepping forward intention. CSP-based methods were adopted for feature extraction and three ML-based methods (SVM, K-nearest neighbor, and RF) were used for classification. The best ACA reached 73.70 ± 12.47% under MI conditions. Wei et al. [33] proposed three sub-band cascaded CSP algorithms to extract common EEG features across different subjects, and deep transfer learning-based AlexNet was used as the classifier, which realized the cross-subject MI classification of the following three MI tasks: left stepping, right stepping, and idle state. Zhang et al. [49] combined traditional ML with DL techniques to form a shallow and deep ensemble method for classifying lower-limb MI across the left leg, right leg, and rest. The shallow part included wavelet transformation and FBCSP, while the deep part included CNN, MSA, and TCN. The proposed method achieved an ACA of 60.27% for three classes and 64.20% for two classes (lower-limb MI vs. rest).

In summary, ML-based techniques, such as the CSP approach, which requires manual feature extraction, are predominantly utilized for lower-limb MI classification. In contrast, DL-based methods for lower-limb MI decoding are gradually being explored and are gaining increasing attention.

The concept of multidimensional attention has garnered significant interest across various domains, including EEG signal processing, machine life prediction, speech separation, human activity recognition, and status recognition. It allows models to capture and focus on relevant features from different perspectives, such as temporal, spatial, and channel dimensions. Several studies have explored the application of multidimensional attention mechanisms, leading to substantial improvements in performance across a range of tasks. Xu et al. [50] proposed an attention-based multiple dimensional EEG transformer for emotion recognition, which extracted 3D spectral-spatial-temporal features from EEG signals and achieved a competitive recognition performance. Similarly, Xie et al. [51] designed a CNN with a 3D attention mechanism to obtain synthetic degradation features in machine remaining useful life prediction. In the domain of speech separation, Li et al. [52] proposed a multidimensional attention fusion network, which extracted information of narrow and full bands along temporal, spatial, and channel dimensions and maximized the utilization of different attention features. Liang et al. [34] proposed a 3D weight attention module that considers both spatial and weight information and could be integrated with convolutional models to enhance model performance without increasing computational burden for human activity recognition. Liu et al. [53] proposed an improved 3D CNN with a separable structure and multidimensional attention for welding status recognition, and the multidimensional attention mechanism compensated for accuracy.

These studies demonstrate the broad applicability and effectiveness of multidimensional attention mechanisms across various fields. EEG signals have low SNR and exhibit high intersubject variability. This paper leverages multidimensional attention across temporal, spatial, and channel dimensions and proposes MACNet to address the inherent challenges of the noisy and complex lower-limb MI-EEG signals. Extracting meaningful features from EEG signals is crucial, as it improves classification accuracy and model generalization.

Learning global temporal dependencies from EEG signals significantly improves the decoding accuracy in MI classification [17]. Within this context, a self-attention mechanism within the temporal refining module is employed to capture long-term dependencies of the EEG signals of each channel in the temporal dimension. This mechanism allows the model to discern the most relevant temporal features while minimizing the influence of irrelevant or noisy information. The temporal refining module effectively filters out noise and enhances the quality of the raw EEG signals, thereby addressing the challenge of low SNR and improving the data quality. As introduced in [20], for the self-attention mechanism, the self-attention layer consists of three components, query (Q), key (K), and value (V), where Q and K are obtained as follows:(1)Q=WQX∈RC×T(2)K=WKX∈RC×T where X is the input sequence. The attention map can be calculated as follows:(3)Attention(Q,K,V)=Softmax(QKTd)V where d represents the length of keys and queries, and V is defined as the input sequence. This design allows the raw input information to be preserved and dynamically refined by attention weights. In this context, a single-head self-attention with a head dimension of 61 is employed to calculate the attention score. Then, the temporally refined EEG signals are computed as follows:(4)X∧=Attention(Q,K,V)

To enhance the representation ability and avoid losing information, residual connection is used, and the outputis added to the input sequence to obtain the final temporally refined EEG trials, which is given as follows: X ∧ X ~ (5) X ~ X ∧ = + X

EEG signals represent neural activity and are characterized by both temporal coherence and interchannel relationships, making it critical to capture information across these dimensions in EEG decoding. Inspired by [19,35], we designed an attention-enhanced convolutional module that integrates a temporal convolutional layer, the CBAM, and a spatial convolutional layer to address the challenge of decoding complex spatiotemporal dependencies of EEG signals.

The temporal convolution is operated over the time dimension via 40 filters with a kernel size of (1, 25) and a stride of (1, 1), allowing it to extract temporal features corresponding to frequencies above 10 Hz.

Following temporal feature extraction, CBAM is embedded by leveraging both channel and spatial attention mechanisms, since CBAM is lightweight and effective at strengthening the feature representation ability of the network. CBAM plays a vital role in handling the complex spatiotemporal dependencies of EEG data by adaptively enhancing the feature maps along the channel and spatial dimensions. Through its channel attention mechanism, CBAM adaptively emphasizes the most important feature maps by selectively highlighting relevant feature channels. Moreover, the spatial attention mechanism refines the spatial distribution of features within each feature map, capturing critical spatial patterns. A skip connection is included to ensure that important features from the temporal convolution are preserved and enhanced, further improving the feature extraction pipeline.

The CBAM is composed of channel and spatial attention mechanisms with sequential arrangements. The intermediate feature map is defined as; the attention process is summarized as follows:whereis the channel attention map,is the spatial attention map,denotes the refined feature map from channel attention, andis the final refined output feature map. Channel attention operates via average-pooling and max-pooling simultaneously to obtain two spatial context descriptorsand, and both are fed to a shared multilayer perceptron (MLP) network. The MLP consists of two fully connected layers, where the hidden layer reduces the channel dimension by a ratio of 16, followed by a ReLU activation, and the output layer restores the original channel dimension. The channel attentionis calculated as follows:whererepresents the sigmoid function andandare the weights of the MLP. F = ℝ C H W × × M c M S F ′ F ″ F avg C F max C M c σ W 0 W 1 (6) F ′ = ( ) ⊗ M c F F (7) F ″ F ′ F ′ = ( ) ⊗ M S (8) M c ( ) = ( ( ( ) ) + ( ( ) ) ) F σ M L P A v g P o o l F M L P M a x P o o l F = ( ( ( ) ) + ( ( ) ) ) σ W 1 W 0 W 1 W 0 F avg C F max C

Based on the channel-refined feature map, the spatial attention is employed to obtain deeper refined features by focusing on the significance of ‘where in the interspatial areas of the feature map. The average-pooling and max-pooling operations along the channel dimension of the feature map generate feature descriptorsand, which are subsequently concatenated and passed to a convolutional layer with a kernel size of 7 and padding of 3. Finally, the spatial attentionis computed as follows: F avg S F max S M s ( ) F (9) M s ( ) = ( ( [ ( ) ; ( ) ] ) ) F σ A v g P o o l F M a x P o o l F f 7 7 × = ( ( [ ; ] ) ) σ f 7 7 × F avg S F max S

Overall, CBAM helps to enhance the representation of critical EEG features by adaptively emphasizing informative features. Through channel and spatial attention mechanisms, CBAM adaptively refines the input feature maps by multiplying the attention and dynamically weighing them based on both channel and spatial importance.

Next, spatial convolution is executed with 40 filters of size (C, 1), where C represents electrode channels. Spatial features are extracted from the refined feature maps. Then, batch normalization is utilized to improve the training efficiency and model stability, and average-pooling along the temporal dimension and a dropout strategy are performed to prevent overfitting; the hyperparameters are detailed in Table 1. Finally, a safe log calculation is adopted to increase numerical stability. The final features obtained are the output of this module.

A classifying convolutional layer with four filters of size (1, 11) is used as the classifier, and the output M-dimensional vector is computed via the SoftMax function expressed as follows: (10)Pi=ezi∑j=1Mezj where M is the number of MI task categories and zi is the score of the i-th category. Subsequently, cross-entropy loss is applied as the loss function for model training, which is formulated as follows:(11)L=-1Nb∑i=1Nb∑c=1Mylog(y∧) where Nb denotes the sample number for each batch, and y and y∧ denote the real and predicted labels, respectively.

Totally, the preprocessed EEG data are fed to the model, and sequentially passed through the temporal refining module to obtain the refined EEG signals. The refined signals go through the attention-enhanced convolutional module to acquire refined EEG features, followed by the classifier module, which outputs the classification results.

The two most common metrics for classification accuracy (ACC) and the Kappa coefficient [1,59] are used to assess the classification performance. ACC is the ratio of the correctly predicted quantity to the total predicted quantity, offering a straightforward indicator of the model’s predictive performance. The Kappa coefficient is useful for addressing imbalanced data and random classification effects because it accounts for chance agreement, providing a more balanced measure of model performance. Kappa measures the ACC based on a confusion matrix (CM). The ACC and Kappa are defined as follows: (13)Accuracy(ACC)=TP+TNTP+TN+FP+FN where TP, FP, TN, and FN represent the number of correctly predicted positive samples, incorrectly predicted positive samples, correctly predicted negative samples, and incorrectly predicted negative samples, respectively. (14)Kappa=P0+Pe1−Pe where P_e denotes chance consistency, and P₀ is the accuracy.

Next, the CM provides detailed insight into the classification performance across different classes, and it reveals how well the model performs in predicting different classes. To statistically compare the proposed model with other comparison models, we employ the Wilcoxon signed rank test [60], a nonparametric test well-suited for small sample sizes and non-normally distributed data, to evaluate the statistical significance of performance differences across various methods. This ensures the robustness of the superiority of the proposed model. Specifically, a test is performed on the classification accuracies of MACNet and the baseline models for each subject. We defined a p value less than 0.05 as indicating a statistically significant difference.

To evaluate the generalizability of the proposed MACNet, we implement it using the benchmark public MI-EEG dataset of the BCIC IV 2a. The dataset contains nine subjects with four MI tasks, left hand, right hand, feet, and tongue. Two sessions (288 trials/session) of EEG data were acquired with a sampling rate of 250 Hz for each subject, and the electrode number is 22 channels. The first session is the training set, and the second session is the test set.

The proposed model is evaluated on the lower-limb MI dataset using a subject-dependent approach. The dataset includes four MI classes, with each class containing an equal number of trials per session to maintain a balanced class distribution. The MI-EEG data for each subject are merged and randomly divided into 10 equal groups for tenfold cross-validation. In each fold, 90% of the data are used for training and 10% for testing, and the model performance is averaged over all folds [1,61].

The proposed MACNet is compared with six advanced MI-EEG classification models: FBCSP [62], deep ConvNet [35], EEGNet [37], FBCNet [36], ATCNet [6], and Conformer [17]. We reproduced all the comparison models with hyperparameters in accordance with their original published articles. The data preprocessing, training, and evaluation steps follow the same procedure as that of the proposed model for fairness. Then, an ablation study on the lower-limb MI dataset is performed to evaluate the performance of each module in MACNet.

The MI-decoding models are built via Python 3.9.18 with the PyTorch 2.1.0 library. The models are trained and evaluated using an NVIDIA GeForce RTX 4090 GPU. For all the experiments, the Adam optimizer is used with a learning rate of 0.0001, and a learning rate scheduler based on the cosine annealing strategy is used to adjust the learning rate. The batch size is 64, the cross-entropy loss is set as the loss function, and an exponential moving average model is constructed to smooth the model parameters during training for robustness enhancement.

A subject-dependent experiment is conducted on the lower-limb MI dataset to validate the effectiveness and evaluate the performance of the proposed MACNet model. The performance of MACNet is compared with that of six advanced MI-EEG decoding models. For the comparison models, the FBCSP [62] uses handcrafted spatial EEG features and previously won the BCI Competition IV. Many SOTA DL-based models, including deep ConvNet [35], EEGNet [37], FBCNet [36], ATCNet [6], and Conformer [17], were inspired by the FBCSP. Deep ConvNet incorporates five convolutional layers and a Softmax layer, and it was specifically designed to classify oscillatory signals. EEGNet performs a two-step convolution from the temporal domain to the spatial domain to learn frequency and spatial features; then, separable convolution is employed to reduce parameter counts and decouple feature map relationships. EEGNet is robust in learning various features across different BCI paradigms. FBCNet extracts spectro-spatial features from multi-frequency-band signals, making it less susceptible to small training sets. ATCNet and Conformer are the existing advanced transformer models in EEG decoding. ATCNet recently showed excellent feature extraction ability and obtained the highest classification performance on BCIC IV 2a. Conformer effectively decoded oscillatory EEG signals in subject-specific experiments. These comparison models have achieved impressive performance on public EEG datasets and are considered the backbone in the BCI field.

Table 2 summarizes the classification performance of each subject and the total average results, comparing the proposed MACNet with other models on the lower-limb MI dataset with respect to accuracy and Kappa. MACNet achieves the highest classification accuracy and Kappa among all the models, with an ACA across all the subjects of 98.54% and an average Kappa of 0.98. This model yields significant improvements over FBSCP (p < 0.005), EEGNet (p < 0.005), FBCNet (p < 0.005), deep ConvNet (p < 0.005), ATCNet (p < 0.005), and Conformer (p < 0.005) by 43.79%, 41.87%, 41.64%, 2.99%, 8.85%, and 11.17%, respectively. Deep ConvNet yields better results than the other comparison models do. Regarding the other SOTA models, ATCNet, which incorporates TCN and attention mechanisms, achieves 89.69% ACA; Conformer, which combines convolutional layers with transformer blocks, achieves 87.38% ACA. Both achieve relatively high ACAs on ten subjects, as they both extract local EEG features and learn global correlations from the temporal domain, resulting in better classification performance than that of pure CNN-based models (EEGNet and FBCNet) and the traditional FBCSP model. The results of ATCNet and Conformer reveal that transformer-based models are effective in MI-EEG decoding. However, transformers need larger datasets to realize their full potential. Alternatively, the proposed MACNet extracts refined local and global EEG features from both the temporal and spatial domains by integrating multidimensional attention mechanisms with ConvNet, leading to superior results and significant upgrades in terms of accuracy and Kappa. The results obtained by ATCNet align with those reported in [63], where a transformer-based EEG decoding model achieved an ACA of 97.33 ± 1.86% in LLMI recognition. Additionally, statistical analysis via the Wilcoxon signed-rank test confirms that the differences between the proposed MACNet and all the comparison models are statistically significant (p < 0.05), further demonstrating the robustness and effectiveness of the proposed model.

An extensive experiment is carried out to validate the robustness of the proposed MACNet via the hold-out method. The data of one random fold are used as the test set, while the data from the other nine folds serve as the training set. Figure 6 illustrates the classification accuracy for each subject based on the hold-out method and tenfold cross-validation method. The classification accuracy obtained using the hold-out method is closer to the results obtained under the tenfold cross-validation method, and the variation trends of classification accuracy based on the two experiments are similar. The results demonstrate that MACNet is robust for lower-limb MI classification.

Ablation studies are performed on the lower-limb MI dataset to investigate the impact of different modules and the robustness of MACNet under varying channel configurations. MACNet is enhanced over ConvNet by the incorporation of temporal self-attention for reconstructing EEG signals and CBAM for refining EEG features across channel and spatial dimensions. The model branches are defined as follows: MACNet-Branch 0 serves as the baseline ConvNet, MACNet-Branch I consists of ConvNet and a temporal self-attention layer, and MACNet-Branch II is composed of ConvNet, a temporal self-attention layer, and CBAM. The ablation study results are presented in Figure 7. From Figure 7a, the classification accuracy gradually improves from MACNet-Branch 0 to MACNet-Branch II. The temporal self-attention mechanism helps in focusing on the most salient temporal characteristics and ignores less important details, and CBAM captures complex spatiotemporal dependencies in EEG data, demonstrating that both temporal self-attention and CBAM are successful in improving MI-EEG classification. Additionally, channel reduction analysis of Figure 7b demonstrates MACNet’s robustness with fewer channels; as the number of channels increases, the performance also improves.

These findings confirm the effectiveness of MACNet’s multidimensional attention mechanism in achieving robust feature extraction and classification across varying input and configuration conditions.

To show the interpretability of the MACNet model, visualization results of the CM, t-distributed stochastic neighbor embedding (t-SNE)-based method mapping of high-dimensional features, and EEG topography are illustrated.

The average CMs of MACNet and the reproduced models of subject 5 are shown in Figure 8. MACNet yields the best ACA in MI classification for all four MI classes, which is better than those of the other six models. Moreover, Figure 8g,h,i show the CM of MACNet and its branches 0 and I, respectively. The classification accuracy of each class is greater than that of the branch modules. The effectiveness of the proposed MACNet is further demonstrated.

In addition, t-SNE is applied to visualize the feature distribution in a two-dimensional embedding space. Embedding helps to evaluate the discriminative power of high-dimensional features extracted by the proposed MACNet. As depicted in Figure 9, the distribution of the raw data is mixed. Compared with those of the FBCSP, EEGNet, FBCNet, deep ConvNet, ATCNet, and Conformer models, the features of different categories of MACNet are more separable. The effects of deep ConvNet, ATCNet, and Conformer are the second greatest. The aliasing between categories is evident in FBCSP, EEGNet, and FBCNet, where the features of different categories are relatively close. Figure 9h shows that the intercategory distance increases and the intracategory distance decreases, suggesting that the attention mechanisms in MACNet enable the extraction of more discriminative features. For example, separation in MI tasks such as walking and standing is particularly noticeable, where the brain activity patterns are more complex and difficult to distinguish. This finding indicates that MACNet can learn the most discriminative EEG features in lower-limb MI tasks.

In previous studies [64,65], the imagination of standing was used as a baseline. Topographies of PSD differences (S-ST minus standing, ST-S minus standing, and walking minus standing) are displayed in Figure 10, depicting the PSD changes in brain areas through differences in characteristic activation values of each electrode channel across frequency bands ranging from 7 to 13 Hz (located in the mu band [8]). PSD is decreased in the mu band over the motor and sensory areas for the S-ST, ST-S, and walking MI tasks compared with the standing MI. These findings indicate that when the sensorimotor cortex of the brain is activated, event-related desynchronization (ERD) occurs in the mu rhythm. Which validates the neurophysiological relevance of the collected lower-limb MI-EEG data and aligns with established patterns in MI research.

For the performance on the widely used public dataset BCIC IV 2a, the classification accuracies of the proposed MACNet and comparison models are presented in Table 3. The accuracy achieved by MACNet is superior to that of FBCSP (p < 0.005), EEGNet (p < 0.05), FBCNet (p < 0.005), and deep ConvNet (p < 0.05), confirming its robust performance across different subjects. Although MACNet does not achieve the highest ACA, it is statistically comparable to the SOTA methods ATCNet and Conformer. The observed differences between these models’ performances are not statistically significant, as confirmed by the Wilcoxon signed-rank test (p > 0.05). This finding suggests that MACNet provides competitive performance on par with the best existing methods. Note that while ATCNet and Conformer require over 1000 training epochs to achieve optimum, MACNet achieves comparable accuracy with fewer than 500 epochs, demonstrating its efficiency in MI classification. This suggests MACNet is not data-dependent in MI-EEG decoding.

The results indicate that MACNet not only achieves statistically significant improvements in accuracy over most comparison models but also demonstrates comparable performance to SOTA methods, suggesting its robustness and generalizability in MI classification tasks.

This paper presents MACNet for the exploration of multiple lower-limb MI classifications. The attention mechanism is generally incorporated from multiple dimensions such as channel, temporal, spatial, branch, or hybrid dimensions to improve network performance. MACNet integrates multidimensional attention mechanisms with ConvNet, including a temporal refining module and an attention-enhanced convolutional module, to obtain highly significant features from MI-EEG signals, thus enhancing the representative ability of the network. ConvNet comprises temporal and spatial convolution layers, where temporal convolution with a small kernel size captures high-frequency features, and spatial convolution operates along the electrode channels to extract spatial features, with the kernels acting as spatial filters [8]. The temporal refining module plays a crucial role in reducing noise from the raw EEG signals, enhancing the model’s ability to handle the low SNR associated with lower-limb MI tasks. The attention-enhanced convolutional module incorporating CBAM refines the feature maps along both the channel and spatial dimensions. This mechanism helps the model dynamically focus on task-relevant features while ignoring less significant features. The CBAM combines channel attention, which emphasizes “what” is important in the feature map, and spatial attention, which highlights “where the critical features are located. By adaptively recalibrating the attention weights, MACNet demonstrates better discrimination capability across different lower-limb MI tasks in terms of feature distribution, as shown in Figure 9. This superior feature separation contributes to the model’s improved classification accuracy, particularly in MI tasks with higher complexity.

For the lower-limb MI task design, we choose four general LLMPs, namely, S-ST, ST-S, walking, and standing, for research on LLMI recognition. Most published studies on lower-limb MI decoding focus on classifying tasks, such as left-leg and right-leg movements, walking and relaxation, and walking and turning. Table 4 summarizes the comparison between the proposed model and those of previous works for LLMI recognition. The classification accuracies of different models are compared, and the proposed model offers advantages from a comprehensive perspective. Traditional ML techniques are commonly utilized to classify lower-limb MI tasks. This paper studies the multidimensional attention-based ConvNet model for decoding multiple lower-limb MI tasks and achieves remarkable results in subject-specific classification. We attribute this excellent result to two points: (1) the attention mechanism directs where to focus and improves the representation ability of interests, thus improving the classification performance; and (2) the baseline framework ConvNet performs well. Notably, we can see that deep ConvNet is also based on ConvNet and obtains outstanding classification accuracy, as listed in Table 2. When the ConvNet framework is separated into modules (ConvNet without temporal convolution, ConvNet without spatial convolution) to classify MI-EEG signals, the classification accuracy in Figure 11 shows that the overall ConvNet works better than when some modules are lacking, and removing the CNN module primarily affects the model’s ability to capture localized temporal and spatial patterns. In addition, the cross-validation approach may overrate accuracy among all EEG decoding models [66].

MACNet is validated via comparison experiments and ablation studies. The comparative results in Table 2 reveal that MACNet outperforms the SOTA models in terms of classification performance on the lower-limb MI dataset, such as EEGNet, FBCSP, FBCNet, deep ConvNet, ATCNet, and Conformer, across all subjects. Among these advanced comparison models, EEGNet is a general EEG decoding model that yields relatively low classification performance, possibly due to preprocessing issues. Deep ConvNet performs reasonably well, achieving high accuracy in some cases. However, its performance is inferior to that of MACNet; this can be explained by deep ConvNet’s lack of attention mechanisms, which limits its ability to selectively focus on the most informative channels and time periods. MACNet achieves superior classification performance compared with ATCNet and Conformer; this may be due to the action of multidimensional attention mechanisms in MACNet. The excellent classification performance can be attributed to MACNet’s advanced attention mechanisms, which adaptively focus on both spatial and temporal features, allowing it to handle the inherently noisy and complex EEG signals associated with lower-limb MI tasks. The confusion matrices (Figure 8) provide further insights into the classification performance of different models. Compared with the other models, MACNet results in significantly fewer misclassifications across all four lower-limb MI tasks (S-ST, ST-S, walking, and standing). For example, EEGNet and FBCSP exhibit greater confusion between tasks such as walking and standing, likely due to their inability to fully capture the subtle differences in brain activity associated with these similar tasks. In contrast, MACNet achieves extremely high classification in these tasks, which can be attributed to its attention-enhanced ability to distinguish task-relevant features, as well as the noise reduction provided by the temporal refining module.

The ablation study in Figure 7 clearly indicates that both temporal self-attention and CBAM contribute to the MACNet model. Specifically, MACNet-Branch II consistently outperforms MACNet-Branch I and MACNet-Branch 0 across all the subjects. This result highlights the roles of the temporal refining module and CBAM in capturing global patterns, complementing the local feature extraction capability of the ConvNet module. This enables MACNet to effectively handle the inherently noisy and complex spatiotemporal dependencies of EEG signals in lower-limb MI tasks.

Figure 10 shows that S-ST, ST-S, and walking imagery tasks activate the sensorimotor cortex of the brain and induce ERD in mu rhythm, demonstrating neurophysiological patterns characteristic in MI. This corresponds with previous psychological findings [2,72,73,74]. This consistency validates the scientific credibility of the experimental data and reinforces its relevance for studying MI classification. Furthermore, the subtle PSD changes underscore the inherent difficulty in distinguishing lower-limb MI tasks based solely on traditional features such as PSD. Classifying lower-limb MI is an intuitive and natural way to detect HMI [75]. This differentiates MI-based systems from other EEG paradigms like SSVEPs or event-related potentials (ERPs), which are driven by external stimuli. For instance, the CSP approach was utilized to enhance spatial filtering for MI classification, focusing on optimal feature selection [75]. However, CSP methods primarily address spatial domains, whereas MACNet’s multidimensional attention mechanism dynamically captures both spatial and temporal dependencies. For other EEG paradigms, ERP classification was studied by using a multiscale convolutional network (MOCNN) [76], and ERP signals are event-related potentials that are time-locked to specific sensory or cognitive stimuli. This structured nature allowed the MOCNN to capture local and global temporal features effectively using multiscale convolutional layers. Unlike these paradigms, MI lacks external event triggers, requiring more sophisticated methods to capture subtle spatial and temporal dependencies in the EEG data. Thus, the proposed MACNet extends the concept of multiscale convolution by incorporating attention mechanisms and a multidimensional feature extraction method to address the more diffuse and complex nature of MI signals. MACNet is specifically designed to address lower-limb MI classification, which shows promise for application in user-friendly BCI systems, such as lower-limb exoskeletons (LLEs), wheelchairs, and prostheses, where the real-time, accurate detection of MI is essential for the practical human–machine interaction.

In addition to classification accuracy, computational efficiency is crucial for real-time BCI systems. Thus, we evaluate the computational overhead and complexity of MACNet by comparing the number of parameters, floating point operations (FLOPs), and inference time with those of other SOTA models in the MI-EEG classification. The comparison results of different models with a data batch size of 64 are listed in Table 5, and the inference time is averaged over 100 runs based on Intel(R) Pentium(R) Gold G5420 and NVIDIA GeForce RTX 4090. As shown in Table 5, MACNet incorporates additional attention modules compared with shallow ConvNet, resulting in slightly higher parameter counts and a marginal increase in FLOPs. For the proposed MACNet, the inclusion of the temporal refining module and CBAM increases the computational cost because of the additional operations in the temporal, channel, and spatial dimensions. However, the corresponding increase in inference time is minimal, indicating that the attention mechanisms significantly enhance feature extraction without substantially compromising real-time performance. Compared with other SOTA models, MACNet maintains a competitive balance between model complexity, computational overhead, and real-time feasibility. Although some models, such as EEGNet and FBCNet, offer lower computational costs, they do so at the expense of classification accuracy. More complex models, like ATCNet and Conformer, offer high accuracy but also substantially increase higher computational costs. On the other hand, MACNet, by integrating multidimensional attention mechanisms, achieves higher accuracy with only a marginal increase in computational demand, making it highly suitable for real-time BCI applications. Overall, the trade-off between computational complexity and accuracy is carefully managed in MACNet, ensuring that it maintains high accuracy while being computationally feasible for practical real-time BCI applications [77].

Although the proposed MACNet achieves superior performance for LLMI classification, several limitations remain that can be improved in the future.

The relatively small size of the lower-limb MI-EEG dataset may affect the model’s generalizability. We employed a data augmentation technique, including a nonoverlapping sliding window approach, to increase the dataset size. This method segments continuous EEG signals into multiple smaller windows, significantly expanding the training data and reducing the risk of overfitting. However, we recognize that sliding window augmentation alone may not be sufficient to fully overcome the limitations posed by small datasets. Other data augmentation techniques, such as adversarial training and synthetic data generation, could be explored to further enhance model robustness. In our previous research, we experimented with generative adversarial networks (GANs) to generate synthetic EEG data [78], which yielded promising results in increasing data diversity and improving model performance on public EEG datasets. Nevertheless, the GAN-based approach is effective in augmenting data but introduces a significant computational overhead simultaneously. As a result, we choose the traditional sliding window technique for data augmentation in LLMI classification, considering the computational constraints. We also validated the proposed model on a public MI-EEG dataset and obtained competitive classification performance, further confirming its generalizability.

EEG signals exhibit considerable individual differences and high variability from session to session [8]. Intersubject variability remains challenging in EEG-based MI classification. Differences in brain anatomy, cortical activation patterns, and signal quality across participants affect model generalizability. Although MACNet’s multidimensional attention mechanisms have demonstrated adaptability to subject-specific features, it is limited in learning useful information from other subjects, further improvements are required to address intersubject variability. Future work could explore additional strategies, such as subject-independent training methods and transfer learning that enable the rapid adaptation to new subjects, to promote the universality of the model and reduce the calibration time in real-time BCI systems.

Although MACNet has been validated on benchmark datasets, cross-validation and experiments with larger and more diverse lower-limb MI-EEG datasets will also be considered to further validate the robustness and universality of this model. MACNet is an end-to-end framework that directly integrates feature extraction and classification into a unified pipeline. Although it currently relies on preprocessed EEG signals, future work will focus on integrating preprocessing steps, such as artifact removal and bandpass filtering, directly within the network to increase practicality and adaptability in real-world BCI applications. Additionally, exploring channel selection or reduction techniques is another avenue to prioritize the most informative EEG channels, thereby optimizing computational efficiency and facilitating deployment in resource-constrained BCI systems.