What this is
- CRISPR Cas-9 is a powerful genome-editing tool, but it can cause unintended modifications known as .
- Accurate prediction of these is crucial for safe applications in biotechnology and medicine.
- This research presents CRISPR-DIPOFF, a deep learning model that improves Off-Target prediction while providing interpretability.
- The model balances precision and recall, addressing limitations of previous approaches.
Essence
- CRISPR-DIPOFF enhances Off-Target prediction accuracy using recurrent neural networks while offering insights into model decision-making, particularly regarding seed region mismatches.
Key takeaways
- CRISPR-DIPOFF demonstrates improved Off-Target prediction performance compared to existing models, addressing the .
- The model reveals important biological insights, particularly the significance of two sub-regions in the seed region of the single guide RNA.
- Hyperparameter tuning via genetic algorithms significantly enhances model performance, emphasizing the importance of optimization in deep learning.
Caveats
- The study focuses solely on substitution-type mismatches, excluding insertion and deletion types, which limits its applicability.
- Only one dataset was used for training and testing, which may affect the generalizability of the findings.
- The performance of the ELECTRA model was below expectations, indicating potential limitations in its pretraining approach.
Definitions
- Off-Target effects: Unintended modifications to DNA caused by the CRISPR Cas-9 system, potentially leading to adverse outcomes.
- Precision-recall trade-off: The balance between the precision of a model (correct positive predictions) and its recall (ability to identify all relevant instances).
Simplified
INTRODUCTION
CRISPR Cas-9 stands for Clustered Regularly Interspaced Short Palindromic Repeats and CRISPR-associated enzyme Cas-9. It utilizes a bacterial defense mechanism to precisely cut DNA sequences, which has been repurposed by scientists to cause a double-strand break at any portion of the DNA by providing the Cas-9 enzyme with the exact sequence of that DNA portion [1]. After the break, DNA’s natural healing mechanism is triggered, and due to mutation, the repair region’s biological function gets disrupted. Thus, the CRISPR Cas-9 system can be exploited to knock off specific genes in an organism’s genome. This repair mechanism can be controlled by providing a template to replace the existing DNA sequence [2, 3], and thus the gene gets edited.
The applications of CRISPR are beyond imagination. CRISPR can be used to map the functionality of genes in different organisms. It can revolutionize the agriculture sector by designing disease and weed-resistant high-yielding crops [4, 5]. It can also be used for curing genetic diseases [5]. However, like any other groundbreaking advancement, CRISPR comes with its own set of challenges, particularly in the form of Off-Target effects where the single guide RNA (sgRNA) causes a double-strand break of the DNA even with the presence of a few nucleotide mismatches. Identifying these Off-Targets through traditional laboratory validation methods can be costly and time-consuming. This presents a valuable opportunity for computational methods to offer a rapid, reliable and cost-effective solution to detect and predict Off-Target effects in genome-editing.
The detection of potential Off-Target effects is very crucial for the successful application of the CRISPR Cas-9 system. In recent years, computational approaches, especially machine learning-based ones, have contributed to a great extent in fields like genomic sequence analysis, drug discovery, motif finding, etc. [6]. Following the same trend, computational approaches have also been used to predict the Off-Target effects [7]. The early detection methods relied on score-based approaches [8–11] primarily focusing on identifying Off-Target effects based on the presence and positions of mismatches between the target site and the guide RNA assigning scores based on the mismatches. These methods do not consider the complex relationship among the sequences, and they may suffer from experimental variations and limitations [12].
Traditional machine learning approaches for Off-Target prediction typically involve the use of manually engineered features. These features can include sequence-based characteristics, such as mismatch patterns or nucleotide composition, and contextual information surrounding the target site. Abadi et al. [13] aggregated datasets from three different sources: GUIDE-Seq [14], HTGTS [15] and BLESS [16], and trained a random forest regression model. Peng et al. [17] trained several support vector machine (SVM) models and ensembled them where the number of positive samples and negative samples were maintained the same. Chen et al. [18] performed their experiments on their CRISPEY dataset and applied Synthetic Minority Over-Sampling Technique [19] to generate synthetic positive samples and trained Logistic Regression, SVM, Random Forest and Neural Network models. Zhang et al. [20] used the scores of score-based methods along with biological properties as features for training an Adaboost [21] model.
Over time, the dataset for Off-Target prediction has grown significantly, and recently most of the works have focused on deep learning for predicting Off-Target sites as traditional machine learning models do not scale well for large datasets [22]. Lin et al. [12] presented a convolutional neural network (CNN) based model CNN_Std trained on CRISPOR [23] dataset. They encoded the sgRNA and targeted DNA sequences with the one hot encoding method and superposed (logically ORed) those encodings. A similar encoding approach was followed by other studies, too [12, 24]. Chuai et al. [24] produced their own dataset (DeepCRISPR), which has been used by most of the later works as a benchmark dataset. The authors pretrained a Deep Convolutionary Denoising Neural Network based autoencoder using the whole human genome sequence containing almost 0.68 billion unlabeled samples. They used epigenetic features and sequence data to train their CNN model.
Natural language processing (NLP) based techniques have also been employed for Off-Target prediction. For example, Liu et al. [25] applied transformers [26] for Off-Target prediction in their proposed tool AttnToMismtach_CNN. Recently, Guan et al. [27] have proposed a transformer-based model with a new encoding scheme and they have also designed loss functions to tackle the noise issue in the Off-Target datasets. Chen et al. [28] pretrained a BERT [29] model specifically targeting the Off-Target prediction task and used the BERT model’s embedding as features and along with other engineered features for training a LightGBM model [30]. Liu et al. [31] proposed CnnCrispr, where they introduced GloVe (Global vectors for word representation) embedding [32] for the input sequences. Yan et al. [33] benchmarked various datasets and Off-Target prediction tools and found that structural and energy-based approaches tend to perform better than learning-based methods. However, this finding might just be outdated with the proliferation of new and improved deep-learning-based algorithms [27, 31, 34–36], since this study (more on this in the Discussions Section).
The studies mentioned so far consider only substitution-type mismatches, which is the common cause of Off-Target and the focus of this study. However, a few studies have also considered insertion and deletion-based mismatches between sgRNA and target DNA [34–36]. Here, the work of Störtz et al. [35], which we came across during the manuscript writing phase of our research work, deserves some attention. Their model, namely, piCRISPR, utilized the context regions near the target DNA for feature engineering and used a larger dataset for training. However, piCRISPR is incomparable with our work as we only focus on substitution-type mismatches, as mentioned earlier.
Though the application of deep learning has improved the accuracy of Off-Target prediction, the datasets of Off-Target prediction are highly imbalanced, and thus most of the models suffer from precision-recall trade-offs. Another shortcoming of the existing studies is the lack of focus on interpreting the models. A few traditional machine learning models applied feature rankings [13, 37, 38] for On-Target and Off-Target prediction, whereas most of the deep learning models do not provide any interpretation at all. A few deep learning models provide brief interpretation with saliency map [24] or input perturbation-based feature ranking [25] but they do not explore the complex decision-making process with a in-depth analysis. Due to the sensitivity of the application of CRISPR in real life, it is absolutely necessary to understand the inner workings and decision-making process of the prediction models.
In this study, we have attempted to address existing issues in Off-Target prediction. This paper makes the following key contributions.
METHODS
Dataset
The dataset used in the DeepCRISPR [24] study serves as a foundational resource for our analysis and it has been used as the benchmark dataset in most of the previous studies too. It comprises a collection of 30 sgRNA sequences from two cell lines, k562 (12 sgRNAs) and hek293 (18 sgRNAs), along with their corresponding potential Off-Target sites. The authors collected 153 233 loci across the whole genome using a tool named Bowtie 2 [40], allowing up to six mismatches. The dataset was labeled using various whole genome Off-Target detection methods [8, 14, 16, 41, 42]. The dataset is highly imbalanced with a 1:230 ratio of positive and negative samples.
For our experiments, we split the dataset into three subsets: training (60%), validation (20%) and test (20%). The purpose of these subsets is to train the model, tune hyperparameters and evaluate its performance, respectively.
We also have used the GRCh38 (Genome Reference Consortium Human Build 38) dataset the details of which is provided in Section B of the supplementary file. This dataset is used for pretraining purposes in our small-scale experiments using a transformer-based ELECTRA [43] model.
Performance metrics
For performance evaluation, we have employed various performance metrics like accuracy, precision, recall, F-1 score, Area Under the Receiver Operating Characteristics (AUROC) and Area Under the Precision-Recall Curve (AUPRC). Most of the previous studies reported and compared their results in terms of AUROC score. But AUPRC is often considered a better measure for imbalanced datasets compared with AUROC [25, 44]. Hence in our study, we have used AUPRC as the primary metric for comparison.
Recurrent neural network-based approach
The Off-Target prediction task involves analyzing two input sequences: sgRNA and target DNA. The objective is to determine the likelihood of potential Off-Target sites. To achieve this objective, we were motivated to explore recurrent neural networks (RNNs) and their variants. RNNs have gained significant popularity and success in sequence prediction tasks across various domains, such as NLP, speech recognition, time series analysis and genomics. While previous works have employed Long Short-Term Memory (LSTM) [20, 25], it has typically been used as a component within a larger model with heterogeneous components. We explored different variants of RNN, like vanilla RNN, LSTM and GRU (Gated Recurrent Unit), and their different compositions such as Bi-directional RNN and Stacked RNN.
Data preprocessing
First, we preprocessed the raw sgRNA and target DNA sequences using two one-hot encoding methods (4-channel and 5-channel) as shown in Figure 1. We followed the same approach used by CNN_Std [12]. For, 4-channel encoding, we converted both sgRNA and DNA sequences to one hot encoded matrix independently and superposed them using logical OR operation to produce the final encoding. We added an additional directional channel in the 5-channel encoding method. In our encoding scheme, the precedence of encoding is ‘A’, ‘T’, ‘C’ and ‘G’. In case of a mismatch, if the higher precedence nucleotide originates from the target DNA sequence, the direction channel is set to ‘1’.
Example of One-hot Encoding. (A) The sgRNA and DNA sequence are encoded with four channels independently and then superposed using logical OR operation. (B) An additional channel has been added for direction.
Generic model design
We designed a generic parameterized RNN model which would ease the process of hyperparameter tuning. A simplified block diagram of the model is shown in Figure 2. The model’s input would be the 4/5-channel encoded matrix. The input would then be fed to a recurrent layer(s). Based on the parameter there could be one or two recurrent layers and each layer could be bidirectional or unidirectional. The output from the recurrent layer is passed through a series of fully connected hidden layers each of which gradually halves the size of its previous layer. Finally, there is an output layer. We used CrossEntropy as the loss function and Adam (Adaptive moment estimation) [45] as the optimizer. All the hidden layers were followed by the ReLU activation layer and a dropout layer to tackle overfitting during training.
Block Diagram of Generic RNN Based Model. The encoded input matrix passes through one or two uni or bi-directional Recurrent layers, followed by a series of hidden layers. There are ReLU activation and dropout layers in between the hidden layers.
Model selection with genetic algorithms
In order to find the best set of hyperparameters for our model we employed genetic algorithms which are optimization algorithms inspired by the process of natural evolution. Most of the previous works performed model selection only on a limited scale and that too was mostly based on trial and error or adding or removing different components of the model [12, 31]. We selected the eight hyperparameters shown in Table 1 to be tuned for our generic model. Many of the parameters take on continuous values. But due to our limitation of computational resources we prepared a list of possible values for each parameter to be optimized which were reflected in general practice and findings from different empirical studies [46].
Among the hyperparameters, the possible values for the number of hidden layers are dependent on the hidden size of the recurrent unit. Even without including this hyperparameter, there would be 72 000 possible combinations of the hyperparameters. Training each model on our computational environment took around 10 to 30 min based on the selected hyperparameters. If we were to exhaustively search each combination, it would take about 1000 days! Hence we choose to apply genetic algorithm to intelligently explore the search space. The steps of the genetic algorithm are discussed below.
In our study, the population, elite and tournament size were 20, 4 and 2, respectively. The crossover and mutation probabilities were 0.3 and 0.2. The genetic algorithm was run for 20 generations. Each of the RNN, LSTM and GRU models works a bit differently so it was not conducive to performing crossover among different types of recurrent networks. Hence we run the genetic algorithm and its elitist variant for each of the recurrent model types independently.
| Hyperparameter Name | Range/ Values | Step Size for Range |
|---|---|---|
| Batch Size | 32–256 | Doubled each step |
| Epochs | 10–100 | 10 |
| Number of Recurrent Layers | 1,2 | – |
| Bi-directional Recurrent Layer | True, False | – |
| Dropout Probability | 0.1–0.5 | 0.05 |
| Hidden Size | 32–512 | Doubled each step |
| Number of Hidden Layers | 0–6 | 1 |
| Learning Rate | –0.5 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} $1 \times 10^{-5}$\end{document} | Multiple of 5, 10, 50, 100, etc. of the base value |
Transformer-based approach
In recent years, transformer-based large language models (LLMs) like BERT [29] and ELECTRA [43] have revolutionized NLP tasks. We performed small-scale experiments with ELECTRA by pretraining it with unlabeled samples obtained from the GRCh38 dataset which is a comprehensive assembly of the human genome [48]. The pretraining data preparation process was similar to DeepCRISPR [24]. The pretrained models were finetuned with the labeled Off-Target dataset. The details of this approach is provided in Section B of the supplementary file.
Code, environment and availability
All the deep learning models were designed using the PyTorch deep learning library [49]. The RNN-based experiments including the hyperparameter tuning with genetic algorithm were performed on an Nvidia RTX3090 GPU with 24GB VRAM. The ELECTRA models were pretrained and finetuned on an Nvidia Titan X GPU with 12 GB of VRAM. We used PyTorch Captum [50] library for model interpretation which provides the implementation of integrated gradients. The dataset and source codes are available at https://github.com/tzpranto/CRISPR-DIPOFF↗.
RESULTS
Model selection
In model selection for RNN-based models, we ranked and selected the models based on their AUPRC score on the validation set across all the experiments. The best RNN, LSTM and GRU model’s hyperparameters along with their performance on validation and test set are shown in Table 2. The detailed results of the individual experiments are presented in Section A of the supplementary file.
According to the results, all the models have performed quite well on the validation set in terms of the AUPRC score. We observed that most of the models from the plain genetic algorithm were already found on iterations 0, 1 or 2. This indicates that the best-performing models from the initial generation were not preserved in the subsequent generations and some useful parameter values might have got lost. This effect was reduced using the elitist version of the genetic algorithm which had produced the best-performing RNN and LSTM models. Surprisingly, in both versions of genetic algorithms, the performance of 5-channel encoded data was worse.
The selected best models were trained on the combined training and validation set and their performances were measured on the test set for further comparison with previous studies. The AUPRC scores on the test set have decreased compared with the AUPRC score on the validation set and it is more prominent for the RNN model (rightmost two columns of Table 2). In general, LSTM and GRU can generally store long-term dependency and capture complex patterns compared with RNN [51].
| Model Type | Hidden Size | LSTM Layers | Bi LSTM | Hidden Layers | Dropout Probability | Batch Size | Epochs | Learning Rate | AUPRC on Validation Set | AUPRC on Test Set |
|---|---|---|---|---|---|---|---|---|---|---|
| RNN | 256 | 2 | TRUE | 0 | 0.4 | 256 | 60 | 0.001 | 0.7643 | 0.5711 |
| LSTM | 512 | 1 | TRUE | 2 | 0.4 | 64 | 50 | 0.0001 | 0.7403 | 0.7208 |
| GRU | 128 | 2 | TRUE | 0 | 0.1 | 64 | 30 | 0.0005 | 0.7427 | 0.6859 |
Comparison with previous studies
We retrained the deep learning models from previous studies whose source code is publicly available among the models discussed in Section 1. Only the DeepCRISPR [24] model could not be retrained due to the unavailability of source code and data for pretraining but the published model was already trained with the whole DeepCRISPR dataset. All the models were evaluated with the test set and compared with our best-performing RNN, LSTM, GRU and ELECTRA models. In the case of ELECTRA model, we have reported the results of the best finetuned model obtained from our preliminary experiments described in Section B of the supplementary file. The results are shown in Figure 3. The underlying values are reported in Table 8 of the supplementary file.
Both our LSTM and GRU models outperformed all other models in terms of AUPRC score. Although our RNN model showed a slight decrease in performance on the test set, it still performed better than some of the previous models. Contrary to our expectations, the finetuned ELECTRA model did not yield the desired results. Among the baseline models, CnnCrispr demonstrated the most balanced performance, followed by CRISPR-IP. Other baseline models, such as CNN_Std, DeepCRISPR, AttnToMismtach_CNN, FNN_8Ch, demonstrated varying levels of performance.
As the dataset is highly imbalanced toward the negative class, it is expected that accuracy would be high even if a model is bad at predicting the positive class. Hence we observe that all the models have quite similar accuracy. The highest accuracy is 0.997 achieved by LSTM, GRU and the FFN_8Ch model. We can observe the same trend for the AUROC score too. Here the best score 0f 0.991 is achieved by our GRU model. We observe varying scores across the other performance metrics. There is a clear precision-recall trade-off among the models. For example, the CRISPR_IP model has the highest precision (0.882) but the 2nd lowest recall. On the other hand, our RNN model has the highest recall (0.886) but quite low precision. This trade-off is more prevalent in the baseline models and is evident in F-1 scores which are the harmonic mean of precision and recall. Both our LSTM and GRU models have relatively balanced precision and recall with the precision being slightly higher. Hence these two models have the highest F1 score of 0.667 and 0.656, respectively. The other performance metric is AUPRC which is considered a better indicator of performance for highly imbalanced datasets. The highest AUPRC score (0.721) was achieved by our LSTM model and it is followed by our GRU model. The LSTM model has achieved a 6.34% increase in AUPRC score and an 11.35% increase in F-1 score as compared with CnnCrispr, which is currently the best baseline model. Overall, the RNN models, particularly LSTM and GRU, significantly outperformed the baselines. This indicates the superiority of sequence-based models and the potential of hyperparameter-tuned models for improved performance in CRISPR Off-Target prediction. Another important observation is the simplicity of the high-performing models. Further elaborations along this line are done in Section. 4
During the first review cycle of this manuscript, we came across CrisprDNT [27], which is a deep learning model consisting of CNN, Bi-LSTM and Transformers for Off-Target prediction focusing on a new encoding scheme and custom loss function to tackle noise in the datasets. Though the authors have performed experiments on various datasets, many of these have overlaps with the DeepCRISPR dataset [24] used in our work. Our preliminary comparison between CRISPR-DIPOFF and CrisprDNT shows a better F1-score for the former (0.667 versus 0.652) with CRISPR-DIPOFF showing a more balanced precision-recall tradeoff: precision and recall of CRISPR-DIPOFF (CrisprDNT) are 0.734 (0.758) and 0.611 (0.573), respectively. In this connection, the much simpler architecture of CRISPR-DIPOFF is worth recalling.
Interestingly, CrisprDNT was tested on the DeepCRISPR dataset (as an independent dataset) and reported to show good generalization. While this sounds promising, the overlap between the DeepCRISPR dataset and the dataset used to train CrisprDNT suggests a data leakage, thereby reducing the appeal of this result. Nevertheless, irrespective of the results, the novel encoding scheme applied by the authors could be explored in the future for further improvement of our method.
Comparison of different models based on different performance metrics. The accuracy and AUROC score are quite similar for all the models. Other metrics have varied a lot. Our LSTM model achieved the most balanced score with the highest F-1 and AUPRC scores. The underlying values are reported inof the. Table 8 supplementary file
DISCUSSIONS
Some interesting and insightful observations may be derived from the results and analysis conducted in our experiments. Firstly, the ‘intelligent’ search mechanism for hyperparameter tuning through the use of genetic algorithms has yielded models with superior performance, despite the model having simple architecture. This observation highlights two important insights. One is the emphasis on the criticality of proper hyperparameter tuning in achieving superior performance. By carefully selecting and tuning the hyperparameters, we apparently were able to unlock the full potential of the RNN-type models and surpass the performance of the more intricate and complex baseline models. The other insight aligns with the principle of Occam’s Razor, which suggests that simpler models are often more likely to be correct among competing hypotheses. In the context of our study, this implies that a well-optimized and properly tuned simpler RNN-based architecture can effectively capture the underlying patterns and dependencies in CRISPR Cas-9 Off-target prediction better than more complex models.
Secondly, among the different encoding schemes used, the 4-channel encoding approach has consistently produced the best results for the RNN, LSTM and GRU models. This finding suggests that the addition of the directional feature in 5-channel encoding may have interfered with the generalization capability of the RNN models. Alternatively, it is possible that the specific mismatch patterns themselves, rather than the direction of the mismatches, play a more significant role in determining Off-Target prediction. Further analysis is needed to better understand the underlying mechanisms driving the performance variations observed across different encoding schemes.
Additionally, the finetuned ELECTRA model did not yield the expected output, which can be attributed to the limited scale pretraining. Despite its below-par performance, we find it worth discussing as a second independent (computational) validation for our interesting observation obtained from the LSTM model’s interpretation, as discussed in the following section.
Model interpretation
We have applied integrated gradients [39], an axiomatic interpretation technique, for explaining the predictions of our deep learning models, which quantifies the impact of each feature by computing the accumulated gradients along the integration path from a baseline input to the actual input.
Interpretation of the LSTM model
In our study, the best-performing model is an LSTM model which is trained on the 4-channel encoded input. This model’s architecture is shown in Figure 4. The model has a bidirectional LSTM layer followed by two hidden layers and an output layer. The hidden layers have 512 and 256 neurons, respectively.
Architecture of best-performing LSTM Model. The model has a bidirectional LSTM followed by two hidden layers and an output layer.
Feature importance
We calculated attribution scores for all the test samples using Integrated Gradients with respect to the positive class. For each sample, we normalized the attribution scores for all the features present in that sample. then we averaged the score of each feature across all the samples. Based on this score, the top 15 contributing features for positive, negative and overall prediction of the model are shown in Figure 5. Due to the higher ratio of negative samples, the most contributing features for negative and overall prediction are almost the same. We have drawn a few major observations from the importance scores as follows.
Top 15 features contributing most to (A) positive, (B) negative and (C) overall prediction with respect to positive class.
Layer importance
Layer attributions calculated with integrated gradients were used to interpret layer importance. It represents the significance of all the neurons contributing to the output of a specific layer. For each test sample, the attribution score of each neuron of each layer was calculated and averaged across positive, negative and all samples. Our LSTM model has two hidden layers with 512 and 256 neurons, respectively. The top 15 contributing neurons on positive, negative and overall predictions are shown in Figure 6.
All the neurons in Figure 6 are from hidden layer 1. In fact, only 16 out of the top 100 neurons in terms of overall prediction are from the 2nd hidden layer. This clearly indicates that the 1st hidden layer has more leverage on the final model output. Being directly fed with the LSTM’s output, the 1st hidden layer is able to capture more generalized context. The dominance of 1st hidden layer over the output is clearly visible from the heatmaps shown in Figure 7. The heatmap of the 1st hidden layer is filled with reddish colors, whereas the 2nd hidden layer contains dark patches denoting minimal activity.
Interpretation of the ELECTRA model
We only interpreted the embedding layers of the finetuned ELECTRA model (8). Similar to the interpretation of our LSTM model, we observe that the attribution scores indicate four distinguishable regions in the DNA tokens in terms of consecutive positive and negative scores. The regions are PAM, PAM distal (position 1 to 7) and two sub-regions in the seed region. One of the sub-regions (position 16 to 20) has a positive correlation with the Off-Target effect, and the other has a negative correlation. This is consistent with our findings in the LSTM model’s interpretation. This further strengthens our earlier observation that there are two sub-regions in the seed regions and one of them might be tolerable for mismatches.
Following some recent literature [38, 55], one of the motivations of this research was to apply only sequence data for the task at hand. This motivation stems from the fact that if we incorporate various other structural information, we usually need to use some other models and while doing so introduce inherent inaccuracies therefrom. Hence, we did not incorporate epigenetic, structural and energy-based features. Informatively, in a benchmarking study, Yan et al. [33] found that energy-based models tend to outperform learning-based methods, including DeepCRISPR [24], which attained the highest AUROC and AUPRC score after the energy-based models in their study. However, this finding seems outdated as many deep learning-based models with better efficacy have been published since then (some are considered in out comparative analysis as reported in Figure 3 ). Crucially, in the recently published work of Störtz et al. [35], the authors have found less importance in energy-based features compared with sequence-based features. Consequently, we did not include the non-learning-based studies presented in the above benchmarking paper in our comparative analysis.
Attribution scores at embedding layers for all the input tokens of finetuned ELECTRA model.
CONCLUSION
CRISPR Cas-9 has shown great promise in the field of genome-editing for the past few years but the possibility of Off-Target effects still remains a monumental challenge. The current state-of-the-art deep learning models suffer from precision-recall trade-off and due to the lack of interpretations, these models are still black boxes to the end users. Here we have successfully developed interpretable deep learning models capable of predicting Off-Target sites with high efficacy surpassing the previous studies using only sequence data. Through the interpretation of our models, we seem to have unraveled the existence of sub-regions within the seed region. This can be seen as an extension/elaboration of the current hypothesis and it calls for further biological and computational research.
Future Extensions: As an immediate future work, other Cas variants like Cas-12 and Cas-13, insertion–deletion-type mismatches and larger context regions of the target DNA could be explored. Also, more benchmark datasets from different cell types and species could be explored to enhance the applicability and robustness of our proposed methodology. We are thus swayed to conduct a comprehensive benchmark study including more datasets and more models (both learning and non-learning). In addition, proper pretraining of ELECTRA or similar LLMs, particularly on the whole genome sequence, could be used for both On-Target and Off-Target predictions at the same time and even for any other predictive tasks requiring genomic contexts.
Limitation of this study: Our study has some limitations. Firstly, as has already been alluded to above, we only considered substitutions which is the common cause of Off-Target and did not consider insertion and deletion-based mismatches. Secondly, we did not consider the structural and energy based models in our comparative study because as per recent evidences, deep learning based models are expected to significantly outperform them. Finally, we experimented with only one dataset. While the DeepCRISPR dataset that we used is a comprehensive dataset, and many of the datasets found in the literature are included therein, we actively will explore more datasets as and when available in the literature.