Innovative TRAFICA model from HKBU transforms transcription factor binding analysis

Transcription factors are central to gene regulation, dictating biological processes by binding to specific DNA sequences. While computational prediction tools have long sought to map these interactions, existing models predominantly rely on in vitro binding profiles, often ignoring a fundamental in vivo constraint: transcription factors typically bind to sequences located within open chromatin regions. 

Led by Professor Lyu Aiping (Vice-President (Research and Development) and Dean of Graduate School) and Professor Zhang Lu (Associate Professor, Department of Computer Science), this study introduces an open chromatin language model specifically designed to overcome this limitation. By integrating the sequence characteristics of open chromatin regions with high-throughput in vitro binding profiles, the model, named TRAFICA, achieves state-of-the-art predictive performance in determining transcription factor-DNA binding affinity. 

The biological reality of chromatin accessibility 

In living cells, the physical positioning of nucleosomes restricts access to the genome, meaning transcription factors can only interact with DNA that is structurally accessible. Consequently, in vivo binding affinity is a highly complex mechanism influenced by cooperative binding dynamics alongside intrinsic sequence preferences. Traditional machine learning and deep learning tools, including DeepBind and DNAshapeR, generally evaluate nucleotide sequences without contextualising whether those sequences reside in accessible regions. 

The architecture outlined in the study directly addresses this blind spot through a comprehensive pretraining phase. The researchers trained the model on a vast dataset comprising over 2.8 million accessible genomic sequences, which were derived from 197 ATAC-seq experiments covering 115 unique human cell lines. Through masked language modelling, the system learned natural, in vivo binding contexts. Validation of this dataset revealed that 93.23% of the sequences overlapped with known enhancers and 37.64% with promoters, confirming their vital regulatory functions. 

A dual-stage training architecture 

The foundation of this predictive tool is a Transformer-encoder structure enhanced with Rotary Position Embedding, enabling it to process sequences of varying lengths. Following the pretraining on open chromatin data, the model undergoes a highly targeted fine-tuning stage. During this phase, the researchers employed low-rank adaptation to update only specific projection layers within the multi-head self-attention modules. This strategic freezing of parameters ensures that the foundational open chromatin features are preserved while the model learns the intrinsic binding preferences of specific transcription factors. The fine-tuning process leveraged extensive datasets from Protein Binding Microarrays and High-Throughput Systematic Evolution of Ligands by EXponential enrichment experiments. 

Evaluating predictive superiority 

Systematic benchmarking demonstrates that incorporating chromatin accessibility dramatically improves prediction accuracy. When evaluated using the DREAM5 challenge datasets, the model consistently outperformed existing analytical tools and advanced language models like DNABERT2. Specifically, the version of the model utilising 5-mer sequence tokenisation proved exceptionally robust, securing an average Pearson correlation coefficient of 0.647, which significantly surpassed competing methods. 

Comprehensive ablation studies further validated the methodology, revealing that training the model from scratch without the initial ATAC-seq pretraining resulted in a substantial decline in predictive capability. This confirms that integrating natural chromatin contexts as prior knowledge is essential. Furthermore, when tested on in vivo ChIP-seq datasets encompassing 141 transcription factors across 18 distinct cell lines, the tool maintained its superiority over models pretrained merely on whole reference genomes.

Biological relevance and future potential 

Beyond generating raw performance metrics, attention-score analysis indicates that the model genuinely captures biologically relevant binding patterns. The sequence motifs extracted directly from the model's attention mechanisms align closely with validated motifs documented in the JASPAR database. Detailed analysis also revealed that individual attention heads within the network successfully recognise distinct sequence characteristics, ranging from specific motif fragments to broader nucleotide compositions. Designed with a relatively compact architecture of approximately 26 million parameters, the model requires fewer computational resources than larger alternatives, though its generalisability can decrease when overwhelmed by noise in extremely large fine-tuning datasets. The framework presented in this study establishes a powerful computational foundation, setting the stage for future integrations with single-cell multiomics to predict highly specific, cell-type-dependent binding affinities. 

Full paper on Bioinformatics: https://academic.oup.com/bioinformatics/article/41/11/btaf469/8240329

Professor Lyu Aiping

School of Chinese Medicine

Professor Zhang Lu

Faculty of Science and Technology