Retrosynthesis, the iterative process of breaking down a molecule into simpler precursors, has traditionally been the domain of expert organic chemists. However, even for experienced chemists, this approach presents challenges due to the vast chemical space of potential transformations and the incomplete understanding of reaction mechanisms and their dependence on reaction conditions. To overcome these challenges, efforts have persisted since the 1970s to integrate computation into synthetic planning by developing Computer-Aided Synthesis Planning (CASP) tools, with one of the earliest examples being the Logic and Heuristics Applied to Synthetic Analysis (LHASA) by Pensak et al. Despite numerous attempts, CASP tools had limited success until recently. Significant progress in CASP tools has occurred in the last decade, driven by advances in machine learning (ML) methodologies and the availability of chemical datasets, such as Lowe's US Patents Office (USPTO) reaction extracts. Following the seminal work by Segler et al. on the use of neural networks and search algorithms in the 3N-MCTS CASP tool, there has been a proliferation of new ML models for retrosynthesis prediction. These models can be broadly classified into two categories: template-based and template-free methods. Templatebased methods rely on predefined reaction rules extracted from datasets, where algorithms match a target molecule with predefined templates. CASP tools utilising such models include ASKCOS, AiZynthFinder, and Retro*. In contrast, template-free methods, such as graph-based or sequence-to-sequence (seq2seq) approaches, bypass the use of an external template database by directly training on raw reaction data. While early seq2seq models were based on long-short term memory networks (LSTMs), the breakthrough in seq2seq reaction prediction came when Schwaller et al. applied the transformer model commonly used in Natural Language Processing (NLP) for forward reaction prediction, creating the Molecular Transformer. In this case, reaction prediction is treated as a translation problem using Simplified Molecular Input Line Entry System (SMILES) strings to represent the chemical transformation. Since then, seq2seq retrosynthesis prediction models have shown high accuracies on public benchmarking test sets, with the Augumented Transformer 11 achieving 46.2% top-1 reactant accuracy on the USPTO-full dataset. The recent developments have led to transformers emerging as a premier architecture for retrosynthesis planning utilised in platforms such as IBM RXN. Despite the high efficacy of CASP tools on general reaction datasets, predicting retrosynthetic disconnections for specific, less prevalent areas of chemistry remains a significant challenge due to dataset bias. Heterocycle formation reactions are an example of underrepresented reaction classes, accounting for only 5% of reported chemical reactions in the USPTO dataset. However, heterocycles are key motifs in drug design, with 85% of the top 200 best-selling small molecule drugs of 2022 featuring heterocyclic rings, where they act as bioisosteric replacements improving pharmacokinetic and toxicological properties of drug targets. Although numerous virtual libraries document theoretically synthesisable heterocyclic scaffolds, synthetic pathways towards novel heterocycles remain underexplored, with the focus in medicinal chemistry being on ring derivatisation rather than ring formation. Enhancing the prediction capacity of CASP tools for reactions forming these crucial chemical motifs could stimulate the exploration of novel heterocyclic molecules, potentially fuelling new therapeutic breakthroughs. This work aims to enhance the performance of CASP tools for heterocycle retrosynthesis by combining seq2seq models and transfer learning, where knowledge learned from one task is used to boost the performance on a related task (Figure ). Two transfer learning approaches, fine-tuning and multi-task learning, have been previously successfully applied for the forward reaction prediction of carbohydrate reactions and Heck reactions, as well as forward and retrosynthesis prediction of enzymatic reactions (Figure ). However, both of those approaches come with limitations. For example, in the reported examples, fine-tuning has shown a quick training time and increased accuracy for reactions of interest but exhibited a large drop in performance for other, more common reaction types. Conversely, multi-task learning maintained good performance on all reaction types but required longer training time, making it less suitable for frequent retraining as new reaction data becomes available. To address these limitations, here we evaluate mixed fine-tuning 31 and ensemble decoding, which have previously proven effective in language translation tasks but have not been used in retrosynthesis prediction (Figure ). We compare those methods to the template-based approach reported by Thakkar et al. specifically for ring-forming reaction prediction in the 'Ring Breaker'. We use two datasets to train these models: a large dataset of all reaction types based on USPTO ("General ") and a smaller dataset of just heterocycle formations ("Ring"). We show that the mixed fine-tuned model is the best for use in multi-step retrosynthesis, with 10% increase in accuracy over the baseline for heterocycle formations and only a marginal decrease in performance for other reactions. We then further demonstrate its applicability by predicting retrosynthetic routes towards two recently published heterocycle-containing drug-like targets. Finally, we test the mixed-fine tuned model on recently developed heterocycle formations and demonstrate how it can be further fine-tuned to improve its accuracy on this new data.