{'fraction_non_alphanumeric': 0.026461693548387098, 'fraction_numerical': 0.013835685483870968, 'mean_word_length': 4.812362677603633, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DARE: A large-scale handwritten date recognition system

Handwritten text recognition for historical documents is an important task but it remains difficult due to a lack of sufficient training data in combination with a large variability of writing styles and degradation of historical documents. While recurrent neural network architectures are commonly used for handwritten text recognition, they are often computationally expensive to train and the benefit of recurrence drastically differs by task. For these reasons, it is important to consider non-recurrent architectures. In the context of handwritten date recognition, we propose an architecture based on the EfficientNetV2 class of models that is fast to train, robust to parameter choices, and accurately transcribes handwritten dates from a number of sources. For training, we introduce a database containing almost 10 million tokens, originating from more than 2.2 million handwritten dates which are segmented from different historical documents. As dates are some of the most common information on historical documents, and with historical archives containing millions of such documents, the efficient and automatic transcription of dates has the potential to lead to significant cost-savings over manual transcription. We show that training on handwritten text with high variability in writing styles result in robust models for general handwritten text recognition and that transfer learning from the DARE system increases transcription accuracy substantially, allowing one to obtain high accuracy even when using a relatively small training sample.


Introduction
Handwritten text recognition (HTR) is an important step towards the transcription and preservation of historical archives but it remains a challenging task. One obstacle is that deep learning models often require large-scale annotated datasets to perform well, which is particularly a concern for HTR on historical documents due to the high variation in writing styles and that historical documents often suffer from bleedthrough, fading of text, and general degradation. One step towards solving this is to introduce more and larger real-world datasets that can increase the robustness of HTR models. Even though dates are some of the most important and frequently represented information on historical documents, little research have been invested into the specific recognition of handwritten dates from historical documents. To solve the challenge of handwritten date transcription from historical documents, we introduce the DARE database and system, which together facilitate the transcription of dates even from difficult-to-read historical documents. 1 The DARE database is the largest available database of handwritten dates and consists of segmented dates from a number of different historical documents, meaning that the database contains many types of documents from different time periods written by a large number of authors and suffering from varying amounts of degradation. This results in significant variation between the handwritten dates, see, for example, Figure 1, which enables us to train robust models for transcription, which in turn perform well for transfer learning. In total, the DARE database is derived from 3,145,922 images, originating from six different data sources. While a large number of the segmented images are empty, we still obtain more than 2.2 million manually transcribed dates, totalling almost 10 million individual tokens. 2 We use this large database of handwritten dates to train neural networks to transcribe dates: The DARE system. These networks achieve date transcription accuracies of between 92% to 99%, with the exception of one particularly difficult dataset. 3 To put this performance into context, we compare the performance on one of these datasets to previous work and demonstrate significant improvements over manual labelling. While it is reassuring that the DARE system achieves high levels of transcription accuracies on the test sets, our ultimate objective is to create a system that improves automated transcription of dates from new data sources. To illustrate the usefulness of the DARE system for such tasks, we first use the DARE system for transfer learning, showing that it significantly improves transcription performance over networks trained from scratch as well as networks transfer learned on other data sources. Using the DARE system, we are able to achieve high levels of transcription accuracy even when using only a small training sample.
Second, we use Danish census data from 1916 to further illustrate the usefulness of the DARE system for transfer learning and for zero-shot transcription. A subset (less than 5%) of the 3.7 million entries recorded in the 1916 census, for which we have image files, are labelled manually. However, the labels come with no link back 2 We still include the empty images when training our models, as we want models that are robust in the sense of being able to inform when a date is not present on an image.
3 This is primarily due to errors in the manual labels and the segmentation. However, we still believe that this dataset is helpful to improve the overall performance of our neural networks.
to the source images, so the labels cannot directly be used as training data. We show that by transcribing images in a zero-shot fashion using the DARE system and subsequently linking the images to the labelled data, we can gradually create a larger training set to which we can eventually apply transfer learning and obtain a more accurate model. This procedure is repeated until a sufficient degree of accuracy is obtained.
This procedure allows us to match nearly all of the labelled entries to the source images, resulting in a large training sample that makes it possible to transcribe the census with a date recognition accuracy of around 95%. The resulting database comprising more than 3.7 million rows contains unique information on name, birth date, residence, civil status, income, and wealth for every single individual living in Denmark (excluding Copenhagen residents) in 1916.
The rest of the paper proceeds as follows: We start by surveying some of the most relevant literature within HTR. Next, in Section 3 we describe the database, network architecture, and technical details of the training pipeline. Section 4 shows the within-distribution performance of the DARE system, by studying the performance of the system on the test sets of the DARE database. Section 5 describes the pipeline for transfer learning and presents the associated results. Section 6 shows the process of linking images to transcriptions. Section 7 concludes.

Related work
Performance of state-of-the-art deep learning HTR models have improved considerably within recent years. From an initial focus on hidden Markov models (HMM) [Toselli et al., 2004, Bertolami and, more recently, focus has shifted towards deep learning models such as CNNs [Yousef and Bishop, 2020], hybrid CNN-RNNs [Dutta et al., 2018], or other types of RNNs, including LSTM models for HTR challenges, such as bidirectional LSTMs (BLSTM) , Wigington et al., 2018, multidimensional LSTMs (MDLSTM) , MDLSTM-RNNs [Puigcerver, 2017], or deformable convolutional-recurrent networks [Cascianelli et al., 2022]. The introduction of the connectionist temporal classification (CTC) loss [Graves et al., 2006] played a significant role and provided a significant advantage for RNNs over the previous HMMs [Wigington et al., 2018]. However, one of the challenges with many of the different variants of the recurrent models is that they are computationally slow due to the lack of parallelization during training [Kang et al., 2022].
Another limiting factor for deep learning models, especially for HTR models where there is a huge variability of writing styles, is the need for large amounts of training data. To overcome this, it has been suggested to perform better data augmentation in order to increase the variability of the training data [Puigcerver, 2017] and to enhance existing pre-processing procedures to mitigate some of the concerns regarding the degradation of historical documents, e.g., through improving upon the grayscale algorithm as proposed by Bouillon et al. [2019]. Another possibility to overcome the lack of training data and reduce the cost of manual labelling is to use GANs [Goodfellow et al., 2014] as a semi-supervised tool to create synthetic handwritten text [Fogel et al., 2020].
Inspired by the more recent interest in sequence-to-sequence models for HTR [Aberdam et al., 2021, Geetha et al., 2021, Transformers [Vaswani et al., 2017] have also received an increased interest. Transformer-based models for HTR challenges are presented in both Li et al. [2021] and Kang et al. [2022] from where it seems that in order to obtain results similar to those acquired by other architectures, these currently require pre-training on large synthetic datasets followed by fine-tuning on real annotated data. This paper departs from much of the HTR literature by proposing a non-recurrent architecture. Our focus is on a relatively light-weight architecture that is fast to train and performs well in the absence of large amounts of labelled data. To achieve this, we draw inspiration from the literature on image classification and adopt many of the optimization techniques used there, e.g., RandAugment [Cubuk et al., 2020], label smoothing [Szegedy et al., 2016], and stochastic depth [Huang et al., 2016].
We use an EfficientNetV2 [Tan and Le, 2021] as the backbone of the architecture and a multi-head classification approach inspired by the house number recognition model proposed in Goodfellow et al. [2013]. Our work shows that the non-recurrent architecture we propose is not a limitation in the setting of date recognition, which we believe is due to the limited additional information on the next token in a date sequence by knowing the former token. 4 Instead it leads to faster training and 4 For example, knowing that it is the 14th provides limited information about which month it is. In certain cases the reverse is true: For example, knowing that it is the 31st excludes a large number of months.
inference due to additional available parallelization.

The DARE system
The DARE system we are proposing consists of several unique neural networks.
Each of the networks are trained on specific subsets of the DARE database and each network has a unique task in relation to transcribing the dates. In this section, we present the DARE database, the neural networks, including their architecture and how they are trained.

Database
The DARE database consists of 3,145,922 images, of which 2,221,954 contain handwritten dates. The handwritten dates are typically written on a single line. The source images vary in size, with widths up to 1400 pixels and heights up to 356 pixels. Figure 1 illustrates the large variance in style. We briefly describe our data acquisition strategy in A. The visualization in A together with Figure 1 illustrates that both within and between datasets, the style -in terms of handwriting, structure, and content -varies significantly. Table 1 shows an overview of the different datasets, including what information is available (i.e., how complete the date is), which source the dataset originates from, and how many labelled images are in each dataset (split into a train and test sample).
We split the database into a train and test set for each dataset separately. The test set is randomly selected and consists of approximately 5% of each dataset. The split enables us to test the performance on each dataset internally, which is important as the difficulty (in part due to image, segmentation, and label quality) differ between the datasets. There is also significant variation between the different handwritten dates across the sources, time-period and writing styles, of which an example is provided in Figure 1.
For ease of readability, we use an abbreviated name for each dataset (also visible in the table). The general format of the date sequences across data sources is DD-M-YYYY whenever the year, including the century, is present on the images. 5 For many of the datasets part of (or the entire) year is not present: When only the last two digits of the year are available, we write DD-M-YY, and when the entire year is missing, we write DD-M.

Network architecture
Each neural network uses an EfficientNetV2-S as its feature extractor [Tan and Le, 2021]. This choice is motivated by the unprecedented trade-off between accuracy and required computation achieved by the family of EfficientNetV2 models, both compared to other convolutional neural networks and vision transformers and both in terms of training and inference speed. While our models are largely identical, the networks differ in terms of their classification heads. As shown in Table 1 To construct classification heads able to predict dates, we draw inspiration from Goodfellow et al. [2013] and construct different heads for each part of the date. For the architecture that predicts both the day, month, and entire year, we construct seven classification heads in total: Two heads to predict the date (one for the first and one for the second digit), another head to predict the month, and four heads to predict the year (one for each of the four digits). As opposed to Goodfellow et al. [2013], we do not estimate the length of our sequences; while house numbers differ in lengths, the length of a date is not well-defined. 6 Further, the number of classes differs between our classification heads: The first day classification head can predict four classes (1-3 and missing), the second day classification head can predict 11 classes (0-9 and missing), the month classification head can predict 14 classes (1-12, missing, and wildcard 7 ), and all four year classification heads can predict 11 classes (0-9 and missing). 8 For the network predicting the day, month, and last two digits of the year, we remove the other two heads related to the year (leading to a total of five classification heads). For the network predicting the day and month, we remove all heads related to the year (leading to a total of three classification heads).

Optimization
We use the same basic recipe to train our networks, regardless of the specific architecture. We use stochastic gradient descent with momentum as our optimizer and a cosine annealing learning rate scheduler with warmup to control the learning rate.
As our optimization criterion, we draw inspiration from Goodfellow et al. [2013] and estimate losses separately for each token, after which we combine the token losses to obtain the total loss. However, our approach incorporates label smoothing and allows for a different number of classes across classification heads. Further, we normalize the loss by the sequence length, to obtain more comparable losses across models with different numbers of classification heads. For a sequence of t = 1, . . . , T elements, each with C t distinct classes, we seek to minimize the loss represented by Equation 1. Here, α t controls the degree of label smoothing, T the number of classification heads, and C t the number of classes for classification head t, t = 1, . . . , T . For α t = 0 for all t, this maximizes the probability of a sequence given the image, provided the different elements are (conditionally) independent. If additionally T = 1, this is the standard cross entropy loss commonly used in, e.g., image classification, with C 1 number of classes.
Across all our models, we resize our images to a height of 160 pixels and a width of 352 pixels. 9 As initial weights, we use the weights from EfficientNetV2-S pretrained on ImageNet21k [Russakovsky et al., 2015]. As a default, we use a batch size of 256 and a learning rate of 0.6, and train for a total of 250 epochs, where the first ten are used to warm up the learning rate. However, for some of our modelsthose using a particularly large number of observations -we change the batch size, learning rate, and number of epochs used to warm up the learning rate. More details on the specific parameters are presented in Table 5 in B.

Networks
In total, we train 11 neural networks as part of the DARE system, each on a different subset of the DARE database; here, we use PyTorch Image Models [Wightman, 2021]. First, for each of the eight individual datasets (see Table 1

Performance measures
To measure the performance of our networks, we focus on the sequence accuracy (SeqAcc) of our transcriptions. A date prediction is deemed correct if all parts of the transcription is correct, and thus incorrect if any one of up to seven token predictions is incorrect; this also means that the token accuracies of our networks are significantly higher than the reported sequence accuracies.
4 Transcription performance Table 2 shows the performance of our date transcription models for each individual dataset of the DARE database, with the best result for each dataset marked in bold. Across all but one dataset (PR-2), all models reach high levels of SeqAccs, ranging from 92% to more than 99%. 10 In comparison, Dahl et al. [2021] found that Transkribus obtained a sequence accuracy of 74.9% and 96.3% using crowdsourcing on DC-1. In comparison, our models achieve a SeqAcc of over 97% on this subset 10 The poor performance on PR-2 stems from primarily two causes: First, the segmentation of these fields is quite poor, and thus the date is not always present on the segmented fields which is a common issue in this sequential segmentation and recognition pipeline [Wigington et al., 2018]. Second, the label quality is poor, meaning that the label does not always reflect the date that is actually present on the image. For these reasons, the performance on this dataset does not reflect actual performance directly, as it is impossible to reach anywhere close to 100% accuracy due to the poor quality of the data. Nonetheless, we find that the data is still usable for training, improving the generalization of our models.

Transfer learning
While Section 4 demonstrated the performance of the DARE system in an internal sense (achieving high transcription performance on the test splits of the datasets used to train the system), our ultimate goal for the DARE system is to show its generalizability to other datasets. In this section, we document that using the DARE system for transfer learning leads to significant improvements in transcription accuracy of dates from historical Swedish grade sheets. To this end, we train networks to transcribe dates from Swedish grade sheets with and without transfer learning from the DARE system, demonstrating that using the DARE system for transfer learning improves performance. Further, we demonstrate that transfer learning from the DARE system is better than transfer learning from ImageNet21k [Russakovsky et al., 2015], a dataset of 13 million images that is commonly used to pre-train models before finetuning on downstream tasks. That transfer learning from the DARE database is significantly better is particularly promising, as pre-training on the ImageNet21k dataset is used to achieve the best performance in image classification, aside from the use of non-public Instagram/JFT images. We train eight neural networks to transcribe these grade sheets: One trained from scratch, one using ImageNet21k pre-trained weights (similar to how we train the networks in Section 3), and one using DARE pre-trained weights. We use largely the same architecture and training recipe as discussed in Section 3 to train these networks. As the century is rarely available on the image and never differs when available (only ever being 19) we only transfer from models trained on a sequence containing some year information, using the architecture with five heads (M-DDMYY). When training our networks, we use the same recipe with the exception of the learning rate and number of epochs. 11 Due to the difference in training set size and the use of transfer learning for seven of the eight models, we perform a small search over exponentially distanced learning rates for each network. Further, we experiment with a longer training schedule in terms of number of epochs (due to the small training set size). We then use the learning rate and number of epochs that perform best on the 2,000 training images and finally evaluate the performance of the models on the test set of 4,139 images. Notes: The table shows the SeqAcc on the Swedish grade sheets of eight different models: One trained from scratch, one finetuned from ImageNet21k pre-trained weights, and six finetuned from DARE pre-trained weights. It also shows the SeqAcc improvement and the error rate reduction obtained by using transfer learning, compared to the baseline model trained from scratch. Rows are ordered according the their imporovement over the baseline model. Table 3 shows the SeqAcc of all eight models as well as the SeqAcc improvement and the error rate reduction obtained by using transfer learning compared to the model trained from scratch. There is significant variation in the transcription performance achieved by the different models, with the SeqAcc ranging from 85.17% for the model trained from scratch to 96.18% for the best model transfer learning from the DARE system. We find that transfer learning from the full database is superior to transfer learning from subsets of the DARE database. While PR-2 is the second largest dataset, transfer learning from this dataset is outperformed by two smaller dataset (DC-2 and SWE-BD). We conclude from this that it is not only the size of the transfer learning dataset that is important, but rather the interaction between quality and quantity. Hence, the best performance is achieved when transfer learning from the M-DDMYY model -outperforming not only training from scratch but also transfer learning from smaller subsets of the DARE database or ImageNet21k. This demonstrates the usefulness of the DARE system: At no cost in required training resources or time, date transcription accuracy is significantly improved on this new

Figure 3: Performance on Swedish Grade Sheets
Notes: The figure shows the accuracy at different levels of data coverage on the Swedish grade sheet data for three models: One trained from scratch, one using ImageNet21k pre-trained weights, and one using DARE (here represented by M-DDMYY) pre-trained weights. The models are evaluated on between 5% and 100% data coverage, meaning that the leftmost points are measures of the accuracy if we keep only the 5% of predictions that the network is most confident about and the rightmost points are the SeqAccs.
dataset. In fact, in order to obtain the results for the model trained from scratch, a much longer training schedule was used, meaning that transfer learning reduced the computational resources required to train the model.
Another useful feature of our date transcription model is presented in Figure 3, which shows the SeqAcc of three models (no transfer learning, ImageNet21k, and M-DDMYY) at different levels of data coverage. Specifically, we gradually remove predictions where the models are the least certain and evaluate the models on the remaining images. Similarly for all models, removing 10 to 20% of images where they are the least certain significantly improves their performance on the remaining set. This shows that all the models are able to identify the subset of images where they might need assistance, and the large subset where our confidence in their per-formance is high. Further, it shows that achieving, e.g., 98% SeqAcc can be achieved at a much higher data coverage for the model transfer learning from the DARE database compared to the other models. More broadly, the figure shows that the models all perform fairly equally up until 70% data coverage. From this point on the model trained without transfer learning experiences a sizable fall in its performance.
We find a similar pattern at around 80% data coverage for the model using the pre-trained weights from ImageNet21k. To achieve, e.g., 98% SeqAcc at above 80% data coverage, only the model transfer learning from the DARE database is adequate and able to retain this high accuracy until around 95% data coverage. Further, the approach we use is completely generalizable to all settings where segmented fields of dates require transcription -no part of this is made specifically for the Swedish grade sheets. This allows others to directly use the DARE system for their own projects on any dataset and potentially achieve significantly improved transcription performance. To further showcase the importance of the combined DARE database we illustrate the respective performance achieved by transfer learning from only parts of the database relative to the full M-DDMYY model in Figure 4. Figure 4 shows the error rate reduction achieved by transfer learning from six different models. We find that there seems to be a connection between the size (and quality) of the dataset used for transfer learning since these are sorted according to the accuracy from left to right and this roughly corresponds to the size of the different datasets. The exception to this is M-PR-2 which performs worse than M-DC-2 and M-SWE-BD (note that SWE-BD "only" consists of 198,096 transcribed dates). We believe this likely is caused by the relatively poor quality of the PR-2 dataset. In

Figure 4: Error Rate Reduction on Swedish Grade Sheets
Notes: The figure shows the error rate reduction achieved by transfer learning from six different models from the DARE system (the M-DDMYY model), compared to training from scratch, on Swedish grade sheets. The error rate reduction is calculated as the relative reduction in SeqAcc achieved by the models compared to the baseline model (no transfer learning). The models are ordered according to the error rate reduction they achieve, such that the rightmost bar corresponds to the model that achieves the largest error rate reduction.
addition, we find that transfer learning from the combined DARE database outperform the performance relative to the other models trained on only a single dataset.
Although, the absolute performance in terms of accuracy "only" ranges from 95.19% to 96.18% (see Table 3) we find that there is an advantage from transfer learning from several and larger datasets.

Linking pipeline
To illustrate another important application of the DARE system, we use it for linking and matching more than 140,000 individuals. Originally, information on these individuals has been manually transcribed by crowd sourcing but links back to the source images was not part of the transcription. Below we discuss how to link the manual transcriptions to the source images and subsequently we describe how to use the linked entries for training and transcribing a total of 3.7 million entries from the 1916 Danish Census, without requiring additional manual transcriptions.
As a starting point, we obtained 156,712 manually transcribed census records from the National Archives covering parts of the population of Denmark in 1916. As already mentioned, the manual transcriptions are not linked to the scanned records.
They only contain a reference to the folder from which they were extracted. Therefore, in order to use the transcriptions for training, we have to link the scanned documents to the transcriptions, using matching. We suggest to use a sequential matching procedure which in many ways are similar to a linking pipeline when combining corpuses across data sources. In addition, we avoid manual labelling by implementing a pretrained date network, namely the M-DDMYY model. While the date alone is not enough to identify individuals (due to many individuals with the same birth date), using the first and last name in combination with the date allows us to identify most individuals uniquely (in combination with information on the reference to the source folder). We therefore also transcribe the names (first and last) of the individuals in the census, using networks from Dahl et al. [2022a].
Our proposed linking pipeline use the DARE system to obtain date predictions for all dates from the 1916 Danish Census. We then match predictions that fulfill the criterion of having at least two equal (prediction, transcription)-pairs within the specific regions. Specifically, we match an image to a transcription if at least two of three of the date, first name, and last name match exactly and there are no duplicate cases. After matching, we use the matched data as a training set, and transfer learn from the three models (date, first name, last name). These models are used for predicting all entries from the 1916 census and we can repeat the matching process with the (expected) improved predictions. We perform several (5) rounds of iteratively predicting, matching and training to obtain additional linked images to the transcribed data. Each of these rounds is similar, differing only with respect to what training data is available for use. However, we are faced with one important issue: If we transcribe an image incorrectly, but this incorrect transcription leads to a match, our networks may, due to simply memorizing the training data, keep transcribing this image incorrectly. To combat this issue, we need a process in our matching procedure to "push" out such cases, even if they enter our training data at some point. To do so, we split the training sample into two and train networks separately on each of the two sets (that is, we train a total of six networks, two for each set, in each round). We then use each set of networks to transcribe and match all images, and only keep the intersect of the matched entries between the two sets of networks. If an incorrect transcription enter our training data, this image will only be used to train one set of networks, and it is unlikely that the other set should transcribe (and match) this image incorrectly in the exact same way as the other set, and the incorrect transcription will thereby be pushed out of our training data. Only in round 0 do we perform the 0th step, and for rounds 1-5 we go through the steps 1-4. After round 0, the steps are performed as: Step 0 Step 1 Step 2 Step 3 Step 4 Notes: The figure shows the process for linking and matching to the Danish census data. Initially we predict on the full census data from 1916. We match the predictions to the manually transcribed records. We match exactly by region and two (or more) other identifiers (first name, last name, and date). After the initial round we match our predictions twice each round, one time using the predictions obtained from the first set of models and once using the predictions from the second set of models. Subsequently, we only use the intersection of the two matched samples obtained from the two sets of models. Once the predictions are matched to the transcribed records, we extract these matches and create two new tuning datasets (each containing 50% of the intersected matched entries) that can be used for transfer learning. From the transfer learned models we perform the final step of predicting on all census records from 1916. We define these steps as a single round, which we in turn do a total of five times before obtaining the final transcriptions.
Step 1: Using predictions, match transcribed images to the transcribed data.
Using only the intersection of the matched sample from the two sets of models.
Step 2: Extract data from the intersection of the matched and transcribed images to use as a training sample. Split sample into two sets with respectively 50% in each set.
Step 3: For both subsets, train a date, a first name, and a last name transcription network, transfer learning from the M-DDMYY model for the date transcription and the models from Dahl et al. [2022a] for the name transcription.
Step 4: Use the trained models to predict the dates and names of all images, resulting in two transcriptions for each image.
In the setting of the 1916 Danish Census, the initial round leads to almost 8,000 matches. We split these observations into two sets and train models based on the matched transcriptions. Hence, based on 50% of the 8,000 matches we train three models, one for date, one for first name, and one for last name) and the other half we use to train three other models. These two sets of models are after training used to predict all images once again, and we use the intersection of the matches obtained from these two sets of models, to define the labels for the following round.
As we repeat this process for several rounds we obtain more than 143,000 matched observations (out of the 156,712 potential matches). We use this to train the final models that uses 100% of the available training data and predict the entire Danish census. These predictions constitute our final transcriptions, where we manage to match 146,081 of the 156,712 observations (a match rate of over 93%). While our matching procedure attempts to remove any cases of incorrect transcriptions, we perform a test to verify that the final transcriptions are correct. Specifically, we manually review transcriptions of 1,000 randomly selected observations that were not used as training data in the final round. For both the date, first name, and last name, we manually compare the transcription to the source image, noting down whether (1) it is correct and (2) whether we are able to read the source image.
Statement (2) is also illustrated in Figure 6 where we provide some transcriptions where especially the date were difficult to validate and we decided that we were not confident in labelling the transcription of the date as either correct or incorrect.
Due to this complication, we define the projected SeqAcc as the SeqAcc on the sample we are able to read, as well as upper and lower bounds (defined, respectively, by labelling all illegible cases as correct or incorrect). Table 4 shows these SeqAccs.
Both the date and the first name are transcribed with high accuracy, with a a pro- cases where no last name were stated. This unfortunately occurs quite frequently since the census records are sorted by families and a notation for "same as above" are quite frequently used for the last name. Even so, the projected accuracy is 87.46% for the last name.

Discussion and Conclusion
This paper introduces the DARE database and the DARE system. The DARE database is the largest publicly available database of handwritten dates and consists of almost 10 million tokens from 3.1 million images with more than 2.2 million handwritten dates. We advocate that larger and more challenging publicly available datasets, together with the development of better and more efficient architectures for HTR models, will aid in overcoming the challenge of HTR for historical documents.
Using this novel database, we train a number of neural networks able to transcribe dates -the DARE system. Similar to other authors [Puigcerver, 2017, Kang et al., 2022 we do not use recurrent architectures and adopt a computationally more efficient model which is still able to obtain high accuracy. To illustrate another application for the DARE system we undertook the task of transcribing the name and date fields of the 1916 Danish Census with more than 3.7M observations in Section 6. Even though some of this data were already labeled, they were missing links to their image counterparts and thus no (image, label)-pairs were available for training. To facilitate the transcription of the census, we first use the DARE system in a zero-shot way to transcribe dates in combination with name transcription models from Dahl et al. [2022a] in order to bootstrap a linking process between the labelled data and the images. Directly using the predictions of our networks, we were able to link a small subset of the labels and images. We used this to start an iterative process of gradually training better models and linking more images and labels; using transfer learning from the DARE system. Ultimately we were able to train models that can transcribe the name and date fields of the 1916 Danish Census (excluding Copenhagen) in its entirety. 13 This illustrates that the DARE system is not only useful for transfer learning: It also has direct zero-shot use and it can be used to obtain vital information to link individuals across, for example, censuses.
For future work, there are several dimensions along which the DARE system could be extended and improved. First, the DARE database could potentially benefit from including more datasets of handwritten dates. Also the quality of the datasets, i.e., the image snippets, could potentially be improved. For example, the segmentation process (producing image snippets) rely on the tabulated forms to be consistent and while this method is largely successful, it is not without error, and improving the segmentation of the fields would in all likelihood lead to a better database.
Finally, the models of the DARE system should be maintained and updated over time. Whenever the DARE database is extended, re-training models on the larger set of datasets will be beneficial. In addition, over time new architectures and new optimization strategies will be emerging. They must be evaluated and potentially (eventually) replace the existing models and strategies.

A Details on Data Sources
In this section, we first describe the construction of the DARE database, including the data sources used and how we extract the dates from the source images. We then introduce two new datasets, the first dataset (Swedish grade sheets) is used directly in our transfer learning example and the second (1916 Danish Census) we use in combination with our linking method to match the images to a small labelled subset of the census. Finally, we provide additional visualizations of the datasets used to construct the DARE database.

A.1 Datasets of the DARE Database
The DARE database consists of eight datasets. We use Danish death certificates, Danish police records, Danish funeral records, Danish nurse home visit documents, and Swedish causes of death records.

A.2 Image Segmentation
To segment the data (i.e., extract the parts of the scanned pages that contain dates), we use point set registration. Point set registration refers to the problem of aligning point spaces across a template image to an input image [Besl and McKay, 1992].
To find point spaces that roughly correspond to each other across semi-structured documents, we extract horizontal and vertical lines from the documents.
We use the intersections as the point space, which we align with the template points. This method is identical to the one described in Dahl et al. [2021].

A.3 Out-of-Distribution: Swedish Grade Sheets
To illustrate how the DARE system can be used for transfer learning, we use data

A.4 Out-of-Distribution: Danish Census Data
We use data only from the 1916 Danish Census (excluding Copenhagen) to illustrate the performance of the DARE system for matching and linking purposes. Figure 7 shows an example of the Danish census data. Aside from illustrating the performance

B Model Parameters
This section introduces the parameters of the DARE system. We use two different parameter settings with a varying number of epochs. Since there is a large difference in the sizes of the different datasets, some required more epochs for training. Model setting (2) is used in most of our applications, including the larger M-DDMYY model which is used in our transfer learning experiments.