Simple Natural Language Processing Tools for Danish
Leon Derczynski


Magnetic Cataclysmic Variables discovered in hard X-rays

    
M. Falanga1,
D. de Martino2,
F. Bernardini3,2,
K. Mukai4,5
          
    Received: date / Revised version: date
==========================================================================




This technical note describes a set of baseline tools for automatic processing of Danish text. 
The tools are machine-learning based, using natural language processing models trained over previously annotated documents.
They are maintained at ITU Copenhagen and will always be freely available.









§ INTRODUCTION

Danish, a language with few resources for automatic processing, is spoken by six million people, largely concentrated in the Scandinavian country of Denmark.
At the NLP (natural language processing) group at ITU Copenhagen, one of our foci is Nordic and Danish languages.
Thus, we aim to include local languages in our NLP research whenever possible, and to provide tools for others working on these languages.
Despite a modest number of researcher in the country, there has been a gap in usable Danish NLP tools that are available to the wider community.
As in every time that new technology is not effectively mediated to those who can use it, this restriction stymies the impact and development NLP in Denmark.
To bridge this artificial lacuna, a basic set of easy-to-run utilities that build on existing datasets and open tools has been developed at ITU, which this note introduces.


The tools are available at <https://nlp.itu.dk/resources/>.



§ DANER: NAMED ENTITY RECOGNITION


Named entity recognition means finding words in text that refer to things by a specific name.
This could be e.g. using “Nanna" to refer to a specific individual, instead of “them" or “she" or “the person"; or referring to “Moscow" instead of “the nearest city".
The important thing is that one mentions a name specifically.
Named entity recognition (NER) finds those names. In the case of daner, names of Locations, Organisation and Persons are tagged.

The daner tool wraps the CoreNLP <cit.> named entity recognition component <cit.>, using the DKIE data <cit.> developed as part of the GATE tool <cit.>, which is derived from the UD part of the Copenhagen Dependency Treebank <cit.>, itself including data from the Danish Dependency Treebank <cit.>.
Further ad-hoc data is added as required, from newswire and other sources.
The data overall comprises 20K tokens from the CDT (largely 1990s newswire), 3K tokens from non-newswire Danish, and 10K tokens from post-2015 newswire.
The mode performs best over Danish newswire text.
The data is annotated for three classes, person (PER), location (LOC), and organisation (ORG).
Tags are in BIO format.

The tool recognises names of people, of organisations, and of locations.
It performs automatic tokenization, and outputs in slashed-tag format.
For example:


    En/O stor/O reform/O skal/O derfor/O blandt/O andet/O styrke/O tilliden/O til/O politikere/O og/O medier/O ,/O genopbygge/O tilliden/O til/O Skat/O og/O mindske/O de/O økonomiske/O forskelle/O i/O Danmark/B-LOC ./O 


Here, “Danmark" is tagged as a location. The B indicates that “Danmark" is the first token in the location name (the beginning ).

Additional data is added to an internal repository that is also used to train daner; some of this data has copyright restrictions, and so cannot be distributed, though the aggregated statistical representation in daner's model is free from that restriction, and so can be downloaded by anyone.

The NER model is configured to use contextual features, including n-grams up to 5-grams, and to use type sequences and word shape features.

daner uses Brown clusters induced over large Danish-language corpus. These are created using the Generalised Brown formulation <cit.>, which offers more flexibility in use than the classical version – specifically, one clustering run can be used to generate clusterings of any size. For daner, 2500 clusters[A size that works well for NER in English <cit.>] are induced from a clustering with window size of 5000, over a set of 134M Danish tokens. 
These 134M tokens are taken from Danish Wikipedia and Danish content from the Common Crawl corpus filtered through the FastText language ID <cit.>, both using June 2019 data.
The clusters are used inside the model, but also available separately at <http://itu.dk/ leod/dansk-brown.tar.bz2>.





§ DAPIPE: TOKENIZATION, PART-OF-SPEECH TAGGING, AND DEPENDENCY PARSING


Another utility in the toolkit, dapipe, provides three functions: breaking text files into words and sentences, finding what class each word is through PoS tagging, and determining one part of the syntax that links a sentence's words together, through dependency parsing.

Defining where words and sentences start and stop is very important if we are going to do NLP.
These processes are both called tokenization.
dapipe uses the Universal dependencies method for tokenizing, breaking text into sentences and words.

The next step is to give a “part of speech" to each word.
This would be something like a noun (substantiv), verb (verbum), or preposition (præposition).
This tells us the function each particular word has.
dapipe works generally well, though for social media text, I recommend structbilty,[See <https://github.com/bplank/bilstm-aux>] which is more resilient <cit.> to the unusual words and noise found in this colloquial setting <cit.>.

Once the words and their classes are known, one may continue by parsing the words in the sentence.
The dapipe tool uses the Universal Dependencies schema[See <https://universaldependencies.org/introduction.html>] for this, finding the main grammatical relations between each word to construct a dependency tree that links every word in the sentence together.

All these tasks are achieved by using UDPipe <cit.> based on the universal dependencies <cit.> data for Danish.
Sample output follows.






§ SUMMARY


This note describes a free toolkit for basic natural language processing. 
The two tools daner and dapipe are built on freely-available resources and maintaned at ITU Copenhagen.

Feedback, requests, and so on can be addressed to the ITU NLP research group: <http://nlp.itu.dk/>

unsrt  
  


