Fresh start without large files

.gitignore

@@ -0,0 +1,15 @@
+model_0.bin
+model_1.bin
+pytorch_model.bin
+model.bin
+split.ipynb
+decode.ipynb
+upload.py
+results/
+vocab.json
+model_en.bin
+model_zh.bin
+__pycache__
+transfer.py
+word_frequency.json
+word_frequency_en.json

README.MD

@@ -0,0 +1,163 @@
+# BriLLM: Brain-inspired Large Language Model
+
+We release BriLLM-Chinese and BriLLM-English.
+
+Our paper: https://arxiv.org/pdf/2503.11299
+
+Our huggingface: https://huggingface.co/BriLLM/BriLLM0.5
+
+
+## Overview
+This work introduces the first brain-inspired large language model (BriLLM). This is a non-Transformer, non-GPT, non-traditional machine learning input-output controlled generative language model. The model is based on the Signal Fully-connected flowing (SiFu) definition on the directed graph in terms of the neural network, and has the interpretability of all nodes on the graph of the whole model, instead of the traditional machine learning model that only has limited interpretability at the input and output ends. 
+
+
+## SiFU Mechanism
+![](./figs/fig1.png)
+> As shown in Figure 1, SiFu model is a graph composed of multiple nodes, which are sparsely activated and utilize tensors to transmit a nominal signal.
+Each node (ideally, a layer of neurons) represents a certain concept or word, e.g., a noun, a verb, etc.
+Each edge models the relationship between every node pair.
+The signal is transmitted by the magnitude of the energy. The energy will be strengthened, i.e., maximized, if it is in the right route. Or, at least, the right path always keeps the maximal energy for the transmitted signal.
+Each node is sequentially activated in terms of the maximized energy.
+Route or path is determined in a competitive way, i.e., the next node will be activated only if the energy can be maximally delivered in this node.
+
+
+## Architecture
+![](./figs/fig2.png)
+> As shown in Figure 2, BriLLM implements SiFu neural network for language modeling. 
+Each token in the vocabulary is modeled as a node, which is defined by a hidden layer of neurons in the neural network.
+
+
+## Training Network
+![](./figs/fig3.png)
+> To train a sample in BriLLM, every time we build an individual common neural network to perform the regular BP training. This network consists of two parts, in which the front part connects all input nodes (i.e., tokens), then it follows the rear parts which connect all possible paths in order. At last, a softmax layer collects all paths' energy tensors to indicate the right path with a 0-1 ground truth vector. We adopt a cross-entropy loss for training.
+
+
+## Dataset
+We use the subset from the Chinese version of Wikipedia, which contains over 100M Chinese characters. We truncate the long sentences into small sentences with a maximum length of 16.
+We select a vocabulary of 4,000 tokens consisting of the most frequently used Chinese characters.
+
+
+## Implementation Details.
+BriLLM is implemented using PyTorch. 
+It uses sinusoidal positional encoding, GeLU as the activation function, cross-entropy loss for next-token prediction, and an embedding size of $d_{model} = 32$. 
+We used the AdamW optimizer with $\beta_1 = 0.9$, $\beta_2 = 0.999$ and $\epsilon = 10^{-8}$. 
+The model size is about $512 + 4000 * 4000 * (32 * 32 + 32) \approx 16B$.
+We trained our models on one machine with 8 NVIDIA A800 GPUs for 1.5k steps. 
+![](./figs/fig4.png)
+
+
+## Complexity
+$n$ is the sequence length, $v$ is the vocabulary size, and $d$ is the representation dimension. The computational complexity is $O(n \cdot v \cdot d^2)$.
+
+
+## Case Study
+![](./figs/fig5.png)
+![](./figs/fig7.png)
+
+
+## Comparison of LLM and BriLLM
+![](./figs/fig6.png)
+
+
+## Installation
+```bash
+pip install torch
+```
+
+
+## Checkpoint
+[BriLLM0.5](https://huggingface.co/BriLLM/BriLLM0.5)
+
+
+## Train
+### BriLLM-Chinese
+```bash
+bash run_zh.sh
+```
+
+### BriLLM-English
+```bash
+bash run_en.sh
+```
+
+
+
+## Inference
+### BriLLM-Chinese
+```python
+import json
+import torch
+from model import BraLM, Vocab
+
+with open("vocab_wiki_4k.json") as f:
+     node_dict = json.load(f)
+vocab = Vocab.from_node_dict(node_dict)
+
+with open('word_frequency.json', 'r') as f:
+    freq_dict = json.load(f)
+
+zero_freq_edges = {}
+for s in freq_dict:
+    zero_freq_edges[s] = []
+    for t in freq_dict[s]:
+        if freq_dict[s][t] == 0:
+            zero_freq_edges[s].append(t)
+
+model = BraLM(hidden_size=32, zero_freq_edges=zero_freq_edges, vocab=vocab)
+model.prepare_network(vocab)
+
+state_dict = torch.load("model_zh.bin", weights_only=True)
+model.load_state_dict(state_dict)
+model.to_device("cuda:6")
+
+head = "《罗马》描述了"
+max_token = 32 - len(head)
+
+start = [vocab((head[i]+ '->' +head[i+1])) for i in range(len(head)-1)]
+ret = model.decode(start, vocab, max_token)
+decode_tuple_list = [vocab.decode(p) for p in ret]
+decode_sentence = decode_tuple_list[0][0] + "".join([p[-1] for p in decode_tuple_list])
+
+print(decode_sentence)
+```
+
+
+### BriLLM-English
+```python
+import json
+import torch
+from model import BraLM, Vocab
+from tokenizers import Tokenizer
+
+bpe_tokenizer = Tokenizer.from_file("wiki_bpe_tokenizer_4000_bytelevel.json")
+
+def decode_en_sentence(head, max_token=32, do_sample=False):
+    bpe_tokens = bpe_tokenizer.encode(head).tokens
+    if len(bpe_tokens) < 2:
+        return head
+    start = [vocab((bpe_tokens[i] + '->' + bpe_tokens[i+1])) for i in range(len(bpe_tokens)-1)]
+    ret = model.decode(start, vocab, max_token, do_sample)
+    decode_tuple_list = [vocab.decode(p).split('->') for p in ret]
+    decode_sentence = decode_tuple_list[0][0] + "".join([p[-1] for p in decode_tuple_list])
+    return decode_sentence
+
+
+with open("./vocab_wiki_4k_en.json") as f:
+     node_dict = json.load(f)
+vocab = Vocab.from_node_dict(node_dict)
+
+model = BraLM(hidden_size=32)
+model.prepare_network(vocab)
+
+state_dict = torch.load("model_en.bin", weights_only=True)
+model.load_state_dict(state_dict)
+model.to_device("cuda:6")
+
+head = "In frogs, the hind legs are larger"
+encoding = bpe_tokenizer.encode(head)
+token_len = len(encoding.ids)
+max_token = 32 - token_len
+decode_sentence = decode_en_sentence(head, max_token).replace("Ġ", " ")
+
+print(decode_sentence)
+```

brain.yaml

@@ -0,0 +1,11 @@
+compute_environment: LOCAL_MACHINE
+deepspeed_config: {}
+distributed_type: MULTI_GPU
+fsdp_config: {}
+machine_rank: 0
+main_process_ip: 192.168.0.0
+main_process_port: 29500
+main_training_function: main
+num_machines: 1
+num_processes: 8
+use_cpu: false

infer_en.py

@@ -0,0 +1,36 @@
+import json
+import torch
+from model import BraLM, Vocab
+from tokenizers import Tokenizer
+
+bpe_tokenizer = Tokenizer.from_file("wiki_bpe_tokenizer_4000_bytelevel.json")
+
+def decode_en_sentence(head, max_token=32, do_sample=False):
+    bpe_tokens = bpe_tokenizer.encode(head).tokens
+    if len(bpe_tokens) < 2:
+        return head
+    start = [vocab((bpe_tokens[i] + '->' + bpe_tokens[i+1])) for i in range(len(bpe_tokens)-1)]
+    ret = model.decode(start, vocab, max_token, do_sample)
+    decode_tuple_list = [vocab.decode(p).split('->') for p in ret]
+    decode_sentence = decode_tuple_list[0][0] + "".join([p[-1] for p in decode_tuple_list])
+    return decode_sentence
+
+
+with open("./vocab_wiki_4k_en.json") as f:
+     node_dict = json.load(f)
+vocab = Vocab.from_node_dict(node_dict)
+
+model = BraLM(hidden_size=32)
+model.prepare_network(vocab)
+
+state_dict = torch.load("model_en.bin", weights_only=True)
+model.load_state_dict(state_dict)
+model.to_device("cuda:6")
+
+head = "In frogs, the hind legs are larger"
+encoding = bpe_tokenizer.encode(head)
+token_len = len(encoding.ids)
+max_token = 32 - token_len
+decode_sentence = decode_en_sentence(head, max_token).replace("Ġ", " ")
+
+print(decode_sentence)

infer_zh.py

@@ -0,0 +1,34 @@
+import json
+import torch
+from model import BraLM, Vocab
+
+with open("vocab_wiki_4k.json") as f:
+     node_dict = json.load(f)
+vocab = Vocab.from_node_dict(node_dict)
+
+with open('word_frequency.json', 'r') as f:
+    freq_dict = json.load(f)
+
+zero_freq_edges = {}
+for s in freq_dict:
+    zero_freq_edges[s] = []
+    for t in freq_dict[s]:
+        if freq_dict[s][t] == 0:
+            zero_freq_edges[s].append(t)
+
+model = BraLM(hidden_size=32, zero_freq_edges=zero_freq_edges, vocab=vocab)
+model.prepare_network(vocab)
+
+state_dict = torch.load("model_zh.bin", weights_only=True)
+model.load_state_dict(state_dict)
+model.to_device("cuda:6")
+
+head = "《罗马》描述了"
+max_token = 32 - len(head)
+
+start = [vocab((head[i]+ '->' +head[i+1])) for i in range(len(head)-1)]
+ret = model.decode(start, vocab, max_token)
+decode_tuple_list = [vocab.decode(p) for p in ret]
+decode_sentence = decode_tuple_list[0][0] + "".join([p[-1] for p in decode_tuple_list])
+
+print(decode_sentence)

model.py

@@ -0,0 +1,305 @@
+import torch
+import torch.nn as nn
+import random
+from torch.autograd import Variable
+
+class BraLM(nn.Module):
+    def __init__(self, hidden_size, use_ds=False, zero_freq_edges=None, vocab=None):
+        super().__init__()
+        self.hidden_size = hidden_size
+        self.activation = nn.GELU()
+        self.positions = nn.Parameter(torch.ones(1, 512, 1))
+        self.device = None
+        
+        # for fsdp
+        self._tied_weights_keys = []
+
+        self.use_ds = use_ds
+        self.zero_freq_edges = zero_freq_edges
+        self.vocab = vocab
+
+    def prepare_network(self, vocab):
+        # Create index mappings for the flattened structure
+        self.weight_indices = {}  # Maps (s_idx, t_idx) to parameter index
+        self.shared_param_idx = 0
+        
+        # Current index for new parameters
+        current_idx = 1
+        
+        # Populate parameters and mappings
+        for s_idx, s in enumerate(vocab.edge_dict):
+            for t_idx, t in enumerate(vocab.edge_dict[s]):
+                if self.zero_freq_edges is not None and t in self.zero_freq_edges[s]:
+                    # Use shared parameters
+                    self.weight_indices[(s_idx, t_idx)] = self.shared_param_idx
+                else:
+                    self.weight_indices[(s_idx, t_idx)] = current_idx
+                    current_idx += 1
+
+        # Create new parameters
+        self.weights = nn.Parameter(torch.randn(current_idx, self.hidden_size, self.hidden_size).uniform_(-0.5, 0.5))
+        self.biases = nn.Parameter(torch.randn(current_idx, 1, self.hidden_size).uniform_(-0.5, 0.5))
+
+        self.node_bias = nn.Parameter(torch.randn(len(vocab.edge_dict), 1, self.hidden_size).uniform_(-0.5, 0.5))
+
+    def to_device(self, device):
+        self.weights.to(device)
+        self.biases.to(device)
+        self.node_bias.to(device)
+        self.positions.data = self.positions.data.to(device)
+        self.device = device
+
+    @staticmethod
+    def _reshape12(x):
+        return x.reshape(-1, x.size(-2), x.size(-1))
+    
+    def get_positional_encoding(self, seq_len, d_model):
+        position = torch.arange(0, seq_len).reshape(-1, 1)
+        div_term = 10000.0 ** (torch.arange(0, d_model, 2) / d_model)
+        position_encoding = torch.zeros(seq_len, d_model)
+        position_encoding[:, 0::2] = torch.sin(position * div_term)
+        position_encoding[:, 1::2] = torch.cos(position * div_term)
+        return position_encoding.unsqueeze(0).to(self.device)
+    
+    def get_initial_tensor(self, batch_size, d, pe):
+        # initialize energy_tensor
+        energy_tensor = torch.ones(batch_size, 1, self.hidden_size) / self.hidden_size   #(bs, 1, hs)
+        energy_tensor = energy_tensor.to(self.device)
+        
+        # Ensure d is on the same device as node_bias
+        d = d.to(self.device)
+        node_bias = self.node_bias[d[:, 0, 0]]
+        energy_tensor = self.activation(energy_tensor + node_bias + pe[:,0])
+        return energy_tensor
+
+
+    def forward(self, neighbor_ids):
+        # neighbor_ids: (bs, sen_len, 1+k, 2) ; k is the number of negative samples
+        batch_size = neighbor_ids.size(0)
+        loss = 0
+
+        pe = self.get_positional_encoding(512, self.hidden_size)  #(1, 512, hs)
+
+        for i in range(neighbor_ids.size(1)):
+            d = neighbor_ids[:, i]  #(bs, 1+k, 2)
+            
+            if i == 0:
+                # for the first token, initialize energy_tensor as an all-one tensor
+                energy_tensor = self.get_initial_tensor(batch_size, d, pe)  #(bs, 1, hs) 
+            else:
+                energy_tensor = (energy_cache * self.positions[:, :i, :].softmax(1)).sum(1, keepdim=True)   #(bs, 1, hs) :fix dim bug
+
+            # Vectorized parameter lookup
+            src_idx = d[..., 0]  # (bs, 1+k)
+            tgt_idx = d[..., 1]  # (bs, 1+k)
+            param_indices = torch.tensor([self.weight_indices.get((s.item(), t.item()), self.shared_param_idx) 
+                                        for s, t in zip(src_idx.reshape(-1), tgt_idx.reshape(-1))], 
+                                       device=self.device).reshape(batch_size, -1)  # (bs, 1+k)
+            
+            # Batch gather operation
+            w = self.weights[param_indices]  # (bs, 1+k, hidden_size, hidden_size)
+            b = self.biases[param_indices]   # (bs, 1+k, 1, hidden_size)
+
+            expand_energy_tensor = self._reshape12(energy_tensor.unsqueeze(1).repeat(1, w.size(1), 1, 1))  #(bs*(1+k), 1, hs)
+            # for deepspeed fp16: expand_energy_tensor.half()
+            if self.use_ds:
+                expand_energy_tensor = expand_energy_tensor.half()
+            nxt_energy_tensor = self.activation(expand_energy_tensor.bmm(self._reshape12(w))+self._reshape12(b)+Variable(pe[:,i+1], requires_grad=False))  #(bs*(1+k), 1, hs)
+            output_tensor = nxt_energy_tensor.reshape(batch_size, -1, nxt_energy_tensor.size(-2), nxt_energy_tensor.size(-1))  #(bs, 1+k, 1, hs)
+
+            if i == 0:
+                energy_cache = output_tensor[:,0]   #(bs, 1, hs)
+            else:
+                energy_cache = torch.cat([energy_cache, output_tensor[:,0]], dim=1)  #(bs, i+1, hs)
+
+            if 1:
+                energy = output_tensor.norm(2, (-2, -1))
+                label = torch.LongTensor([0 for _ in range(batch_size)]).to(self.device)
+                loss += nn.CrossEntropyLoss()(energy, label)
+
+        return loss / neighbor_ids.size(1)
+
+    def decode(self, start, vocab, max_new_tokens=16, do_sample=False, temperature=1):
+        ret = []
+        pe = self.get_positional_encoding(512, self.hidden_size)
+        
+        for i, pair in enumerate(start):
+            if i == 0:
+                energy_tensor = self.get_initial_tensor(batch_size=1, d=torch.tensor([[pair]], device=self.device), pe=pe).squeeze(0)
+            else:
+                energy_tensor = (energy_cache * self.positions[:, :i, :].softmax(1)).sum(1, keepdim=True).squeeze(0)
+            
+            # Get parameter index for this edge
+            param_idx = self.weight_indices.get((pair[0], pair[1]), self.shared_param_idx)
+            
+            # Get weights and biases using parameter index
+            w = self.weights[param_idx].to(self.device)
+            b = self.biases[param_idx].to(self.device)
+
+            energy_tensor = self.activation(energy_tensor.mm(w) + b + pe.squeeze(0)[i])
+            if i == 0:
+                energy_cache = energy_tensor.unsqueeze(0)  # Add batch dimension
+            else:
+                energy_cache = torch.cat([energy_cache, energy_tensor.unsqueeze(0)], dim=1)
+            ret += [pair]
+        
+        x = pair[1]
+        prev_i = len(start)
+
+        for i in range(max_new_tokens):
+            candidates = vocab(vocab.get_neighbor_of_node(x, -1))
+            
+            # Get parameter indices for all candidates
+            param_indices = torch.tensor([self.weight_indices.get((x, t[1]), self.shared_param_idx) 
+                                        for t in candidates], device=self.device)
+            
+            # Get weights and biases for all candidates
+            all_w = self.weights[param_indices].to(self.device)
+            all_b = self.biases[param_indices].to(self.device)
+
+            curr_i = prev_i + i
+            energy_tensor = (energy_cache * self.positions[:, :curr_i, :].softmax(1)).sum(1, keepdim=True)
+            expand_energy_tensor = energy_tensor.unsqueeze(1).repeat(1, all_w.size(0), 1, 1)
+            expand_energy_tensor = self._reshape12(expand_energy_tensor)
+
+            nxt_energy_tensor = self.activation(expand_energy_tensor.bmm(self._reshape12(all_w)) + self._reshape12(all_b) + pe[:,curr_i].unsqueeze(0))
+            output_tensor = nxt_energy_tensor.reshape(1, -1, nxt_energy_tensor.size(-2), nxt_energy_tensor.size(-1))
+
+            energy = output_tensor.norm(2, (-2,-1)).squeeze()
+
+            probs = torch.softmax(energy, dim=-1)
+            if temperature > 0:
+                probs = probs / temperature
+            if do_sample:
+                index = torch.multinomial(probs, 1).item()
+            else:
+                index = probs.argmax(-1).item()
+
+            y = candidates[index][-1]
+            ret += [(x, y)]
+
+            energy_tensor = output_tensor[0, index]
+            x = y
+
+            energy_cache = torch.cat([energy_cache, energy_tensor.unsqueeze(0)], dim=1)
+
+        return ret
+
+
+class Vocab:
+    def __init__(self, node_dict, nodeindex_dict, edge_dict, edge_decode_dict):
+        self.node_dict = node_dict              #{'node_p': index_p}    ----    size: num_nodes
+        self.nodeindex_dict = nodeindex_dict    #{index_p: 'node_p'}    ----    size: num_nodes
+        self.edge_dict = edge_dict              #{'node_p': {'node_q': (index_p, index_q), 'node_m': (index_p, index_m)},...}    ----    size: num_nodes
+        self.edge_decode_dict = edge_decode_dict    #{(index_p, index_q): 'node_p->node_q'}    ----    size: num_nodes*num_nodes
+
+    def __call__(self, x):
+        if isinstance(x, list):
+            return [self.__call__(_) for _ in x]
+        else:
+            return self.fetch(x)
+
+    def fetch(self, x):
+        s, t = x.split("->")
+        return self.edge_dict[s][t] if s in self.edge_dict and t in self.edge_dict[s] else self.edge_dict[""][""]
+
+    @classmethod
+    def from_node_dict(cls, dictname):
+        node_dict = dict()
+        nodeindex_dict = dict()
+        edge_dict = dict()
+        edge_decode_dict = dict()
+        for s in dictname:
+            node_dict[s] = dictname[s]
+            nodeindex_dict[dictname[s]] = s # nodeindex_dict: {index_p: 'node_p'}
+            edge_dict[s] = {} # edge_dict: {'node_p': {'node_q': (index_p, index_q), 'node_m': (index_p, index_m)}}
+            for t in dictname:
+                edge_dict[s][t] = (dictname[s], dictname[t])
+                edge_decode_dict[(dictname[s], dictname[t])] = "->".join([s, t])
+        return cls(node_dict, nodeindex_dict, edge_dict, edge_decode_dict)
+
+    @classmethod
+    def from_edge(cls, filename):
+        edge_dict = dict()
+        edge_dict[""] = {}
+        edge_dict[""][""] = (0, 0)
+        edge_decode_dict = dict()
+        with open(filename) as f:
+            for line in f:
+                # line: node_p->node_q
+                s, t = line.strip().split("->")
+                if s not in edge_dict:
+                    i = len(edge_dict)
+                    j = 0
+                    edge_dict[s] = dict()
+                else:
+                    i = edge_dict[s][list(edge_dict[s].keys())[0]][0]
+                    j = len(edge_dict[s])
+                edge_dict[s][t] = (i, j)
+                edge_decode_dict[(i, j)] = "->".join([s, t])
+        return cls(None, edge_dict, edge_decode_dict)
+
+    def get_neighbor_of_edge(self, key, k, frequency_dict=None):
+        s, t = key.split("->") # s, t: node
+        _s = s if s in self.edge_dict else ""
+        
+        # if s in self.edge_dict:
+        #     ret = ["->".join([s, _t]) for _t in self.edge_dict[s].keys() if _t != t]
+        # else:
+        #     ret = ["->".join([s, _t]) for _t in self.edge_dict[""].keys() if _t != t]
+        # ret = ["->".join([s, _t]) for _t in self.edge_dict[s].keys() if _t != t]
+        
+        # select by word_frequency
+        if frequency_dict:
+            frequency_lst = list(frequency_dict[_s].keys())
+            # index = frequency_lst.index(t)
+            # half = k // 2
+            # if index <= k:
+            #     t_lst = [x for i, x in enumerate(frequency_lst[:k+1]) if i != index]
+            # else:
+            #     t_lst = frequency_lst[:half] + frequency_lst[index-half:index]
+            t_lst = [x for i, x in enumerate(frequency_lst[:k+1]) if x != t][:k]
+            ret = ["->".join([_s, _t]) for _t in t_lst]
+            random.shuffle(ret)
+            return ret
+        # randomly select k negative samples
+        else:
+            ret = ["->".join([_s, _t]) for _t in self.edge_dict[_s].keys() if _t != t]
+            random.shuffle(ret)
+            return ret[:k] if k != -1 else ret
+
+    def get_neighbor_of_node(self, key, k):
+        #key :index
+        s = self.nodeindex_dict[key] #node
+        #_t: node
+        ret = ["->".join([s, _t]) for _t in self.edge_dict[s].keys() if _t != s]
+        
+        # randomly select k negative samples
+        random.shuffle(ret) 
+        return ret[:k] if k != -1 else ret
+    
+    def get_neighbor_of_edge_broadcast(self, key, edges, k=100):
+        s, t = key.split("->")
+        _ret = [_t for _t in self.edge_dict[s].keys() if _t != t] # all neighbors of s except t
+        random.shuffle(_ret)
+        ret = []
+        for edge in edges:
+            s, t = edge.split("->")
+            ret += [["->".join([s, _t]) for _t in _ret[:k]]] 
+        return ret
+
+    @staticmethod
+    def to_path(tokens):
+        path = []
+        for left, right in zip(tokens[:-1], tokens[1:]):
+            path.append("->".join([left, right]))
+        return path
+
+    def get_edge_of_node(self, key):
+        return list(self.edge_dict[key].values())
+
+    def decode(self, x):
+        return self.edge_decode_dict[x]
+    
+
+    

run_en.sh

@@ -0,0 +1,31 @@
+set -xe
+
+BS=32
+LR=5e-2
+HS=32
+LEN=32
+INITIAL_F=0
+END_F=0
+EPOCH=25
+
+accelerate launch --config_file brain.yaml train.py \
+  --data_dir data \
+  --do_train \
+  --output_dir en_checkpoints_len${LEN}_sparse \
+  --hidden_size $HS \
+  --train_batch_size $BS \
+  --max_seq_length $LEN \
+  --learning_rate $LR \
+  --num_train_epochs $EPOCH \
+  --num_neg_samples 400 \
+  --initial_file_number $INITIAL_F \
+  --end_file_number $END_F \
+  --num_workers 8 \
+  --fp16 \
+  --run_name "BS${BS}_LR${LR}_HS${HS}_LEN${LEN}_f${INITIAL_F}_EPOCH${EPOCH}" \
+  --vocab_path vocab_wiki_4k_en.json \
+  --train_full data \
+  --sparse \
+  --use_frequency \
+  --use_bpe \
+  --bpe_tokenizer wiki_bpe_tokenizer_4000_bytelevel.json

run_zh.sh

@@ -0,0 +1,29 @@
+set -xe
+
+BS=32
+LR=5e-2
+HS=32
+LEN=32
+INITIAL_F=0
+END_F=0
+EPOCH=25
+
+accelerate launch --config_file brain.yaml train.py \
+  --data_dir data \
+  --do_train \
+  --output_dir zh_checkpoints_len${LEN}_sparse \
+  --hidden_size $HS \
+  --train_batch_size $BS \
+  --max_seq_length $LEN \
+  --learning_rate $LR \
+  --num_train_epochs $EPOCH \
+  --num_neg_samples 400 \
+  --initial_file_number $INITIAL_F \
+  --end_file_number $END_F \
+  --num_workers 8 \
+  --fp16 \
+  --run_name "BS${BS}_LR${LR}_HS${HS}_LEN${LEN}_f${INITIAL_F}_EPOCH${EPOCH}" \
+  --vocab_path vocab_wiki_4k.json \
+  --train_full data \
+  --sparse \
+  --use_frequency 

train.py

@@ -0,0 +1,852 @@
+import argparse
+import logging
+import os
+import random
+import math
+import copy
+import json
+import numpy as np
+import torch
+import torch.nn as nn
+import glob
+from tqdm.auto import tqdm, trange
+from torch.autograd import Variable
+from accelerate import Accelerator, DistributedDataParallelKwargs
+from accelerate.utils import InitProcessGroupKwargs
+from torch.utils.data import IterableDataset, DataLoader, Dataset
+import time
+import torch.distributed as dist
+import gc
+from datetime import timedelta
+from tokenizers import Tokenizer
+
+import wandb
+
+os.environ["WANDB_WATCH"] = "false"
+
+
+class BraLM(nn.Module):
+    def __init__(self, hidden_size, use_ds=False, zero_freq_edges=None, vocab=None):
+        super().__init__()
+        self.hidden_size = hidden_size
+        self.activation = nn.GELU()
+        self.positions = nn.Parameter(torch.ones(1, 512, 1))
+        self.device = None
+        
+        # for fsdp
+        self._tied_weights_keys = []
+
+        self.use_ds = use_ds
+        self.zero_freq_edges = zero_freq_edges
+        self.vocab = vocab
+
+    def prepare_network(self, vocab):
+        # Create index mappings for the flattened structure
+        self.weight_indices = {}  # Maps (s_idx, t_idx) to parameter index
+        self.shared_param_idx = 0
+        
+        # Current index for new parameters
+        current_idx = 1
+        
+        # Populate parameters and mappings
+        for s_idx, s in enumerate(vocab.edge_dict):
+            for t_idx, t in enumerate(vocab.edge_dict[s]):
+                if self.zero_freq_edges is not None and t in self.zero_freq_edges[s]:
+                    # Use shared parameters
+                    self.weight_indices[(s_idx, t_idx)] = self.shared_param_idx
+                else:
+                    self.weight_indices[(s_idx, t_idx)] = current_idx
+                    current_idx += 1
+
+        # Create new parameters
+        self.weights = nn.Parameter(torch.randn(current_idx, self.hidden_size, self.hidden_size).uniform_(-0.5, 0.5))
+        self.biases = nn.Parameter(torch.randn(current_idx, 1, self.hidden_size).uniform_(-0.5, 0.5))
+
+        self.node_bias = nn.Parameter(torch.randn(len(vocab.edge_dict), 1, self.hidden_size).uniform_(-0.5, 0.5))
+
+    def to_device(self, device):
+        self.weights.to(device)
+        self.biases.to(device)
+        self.positions.data = self.positions.data.to(device)
+        self.device = device
+
+    @staticmethod
+    def _reshape12(x):
+        return x.reshape(-1, x.size(-2), x.size(-1))
+    
+    def get_positional_encoding(self, seq_len, d_model):
+        position = torch.arange(0, seq_len).reshape(-1, 1)
+        div_term = 10000.0 ** (torch.arange(0, d_model, 2) / d_model)
+        position_encoding = torch.zeros(seq_len, d_model)
+        position_encoding[:, 0::2] = torch.sin(position * div_term)
+        position_encoding[:, 1::2] = torch.cos(position * div_term)
+        return position_encoding.unsqueeze(0).to(self.device)
+
+    # def get_initial_tensor(self, batch_size, max_norm=1.0):
+    #     # initialize energy_tensor
+    #     energy_tensor = torch.zeros(batch_size, 1, self.hidden_size).normal_(0, 1).to(self.device)
+    #     delta_norm = torch.norm(energy_tensor.view(energy_tensor.shape[0], -1), dim=-1, p="fro").detach()
+    #     clip_mask = (delta_norm > max_norm).to(energy_tensor)
+    #     clip_weights = max_norm / delta_norm * clip_mask + (1 - clip_mask)
+    #     energy_tensor = (energy_tensor * clip_weights.view(-1, 1, 1)).detach()    #(bs, 1, hs)
+    #     return energy_tensor
+    
+    def get_initial_tensor(self, batch_size, d, pe):
+        # initialize energy_tensor
+        energy_tensor = torch.ones(batch_size, 1, self.hidden_size) / self.hidden_size   #(bs, 1, hs)
+        energy_tensor = energy_tensor.to(self.device)
+
+        node_bias = self.node_bias[d[:, 0, 0]]
+        energy_tensor = self.activation(energy_tensor + node_bias + Variable(pe[:,0], requires_grad=False))
+        return energy_tensor
+
+
+    def forward(self, neighbor_ids):
+        # neighbor_ids: (bs, sen_len, 1+k, 2) ; k is the number of negative samples
+        batch_size = neighbor_ids.size(0)
+        loss = 0
+
+        pe = self.get_positional_encoding(512, self.hidden_size)  #(1, 512, hs)
+
+        for i in range(neighbor_ids.size(1)):
+            d = neighbor_ids[:, i]  #(bs, 1+k, 2)
+            
+            if i == 0:
+                # for the first token, initialize energy_tensor as an all-one tensor
+                energy_tensor = self.get_initial_tensor(batch_size, d, pe)  #(bs, 1, hs) 
+            else:
+                energy_tensor = (energy_cache * self.positions[:, :i, :].softmax(1)).sum(1, keepdim=True)   #(bs, 1, hs) :fix dim bug
+
+            # Vectorized parameter lookup
+            src_idx = d[..., 0]  # (bs, 1+k)
+            tgt_idx = d[..., 1]  # (bs, 1+k)
+            param_indices = torch.tensor([self.weight_indices.get((s.item(), t.item()), self.shared_param_idx) 
+                                        for s, t in zip(src_idx.reshape(-1), tgt_idx.reshape(-1))], 
+                                       device=self.device).reshape(batch_size, -1)  # (bs, 1+k)
+            
+            # Batch gather operation
+            w = self.weights[param_indices]  # (bs, 1+k, hidden_size, hidden_size)
+            b = self.biases[param_indices]   # (bs, 1+k, 1, hidden_size)
+
+            expand_energy_tensor = self._reshape12(energy_tensor.unsqueeze(1).repeat(1, w.size(1), 1, 1))  #(bs*(1+k), 1, hs)
+            # for deepspeed fp16: expand_energy_tensor.half()
+            if self.use_ds:
+                expand_energy_tensor = expand_energy_tensor.half()
+            nxt_energy_tensor = self.activation(expand_energy_tensor.bmm(self._reshape12(w))+self._reshape12(b)+Variable(pe[:,i+1], requires_grad=False))  #(bs*(1+k), 1, hs)
+            output_tensor = nxt_energy_tensor.reshape(batch_size, -1, nxt_energy_tensor.size(-2), nxt_energy_tensor.size(-1))  #(bs, 1+k, 1, hs)
+
+            if i == 0:
+                energy_cache = output_tensor[:,0]   #(bs, 1, hs)
+            else:
+                energy_cache = torch.cat([energy_cache, output_tensor[:,0]], dim=1)  #(bs, i+1, hs)
+
+            if 1:
+                energy = output_tensor.norm(2, (-2, -1))
+                label = torch.LongTensor([0 for _ in range(batch_size)]).to(self.device)
+                loss += nn.CrossEntropyLoss()(energy, label)
+
+        return loss / neighbor_ids.size(1)
+
+    def decode(self, start, vocab, max_new_tokens=16, do_sample=False, temperature=1):
+        ret = []
+        pe = self.get_positional_encoding(512, self.hidden_size)
+        
+        for i, pair in enumerate(start):
+            if i == 0:
+                energy_tensor = self.get_initial_tensor(batch_size=1, d=torch.tensor([[pair]], device=self.device), pe=pe).squeeze(0)
+            else:
+                energy_tensor = (energy_cache * self.positions[:, :i, :].softmax(1)).sum(1, keepdim=True).squeeze(0)
+            
+            # Get parameter index for this edge
+            param_idx = self.weight_indices.get((pair[0], pair[1]), self.shared_param_idx)
+            
+            # Get weights and biases using parameter index
+            w = self.weights[param_idx].to(self.device)
+            b = self.biases[param_idx].to(self.device)
+
+            energy_tensor = self.activation(energy_tensor.mm(w) + b + pe.squeeze(0)[i])
+            if i == 0:
+                energy_cache = energy_tensor.unsqueeze(0)  # Add batch dimension
+            else:
+                energy_cache = torch.cat([energy_cache, energy_tensor.unsqueeze(0)], dim=1)
+            ret += [pair]
+        
+        x = pair[1]
+        prev_i = len(start)
+
+        for i in range(max_new_tokens):
+            candidates = vocab(vocab.get_neighbor_of_node(x, -1))
+            
+            # Get parameter indices for all candidates
+            param_indices = torch.tensor([self.weight_indices.get((x, t[1]), self.shared_param_idx) 
+                                        for t in candidates], device=self.device)
+            
+            # Get weights and biases for all candidates
+            all_w = self.weights[param_indices].to(self.device)
+            all_b = self.biases[param_indices].to(self.device)
+
+            curr_i = prev_i + i
+            energy_tensor = (energy_cache * self.positions[:, :curr_i, :].softmax(1)).sum(1, keepdim=True)
+            expand_energy_tensor = energy_tensor.unsqueeze(1).repeat(1, all_w.size(0), 1, 1)
+            expand_energy_tensor = self._reshape12(expand_energy_tensor)
+
+            nxt_energy_tensor = self.activation(expand_energy_tensor.bmm(self._reshape12(all_w)) + self._reshape12(all_b) + pe[:,curr_i].unsqueeze(0))
+            output_tensor = nxt_energy_tensor.reshape(1, -1, nxt_energy_tensor.size(-2), nxt_energy_tensor.size(-1))
+
+            energy = output_tensor.norm(2, (-2,-1)).squeeze()
+
+            probs = torch.softmax(energy, dim=-1)
+            if temperature > 0:
+                probs = probs / temperature
+            if do_sample:
+                index = torch.multinomial(probs, 1).item()
+            else:
+                index = probs.argmax(-1).item()
+
+            y = candidates[index][-1]
+            ret += [(x, y)]
+
+            energy_tensor = output_tensor[0, index]
+            x = y
+
+            energy_cache = torch.cat([energy_cache, energy_tensor.unsqueeze(0)], dim=1)
+
+        return ret
+
+
+class Vocab:
+    def __init__(self, node_dict, nodeindex_dict, edge_dict, edge_decode_dict):
+        self.node_dict = node_dict              #{'node_p': index_p}    ----    size: num_nodes
+        self.nodeindex_dict = nodeindex_dict    #{index_p: 'node_p'}    ----    size: num_nodes
+        self.edge_dict = edge_dict              #{'node_p': {'node_q': (index_p, index_q), 'node_m': (index_p, index_m)},...}    ----    size: num_nodes
+        self.edge_decode_dict = edge_decode_dict    #{(index_p, index_q): 'node_p->node_q'}    ----    size: num_nodes*num_nodes
+
+    def __call__(self, x):
+        if isinstance(x, list):
+            return [self.__call__(_) for _ in x]
+        else:
+            return self.fetch(x)
+
+    def fetch(self, x):
+        s, t = x.split("->")
+        return self.edge_dict[s][t] if s in self.edge_dict and t in self.edge_dict[s] else self.edge_dict[""][""]
+
+    @classmethod
+    def from_node_dict(cls, dictname):
+        node_dict = dict()
+        nodeindex_dict = dict()
+        edge_dict = dict()
+        edge_decode_dict = dict()
+        for s in dictname:
+            node_dict[s] = dictname[s]
+            nodeindex_dict[dictname[s]] = s # nodeindex_dict: {index_p: 'node_p'}
+            edge_dict[s] = {} # edge_dict: {'node_p': {'node_q': (index_p, index_q), 'node_m': (index_p, index_m)}}
+            for t in dictname:
+                edge_dict[s][t] = (dictname[s], dictname[t])
+                edge_decode_dict[(dictname[s], dictname[t])] = "->".join([s, t])
+        return cls(node_dict, nodeindex_dict, edge_dict, edge_decode_dict)
+
+    @classmethod
+    def from_edge(cls, filename):
+        edge_dict = dict()
+        edge_dict[""] = {}
+        edge_dict[""][""] = (0, 0)
+        edge_decode_dict = dict()
+        with open(filename) as f:
+            for line in f:
+                # line: node_p->node_q
+                s, t = line.strip().split("->")
+                if s not in edge_dict:
+                    i = len(edge_dict)
+                    j = 0
+                    edge_dict[s] = dict()
+                else:
+                    i = edge_dict[s][list(edge_dict[s].keys())[0]][0]
+                    j = len(edge_dict[s])
+                edge_dict[s][t] = (i, j)
+                edge_decode_dict[(i, j)] = "->".join([s, t])
+        return cls(None, edge_dict, edge_decode_dict)
+
+    def get_neighbor_of_edge(self, key, k, frequency_dict=None):
+        s, t = key.split("->") # s, t: node
+        _s = s if s in self.edge_dict else ""
+        
+        # if s in self.edge_dict:
+        #     ret = ["->".join([s, _t]) for _t in self.edge_dict[s].keys() if _t != t]
+        # else:
+        #     ret = ["->".join([s, _t]) for _t in self.edge_dict[""].keys() if _t != t]
+        # ret = ["->".join([s, _t]) for _t in self.edge_dict[s].keys() if _t != t]
+        
+        # select by word_frequency
+        if frequency_dict:
+            frequency_lst = list(frequency_dict[_s].keys())
+            # index = frequency_lst.index(t)
+            # half = k // 2
+            # if index <= k:
+            #     t_lst = [x for i, x in enumerate(frequency_lst[:k+1]) if i != index]
+            # else:
+            #     t_lst = frequency_lst[:half] + frequency_lst[index-half:index]
+            t_lst = [x for i, x in enumerate(frequency_lst[:k+1]) if x != t][:k]
+            ret = ["->".join([_s, _t]) for _t in t_lst]
+            random.shuffle(ret)
+            return ret
+        # randomly select k negative samples
+        else:
+            ret = ["->".join([_s, _t]) for _t in self.edge_dict[_s].keys() if _t != t]
+            random.shuffle(ret)
+            return ret[:k] if k != -1 else ret
+
+    def get_neighbor_of_node(self, key, k):
+        #key :index
+        s = self.nodeindex_dict[key] #node
+        #_t: node
+        ret = ["->".join([s, _t]) for _t in self.edge_dict[s].keys() if _t != s]
+        
+        # randomly select k negative samples
+        random.shuffle(ret) 
+        return ret[:k] if k != -1 else ret
+    
+    def get_neighbor_of_edge_broadcast(self, key, edges, k=100):
+        s, t = key.split("->")
+        _ret = [_t for _t in self.edge_dict[s].keys() if _t != t] # all neighbors of s except t
+        random.shuffle(_ret)
+        ret = []
+        for edge in edges:
+            s, t = edge.split("->")
+            ret += [["->".join([s, _t]) for _t in _ret[:k]]] 
+        return ret
+
+    @staticmethod
+    def to_path(tokens):
+        path = []
+        for left, right in zip(tokens[:-1], tokens[1:]):
+            path.append("->".join([left, right]))
+        return path
+
+    def get_edge_of_node(self, key):
+        return list(self.edge_dict[key].values())
+
+    def decode(self, x):
+        return self.edge_decode_dict[x]
+
+
+logging.basicConfig(format="%(asctime)s - %(levelname)s - %(name)s -   %(message)s",
+                    datefmt="%m/%d/%Y %H:%M:%S",
+                    level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+
+def stdf(string):
+    def _h(char):
+        inside_code = ord(char)
+        if inside_code == 0x3000:
+            inside_code = 0x0020
+        else:
+            inside_code -= 0xfee0
+        if inside_code < 0x0020 or inside_code > 0x7e:
+            return char
+        return chr(inside_code)
+
+    return "".join([_h(char) for char in string])
+
+
+class WikiDataset(Dataset):
+    """
+    Processor for wiki data.
+    """
+    def __init__(self, filename, vocab, max_seq_length, num_neg_samples, seed, buffer_size=100000, shuffle=True, use_frequency=False, use_bpe=False, bpe_tokenizer=None):
+        super().__init__()
+        self.vocab = vocab
+        self.max_seq_length = max_seq_length
+        self.num_neg_samples = num_neg_samples
+        self.generator = np.random.default_rng(seed=seed)
+        self.use_bpe = use_bpe
+        self.bpe_tokenizer = bpe_tokenizer
+        
+        self.data = self.read(filename)
+        
+        if use_frequency:
+            freq_file = 'word_frequency_en.json' if use_bpe else 'word_frequency.json'
+            with open(freq_file, 'r') as f:
+                self.frequency_dict = json.load(f)
+        else:
+            self.frequency_dict = None
+
+    def read(self, filename):
+        lines = []
+        with open(filename, "r", encoding="utf-8") as f:
+            for line in f:
+                if self.use_bpe:
+                    lines.append(line.strip())
+                else:
+                    src = list(line.strip()[:self.max_seq_length])
+                    lines.append(src)
+        return lines
+
+    def __len__(self):
+        return len(self.data)
+
+    def __getitem__(self, idx):
+        src = self.data[idx]
+        return self.vectorize(src)
+
+    def vectorize(self, src):
+        if self.use_bpe:
+            # For English with BPE
+            bpe_tokens = self.bpe_tokenizer.encode(src).tokens
+            # Truncate/pad
+            pad_token = "[PAD]"
+            if len(bpe_tokens) > self.max_seq_length:
+                bpe_tokens = bpe_tokens[:self.max_seq_length]
+            else:
+                bpe_tokens.extend(pad_token for _ in range(self.max_seq_length - len(bpe_tokens)))
+            tokens = bpe_tokens
+        else:
+            # For Chinese without BPE
+            if len(src) > self.max_seq_length:
+                src = src[:self.max_seq_length]
+            else:
+                src.extend("" for _ in range(self.max_seq_length-len(src)))
+            tokens = src
+            
+        edges = self.vocab.to_path(tokens)
+        edge_ids = self.vocab(edges)
+        edge_ids = edge_ids[:self.max_seq_length]
+        neighbor_ids = [self.vocab(self.vocab.get_neighbor_of_edge(e, self.num_neg_samples, self.frequency_dict)) for e in edges]
+
+        new_neighbor_ids = []
+        for i, e_ids in enumerate(edge_ids):
+            new_neighbor_ids.append([e_ids] + neighbor_ids[i])
+        return torch.LongTensor(new_neighbor_ids)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+
+    # Data config
+    parser.add_argument("--data_dir", type=str, default="data/wiki",
+                        help="Directory to contain the input data for all tasks.")
+    parser.add_argument("--output_dir", type=str, default="model/",
+                        help="Directory to output predictions and checkpoints.")
+    parser.add_argument("--load_state_dict", type=str, default=None,
+                        help="Trained model weights to load for evaluation if needed.")
+
+    # Training config
+    parser.add_argument("--do_train", action="store_true",
+                        help="Whether to run training.")
+    parser.add_argument("--do_eval", action="store_true",
+                        help="Whether to evaluate on the dev set.")
+    parser.add_argument("--num_neg_samples", type=int, default=100,
+                        help="Number of negative samples.")
+    parser.add_argument("--max_seq_length", type=int, default=128,
+                        help="Maximum total input sequence length after word-piece tokenization.")
+    parser.add_argument("--train_batch_size", type=int, default=128,
+                        help="Total batch size for training.")
+    parser.add_argument("--eval_batch_size", type=int, default=128,
+                        help="Total batch size for evaluation.")
+    parser.add_argument("--learning_rate", type=float, default=5e-5,
+                        help="Initial learning rate for Adam.")
+    parser.add_argument("--num_train_epochs", type=float, default=3.0,
+                        help="Total number of training epochs to perform.")
+    parser.add_argument("--max_train_steps", type=int, default=None,
+                        help="Total number of training steps to perform. If provided, overrides training epochs.")
+    parser.add_argument("--weight_decay", type=float, default=0.,
+                        help="L2 weight decay for training.")
+    parser.add_argument("--gradient_accumulation_steps", type=int, default=1,
+                        help="Number of updates steps to accumulate before performing a backward pass.")
+    parser.add_argument("--no_cuda", action="store_true",
+                        help="Whether not to use CUDA when available.")
+    parser.add_argument("--fp16", action="store_true",
+                        help="Whether to use mixed precision.")
+    parser.add_argument("--seed", type=int, default=42,
+                        help="Random seed for initialization.")
+    parser.add_argument("--save_steps", type=int, default=500,
+                        help="How many steps to save the checkpoint once.")
+    parser.add_argument("--hidden_size", type=int, default=32,
+                        help="Mask rate for masked-fine-tuning.")
+    parser.add_argument("--local_rank", type=int)
+    parser.add_argument("--initial_file_number", type=int, default=0,
+                        help="From which file to begin training.")
+    parser.add_argument("--end_file_number", type=int, default=0,
+                        help="End file number for training.")
+    parser.add_argument("--wiki_sorted_size", type=int, default=70,
+                        help="Total file numbers for sorted wikidata.")
+    parser.add_argument("--run_name", type=str, default="plusb_pluspe_order",
+                        help="Run name for wandb.")
+
+    parser.add_argument("--use_frequency", action="store_true",
+                        help="Whether to use word frequency.")
+    parser.add_argument("--train_full", type=str, default=None,
+                        help="Path to train on full text.")
+    parser.add_argument("--checkpoint_save_step", type=int, default=0,
+                        help="Interval to save checkpoint.(Only support when train_full is True)")
+    parser.add_argument("--resume_from_checkpoint", type=str, default=None,
+                        help="Path to checkpoint to resume training from")
+    parser.add_argument("--num_workers", type=int, default=8,
+                        help="Number of workers for data loading.")
+    parser.add_argument("--vocab_path", type=str, default="vocab_wiki_4k.json",
+                        help="Path to vocab file.")
+    parser.add_argument("--use_ds", action="store_true",
+                        help="Whether to use deepspeed.")
+    parser.add_argument("--sparse", action="store_true",
+                        help="Whether to use sparse.")
+    parser.add_argument("--use_bpe", action="store_true",
+                        help="Whether to use BPE tokenizer for English.")
+    parser.add_argument("--bpe_tokenizer_path", type=str, default="wiki_bpe_tokenizer_4000_bytelevel.json",
+                        help="Path to BPE tokenizer file.")
+
+    args = parser.parse_args()
+    
+    
+    device = torch.device("cuda" if torch.cuda.is_available() and not args.no_cuda else "cpu")
+    n_gpu = torch.cuda.device_count()
+    logger.info("device: {}, n_gpu: {}, distributed training: {}, 16-bits training: {}".format(
+        device, n_gpu, "-accelerate", args.fp16))
+
+    args.train_batch_size = args.train_batch_size // args.gradient_accumulation_steps
+
+    random.seed(args.seed)
+    np.random.seed(args.seed)
+    torch.manual_seed(args.seed)
+    if n_gpu > 0:
+        torch.cuda.manual_seed_all(args.seed)
+
+    # if not os.path.exists(args.output_dir):
+    #     os.makedirs(args.output_dir)
+
+    with open(args.vocab_path) as f:
+        node_dict = json.load(f)
+    vocab = Vocab.from_node_dict(node_dict)
+
+    if args.sparse:
+        with open('word_frequency.json', 'r') as f:
+            freq_dict = json.load(f)
+
+        zero_freq_edges = {}
+        for s in freq_dict:
+            zero_freq_edges[s] = []
+            for t in freq_dict[s]:
+                if freq_dict[s][t] == 0:
+                    zero_freq_edges[s].append(t)
+    else:
+        zero_freq_edges = None
+    
+
+    def stat_cuda(epoch, cur_file_num, step, location):
+        if accelerator.is_local_main_process:
+            with open("cuda_stat.txt", "a") as f:
+                if epoch is not None:
+                    f.write('epoch: %d, cur_file_num: %d, step: %d\n' % (epoch, cur_file_num, step))
+                f.write(f'--{location}\n')
+                f.write('allocated: %dG, max allocated: %dG, cached: %dG, max cached: %dG\n' % (
+                    torch.cuda.memory_allocated() / 1024 / 1024 / 1024,
+                    torch.cuda.max_memory_allocated() / 1024 / 1024 / 1024,
+                    torch.cuda.memory_reserved() / 1024 / 1024 / 1024,
+                    torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024
+            ))
+
+    if args.do_train:
+        # training arguments
+        os.environ["NCCL_DEBUG"] = "WARN"
+        os.environ["TORCH_NCCL_BLOCKING_WAIT"] = "1"
+        ddp_kwargs = DistributedDataParallelKwargs(find_unused_parameters=True)
+        init_kwargs = InitProcessGroupKwargs(timeout=timedelta(seconds=1080000))
+        accelerator = Accelerator(kwargs_handlers=[ddp_kwargs, init_kwargs], cpu=args.no_cuda, mixed_precision="fp16" if args.fp16 else "no")
+        device = accelerator.device
+
+        # prepare model
+        model = BraLM(args.hidden_size, args.use_ds, zero_freq_edges, vocab=vocab)
+        model.prepare_network(vocab)
+        # model.shared_weight.requires_grad = False
+        # model.shared_bias.requires_grad = False
+
+        # load model from checkpoint
+        if args.load_state_dict:
+            print(f"Loading model from checkpoint: {args.load_state_dict}")
+            checkpoint = torch.load(args.load_state_dict, map_location="cpu")
+            #model.load_state_dict(checkpoint["model_state_dict"])
+            model.load_old(checkpoint["model_state_dict"])
+
+
+        # Load checkpoint if specified
+        wandb_id = None
+        global_step = 0
+        if args.resume_from_checkpoint:
+            print(f"Resuming from checkpoint: {args.resume_from_checkpoint}")
+            checkpoint = torch.load(args.resume_from_checkpoint, map_location="cpu")
+            model.load_state_dict(checkpoint["model_state_dict"])
+            #optimizer.load_state_dict(checkpoint["optimizer_state_dict"])
+            start_epoch = checkpoint["epoch"] # + 1
+            global_step = checkpoint.get("global_step", 0)  # Get saved global step
+            wandb_id = checkpoint.get("wandb_id")
+        else:
+            start_epoch = 0
+
+        # if accelerator.is_local_main_process:   
+        #     for name, param in model.named_parameters():
+        #         print(name)
+
+        model.to_device(device)
+
+        
+        if accelerator.is_local_main_process:   
+            print(f"start_epoch: {start_epoch}, global_step: {global_step}")
+        
+        # prepare optimizer
+        no_decay = ["bias", "LayerNorm.bias", "LayerNorm.weight"]
+        optimizer_grouped_parameters = [
+            {
+                "params": [p for n, p in model.named_parameters() if not any(nd in n for nd in no_decay)],
+                "weight_decay": args.weight_decay
+            },
+            {
+                "params": [p for n, p in model.named_parameters() if any(nd in n for nd in no_decay)],
+                "weight_decay": 0.0
+            }
+        ]
+        optimizer = torch.optim.AdamW(optimizer_grouped_parameters, lr=args.learning_rate)
+
+        if args.resume_from_checkpoint:
+            optimizer.load_state_dict(checkpoint["optimizer_state_dict"])
+        
+        if accelerator.is_local_main_process:
+            print(f"before prepare")
+        #input('-' * 10)
+        #stat_cuda(None, None, None, "before prepare")
+        #print(f"{accelerator.device}, model: {model.weights.device}, tensor: {model.tensor.device}, pe: {model.positions.device}")
+        if not args.use_ds:
+            model, optimizer = accelerator.prepare(model, optimizer) # for deepspeed: # this line
+        #stat_cuda(None, None, None, "after prepare")
+        #print(f"{accelerator.device}, model: {model.module.weights.device}, tensor: {model.module.tensor.device}, pe: {model.module.positions.device}")
+        if accelerator.is_local_main_process:
+            print(f"after prepare")
+
+    if args.do_train:
+
+        if accelerator.is_local_main_process:
+            # init wandb
+            wandb.init(
+                project="brain",
+                name=args.run_name,
+                id=wandb_id,  # 如果有之前的run_id，使用它；否则会创建新的
+                resume="allow",  # "allow"表示如果有id就恢复，没有就创建新的
+                config=vars(args)
+            )
+            wandb.define_metric("custom_step")
+            wandb.define_metric("batch_*", step_metric="custom_step")
+            wandb.define_metric("epoch")
+            wandb.define_metric("epoch_*", step_metric="epoch")
+            print(f"Started wandb run with id: {wandb.run.id}")
+            print(f"View run at: {wandb.run.get_url()}")
+        
+        if args.train_full:
+            cur_file_num = args.train_full
+            cur_filename = f"{cur_file_num}.txt"
+            if args.use_bpe:
+                with open(args.bpe_tokenizer_path, 'r') as f:
+                    bpe_tokenizer = json.load(f)
+            else:
+                bpe_tokenizer = None
+            dataset = WikiDataset(
+                os.path.join(args.data_dir, cur_filename), 
+                vocab, 
+                args.max_seq_length, 
+                args.num_neg_samples, 
+                seed=args.seed, 
+                shuffle=True, 
+                use_frequency=args.use_frequency,
+                use_bpe=args.use_bpe,
+                bpe_tokenizer=bpe_tokenizer
+            )
+            train_dataloader = DataLoader(dataset, batch_size=args.train_batch_size, num_workers=args.num_workers, pin_memory=True)
+            train_dataloader = accelerator.prepare(train_dataloader)
+        elif args.resume_from_checkpoint:
+            cur_file_num = checkpoint["cur_file_num"]
+            if isinstance(cur_file_num, int) or cur_file_num.isdigit():
+                cur_file_num = int(cur_file_num) + 1
+                #start_epoch = start_epoch - 1
+        else:
+            cur_file_num = args.initial_file_number
+
+        if args.resume_from_checkpoint and global_step > 0:
+            if args.train_full and global_step % len(train_dataloader) == 0:
+                start_epoch = start_epoch + 1
+            if not args.train_full and cur_file_num > args.end_file_number:
+                start_epoch = start_epoch + 1
+                cur_file_num = args.initial_file_number
+        
+
+        for epoch in trange(start_epoch, int(args.num_train_epochs), desc="Epoch"):
+            # traverse all wiki files
+            if epoch != start_epoch or args.train_full:
+                cur_file_num = args.initial_file_number
+            while cur_file_num <= args.wiki_sorted_size:
+                if args.train_full:
+                    cur_file_num = args.train_full
+                logger.info("***** Running training for wiki = %s *****", cur_file_num)
+                logger.info("  Batch size = %d", args.train_batch_size * accelerator.num_processes)
+                
+                # prepare data
+                if not args.train_full:
+                    cur_filename = f"{cur_file_num}.txt"
+                    if args.use_bpe:
+                        with open(args.bpe_tokenizer_path, 'r') as f:
+                            bpe_tokenizer = json.load(f)
+                    else:
+                        bpe_tokenizer = None
+                    dataset = WikiDataset(
+                        os.path.join(args.data_dir, cur_filename), 
+                        vocab, 
+                        args.max_seq_length, 
+                        args.num_neg_samples, 
+                        seed=args.seed, 
+                        shuffle=True, 
+                        use_frequency=args.use_frequency,
+                        use_bpe=args.use_bpe,
+                        bpe_tokenizer=bpe_tokenizer
+                    )
+                    train_dataloader = DataLoader(dataset, batch_size=args.train_batch_size, num_workers=args.num_workers, pin_memory=True)
+                    if not args.use_ds:
+                        train_dataloader = accelerator.prepare(train_dataloader)
+                    else:
+                        model, optimizer, train_dataloader = accelerator.prepare(model, optimizer, train_dataloader) # for deepspeed
+
+
+                # training
+                train_loss = 0
+                num_train_examples = 0
+                if accelerator.is_local_main_process:
+                    progress_bar = tqdm(train_dataloader, desc="Iteration")
+                    # start_time = time.time()
+                
+                #for _ in range(3):
+                for step, batch in enumerate(train_dataloader, start=global_step % len(train_dataloader)):
+                    # batch: (bs, sen_len, 1+k, 2)
+                    batch_train_loss = 0
+                    batch_num_train_examples = 0
+                    #for ind in range(2, batch.size(1)):
+                    for ind in range(batch.size(1) - 1, batch.size(1)): # fix: only use the sen_len-1
+                        # ind: 2, 3, ..., sen_len-1
+                        # if accelerator.is_local_main_process:
+                        #     end_time = time.time()
+                        #     step_time = end_time - start_time
+                        #     logger.info(f"Step training time: {step_time:.2f} seconds")
+
+                        model.train()
+                        neighbor_ids = batch[:, :ind]   #(bs, ind, 1+k, 2)
+
+                        #stat_cuda(epoch, cur_file_num, global_step, "before forward")   
+                        outputs = model(neighbor_ids)
+                        loss = outputs
+
+                        # if n_gpu > 1:
+                        #     loss = loss.mean()
+
+                        if args.gradient_accumulation_steps > 1:
+                            loss = loss / args.gradient_accumulation_steps
+                        accelerator.backward(loss)
+
+                        if n_gpu > 1:
+                            dist.all_reduce(loss)
+                            loss = loss / dist.get_world_size()
+
+                        train_loss += loss.detach().item()
+                        batch_train_loss += loss.detach().item()
+
+                        num_train_examples += 1
+                        batch_num_train_examples += 1
+                        
+                        del outputs
+                        del loss
+                        del neighbor_ids
+                        gc.collect()
+                        # if step % 5 == 0:
+                        #    torch.cuda.empty_cache()
+
+                        if (step + 1) % args.gradient_accumulation_steps == 0:
+                            optimizer.step()
+                            optimizer.zero_grad()
+
+                            ## modified
+
+                            ppl = math.exp(batch_train_loss / batch_num_train_examples)
+                        
+                            if accelerator.is_local_main_process:
+                                progress_bar.update(1)
+                                progress_bar.set_postfix(loss=batch_train_loss / batch_num_train_examples, perplexity=ppl)
+                                
+                                wandb.log({
+                                    "batch_loss": batch_train_loss / batch_num_train_examples, 
+                                    "batch_perplexity": math.exp(batch_train_loss / batch_num_train_examples),
+                                    "batch_epoch": epoch,
+                                    #"step": global_step,
+                                    "custom_step": global_step
+                                })#, step=global_step)
+
+                        global_step += 1
+
+                        # Save checkpoint every checkpoint_save_step steps at the end of each step
+                        if accelerator.is_local_main_process and args.checkpoint_save_step > 0 and global_step % args.checkpoint_save_step == 0:
+                            output_dir_f = f"{args.output_dir}/HS{args.hidden_size}/step_{global_step}/"
+                            if not os.path.exists(output_dir_f):
+                                os.makedirs(output_dir_f)
+                            output_model_file = os.path.join(output_dir_f, f"checkpoint_{global_step}.bin")
+                            
+                            model_to_save = model.module if hasattr(model, "module") else model
+                            checkpoint = {
+                                "model_state_dict": model_to_save.state_dict(),
+                                "optimizer_state_dict": optimizer.state_dict(), 
+                                "epoch": epoch,
+                                "global_step": global_step,
+                                "args": vars(args),
+                                "wandb_id": wandb.run.id
+                            }
+                            if not args.train_full:
+                                checkpoint["cur_file_num"] = cur_file_num
+                            print(f"Saving checkpoint to {output_model_file}")
+                            torch.save(checkpoint, output_model_file)
+                            print(f"Checkpoint saved to {output_model_file}")
+
+
+                # save model for current training file
+                if accelerator.is_local_main_process:
+                    epoch_avg_loss = train_loss / num_train_examples
+                    epoch_ppl = math.exp(epoch_avg_loss)
+                    wandb.log({
+                        "epoch_loss": epoch_avg_loss,
+                        "epoch_perplexity": epoch_ppl,
+                        "epoch": epoch,
+                    })#, step=global_step)
+                    
+                    model_to_save = model.module if hasattr(model, "module") else model
+                    output_dir_f = f"{args.output_dir}/HS{args.hidden_size}/EPOCH{epoch}/"
+                    if not os.path.exists(output_dir_f):
+                        os.makedirs(output_dir_f)
+                    output_model_file = os.path.join(output_dir_f, "f{}_pytorch_model.bin".format(cur_file_num))
+                    # only save the last model
+                    if args.train_full or cur_file_num == args.end_file_number:
+                        #torch.save(model_to_save.state_dict(), output_model_file)
+                        checkpoint = {
+                            "model_state_dict": model_to_save.state_dict(),
+                            "optimizer_state_dict": optimizer.state_dict(),
+                            "epoch": epoch,
+                            "global_step": global_step,  # Save global step
+                            "args": vars(args),
+                            "wandb_id": wandb.run.id  # 保存当前运行的wandb_id
+                        }
+                        if not args.train_full:
+                            checkpoint["cur_file_num"] = cur_file_num
+                        print(f"Saving model to {output_model_file}")
+                        torch.save(checkpoint, output_model_file)
+                        print(f"Model saved to {output_model_file}")
+
+                if args.train_full:
+                    break
+                cur_file_num += 1
+                if cur_file_num > args.end_file_number:
+                    break
+
+
+
+if __name__ == "__main__":
+    main()

vocab_wiki_4k.json

@@ -0,0 +1,4003 @@
+{
+    "<unk>": 0,
+    ",": 1,
+    "的": 2,
+    "。": 3,
+    "1": 4,
+    "年": 5,
+    "、": 6,
+    "在": 7,
+    "0": 8,
+    "一": 9,
+    "为": 10,
+    "中": 11,
+    "国": 12,
+    "2": 13,
+    "9": 14,
+    "是": 15,
+    "人": 16,
+    "有": 17,
+    "大": 18,
+    "于": 19,
+    "和": 20,
+    "后": 21,
+    "了": 22,
+    "以": 23,
+    "时": 24,
+    "月": 25,
+    "日": 26,
+    "会": 27,
+    "不": 28,
+    "学": 29,
+    "(": 30,
+    ")": 31,
+    "地": 32,
+    "成": 33,
+    "他": 34,
+    "5": 35,
+    "上": 36,
+    "3": 37,
+    "8": 38,
+    "4": 39,
+    "出": 40,
+    "之": 41,
+    "个": 42,
+    "军": 43,
+    "6": 44,
+    "与": 45,
+    "到": 46,
+    "部": 47,
+    "7": 48,
+    "行": 49,
+    "主": 50,
+    "发": 51,
+    "民": 52,
+    "其": 53,
+    "而": 54,
+    "生": 55,
+    "作": 56,
+    "并": 57,
+    "斯": 58,
+    "这": 59,
+    "对": 60,
+    "法": 61,
+    "公": 62,
+    "被": 63,
+    "分": 64,
+    "家": 65,
+    "来": 66,
+    "政": 67,
+    "及": 68,
+    "方": 69,
+    "同": 70,
+    "第": 71,
+    "区": 72,
+    "多": 73,
+    "用": 74,
+    "西": 75,
+    "长": 76,
+    "自": 77,
+    "下": 78,
+    "名": 79,
+    "开": 80,
+    "由": 81,
+    "《": 82,
+    "》": 83,
+    "南": 84,
+    "本": 85,
+    "也": 86,
+    "任": 87,
+    "动": 88,
+    "「": 89,
+    "因": 90,
+    "“": 91,
+    "此": 92,
+    "」": 93,
+    "尔": 94,
+    "北": 95,
+    "可": 96,
+    "”": 97,
+    "文": 98,
+    "经": 99,
+    "要": 100,
+    "最": 101,
+    "子": 102,
+    "战": 103,
+    "过": 104,
+    "前": 105,
+    "员": 106,
+    "所": 107,
+    "进": 108,
+    "得": 109,
+    "新": 110,
+    "当": 111,
+    "台": 112,
+    "东": 113,
+    "事": 114,
+    "等": 115,
+    "高": 116,
+    "建": 117,
+    "加": 118,
+    "三": 119,
+    "立": 120,
+    "教": 121,
+    "间": 122,
+    "但": 123,
+    "将": 124,
+    "代": 125,
+    "市": 126,
+    "位": 127,
+    "特": 128,
+    "现": 129,
+    "海": 130,
+    "至": 131,
+    "入": 132,
+    "德": 133,
+    "·": 134,
+    "内": 135,
+    "能": 136,
+    "期": 137,
+    "工": 138,
+    "外": 139,
+    "利": 140,
+    "世": 141,
+    "定": 142,
+    "里": 143,
+    "道": 144,
+    "次": 145,
+    "理": 146,
+    "天": 147,
+    "美": 148,
+    "二": 149,
+    "机": 150,
+    "克": 151,
+    "制": 152,
+    "者": 153,
+    "队": 154,
+    "重": 155,
+    "路": 156,
+    "全": 157,
+    "使": 158,
+    "十": 159,
+    "力": 160,
+    "都": 161,
+    "面": 162,
+    "度": 163,
+    "亚": 164,
+    "表": 165,
+    "山": 166,
+    "电": 167,
+    "合": 168,
+    "共": 169,
+    "业": 170,
+    "通": 171,
+    "物": 172,
+    "两": 173,
+    "体": 174,
+    "场": 175,
+    "小": 176,
+    ":": 177,
+    "化": 178,
+    "马": 179,
+    "们": 180,
+    "王": 181,
+    " ": 182,
+    "称": 183,
+    "设": 184,
+    "平": 185,
+    "性": 186,
+    "相": 187,
+    "联": 188,
+    "府": 189,
+    "院": 190,
+    "安": 191,
+    "关": 192,
+    "赛": 193,
+    "拉": 194,
+    "正": 195,
+    "州": 196,
+    "车": 197,
+    "明": 198,
+    "数": 199,
+    "球": 200,
+    "种": 201,
+    "从": 202,
+    "起": 203,
+    "华": 204,
+    "原": 205,
+    "统": 206,
+    "总": 207,
+    "城": 208,
+    "议": 209,
+    "然": 210,
+    "治": 211,
+    "达": 212,
+    "系": 213,
+    ";": 214,
+    "线": 215,
+    "就": 216,
+    "比": 217,
+    "港": 218,
+    "向": 219,
+    "式": 220,
+    "如": 221,
+    "金": 222,
+    "选": 223,
+    "士": 224,
+    "元": 225,
+    "基": 226,
+    "罗": 227,
+    "布": 228,
+    "水": 229,
+    "提": 230,
+    "受": 231,
+    "改": 232,
+    "组": 233,
+    "产": 234,
+    "四": 235,
+    "英": 236,
+    "-": 237,
+    "运": 238,
+    "实": 239,
+    "约": 240,
+    "该": 241,
+    "常": 242,
+    "说": 243,
+    "党": 244,
+    "号": 245,
+    "或": 246,
+    "无": 247,
+    "义": 248,
+    "影": 249,
+    "计": 250,
+    "心": 251,
+    "书": 252,
+    "接": 253,
+    "些": 254,
+    "始": 255,
+    "委": 256,
+    "意": 257,
+    "科": 258,
+    "格": 259,
+    "则": 260,
+    "量": 261,
+    "司": 262,
+    "务": 263,
+    "认": 264,
+    "县": 265,
+    "领": 266,
+    "界": 267,
+    "已": 268,
+    "门": 269,
+    "兰": 270,
+    "林": 271,
+    "首": 272,
+    "保": 273,
+    "结": 274,
+    "交": 275,
+    "女": 276,
+    "传": 277,
+    "参": 278,
+    "校": 279,
+    "还": 280,
+    "回": 281,
+    "尼": 282,
+    "直": 283,
+    "万": 284,
+    "口": 285,
+    "目": 286,
+    "巴": 287,
+    "处": 288,
+    "风": 289,
+    "持": 290,
+    "展": 291,
+    "各": 292,
+    "团": 293,
+    "程": 294,
+    "指": 295,
+    "导": 296,
+    "湾": 297,
+    "反": 298,
+    "香": 299,
+    "手": 300,
+    "广": 301,
+    ".": 302,
+    "史": 303,
+    "社": 304,
+    "更": 305,
+    "阿": 306,
+    "师": 307,
+    "���": 308,
+    "只": 309,
+    "曾": 310,
+    "语": 311,
+    "包": 312,
+    "演": 313,
+    "太": 314,
+    "击": 315,
+    "应": 316,
+    "权": 317,
+    "没": 318,
+    "省": 319,
+    "站": 320,
+    "色": 321,
+    "朝": 322,
+    "命": 323,
+    "形": 324,
+    "身": 325,
+    "五": 326,
+    "获": 327,
+    "列": 328,
+    "官": 329,
+    "米": 330,
+    "管": 331,
+    "信": 332,
+    "带": 333,
+    "决": 334,
+    "流": 335,
+    "变": 336,
+    "取": 337,
+    "局": 338,
+    "造": 339,
+    "集": 340,
+    "著": 341,
+    "视": 342,
+    "张": 343,
+    "情": 344,
+    "点": 345,
+    "族": 346,
+    "论": 347,
+    "少": 348,
+    "乐": 349,
+    "属": 350,
+    "古": 351,
+    "级": 352,
+    "维": 353,
+    "兵": 354,
+    "空": 355,
+    "气": 356,
+    "举": 357,
+    "强": 358,
+    "神": 359,
+    "清": 360,
+    "示": 361,
+    "解": 362,
+    "纪": 363,
+    "支": 364,
+    "资": 365,
+    "京": 366,
+    "派": 367,
+    "别": 368,
+    "近": 369,
+    "据": 370,
+    "类": 371,
+    "活": 372,
+    "江": 373,
+    "又": 374,
+    "历": 375,
+    "河": 376,
+    "先": 377,
+    "武": 378,
+    "铁": 379,
+    "往": 380,
+    "她": 381,
+    "记": 382,
+    "岛": 383,
+    "即": 384,
+    "非": 385,
+    "去": 386,
+    "品": 387,
+    "帝": 388,
+    "争": 389,
+    "再": 390,
+    "星": 391,
+    "苏": 392,
+    "放": 393,
+    "转": 394,
+    "术": 395,
+    "李": 396,
+    "字": 397,
+    "条": 398,
+    "夫": 399,
+    "件": 400,
+    "音": 401,
+    "令": 402,
+    "商": 403,
+    "石": 404,
+    "光": 405,
+    "头": 406,
+    "随": 407,
+    "初": 408,
+    "且": 409,
+    "研": 410,
+    "很": 411,
+    "宗": 412,
+    "洲": 413,
+    "未": 414,
+    "亲": 415,
+    "卡": 416,
+    "单": 417,
+    "图": 418,
+    "周": 419,
+    "调": 420,
+    "际": 421,
+    "收": 422,
+    "汉": 423,
+    "攻": 424,
+    "许": 425,
+    "复": 426,
+    "职": 427,
+    "知": 428,
+    "它": 429,
+    "专": 430,
+    "果": 431,
+    "节": 432,
+    "打": 433,
+    "连": 434,
+    "划": 435,
+    "完": 436,
+    "离": 437,
+    "土": 438,
+    "波": 439,
+    "求": 440,
+    "亦": 441,
+    "陆": 442,
+    "问": 443,
+    "死": 444,
+    "续": 445,
+    "型": 446,
+    "给": 447,
+    "游": 448,
+    "今": 449,
+    "观": 450,
+    "办": 451,
+    "除": 452,
+    "题": 453,
+    "每": 454,
+    "括": 455,
+    "奥": 456,
+    "规": 457,
+    "居": 458,
+    "龙": 459,
+    "见": 460,
+    "六": 461,
+    "言": 462,
+    "普": 463,
+    "究": 464,
+    "己": 465,
+    "我": 466,
+    "白": 467,
+    "角": 468,
+    "较": 469,
+    "剧": 470,
+    "案": 471,
+    "根": 472,
+    "纳": 473,
+    "育": 474,
+    "修": 475,
+    "片": 476,
+    "好": 477,
+    "创": 478,
+    "宣": 479,
+    "众": 480,
+    "段": 481,
+    "伊": 482,
+    "济": 483,
+    "引": 484,
+    "致": 485,
+    "终": 486,
+    "投": 487,
+    "威": 488,
+    "八": 489,
+    "标": 490,
+    "功": 491,
+    "服": 492,
+    "福": 493,
+    "样": 494,
+    "另": 495,
+    "失": 496,
+    "质": 497,
+    "助": 498,
+    "舰": 499,
+    "装": 500,
+    "协": 501,
+    "陈": 502,
+    "皇": 503,
+    "阳": 504,
+    "营": 505,
+    "希": 506,
+    "率": 507,
+    "兴": 508,
+    "器": 509,
+    "速": 510,
+    "增": 511,
+    "卫": 512,
+    "查": 513,
+    "黄": 514,
+    "构": 515,
+    "护": 516,
+    "继": 517,
+    "告": 518,
+    "那": 519,
+    "存": 520,
+    "备": 521,
+    "季": 522,
+    "准": 523,
+    "越": 524,
+    "九": 525,
+    "欧": 526,
+    "早": 527,
+    "央": 528,
+    "占": 529,
+    "推": 530,
+    "园": 531,
+    "整": 532,
+    "火": 533,
+    "%": 534,
+    "边": 535,
+    "故": 536,
+    "响": 537,
+    "具": 538,
+    "百": 539,
+    "置": 540,
+    "才": 541,
+    "仍": 542,
+    "爱": 543,
+    "供": 544,
+    "足": 545,
+    "镇": 546,
+    "担": 547,
+    "围": 548,
+    "青": 549,
+    "超": 550,
+    "便": 551,
+    "杀": 552,
+    "例": 553,
+    "客": 554,
+    "让": 555,
+    "花": 556,
+    "源": 557,
+    "革": 558,
+    "织": 559,
+    "警": 560,
+    "独": 561,
+    "落": 562,
+    "项": 563,
+    "圣": 564,
+    "想": 565,
+    "馆": 566,
+    "座": 567,
+    "份": 568,
+    "境": 569,
+    "儿": 570,
+    "需": 571,
+    "田": 572,
+    "象": 573,
+    "防": 574,
+    "航": 575,
+    "何": 576,
+    "父": 577,
+    "志": 578,
+    "村": 579,
+    "印": 580,
+    "密": 581,
+    "升": 582,
+    "留": 583,
+    "易": 584,
+    "塔": 585,
+    "席": 586,
+    "把": 587,
+    "施": 588,
+    "老": 589,
+    "证": 590,
+    "哈": 591,
+    "副": 592,
+    "均": 593,
+    "编": 594,
+    "败": 595,
+    "歌": 596,
+    "班": 597,
+    "伯": 598,
+    "母": 599,
+    "热": 600,
+    "容": 601,
+    "感": 602,
+    "极": 603,
+    "步": 604,
+    "半": 605,
+    "声": 606,
+    "住": 607,
+    "考": 608,
+    "着": 609,
+    "七": 610,
+    "几": 611,
+    "医": 612,
+    "止": 613,
+    "红": 614,
+    "吉": 615,
+    "曲": 616,
+    "飞": 617,
+    "环": 618,
+    "版": 619,
+    "盟": 620,
+    "看": 621,
+    "评": 622,
+    "律": 623,
+    "播": 624,
+    "域": 625,
+    "层": 626,
+    "萨": 627,
+    "沙": 628,
+    "录": 629,
+    "群": 630,
+    "农": 631,
+    "刘": 632,
+    "降": 633,
+    "室": 634,
+    "低": 635,
+    "伦": 636,
+    "真": 637,
+    "守": 638,
+    "双": 639,
+    "技": 640,
+    "艺": 641,
+    "承": 642,
+    "筑": 643,
+    "批": 644,
+    "势": 645,
+    "深": 646,
+    "博": 647,
+    "佛": 648,
+    "登": 649,
+    "试": 650,
+    "费": 651,
+    "写": 652,
+    "奖": 653,
+    "网": 654,
+    "察": 655,
+    "移": 656,
+    "抗": 657,
+    "胜": 658,
+    "仅": 659,
+    "楼": 660,
+    "积": 661,
+    "食": 662,
+    "模": 663,
+    "话": 664,
+    "洋": 665,
+    "却": 666,
+    "票": 667,
+    "黑": 668,
+    "采": 669,
+    "鲁": 670,
+    "排": 671,
+    "雷": 672,
+    "画": 673,
+    "洛": 674,
+    "确": 675,
+    "虽": 676,
+    "像": 677,
+    "责": 678,
+    "范": 679,
+    "—": 680,
+    "督": 681,
+    "病": 682,
+    "请": 683,
+    "远": 684,
+    "消": 685,
+    "满": 686,
+    "余": 687,
+    "射": 688,
+    "岁": 689,
+    "戏": 690,
+    "乡": 691,
+    "征": 692,
+    "干": 693,
+    "湖": 694,
+    "望": 695,
+    "勒": 696,
+    "堂": 697,
+    "料": 698,
+    "千": 699,
+    "负": 700,
+    "显": 701,
+    "做": 702,
+    "限": 703,
+    "思": 704,
+    "男": 705,
+    "船": 706,
+    "木": 707,
+    "效": 708,
+    "顿": 709,
+    "难": 710,
+    "依": 711,
+    "压": 712,
+    "景": 713,
+    "状": 714,
+    "控": 715,
+    "破": 716,
+    "俄": 717,
+    "街": 718,
+    "退": 719,
+    "康": 720,
+    "断": 721,
+    "川": 722,
+    "藏": 723,
+    "宫": 724,
+    "念": 725,
+    "精": 726,
+    "房": 727,
+    "必": 728,
+    "氏": 729,
+    "托": 730,
+    "诺": 731,
+    "预": 732,
+    "习": 733,
+    "友": 734,
+    "严": 735,
+    "温": 736,
+    "态": 737,
+    "候": 738,
+    "哥": 739,
+    "典": 740,
+    "塞": 741,
+    "桥": 742,
+    "森": 743,
+    "云": 744,
+    "曼": 745,
+    "章": 746,
+    "照": 747,
+    "配": 748,
+    "突": 749,
+    "况": 750,
+    "莱": 751,
+    "宁": 752,
+    "价": 753,
+    "素": 754,
+    "遭": 755,
+    "暴": 756,
+    "蒙": 757,
+    "久": 758,
+    "停": 759,
+    "测": 760,
+    "亡": 761,
+    "执": 762,
+    "奇": 763,
+    "策": 764,
+    "底": 765,
+    "斗": 766,
+    "走": 767,
+    "述": 768,
+    "附": 769,
+    "唐": 770,
+    "庄": 771,
+    "伤": 772,
+    "瓦": 773,
+    "甲": 774,
+    "识": 775,
+    "载": 776,
+    "判": 777,
+    "绝": 778,
+    "恩": 779,
+    "算": 780,
+    "左": 781,
+    "逐": 782,
+    "封": 783,
+    "冠": 784,
+    "银": 785,
+    "快": 786,
+    "授": 787,
+    "礼": 788,
+    "韩": 789,
+    "弹": 790,
+    "届": 791,
+    "验": 792,
+    "注": 793,
+    "祖": 794,
+    "词": 795,
+    "富": 796,
+    "户": 797,
+    "厅": 798,
+    "永": 799,
+    "毛": 800,
+    "般": 801,
+    "寺": 802,
+    "监": 803,
+    "宝": 804,
+    "泰": 805,
+    "庆": 806,
+    "拿": 807,
+    "闻": 808,
+    "牙": 809,
+    "姆": 810,
+    "澳": 811,
+    "临": 812,
+    "丽": 813,
+    "贝": 814,
+    "厂": 815,
+    "延": 816,
+    "拥": 817,
+    "迪": 818,
+    "邦": 819,
+    "旅": 820,
+    "索": 821,
+    "读": 822,
+    "右": 823,
+    "夏": 824,
+    "孙": 825,
+    "吴": 826,
+    "归": 827,
+    "瑞": 828,
+    "息": 829,
+    "尽": 830,
+    "值": 831,
+    "须": 832,
+    "堡": 833,
+    "弟": 834,
+    "晚": 835,
+    "甚": 836,
+    "切": 837,
+    "埃": 838,
+    "丹": 839,
+    "岸": 840,
+    "役": 841,
+    "杨": 842,
+    "束": 843,
+    "叶": 844,
+    "春": 845,
+    "冲": 846,
+    "署": 847,
+    "坦": 848,
+    "雄": 849,
+    "庭": 850,
+    "轮": 851,
+    "宋": 852,
+    "渐": 853,
+    "送": 854,
+    "隆": 855,
+    "侧": 856,
+    "旧": 857,
+    "似": 858,
+    "臣": 859,
+    "轻": 860,
+    "练": 861,
+    "迁": 862,
+    "麦": 863,
+    "救": 864,
+    "野": 865,
+    "害": 866,
+    "驻": 867,
+    "减": 868,
+    "核": 869,
+    "优": 870,
+    "阵": 871,
+    "库": 872,
+    "异": 873,
+    "灵": 874,
+    "细": 875,
+    "烈": 876,
+    "末": 877,
+    "爆": 878,
+    "阶": 879,
+    "丁": 880,
+    "乌": 881,
+    "草": 882,
+    "善": 883,
+    "财": 884,
+    "含": 885,
+    "罪": 886,
+    "遗": 887,
+    "尚": 888,
+    "唱": 889,
+    "兼": 890,
+    "略": 891,
+    "牌": 892,
+    "嘉": 893,
+    "免": 894,
+    "审": 895,
+    "劳": 896,
+    "简": 897,
+    "若": 898,
+    "油": 899,
+    "良": 900,
+    "敌": 901,
+    "乘": 902,
+    "杰": 903,
+    "松": 904,
+    "币": 905,
+    "亿": 906,
+    "店": 907,
+    "援": 908,
+    "签": 909,
+    "郡": 910,
+    "散": 911,
+    "婚": 912,
+    "挥": 913,
+    "养": 914,
+    "津": 915,
+    "酒": 916,
+    "毕": 917,
+    "鲜": 918,
+    "换": 919,
+    "胡": 920,
+    "予": 921,
+    "诗": 922,
+    "顺": 923,
+    "短": 924,
+    "徒": 925,
+    "端": 926,
+    "释": 927,
+    "追": 928,
+    "训": 929,
+    "差": 930,
+    "钟": 931,
+    "莫": 932,
+    "荣": 933,
+    "艾": 934,
+    "盛": 935,
+    "夺": 936,
+    "贵": 937,
+    "树": 938,
+    "晋": 939,
+    "皮": 940,
+    "泽": 941,
+    "禁": 942,
+    "秀": 943,
+    "乱": 944,
+    "朱": 945,
+    "梅": 946,
+    "够": 947,
+    "讨": 948,
+    "辑": 949,
+    "血": 950,
+    "架": 951,
+    "启": 952,
+    "摩": 953,
+    "廷": 954,
+    "药": 955,
+    "旗": 956,
+    "君": 957,
+    "扩": 958,
+    "籍": 959,
+    "玛": 960,
+    "仁": 961,
+    "补": 962,
+    "朗": 963,
+    "召": 964,
+    "丰": 965,
+    "撤": 966,
+    "假": 967,
+    "午": 968,
+    "摄": 969,
+    "访": 970,
+    "庙": 971,
+    "梁": 972,
+    "辖": 973,
+    "检": 974,
+    "枪": 975,
+    "沿": 976,
+    "鱼": 977,
+    "益": 978,
+    "/": 979,
+    "输": 980,
+    "宪": 981,
+    "诸": 982,
+    "逃": 983,
+    "佳": 984,
+    "迫": 985,
+    "屋": 986,
+    "杂": 987,
+    "玉": 988,
+    "抵": 989,
+    "雅": 990,
+    "喜": 991,
+    "遇": 992,
+    "袭": 993,
+    "弗": 994,
+    "菲": 995,
+    "讲": 996,
+    "顶": 997,
+    "秘": 998,
+    "互": 999,
+    "仪": 1000,
+    "款": 1001,
+    "帮": 1002,
+    "微": 1003,
+    "否": 1004,
+    "拜": 1005,
+    "险": 1006,
+    "企": 1007,
+    "拍": 1008,
+    "什": 1009,
+    "觉": 1010,
+    "返": 1011,
+    "蒂": 1012,
+    "媒": 1013,
+    "股": 1014,
+    "盖": 1015,
+    "耶": 1016,
+    "齐": 1017,
+    "诉": 1018,
+    "找": 1019,
+    "炮": 1020,
+    "捕": 1021,
+    "适": 1022,
+    "借": 1023,
+    "愿": 1024,
+    "犯": 1025,
+    "序": 1026,
+    "童": 1027,
+    "黎": 1028,
+    "背": 1029,
+    "饰": 1030,
+    "乎": 1031,
+    "昌": 1032,
+    "凯": 1033,
+    "植": 1034,
+    "透": 1035,
+    "郑": 1036,
+    "灭": 1037,
+    "板": 1038,
+    "销": 1039,
+    "跟": 1040,
+    "侵": 1041,
+    "刻": 1042,
+    "材": 1043,
+    "夜": 1044,
+    "货": 1045,
+    "额": 1046,
+    "纽": 1047,
+    "\"": 1048,
+    "坚": 1049,
+    "按": 1050,
+    "谈": 1051,
+    "待": 1052,
+    "途": 1053,
+    "旋": 1054,
+    "奉": 1055,
+    "赫": 1056,
+    "赵": 1057,
+    "私": 1058,
+    "损": 1059,
+    "谋": 1060,
+    "隶": 1061,
+    "充": 1062,
+    "某": 1063,
+    "么": 1064,
+    "介": 1065,
+    "吸": 1066,
+    "刑": 1067,
+    "谷": 1068,
+    "距": 1069,
+    "缺": 1070,
+    "巡": 1071,
+    "激": 1072,
+    "骑": 1073,
+    "忠": 1074,
+    "听": 1075,
+    "献": 1076,
+    "毁": 1077,
+    "聚": 1078,
+    "钱": 1079,
+    "雨": 1080,
+    "尾": 1081,
+    "课": 1082,
+    "废": 1083,
+    "震": 1084,
+    "杯": 1085,
+    "舞": 1086,
+    "巨": 1087,
+    "宾": 1088,
+    "售": 1089,
+    "混": 1090,
+    "购": 1091,
+    "刺": 1092,
+    "洪": 1093,
+    "怀": 1094,
+    "穿": 1095,
+    "皆": 1096,
+    "牛": 1097,
+    "欢": 1098,
+    "疑": 1099,
+    "卢": 1100,
+    "避": 1101,
+    "荷": 1102,
+    "郎": 1103,
+    "择": 1104,
+    "顾": 1105,
+    "露": 1106,
+    "谢": 1107,
+    "竞": 1108,
+    "智": 1109,
+    "固": 1110,
+    "疗": 1111,
+    "圆": 1112,
+    "讯": 1113,
+    "唯": 1114,
+    "眼": 1115,
+    "申": 1116,
+    "探": 1117,
+    "兹": 1118,
+    "休": 1119,
+    "徐": 1120,
+    "御": 1121,
+    "穆": 1122,
+    "帕": 1123,
+    "玩": 1124,
+    "映": 1125,
+    "订": 1126,
+    "杜": 1127,
+    "急": 1128,
+    "描": 1129,
+    "你": 1130,
+    "赞": 1131,
+    "殖": 1132,
+    "溪": 1133,
+    "爵": 1134,
+    "宇": 1135,
+    "坏": 1136,
+    "敦": 1137,
+    "殿": 1138,
+    "恶": 1139,
+    "刚": 1140,
+    "尤": 1141,
+    "孩": 1142,
+    "贸": 1143,
+    "刊": 1144,
+    "竹": 1145,
+    "靠": 1146,
+    "兄": 1147,
+    "峰": 1148,
+    "幕": 1149,
+    "轨": 1150,
+    "呼": 1151,
+    "秦": 1152,
+    "译": 1153,
+    "紧": 1154,
+    "炸": 1155,
+    "妻": 1156,
+    "潮": 1157,
+    "毒": 1158,
+    "涉": 1159,
+    "坡": 1160,
+    "秋": 1161,
+    "阻": 1162,
+    "弃": 1163,
+    "址": 1164,
+    "羽": 1165,
+    "乔": 1166,
+    "替": 1167,
+    "翰": 1168,
+    "姓": 1169,
+    "汇": 1170,
+    "魔": 1171,
+    "脱": 1172,
+    "泛": 1173,
+    "雪": 1174,
+    "拒": 1175,
+    "招": 1176,
+    "危": 1177,
+    "魏": 1178,
+    "宽": 1179,
+    "蓝": 1180,
+    "骨": 1181,
+    "频": 1182,
+    "操": 1183,
+    "陵": 1184,
+    "翻": 1185,
+    "买": 1186,
+    "渡": 1187,
+    "耳": 1188,
+    "尊": 1189,
+    "辆": 1190,
+    "迎": 1191,
+    "蒋": 1192,
+    "础": 1193,
+    "…": 1194,
+    "融": 1195,
+    "培": 1196,
+    "寻": 1197,
+    "衣": 1198,
+    "戴": 1199,
+    "��": 1200,
+    "董": 1201,
+    "绿": 1202,
+    "裁": 1203,
+    "针": 1204,
+    "贡": 1205,
+    "恐": 1206,
+    "}": 1207,
+    "卷": 1208,
+    "误": 1209,
+    "{": 1210,
+    "弱": 1211,
+    "宜": 1212,
+    "娜": 1213,
+    "舍": 1214,
+    "横": 1215,
+    "径": 1216,
+    "丝": 1217,
+    "肯": 1218,
+    "冷": 1219,
+    "偏": 1220,
+    "邻": 1221,
+    "健": 1222,
+    "灾": 1223,
+    "倒": 1224,
+    "沃": 1225,
+    "锦": 1226,
+    "租": 1227,
+    "惠": 1228,
+    "付": 1229,
+    "隔": 1230,
+    "霍": 1231,
+    "困": 1232,
+    "妇": 1233,
+    "尺": 1234,
+    "掌": 1235,
+    "岩": 1236,
+    "腊": 1237,
+    "柏": 1238,
+    "侯": 1239,
+    "脑": 1240,
+    "哲": 1241,
+    "陷": 1242,
+    "绩": 1243,
+    "俗": 1244,
+    "阴": 1245,
+    "恢": 1246,
+    "符": 1247,
+    "奏": 1248,
+    "税": 1249,
+    "卖": 1250,
+    "染": 1251,
+    "毫": 1252,
+    "暗": 1253,
+    "辞": 1254,
+    "篇": 1255,
+    "冰": 1256,
+    "扬": 1257,
+    "曹": 1258,
+    "迹": 1259,
+    "赖": 1260,
+    "墨": 1261,
+    "犹": 1262,
+    "剑": 1263,
+    "汽": 1264,
+    "楚": 1265,
+    "伐": 1266,
+    "络": 1267,
+    "绍": 1268,
+    "繁": 1269,
+    "镜": 1270,
+    "赢": 1271,
+    "块": 1272,
+    "努": 1273,
+    "暂": 1274,
+    "圈": 1275,
+    "套": 1276,
+    "佐": 1277,
+    "概": 1278,
+    "乃": 1279,
+    "呈": 1280,
+    "阁": 1281,
+    "稳": 1282,
+    "错": 1283,
+    "辛": 1284,
+    "廉": 1285,
+    "鼓": 1286,
+    "莉": 1287,
+    "艘": 1288,
+    "彩": 1289,
+    "坐": 1290,
+    "崇": 1291,
+    "盘": 1292,
+    "墙": 1293,
+    "井": 1294,
+    "矿": 1295,
+    "遣": 1296,
+    "泉": 1297,
+    "佩": 1298,
+    "肉": 1299,
+    "芬": 1300,
+    "扎": 1301,
+    "纸": 1302,
+    "鸟": 1303,
+    "症": 1304,
+    "奴": 1305,
+    "韦": 1306,
+    "赴": 1307,
+    "锡": 1308,
+    "味": 1309,
+    "驱": 1310,
+    "珠": 1311,
+    "辅": 1312,
+    "裂": 1313,
+    "默": 1314,
+    "笔": 1315,
+    "虎": 1316,
+    "恒": 1317,
+    "翼": 1318,
+    "患": 1319,
+    "郭": 1320,
+    "孔": 1321,
+    "幼": 1322,
+    "墓": 1323,
+    "脚": 1324,
+    "叛": 1325,
+    "]": 1326,
+    "叫": 1327,
+    "估": 1328,
+    "邀": 1329,
+    "[": 1330,
+    "胞": 1331,
+    "促": 1332,
+    "昭": 1333,
+    "触": 1334,
+    "甘": 1335,
+    "陶": 1336,
+    "静": 1337,
+    "障": 1338,
+    "亮": 1339,
+    "振": 1340,
+    "答": 1341,
+    "筹": 1342,
+    "狱": 1343,
+    "仙": 1344,
+    "祭": 1345,
+    "妹": 1346,
+    "洞": 1347,
+    "寿": 1348,
+    "池": 1349,
+    "遍": 1350,
+    "浪": 1351,
+    "誉": 1352,
+    "伍": 1353,
+    "幸": 1354,
+    "残": 1355,
+    "蔡": 1356,
+    "茨": 1357,
+    "缘": 1358,
+    "迷": 1359,
+    "沉": 1360,
+    "柱": 1361,
+    "冬": 1362,
+    "?": 1363,
+    "潜": 1364,
+    "铜": 1365,
+    "烧": 1366,
+    "漫": 1367,
+    "析": 1368,
+    "颁": 1369,
+    "闭": 1370,
+    "遂": 1371,
+    "仰": 1372,
+    "邓": 1373,
+    "钢": 1374,
+    "截": 1375,
+    "纵": 1376,
+    "逊": 1377,
+    "违": 1378,
+    "辽": 1379,
+    "拔": 1380,
+    "埔": 1381,
+    "迅": 1382,
+    "浙": 1383,
+    "酸": 1384,
+    "赤": 1385,
+    "燃": 1386,
+    "凤": 1387,
+    "幅": 1388,
+    "吕": 1389,
+    "梦": 1390,
+    "苦": 1391,
+    "伟": 1392,
+    "浦": 1393,
+    "麻": 1394,
+    "虑": 1395,
+    "缩": 1396,
+    "仔": 1397,
+    "绪": 1398,
+    "戈": 1399,
+    "烟": 1400,
+    "纷": 1401,
+    "勇": 1402,
+    "享": 1403,
+    "沈": 1404,
+    "龄": 1405,
+    "厚": 1406,
+    "桑": 1407,
+    "衡": 1408,
+    "句": 1409,
+    "伴": 1410,
+    "绘": 1411,
+    "允": 1412,
+    "莎": 1413,
+    "盐": 1414,
+    "颜": 1415,
+    "豪": 1416,
+    "琴": 1417,
+    "液": 1418,
+    "牧": 1419,
+    "综": 1420,
+    "!": 1421,
+    "拆": 1422,
+    "隐": 1423,
+    "宿": 1424,
+    "谓": 1425,
+    "硬": 1426,
+    "脉": 1427,
+    "葡": 1428,
+    "纯": 1429,
+    "宅": 1430,
+    "殊": 1431,
+    "旁": 1432,
+    "册": 1433,
+    "折": 1434,
+    "慢": 1435,
+    "厦": 1436,
+    "彼": 1437,
+    "飓": 1438,
+    "疆": 1439,
+    "捷": 1440,
+    "俱": 1441,
+    "徽": 1442,
+    "祥": 1443,
+    "汗": 1444,
+    "辉": 1445,
+    "滨": 1446,
+    "跑": 1447,
+    "孝": 1448,
+    "熙": 1449,
+    "敬": 1450,
+    "叙": 1451,
+    "零": 1452,
+    "挑": 1453,
+    "珍": 1454,
+    "缓": 1455,
+    "裔": 1456,
+    "吃": 1457,
+    "敏": 1458,
+    "冯": 1459,
+    "冈": 1460,
+    "亨": 1461,
+    "瑟": 1462,
+    "跨": 1463,
+    "岭": 1464,
+    "锋": 1465,
+    "刀": 1466,
+    "驶": 1467,
+    "茶": 1468,
+    "滑": 1469,
+    "赏": 1470,
+    "萧": 1471,
+    "撒": 1472,
+    "倾": 1473,
+    "坛": 1474,
+    "逝": 1475,
+    "掉": 1476,
+    "侍": 1477,
+    "氧": 1478,
+    "柯": 1479,
+    "磁": 1480,
+    "灯": 1481,
+    "仓": 1482,
+    "慕": 1483,
+    "碑": 1484,
+    "晨": 1485,
+    "丘": 1486,
+    "绕": 1487,
+    "艇": 1488,
+    "覆": 1489,
+    "械": 1490,
+    "档": 1491,
+    "餐": 1492,
+    "昆": 1493,
+    "伏": 1494,
+    "熟": 1495,
+    "榜": 1496,
+    "雕": 1497,
+    "贯": 1498,
+    "燕": 1499,
+    "糖": 1500,
+    "诚": 1501,
+    "软": 1502,
+    "汤": 1503,
+    "虫": 1504,
+    "疾": 1505,
+    "痛": 1506,
+    "葬": 1507,
+    "朋": 1508,
+    "凡": 1509,
+    "慈": 1510,
+    "孟": 1511,
+    "欲": 1512,
+    "壁": 1513,
+    "搭": 1514,
+    "贤": 1515,
+    "胁": 1516,
+    "桃": 1517,
+    "偶": 1518,
+    "恋": 1519,
+    "乏": 1520,
+    "邮": 1521,
+    "桂": 1522,
+    "袁": 1523,
+    "菜": 1524,
+    "藤": 1525,
+    "贺": 1526,
+    "惊": 1527,
+    "虚": 1528,
+    "莲": 1529,
+    "览": 1530,
+    "倍": 1531,
+    "猪": 1532,
+    "猎": 1533,
+    "戒": 1534,
+    "勋": 1535,
+    "驾": 1536,
+    "粉": 1537,
+    "丧": 1538,
+    "藩": 1539,
+    "姐": 1540,
+    "趣": 1541,
+    "罚": 1542,
+    "勤": 1543,
+    "坑": 1544,
+    "妮": 1545,
+    "玄": 1546,
+    "储": 1547,
+    "饮": 1548,
+    "蛋": 1549,
+    "萄": 1550,
+    "塘": 1551,
+    "凌": 1552,
+    "彭": 1553,
+    "番": 1554,
+    "措": 1555,
+    "撞": 1556,
+    "峡": 1557,
+    "匹": 1558,
+    "励": 1559,
+    "葛": 1560,
+    "拟": 1561,
+    "迈": 1562,
+    "沟": 1563,
+    "抚": 1564,
+    "轴": 1565,
+    "伸": 1566,
+    "屯": 1567,
+    "既": 1568,
+    "赶": 1569,
+    "剂": 1570,
+    "奈": 1571,
+    "鬼": 1572,
+    "旦": 1573,
+    "粮": 1574,
+    "僧": 1575,
+    "淡": 1576,
+    "挂": 1577,
+    "罕": 1578,
+    "纲": 1579,
+    "岳": 1580,
+    "污": 1581,
+    "陕": 1582,
+    "忆": 1583,
+    "袖": 1584,
+    "牠": 1585,
+    "紫": 1586,
+    "贾": 1587,
+    "扰": 1588,
+    "累": 1589,
+    "菌": 1590,
+    "键": 1591,
+    "仲": 1592,
+    "吨": 1593,
+    "鉴": 1594,
+    "笼": 1595,
+    "琳": 1596,
+    "跳": 1597,
+    "昂": 1598,
+    "晓": 1599,
+    "乙": 1600,
+    "兽": 1601,
+    "灰": 1602,
+    "町": 1603,
+    "涯": 1604,
+    "禄": 1605,
+    "凭": 1606,
+    "洗": 1607,
+    "怒": 1608,
+    "尖": 1609,
+    "箭": 1610,
+    "赐": 1611,
+    "卓": 1612,
+    "衰": 1613,
+    "辩": 1614,
+    "潘": 1615,
+    "彻": 1616,
+    "诞": 1617,
+    "尉": 1618,
+    "芳": 1619,
+    "颗": 1620,
+    "搜": 1621,
+    "胶": 1622,
+    "炎": 1623,
+    "怪": 1624,
+    "锁": 1625,
+    "浮": 1626,
+    "梯": 1627,
+    "塑": 1628,
+    "鹿": 1629,
+    "『": 1630,
+    "羊": 1631,
+    "阮": 1632,
+    "』": 1633,
+    "谱": 1634,
+    "捐": 1635,
+    "俊": 1636,
+    "俘": 1637,
+    "祝": 1638,
+    "详": 1639,
+    "苗": 1640,
+    "匈": 1641,
+    "熊": 1642,
+    "乾": 1643,
+    "粒": 1644,
+    "衔": 1645,
+    "剩": 1646,
+    "帅": 1647,
+    "轰": 1648,
+    "圳": 1649,
+    "盾": 1650,
+    "隧": 1651,
+    "逢": 1652,
+    "幻": 1653,
+    "割": 1654,
+    "耀": 1655,
+    "盗": 1656,
+    "粤": 1657,
+    "荒": 1658,
+    "咸": 1659,
+    "拓": 1660,
+    "齿": 1661,
+    "泥": 1662,
+    "跃": 1663,
+    "乳": 1664,
+    "贫": 1665,
+    "闽": 1666,
+    "询": 1667,
+    "柳": 1668,
+    "枢": 1669,
+    "弥": 1670,
+    "巧": 1671,
+    "彰": 1672,
+    "渔": 1673,
+    "儒": 1674,
+    "〈": 1675,
+    "函": 1676,
+    "〉": 1677,
+    "笑": 1678,
+    "撰": 1679,
+    "抢": 1680,
+    "詹": 1681,
+    "贞": 1682,
+    "契": 1683,
+    "旺": 1684,
+    "~": 1685,
+    "慧": 1686,
+    "尸": 1687,
+    "尝": 1688,
+    "缅": 1689,
+    "屿": 1690,
+    "仑": 1691,
+    "鸿": 1692,
+    "握": 1693,
+    "阅": 1694,
+    "侦": 1695,
+    "聘": 1696,
+    "杭": 1697,
+    "疏": 1698,
+    "肃": 1699,
+    "搬": 1700,
+    "瓜": 1701,
+    "棒": 1702,
+    "逼": 1703,
+    "淮": 1704,
+    "悬": 1705,
+    "厘": 1706,
+    "宙": 1707,
+    "仿": 1708,
+    "劝": 1709,
+    "屏": 1710,
+    "丈": 1711,
+    "裕": 1712,
+    "郊": 1713,
+    "伙": 1714,
+    "碍": 1715,
+    "壮": 1716,
+    "卿": 1717,
+    "霸": 1718,
+    "杉": 1719,
+    "叔": 1720,
+    "弘": 1721,
+    "牵": 1722,
+    "芝": 1723,
+    "夷": 1724,
+    "罢": 1725,
+    "崔": 1726,
+    "循": 1727,
+    "柴": 1728,
+    "孤": 1729,
+    "勃": 1730,
+    "浓": 1731,
+    "砲": 1732,
+    "硕": 1733,
+    "拳": 1734,
+    "扮": 1735,
+    "湘": 1736,
+    "钦": 1737,
+    "粹": 1738,
+    "惯": 1739,
+    "晶": 1740,
+    "亭": 1741,
+    "禅": 1742,
+    "纹": 1743,
+    "秒": 1744,
+    "枚": 1745,
+    "迟": 1746,
+    "傅": 1747,
+    "押": 1748,
+    "岗": 1749,
+    "旨": 1750,
+    "宏": 1751,
+    "靖": 1752,
+    "踪": 1753,
+    "床": 1754,
+    "填": 1755,
+    "琉": 1756,
+    "茂": 1757,
+    "妃": 1758,
+    "贴": 1759,
+    "偷": 1760,
+    "赠": 1761,
+    "猫": 1762,
+    "逮": 1763,
+    "疫": 1764,
+    "悉": 1765,
+    "饭": 1766,
+    "狂": 1767,
+    "邨": 1768,
+    "忍": 1769,
+    "甸": 1770,
+    "煤": 1771,
+    "貌": 1772,
+    "侠": 1773,
+    "抽": 1774,
+    "赋": 1775,
+    "净": 1776,
+    "庞": 1777,
+    "姻": 1778,
+    "稿": 1779,
+    "坊": 1780,
+    "寒": 1781,
+    "棋": 1782,
+    "玻": 1783,
+    "颇": 1784,
+    "翌": 1785,
+    "箱": 1786,
+    "崎": 1787,
+    "伪": 1788,
+    "涅": 1789,
+    "娱": 1790,
+    "妈": 1791,
+    "揭": 1792,
+    "铸": 1793,
+    "肥": 1794,
+    "喀": 1795,
+    "诏": 1796,
+    "蛇": 1797,
+    "扶": 1798,
+    "屠": 1799,
+    "遵": 1800,
+    "劫": 1801,
+    "吐": 1802,
+    "雇": 1803,
+    "鸡": 1804,
+    "浅": 1805,
+    "趋": 1806,
+    "递": 1807,
+    "胎": 1808,
+    "耗": 1809,
+    "嫌": 1810,
+    "娘": 1811,
+    "泊": 1812,
+    "焦": 1813,
+    "敢": 1814,
+    "浩": 1815,
+    "洁": 1816,
+    "偿": 1817,
+    "湿": 1818,
+    "狮": 1819,
+    "寄": 1820,
+    "舒": 1821,
+    "祀": 1822,
+    "曰": 1823,
+    "吾": 1824,
+    "稍": 1825,
+    "稣": 1826,
+    "陀": 1827,
+    "愈": 1828,
+    "冒": 1829,
+    "插": 1830,
+    "猛": 1831,
+    "狄": 1832,
+    "旬": 1833,
+    "卜": 1834,
+    "摇": 1835,
+    "涌": 1836,
+    "惨": 1837,
+    "怕": 1838,
+    "券": 1839,
+    "廊": 1840,
+    "垒": 1841,
+    "鲍": 1842,
+    "薄": 1843,
+    "枝": 1844,
+    "篮": 1845,
+    "窗": 1846,
+    "怖": 1847,
+    "喷": 1848,
+    "仇": 1849,
+    "魂": 1850,
+    "荐": 1851,
+    "铺": 1852,
+    "奔": 1853,
+    "润": 1854,
+    "惟": 1855,
+    "歧": 1856,
+    "碎": 1857,
+    "腹": 1858,
+    "扫": 1859,
+    "串": 1860,
+    "铭": 1861,
+    "醒": 1862,
+    "腐": 1863,
+    "欣": 1864,
+    "垂": 1865,
+    "壤": 1866,
+    "朴": 1867,
+    "矛": 1868,
+    "债": 1869,
+    "婆": 1870,
+    "摆": 1871,
+    "潭": 1872,
+    "滩": 1873,
+    "刷": 1874,
+    "悲": 1875,
+    "鸣": 1876,
+    "堆": 1877,
+    "泳": 1878,
+    "吹": 1879,
+    "汪": 1880,
+    "碳": 1881,
+    "侨": 1882,
+    "彦": 1883,
+    "逆": 1884,
+    "壳": 1885,
+    "抓": 1886,
+    "柔": 1887,
+    "杆": 1888,
+    "倡": 1889,
+    "–": 1890,
+    "页": 1891,
+    "鹰": 1892,
+    "狗": 1893,
+    "兆": 1894,
+    "崩": 1895,
+    "忽": 1896,
+    "璃": 1897,
+    "豆": 1898,
+    "竟": 1899,
+    "阔": 1900,
+    "嗣": 1901,
+    "脏": 1902,
+    "携": 1903,
+    "宰": 1904,
+    "豫": 1905,
+    "忙": 1906,
+    "夕": 1907,
+    "邪": 1908,
+    "堪": 1909,
+    "寨": 1910,
+    "抱": 1911,
+    "斜": 1912,
+    "溃": 1913,
+    "迦": 1914,
+    "淘": 1915,
+    "炉": 1916,
+    "衍": 1917,
+    "辟": 1918,
+    "穷": 1919,
+    "蒸": 1920,
+    "翁": 1921,
+    "隋": 1922,
+    "腾": 1923,
+    "逻": 1924,
+    "诊": 1925,
+    "炼": 1926,
+    "─": 1927,
+    "滚": 1928,
+    "龟": 1929,
+    "姚": 1930,
+    "掘": 1931,
+    "毅": 1932,
+    "吏": 1933,
+    "坎": 1934,
+    "膜": 1935,
+    "斩": 1936,
+    "耕": 1937,
+    "妙": 1938,
+    "苑": 1939,
+    "鼎": 1940,
+    "削": 1941,
+    "陪": 1942,
+    "埋": 1943,
+    "掠": 1944,
+    "添": 1945,
+    "凝": 1946,
+    "栏": 1947,
+    "涵": 1948,
+    "摧": 1949,
+    "珊": 1950,
+    "狼": 1951,
+    "姬": 1952,
+    "癌": 1953,
+    "拖": 1954,
+    "奋": 1955,
+    "孕": 1956,
+    "沪": 1957,
+    "薪": 1958,
+    "磨": 1959,
+    "薛": 1960,
+    "嫁": 1961,
+    "拨": 1962,
+    "愤": 1963,
+    "拘": 1964,
+    "寸": 1965,
+    "灌": 1966,
+    "吞": 1967,
+    "坪": 1968,
+    "螺": 1969,
+    "丸": 1970,
+    "韵": 1971,
+    "巫": 1972,
+    "稻": 1973,
+    "歇": 1974,
+    "弦": 1975,
+    "伽": 1976,
+    "募": 1977,
+    "甫": 1978,
+    "鼠": 1979,
+    "荡": 1980,
+    "挪": 1981,
+    "绳": 1982,
+    "’": 1983,
+    "厄": 1984,
+    "扣": 1985,
+    "栖": 1986,
+    "琼": 1987,
+    "掩": 1988,
+    "驳": 1989,
+    "舱": 1990,
+    "狭": 1991,
+    "誓": 1992,
+    "寮": 1993,
+    "‘": 1994,
+    "屈": 1995,
+    "雍": 1996,
+    "麟": 1997,
+    "噶": 1998,
+    "砖": 1999,
+    "寓": 2000,
+    "雾": 2001,
+    "盆": 2002,
+    "奶": 2003,
+    "蜀": 2004,
+    "脊": 2005,
+    "辐": 2006,
+    "谭": 2007,
+    "厢": 2008,
+    "渊": 2009,
+    "溶": 2010,
+    "凉": 2011,
+    "挖": 2012,
+    "魁": 2013,
+    "鄂": 2014,
+    "涂": 2015,
+    "贩": 2016,
+    "扁": 2017,
+    "舟": 2018,
+    "喇": 2019,
+    "坝": 2020,
+    "荆": 2021,
+    "婴": 2022,
+    "吁": 2023,
+    "幽": 2024,
+    "邑": 2025,
+    "粗": 2026,
+    "鹤": 2027,
+    "斥": 2028,
+    "袋": 2029,
+    "忧": 2030,
+    "鼻": 2031,
+    "栋": 2032,
+    "纬": 2033,
+    "姊": 2034,
+    "肌": 2035,
+    "汰": 2036,
+    "斋": 2037,
+    "恭": 2038,
+    "贪": 2039,
+    "脂": 2040,
+    "奸": 2041,
+    "氢": 2042,
+    "漠": 2043,
+    "矶": 2044,
+    "胸": 2045,
+    "诱": 2046,
+    "祠": 2047,
+    "卵": 2048,
+    "忘": 2049,
+    "暖": 2050,
+    "咨": 2051,
+    "桓": 2052,
+    "聪": 2053,
+    "趁": 2054,
+    "脸": 2055,
+    "抑": 2056,
+    "廖": 2057,
+    "溯": 2058,
+    "卑": 2059,
+    "泡": 2060,
+    "翔": 2061,
+    "谦": 2062,
+    "斤": 2063,
+    "郁": 2064,
+    "宴": 2065,
+    "斑": 2066,
+    "犬": 2067,
+    "贼": 2068,
+    "腓": 2069,
+    "襄": 2070,
+    "惧": 2071,
+    "恨": 2072,
+    "叠": 2073,
+    "碧": 2074,
+    "恰": 2075,
+    "顷": 2076,
+    "娃": 2077,
+    "酷": 2078,
+    "夹": 2079,
+    "爷": 2080,
+    "擅": 2081,
+    "雌": 2082,
+    "哪": 2083,
+    "链": 2084,
+    "谁": 2085,
+    " ": 2086,
+    "牲": 2087,
+    "慰": 2088,
+    "钻": 2089,
+    "辰": 2090,
+    "砂": 2091,
+    "闸": 2092,
+    "闲": 2093,
+    "惜": 2094,
+    "冀": 2095,
+    "辨": 2096,
+    "锐": 2097,
+    "尹": 2098,
+    "嘴": 2099,
+    "瓶": 2100,
+    "肖": 2101,
+    "挡": 2102,
+    "翠": 2103,
+    "披": 2104,
+    "姜": 2105,
+    "凶": 2106,
+    "奎": 2107,
+    "瓷": 2108,
+    "奠": 2109,
+    "劲": 2110,
+    "贷": 2111,
+    "绥": 2112,
+    "腔": 2113,
+    "巢": 2114,
+    "厝": 2115,
+    "焚": 2116,
+    "姑": 2117,
+    "踏": 2118,
+    "剪": 2119,
+    "劣": 2120,
+    "斐": 2121,
+    "阪": 2122,
+    "缴": 2123,
+    "赔": 2124,
+    "帽": 2125,
+    "菩": 2126,
+    "弯": 2127,
+    "碰": 2128,
+    "丞": 2129,
+    "蜂": 2130,
+    "邱": 2131,
+    "骚": 2132,
+    "亩": 2133,
+    "囚": 2134,
+    "卸": 2135,
+    "°": 2136,
+    "宠": 2137,
+    "纠": 2138,
+    "践": 2139,
+    "株": 2140,
+    "睡": 2141,
+    "渠": 2142,
+    "拼": 2143,
+    "闪": 2144,
+    "慎": 2145,
+    "'": 2146,
+    "棉": 2147,
+    "侣": 2148,
+    "冕": 2149,
+    "屡": 2150,
+    "殷": 2151,
+    "骗": 2152,
+    "宛": 2153,
+    "虏": 2154,
+    "蹈": 2155,
+    "虹": 2156,
+    "惩": 2157,
+    "巷": 2158,
+    "颠": 2159,
+    "绑": 2160,
+    "谊": 2161,
+    "怎": 2162,
+    "耐": 2163,
+    "丑": 2164,
+    "忌": 2165,
+    "逾": 2166,
+    "擎": 2167,
+    "胆": 2168,
+    "茅": 2169,
+    "庇": 2170,
+    "秩": 2171,
+    "辈": 2172,
+    "涛": 2173,
+    "赌": 2174,
+    "栗": 2175,
+    "矩": 2176,
+    "舆": 2177,
+    "亥": 2178,
+    "帐": 2179,
+    "稀": 2180,
+    "姿": 2181,
+    "遥": 2182,
+    "祸": 2183,
+    "拱": 2184,
+    "厉": 2185,
+    "跌": 2186,
+    "鲸": 2187,
+    "颈": 2188,
+    "腿": 2189,
+    "蕾": 2190,
+    "鸦": 2191,
+    "祈": 2192,
+    "抛": 2193,
+    "厥": 2194,
+    "蕃": 2195,
+    "澄": 2196,
+    "琪": 2197,
+    "娶": 2198,
+    "讼": 2199,
+    "卒": 2200,
+    "吧": 2201,
+    "弓": 2202,
+    "堤": 2203,
+    "催": 2204,
+    "淑": 2205,
+    "抄": 2206,
+    "叉": 2207,
+    "蜜": 2208,
+    "悟": 2209,
+    "仆": 2210,
+    "汀": 2211,
+    "鹏": 2212,
+    "漏": 2213,
+    "逸": 2214,
+    "肢": 2215,
+    "肺": 2216,
+    "哀": 2217,
+    "欠": 2218,
+    "汝": 2219,
+    "雀": 2220,
+    "筒": 2221,
+    "镑": 2222,
+    "庸": 2223,
+    "剥": 2224,
+    "浸": 2225,
+    "弄": 2226,
+    "珀": 2227,
+    "禧": 2228,
+    "暨": 2229,
+    "囊": 2230,
+    "涨": 2231,
+    "僚": 2232,
+    "尿": 2233,
+    "纤": 2234,
+    "歼": 2235,
+    "悠": 2236,
+    "阀": 2237,
+    "硫": 2238,
+    "邵": 2239,
+    "咖": 2240,
+    "怨": 2241,
+    "蓄": 2242,
+    "胀": 2243,
+    "穴": 2244,
+    "霞": 2245,
+    "挤": 2246,
+    "吊": 2247,
+    "鸭": 2248,
+    "炭": 2249,
+    "畅": 2250,
+    "匪": 2251,
+    "梵": 2252,
+    "丛": 2253,
+    "躲": 2254,
+    "蒲": 2255,
+    "泄": 2256,
+    "坟": 2257,
+    "豹": 2258,
+    "寇": 2259,
+    "垦": 2260,
+    "肤": 2261,
+    "讽": 2262,
+    "丙": 2263,
+    "妖": 2264,
+    "玲": 2265,
+    "℃": 2266,
+    "肠": 2267,
+    "尘": 2268,
+    "伞": 2269,
+    "歉": 2270,
+    "勘": 2271,
+    "坂": 2272,
+    "挺": 2273,
+    "扑": 2274,
+    "妥": 2275,
+    "塌": 2276,
+    "盈": 2277,
+    "肩": 2278,
+    "铃": 2279,
+    "厌": 2280,
+    "臂": 2281,
+    "辱": 2282,
+    "喝": 2283,
+    "旱": 2284,
+    "蚀": 2285,
+    "帆": 2286,
+    "遮": 2287,
+    "嘛": 2288,
+    "仕": 2289,
+    "夸": 2290,
+    "昏": 2291,
+    "腺": 2292,
+    "锅": 2293,
+    "饥": 2294,
+    "贬": 2295,
+    "滥": 2296,
+    "饼": 2297,
+    "纺": 2298,
+    "谴": 2299,
+    "畏": 2300,
+    "轩": 2301,
+    "悦": 2302,
+    "徙": 2303,
+    "牺": 2304,
+    "颂": 2305,
+    "巩": 2306,
+    "札": 2307,
+    "肇": 2308,
+    "窄": 2309,
+    "骏": 2310,
+    "猜": 2311,
+    "唤": 2312,
+    "谨": 2313,
+    "卧": 2314,
+    "璋": 2315,
+    "怡": 2316,
+    "赣": 2317,
+    "椎": 2318,
+    "瑜": 2319,
+    "啡": 2320,
+    "剿": 2321,
+    "蓬": 2322,
+    "搞": 2323,
+    "昔": 2324,
+    "颖": 2325,
+    "薇": 2326,
+    "钓": 2327,
+    "爪": 2328,
+    "腰": 2329,
+    "圭": 2330,
+    "捉": 2331,
+    "滋": 2332,
+    "戚": 2333,
+    "删": 2334,
+    "丢": 2335,
+    "礁": 2336,
+    "槽": 2337,
+    "爬": 2338,
+    "嫩": 2339,
+    "芦": 2340,
+    "谐": 2341,
+    "朵": 2342,
+    "庶": 2343,
+    "撑": 2344,
+    "缝": 2345,
+    "肝": 2346,
+    "葵": 2347,
+    "尧": 2348,
+    "柜": 2349,
+    "佣": 2350,
+    "贿": 2351,
+    "晴": 2352,
+    "冻": 2353,
+    "缔": 2354,
+    "喻": 2355,
+    "坤": 2356,
+    "矮": 2357,
+    "扭": 2358,
+    "框": 2359,
+    "芒": 2360,
+    "肿": 2361,
+    "谣": 2362,
+    "汶": 2363,
+    "绵": 2364,
+    "荃": 2365,
+    "坞": 2366,
+    "履": 2367,
+    "漂": 2368,
+    "虐": 2369,
+    "坠": 2370,
+    "蛮": 2371,
+    "擦": 2372,
+    "埠": 2373,
+    "醉": 2374,
+    "欺": 2375,
+    "畜": 2376,
+    "痕": 2377,
+    "崖": 2378,
+    "醇": 2379,
+    "橡": 2380,
+    "柬": 2381,
+    "滕": 2382,
+    "沦": 2383,
+    "甜": 2384,
+    "煌": 2385,
+    "窝": 2386,
+    "阎": 2387,
+    "咒": 2388,
+    "о": 2389,
+    "艰": 2390,
+    "呎": 2391,
+    "佑": 2392,
+    "牢": 2393,
+    "澎": 2394,
+    "炳": 2395,
+    "勾": 2396,
+    "桌": 2397,
+    "苹": 2398,
+    "茵": 2399,
+    "鹅": 2400,
+    "吓": 2401,
+    "梨": 2402,
+    "赦": 2403,
+    "菊": 2404,
+    "蝠": 2405,
+    "鞋": 2406,
+    "荫": 2407,
+    "淹": 2408,
+    "・": 2409,
+    "酿": 2410,
+    "竣": 2411,
+    "踢": 2412,
+    "毗": 2413,
+    "睛": 2414,
+    "祷": 2415,
+    "眠": 2416,
+    "哭": 2417,
+    "闹": 2418,
+    "秉": 2419,
+    "桐": 2420,
+    "攀": 2421,
+    "饶": 2422,
+    "铎": 2423,
+    "霖": 2424,
+    "疯": 2425,
+    "拦": 2426,
+    "膨": 2427,
+    "祐": 2428,
+    "饱": 2429,
+    "匠": 2430,
+    "眉": 2431,
+    "芙": 2432,
+    "瘤": 2433,
+    "椅": 2434,
+    "棕": 2435,
+    "蝙": 2436,
+    "扇": 2437,
+    "窑": 2438,
+    "泪": 2439,
+    "幢": 2440,
+    "仗": 2441,
+    "肆": 2442,
+    "堵": 2443,
+    "茎": 2444,
+    "拯": 2445,
+    "跋": 2446,
+    "钧": 2447,
+    "芭": 2448,
+    "爸": 2449,
+    "娅": 2450,
+    "闯": 2451,
+    "斌": 2452,
+    "骂": 2453,
+    "绅": 2454,
+    "畔": 2455,
+    "苍": 2456,
+    "糊": 2457,
+    "烦": 2458,
+    "裴": 2459,
+    "+": 2460,
+    "滞": 2461,
+    "沼": 2462,
+    "碱": 2463,
+    "咏": 2464,
+    "阐": 2465,
+    "狐": 2466,
+    "氛": 2467,
+    "а": 2468,
+    "妓": 2469,
+    "浑": 2470,
+    "吻": 2471,
+    "棚": 2472,
+    "帖": 2473,
+    "氨": 2474,
+    "浆": 2475,
+    "懂": 2476,
+    "涡": 2477,
+    "惑": 2478,
+    "盲": 2479,
+    "庐": 2480,
+    "谕": 2481,
+    "窃": 2482,
+    "宦": 2483,
+    "祯": 2484,
+    "帜": 2485,
+    "漳": 2486,
+    "虞": 2487,
+    "冥": 2488,
+    "聂": 2489,
+    "烂": 2490,
+    "罩": 2491,
+    "糕": 2492,
+    "悼": 2493,
+    "酬": 2494,
+    "厨": 2495,
+    "黛": 2496,
+    "熔": 2497,
+    "蝶": 2498,
+    "漆": 2499,
+    "柄": 2500,
+    "喊": 2501,
+    "谍": 2502,
+    "凰": 2503,
+    "樱": 2504,
+    "诈": 2505,
+    "淳": 2506,
+    "掷": 2507,
+    "牟": 2508,
+    "彪": 2509,
+    "兔": 2510,
+    "艳": 2511,
+    "铅": 2512,
+    "吗": 2513,
+    "氯": 2514,
+    "朔": 2515,
+    "寡": 2516,
+    "禹": 2517,
+    "鞍": 2518,
+    "浴": 2519,
+    "猴": 2520,
+    "驰": 2521,
+    "渗": 2522,
+    "酶": 2523,
+    "笃": 2524,
+    "垄": 2525,
+    "‧": 2526,
+    "萝": 2527,
+    "庚": 2528,
+    "勉": 2529,
+    "苯": 2530,
+    "哺": 2531,
+    "酋": 2532,
+    "庵": 2533,
+    "舌": 2534,
+    "榄": 2535,
+    "焰": 2536,
+    "诵": 2537,
+    "и": 2538,
+    "瑶": 2539,
+    "藻": 2540,
+    "冶": 2541,
+    "岐": 2542,
+    "缠": 2543,
+    "陨": 2544,
+    "锣": 2545,
+    "峻": 2546,
+    "窟": 2547,
+    "邸": 2548,
+    "彬": 2549,
+    "旭": 2550,
+    "藉": 2551,
+    "凸": 2552,
+    "悔": 2553,
+    "枫": 2554,
+    "呢": 2555,
+    "砍": 2556,
+    "滤": 2557,
+    "褐": 2558,
+    "垣": 2559,
+    "滴": 2560,
+    "赚": 2561,
+    "弊": 2562,
+    "缉": 2563,
+    "毙": 2564,
+    "孜": 2565,
+    "裸": 2566,
+    "叹": 2567,
+    "饲": 2568,
+    "骤": 2569,
+    "皖": 2570,
+    "劾": 2571,
+    "翅": 2572,
+    "虾": 2573,
+    "畴": 2574,
+    "乞": 2575,
+    "赎": 2576,
+    "擒": 2577,
+    "衙": 2578,
+    "诛": 2579,
+    "傲": 2580,
+    "肾": 2581,
+    "淫": 2582,
+    "焕": 2583,
+    "淀": 2584,
+    "稽": 2585,
+    "磷": 2586,
+    "廿": 2587,
+    "倪": 2588,
+    "衷": 2589,
+    "谥": 2590,
+    "咀": 2591,
+    "沛": 2592,
+    "纂": 2593,
+    "烯": 2594,
+    "墟": 2595,
+    "蔽": 2596,
+    "鞭": 2597,
+    "犁": 2598,
+    "挫": 2599,
+    "摘": 2600,
+    "倭": 2601,
+    "汕": 2602,
+    "匿": 2603,
+    "闵": 2604,
+    "婉": 2605,
+    "镶": 2606,
+    "麓": 2607,
+    "寂": 2608,
+    "胃": 2609,
+    "挽": 2610,
+    "狩": 2611,
+    "矢": 2612,
+    "垃": 2613,
+    "唇": 2614,
+    "戎": 2615,
+    "е": 2616,
+    "喉": 2617,
+    "耿": 2618,
+    "燥": 2619,
+    "饿": 2620,
+    "蔓": 2621,
+    "胺": 2622,
+    "碟": 2623,
+    "纱": 2624,
+    "棠": 2625,
+    "爽": 2626,
+    "窦": 2627,
+    "暑": 2628,
+    "粘": 2629,
+    "圾": 2630,
+    "冢": 2631,
+    "僵": 2632,
+    "笛": 2633,
+    "卦": 2634,
+    "彗": 2635,
+    "侄": 2636,
+    "竭": 2637,
+    "茄": 2638,
+    "嵌": 2639,
+    "簧": 2640,
+    "堕": 2641,
+    "寅": 2642,
+    "拾": 2643,
+    "缆": 2644,
+    "铝": 2645,
+    "筋": 2646,
+    "噪": 2647,
+    "沧": 2648,
+    "濒": 2649,
+    "盒": 2650,
+    "殉": 2651,
+    "梭": 2652,
+    "桶": 2653,
+    "迄": 2654,
+    "谏": 2655,
+    "棱": 2656,
+    "矣": 2657,
+    "袍": 2658,
+    "亏": 2659,
+    "疲": 2660,
+    "浊": 2661,
+    "抬": 2662,
+    "崛": 2663,
+    "嫡": 2664,
+    "啸": 2665,
+    "诠": 2666,
+    "峙": 2667,
+    "羌": 2668,
+    "槟": 2669,
+    "滇": 2670,
+    "趾": 2671,
+    "兑": 2672,
+    "妆": 2673,
+    "舅": 2674,
+    "萍": 2675,
+    "竖": 2676,
+    "阜": 2677,
+    "弼": 2678,
+    "凿": 2679,
+    "陇": 2680,
+    "弧": 2681,
+    "雁": 2682,
+    "煮": 2683,
+    "湛": 2684,
+    "愚": 2685,
+    "韶": 2686,
+    "棍": 2687,
+    "芜": 2688,
+    "咬": 2689,
+    "稚": 2690,
+    "琦": 2691,
+    "绣": 2692,
+    "骸": 2693,
+    "曝": 2694,
+    "舜": 2695,
+    "黏": 2696,
+    "镰": 2697,
+    "霜": 2698,
+    "瑰": 2699,
+    "辣": 2700,
+    "粟": 2701,
+    "瑚": 2702,
+    "匡": 2703,
+    "鳍": 2704,
+    "搁": 2705,
+    "泌": 2706,
+    "辜": 2707,
+    "鳄": 2708,
+    "钩": 2709,
+    "摊": 2710,
+    "汁": 2711,
+    "蛛": 2712,
+    "邯": 2713,
+    "孚": 2714,
+    "俞": 2715,
+    "隙": 2716,
+    "禾": 2717,
+    "豚": 2718,
+    "鳞": 2719,
+    "蜥": 2720,
+    "荔": 2721,
+    "摔": 2722,
+    "懿": 2723,
+    "н": 2724,
+    "挝": 2725,
+    "栽": 2726,
+    "敷": 2727,
+    "蕴": 2728,
+    "椒": 2729,
+    "奕": 2730,
+    "氮": 2731,
+    "啤": 2732,
+    "嵩": 2733,
+    "谜": 2734,
+    "颅": 2735,
+    "胤": 2736,
+    "杏": 2737,
+    "眷": 2738,
+    "邹": 2739,
+    "蓉": 2740,
+    "榴": 2741,
+    "毓": 2742,
+    "渤": 2743,
+    "罹": 2744,
+    "酵": 2745,
+    "芽": 2746,
+    "鲨": 2747,
+    "蔬": 2748,
+    "龚": 2749,
+    "穗": 2750,
+    "愉": 2751,
+    "殴": 2752,
+    "钞": 2753,
+    "厕": 2754,
+    "匾": 2755,
+    "绸": 2756,
+    "锥": 2757,
+    "雏": 2758,
+    "磅": 2759,
+    "慌": 2760,
+    "钉": 2761,
+    "傍": 2762,
+    "晖": 2763,
+    "褒": 2764,
+    "凹": 2765,
+    "裤": 2766,
+    "烤": 2767,
+    "玫": 2768,
+    "袜": 2769,
+    "敕": 2770,
+    "渴": 2771,
+    "剌": 2772,
+    "澜": 2773,
+    "の": 2774,
+    "罐": 2775,
+    "铀": 2776,
+    "→": 2777,
+    "壶": 2778,
+    "淞": 2779,
+    "耻": 2780,
+    "禽": 2781,
+    "烷": 2782,
+    "哨": 2783,
+    "晤": 2784,
+    "萌": 2785,
+    "廓": 2786,
+    "账": 2787,
+    "晰": 2788,
+    "轿": 2789,
+    "郝": 2790,
+    "顽": 2791,
+    "飘": 2792,
+    "骆": 2793,
+    "俩": 2794,
+    "巾": 2795,
+    "砌": 2796,
+    "×": 2797,
+    "沫": 2798,
+    "缪": 2799,
+    "墩": 2800,
+    "垫": 2801,
+    "蛙": 2802,
+    "聊": 2803,
+    "讳": 2804,
+    "р": 2805,
+    "谅": 2806,
+    "臭": 2807,
+    "驼": 2808,
+    "梓": 2809,
+    "猩": 2810,
+    "酱": 2811,
+    "剖": 2812,
+    "暹": 2813,
+    "瞬": 2814,
+    "疼": 2815,
+    "橄": 2816,
+    "膀": 2817,
+    "蔚": 2818,
+    "渭": 2819,
+    "抹": 2820,
+    "肚": 2821,
+    "樊": 2822,
+    "敲": 2823,
+    "墅": 2824,
+    "祺": 2825,
+    "莽": 2826,
+    "寝": 2827,
+    "蟹": 2828,
+    "棺": 2829,
+    "惹": 2830,
+    "膝": 2831,
+    "椭": 2832,
+    "妾": 2833,
+    "刹": 2834,
+    "舶": 2835,
+    "绰": 2836,
+    "舖": 2837,
+    "汐": 2838,
+    "郸": 2839,
+    "羞": 2840,
+    "塾": 2841,
+    "肪": 2842,
+    "奢": 2843,
+    "祇": 2844,
+    "驹": 2845,
+    "俭": 2846,
+    "裙": 2847,
+    "溥": 2848,
+    "瑙": 2849,
+    "淋": 2850,
+    "泼": 2851,
+    "宵": 2852,
+    "檀": 2853,
+    "杖": 2854,
+    "抨": 2855,
+    "哩": 2856,
+    "脆": 2857,
+    "瀑": 2858,
+    "揽": 2859,
+    "冤": 2860,
+    "甄": 2861,
+    "谎": 2862,
+    "楠": 2863,
+    "氟": 2864,
+    "橙": 2865,
+    "$": 2866,
+    "勿": 2867,
+    "彝": 2868,
+    "嘱": 2869,
+    "虔": 2870,
+    "缮": 2871,
+    "榆": 2872,
+    "俯": 2873,
+    "炒": 2874,
+    "с": 2875,
+    "檐": 2876,
+    "魅": 2877,
+    "骄": 2878,
+    "您": 2879,
+    "烹": 2880,
+    "岱": 2881,
+    "瞄": 2882,
+    "丕": 2883,
+    "蜘": 2884,
+    "钥": 2885,
+    "壹": 2886,
+    "菱": 2887,
+    "濑": 2888,
+    "绎": 2889,
+    "岑": 2890,
+    "洽": 2891,
+    "嘲": 2892,
+    "煽": 2893,
+    "琅": 2894,
+    "牡": 2895,
+    "笠": 2896,
+    "楷": 2897,
+    "畿": 2898,
+    "祂": 2899,
+    "倚": 2900,
+    "睹": 2901,
+    "т": 2902,
+    "糟": 2903,
+    "苻": 2904,
+    "栅": 2905,
+    "莞": 2906,
+    "酮": 2907,
+    "膏": 2908,
+    "扳": 2909,
+    "诬": 2910,
+    "祁": 2911,
+    "蚁": 2912,
+    "娥": 2913,
+    "戊": 2914,
+    "蜡": 2915,
+    "裹": 2916,
+    "驿": 2917,
+    "恼": 2918,
+    "梧": 2919,
+    "薯": 2920,
+    "磡": 2921,
+    "掀": 2922,
+    "撕": 2923,
+    "溉": 2924,
+    "谟": 2925,
+    "敖": 2926,
+    "匀": 2927,
+    "榕": 2928,
+    "缸": 2929,
+    "侬": 2930,
+    "凑": 2931,
+    "睿": 2932,
+    "钠": 2933,
+    "橘": 2934,
+    "匣": 2935,
+    "黔": 2936,
+    "锤": 2937,
+    "胚": 2938,
+    "戍": 2939,
+    "晏": 2940,
+    "哇": 2941,
+    "泗": 2942,
+    "睦": 2943,
+    "篡": 2944,
+    "捞": 2945,
+    "婿": 2946,
+    "璧": 2947,
+    "绒": 2948,
+    "棣": 2949,
+    "碗": 2950,
+    "憾": 2951,
+    "沁": 2952,
+    "焊": 2953,
+    "沂": 2954,
+    "舷": 2955,
+    "邢": 2956,
+    "瘟": 2957,
+    "棘": 2958,
+    "嫔": 2959,
+    "焉": 2960,
+    "泾": 2961,
+    "樟": 2962,
+    "刃": 2963,
+    "慨": 2964,
+    "饷": 2965,
+    "缀": 2966,
+    "荀": 2967,
+    "琛": 2968,
+    "盔": 2969,
+    "堀": 2970,
+    "摸": 2971,
+    "雯": 2972,
+    "枯": 2973,
+    "釜": 2974,
+    "乒": 2975,
+    "渝": 2976,
+    "啊": 2977,
+    "堰": 2978,
+    "​": 2979,
+    "髓": 2980,
+    "吟": 2981,
+    "胖": 2982,
+    "晃": 2983,
+    "莹": 2984,
+    "沽": 2985,
+    "渣": 2986,
+    "茹": 2987,
+    "妄": 2988,
+    "钮": 2989,
+    "颌": 2990,
+    "伺": 2991,
+    "瘦": 2992,
+    "镍": 2993,
+    "狙": 2994,
+    "蕉": 2995,
+    "鲤": 2996,
+    "邺": 2997,
+    "挟": 2998,
+    "淆": 2999,
+    "灿": 3000,
+    "′": 3001,
+    "茜": 3002,
+    "妨": 3003,
+    "衬": 3004,
+    "黜": 3005,
+    "桩": 3006,
+    "侮": 3007,
+    "溢": 3008,
+    "•": 3009,
+    "抒": 3010,
+    "捍": 3011,
+    "洒": 3012,
+    "桦": 3013,
+    "鹦": 3014,
+    "辗": 3015,
+    "傀": 3016,
+    "钙": 3017,
+    "刮": 3018,
+    "搏": 3019,
+    "炬": 3020,
+    "瓣": 3021,
+    "缚": 3022,
+    "擢": 3023,
+    "棵": 3024,
+    "娄": 3025,
+    "埤": 3026,
+    "麾": 3027,
+    "昼": 3028,
+    "瀛": 3029,
+    "硝": 3030,
+    "蔗": 3031,
+    "汾": 3032,
+    "儡": 3033,
+    "嘎": 3034,
+    "俸": 3035,
+    "溺": 3036,
+    "俾": 3037,
+    "恪": 3038,
+    "兀": 3039,
+    "砸": 3040,
+    "拢": 3041,
+    "翡": 3042,
+    "浚": 3043,
+    "怜": 3044,
+    "雳": 3045,
+    "菁": 3046,
+    "隍": 3047,
+    "贱": 3048,
+    "в": 3049,
+    "锯": 3050,
+    "膛": 3051,
+    "陛": 3052,
+    "觅": 3053,
+    "熹": 3054,
+    "衫": 3055,
+    "捧": 3056,
+    "鸽": 3057,
+    "衞": 3058,
+    "汞": 3059,
+    "瞻": 3060,
+    "粪": 3061,
+    "簿": 3062,
+    "锻": 3063,
+    "犀": 3064,
+    "吵": 3065,
+    "偕": 3066,
+    "忒": 3067,
+    "硅": 3068,
+    "扔": 3069,
+    "梳": 3070,
+    "讷": 3071,
+    "馈": 3072,
+    "蹄": 3073,
+    "鹉": 3074,
+    "蚌": 3075,
+    "谬": 3076,
+    "迭": 3077,
+    "挠": 3078,
+    "л": 3079,
+    "浏": 3080,
+    "禺": 3081,
+    "斧": 3082,
+    "溜": 3083,
+    "吋": 3084,
+    "灶": 3085,
+    "匙": 3086,
+    "瘫": 3087,
+    "兖": 3088,
+    "麒": 3089,
+    "玺": 3090,
+    "霹": 3091,
+    "逗": 3092,
+    "洼": 3093,
+    "羚": 3094,
+    "赂": 3095,
+    "泵": 3096,
+    "媛": 3097,
+    "α": 3098,
+    "酯": 3099,
+    "瞒": 3100,
+    "枕": 3101,
+    "ー": 3102,
+    "阙": 3103,
+    "皓": 3104,
+    "梗": 3105,
+    "钊": 3106,
+    "壬": 3107,
+    "躯": 3108,
+    "荧": 3109,
+    "壕": 3110,
+    "蜒": 3111,
+    "娇": 3112,
+    "诃": 3113,
+    "涩": 3114,
+    "垮": 3115,
+    "莺": 3116,
+    "桨": 3117,
+    "鸠": 3118,
+    "斡": 3119,
+    "狸": 3120,
+    "羲": 3121,
+    "蜿": 3122,
+    "=": 3123,
+    "瑾": 3124,
+    "跪": 3125,
+    "韬": 3126,
+    "戌": 3127,
+    "杠": 3128,
+    "馨": 3129,
+    "磺": 3130,
+    "豁": 3131,
+    "炯": 3132,
+    "揆": 3133,
+    "庾": 3134,
+    "铲": 3135,
+    "猿": 3136,
+    "篷": 3137,
+    "掳": 3138,
+    "烛": 3139,
+    "譬": 3140,
+    "瑛": 3141,
+    "腕": 3142,
+    "懋": 3143,
+    "挣": 3144,
+    "淤": 3145,
+    "稠": 3146,
+    "骼": 3147,
+    "敛": 3148,
+    "芯": 3149,
+    "蚕": 3150,
+    "舵": 3151,
+    "讶": 3152,
+    "戮": 3153,
+    "琰": 3154,
+    "霉": 3155,
+    "羁": 3156,
+    "腥": 3157,
+    "婷": 3158,
+    "颐": 3159,
+    "晕": 3160,
+    "い": 3161,
+    "胰": 3162,
+    "耆": 3163,
+    "к": 3164,
+    "〔": 3165,
+    "苛": 3166,
+    "闰": 3167,
+    "霆": 3168,
+    "箕": 3169,
+    "泣": 3170,
+    "岂": 3171,
+    "噬": 3172,
+    "瘾": 3173,
+    "〕": 3174,
+    "帛": 3175,
+    "痪": 3176,
+    "绞": 3177,
+    "喂": 3178,
+    "诰": 3179,
+    "肋": 3180,
+    "陡": 3181,
+    "渥": 3182,
+    "喙": 3183,
+    "孵": 3184,
+    "芮": 3185,
+    "膳": 3186,
+    "笨": 3187,
+    "龛": 3188,
+    "恕": 3189,
+    "陋": 3190,
+    "乓": 3191,
+    "蝉": 3192,
+    "诫": 3193,
+    "哉": 3194,
+    "坜": 3195,
+    "遏": 3196,
+    "蝴": 3197,
+    "扯": 3198,
+    "汴": 3199,
+    "霄": 3200,
+    "鑫": 3201,
+    "鞑": 3202,
+    "岚": 3203,
+    "曜": 3204,
+    "诅": 3205,
+    "祚": 3206,
+    "铉": 3207,
+    "呆": 3208,
+    "萃": 3209,
+    "恺": 3210,
+    "葱": 3211,
+    "隘": 3212,
+    "卯": 3213,
+    "锚": 3214,
+    "穹": 3215,
+    "谤": 3216,
+    "苟": 3217,
+    "巅": 3218,
+    "槛": 3219,
+    "驯": 3220,
+    "埗": 3221,
+    "咎": 3222,
+    "盎": 3223,
+    "蔑": 3224,
+    "沸": 3225,
+    "枣": 3226,
+    "兜": 3227,
+    "昊": 3228,
+    "悄": 3229,
+    "釉": 3230,
+    "闷": 3231,
+    "沮": 3232,
+    "潼": 3233,
+    "崁": 3234,
+    "汲": 3235,
+    "钾": 3236,
+    "泻": 3237,
+    "萎": 3238,
+    "桢": 3239,
+    "艮": 3240,
+    "隅": 3241,
+    "臀": 3242,
+    "【": 3243,
+    "坍": 3244,
+    "屑": 3245,
+    "於": 3246,
+    "垩": 3247,
+    "靡": 3248,
+    "嗜": 3249,
+    "晒": 3250,
+    "钵": 3251,
+    "阱": 3252,
+    "剃": 3253,
+    "】": 3254,
+    "臧": 3255,
+    "魄": 3256,
+    "楞": 3257,
+    "卤": 3258,
+    "矫": 3259,
+    "氦": 3260,
+    "毘": 3261,
+    "峨": 3262,
+    "夭": 3263,
+    "靶": 3264,
+    "佬": 3265,
+    "稷": 3266,
+    "薰": 3267,
+    "皈": 3268,
+    "聆": 3269,
+    "谒": 3270,
+    "恳": 3271,
+    "殆": 3272,
+    "敞": 3273,
+    "臻": 3274,
+    "皋": 3275,
+    "萤": 3276,
+    "锌": 3277,
+    "翟": 3278,
+    "毯": 3279,
+    "韧": 3280,
+    "桅": 3281,
+    "诡": 3282,
+    "贮": 3283,
+    "埕": 3284,
+    "煞": 3285,
+    "磐": 3286,
+    "秃": 3287,
+    "甬": 3288,
+    "姨": 3289,
+    "叡": 3290,
+    "筛": 3291,
+    "靼": 3292,
+    "沥": 3293,
+    "妒": 3294,
+    "〇": 3295,
+    "孢": 3296,
+    "幡": 3297,
+    "朽": 3298,
+    "栈": 3299,
+    "藔": 3300,
+    "赈": 3301,
+    "盼": 3302,
+    "伶": 3303,
+    "蚊": 3304,
+    "咪": 3305,
+    "竺": 3306,
+    "辕": 3307,
+    "媚": 3308,
+    "漕": 3309,
+    "奚": 3310,
+    "聋": 3311,
+    "弩": 3312,
+    "昧": 3313,
+    "鹫": 3314,
+    "侏": 3315,
+    "攸": 3316,
+    "旷": 3317,
+    "暮": 3318,
+    "ン": 3319,
+    "圻": 3320,
+    "痴": 3321,
+    "翘": 3322,
+    "锈": 3323,
+    "β": 3324,
+    "皂": 3325,
+    "濠": 3326,
+    "幌": 3327,
+    "炀": 3328,
+    "躁": 3329,
+    "匆": 3330,
+    "鸾": 3331,
+    "し": 3332,
+    "胥": 3333,
+    "晟": 3334,
+    "愁": 3335,
+    "殡": 3336,
+    "侃": 3337,
+    "陌": 3338,
+    "潟": 3339,
+    "沾": 3340,
+    "峭": 3341,
+    "啦": 3342,
+    "栓": 3343,
+    "乍": 3344,
+    "悖": 3345,
+    "岔": 3346,
+    "曦": 3347,
+    "疟": 3348,
+    "淇": 3349,
+    "冉": 3350,
+    "憩": 3351,
+    "珂": 3352,
+    "瞩": 3353,
+    "菇": 3354,
+    "瞿": 3355,
+    "芸": 3356,
+    "钝": 3357,
+    "遁": 3358,
+    "哗": 3359,
+    "咽": 3360,
+    "燮": 3361,
+    "茱": 3362,
+    "煎": 3363,
+    "м": 3364,
+    "捣": 3365,
+    "踩": 3366,
+    "沐": 3367,
+    "蕊": 3368,
+    "撼": 3369,
+    "炽": 3370,
+    "疹": 3371,
+    "笙": 3372,
+    "拐": 3373,
+    "娴": 3374,
+    "苓": 3375,
+    "毋": 3376,
+    "瓮": 3377,
+    "朕": 3378,
+    "寰": 3379,
+    "婢": 3380,
+    "娟": 3381,
+    "竿": 3382,
+    "倦": 3383,
+    "曙": 3384,
+    "歪": 3385,
+    "窜": 3386,
+    "蟾": 3387,
+    "钰": 3388,
+    "衮": 3389,
+    "懒": 3390,
+    "た": 3391,
+    "秽": 3392,
+    "躺": 3393,
+    "д": 3394,
+    ">": 3395,
+    "蓟": 3396,
+    "侈": 3397,
+    "琶": 3398,
+    "桧": 3399,
+    "撃": 3400,
+    "崭": 3401,
+    "赁": 3402,
+    "蝇": 3403,
+    "篆": 3404,
+    "琵": 3405,
+    "狠": 3406,
+    "拷": 3407,
+    "濬": 3408,
+    "捆": 3409,
+    "褚": 3410,
+    "甥": 3411,
+    "辍": 3412,
+    "镀": 3413,
+    "莒": 3414,
+    "骁": 3415,
+    "耍": 3416,
+    "禀": 3417,
+    "癸": 3418,
+    "靴": 3419,
+    "醛": 3420,
+    "焘": 3421,
+    "卉": 3422,
+    "愧": 3423,
+    "挚": 3424,
+    "纶": 3425,
+    "裘": 3426,
+    "绮": 3427,
+    "喘": 3428,
+    "翊": 3429,
+    "窥": 3430,
+    "肛": 3431,
+    "衅": 3432,
+    "椰": 3433,
+    "う": 3434,
+    "绛": 3435,
+    "嫉": 3436,
+    "碉": 3437,
+    "莓": 3438,
+    "睐": 3439,
+    "淄": 3440,
+    "诽": 3441,
+    "觐": 3442,
+    "悍": 3443,
+    "巳": 3444,
+    "に": 3445,
+    "颚": 3446,
+    "玮": 3447,
+    "樵": 3448,
+    "暇": 3449,
+    "酉": 3450,
+    "倩": 3451,
+    "苔": 3452,
+    "灼": 3453,
+    "醋": 3454,
+    "臼": 3455,
+    "泸": 3456,
+    "у": 3457,
+    "萱": 3458,
+    "羡": 3459,
+    "镐": 3460,
+    "炫": 3461,
+    "糙": 3462,
+    "饪": 3463,
+    "侗": 3464,
+    "琐": 3465,
+    "渎": 3466,
+    "腻": 3467,
+    "濮": 3468,
+    "烃": 3469,
+    "彤": 3470,
+    "媳": 3471,
+    "|": 3472,
+    "襟": 3473,
+    "磊": 3474,
+    "浒": 3475,
+    "昙": 3476,
+    "裳": 3477,
+    "蠢": 3478,
+    "牒": 3479,
+    "脾": 3480,
+    "捡": 3481,
+    "绶": 3482,
+    "恤": 3483,
+    "迥": 3484,
+    "葫": 3485,
+    "昨": 3486,
+    "帘": 3487,
+    "耸": 3488,
+    "酰": 3489,
+    "莆": 3490,
+    "卅": 3491,
+    "叱": 3492,
+    "○": 3493,
+    "镖": 3494,
+    "か": 3495,
+    "囱": 3496,
+    "僻": 3497,
+    "槃": 3498,
+    "遴": 3499,
+    "晦": 3500,
+    "珪": 3501,
+    "弈": 3502,
+    "瓯": 3503,
+    "戟": 3504,
+    "槐": 3505,
+    "鄙": 3506,
+    "谛": 3507,
+    "銮": 3508,
+    "颍": 3509,
+    "镁": 3510,
+    "蕙": 3511,
+    "捏": 3512,
+    "僖": 3513,
+    "扼": 3514,
+    "湄": 3515,
+    "葆": 3516,
+    "と": 3517,
+    "艏": 3518,
+    "ん": 3519,
+    "铨": 3520,
+    "潢": 3521,
+    "埼": 3522,
+    "衢": 3523,
+    "馏": 3524,
+    "咐": 3525,
+    "贻": 3526,
+    "谚": 3527,
+    "绫": 3528,
+    "迂": 3529,
+    "虢": 3530,
+    "膺": 3531,
+    "佃": 3532,
+    "苇": 3533,
+    "呕": 3534,
+    "砾": 3535,
+    "飙": 3536,
+    "憎": 3537,
+    "陂": 3538,
+    "哑": 3539,
+    "惕": 3540,
+    "鹃": 3541,
+    "鸥": 3542,
+    "烘": 3543,
+    "﹐": 3544,
+    "趟": 3545,
+    "矽": 3546,
+    "慷": 3547,
+    "扈": 3548,
+    "杞": 3549,
+    "傻": 3550,
+    "鹭": 3551,
+    "肴": 3552,
+    "畸": 3553,
+    "る": 3554,
+    "佚": 3555,
+    "挨": 3556,
+    "丐": 3557,
+    "艋": 3558,
+    "昱": 3559,
+    "逵": 3560,
+    "咤": 3561,
+    "舺": 3562,
+    "氰": 3563,
+    "亵": 3564,
+    "忏": 3565,
+    "柑": 3566,
+    "伎": 3567,
+    "峇": 3568,
+    "倘": 3569,
+    "奘": 3570,
+    "滔": 3571,
+    "粥": 3572,
+    "磋": 3573,
+    "濂": 3574,
+    "涧": 3575,
+    "芥": 3576,
+    "氹": 3577,
+    "鄞": 3578,
+    "铠": 3579,
+    "蜗": 3580,
+    "匮": 3581,
+    "轼": 3582,
+    "诀": 3583,
+    "_": 3584,
+    "冗": 3585,
+    "泷": 3586,
+    "俨": 3587,
+    "鹘": 3588,
+    "乂": 3589,
+    "笋": 3590,
+    "惶": 3591,
+    "湳": 3592,
+    "骠": 3593,
+    "﹑": 3594,
+    "姪": 3595,
+    "孺": 3596,
+    "︰": 3597,
+    "骇": 3598,
+    "瑕": 3599,
+    "闾": 3600,
+    "*": 3601,
+    "尬": 3602,
+    "氓": 3603,
+    "<": 3604,
+    "鳌": 3605,
+    "雉": 3606,
+    "浇": 3607,
+    "驴": 3608,
+    "佥": 3609,
+    "稼": 3610,
+    "榈": 3611,
+    "妊": 3612,
+    "馅": 3613,
+    "尴": 3614,
+    "鹊": 3615,
+    "酪": 3616,
+    "芹": 3617,
+    "妲": 3618,
+    "绯": 3619,
+    "刁": 3620,
+    "弋": 3621,
+    "皱": 3622,
+    "洮": 3623,
+    "醮": 3624,
+    "隼": 3625,
+    "吼": 3626,
+    "圃": 3627,
+    "悚": 3628,
+    "芷": 3629,
+    "锂": 3630,
+    "辙": 3631,
+    "莘": 3632,
+    "熏": 3633,
+    "偃": 3634,
+    "碘": 3635,
+    "酌": 3636,
+    "邕": 3637,
+    "跆": 3638,
+    "搅": 3639,
+    "炜": 3640,
+    "鸮": 3641,
+    "脖": 3642,
+    "吠": 3643,
+    "疮": 3644,
+    "绢": 3645,
+    "弑": 3646,
+    "钛": 3647,
+    "枋": 3648,
+    "懈": 3649,
+    "缎": 3650,
+    "肘": 3651,
+    "钗": 3652,
+    "傣": 3653,
+    "潞": 3654,
+    "茫": 3655,
+    "咳": 3656,
+    "劈": 3657,
+    "碁": 3658,
+    "孀": 3659,
+    "浜": 3660,
+    "黯": 3661,
+    "熄": 3662,
+    "琮": 3663,
+    "拙": 3664,
+    "猷": 3665,
+    "榔": 3666,
+    "筝": 3667,
+    "跻": 3668,
+    "奄": 3669,
+    "郢": 3670,
+    "#": 3671,
+    "汛": 3672,
+    "诣": 3673,
+    "酗": 3674,
+    "诲": 3675,
+    "筠": 3676,
+    "妳": 3677,
+    "吡": 3678,
+    "蝎": 3679,
+    "沅": 3680,
+    "骅": 3681,
+    "褶": 3682,
+    "拂": 3683,
+    "髅": 3684,
+    "畑": 3685,
+    "ル": 3686,
+    "唆": 3687,
+    "丫": 3688,
+    "颉": 3689,
+    "萼": 3690,
+    "剔": 3691,
+    "佗": 3692,
+    "珲": 3693,
+    "翎": 3694,
+    "胄": 3695,
+    "は": 3696,
+    "攘": 3697,
+    "蟒": 3698,
+    "琏": 3699,
+    "颤": 3700,
+    "驸": 3701,
+    "苷": 3702,
+    "ス": 3703,
+    "п": 3704,
+    "樽": 3705,
+    "讥": 3706,
+    "逍": 3707,
+    "椿": 3708,
+    "耦": 3709,
+    "宸": 3710,
+    "怠": 3711,
+    "靳": 3712,
+    "巍": 3713,
+    "镕": 3714,
+    "ы": 3715,
+    "な": 3716,
+    "暱": 3717,
+    "蜴": 3718,
+    "骷": 3719,
+    "□": 3720,
+    "掖": 3721,
+    "嚣": 3722,
+    "劭": 3723,
+    "嗅": 3724,
+    "苞": 3725,
+    "烽": 3726,
+    "鄱": 3727,
+    "晷": 3728,
+    "宕": 3729,
+    "り": 3730,
+    "楔": 3731,
+    "贲": 3732,
+    "酚": 3733,
+    "孛": 3734,
+    "荼": 3735,
+    "乖": 3736,
+    "瞳": 3737,
+    "崧": 3738,
+    "沌": 3739,
+    "蒜": 3740,
+    "昶": 3741,
+    "羟": 3742,
+    "篱": 3743,
+    "鞠": 3744,
+    "檎": 3745,
+    "菅": 3746,
+    "堑": 3747,
+    "酥": 3748,
+    "て": 3749,
+    "慑": 3750,
+    "窘": 3751,
+    "躬": 3752,
+    "荥": 3753,
+    "拌": 3754,
+    "柠": 3755,
+    "耽": 3756,
+    "岷": 3757,
+    "枭": 3758,
+    "癫": 3759,
+    "谔": 3760,
+    "й": 3761,
+    "寥": 3762,
+    "糯": 3763,
+    "潍": 3764,
+    "叩": 3765,
+    "滦": 3766,
+    "峪": 3767,
+    "瓒": 3768,
+    "喃": 3769,
+    "嬴": 3770,
+    "覃": 3771,
+    "赃": 3772,
+    "肽": 3773,
+    "巽": 3774,
+    "涤": 3775,
+    "鞘": 3776,
+    "嫂": 3777,
+    "葺": 3778,
+    "槿": 3779,
+    "茉": 3780,
+    "悯": 3781,
+    "渲": 3782,
+    "徘": 3783,
+    "曳": 3784,
+    "叮": 3785,
+    "骧": 3786,
+    "哮": 3787,
+    "‰": 3788,
+    "я": 3789,
+    "涿": 3790,
+    "呀": 3791,
+    "俑": 3792,
+    "娼": 3793,
+    "澡": 3794,
+    "佟": 3795,
+    "漱": 3796,
+    "徊": 3797,
+    "纾": 3798,
+    "渚": 3799,
+    "迳": 3800,
+    "撇": 3801,
+    "僭": 3802,
+    "溧": 3803,
+    "簇": 3804,
+    "邝": 3805,
+    "璜": 3806,
+    "貂": 3807,
+    "イ": 3808,
+    "艉": 3809,
+    "粽": 3810,
+    "炙": 3811,
+    "榨": 3812,
+    "蛟": 3813,
+    "砚": 3814,
+    "瀚": 3815,
+    "檬": 3816,
+    "斛": 3817,
+    "蚂": 3818,
+    "怯": 3819,
+    "屁": 3820,
+    "峦": 3821,
+    "钚": 3822,
+    "瑄": 3823,
+    "叭": 3824,
+    "岌": 3825,
+    "镛": 3826,
+    "ま": 3827,
+    "颓": 3828,
+    "铂": 3829,
+    "г": 3830,
+    "踞": 3831,
+    "讹": 3832,
+    "夔": 3833,
+    "跤": 3834,
+    "聿": 3835,
+    "祜": 3836,
+    "柩": 3837,
+    "を": 3838,
+    "墘": 3839,
+    "骥": 3840,
+    "妍": 3841,
+    "纥": 3842,
+    "霰": 3843,
+    "娠": 3844,
+    "が": 3845,
+    "琥": 3846,
+    "き": 3847,
+    "兮": 3848,
+    "迺": 3849,
+    "惰": 3850,
+    "睾": 3851,
+    "鼬": 3852,
+    "秤": 3853,
+    "拣": 3854,
+    "贰": 3855,
+    "眺": 3856,
+    "娩": 3857,
+    "摹": 3858,
+    "笈": 3859,
+    "铿": 3860,
+    "髻": 3861,
+    "沱": 3862,
+    "籽": 3863,
+    "睢": 3864,
+    "谯": 3865,
+    "痒": 3866,
+    "帧": 3867,
+    "蔷": 3868,
+    "赓": 3869,
+    "煜": 3870,
+    "呐": 3871,
+    "牦": 3872,
+    "戡": 3873,
+    "洙": 3874,
+    "仄": 3875,
+    "砷": 3876,
+    "ラ": 3877,
+    "昕": 3878,
+    "歹": 3879,
+    "嬷": 3880,
+    "饵": 3881,
+    "嬉": 3882,
+    "痊": 3883,
+    "钳": 3884,
+    "纣": 3885,
+    "酝": 3886,
+    "璞": 3887,
+    "堃": 3888,
+    "涟": 3889,
+    "勐": 3890,
+    "腌": 3891,
+    "恃": 3892,
+    "浔": 3893,
+    "畠": 3894,
+    "掏": 3895,
+    "珅": 3896,
+    "颊": 3897,
+    "遐": 3898,
+    "檄": 3899,
+    "掺": 3900,
+    "苳": 3901,
+    "璟": 3902,
+    "盏": 3903,
+    "潇": 3904,
+    "绊": 3905,
+    "锟": 3906,
+    "さ": 3907,
+    "枉": 3908,
+    "窒": 3909,
+    "馥": 3910,
+    "锭": 3911,
+    "瞭": 3912,
+    "崴": 3913,
+    "缙": 3914,
+    "灏": 3915,
+    "辄": 3916,
+    "筿": 3917,
+    "羹": 3918,
+    "歙": 3919,
+    "锺": 3920,
+    "唾": 3921,
+    "呜": 3922,
+    "ь": 3923,
+    "咕": 3924,
+    "璐": 3925,
+    "碌": 3926,
+    "蝗": 3927,
+    "熬": 3928,
+    "莪": 3929,
+    "琨": 3930,
+    "疵": 3931,
+    "抖": 3932,
+    "鳃": 3933,
+    "栾": 3934,
+    "淦": 3935,
+    "筏": 3936,
+    "狡": 3937,
+    "熵": 3938,
+    "懦": 3939,
+    "窖": 3940,
+    "戛": 3941,
+    "恬": 3942,
+    "忱": 3943,
+    "杵": 3944,
+    "嗓": 3945,
+    "孽": 3946,
+    "螂": 3947,
+    "铮": 3948,
+    "撮": 3949,
+    "茧": 3950,
+    "卞": 3951,
+    "甩": 3952,
+    "锷": 3953,
+    "荟": 3954,
+    "泮": 3955,
+    "钨": 3956,
+    "秧": 3957,
+    "掸": 3958,
+    "槌": 3959,
+    "彧": 3960,
+    "醚": 3961,
+    "踝": 3962,
+    "岬": 3963,
+    "凋": 3964,
+    "惇": 3965,
+    "姥": 3966,
+    "骞": 3967,
+    "ч": 3968,
+    "驭": 3969,
+    "柿": 3970,
+    "リ": 3971,
+    "拇": 3972,
+    "吩": 3973,
+    "屎": 3974,
+    "涪": 3975,
+    "辫": 3976,
+    "圩": 3977,
+    "旌": 3978,
+    "鼐": 3979,
+    "筵": 3980,
+    "溅": 3981,
+    "螈": 3982,
+    "祉": 3983,
+    "惮": 3984,
+    "阇": 3985,
+    "癖": 3986,
+    "峤": 3987,
+    "氐": 3988,
+    "瞰": 3989,
+    "歆": 3990,
+    "铯": 3991,
+    "蛾": 3992,
+    "轧": 3993,
+    "镳": 3994,
+    "汹": 3995,
+    "璇": 3996,
+    "ト": 3997,
+    "佯": 3998,
+    "楣": 3999,
+    "哄": 4000
+}

vocab_wiki_4k_en.json

@@ -0,0 +1,4002 @@
+{
+  "[PAD]": 0,
+  "[UNK]": 1,
+  "[CLS]": 2,
+  "[SEP]": 3,
+  "[MASK]": 4,
+  "!": 5,
+  "\"": 6,
+  "'": 7,
+  "(": 8,
+  ")": 9,
+  ",": 10,
+  "-": 11,
+  ".": 12,
+  "0": 13,
+  "1": 14,
+  "2": 15,
+  "3": 16,
+  "4": 17,
+  "5": 18,
+  "6": 19,
+  "7": 20,
+  "8": 21,
+  "9": 22,
+  ":": 23,
+  ";": 24,
+  "?": 25,
+  "A": 26,
+  "B": 27,
+  "C": 28,
+  "D": 29,
+  "E": 30,
+  "F": 31,
+  "G": 32,
+  "H": 33,
+  "I": 34,
+  "J": 35,
+  "K": 36,
+  "L": 37,
+  "M": 38,
+  "N": 39,
+  "O": 40,
+  "P": 41,
+  "Q": 42,
+  "R": 43,
+  "S": 44,
+  "T": 45,
+  "U": 46,
+  "V": 47,
+  "W": 48,
+  "X": 49,
+  "Y": 50,
+  "Z": 51,
+  "[": 52,
+  "]": 53,
+  "a": 54,
+  "b": 55,
+  "c": 56,
+  "d": 57,
+  "e": 58,
+  "f": 59,
+  "g": 60,
+  "h": 61,
+  "i": 62,
+  "j": 63,
+  "k": 64,
+  "l": 65,
+  "m": 66,
+  "n": 67,
+  "o": 68,
+  "p": 69,
+  "q": 70,
+  "r": 71,
+  "s": 72,
+  "t": 73,
+  "u": 74,
+  "v": 75,
+  "w": 76,
+  "x": 77,
+  "y": 78,
+  "z": 79,
+  "\u0109": 80,
+  "\u010a": 81,
+  "\u0120": 82,
+  "\u0120t": 83,
+  "\u0120a": 84,
+  "he": 85,
+  "in": 86,
+  "\u0120the": 87,
+  "er": 88,
+  "on": 89,
+  "re": 90,
+  "\u0120o": 91,
+  "at": 92,
+  "\u0120s": 93,
+  "en": 94,
+  "ed": 95,
+  "an": 96,
+  "is": 97,
+  "\u0120c": 98,
+  "\u0120w": 99,
+  "es": 100,
+  "it": 101,
+  "or": 102,
+  "\u0120of": 103,
+  "al": 104,
+  "\u0120an": 105,
+  "\u0120b": 106,
+  "ar": 107,
+  "\u0120in": 108,
+  "\u0120f": 109,
+  "\u0120p": 110,
+  "ic": 111,
+  "\u0120and": 112,
+  "ing": 113,
+  "ion": 114,
+  "as": 115,
+  "ro": 116,
+  "\u0120m": 117,
+  "\u0120to": 118,
+  "\u0120d": 119,
+  "le": 120,
+  "\u0120A": 121,
+  "\u0120h": 122,
+  "ent": 123,
+  "ou": 124,
+  "\u0120T": 125,
+  "om": 126,
+  "\u0120re": 127,
+  "st": 128,
+  "ly": 129,
+  "ct": 130,
+  "\u0120e": 131,
+  "\u0120th": 132,
+  "\u0120l": 133,
+  "il": 134,
+  "\u0120C": 135,
+  "\u0120n": 136,
+  "\u0120S": 137,
+  "\u0120B": 138,
+  "ol": 139,
+  "ve": 140,
+  "us": 141,
+  "\u0120be": 142,
+  "am": 143,
+  "ation": 144,
+  "ur": 145,
+  "\u0120The": 146,
+  "ce": 147,
+  "id": 148,
+  "\u0120is": 149,
+  "ch": 150,
+  "ad": 151,
+  "im": 152,
+  "\u0120I": 153,
+  "\u0120as": 154,
+  "un": 155,
+  "\u01201": 156,
+  "\u0120for": 157,
+  "ot": 158,
+  "ut": 159,
+  "ver": 160,
+  "ig": 161,
+  "et": 162,
+  "\u0120g": 163,
+  "ir": 164,
+  "el": 165,
+  "\u0120on": 166,
+  "\u0120M": 167,
+  "ul": 168,
+  "ter": 169,
+  "ith": 170,
+  "\u0120was": 171,
+  "ow": 172,
+  "ra": 173,
+  "\u0120(": 174,
+  "\u0120st": 175,
+  "\u0120con": 176,
+  "her": 177,
+  "\u0120with": 178,
+  "\u0120P": 179,
+  "ist": 180,
+  "\u0120al": 181,
+  "\u0120by": 182,
+  "\u0120that": 183,
+  "ay": 184,
+  "\u0120H": 185,
+  "em": 186,
+  "os": 187,
+  "\u0120wh": 188,
+  "ri": 189,
+  "th": 190,
+  "um": 191,
+  "and": 192,
+  "\u0120pro": 193,
+  "od": 194,
+  "ers": 195,
+  "'s": 196,
+  "\u0120or": 197,
+  "op": 198,
+  "\u0120R": 199,
+  "\u0120E": 200,
+  "\u0120com": 201,
+  "\u0120D": 202,
+  "\u0120are": 203,
+  "\u0120de": 204,
+  "\u0120In": 205,
+  "ag": 206,
+  "\u0120F": 207,
+  "est": 208,
+  "if": 209,
+  "\u0120it": 210,
+  "\u0120G": 211,
+  "rom": 212,
+  "ac": 213,
+  "\u0120W": 214,
+  "\u0120L": 215,
+  "\u0120su": 216,
+  "ate": 217,
+  "\u0120\"": 218,
+  "ain": 219,
+  "ies": 220,
+  "\u012019": 221,
+  "res": 222,
+  "ab": 223,
+  "\u0120at": 224,
+  "\u01202": 225,
+  "ity": 226,
+  "pe": 227,
+  "\u0120ex": 228,
+  "\u0120N": 229,
+  "\u0120from": 230,
+  "oun": 231,
+  "\u0120v": 232,
+  "\u0120he": 233,
+  "se": 234,
+  "ud": 235,
+  "ive": 236,
+  "oc": 237,
+  "qu": 238,
+  "ment": 239,
+  "ant": 240,
+  "iv": 241,
+  "\u0120se": 242,
+  "ich": 243,
+  "ort": 244,
+  "ld": 245,
+  "ess": 246,
+  "ical": 247,
+  "ian": 248,
+  "ial": 249,
+  "ore": 250,
+  "\u0120O": 251,
+  "art": 252,
+  "\u0120J": 253,
+  "\u0120us": 254,
+  "\u0120r": 255,
+  "\u0120un": 256,
+  "pp": 257,
+  "\u0120his": 258,
+  "all": 259,
+  "pt": 260,
+  "ia": 261,
+  "igh": 262,
+  "ated": 263,
+  "ill": 264,
+  "00": 265,
+  "ard": 266,
+  "\u0120which": 267,
+  "ew": 268,
+  "ere": 269,
+  "gh": 270,
+  "ber": 271,
+  "ction": 272,
+  "\u0120ch": 273,
+  "ak": 274,
+  "ther": 275,
+  "\u0120U": 276,
+  "ge": 277,
+  "ave": 278,
+  "ost": 279,
+  "end": 280,
+  "og": 281,
+  "\u0120were": 282,
+  "ure": 283,
+  "\u0120not": 284,
+  "ary": 285,
+  "cl": 286,
+  "ip": 287,
+  "ine": 288,
+  "\u0120pl": 289,
+  "ast": 290,
+  "ish": 291,
+  "\u0120le": 292,
+  "\u0120comp": 293,
+  "ous": 294,
+  "per": 295,
+  "so": 296,
+  "ong": 297,
+  "\u0120ar": 298,
+  "du": 299,
+  "our": 300,
+  "ell": 301,
+  "ame": 302,
+  "\u0120k": 303,
+  "rit": 304,
+  "\u0120have": 305,
+  "ome": 306,
+  "av": 307,
+  "\u0120K": 308,
+  "cc": 309,
+  "ans": 310,
+  "ect": 311,
+  "ition": 312,
+  "ight": 313,
+  "ord": 314,
+  "\u0120sh": 315,
+  "\u0120Ch": 316,
+  "ire": 317,
+  "iz": 318,
+  "\u0120cont": 319,
+  "ult": 320,
+  "up": 321,
+  "\u0120int": 322,
+  "ide": 323,
+  "ub": 324,
+  "ear": 325,
+  "ff": 326,
+  "ry": 327,
+  "ap": 328,
+  "\u0120has": 329,
+  "\u0120also": 330,
+  "mer": 331,
+  "\u0120ad": 332,
+  "ence": 333,
+  "ations": 334,
+  "\u0120St": 335,
+  "der": 336,
+  "ib": 337,
+  "ust": 338,
+  "ally": 339,
+  "\u0120had": 340,
+  "\u0120whe": 341,
+  "\u0120their": 342,
+  "ack": 343,
+  "ign": 344,
+  "\u0120its": 345,
+  "ass": 346,
+  "pl": 347,
+  "),": 348,
+  "\u0120ab": 349,
+  "\u012020": 350,
+  "\u0120part": 351,
+  "ue": 352,
+  "age": 353,
+  "\u0120other": 354,
+  "\u0120but": 355,
+  "\u0120res": 356,
+  "ang": 357,
+  "\u0120An": 358,
+  "ould": 359,
+  "\u0120tr": 360,
+  "low": 361,
+  "\u0120Th": 362,
+  "act": 363,
+  "\u0120cl": 364,
+  "ater": 365,
+  "ok": 366,
+  "clud": 367,
+  "\u0120Al": 368,
+  "\u0120can": 369,
+  "fer": 370,
+  "are": 371,
+  "\u0120y": 372,
+  "\u0120wor": 373,
+  "\u0120im": 374,
+  "ount": 375,
+  "ph": 376,
+  "vel": 377,
+  "\u0120dis": 378,
+  "\u0120en": 379,
+  "ie": 380,
+  "ru": 381,
+  "ory": 382,
+  "irst": 383,
+  "ree": 384,
+  "ates": 385,
+  "any": 386,
+  "\u0120been": 387,
+  "erm": 388,
+  "\u0120Un": 389,
+  "tern": 390,
+  "\u0120this": 391,
+  "over": 392,
+  "ions": 393,
+  "ance": 394,
+  "\u0120includ": 395,
+  "orm": 396,
+  "\u0120one": 397,
+  "ric": 398,
+  "\u0120bec": 399,
+  "\u0120comm": 400,
+  "out": 401,
+  "ace": 402,
+  "con": 403,
+  "\u0120first": 404,
+  "\u0120V": 405,
+  "own": 406,
+  "ime": 407,
+  "able": 408,
+  "ach": 409,
+  "wo": 410,
+  "hed": 411,
+  "ress": 412,
+  "vers": 413,
+  "ile": 414,
+  "fter": 415,
+  "\u0120such": 416,
+  "ugh": 417,
+  "ke": 418,
+  "\u0120used": 419,
+  ").": 420,
+  "te": 421,
+  "port": 422,
+  "\u0120It": 423,
+  "\u0120te": 424,
+  "ound": 425,
+  "\u0120ro": 426,
+  "land": 427,
+  "und": 428,
+  "ced": 429,
+  "ces": 430,
+  "ens": 431,
+  "\u0120pre": 432,
+  "\u0120Ar": 433,
+  "\u0120who": 434,
+  "ep": 435,
+  "\u0120sy": 436,
+  "\u0120per": 437,
+  "ice": 438,
+  "\u0120spe": 439,
+  "ough": 440,
+  "\u0120cons": 441,
+  "erv": 442,
+  "lic": 443,
+  "\u0120all": 444,
+  "ren": 445,
+  "uring": 446,
+  "\u0120ne": 447,
+  "ose": 448,
+  "\u0120des": 449,
+  "\u0120form": 450,
+  "\u0120ag": 451,
+  "ors": 452,
+  "ational": 453,
+  "ari": 454,
+  "\u0120200": 455,
+  "\u0120cent": 456,
+  "\u0120sp": 457,
+  "ron": 458,
+  "les": 459,
+  "\u0120they": 460,
+  "ov": 461,
+  "\u0120more": 462,
+  "ase": 463,
+  "we": 464,
+  "\u0120ev": 465,
+  "\u0120He": 466,
+  "\u0120sc": 467,
+  "ism": 468,
+  "ents": 469,
+  "\u0120into": 470,
+  "ite": 471,
+  "\u0120two": 472,
+  "\u0120most": 473,
+  "ks": 474,
+  "ade": 475,
+  "ited": 476,
+  "ts": 477,
+  "\u0120j": 478,
+  "ied": 479,
+  "\u0120rec": 480,
+  "ular": 481,
+  "\u0120201": 482,
+  "olog": 483,
+  "aw": 484,
+  "ative": 485,
+  "\u0120ele": 486,
+  "\u0120rel": 487,
+  "amp": 488,
+  "aus": 489,
+  "\u0120over": 490,
+  "reat": 491,
+  "\u0120me": 492,
+  "\u0120year": 493,
+  "\u012018": 494,
+  "\u0120reg": 495,
+  "rib": 496,
+  "\u0120col": 497,
+  "\u0120att": 498,
+  "\u0120As": 499,
+  "meric": 500,
+  "one": 501,
+  "ual": 502,
+  "\u0120some": 503,
+  "oin": 504,
+  "ft": 505,
+  "ath": 506,
+  "rough": 507,
+  "ood": 508,
+  "\u0120bet": 509,
+  "ne": 510,
+  "\u0120out": 511,
+  "\u0120gro": 512,
+  "wn": 513,
+  "\u0120num": 514,
+  "\u0120than": 515,
+  "ased": 516,
+  "ted": 517,
+  "ics": 518,
+  "\u0120after": 519,
+  "oth": 520,
+  "\u0120app": 521,
+  "\u0120am": 522,
+  "\u0120produ": 523,
+  "ings": 524,
+  "\u0120gen": 525,
+  "\u0120This": 526,
+  "form": 527,
+  "\u0120Americ": 528,
+  "\u0120sub": 529,
+  "ten": 530,
+  "ook": 531,
+  "red": 532,
+  "\u0120kn": 533,
+  "iss": 534,
+  "ific": 535,
+  "ob": 536,
+  "ons": 537,
+  "\u0120would": 538,
+  "\u0120lar": 539,
+  "ury": 540,
+  "old": 541,
+  "ark": 542,
+  "\u0120may": 543,
+  "\u0120time": 544,
+  "stem": 545,
+  "\u0120under": 546,
+  "ween": 547,
+  "\u0120En": 548,
+  "\u0120only": 549,
+  "\u0120play": 550,
+  "gan": 551,
+  "ys": 552,
+  "\u0120ac": 553,
+  "\u0120when": 554,
+  "\u0120inter": 555,
+  "\u0120so": 556,
+  "\u0120ear": 557,
+  "\u0120acc": 558,
+  "ind": 559,
+  "als": 560,
+  "ail": 561,
+  "\u0120mod": 562,
+  "the": 563,
+  "ove": 564,
+  "\u0120between": 565,
+  "\u0120ass": 566,
+  "ever": 567,
+  "\u0120new": 568,
+  "\u0120ph": 569,
+  "\u0120ra": 570,
+  "ran": 571,
+  "\u0120up": 572,
+  "\u0120there": 573,
+  "li": 574,
+  "\u0120spec": 575,
+  "\u0120many": 576,
+  "\u0120ind": 577,
+  "ral": 578,
+  "\u0120off": 579,
+  "ild": 580,
+  "\".": 581,
+  "overn": 582,
+  "\u0120them": 583,
+  "\u0120use": 584,
+  "\u0120him": 585,
+  "cre": 586,
+  "ities": 587,
+  "\u0120dif": 588,
+  "urn": 589,
+  "\u0120pe": 590,
+  "\u0120199": 591,
+  "\u01203": 592,
+  "velop": 593,
+  "ram": 594,
+  "ince": 595,
+  "\u0120Y": 596,
+  "\u0120no": 597,
+  "\u0120system": 598,
+  "\u0120count": 599,
+  "\u0120def": 600,
+  "\u0120Com": 601,
+  "ars": 602,
+  "\u0120pr": 603,
+  "\u0120trans": 604,
+  "\u0120qu": 605,
+  "\u0120through": 606,
+  "\u0120act": 607,
+  "ject": 608,
+  "\u0120her": 609,
+  "\u0120bl": 610,
+  "ution": 611,
+  "ines": 612,
+  "\u0120Se": 613,
+  "ern": 614,
+  "\u0120call": 615,
+  "ollow": 616,
+  "\u0120pos": 617,
+  "ward": 618,
+  "uc": 619,
+  "\u0120Brit": 620,
+  "\u0120pres": 621,
+  "\u0120where": 622,
+  "\u0120we": 623,
+  "ond": 624,
+  "orn": 625,
+  "ating": 626,
+  "\u0120pub": 627,
+  "\u0120rem": 628,
+  "ished": 629,
+  "\u0120during": 630,
+  "ake": 631,
+  "ists": 632,
+  "ict": 633,
+  "oy": 634,
+  "hes": 635,
+  "\u0120work": 636,
+  "arch": 637,
+  "\u0120em": 638,
+  "\u0120ob": 639,
+  "\u0120supp": 640,
+  "\u0120prov": 641,
+  "ident": 642,
+  "\u0120develop": 643,
+  "\u0120Re": 644,
+  "\u0120do": 645,
+  "\u0120inst": 646,
+  "tle": 647,
+  "\u0120fil": 648,
+  "cess": 649,
+  "ug": 650,
+  "\u0120known": 651,
+  "ivers": 652,
+  "\u0120set": 653,
+  "ious": 654,
+  "\u0120pol": 655,
+  "uro": 656,
+  "\u0120inv": 657,
+  "\u0120met": 658,
+  "ays": 659,
+  "ause": 660,
+  "\",": 661,
+  "ures": 662,
+  "\u0120man": 663,
+  "ower": 664,
+  "\u0120li": 665,
+  "very": 666,
+  "led": 667,
+  "\u0120pop": 668,
+  "ically": 669,
+  "\u0120inf": 670,
+  "ments": 671,
+  "\u0120mem": 672,
+  "hip": 673,
+  "\u0120gener": 674,
+  "ient": 675,
+  "owever": 676,
+  "oh": 677,
+  "\u0120vari": 678,
+  "\u0120found": 679,
+  "cept": 680,
+  "fore": 681,
+  "\u0120these": 682,
+  "\u0120tra": 683,
+  "\u0120ap": 684,
+  "ix": 685,
+  "ually": 686,
+  "\u0120Con": 687,
+  "ton": 688,
+  "\u0120about": 689,
+  "outh": 690,
+  "\u0120govern": 691,
+  "\u0120De": 692,
+  "\u0120Eng": 693,
+  "aj": 694,
+  "ized": 695,
+  "cted": 696,
+  "ah": 697,
+  "tain": 698,
+  "\u0120years": 699,
+  "\u0120char": 700,
+  "ible": 701,
+  "\u0120New": 702,
+  "\u0120follow": 703,
+  "\u0120fl": 704,
+  "ins": 705,
+  "\u0120dist": 706,
+  "\u0120co": 707,
+  "ium": 708,
+  "\u0120exp": 709,
+  "\u0120loc": 710,
+  "hen": 711,
+  "ople": 712,
+  "\u0120century": 713,
+  "ved": 714,
+  "\u0120number": 715,
+  "angu": 716,
+  "\u0120add": 717,
+  "oci": 718,
+  "ise": 719,
+  "\u0120eff": 720,
+  "ered": 721,
+  "\u0120American": 722,
+  "\u0120On": 723,
+  "yp": 724,
+  "\u0120including": 725,
+  "\u0120made": 726,
+  "\u0120rep": 727,
+  "\u0120differ": 728,
+  "\u0120fe": 729,
+  "ctions": 730,
+  "inal": 731,
+  "lish": 732,
+  "\u0120film": 733,
+  "\u0120sur": 734,
+  "19": 735,
+  "ural": 736,
+  "\u0120well": 737,
+  "orth": 738,
+  "\u0120being": 739,
+  "\u0120At": 740,
+  "\u0120imp": 741,
+  "\u0120both": 742,
+  "\u0120cre": 743,
+  "\u0120For": 744,
+  "io": 745,
+  "lex": 746,
+  "\u0120United": 747,
+  "\u0120serv": 748,
+  "\u0120called": 749,
+  "\u0120writ": 750,
+  "\u0120main": 751,
+  "\u0120car": 752,
+  "\u0120bu": 753,
+  "\u0120ext": 754,
+  "man": 755,
+  "ife": 756,
+  "\u0120again": 757,
+  "cy": 758,
+  "\u0120sign": 759,
+  "ature": 760,
+  "round": 761,
+  "ants": 762,
+  "iel": 763,
+  "\u0120any": 764,
+  "\u0120then": 765,
+  "uch": 766,
+  "its": 767,
+  ".\"": 768,
+  "\u0120ent": 769,
+  "\u0120Ap": 770,
+  "\u0120while": 771,
+  "bers": 772,
+  "cial": 773,
+  "\u0120stud": 774,
+  "\u0120hist": 775,
+  "imes": 776,
+  "\u0120end": 777,
+  "\u0120Am": 778,
+  "\u0120later": 779,
+  "ility": 780,
+  "\u0120198": 781,
+  "cond": 782,
+  "\u0120sim": 783,
+  "\u0120Is": 784,
+  "\u0120became": 785,
+  "ters": 786,
+  "\u0120dec": 787,
+  "\u01204": 788,
+  "\u0120art": 789,
+  "\u0120three": 790,
+  "\u0120sm": 791,
+  "ather": 792,
+  "ock": 793,
+  "\u0120British": 794,
+  "hem": 795,
+  "\u0120197": 796,
+  "\u0120often": 797,
+  "\u0120langu": 798,
+  "\u0120Euro": 799,
+  "ll": 800,
+  "ick": 801,
+  "\u0120term": 802,
+  "\u0120dep": 803,
+  "\u0120early": 804,
+  "ians": 805,
+  "ros": 806,
+  "ract": 807,
+  "other": 808,
+  "elf": 809,
+  "\u0120group": 810,
+  "\u0120const": 811,
+  "\u0120mon": 812,
+  "\u0120ser": 813,
+  "ames": 814,
+  "\u0120large": 815,
+  "\u0120aut": 816,
+  "\u0120high": 817,
+  "\u0120Le": 818,
+  "\u0120air": 819,
+  "ork": 820,
+  "ily": 821,
+  "gest": 822,
+  "rist": 823,
+  "ision": 824,
+  "ck": 825,
+  "\u0120Bl": 826,
+  "\u0120people": 827,
+  "\u0120sever": 828,
+  "ages": 829,
+  "\u0120.": 830,
+  "\u0120desc": 831,
+  "ology": 832,
+  "\u0120contin": 833,
+  "\u0120partic": 834,
+  "ting": 835,
+  "rop": 836,
+  "\u0120Af": 837,
+  "ful": 838,
+  "\u0120Europe": 839,
+  "\u0120Be": 840,
+  "\u0120est": 841,
+  "\u0120state": 842,
+  "\u0120common": 843,
+  "lished": 844,
+  "ajor": 845,
+  "\u0120mus": 846,
+  "\u0120dire": 847,
+  "\u0120orig": 848,
+  "ues": 849,
+  "ty": 850,
+  "\u0120city": 851,
+  "ann": 852,
+  "\u0120long": 853,
+  "\u0120Col": 854,
+  "\u0120appe": 855,
+  "\u0120196": 856,
+  "ool": 857,
+  "apt": 858,
+  "\u01205": 859,
+  "\u0120world": 860,
+  "vent": 861,
+  "\u0120Ind": 862,
+  "\u0120consid": 863,
+  "iversity": 864,
+  "\u0120result": 865,
+  "\u0120ret": 866,
+  "\u0120oper": 867,
+  "air": 868,
+  "ex": 869,
+  "\u0120pers": 870,
+  "ield": 871,
+  "iew": 872,
+  "ax": 873,
+  "\u0120Sh": 874,
+  "\u0120ant": 875,
+  "\u0120could": 876,
+  "ger": 877,
+  "\u0120lead": 878,
+  "\u0120Ad": 879,
+  "ined": 880,
+  "til": 881,
+  "az": 882,
+  "uss": 883,
+  "\u0120same": 884,
+  "ample": 885,
+  "\u0120own": 886,
+  "osed": 887,
+  "ange": 888,
+  "\u0120Aust": 889,
+  "though": 890,
+  "\u0120like": 891,
+  "ital": 892,
+  "\u0120War": 893,
+  "\u0120government": 894,
+  "ale": 895,
+  "ble": 896,
+  "\u0120occ": 897,
+  "\u0120min": 898,
+  "\u0120Sp": 899,
+  "\u0120Christ": 900,
+  "lo": 901,
+  "\u0120Sc": 902,
+  "\u0120States": 903,
+  "oman": 904,
+  "raph": 905,
+  "\u0120war": 906,
+  "\u0120small": 907,
+  "000": 908,
+  "omet": 909,
+  "acter": 910,
+  "ived": 911,
+  "ae": 912,
+  "rol": 913,
+  "\u0120ed": 914,
+  "\u0120before": 915,
+  "\u0120will": 916,
+  "\u0120she": 917,
+  "\u0120second": 918,
+  "ired": 919,
+  "\u0120different": 920,
+  "ank": 921,
+  "\u0120,": 922,
+  "ared": 923,
+  "\u0120fam": 924,
+  "\u0120med": 925,
+  "\u0120Joh": 926,
+  "urch": 927,
+  "\u012017": 928,
+  "ute": 929,
+  "gy": 930,
+  "ional": 931,
+  "\u0120each": 932,
+  "\u0120disc": 933,
+  "\u0120design": 934,
+  "\u0120several": 935,
+  "ives": 936,
+  "\u0120major": 937,
+  "ene": 938,
+  "\u0120bro": 939,
+  "\u0120support": 940,
+  "chn": 941,
+  "els": 942,
+  "\u0120bel": 943,
+  "\u0120because": 944,
+  "\u0120prog": 945,
+  "\u0120Ab": 946,
+  "\u0120include": 947,
+  "ised": 948,
+  "\u0120Germ": 949,
+  "\u0120example": 950,
+  "\u0120arm": 951,
+  "ety": 952,
+  "\u0120remain": 953,
+  "\u0120if": 954,
+  "\u0120law": 955,
+  "aking": 956,
+  "ccess": 957,
+  "iod": 958,
+  "\u0120hum": 959,
+  "\u0120effect": 960,
+  "\u0120Pro": 961,
+  "\u0120Wor": 962,
+  "ording": 963,
+  "min": 964,
+  "\u0120against": 965,
+  "\u0120land": 966,
+  "riv": 967,
+  "\u0120sch": 968,
+  "\u0120However": 969,
+  "\u0120There": 970,
+  "\u0120since": 971,
+  "iver": 972,
+  "aid": 973,
+  "\u0120John": 974,
+  "oint": 975,
+  "\u0120commun": 976,
+  "\u0120Z": 977,
+  "ash": 978,
+  "\u0120name": 979,
+  "atic": 980,
+  "ides": 981,
+  "ries": 982,
+  "ided": 983,
+  "\u0120po": 984,
+  "ilit": 985,
+  "ices": 986,
+  "\u0120str": 987,
+  "ning": 988,
+  "ird": 989,
+  "\u0120trad": 990,
+  "\u0120Comm": 991,
+  "\u0120incre": 992,
+  "\u0120fun": 993,
+  "\u0120four": 994,
+  "\u0120public": 995,
+  "ina": 996,
+  "\u0120until": 997,
+  "\u0120allow": 998,
+  "cent": 999,
+  "\u0120requ": 1000,
+  "\u0120cult": 1001,
+  "ox": 1002,
+  "\u0120leg": 1003,
+  "\u0120book": 1004,
+  "ulation": 1005,
+  "\u0120Wh": 1006,
+  "\u01206": 1007,
+  "\u0120ins": 1008,
+  "\u0120det": 1009,
+  "\u0120World": 1010,
+  "20": 1011,
+  "ued": 1012,
+  "\u0120English": 1013,
+  "ley": 1014,
+  "\u0120Cl": 1015,
+  "get": 1016,
+  "\u0120back": 1017,
+  "way": 1018,
+  "\u0120ref": 1019,
+  "lu": 1020,
+  "eral": 1021,
+  "\u0120power": 1022,
+  "ified": 1023,
+  "itions": 1024,
+  "ged": 1025,
+  "\u0120country": 1026,
+  "\u0120organ": 1027,
+  "conom": 1028,
+  "ven": 1029,
+  "yn": 1030,
+  "\u0120Ph": 1031,
+  "ruct": 1032,
+  "\u0120mat": 1033,
+  "\u01207": 1034,
+  "yl": 1035,
+  "\u0120equ": 1036,
+  "\u0120import": 1037,
+  "mber": 1038,
+  "raft": 1039,
+  "\u0120though": 1040,
+  "arm": 1041,
+  "ius": 1042,
+  "\u0120cap": 1043,
+  "ission": 1044,
+  "\u0120success": 1045,
+  "\u0120val": 1046,
+  "ases": 1047,
+  "ience": 1048,
+  "\u0120They": 1049,
+  "\u0120sing": 1050,
+  "\u0120elect": 1051,
+  "to": 1052,
+  "\u0120process": 1053,
+  "son": 1054,
+  "\u0120cor": 1055,
+  "lect": 1056,
+  "\u0120bat": 1057,
+  "\u0120University": 1058,
+  "\u0120op": 1059,
+  "xt": 1060,
+  "por": 1061,
+  "enn": 1062,
+  "ris": 1063,
+  "ier": 1064,
+  "\u0120poss": 1065,
+  "\u0120show": 1066,
+  "ternational": 1067,
+  "\u0120rele": 1068,
+  "\u0120Can": 1069,
+  "ner": 1070,
+  "inc": 1071,
+  "oph": 1072,
+  "ason": 1073,
+  "ended": 1074,
+  "eng": 1075,
+  "rad": 1076,
+  "\u0120refer": 1077,
+  "ring": 1078,
+  "\u0120those": 1079,
+  "ross": 1080,
+  "\u0120These": 1081,
+  "ement": 1082,
+  "\u0120period": 1083,
+  "\u0120based": 1084,
+  "\u0120did": 1085,
+  "ouse": 1086,
+  "\u0120describ": 1087,
+  "com": 1088,
+  "\u0120conf": 1089,
+  "\u0120cur": 1090,
+  "\u0120par": 1091,
+  "\u0120right": 1092,
+  "aim": 1093,
+  "oot": 1094,
+  "ize": 1095,
+  "\u0120program": 1096,
+  "\u0120II": 1097,
+  "ness": 1098,
+  "rodu": 1099,
+  "\u0120offic": 1100,
+  "pect": 1101,
+  "ilar": 1102,
+  "\u0120polit": 1103,
+  "\u0120person": 1104,
+  "ull": 1105,
+  "\u0120life": 1106,
+  "ript": 1107,
+  "\u0120order": 1108,
+  "ets": 1109,
+  "ently": 1110,
+  "ization": 1111,
+  "ences": 1112,
+  "ration": 1113,
+  "resent": 1114,
+  "\u0120around": 1115,
+  "\u0120div": 1116,
+  "\u0120area": 1117,
+  "\u0120near": 1118,
+  "\u0120Or": 1119,
+  "\u0120class": 1120,
+  "\u0120Arm": 1121,
+  "ane": 1122,
+  "\u0120non": 1123,
+  "\u0120sl": 1124,
+  "\u0120mar": 1125,
+  "\u0120language": 1126,
+  "\u0120even": 1127,
+  "\u0120Car": 1128,
+  "\u0120began": 1129,
+  "\u0120194": 1130,
+  "\u0120direct": 1131,
+  "\u0120After": 1132,
+  "ody": 1133,
+  "dom": 1134,
+  "\u0120within": 1135,
+  "\u0120estab": 1136,
+  "\u0120ke": 1137,
+  "aced": 1138,
+  "par": 1139,
+  "aving": 1140,
+  "\u0120using": 1141,
+  "\u0120record": 1142,
+  "mp": 1143,
+  "\u0120techn": 1144,
+  "\u0120modern": 1145,
+  "\u0120expl": 1146,
+  "\u0120death": 1147,
+  "\u0120Afric": 1148,
+  "\u0120character": 1149,
+  "rid": 1150,
+  "\u0120belie": 1151,
+  "ode": 1152,
+  "\u0120lit": 1153,
+  "\u0120among": 1154,
+  "\u0120typ": 1155,
+  "\u0120Alex": 1156,
+  "amed": 1157,
+  "eg": 1158,
+  "\u0120following": 1159,
+  "\u0120perform": 1160,
+  "\u0120av": 1161,
+  "que": 1162,
+  "\u0120pass": 1163,
+  "\u0120much": 1164,
+  "\u0120195": 1165,
+  "lands": 1166,
+  "\u0120influ": 1167,
+  "\u0120Gree": 1168,
+  "ories": 1169,
+  "\u0120giv": 1170,
+  "\u0120exper": 1171,
+  "ograph": 1172,
+  "\u0120prop": 1173,
+  "ec": 1174,
+  "\u0120opp": 1175,
+  "ober": 1176,
+  "gen": 1177,
+  "\u012016": 1178,
+  "\u0120comput": 1179,
+  "\u0120water": 1180,
+  "\u0120econom": 1181,
+  "\u0120series": 1182,
+  "hor": 1183,
+  "ially": 1184,
+  "\u0120Austral": 1185,
+  "\u0120Mar": 1186,
+  "tic": 1187,
+  "\u0120'": 1188,
+  "\u0120appro": 1189,
+  "\u0120stand": 1190,
+  "\u0120Sch": 1191,
+  "\u0120adv": 1192,
+  "ony": 1193,
+  "epend": 1194,
+  "estern": 1195,
+  "\u0120team": 1196,
+  "\u0120Some": 1197,
+  "\u0120South": 1198,
+  "ik": 1199,
+  "\u0120word": 1200,
+  "ological": 1201,
+  "\u0120due": 1202,
+  "\u0120control": 1203,
+  "\u0120mark": 1204,
+  "\u0120stat": 1205,
+  "ball": 1206,
+  "ves": 1207,
+  "read": 1208,
+  "\u0120resp": 1209,
+  "\u0120sol": 1210,
+  "\u0120music": 1211,
+  "\u0120cour": 1212,
+  "ately": 1213,
+  "ister": 1214,
+  "\u0120considered": 1215,
+  "\u012015": 1216,
+  "ilitary": 1217,
+  "\u0120rest": 1218,
+  "\u0120bas": 1219,
+  "eb": 1220,
+  "\u0120Cent": 1221,
+  "ense": 1222,
+  "ament": 1223,
+  "ids": 1224,
+  "ander": 1225,
+  "row": 1226,
+  "\u0120King": 1227,
+  "\u0120North": 1228,
+  "\u0120ann": 1229,
+  "\u0120still": 1230,
+  "\u0120popular": 1231,
+  "ior": 1232,
+  "itional": 1233,
+  "\u0120eng": 1234,
+  "\u0120By": 1235,
+  "\u0120National": 1236,
+  "\u0120along": 1237,
+  "\u0120now": 1238,
+  "\u0120very": 1239,
+  "\u0120es": 1240,
+  "ink": 1241,
+  "\u0120hand": 1242,
+  "ple": 1243,
+  "ards": 1244,
+  "\u0120fin": 1245,
+  "\u0120comple": 1246,
+  "\u0120gu": 1247,
+  "rench": 1248,
+  "craft": 1249,
+  "\u0120comb": 1250,
+  "de": 1251,
+  "\u0120Q": 1252,
+  "\u0120dem": 1253,
+  "ublic": 1254,
+  "\u0120members": 1255,
+  "\u0120particular": 1256,
+  "iet": 1257,
+  "\u0120ter": 1258,
+  "\u0120attack": 1259,
+  "\u0120similar": 1260,
+  "ried": 1261,
+  "\u0120histor": 1262,
+  "\u0120indu": 1263,
+  "\u0120rece": 1264,
+  "\u0120tem": 1265,
+  "\u0120Ang": 1266,
+  "\u0120open": 1267,
+  "\u0120species": 1268,
+  "\u01208": 1269,
+  "\u0120Ag": 1270,
+  "\u0120rad": 1271,
+  "\u0120represent": 1272,
+  "\u0120return": 1273,
+  "oss": 1274,
+  "ole": 1275,
+  "iam": 1276,
+  "\u0120rev": 1277,
+  "\u0120run": 1278,
+  "\u0120fact": 1279,
+  "\u0120Bat": 1280,
+  "\u0120head": 1281,
+  "\u0120bo": 1282,
+  "\u0120mov": 1283,
+  "ains": 1284,
+  "\u0120present": 1285,
+  "\u0120Pl": 1286,
+  "\u0120vers": 1287,
+  "ination": 1288,
+  "omen": 1289,
+  "\u0120various": 1290,
+  "\u0120human": 1291,
+  "ator": 1292,
+  "une": 1293,
+  "\u012010": 1294,
+  "\u0120cer": 1295,
+  "\u0120said": 1296,
+  "\u0120Bar": 1297,
+  "ster": 1298,
+  "me": 1299,
+  "\u0120led": 1300,
+  "\u0120French": 1301,
+  "\u0120what": 1302,
+  "gin": 1303,
+  "af": 1304,
+  "\u0120Air": 1305,
+  "\u0120report": 1306,
+  "\u0120fre": 1307,
+  "\u0120place": 1308,
+  "rent": 1309,
+  "\u0120aff": 1310,
+  "yd": 1311,
+  "\u0120pract": 1312,
+  "ches": 1313,
+  "rab": 1314,
+  "\u0120Roman": 1315,
+  "\u0120important": 1316,
+  "ality": 1317,
+  "empt": 1318,
+  "\u0120start": 1319,
+  "\u0120vis": 1320,
+  "ument": 1321,
+  "ugust": 1322,
+  "\u0120build": 1323,
+  "cing": 1324,
+  "\u0120All": 1325,
+  "\u0120child": 1326,
+  "alth": 1327,
+  "\u0120somet": 1328,
+  "\u0120line": 1329,
+  "hod": 1330,
+  "by": 1331,
+  "\u0120Mus": 1332,
+  "\u0120move": 1333,
+  "\u0120ve": 1334,
+  "\u0120last": 1335,
+  "\u0120bi": 1336,
+  "\u0120crit": 1337,
+  "ray": 1338,
+  "app": 1339,
+  "\u0120great": 1340,
+  "\u0120development": 1341,
+  "\u0120pat": 1342,
+  "\u0120August": 1343,
+  "istic": 1344,
+  "\u0120Acc": 1345,
+  "ither": 1346,
+  "idd": 1347,
+  "ively": 1348,
+  "lin": 1349,
+  "ales": 1350,
+  "\u0120game": 1351,
+  "\u0120BC": 1352,
+  "rew": 1353,
+  "cer": 1354,
+  "ites": 1355,
+  "\u0120school": 1356,
+  "ilt": 1357,
+  "\u0120usually": 1358,
+  "\u0120developed": 1359,
+  "\u0120make": 1360,
+  "\u0120claim": 1361,
+  "ms": 1362,
+  "\u0120given": 1363,
+  "ember": 1364,
+  "\u0120anim": 1365,
+  "\u0120military": 1366,
+  "\u0120general": 1367,
+  "med": 1368,
+  "\u0120short": 1369,
+  "..": 1370,
+  "\u0120less": 1371,
+  "\u0120mult": 1372,
+  "day": 1373,
+  "olic": 1374,
+  "\u0120home": 1375,
+  "\u0120occur": 1376,
+  "\u0120left": 1377,
+  "\u0120And": 1378,
+  "tal": 1379,
+  "\u0120ident": 1380,
+  "amb": 1381,
+  "\u0120Apol": 1382,
+  "\u0120invol": 1383,
+  "\u0120national": 1384,
+  "\u0120German": 1385,
+  "\u0120sug": 1386,
+  "\u0120former": 1387,
+  "\u0120US": 1388,
+  "\u0120become": 1389,
+  "\u0120without": 1390,
+  "\u0120signific": 1391,
+  "\u0120repl": 1392,
+  "\u0120proper": 1393,
+  "\u0120history": 1394,
+  "\u0120ide": 1395,
+  "\u0120old": 1396,
+  "ocial": 1397,
+  "\u0120countries": 1398,
+  "\u0120fore": 1399,
+  "\u0120another": 1400,
+  "\u0120lim": 1401,
+  "\u0120During": 1402,
+  "\u0120anal": 1403,
+  "uted": 1404,
+  "ata": 1405,
+  "\u0120When": 1406,
+  "be": 1407,
+  "\u0120single": 1408,
+  "\u0120colle": 1409,
+  "itution": 1410,
+  "\u0120need": 1411,
+  "\u0120His": 1412,
+  "\u0120Will": 1413,
+  "rand": 1414,
+  "\u0120field": 1415,
+  "\u0120inc": 1416,
+  "ague": 1417,
+  "The": 1418,
+  "\u0120Ass": 1419,
+  "\u0120every": 1420,
+  "\u0120Russ": 1421,
+  "\u0120York": 1422,
+  "astern": 1423,
+  "akes": 1424,
+  "\u0120port": 1425,
+  "ivil": 1426,
+  "\u0120struct": 1427,
+  "\u0120published": 1428,
+  "ots": 1429,
+  "\u0120dev": 1430,
+  "urther": 1431,
+  "wards": 1432,
+  "omin": 1433,
+  "\u0120local": 1434,
+  "\u0120prim": 1435,
+  "ats": 1436,
+  "\u0120arg": 1437,
+  "line": 1438,
+  "ware": 1439,
+  "ule": 1440,
+  "\u0120cr": 1441,
+  "\u0120prot": 1442,
+  "ling": 1443,
+  "\u0120states": 1444,
+  "\u0120obs": 1445,
+  "\u0120Cal": 1446,
+  "\u0120go": 1447,
+  "\u0120family": 1448,
+  "\u0120down": 1449,
+  "ze": 1450,
+  "\u0120Ear": 1451,
+  "cil": 1452,
+  "\u0120bus": 1453,
+  "\u0120late": 1454,
+  "ably": 1455,
+  "\u0120associ": 1456,
+  "ances": 1457,
+  "\u0120Tr": 1458,
+  "\u012012": 1459,
+  "\u0120way": 1460,
+  "\u0120political": 1461,
+  "\u0120origin": 1462,
+  "\u0120arch": 1463,
+  "\u0120hig": 1464,
+  "less": 1465,
+  "ai": 1466,
+  "\u0120European": 1467,
+  "rote": 1468,
+  "\u0120iss": 1469,
+  "\u0120Ge": 1470,
+  "\u0120few": 1471,
+  "\u0120region": 1472,
+  "\u0120Apollo": 1473,
+  "\u0120caus": 1474,
+  "ogn": 1475,
+  "pecial": 1476,
+  "\u0120Canad": 1477,
+  "\u0120view": 1478,
+  "\u0120exist": 1479,
+  "ivid": 1480,
+  "\u0120rul": 1481,
+  "\u0120author": 1482,
+  "\u0120however": 1483,
+  "\u0120mill": 1484,
+  "\u0120Church": 1485,
+  "\u01209": 1486,
+  "iron": 1487,
+  "\u0120cond": 1488,
+  "\u0120treat": 1489,
+  "ones": 1490,
+  "\u0120chang": 1491,
+  "olution": 1492,
+  "\u0120sk": 1493,
+  "\u0120beh": 1494,
+  "ential": 1495,
+  "\u0120level": 1496,
+  "\u0120202": 1497,
+  "aterial": 1498,
+  "\u0120body": 1499,
+  "\u0120Bo": 1500,
+  "ele": 1501,
+  "ops": 1502,
+  "\u0120hel": 1503,
+  "\u0120included": 1504,
+  "\u0120gl": 1505,
+  "\u0120May": 1506,
+  "\u0120independ": 1507,
+  "au": 1508,
+  "\u0120function": 1509,
+  "\u0120indust": 1510,
+  "ured": 1511,
+  "\u0120men": 1512,
+  "lear": 1513,
+  "\u0120held": 1514,
+  "\u0120tele": 1515,
+  "ery": 1516,
+  "\u0120Par": 1517,
+  "aster": 1518,
+  "\u0120prof": 1519,
+  "\u0120kill": 1520,
+  "\u0120Rep": 1521,
+  "\u0120point": 1522,
+  "idence": 1523,
+  "side": 1524,
+  "\u0120anc": 1525,
+  "\u0120sour": 1526,
+  "irc": 1527,
+  "\u0120proble": 1528,
+  "\u0120introdu": 1529,
+  "\u0120pur": 1530,
+  "ead": 1531,
+  "ored": 1532,
+  "\u0120Pr": 1533,
+  "\u0120One": 1534,
+  "hern": 1535,
+  "\u0120although": 1536,
+  "\u0120case": 1537,
+  "\u012011": 1538,
+  "\u0120day": 1539,
+  "\u0120population": 1540,
+  "ruction": 1541,
+  "\u0120significant": 1542,
+  "oon": 1543,
+  "rain": 1544,
+  "\u0120took": 1545,
+  "bit": 1546,
+  "\u0120camp": 1547,
+  "ention": 1548,
+  "ability": 1549,
+  "\u0120193": 1550,
+  "ering": 1551,
+  "\u0120event": 1552,
+  "\u0120prom": 1553,
+  "formation": 1554,
+  "ividual": 1555,
+  "\u0120attempt": 1556,
+  "\u0120Min": 1557,
+  "\u0120Ex": 1558,
+  "rict": 1559,
+  "\u0120Char": 1560,
+  "\u0120init": 1561,
+  "sequ": 1562,
+  "ott": 1563,
+  "\u0120phys": 1564,
+  "\u0120just": 1565,
+  "\u0120space": 1566,
+  "put": 1567,
+  "\u0120wrote": 1568,
+  "\u0120object": 1569,
+  "\u0120suggest": 1570,
+  "\u0120West": 1571,
+  "ges": 1572,
+  "\u0120separ": 1573,
+  "\u0120material": 1574,
+  "\u0120systems": 1575,
+  "\u0120Fran": 1576,
+  "\u0120According": 1577,
+  "\u0120should": 1578,
+  "orthern": 1579,
+  "\u0120does": 1580,
+  "ival": 1581,
+  ".,": 1582,
+  "\u0120God": 1583,
+  "itive": 1584,
+  "\u0120read": 1585,
+  "atural": 1586,
+  "\u0120method": 1587,
+  "\u0120redu": 1588,
+  "\u0120low": 1589,
+  "\u0120activ": 1590,
+  "\u0120having": 1591,
+  "\u0120areas": 1592,
+  "\u0120Oct": 1593,
+  "\u0120relig": 1594,
+  "\u0120largest": 1595,
+  "\u0120Greek": 1596,
+  "\u0120named": 1597,
+  "\u0120Dav": 1598,
+  "aun": 1599,
+  "\u012014": 1600,
+  "\u0120produced": 1601,
+  "\u0120product": 1602,
+  "rated": 1603,
+  "emb": 1604,
+  "most": 1605,
+  "\u0120Ed": 1606,
+  "\u0120role": 1607,
+  "\u0120written": 1608,
+  "for": 1609,
+  "anu": 1610,
+  "\u0120Bro": 1611,
+  "\u0120cal": 1612,
+  "\u0120wide": 1613,
+  "aken": 1614,
+  "\u0120original": 1615,
+  "\u0120Christian": 1616,
+  "ek": 1617,
+  "\u0120individual": 1618,
+  "\u0120band": 1619,
+  "\u0120light": 1620,
+  "\u0120established": 1621,
+  "\u0120times": 1622,
+  "\u0120cell": 1623,
+  "\u0120test": 1624,
+  "ained": 1625,
+  "\u0120black": 1626,
+  "ouncil": 1627,
+  "\u0120Ber": 1628,
+  "\u0120prev": 1629,
+  "\u0120ep": 1630,
+  "\u0120help": 1631,
+  "\u0120er": 1632,
+  "\u0120decl": 1633,
+  "\u0120sym": 1634,
+  "\u0120north": 1635,
+  "che": 1636,
+  "\u0120appear": 1637,
+  "iving": 1638,
+  "iness": 1639,
+  "alf": 1640,
+  "\u012013": 1641,
+  "\u0120Union": 1642,
+  "ril": 1643,
+  "\u0120mean": 1644,
+  "\u0120rese": 1645,
+  "\u0120Sept": 1646,
+  "ape": 1647,
+  "ploy": 1648,
+  "\u0120real": 1649,
+  "\u0120must": 1650,
+  "\u0120mag": 1651,
+  "\u0120Her": 1652,
+  "omb": 1653,
+  "ends": 1654,
+  "\u0120win": 1655,
+  "augh": 1656,
+  "ype": 1657,
+  "ham": 1658,
+  "\u0120either": 1659,
+  "\u0120built": 1660,
+  "ched": 1661,
+  "ctor": 1662,
+  "\u0120sometimes": 1663,
+  "aur": 1664,
+  "aut": 1665,
+  "\u0120take": 1666,
+  "\u0120service": 1667,
+  "ote": 1668,
+  "\u0120bre": 1669,
+  "ways": 1670,
+  "\u0120five": 1671,
+  "ipp": 1672,
+  "alk": 1673,
+  "abor": 1674,
+  "thern": 1675,
+  "oms": 1676,
+  "\u0120king": 1677,
+  "ffic": 1678,
+  "\u0120Arab": 1679,
+  "ilos": 1680,
+  "\u0120released": 1681,
+  "ym": 1682,
+  "\u0120turn": 1683,
+  "bert": 1684,
+  "\u0120theory": 1685,
+  "\u0120England": 1686,
+  "\u0120sent": 1687,
+  "anes": 1688,
+  "\u0120Other": 1689,
+  "\u0120production": 1690,
+  "\u0120my": 1691,
+  "\u0120grow": 1692,
+  "zer": 1693,
+  "atures": 1694,
+  "\u0120del": 1695,
+  "\u0120others": 1696,
+  "\u0120el": 1697,
+  "umb": 1698,
+  "\u0120described": 1699,
+  "\u0120played": 1700,
+  "rian": 1701,
+  "ison": 1702,
+  "uly": 1703,
+  "outhern": 1704,
+  "\u0120games": 1705,
+  "\u0120top": 1706,
+  "\u0120generally": 1707,
+  "acy": 1708,
+  "\u0120base": 1709,
+  "ining": 1710,
+  "\u0120languages": 1711,
+  "\u0120social": 1712,
+  "ency": 1713,
+  "\u0120Although": 1714,
+  "\u0120six": 1715,
+  "ler": 1716,
+  "\u0120sw": 1717,
+  "pecially": 1718,
+  "\u0120south": 1719,
+  "\u0120America": 1720,
+  "\u0120strong": 1721,
+  "used": 1722,
+  "\u0120alb": 1723,
+  "\u0120study": 1724,
+  "face": 1725,
+  "\u0120September": 1726,
+  "eth": 1727,
+  "\u0120Gen": 1728,
+  "\u0120let": 1729,
+  "ille": 1730,
+  "\u0120\u0120": 1731,
+  "\u0120tit": 1732,
+  "ergy": 1733,
+  "overed": 1734,
+  "\u0120dest": 1735,
+  "urg": 1736,
+  "here": 1737,
+  "ected": 1738,
+  "\u0120Hist": 1739,
+  "yle": 1740,
+  "aces": 1741,
+  "\u0120official": 1742,
+  "\u0120relations": 1743,
+  "aries": 1744,
+  "\u0120groups": 1745,
+  "\u0120William": 1746,
+  "\u0120how": 1747,
+  "\u0120works": 1748,
+  "\u0120position": 1749,
+  "self": 1750,
+  "co": 1751,
+  "\u0120mass": 1752,
+  "\u0120special": 1753,
+  "ban": 1754,
+  "\u0120circ": 1755,
+  "aren": 1756,
+  "\u0120free": 1757,
+  "\u0120Act": 1758,
+  "\u0120earli": 1759,
+  "\u0120ver": 1760,
+  "ret": 1761,
+  "incip": 1762,
+  "ung": 1763,
+  "\u0120son": 1764,
+  "\u0120further": 1765,
+  "\u0120While": 1766,
+  "reen": 1767,
+  "ondon": 1768,
+  "\u0120came": 1769,
+  "\u0120Many": 1770,
+  "\u0120vol": 1771,
+  "fic": 1772,
+  "\u0120mot": 1773,
+  "ips": 1774,
+  "\u0120She": 1775,
+  "\u0120Jew": 1776,
+  "\u0120company": 1777,
+  "cul": 1778,
+  "anuary": 1779,
+  "\u0120song": 1780,
+  "\u0120addition": 1781,
+  "\u0120possible": 1782,
+  "\u0120With": 1783,
+  "\u0120rather": 1784,
+  "\u0120Black": 1785,
+  "bs": 1786,
+  "bon": 1787,
+  "\u0120post": 1788,
+  "\u0120Ac": 1789,
+  "\u0120Bel": 1790,
+  "ending": 1791,
+  "vision": 1792,
+  "asing": 1793,
+  "\u0120Br": 1794,
+  "\u0120created": 1795,
+  "\u0120version": 1796,
+  "\u0120October": 1797,
+  "\u0120syn": 1798,
+  "\u0120Republic": 1799,
+  "work": 1800,
+  "\u0120third": 1801,
+  "iation": 1802,
+  "\u0120Gu": 1803,
+  "\u0120January": 1804,
+  "osp": 1805,
+  "ify": 1806,
+  "mon": 1807,
+  "rap": 1808,
+  "\u0120ri": 1809,
+  "\u0120Alexander": 1810,
+  "sh": 1811,
+  "\u0120territ": 1812,
+  "\u0120191": 1813,
+  "\u0120Part": 1814,
+  "\u0120range": 1815,
+  "ridge": 1816,
+  "cember": 1817,
+  "\u0120forces": 1818,
+  "ification": 1819,
+  "let": 1820,
+  "\u0120express": 1821,
+  "\u0120Emp": 1822,
+  "\u0120avail": 1823,
+  "\u0120parts": 1824,
+  "then": 1825,
+  "ociety": 1826,
+  "\u0120certain": 1827,
+  "\u0120Har": 1828,
+  "ovember": 1829,
+  "\u0120chem": 1830,
+  "\u0120London": 1831,
+  "\u0120David": 1832,
+  "atin": 1833,
+  "sc": 1834,
+  "avy": 1835,
+  "\u0120current": 1836,
+  "rance": 1837,
+  "etic": 1838,
+  "\u0120tri": 1839,
+  "\u0120July": 1840,
+  "\u0120sold": 1841,
+  "\u0120International": 1842,
+  "\u0120Book": 1843,
+  "\u0120especially": 1844,
+  "\u0120March": 1845,
+  "\u0120enc": 1846,
+  "lor": 1847,
+  "\u0120East": 1848,
+  "utions": 1849,
+  "\u0120sum": 1850,
+  "\u0120meas": 1851,
+  "\u0120December": 1852,
+  "\u0120-": 1853,
+  "\u0120X": 1854,
+  "\u0120town": 1855,
+  "\u0120numbers": 1856,
+  "arian": 1857,
+  "\u0120April": 1858,
+  "aul": 1859,
+  "enced": 1860,
+  "\u0120Mon": 1861,
+  "\u0120November": 1862,
+  "\u0120aircraft": 1863,
+  "\u0120ast": 1864,
+  "\u0120bar": 1865,
+  "\u0120complex": 1866,
+  "iction": 1867,
+  "ope": 1868,
+  "\u0120food": 1869,
+  "\u0120good": 1870,
+  "eum": 1871,
+  "\u0120side": 1872,
+  "\u0120close": 1873,
+  "\u0120compet": 1874,
+  "\u0120continued": 1875,
+  "BC": 1876,
+  "\u0120capital": 1877,
+  "\u0120Med": 1878,
+  "\u0120Geor": 1879,
+  "arily": 1880,
+  "\u0120broad": 1881,
+  "\u0120recogn": 1882,
+  "\u0120computer": 1883,
+  "ateg": 1884,
+  "\u0120list": 1885,
+  "\u0120natural": 1886,
+  "\u0120data": 1887,
+  "\u0120June": 1888,
+  "\u0120Western": 1889,
+  "ming": 1890,
+  "\u0120interest": 1891,
+  "ps": 1892,
+  "\u0120pri": 1893,
+  "\u0120pot": 1894,
+  "\u0120Press": 1895,
+  "\u0120elements": 1896,
+  "iddle": 1897,
+  "ples": 1898,
+  "like": 1899,
+  "\u0120priv": 1900,
+  "\u0120educ": 1901,
+  "\u0120observ": 1902,
+  "\u0120album": 1903,
+  "ald": 1904,
+  "hers": 1905,
+  "aly": 1906,
+  "ths": 1907,
+  "ert": 1908,
+  "\u0120care": 1909,
+  "\u0120battle": 1910,
+  "\u0120Art": 1911,
+  "\u0120international": 1912,
+  "\u0120abs": 1913,
+  "\u0120Jap": 1914,
+  "\u0120frequ": 1915,
+  "oung": 1916,
+  "\u0120million": 1917,
+  "\u0120Empire": 1918,
+  "\u0120fail": 1919,
+  "aves": 1920,
+  "atter": 1921,
+  "rag": 1922,
+  "\u0120civil": 1923,
+  "\u0120App": 1924,
+  "ief": 1925,
+  "ront": 1926,
+  "\u0120member": 1927,
+  "iment": 1928,
+  "oyal": 1929,
+  "lation": 1930,
+  "\u0120hold": 1931,
+  "ounced": 1932,
+  "\u0120subject": 1933,
+  "\u0120accept": 1934,
+  "mit": 1935,
+  "wh": 1936,
+  "eration": 1937,
+  "\u0120formed": 1938,
+  "\u0120cat": 1939,
+  "ana": 1940,
+  "rug": 1941,
+  "ensive": 1942,
+  "chan": 1943,
+  "\u0120forms": 1944,
+  "\u0120City": 1945,
+  "\u0120Most": 1946,
+  "\u0120terms": 1947,
+  "\u0120received": 1948,
+  "\u0120never": 1949,
+  "\u0120type": 1950,
+  "\u0120season": 1951,
+  "\u0120impro": 1952,
+  "\u0120Army": 1953,
+  "\u0120press": 1954,
+  "\u0120admin": 1955,
+  "\u0120princip": 1956,
+  "int": 1957,
+  "\u0120der": 1958,
+  "viron": 1959,
+  "\u0120won": 1960,
+  "\u0120throughout": 1961,
+  "cur": 1962,
+  "reg": 1963,
+  "ators": 1964,
+  "\u0120cop": 1965,
+  "oid": 1966,
+  "ask": 1967,
+  "\u0120subst": 1968,
+  "eter": 1969,
+  "\u0120Man": 1970,
+  "\u0120children": 1971,
+  "\u0120women": 1972,
+  "\u0120available": 1973,
+  "\u0120192": 1974,
+  "adem": 1975,
+  "\u0120exec": 1976,
+  "\u0120account": 1977,
+  "ising": 1978,
+  "\u0120multip": 1979,
+  "\u0120ge": 1980,
+  "ights": 1981,
+  "ude": 1982,
+  "\u0120colon": 1983,
+  "hib": 1984,
+  "ibr": 1985,
+  "\u0120manag": 1986,
+  "\u0120engine": 1987,
+  "reme": 1988,
+  "\u0120seen": 1989,
+  "\u0120cy": 1990,
+  "\u0120African": 1991,
+  "\u0120host": 1992,
+  "\u0120died": 1993,
+  "\u0120ren": 1994,
+  "\u0120making": 1995,
+  "\u0120least": 1996,
+  "\u0120previous": 1997,
+  "osition": 1998,
+  "ached": 1999,
+  "\u0120laun": 2000,
+  "of": 2001,
+  "\u0120research": 2002,
+  "val": 2003,
+  "rum": 2004,
+  "ened": 2005,
+  "raw": 2006,
+  "\u0120evidence": 2007,
+  "\u0120Cor": 2008,
+  "\u0120central": 2009,
+  "\u0120complete": 2010,
+  "utes": 2011,
+  "ades": 2012,
+  "\u0120ground": 2013,
+  "sp": 2014,
+  "\u0120age": 2015,
+  "\u0120half": 2016,
+  "\u0120himself": 2017,
+  "\u0120Sw": 2018,
+  "\u0120lower": 2019,
+  "\u0120ancient": 2020,
+  "itu": 2021,
+  "\u0120Since": 2022,
+  "\u0120army": 2023,
+  "\u0120feature": 2024,
+  "stand": 2025,
+  "\u0120super": 2026,
+  "inese": 2027,
+  "\u0120Sov": 2028,
+  "\u0120Ant": 2029,
+  "\u0120philos": 2030,
+  "\u0120economic": 2031,
+  "\u0120thus": 2032,
+  "imate": 2033,
+  "istan": 2034,
+  "\u0120tro": 2035,
+  "\u0120capt": 2036,
+  "ria": 2037,
+  "\u0120force": 2038,
+  "\u0120Pres": 2039,
+  "\u0120coast": 2040,
+  "lam": 2041,
+  "\u0120offer": 2042,
+  "\u0120standard": 2043,
+  "\u0120information": 2044,
+  "\u0120regard": 2045,
+  "\u0120Australian": 2046,
+  "iber": 2047,
+  "alt": 2048,
+  "\u0120net": 2049,
+  "\u0120Ne": 2050,
+  "\u0120eth": 2051,
+  "\u0120pain": 2052,
+  "\u0120days": 2053,
+  "ption": 2054,
+  "\u0120den": 2055,
+  "\u0120Canada": 2056,
+  "erous": 2057,
+  "bo": 2058,
+  "\u0120Cath": 2059,
+  "\u0120command": 2060,
+  "kn": 2061,
+  "\u0120deb": 2062,
+  "\u0120If": 2063,
+  "play": 2064,
+  "aper": 2065,
+  "\u0120tre": 2066,
+  "\u0120Great": 2067,
+  "\u0120consist": 2068,
+  "\u0120island": 2069,
+  "ama": 2070,
+  "\u0120Co": 2071,
+  "\u0120above": 2072,
+  "\u0120find": 2073,
+  "\u0120Des": 2074,
+  "inary": 2075,
+  "\u0120Bra": 2076,
+  "ford": 2077,
+  "\u0120Pol": 2078,
+  "ique": 2079,
+  "\u0120Su": 2080,
+  "\u0120gr": 2081,
+  "\u0120across": 2082,
+  "arn": 2083,
+  "\u0120Africa": 2084,
+  "\u0120concept": 2085,
+  "\u0120means": 2086,
+  "\u0120hard": 2087,
+  "\u0120Ser": 2088,
+  "\u0120scient": 2089,
+  "cript": 2090,
+  "\u0120project": 2091,
+  "\u0120conc": 2092,
+  "\u0120join": 2093,
+  "\u0120court": 2094,
+  "\u0120ball": 2095,
+  "\u0120profess": 2096,
+  "\u0120taken": 2097,
+  "ula": 2098,
+  "rael": 2099,
+  "\u0120polic": 2100,
+  "\u0120To": 2101,
+  "lement": 2102,
+  "\u0120entire": 2103,
+  "\u0120Earth": 2104,
+  "\u0120River": 2105,
+  "gypt": 2106,
+  "\u0120best": 2107,
+  "pite": 2108,
+  "\u0120model": 2109,
+  "AS": 2110,
+  "\u0120flow": 2111,
+  "\u0120agre": 2112,
+  "iol": 2113,
+  "\u0120upon": 2114,
+  "ingu": 2115,
+  "\u0120tw": 2116,
+  "\u0120energy": 2117,
+  "\u0120determ": 2118,
+  "\u0120fire": 2119,
+  "\u0120begin": 2120,
+  "\u0120Kingdom": 2121,
+  "\u0120red": 2122,
+  "\u0120next": 2123,
+  "\u0120Comp": 2124,
+  "200": 2125,
+  "\u0120soft": 2126,
+  "inct": 2127,
+  "\u0120prote": 2128,
+  "\u0120Inst": 2129,
+  "\u0120includes": 2130,
+  "ney": 2131,
+  "\u0120located": 2132,
+  "\u0120president": 2133,
+  "\u0120party": 2134,
+  "\u0120traditional": 2135,
+  "\u0120maintain": 2136,
+  "\u0120Sy": 2137,
+  "\u0120access": 2138,
+  "ef": 2139,
+  "lant": 2140,
+  "bor": 2141,
+  "\u0120Egypt": 2142,
+  "\u0120House": 2143,
+  "\u0120Kh": 2144,
+  "\u0120player": 2145,
+  "\u0120indic": 2146,
+  "\u0120market": 2147,
+  "isc": 2148,
+  "\u0120followed": 2149,
+  "\u0120final": 2150,
+  "icro": 2151,
+  "\u0120Const": 2152,
+  "\u0120culture": 2153,
+  "\u0120conn": 2154,
+  "\u0120novel": 2155,
+  "\u0120sex": 2156,
+  "\u0120adop": 2157,
+  "\u0120successful": 2158,
+  "\u0120business": 2159,
+  "cle": 2160,
+  "\u0120total": 2161,
+  "ceed": 2162,
+  "pir": 2163,
+  "\u0120Te": 2164,
+  "\u0120France": 2165,
+  "oly": 2166,
+  "\u0120father": 2167,
+  "\u0120di": 2168,
+  "\u0120change": 2169,
+  "sel": 2170,
+  "\u0120Chinese": 2171,
+  "\u0120relative": 2172,
+  "\u0120full": 2173,
+  "\u0120neg": 2174,
+  "\u0120instead": 2175,
+  "\u0120anti": 2176,
+  "\u0120Res": 2177,
+  "iforn": 2178,
+  "\u0120related": 2179,
+  "ino": 2180,
+  "\u0120El": 2181,
+  "\u0120China": 2182,
+  "\u0120text": 2183,
+  "\u0120liter": 2184,
+  "rast": 2185,
+  "\u0120India": 2186,
+  "fact": 2187,
+  "\u0120doc": 2188,
+  "gether": 2189,
+  "ving": 2190,
+  "\u0120together": 2191,
+  "\u0120Latin": 2192,
+  "\u0120contain": 2193,
+  "\u0120foot": 2194,
+  "\u0120Sm": 2195,
+  "\u0120perm": 2196,
+  "\u0120associated": 2197,
+  "\u0120Soviet": 2198,
+  "\u0120norm": 2199,
+  "ads": 2200,
+  "\u0120thought": 2201,
+  "\u0120Conf": 2202,
+  "vironment": 2203,
+  "\u0120z": 2204,
+  "atory": 2205,
+  "\u0120trade": 2206,
+  "\u0120provide": 2207,
+  "\u0120sound": 2208,
+  "\u0120mid": 2209,
+  "\u0120Athen": 2210,
+  "\u0120cases": 2211,
+  "\u0120foc": 2212,
+  "\u0120Australia": 2213,
+  "\u0120higher": 2214,
+  "\u0120mer": 2215,
+  "\u0120except": 2216,
+  "\u0120Phil": 2217,
+  "\u0120Bas": 2218,
+  "udd": 2219,
+  "sy": 2220,
+  "\u0120invest": 2221,
+  "ebru": 2222,
+  "\u0120almost": 2223,
+  "order": 2224,
+  "\u0120imm": 2225,
+  "\u0120community": 2226,
+  "bra": 2227,
+  "unic": 2228,
+  "ebruary": 2229,
+  "\u0120see": 2230,
+  "aff": 2231,
+  "\u0120mount": 2232,
+  "\u0120becom": 2233,
+  "rest": 2234,
+  "mercial": 2235,
+  "\u0120itself": 2236,
+  "ography": 2237,
+  "action": 2238,
+  "\u0120cover": 2239,
+  "\u0120Pers": 2240,
+  "\u0120Britain": 2241,
+  "\u0120altern": 2242,
+  "\u0120movement": 2243,
+  "ficult": 2244,
+  "\u0120miss": 2245,
+  "\u0120Council": 2246,
+  "\u0120Pe": 2247,
+  "\u0120tar": 2248,
+  "\u0120hyd": 2249,
+  "cret": 2250,
+  "\u0120plan": 2251,
+  "ours": 2252,
+  "ternal": 2253,
+  "ica": 2254,
+  "liament": 2255,
+  "ought": 2256,
+  "not": 2257,
+  "\u0120ox": 2258,
+  "\u0120fac": 2259,
+  "icle": 2260,
+  "\u0120Count": 2261,
+  "\u0120employ": 2262,
+  "\u0120electron": 2263,
+  "\u0120films": 2264,
+  "\u0120key": 2265,
+  "\u0120influence": 2266,
+  "\u0120fav": 2267,
+  "\u0120players": 2268,
+  "\u0120according": 2269,
+  "\u0120introduced": 2270,
+  "\u0120concern": 2271,
+  "\u0120able": 2272,
+  "ports": 2273,
+  "\u0120difficult": 2274,
+  "\u0120temper": 2275,
+  "\u0120Central": 2276,
+  "\u0120particularly": 2277,
+  "aign": 2278,
+  "iers": 2279,
+  "\u0120February": 2280,
+  "\u0120applic": 2281,
+  "\u0120know": 2282,
+  "chie": 2283,
+  "\u0120referred": 2284,
+  "ma": 2285,
+  "\u0120features": 2286,
+  "\u0120reported": 2287,
+  "\u0120poin": 2288,
+  "\u0120prob": 2289,
+  "\u0120typically": 2290,
+  "roy": 2291,
+  "\u0120surv": 2292,
+  "\u0120rights": 2293,
+  "\u0120trib": 2294,
+  "amples": 2295,
+  "\u0120television": 2296,
+  "hest": 2297,
+  "anted": 2298,
+  "\u0120value": 2299,
+  "iles": 2300,
+  "\u0120allowed": 2301,
+  "\u0120building": 2302,
+  "\u0120Californ": 2303,
+  "atives": 2304,
+  "\u0120lack": 2305,
+  "ourt": 2306,
+  "ounter": 2307,
+  "yr": 2308,
+  "oted": 2309,
+  "\u0120respons": 2310,
+  "\u0120stri": 2311,
+  "\u0120tradition": 2312,
+  "\u0120State": 2313,
+  "\u0120defe": 2314,
+  "ush": 2315,
+  "ocr": 2316,
+  "\u0120travel": 2317,
+  "\u0120cross": 2318,
+  "\u0120far": 2319,
+  "\u0120compan": 2320,
+  "\u0120surface": 2321,
+  "medi": 2322,
+  "\u0120story": 2323,
+  "\u0120whose": 2324,
+  "cast": 2325,
+  "att": 2326,
+  "\u0120fri": 2327,
+  "ideo": 2328,
+  "\u0120continu": 2329,
+  "\u0120br": 2330,
+  "\u0120Camb": 2331,
+  "\u0120crew": 2332,
+  "\u0120fall": 2333,
+  "ibility": 2334,
+  "\u0120structure": 2335,
+  "\u0120contrib": 2336,
+  "\u0120fund": 2337,
+  "\u0120once": 2338,
+  "\u0120ste": 2339,
+  "\u0120deg": 2340,
+  "\u0120specific": 2341,
+  "\u0120required": 2342,
+  "\u0120words": 2343,
+  "\u0120Indian": 2344,
+  "imately": 2345,
+  "\u0120Catholic": 2346,
+  "ream": 2347,
+  "\u0120respect": 2348,
+  "\u0120Germany": 2349,
+  "\u0120schol": 2350,
+  "born": 2351,
+  "uel": 2352,
+  "oral": 2353,
+  "ties": 2354,
+  "iverse": 2355,
+  "\u0120size": 2356,
+  "iting": 2357,
+  "uth": 2358,
+  "ara": 2359,
+  "\u0120Arch": 2360,
+  "\u0120books": 2361,
+  "\u0120religious": 2362,
+  "\u0120Cong": 2363,
+  "\u0120From": 2364,
+  "\u0120services": 2365,
+  "\u0120Colle": 2366,
+  "clus": 2367,
+  "\u0120might": 2368,
+  "\u0120190": 2369,
+  "ysis": 2370,
+  "\u0120California": 2371,
+  "\u0120served": 2372,
+  "rought": 2373,
+  "\u0120administ": 2374,
+  "\u0120Park": 2375,
+  "\u0120science": 2376,
+  "\u0120(\"": 2377,
+  "\u0120Tur": 2378,
+  "anish": 2379,
+  "\u0120Me": 2380,
+  "II": 2381,
+  "\u0120star": 2382,
+  "\u0120League": 2383,
+  "\u0120integ": 2384,
+  "istry": 2385,
+  "\u0120mach": 2386,
+  "\u0120remained": 2387,
+  "oll": 2388,
+  "\u0120uses": 2389,
+  "\u0120disp": 2390,
+  "ederal": 2391,
+  "ume": 2392,
+  "uck": 2393,
+  "50": 2394,
+  "\u0120Cro": 2395,
+  "\u01202007": 2396,
+  "\u0120commonly": 2397,
+  "\u0120We": 2398,
+  "ixed": 2399,
+  "lying": 2400,
+  "\u0120types": 2401,
+  "\u0120Per": 2402,
+  "\u0120186": 2403,
+  "\u0120lo": 2404,
+  "\u0120Mc": 2405,
+  "\u0120meet": 2406,
+  "\u0120believed": 2407,
+  "ulf": 2408,
+  "\u0120fig": 2409,
+  "\u0120house": 2410,
+  "oses": 2411,
+  "\u0120industry": 2412,
+  "\u0120prevent": 2413,
+  "\u0120events": 2414,
+  "\u0120communic": 2415,
+  "\u0120gre": 2416,
+  "\u01201990": 2417,
+  "\u0120meaning": 2418,
+  "\u0120style": 2419,
+  "\u0120provided": 2420,
+  "\u0120Atlant": 2421,
+  "itch": 2422,
+  "ols": 2423,
+  "aps": 2424,
+  "\u0120vict": 2425,
+  "\u0120mechan": 2426,
+  "ems": 2427,
+  "\u0120Sea": 2428,
+  "\u0120ill": 2429,
+  "ington": 2430,
+  "rial": 2431,
+  "\u0120Em": 2432,
+  "\u0120jud": 2433,
+  "\u0120announced": 2434,
+  "\u0120white": 2435,
+  "\u0120increasing": 2436,
+  "itect": 2437,
+  "\u0120pen": 2438,
+  "\u0120nuc": 2439,
+  "\u0120commercial": 2440,
+  "\u0120element": 2441,
+  "\u0120tour": 2442,
+  "\u0120larger": 2443,
+  "\u0120emer": 2444,
+  "\u0120behav": 2445,
+  "\u0120Rober": 2446,
+  "ength": 2447,
+  "uries": 2448,
+  "\u0120cause": 2449,
+  "\u0120San": 2450,
+  "\u0120Charles": 2451,
+  "\u0120Reg": 2452,
+  "\u0120killed": 2453,
+  "\u0120northern": 2454,
+  "\u0120site": 2455,
+  "\u0120necess": 2456,
+  "ires": 2457,
+  "ulated": 2458,
+  "\u0120rule": 2459,
+  "\u0120effects": 2460,
+  "\u0120School": 2461,
+  "\u0120moved": 2462,
+  "\u0120mission": 2463,
+  "\u0120interp": 2464,
+  "\u0120campaign": 2465,
+  "\u0120Pal": 2466,
+  "\u0120Port": 2467,
+  "\u0120defin": 2468,
+  "wood": 2469,
+  "\u0120rock": 2470,
+  "\u0120quest": 2471,
+  "lim": 2472,
+  "\u0120whether": 2473,
+  "\u01202006": 2474,
+  "\u0120prin": 2475,
+  "\u012021": 2476,
+  "su": 2477,
+  "\u0120lost": 2478,
+  "199": 2479,
+  "\u0120source": 2480,
+  "\u0120dou": 2481,
+  "iding": 2482,
+  "\u0120ord": 2483,
+  "\u0120designed": 2484,
+  "\u0120critic": 2485,
+  "irth": 2486,
+  "go": 2487,
+  "\u0120transport": 2488,
+  "\u0120widely": 2489,
+  "\u0120god": 2490,
+  "omm": 2491,
+  "\u012030": 2492,
+  "ership": 2493,
+  "\u0120consider": 2494,
+  "\u0120aud": 2495,
+  "orph": 2496,
+  "\u0120put": 2497,
+  "\u0120domin": 2498,
+  "off": 2499,
+  "\u0120George": 2500,
+  "\u0120past": 2501,
+  "istance": 2502,
+  "\u0120gave": 2503,
+  "\u0120James": 2504,
+  "\u01202010": 2505,
+  "oney": 2506,
+  "\u01201980": 2507,
+  "ophy": 2508,
+  "\u0120log": 2509,
+  "\u0120longer": 2510,
+  "\u0120achie": 2511,
+  "\u0120ang": 2512,
+  "\u0120legal": 2513,
+  "ourn": 2514,
+  "\u0120outside": 2515,
+  "\u0120little": 2516,
+  "based": 2517,
+  "ference": 2518,
+  "het": 2519,
+  "\u0120nature": 2520,
+  "\u01202000": 2521,
+  "\u0120live": 2522,
+  "\u0120church": 2523,
+  "\u0120pred": 2524,
+  "\u0120produce": 2525,
+  "\u0120private": 2526,
+  "\u0120caused": 2527,
+  "\u0120purp": 2528,
+  "\u0120Party": 2529,
+  "selves": 2530,
+  "\u0120mother": 2531,
+  "\u0120rail": 2532,
+  "\u0120increased": 2533,
+  "\u0120started": 2534,
+  "anks": 2535,
+  "\u0120fem": 2536,
+  "\u0120target": 2537,
+  "\u0120went": 2538,
+  "\u0120Court": 2539,
+  "\u0120suff": 2540,
+  "\u01202008": 2541,
+  "ranch": 2542,
+  "\u0120distrib": 2543,
+  "\u01201970": 2544,
+  "\u0120symb": 2545,
+  "ilities": 2546,
+  "\u0120Thom": 2547,
+  "\u0120carbon": 2548,
+  "\u0120brought": 2549,
+  "ental": 2550,
+  "\u01202011": 2551,
+  "\u0120squ": 2552,
+  "\u0120blood": 2553,
+  "\u0120health": 2554,
+  "\u0120No": 2555,
+  "\u0120eventually": 2556,
+  "\u0120UK": 2557,
+  "\u0120astron": 2558,
+  "\u0120away": 2559,
+  "\u01202009": 2560,
+  "ibrary": 2561,
+  "\u0120environment": 2562,
+  "\u0120reb": 2563,
+  "\u0120President": 2564,
+  "\u0120[": 2565,
+  "\u0120remains": 2566,
+  "view": 2567,
+  "\u0120conditions": 2568,
+  "\u0120viol": 2569,
+  "\u0120Cont": 2570,
+  "\u0120writing": 2571,
+  "\u0120leading": 2572,
+  "\u0120likely": 2573,
+  "\u0120sat": 2574,
+  "\u0120General": 2575,
+  "itute": 2576,
+  "\u0120gra": 2577,
+  "aching": 2578,
+  "anese": 2579,
+  "\u0120dial": 2580,
+  "\u0120appeared": 2581,
+  "\")": 2582,
+  "\u0120mathem": 2583,
+  "\u0120independent": 2584,
+  "\u0120education": 2585,
+  "\u0120fle": 2586,
+  "\u0120Dem": 2587,
+  "ills": 2588,
+  "eleb": 2589,
+  "anced": 2590,
+  "\u0120relationship": 2591,
+  "\u0120cities": 2592,
+  "light": 2593,
+  "\u0120changes": 2594,
+  "oice": 2595,
+  "\u0120Asia": 2596,
+  "\u0120added": 2597,
+  "reet": 2598,
+  "\u0120election": 2599,
+  "\u0120radio": 2600,
+  "ores": 2601,
+  "\u0120code": 2602,
+  "\u0120returned": 2603,
+  "\u0120Ste": 2604,
+  "\u0120effort": 2605,
+  "\u0120require": 2606,
+  "\u0120replaced": 2607,
+  "\u0120hydro": 2608,
+  "otic": 2609,
+  "\u0120sea": 2610,
+  "rate": 2611,
+  "\u0120coll": 2612,
+  "itiz": 2613,
+  "\u0120Govern": 2614,
+  "\u0120regions": 2615,
+  "\u0120problems": 2616,
+  "hing": 2617,
+  "\u0120clear": 2618,
+  "\u0120et": 2619,
+  "\u0120themselves": 2620,
+  "\u0120problem": 2621,
+  "\u0120software": 2622,
+  "\u0120southern": 2623,
+  "ufact": 2624,
+  "amer": 2625,
+  "\u0120High": 2626,
+  "\u0120physical": 2627,
+  "\u0120vir": 2628,
+  "\u01201960": 2629,
+  "\u0120animals": 2630,
+  "\u0120self": 2631,
+  "\u0120west": 2632,
+  "\u0120acid": 2633,
+  "\u0120particip": 2634,
+  "\u0120eas": 2635,
+  "\u0120conver": 2636,
+  "ricult": 2637,
+  "\u0120189": 2638,
+  "\u0120action": 2639,
+  "\u0120Royal": 2640,
+  "\u0120manufact": 2641,
+  "ett": 2642,
+  "irit": 2643,
+  "\u0120companies": 2644,
+  "\u0120cost": 2645,
+  "uture": 2646,
+  "\u0120condu": 2647,
+  "pper": 2648,
+  "acters": 2649,
+  "\u0120variety": 2650,
+  "\u0120east": 2651,
+  "\u0120defined": 2652,
+  "asion": 2653,
+  "\u0120reason": 2654,
+  "\u0120AD": 2655,
+  "\u0120practice": 2656,
+  "\u0120Bay": 2657,
+  "iff": 2658,
+  "\u0120quant": 2659,
+  "\u0120recorded": 2660,
+  "\u0120ten": 2661,
+  "ows": 2662,
+  "ques": 2663,
+  "\u0120months": 2664,
+  "\u0120limited": 2665,
+  "\u0120dig": 2666,
+  "\u0120Bur": 2667,
+  "\u0120founded": 2668,
+  "lied": 2669,
+  "mar": 2670,
+  "\u0120speak": 2671,
+  "\u0120poly": 2672,
+  "\u0120fight": 2673,
+  "\u0120Columb": 2674,
+  "ael": 2675,
+  "\u0120bank": 2676,
+  "\u0120Gl": 2677,
+  "aval": 2678,
+  "\u0120brother": 2679,
+  "\u0120dom": 2680,
+  "ump": 2681,
+  "elling": 2682,
+  "\u0120subsequ": 2683,
+  "\u0120always": 2684,
+  "\u0120ris": 2685,
+  "\u0120say": 2686,
+  "\u0120Hen": 2687,
+  "\u0120characters": 2688,
+  "\u0120establish": 2689,
+  "onse": 2690,
+  "itor": 2691,
+  "odes": 2692,
+  "\u0120majority": 2693,
+  "\u0120Trans": 2694,
+  "cies": 2695,
+  "\u0120contro": 2696,
+  "isms": 2697,
+  "icles": 2698,
+  "oul": 2699,
+  "\u0120saw": 2700,
+  "\u0120Mor": 2701,
+  "\u0120Paul": 2702,
+  "\u0120video": 2703,
+  "\u0120club": 2704,
+  "\u0120foreign": 2705,
+  "\u0120AC": 2706,
+  "\u0120stated": 2707,
+  "\u0120dam": 2708,
+  "\u0120grad": 2709,
+  "\u0120amount": 2710,
+  "\u0120cultural": 2711,
+  "resp": 2712,
+  "\u0120ess": 2713,
+  "\u0120Day": 2714,
+  "\u0120regular": 2715,
+  "\u0120Israel": 2716,
+  "artment": 2717,
+  "\u0120dise": 2718,
+  "\u0120Frank": 2719,
+  "\u0120chemical": 2720,
+  "\u0120towards": 2721,
+  "\u0120hon": 2722,
+  "\u0120alg": 2723,
+  "\u0120involved": 2724,
+  "\u0120come": 2725,
+  "\u0120percent": 2726,
+  "\u0120approx": 2727,
+  "\u0120Battle": 2728,
+  "\u0120hom": 2729,
+  "\u0120smaller": 2730,
+  "\u0120directly": 2731,
+  "ike": 2732,
+  "\u0120che": 2733,
+  "ror": 2734,
+  "irds": 2735,
+  "\u0120personal": 2736,
+  "\u0120architect": 2737,
+  "\u0120Mich": 2738,
+  "alled": 2739,
+  "phas": 2740,
+  "\u0120historical": 2741,
+  "\u0120territory": 2742,
+  "isions": 2743,
+  "\u01202005": 2744,
+  "ulg": 2745,
+  "\u0120station": 2746,
+  "\u01202014": 2747,
+  "\u0120distinct": 2748,
+  "\u0120Russian": 2749,
+  "\u0120award": 2750,
+  "berg": 2751,
+  "ously": 2752,
+  "\u01202012": 2753,
+  "ali": 2754,
+  "\u0120constitution": 2755,
+  "\u0120lines": 2756,
+  "\u0120Ben": 2757,
+  "\u0120Bab": 2758,
+  "och": 2759,
+  "\u0120celeb": 2760,
+  "\u0120too": 2761,
+  "\u0120today": 2762,
+  "\u0120Mount": 2763,
+  "ects": 2764,
+  "\u01202013": 2765,
+  "\u0120primary": 2766,
+  "\u0120title": 2767,
+  "\u0120treatment": 2768,
+  "\u0120dyn": 2769,
+  "utive": 2770,
+  "\u012025": 2771,
+  "key": 2772,
+  "18": 2773,
+  "itude": 2774,
+  "sych": 2775,
+  "\u0120remov": 2776,
+  "rang": 2777,
+  "\u0120100": 2778,
+  "\u0120opt": 2779,
+  "\u0120adapt": 2780,
+  "\u0120young": 2781,
+  "vention": 2782,
+  "\u0120increase": 2783,
+  "known": 2784,
+  "\u0120names": 2785,
+  "\u0120Hol": 2786,
+  "ands": 2787,
+  "ression": 2788,
+  "\u0120earlier": 2789,
+  "igned": 2790,
+  "ounds": 2791,
+  "\u0120mostly": 2792,
+  "ux": 2793,
+  "\u0120Afgh": 2794,
+  "\u0120veh": 2795,
+  "\u0120Robert": 2796,
+  "\u0120aver": 2797,
+  "\u0120Mod": 2798,
+  "\u0120Its": 2799,
+  "\u0120nar": 2800,
+  "ests": 2801,
+  "peror": 2802,
+  "\u0120results": 2803,
+  "\u0120ax": 2804,
+  "aria": 2805,
+  "\u0120ur": 2806,
+  "anded": 2807,
+  "\u0120Rich": 2808,
+  "\u0120spok": 2809,
+  "ection": 2810,
+  "\u0120fish": 2811,
+  "asc": 2812,
+  "\u0120Sim": 2813,
+  "ables": 2814,
+  "ledge": 2815,
+  "\u0120bene": 2816,
+  "ste": 2817,
+  "\u0120Spe": 2818,
+  "uk": 2819,
+  "\u0120front": 2820,
+  "\u0120Pat": 2821,
+  "nel": 2822,
+  "\u0120situ": 2823,
+  "\u0120lay": 2824,
+  "\u0120Museum": 2825,
+  "\u0120products": 2826,
+  "\u0120destroy": 2827,
+  "ha": 2828,
+  "rative": 2829,
+  "\u0120born": 2830,
+  "porary": 2831,
+  "\u0120placed": 2832,
+  "\u0120sett": 2833,
+  "\u0120occup": 2834,
+  "\u0120gun": 2835,
+  "istics": 2836,
+  "\u0120alc": 2837,
+  "abit": 2838,
+  "\u0120commit": 2839,
+  "\u0120Qu": 2840,
+  "\u0120gold": 2841,
+  "\u0120Albert": 2842,
+  "\u0120elected": 2843,
+  "\u0120separate": 2844,
+  "\u0120188": 2845,
+  "\u0120phot": 2846,
+  "iter": 2847,
+  "agn": 2848,
+  "\u0120sequ": 2849,
+  "\u0120Another": 2850,
+  "\u0120acqu": 2851,
+  "field": 2852,
+  "\u0120eight": 2853,
+  "\u0120studies": 2854,
+  "\u01202017": 2855,
+  "\u0120sil": 2856,
+  "\u0120length": 2857,
+  "\u0120Cons": 2858,
+  "ishop": 2859,
+  "ios": 2860,
+  "\u0120sal": 2861,
+  "aint": 2862,
+  "\u0120multiple": 2863,
+  "ault": 2864,
+  "\u0120living": 2865,
+  "\u0120My": 2866,
+  "\u0120construction": 2867,
+  "respond": 2868,
+  "\u0120But": 2869,
+  "\u0120operations": 2870,
+  "\u01202020": 2871,
+  "\u0120discuss": 2872,
+  "iences": 2873,
+  "\u0120Sam": 2874,
+  "throp": 2875,
+  "\u0120islands": 2876,
+  "unar": 2877,
+  "\u0120alle": 2878,
+  "onic": 2879,
+  "rick": 2880,
+  "\u0120Islands": 2881,
+  "\u0120functions": 2882,
+  "isation": 2883,
+  "ago": 2884,
+  "see": 2885,
+  "\u0120proposed": 2886,
+  "\u0120originally": 2887,
+  "\u0120Apple": 2888,
+  "\u0120Budd": 2889,
+  "\u0120response": 2890,
+  "\u0120document": 2891,
+  "\u0120Congress": 2892,
+  "\u0120cut": 2893,
+  "\u0120working": 2894,
+  "\u0120metal": 2895,
+  "\u0120troops": 2896,
+  "\u0120187": 2897,
+  "\u0120board": 2898,
+  "oe": 2899,
+  "\u0120Eastern": 2900,
+  "\u0120largely": 2901,
+  "\u0120network": 2902,
+  "\u0120beginning": 2903,
+  "ky": 2904,
+  "\u0120transl": 2905,
+  "\u0120sources": 2906,
+  "\u0120tax": 2907,
+  "\u01202004": 2908,
+  "\u0120rules": 2909,
+  "\u0120College": 2910,
+  "\u0120greater": 2911,
+  "ements": 2912,
+  "\u0120rap": 2913,
+  "\u0120Islam": 2914,
+  "\u0120units": 2915,
+  "ancial": 2916,
+  "obal": 2917,
+  "rus": 2918,
+  "izing": 2919,
+  "\u01202001": 2920,
+  "oz": 2921,
+  "\u0120technology": 2922,
+  "\u0120policy": 2923,
+  "year": 2924,
+  "\u0120hit": 2925,
+  "\u01202016": 2926,
+  "\u0120properties": 2927,
+  "\u0120flight": 2928,
+  "\u012024": 2929,
+  "\u0120numerous": 2930,
+  "\u0120economy": 2931,
+  "\u0120schools": 2932,
+  "rel": 2933,
+  "\u0120First": 2934,
+  "ivity": 2935,
+  "\u0120discovered": 2936,
+  "\u0120future": 2937,
+  "\u0120methods": 2938,
+  "\u0120magn": 2939,
+  "\u0120adopted": 2940,
+  "\u0120points": 2941,
+  "'t": 2942,
+  "pected": 2943,
+  "\u0120unit": 2944,
+  "\u0120divided": 2945,
+  "\u0120creat": 2946,
+  "\u0120below": 2947,
+  "\u0120Dep": 2948,
+  "\u0120Gold": 2949,
+  "\u0120seven": 2950,
+  "\u0120color": 2951,
+  "\u0120review": 2952,
+  "\u0120Gro": 2953,
+  "\u0120address": 2954,
+  "\u0120surround": 2955,
+  "anned": 2956,
+  "\u0120saf": 2957,
+  "\u0120plants": 2958,
+  "ready": 2959,
+  "embly": 2960,
+  "jan": 2961,
+  "\u0120Old": 2962,
+  "\u0120understand": 2963,
+  "\u0120office": 2964,
+  "arent": 2965,
+  "\u0120Afghan": 2966,
+  "ech": 2967,
+  "isl": 2968,
+  "\u0120better": 2969,
+  "\u0120analysis": 2970,
+  "aughter": 2971,
+  "\u0120sac": 2972,
+  "aged": 2973,
+  "airs": 2974,
+  "\u0120Batman": 2975,
+  "\u0120highest": 2976,
+  "\u0120look": 2977,
+  "ili": 2978,
+  "\u0120create": 2979,
+  "\u0120give": 2980,
+  "\u0120Je": 2981,
+  "\u0120Island": 2982,
+  "\u0120Associ": 2983,
+  "\u0120growth": 2984,
+  "\u0120Academ": 2985,
+  "\u0120bound": 2986,
+  "\u0120Red": 2987,
+  "ients": 2988,
+  "\u0120threat": 2989,
+  "onst": 2990,
+  "\u0120recent": 2991,
+  "\u0120implement": 2992,
+  "\u0120performed": 2993,
+  "\u0120Scott": 2994,
+  "axim": 2995,
+  "nd": 2996,
+  "\u0120sem": 2997,
+  "\u0120categ": 2998,
+  "\u0120via": 2999,
+  "\u0120correspond": 3000,
+  "omy": 3001,
+  "\u0120gas": 3002,
+  "\u0120oil": 3003,
+  "\u0120epis": 3004,
+  "curity": 3005,
+  "time": 3006,
+  "\u0120society": 3007,
+  "\u0120Histor": 3008,
+  "\u0120cust": 3009,
+  "orial": 3010,
+  "\u0120concent": 3011,
+  "aling": 3012,
+  "rying": 3013,
+  "\u01202018": 3014,
+  "\u0120emb": 3015,
+  "\u0120idea": 3016,
+  "men": 3017,
+  "rim": 3018,
+  "\u01202015": 3019,
+  "\u0120heav": 3020,
+  "\u0120centuries": 3021,
+  "spe": 3022,
+  "now": 3023,
+  "\u0120minor": 3024,
+  "\u0120Spanish": 3025,
+  "\u0120hous": 3026,
+  "osing": 3027,
+  "utch": 3028,
+  "\u0120vill": 3029,
+  "17": 3030,
+  "met": 3031,
+  "\u0120highly": 3032,
+  "\u0120supported": 3033,
+  "ension": 3034,
+  "na": 3035,
+  "\u0120Mal": 3036,
+  "\u0120cells": 3037,
+  "\u0120farm": 3038,
+  "\u0120ships": 3039,
+  "\u0120media": 3040,
+  "\u0120shown": 3041,
+  "\u0120students": 3042,
+  "\u0120Henry": 3043,
+  "\u0120dri": 3044,
+  "ext": 3045,
+  "\u0120Jewish": 3046,
+  "\u0120compon": 3047,
+  "ferred": 3048,
+  "\u0120laws": 3049,
+  "\u0120lab": 3050,
+  "\u0120loss": 3051,
+  "\u0120cannot": 3052,
+  "\u0120training": 3053,
+  ".)": 3054,
+  "\u0120Bu": 3055,
+  "\u0120already": 3056,
+  "olved": 3057,
+  "\u0120stage": 3058,
+  "30": 3059,
+  "anc": 3060,
+  "aring": 3061,
+  "\u0120latter": 3062,
+  "\u0120mathemat": 3063,
+  "\u0120drug": 3064,
+  "\u0120Mass": 3065,
+  "rane": 3066,
+  "\u0120release": 3067,
+  "\u0120river": 3068,
+  "\u0120relatively": 3069,
+  "\u0120hor": 3070,
+  "\u0120rate": 3071,
+  "ampions": 3072,
+  "pro": 3073,
+  "\u0120wind": 3074,
+  "icated": 3075,
+  "\u0120script": 3076,
+  "\u0120additional": 3077,
+  "west": 3078,
+  "\u0120wall": 3079,
+  "gebra": 3080,
+  "\u0120plant": 3081,
+  "\u0120cred": 3082,
+  "\u0120pun": 3083,
+  "\u0120Jack": 3084,
+  "\u0120immedi": 3085,
+  "\u0120active": 3086,
+  "\u0120obtain": 3087,
+  "\u0120estim": 3088,
+  "\u0120neigh": 3089,
+  "\u01202019": 3090,
+  "\u0120belief": 3091,
+  "enes": 3092,
+  "ston": 3093,
+  "\u0120Northern": 3094,
+  "\u0120ens": 3095,
+  "\u01202021": 3096,
+  "\u0120western": 3097,
+  "erson": 3098,
+  "\u0120Rev": 3099,
+  "\u0120mention": 3100,
+  "\u0120therefore": 3101,
+  "\u0120focus": 3102,
+  "ked": 3103,
+  "hel": 3104,
+  "ises": 3105,
+  "\u0120enough": 3106,
+  "ey": 3107,
+  "\u0120cast": 3108,
+  "urches": 3109,
+  "ider": 3110,
+  "\u0120Met": 3111,
+  "\u0120detail": 3112,
+  "\u0120collection": 3113,
+  "bre": 3114,
+  "ta": 3115,
+  "\u0120Mac": 3116,
+  "\u012026": 3117,
+  "\u0120prior": 3118,
+  "\u0120whom": 3119,
+  "\u0120heart": 3120,
+  "\u0120agricult": 3121,
+  "\u0120conflic": 3122,
+  "\u0120alternative": 3123,
+  "change": 3124,
+  "\u0120univers": 3125,
+  "\u0120History": 3126,
+  "10": 3127,
+  "\u0120wood": 3128,
+  "asons": 3129,
+  "oles": 3130,
+  "\u0120rare": 3131,
+  "\u0120medic": 3132,
+  "\u0120Japanese": 3133,
+  "\u0120Net": 3134,
+  "\u0120primarily": 3135,
+  "\u0120Smith": 3136,
+  "eds": 3137,
+  "omp": 3138,
+  "omer": 3139,
+  "unity": 3140,
+  "\u0120Hall": 3141,
+  "\u0120performance": 3142,
+  "\u0120Middle": 3143,
+  "odies": 3144,
+  "\u0120Alexand": 3145,
+  "burg": 3146,
+  "part": 3147,
+  "\u0120birth": 3148,
+  "aches": 3149,
+  "\u0120Jer": 3150,
+  "\u0120path": 3151,
+  "cont": 3152,
+  "\u0120scholars": 3153,
+  "cher": 3154,
+  "ointed": 3155,
+  "\u0120changed": 3156,
+  "ithm": 3157,
+  "ords": 3158,
+  "\u0120wife": 3159,
+  "\u0120()": 3160,
+  "\u0120Rad": 3161,
+  "\u0120Que": 3162,
+  "\u0120counter": 3163,
+  "\u0120citiz": 3164,
+  "\u0120Law": 3165,
+  "\u0120III": 3166,
+  "\u0120independence": 3167,
+  "\u0120Bal": 3168,
+  "\u0120Bell": 3169,
+  "\u0120bur": 3170,
+  "\u0120keep": 3171,
+  "\u0120suggested": 3172,
+  "fficient": 3173,
+  "\u01202003": 3174,
+  "\u0120potential": 3175,
+  "OS": 3176,
+  "war": 3177,
+  "\u0120done": 3178,
+  "\u0120display": 3179,
+  "\u0120memory": 3180,
+  "\u0120represented": 3181,
+  "\u0120property": 3182,
+  "\u0120get": 3183,
+  "\u0120contrast": 3184,
+  "\u01201999": 3185,
+  "\u0120basis": 3186,
+  "\u0120activity": 3187,
+  "\u0120psych": 3188,
+  "\u0120Vir": 3189,
+  "\u0120reached": 3190,
+  "olk": 3191,
+  "ij": 3192,
+  "\u0120slow": 3193,
+  "\u0120Big": 3194,
+  "reng": 3195,
+  "\u0120location": 3196,
+  "\u0120prec": 3197,
+  "inn": 3198,
+  "\u0120Italy": 3199,
+  "\u0120Ze": 3200,
+  "ls": 3201,
+  "aled": 3202,
+  "\u0120Award": 3203,
+  "ift": 3204,
+  "\u0120Wash": 3205,
+  "\u0120you": 3206,
+  "\u0120consum": 3207,
+  "\u0120center": 3208,
+  "\u0120esc": 3209,
+  "\u0120issue": 3210,
+  "\u0120Cat": 3211,
+  "igen": 3212,
+  "owers": 3213,
+  "\u0120Pri": 3214,
+  "\u0120spread": 3215,
+  "\u0120quick": 3216,
+  "\u0120levels": 3217,
+  "\u0120pressure": 3218,
+  "\u0120Sen": 3219,
+  "vert": 3220,
+  "\u0120antib": 3221,
+  "\u0120claimed": 3222,
+  "\u0120atom": 3223,
+  "\u0120ship": 3224,
+  "\u0120expans": 3225,
+  "\u0120enter": 3226,
+  "\u0120average": 3227,
+  "\u0120ever": 3228,
+  "umn": 3229,
+  "pped": 3230,
+  "\u0120status": 3231,
+  ");": 3232,
+  "tt": 3233,
+  "\u0120hyp": 3234,
+  "ulations": 3235,
+  "opp": 3236,
+  "\u0120noted": 3237,
+  "\u0120fell": 3238,
+  "lected": 3239,
+  "\u0120Sl": 3240,
+  "uments": 3241,
+  "\u0120diss": 3242,
+  "198": 3243,
+  "\u0120incor": 3244,
+  "\u0120settle": 3245,
+  "back": 3246,
+  "osaur": 3247,
+  "\u0120Bulg": 3248,
+  "cles": 3249,
+  "\u0120contains": 3250,
+  "\u0120whole": 3251,
+  "\u0120sens": 3252,
+  "\u0120Val": 3253,
+  "\u0120measure": 3254,
+  "\u0120card": 3255,
+  "itors": 3256,
+  "\u0120mole": 3257,
+  "\u0120Im": 3258,
+  "\u0120Wall": 3259,
+  "\u01202002": 3260,
+  "\u0120pie": 3261,
+  "\u0120hun": 3262,
+  "\u0120centre": 3263,
+  "\u0120blue": 3264,
+  "\u0120mainly": 3265,
+  "lines": 3266,
+  "\u0120carried": 3267,
+  "\u01201950": 3268,
+  "ging": 3269,
+  "\u0120intended": 3270,
+  "\u0120objects": 3271,
+  "\u0120materials": 3272,
+  "\u0120draw": 3273,
+  "\u0120nation": 3274,
+  "\u0120extreme": 3275,
+  "\u0120matter": 3276,
+  "\u0120Armen": 3277,
+  "\u0120opened": 3278,
+  "\u0120forced": 3279,
+  "\u0120appears": 3280,
+  "edy": 3281,
+  "\u0120experience": 3282,
+  "\u0120career": 3283,
+  "\u0120degree": 3284,
+  "reland": 3285,
+  "\u0120sense": 3286,
+  "cean": 3287,
+  "\u0120provides": 3288,
+  "\u0120initial": 3289,
+  "\u0120behavior": 3290,
+  "\u0120Ro": 3291,
+  "\u0120kind": 3292,
+  "\u0120Scot": 3293,
+  "\u0120financial": 3294,
+  "\u0120Arabic": 3295,
+  "\u0120Academy": 3296,
+  "\u0120AS": 3297,
+  "\u0120scientific": 3298,
+  "\u0120Washington": 3299,
+  "ee": 3300,
+  "onia": 3301,
+  "\u0120presence": 3302,
+  "\u0120break": 3303,
+  "\u0120Richard": 3304,
+  "amm": 3305,
+  "\u0120Fre": 3306,
+  "\u0120contem": 3307,
+  "\u0120professional": 3308,
+  "\u0120Thomas": 3309,
+  "\u0120coin": 3310,
+  "\u0120nearly": 3311,
+  "\u0120Ca": 3312,
+  "\u0120Society": 3313,
+  "IC": 3314,
+  "\u0120conqu": 3315,
+  "\u0120leader": 3316,
+  "\u0120football": 3317,
+  "\u0120peace": 3318,
+  "\u0120weap": 3319,
+  "\u0120shows": 3320,
+  "\u0120issues": 3321,
+  "ias": 3322,
+  "ellig": 3323,
+  "\u0120influenced": 3324,
+  "head": 3325,
+  "\u0120impact": 3326,
+  "\u0120rapid": 3327,
+  "\u0120soon": 3328,
+  "alian": 3329,
+  "\u0120makes": 3330,
+  "\u0120Ter": 3331,
+  "\u0120Dutch": 3332,
+  "estival": 3333,
+  "\u0120Oly": 3334,
+  "rupt": 3335,
+  "NA": 3336,
+  "erve": 3337,
+  "\u0120Ireland": 3338,
+  "osis": 3339,
+  "\u0120i": 3340,
+  "\u0120question": 3341,
+  "\u0120More": 3342,
+  "\u0120pay": 3343,
+  "imal": 3344,
+  "\u0120resulting": 3345,
+  "\u0120emp": 3346,
+  "\u0120demonst": 3347,
+  "\u0120block": 3348,
+  "ournal": 3349,
+  "\u0120Parliament": 3350,
+  "\u0120federal": 3351,
+  "\u0120algebra": 3352,
+  "\u0120algor": 3353,
+  "\u0120ded": 3354,
+  "\u0120eastern": 3355,
+  "\u0120creation": 3356,
+  "\u0120Bak": 3357,
+  "\u0120Follow": 3358,
+  "\u0120amb": 3359,
+  "\u0120improve": 3360,
+  "\u0120joined": 3361,
+  "\u0120Inter": 3362,
+  "\u0120La": 3363,
+  "\u0120experiment": 3364,
+  "\u0120earliest": 3365,
+  "\u0120ethn": 3366,
+  "\u0120Olymp": 3367,
+  "cks": 3368,
+  "hood": 3369,
+  "ply": 3370,
+  "\u0120worked": 3371,
+  "\u0120Force": 3372,
+  "\u0120era": 3373,
+  "ocratic": 3374,
+  "\u0120machine": 3375,
+  "\u0120BBC": 3376,
+  "\u0120streng": 3377,
+  "\u0120ability": 3378,
+  "\u0120probably": 3379,
+  "erc": 3380,
+  "\u0120bor": 3381,
+  "\u0120inj": 3382,
+  "\u0120Int": 3383,
+  "iance": 3384,
+  "\u0120spirit": 3385,
+  "\u0120Americans": 3386,
+  "\u0120temperature": 3387,
+  "actions": 3388,
+  "\u0120passed": 3389,
+  "\u0120Cambridge": 3390,
+  "rument": 3391,
+  "\u0120table": 3392,
+  "\u0120wat": 3393,
+  "\u0120Association": 3394,
+  "heast": 3395,
+  "\u0120Cyp": 3396,
+  "\u0120Peter": 3397,
+  "\u0120184": 3398,
+  "\u0120humans": 3399,
+  "\u0120poor": 3400,
+  "\u0120Common": 3401,
+  "\u0120Russia": 3402,
+  "\u0120sus": 3403,
+  "\u0120mom": 3404,
+  "iform": 3405,
+  "\u0120Vict": 3406,
+  "\u0120Bec": 3407,
+  "\u0120Muslim": 3408,
+  "\u0120happ": 3409,
+  "\u0120Cap": 3410,
+  "\u0120Mart": 3411,
+  "\u0120true": 3412,
+  "\u0120effective": 3413,
+  "iques": 3414,
+  "\u0120letter": 3415,
+  "\u0120individuals": 3416,
+  "lem": 3417,
+  "\u0120heavy": 3418,
+  "\u0120resour": 3419,
+  "\u0120thous": 3420,
+  "\u012022": 3421,
+  "\u0120decided": 3422,
+  "\u0120protect": 3423,
+  "\u0120Southern": 3424,
+  "\u0120promin": 3425,
+  "\u0120regional": 3426,
+  "\u0120Michael": 3427,
+  "SA": 3428,
+  "ares": 3429,
+  "\u0120Ash": 3430,
+  "\u0120flu": 3431,
+  "\u0120broadcast": 3432,
+  "board": 3433,
+  "12": 3434,
+  "use": 3435,
+  "\u0120Lou": 3436,
+  "\u0120rank": 3437,
+  "astic": 3438,
+  "\u0120values": 3439,
+  "\u0120needed": 3440,
+  "\u0120tend": 3441,
+  "\u0120division": 3442,
+  "\u0120Following": 3443,
+  "vin": 3444,
+  "ont": 3445,
+  "\u0120assum": 3446,
+  "\u0120corre": 3447,
+  "\u0120forest": 3448,
+  "ground": 3449,
+  "riage": 3450,
+  "\u0120Ox": 3451,
+  "\u0120authority": 3452,
+  "\u0120Library": 3453,
+  "uda": 3454,
+  "\u0120Jews": 3455,
+  "itutions": 3456,
+  "\u0120Del": 3457,
+  "ibly": 3458,
+  "tee": 3459,
+  "\u0120pattern": 3460,
+  "aith": 3461,
+  "water": 3462,
+  "\u0120Bron": 3463,
+  "igation": 3464,
+  "\u0120Kn": 3465,
+  "\u0120Despite": 3466,
+  "idents": 3467,
+  "\u0120soldiers": 3468,
+  "\u0120frequently": 3469,
+  "oor": 3470,
+  "tical": 3471,
+  "\u0120inh": 3472,
+  "\u0120Bill": 3473,
+  "\u0120search": 3474,
+  "\u0120becoming": 3475,
+  "\u0120Because": 3476,
+  "\u0120tree": 3477,
+  "ico": 3478,
+  "\u0120Aut": 3479,
+  "verse": 3480,
+  "\u0120composed": 3481,
+  "\u0120medical": 3482,
+  "\u0120basic": 3483,
+  "\u0120Liber": 3484,
+  "\u0120decision": 3485,
+  "\u0120argued": 3486,
+  "eder": 3487,
+  "\u0120Stud": 3488,
+  "ampionship": 3489,
+  "ville": 3490,
+  "igr": 3491,
+  "\u0120step": 3492,
+  "\u0120week": 3493,
+  "\u0120contract": 3494,
+  "\u0120Two": 3495,
+  "\u01201998": 3496,
+  "\u0120behind": 3497,
+  "inated": 3498,
+  "\u0120Both": 3499,
+  "\u0120famous": 3500,
+  "\u0120Institute": 3501,
+  "\u0120expansion": 3502,
+  "\u0120Japan": 3503,
+  "oura": 3504,
+  "irm": 3505,
+  "\u0120daughter": 3506,
+  "oud": 3507,
+  "raction": 3508,
+  "\u0120attacks": 3509,
+  "zil": 3510,
+  "\u0120cry": 3511,
+  "\u0120Aber": 3512,
+  "\u0120playing": 3513,
+  "\u01202022": 3514,
+  "\u0120taking": 3515,
+  "\u0120paper": 3516,
+  "\u0120sexual": 3517,
+  "\u0120lunar": 3518,
+  "asty": 3519,
+  "\u0120spect": 3520,
+  "\u0120letters": 3521,
+  "\u0120Company": 3522,
+  "\u0120ult": 3523,
+  "\u012027": 3524,
+  "\u0120Class": 3525,
+  "\u0120ir": 3526,
+  "itz": 3527,
+  "\u0120Pre": 3528,
+  "undred": 3529,
+  "\u0120...": 3530,
+  "\u0120Ir": 3531,
+  "\u0120accepted": 3532,
+  "\u0120link": 3533,
+  "\u0120Bank": 3534,
+  "\u0120enem": 3535,
+  "\u0120attrib": 3536,
+  "\u0120necessary": 3537,
+  "\u0120risk": 3538,
+  "\u01200": 3539,
+  "\u0120bacter": 3540,
+  "\u0120money": 3541,
+  "\u0120vary": 3542,
+  "onds": 3543,
+  "century": 3544,
+  "\u0120married": 3545,
+  "\u0120Anglic": 3546,
+  "\u0120calcul": 3547,
+  "\u0120featured": 3548,
+  "den": 3549,
+  "rog": 3550,
+  "ressed": 3551,
+  "\u0120u": 3552,
+  "\u0120Rome": 3553,
+  "\u0120yet": 3554,
+  "\u0120Second": 3555,
+  "\u0120avoid": 3556,
+  "\u0120failed": 3557,
+  "ka": 3558,
+  "ras": 3559,
+  "enic": 3560,
+  "\u0120date": 3561,
+  "otes": 3562,
+  "\u0120Moon": 3563,
+  "\u0120deep": 3564,
+  "ested": 3565,
+  "\u012023": 3566,
+  "rown": 3567,
+  "rief": 3568,
+  "izations": 3569,
+  "\u0120appointed": 3570,
+  "\u0120allows": 3571,
+  "\u0120identified": 3572,
+  "\u0120activities": 3573,
+  "eties": 3574,
+  "\u0120programs": 3575,
+  "\u0120cand": 3576,
+  "\u0120ey": 3577,
+  "\u0120Fl": 3578,
+  "\u0120emphas": 3579,
+  "win": 3580,
+  "\u0120native": 3581,
+  "\u0120aspect": 3582,
+  "\u0120inn": 3583,
+  "\u0120burn": 3584,
+  "\u0120Under": 3585,
+  "\u0120Arist": 3586,
+  "\u0120operation": 3587,
+  "\u0120alcoh": 3588,
+  "\u0120our": 3589,
+  "197": 3590,
+  "\u0120System": 3591,
+  "bed": 3592,
+  "rical": 3593,
+  "\u0120That": 3594,
+  "\u012050": 3595,
+  "\u0120synthe": 3596,
+  "\u0120launch": 3597,
+  "\u0120micro": 3598,
+  "vey": 3599,
+  "uman": 3600,
+  "\u0120round": 3601,
+  "\u0120instrument": 3602,
+  "annel": 3603,
+  "ygen": 3604,
+  "\u0120mis": 3605,
+  "\u0120chief": 3606,
+  "\u0120buildings": 3607,
+  "\u0120Canadian": 3608,
+  "\u0120Berlin": 3609,
+  "\u0120responsible": 3610,
+  "ully": 3611,
+  "\u0120Dis": 3612,
+  "\u0120applied": 3613,
+  "\u0120completed": 3614,
+  "\u0120previously": 3615,
+  "unicip": 3616,
+  "\u0120knowledge": 3617,
+  "bour": 3618,
+  "ned": 3619,
+  "\u0120consequ": 3620,
+  "\u0120churches": 3621,
+  "azine": 3622,
+  "comp": 3623,
+  "\u0120\u0120\u0120\u0120": 3624,
+  "iated": 3625,
+  "\u0120discover": 3626,
+  "\u0120combined": 3627,
+  "\u0120takes": 3628,
+  "alle": 3629,
+  "ola": 3630,
+  "\u0120Minister": 3631,
+  "writ": 3632,
+  "wide": 3633,
+  "yond": 3634,
+  "\u0120spl": 3635,
+  "itation": 3636,
+  "\u0120reform": 3637,
+  "\u0120graph": 3638,
+  "\u0120Dan": 3639,
+  "\u0120sec": 3640,
+  "\u0120constant": 3641,
+  "\u0120possess": 3642,
+  ",\"": 3643,
+  "\u0120Tex": 3644,
+  "\u0120alph": 3645,
+  "\u0120section": 3646,
+  "\u0120Street": 3647,
+  "crete": 3648,
+  "\u0120ideas": 3649,
+  "\u0120disease": 3650,
+  "\u0120ut": 3651,
+  "ented": 3652,
+  "\u0120chall": 3653,
+  "\u0120consists": 3654,
+  "\u0120Brazil": 3655,
+  "icted": 3656,
+  "\u0120dr": 3657,
+  "\u0120Beng": 3658,
+  "\u0120distingu": 3659,
+  "\u0120Columbia": 3660,
+  "\u0120brain": 3661,
+  "\u0120compared": 3662,
+  "\u0120techniques": 3663,
+  "ship": 3664,
+  "\u0120collect": 3665,
+  "\u0120mix": 3666,
+  "\u0120literature": 3667,
+  "ien": 3668,
+  "\u0120bomb": 3669,
+  "\u0120dram": 3670,
+  "\u0120decre": 3671,
+  "\u0120Thus": 3672,
+  "\u0120formal": 3673,
+  "\u0120directed": 3674,
+  "\u0120Johnson": 3675,
+  "\u0120initially": 3676,
+  "\u0120derived": 3677,
+  "van": 3678,
+  "itan": 3679,
+  "\u0120retain": 3680,
+  "odox": 3681,
+  "\u0120Wil": 3682,
+  "perial": 3683,
+  "\u0120district": 3684,
+  "\u0120Athens": 3685,
+  "\u0120bra": 3686,
+  "\u0120face": 3687,
+  "\u0120examples": 3688,
+  "\u0120rise": 3689,
+  "\u0120Croat": 3690,
+  "\u0120harm": 3691,
+  "\u0120Italian": 3692,
+  "\u0120simple": 3693,
+  "\u0120aim": 3694,
+  "enth": 3695,
+  "\u0120reject": 3696,
+  "eles": 3697,
+  "\u0120university": 3698,
+  "\u0120conserv": 3699,
+  "\u012040": 3700,
+  "\u0120autom": 3701,
+  "\u0120Prot": 3702,
+  "\u0120philosophy": 3703,
+  "\u0120incorpor": 3704,
+  "wise": 3705,
+  "\u0120birds": 3706,
+  "\u0120beg": 3707,
+  "ocal": 3708,
+  "illed": 3709,
+  "\u01201997": 3710,
+  "edom": 3711,
+  "\u0120Serv": 3712,
+  "\u0120Def": 3713,
+  "\u0120Center": 3714,
+  "\u0120reduced": 3715,
+  "\u0120citizens": 3716,
+  "oids": 3717,
+  "\u0120TV": 3718,
+  "\u0120appearance": 3719,
+  "tles": 3720,
+  "\u0120Alf": 3721,
+  "\u01201996": 3722,
+  "ini": 3723,
+  "\u0120Av": 3724,
+  "\u0120Sever": 3725,
+  "\u0120conne": 3726,
+  "...": 3727,
+  "mark": 3728,
+  "\u0120Sal": 3729,
+  "\u0120Bre": 3730,
+  "\u0120speed": 3731,
+  "\u0120communication": 3732,
+  "\u0120theore": 3733,
+  "\u0120animal": 3734,
+  "\u0120Ke": 3735,
+  "\u0120remaining": 3736,
+  "force": 3737,
+  "\u0120benef": 3738,
+  "\u0120AM": 3739,
+  "\u0120Azer": 3740,
+  "\u0120Hill": 3741,
+  "\u012028": 3742,
+  "\u0120Nor": 3743,
+  "\u0120communities": 3744,
+  "\u0120quickly": 3745,
+  "oln": 3746,
+  "urder": 3747,
+  "\u0120Fil": 3748,
+  "\u0120Fore": 3749,
+  "\u0120Navy": 3750,
+  "\u0120approximately": 3751,
+  "sis": 3752,
+  "\u0120mess": 3753,
+  "\u0120Mat": 3754,
+  "\u0120Jose": 3755,
+  "\u0120speech": 3756,
+  "\u0120friend": 3757,
+  "pr": 3758,
+  "\u0120phen": 3759,
+  "\u0120hours": 3760,
+  "\u0120Leg": 3761,
+  "\u0120continue": 3762,
+  "\u0120definition": 3763,
+  "\u0120screen": 3764,
+  "bai": 3765,
+  "\u0120Mem": 3766,
+  "\u0120secret": 3767,
+  "\u0120device": 3768,
+  "\u0120Mex": 3769,
+  "\u0120arrang": 3770,
+  "\u0120assist": 3771,
+  "\u0120launched": 3772,
+  "\u0120Government": 3773,
+  "baijan": 3774,
+  "zz": 3775,
+  "\u0120internal": 3776,
+  "\u0120Grand": 3777,
+  "\u0120compar": 3778,
+  "\u0120Chap": 3779,
+  "izes": 3780,
+  "cluding": 3781,
+  "rone": 3782,
+  "\u0120models": 3783,
+  "\u0120signed": 3784,
+  "rival": 3785,
+  "\u0120controvers": 3786,
+  "hy": 3787,
+  "\u0120Civil": 3788,
+  "emis": 3789,
+  "\u0120Wales": 3790,
+  "\u0120formation": 3791,
+  "\u0120organization": 3792,
+  "11": 3793,
+  "arc": 3794,
+  "elt": 3795,
+  "\u0120orbit": 3796,
+  "\u0120subs": 3797,
+  "stein": 3798,
+  "25": 3799,
+  "can": 3800,
+  "oned": 3801,
+  "\u0120Bible": 3802,
+  "\u0120alk": 3803,
+  "\u0120rot": 3804,
+  "\u0120purpose": 3805,
+  "\u0120fif": 3806,
+  "\u0120lived": 3807,
+  "\u0120Mary": 3808,
+  "thodox": 3809,
+  "\u0120approach": 3810,
+  "mitted": 3811,
+  "wor": 3812,
+  "\u0120anthrop": 3813,
+  "\u0120repe": 3814,
+  "ela": 3815,
+  "\u0120Ann": 3816,
+  "\u0120driv": 3817,
+  "\u0120Greece": 3818,
+  "\u0120night": 3819,
+  "\u0120sets": 3820,
+  "\u0120White": 3821,
+  "\u0120equal": 3822,
+  "\u0120claims": 3823,
+  "\u0120devices": 3824,
+  "\u0120powers": 3825,
+  "rier": 3826,
+  "\u0120Lord": 3827,
+  "\u0120weight": 3828,
+  "\u0120occurred": 3829,
+  "\u0120Azerbaijan": 3830,
+  "uge": 3831,
+  "\u0120structures": 3832,
+  "aught": 3833,
+  "\u0120bring": 3834,
+  "\u0120middle": 3835,
+  "\u0120Pac": 3836,
+  "\u0120dead": 3837,
+  "ographic": 3838,
+  "\u0120Barb": 3839,
+  "\u0120industrial": 3840,
+  "\u0120observed": 3841,
+  "kin": 3842,
+  "\u0120council": 3843,
+  "\u0120ple": 3844,
+  "\u0120Rec": 3845,
+  "\u0120Corn": 3846,
+  "\u0120Their": 3847,
+  "ado": 3848,
+  "ifically": 3849,
+  "\u0120rat": 3850,
+  "\u0120learn": 3851,
+  "orders": 3852,
+  "\u0120road": 3853,
+  "\u0120allowing": 3854,
+  "\u0120favor": 3855,
+  "\u0120Atlantic": 3856,
+  "onn": 3857,
+  "\u0120pil": 3858,
+  "\u0120places": 3859,
+  "\u0120simply": 3860,
+  "\u0120Spain": 3861,
+  "incoln": 3862,
+  "\u0120Prince": 3863,
+  "\u0120fully": 3864,
+  "\u0120Bor": 3865,
+  "\u0120global": 3866,
+  "\u0120offered": 3867,
+  "iring": 3868,
+  "\u0120direction": 3869,
+  "\u0120summer": 3870,
+  "\u0120reve": 3871,
+  "\u0120Best": 3872,
+  "unk": 3873,
+  "ube": 3874,
+  "\u0120ec": 3875,
+  "\u0120suc": 3876,
+  "obile": 3877,
+  "\u0120refers": 3878,
+  "\u0120declared": 3879,
+  "\u0120Victor": 3880,
+  "15": 3881,
+  "\u0120bal": 3882,
+  "\u0120Bos": 3883,
+  "\u01201994": 3884,
+  "\u0120female": 3885,
+  "\u0120ways": 3886,
+  "\u0120recon": 3887,
+  "\u0120Lincoln": 3888,
+  "\u0120depend": 3889,
+  "\u0120classical": 3890,
+  "\u0120presented": 3891,
+  "aska": 3892,
+  "ilies": 3893,
+  "\u0120climate": 3894,
+  "\u0120extended": 3895,
+  "\u0120entered": 3896,
+  "\u0120interpret": 3897,
+  "ya": 3898,
+  "\u0120Sol": 3899,
+  "ipment": 3900,
+  "\u0120shot": 3901,
+  "\u0120impl": 3902,
+  "\u0120records": 3903,
+  "\u0120turned": 3904,
+  "enh": 3905,
+  "cell": 3906,
+  "elled": 3907,
+  "earch": 3908,
+  "\u0120despite": 3909,
+  "\u0120ancest": 3910,
+  "\u0120songs": 3911,
+  "\u0120spoken": 3912,
+  "\u0120resistance": 3913,
+  "\u0120newsp": 3914,
+  "\u0120serious": 3915,
+  "\u0120ordered": 3916,
+  "\u0120Louis": 3917,
+  "ooth": 3918,
+  "\u0120male": 3919,
+  "aded": 3920,
+  "\u0120fort": 3921,
+  "ipl": 3922,
+  "\u0120arri": 3923,
+  "\u0120clos": 3924,
+  "book": 3925,
+  "oured": 3926,
+  "amin": 3927,
+  "\u0120Each": 3928,
+  "ructed": 3929,
+  "\u0120musical": 3930,
+  "\u0120reference": 3931,
+  "\u0120interview": 3932,
+  "\u0120positive": 3933,
+  "\u0120Add": 3934,
+  "\u0120teams": 3935,
+  "\u0120Portug": 3936,
+  "\u0120moment": 3937,
+  "inet": 3938,
+  "atre": 3939,
+  "rod": 3940,
+  "\u0120closed": 3941,
+  "\u0120Ark": 3942,
+  "\u01201995": 3943,
+  "\u0120Shah": 3944,
+  "\u0120Group": 3945,
+  "16": 3946,
+  "with": 3947,
+  "\u0120rein": 3948,
+  "igu": 3949,
+  "\u0120Elect": 3950,
+  "\u0120deal": 3951,
+  "\u0120exhib": 3952,
+  "sec": 3953,
+  "\u0120Ach": 3954,
+  "\u0120hundred": 3955,
+  "aging": 3956,
+  "\u0120prison": 3957,
+  "\u0120victory": 3958,
+  "CA": 3959,
+  "\u0120cru": 3960,
+  "quar": 3961,
+  "\u0120Chic": 3962,
+  "\u0120negative": 3963,
+  "\u0120applications": 3964,
+  "oes": 3965,
+  "\u0120Tra": 3966,
+  "umin": 3967,
+  "\u0120Div": 3968,
+  "\u0120183": 3969,
+  "\u0120ended": 3970,
+  "\u0120nuclear": 3971,
+  "las": 3972,
+  "iments": 3973,
+  "ulate": 3974,
+  "\u0120share": 3975,
+  "\u0120Atari": 3976,
+  "\u0120occurs": 3977,
+  "\u0120condition": 3978,
+  "\u0120Belg": 3979,
+  "\u0120brief": 3980,
+  "\u0120Sun": 3981,
+  "uls": 3982,
+  "estic": 3983,
+  "\u0120effic": 3984,
+  "\u0120opposed": 3985,
+  "\u0120Franc": 3986,
+  "\u0120Brown": 3987,
+  "\u0120distribution": 3988,
+  "\u0120virt": 3989,
+  "\u0120binary": 3990,
+  "\u0120Ah": 3991,
+  "imens": 3992,
+  "\u0120Mill": 3993,
+  "\u0120Jes": 3994,
+  "\u0120serve": 3995,
+  "\u0120sport": 3996,
+  "\u0120pal": 3997,
+  "\u0120Rob": 3998,
+  "\u0120Later": 3999
+}