We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Error when using ./tools/embeddings_to_torch.py
**I am getting the following error.
Is it harmful and anyone know how to solve it?**


[2018-09-24 21:06:09,964 INFO] From: ./glove_experiment/data.vocab.pt
[2018-09-24 21:06:09,964 INFO] 	* source vocab: 50002 words
[2018-09-24 21:06:09,964 INFO] 	* target vocab: 50004 words
[2018-09-24 21:06:42,008 INFO] Got 400000 encryption embeddings from ./glove/original.txt
[2018-09-24 21:08:21,394 INFO] Got 1142358 decryption embeddings from ./glove/wiki.fr.vec
[2018-09-24 21:08:21,699 INFO] 
Matching: 
[2018-09-24 21:08:21,699 INFO] 	* enc: 19625 match, 30377 missing, (39.25%)
[2018-09-24 21:08:21,699 INFO] 	* dec: 1071 match, 48933 missing, (2.14%)
[2018-09-24 21:08:21,699 INFO] 
Filtered embeddings:
--- Logging error ---
Traceback (most recent call last):
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 993, in emit
    msg = self.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 839, in format
    return fmt.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 576, in format
    record.message = record.getMessage()
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "./tools/embeddings_to_torch.py", line 148, in <module>
    main()
  File "./tools/embeddings_to_torch.py", line 134, in main
    logger.info("\t* enc: ", filtered_enc_embeddings.size())
Message: '\t* enc: '
Arguments: (torch.Size([50002, 300]),)
--- Logging error ---
Traceback (most recent call last):
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 993, in emit
    msg = self.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 839, in format
    return fmt.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 576, in format
    record.message = record.getMessage()
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "./tools/embeddings_to_torch.py", line 148, in <module>
    main()
  File "./tools/embeddings_to_torch.py", line 134, in main
    logger.info("\t* enc: ", filtered_enc_embeddings.size())
Message: '\t* enc: '
Arguments: (torch.Size([50002, 300]),)
--- Logging error ---
Traceback (most recent call last):
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 993, in emit
    msg = self.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 839, in format
    return fmt.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 576, in format
    record.message = record.getMessage()
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "./tools/embeddings_to_torch.py", line 148, in <module>
    main()
  File "./tools/embeddings_to_torch.py", line 135, in main
    logger.info("\t* dec: ", filtered_dec_embeddings.size())
Message: '\t* dec: '
Arguments: (torch.Size([50004, 300]),)
--- Logging error ---
Traceback (most recent call last):
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 993, in emit
    msg = self.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 839, in format
    return fmt.format(record)
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 576, in format
    record.message = record.getMessage()
  File "/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py", line 338, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "./tools/embeddings_to_torch.py", line 148, in <module>
    main()
  File "./tools/embeddings_to_torch.py", line 135, in main
    logger.info("\t* dec: ", filtered_dec_embeddings.size())
Message: '\t* dec: '
Arguments: (torch.Size([50004, 300]),)
[2018-09-24 21:08:21,701 INFO] 
Saving embedding as:
	* enc: ./glove_experiment/embeddings.enc.pt
	* dec: ./glove_experiment/embeddings.dec.pt
[2018-09-24 21:08:22,065 INFO] 
Done.


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `tools/embeddings_to_torch.py`
Content:
```
1 #!/usr/bin/env python
2 # -*- coding: utf-8 -*-
3 from __future__ import print_function
4 from __future__ import division
5 import six
6 import sys
7 import numpy as np
8 import argparse
9 import torch
10 from onmt.utils.logging import init_logger, logger
11 
12 
13 def get_vocabs(dict_file):
14     vocabs = torch.load(dict_file)
15 
16     enc_vocab, dec_vocab = None, None
17 
18     # the vocab object is a list of tuple (name, torchtext.Vocab)
19     # we iterate over this list and associate vocabularies based on the name
20     for vocab in vocabs:
21         if vocab[0] == 'src':
22             enc_vocab = vocab[1]
23         if vocab[0] == 'tgt':
24             dec_vocab = vocab[1]
25     assert enc_vocab is not None and dec_vocab is not None
26 
27     logger.info("From: %s" % dict_file)
28     logger.info("\t* source vocab: %d words" % len(enc_vocab))
29     logger.info("\t* target vocab: %d words" % len(dec_vocab))
30 
31     return enc_vocab, dec_vocab
32 
33 
34 def get_embeddings(file_enc, opt, flag):
35     embs = dict()
36     if flag == 'enc':
37         for (i, l) in enumerate(open(file_enc, 'rb')):
38             if i < opt.skip_lines:
39                 continue
40             if not l:
41                 break
42             if len(l) == 0:
43                 continue
44 
45             l_split = l.decode('utf8').strip().split(' ')
46             if len(l_split) == 2:
47                 continue
48             embs[l_split[0]] = [float(em) for em in l_split[1:]]
49         logger.info("Got {} encryption embeddings from {}".format(len(embs),
50                                                                   file_enc))
51     else:
52 
53         for (i, l) in enumerate(open(file_enc, 'rb')):
54             if not l:
55                 break
56             if len(l) == 0:
57                 continue
58 
59             l_split = l.decode('utf8').strip().split(' ')
60             if len(l_split) == 2:
61                 continue
62             embs[l_split[0]] = [float(em) for em in l_split[1:]]
63         logger.info("Got {} decryption embeddings from {}".format(len(embs),
64                                                                   file_enc))
65     return embs
66 
67 
68 def match_embeddings(vocab, emb, opt):
69     dim = len(six.next(six.itervalues(emb)))
70     filtered_embeddings = np.zeros((len(vocab), dim))
71     count = {"match": 0, "miss": 0}
72     for w, w_id in vocab.stoi.items():
73         if w in emb:
74             filtered_embeddings[w_id] = emb[w]
75             count['match'] += 1
76         else:
77             if opt.verbose:
78                 logger.info(u"not found:\t{}".format(w), file=sys.stderr)
79             count['miss'] += 1
80 
81     return torch.Tensor(filtered_embeddings), count
82 
83 
84 TYPES = ["GloVe", "word2vec"]
85 
86 
87 def main():
88 
89     parser = argparse.ArgumentParser(description='embeddings_to_torch.py')
90     parser.add_argument('-emb_file_enc', required=True,
91                         help="source Embeddings from this file")
92     parser.add_argument('-emb_file_dec', required=True,
93                         help="target Embeddings from this file")
94     parser.add_argument('-output_file', required=True,
95                         help="Output file for the prepared data")
96     parser.add_argument('-dict_file', required=True,
97                         help="Dictionary file")
98     parser.add_argument('-verbose', action="store_true", default=False)
99     parser.add_argument('-skip_lines', type=int, default=0,
100                         help="Skip first lines of the embedding file")
101     parser.add_argument('-type', choices=TYPES, default="GloVe")
102     opt = parser.parse_args()
103 
104     enc_vocab, dec_vocab = get_vocabs(opt.dict_file)
105     if opt.type == "word2vec":
106         opt.skip_lines = 1
107 
108     embeddings_enc = get_embeddings(opt.emb_file_enc, opt, flag='enc')
109     embeddings_dec = get_embeddings(opt.emb_file_dec, opt, flag='dec')
110 
111     filtered_enc_embeddings, enc_count = match_embeddings(enc_vocab,
112                                                           embeddings_enc,
113                                                           opt)
114     filtered_dec_embeddings, dec_count = match_embeddings(dec_vocab,
115                                                           embeddings_dec,
116                                                           opt)
117     logger.info("\nMatching: ")
118     match_percent = [_['match'] / (_['match'] + _['miss']) * 100
119                      for _ in [enc_count, dec_count]]
120     logger.info("\t* enc: %d match, %d missing, (%.2f%%)"
121                 % (enc_count['match'],
122                    enc_count['miss'],
123                    match_percent[0]))
124     logger.info("\t* dec: %d match, %d missing, (%.2f%%)"
125                 % (dec_count['match'],
126                    dec_count['miss'],
127                    match_percent[1]))
128 
129     logger.info("\nFiltered embeddings:")
130     logger.info("\t* enc: ", filtered_enc_embeddings.size())
131     logger.info("\t* dec: ", filtered_dec_embeddings.size())
132 
133     enc_output_file = opt.output_file + ".enc.pt"
134     dec_output_file = opt.output_file + ".dec.pt"
135     logger.info("\nSaving embedding as:\n\t* enc: %s\n\t* dec: %s"
136                 % (enc_output_file, dec_output_file))
137     torch.save(filtered_enc_embeddings, enc_output_file)
138     torch.save(filtered_dec_embeddings, dec_output_file)
139     logger.info("\nDone.")
140 
141 
142 if __name__ == "__main__":
143     init_logger('embeddings_to_torch.log')
144     main()
145
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/tools/embeddings_to_torch.py b/tools/embeddings_to_torch.py
--- a/tools/embeddings_to_torch.py
+++ b/tools/embeddings_to_torch.py
@@ -127,8 +127,8 @@
                    match_percent[1]))
 
     logger.info("\nFiltered embeddings:")
-    logger.info("\t* enc: ", filtered_enc_embeddings.size())
-    logger.info("\t* dec: ", filtered_dec_embeddings.size())
+    logger.info("\t* enc: %s" % str(filtered_enc_embeddings.size()))
+    logger.info("\t* dec: %s" % str(filtered_dec_embeddings.size()))
 
     enc_output_file = opt.output_file + ".enc.pt"
     dec_output_file = opt.output_file + ".dec.pt"

{"golden_diff": "diff --git a/tools/embeddings_to_torch.py b/tools/embeddings_to_torch.py\n--- a/tools/embeddings_to_torch.py\n+++ b/tools/embeddings_to_torch.py\n@@ -127,8 +127,8 @@\n                    match_percent[1]))\n \n     logger.info(\"\\nFiltered embeddings:\")\n-    logger.info(\"\\t* enc: \", filtered_enc_embeddings.size())\n-    logger.info(\"\\t* dec: \", filtered_dec_embeddings.size())\n+    logger.info(\"\\t* enc: %s\" % str(filtered_enc_embeddings.size()))\n+    logger.info(\"\\t* dec: %s\" % str(filtered_dec_embeddings.size()))\n \n     enc_output_file = opt.output_file + \".enc.pt\"\n     dec_output_file = opt.output_file + \".dec.pt\"\n", "issue": "Error when using ./tools/embeddings_to_torch.py\n**I am getting the following error.\r\nIs it harmful and anyone know how to solve it?**\r\n\r\n\r\n[2018-09-24 21:06:09,964 INFO] From: ./glove_experiment/data.vocab.pt\r\n[2018-09-24 21:06:09,964 INFO] \t* source vocab: 50002 words\r\n[2018-09-24 21:06:09,964 INFO] \t* target vocab: 50004 words\r\n[2018-09-24 21:06:42,008 INFO] Got 400000 encryption embeddings from ./glove/original.txt\r\n[2018-09-24 21:08:21,394 INFO] Got 1142358 decryption embeddings from ./glove/wiki.fr.vec\r\n[2018-09-24 21:08:21,699 INFO] \r\nMatching: \r\n[2018-09-24 21:08:21,699 INFO] \t* enc: 19625 match, 30377 missing, (39.25%)\r\n[2018-09-24 21:08:21,699 INFO] \t* dec: 1071 match, 48933 missing, (2.14%)\r\n[2018-09-24 21:08:21,699 INFO] \r\nFiltered embeddings:\r\n--- Logging error ---\r\nTraceback (most recent call last):\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 993, in emit\r\n    msg = self.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 839, in format\r\n    return fmt.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 576, in format\r\n    record.message = record.getMessage()\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 338, in getMessage\r\n    msg = msg % self.args\r\nTypeError: not all arguments converted during string formatting\r\nCall stack:\r\n  File \"./tools/embeddings_to_torch.py\", line 148, in <module>\r\n    main()\r\n  File \"./tools/embeddings_to_torch.py\", line 134, in main\r\n    logger.info(\"\\t* enc: \", filtered_enc_embeddings.size())\r\nMessage: '\\t* enc: '\r\nArguments: (torch.Size([50002, 300]),)\r\n--- Logging error ---\r\nTraceback (most recent call last):\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 993, in emit\r\n    msg = self.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 839, in format\r\n    return fmt.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 576, in format\r\n    record.message = record.getMessage()\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 338, in getMessage\r\n    msg = msg % self.args\r\nTypeError: not all arguments converted during string formatting\r\nCall stack:\r\n  File \"./tools/embeddings_to_torch.py\", line 148, in <module>\r\n    main()\r\n  File \"./tools/embeddings_to_torch.py\", line 134, in main\r\n    logger.info(\"\\t* enc: \", filtered_enc_embeddings.size())\r\nMessage: '\\t* enc: '\r\nArguments: (torch.Size([50002, 300]),)\r\n--- Logging error ---\r\nTraceback (most recent call last):\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 993, in emit\r\n    msg = self.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 839, in format\r\n    return fmt.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 576, in format\r\n    record.message = record.getMessage()\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 338, in getMessage\r\n    msg = msg % self.args\r\nTypeError: not all arguments converted during string formatting\r\nCall stack:\r\n  File \"./tools/embeddings_to_torch.py\", line 148, in <module>\r\n    main()\r\n  File \"./tools/embeddings_to_torch.py\", line 135, in main\r\n    logger.info(\"\\t* dec: \", filtered_dec_embeddings.size())\r\nMessage: '\\t* dec: '\r\nArguments: (torch.Size([50004, 300]),)\r\n--- Logging error ---\r\nTraceback (most recent call last):\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 993, in emit\r\n    msg = self.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 839, in format\r\n    return fmt.format(record)\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 576, in format\r\n    record.message = record.getMessage()\r\n  File \"/home/eeb439/anaconda3/envs/pytorch0.4text0.3/lib/python3.6/logging/__init__.py\", line 338, in getMessage\r\n    msg = msg % self.args\r\nTypeError: not all arguments converted during string formatting\r\nCall stack:\r\n  File \"./tools/embeddings_to_torch.py\", line 148, in <module>\r\n    main()\r\n  File \"./tools/embeddings_to_torch.py\", line 135, in main\r\n    logger.info(\"\\t* dec: \", filtered_dec_embeddings.size())\r\nMessage: '\\t* dec: '\r\nArguments: (torch.Size([50004, 300]),)\r\n[2018-09-24 21:08:21,701 INFO] \r\nSaving embedding as:\r\n\t* enc: ./glove_experiment/embeddings.enc.pt\r\n\t* dec: ./glove_experiment/embeddings.dec.pt\r\n[2018-09-24 21:08:22,065 INFO] \r\nDone.\r\n\n", "before_files": [{"content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\nfrom __future__ import print_function\nfrom __future__ import division\nimport six\nimport sys\nimport numpy as np\nimport argparse\nimport torch\nfrom onmt.utils.logging import init_logger, logger\n\n\ndef get_vocabs(dict_file):\n    vocabs = torch.load(dict_file)\n\n    enc_vocab, dec_vocab = None, None\n\n    # the vocab object is a list of tuple (name, torchtext.Vocab)\n    # we iterate over this list and associate vocabularies based on the name\n    for vocab in vocabs:\n        if vocab[0] == 'src':\n            enc_vocab = vocab[1]\n        if vocab[0] == 'tgt':\n            dec_vocab = vocab[1]\n    assert enc_vocab is not None and dec_vocab is not None\n\n    logger.info(\"From: %s\" % dict_file)\n    logger.info(\"\\t* source vocab: %d words\" % len(enc_vocab))\n    logger.info(\"\\t* target vocab: %d words\" % len(dec_vocab))\n\n    return enc_vocab, dec_vocab\n\n\ndef get_embeddings(file_enc, opt, flag):\n    embs = dict()\n    if flag == 'enc':\n        for (i, l) in enumerate(open(file_enc, 'rb')):\n            if i < opt.skip_lines:\n                continue\n            if not l:\n                break\n            if len(l) == 0:\n                continue\n\n            l_split = l.decode('utf8').strip().split(' ')\n            if len(l_split) == 2:\n                continue\n            embs[l_split[0]] = [float(em) for em in l_split[1:]]\n        logger.info(\"Got {} encryption embeddings from {}\".format(len(embs),\n                                                                  file_enc))\n    else:\n\n        for (i, l) in enumerate(open(file_enc, 'rb')):\n            if not l:\n                break\n            if len(l) == 0:\n                continue\n\n            l_split = l.decode('utf8').strip().split(' ')\n            if len(l_split) == 2:\n                continue\n            embs[l_split[0]] = [float(em) for em in l_split[1:]]\n        logger.info(\"Got {} decryption embeddings from {}\".format(len(embs),\n                                                                  file_enc))\n    return embs\n\n\ndef match_embeddings(vocab, emb, opt):\n    dim = len(six.next(six.itervalues(emb)))\n    filtered_embeddings = np.zeros((len(vocab), dim))\n    count = {\"match\": 0, \"miss\": 0}\n    for w, w_id in vocab.stoi.items():\n        if w in emb:\n            filtered_embeddings[w_id] = emb[w]\n            count['match'] += 1\n        else:\n            if opt.verbose:\n                logger.info(u\"not found:\\t{}\".format(w), file=sys.stderr)\n            count['miss'] += 1\n\n    return torch.Tensor(filtered_embeddings), count\n\n\nTYPES = [\"GloVe\", \"word2vec\"]\n\n\ndef main():\n\n    parser = argparse.ArgumentParser(description='embeddings_to_torch.py')\n    parser.add_argument('-emb_file_enc', required=True,\n                        help=\"source Embeddings from this file\")\n    parser.add_argument('-emb_file_dec', required=True,\n                        help=\"target Embeddings from this file\")\n    parser.add_argument('-output_file', required=True,\n                        help=\"Output file for the prepared data\")\n    parser.add_argument('-dict_file', required=True,\n                        help=\"Dictionary file\")\n    parser.add_argument('-verbose', action=\"store_true\", default=False)\n    parser.add_argument('-skip_lines', type=int, default=0,\n                        help=\"Skip first lines of the embedding file\")\n    parser.add_argument('-type', choices=TYPES, default=\"GloVe\")\n    opt = parser.parse_args()\n\n    enc_vocab, dec_vocab = get_vocabs(opt.dict_file)\n    if opt.type == \"word2vec\":\n        opt.skip_lines = 1\n\n    embeddings_enc = get_embeddings(opt.emb_file_enc, opt, flag='enc')\n    embeddings_dec = get_embeddings(opt.emb_file_dec, opt, flag='dec')\n\n    filtered_enc_embeddings, enc_count = match_embeddings(enc_vocab,\n                                                          embeddings_enc,\n                                                          opt)\n    filtered_dec_embeddings, dec_count = match_embeddings(dec_vocab,\n                                                          embeddings_dec,\n                                                          opt)\n    logger.info(\"\\nMatching: \")\n    match_percent = [_['match'] / (_['match'] + _['miss']) * 100\n                     for _ in [enc_count, dec_count]]\n    logger.info(\"\\t* enc: %d match, %d missing, (%.2f%%)\"\n                % (enc_count['match'],\n                   enc_count['miss'],\n                   match_percent[0]))\n    logger.info(\"\\t* dec: %d match, %d missing, (%.2f%%)\"\n                % (dec_count['match'],\n                   dec_count['miss'],\n                   match_percent[1]))\n\n    logger.info(\"\\nFiltered embeddings:\")\n    logger.info(\"\\t* enc: \", filtered_enc_embeddings.size())\n    logger.info(\"\\t* dec: \", filtered_dec_embeddings.size())\n\n    enc_output_file = opt.output_file + \".enc.pt\"\n    dec_output_file = opt.output_file + \".dec.pt\"\n    logger.info(\"\\nSaving embedding as:\\n\\t* enc: %s\\n\\t* dec: %s\"\n                % (enc_output_file, dec_output_file))\n    torch.save(filtered_enc_embeddings, enc_output_file)\n    torch.save(filtered_dec_embeddings, dec_output_file)\n    logger.info(\"\\nDone.\")\n\n\nif __name__ == \"__main__\":\n    init_logger('embeddings_to_torch.log')\n    main()\n", "path": "tools/embeddings_to_torch.py"}], "after_files": [{"content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\nfrom __future__ import print_function\nfrom __future__ import division\nimport six\nimport sys\nimport numpy as np\nimport argparse\nimport torch\nfrom onmt.utils.logging import init_logger, logger\n\n\ndef get_vocabs(dict_file):\n    vocabs = torch.load(dict_file)\n\n    enc_vocab, dec_vocab = None, None\n\n    # the vocab object is a list of tuple (name, torchtext.Vocab)\n    # we iterate over this list and associate vocabularies based on the name\n    for vocab in vocabs:\n        if vocab[0] == 'src':\n            enc_vocab = vocab[1]\n        if vocab[0] == 'tgt':\n            dec_vocab = vocab[1]\n    assert enc_vocab is not None and dec_vocab is not None\n\n    logger.info(\"From: %s\" % dict_file)\n    logger.info(\"\\t* source vocab: %d words\" % len(enc_vocab))\n    logger.info(\"\\t* target vocab: %d words\" % len(dec_vocab))\n\n    return enc_vocab, dec_vocab\n\n\ndef get_embeddings(file_enc, opt, flag):\n    embs = dict()\n    if flag == 'enc':\n        for (i, l) in enumerate(open(file_enc, 'rb')):\n            if i < opt.skip_lines:\n                continue\n            if not l:\n                break\n            if len(l) == 0:\n                continue\n\n            l_split = l.decode('utf8').strip().split(' ')\n            if len(l_split) == 2:\n                continue\n            embs[l_split[0]] = [float(em) for em in l_split[1:]]\n        logger.info(\"Got {} encryption embeddings from {}\".format(len(embs),\n                                                                  file_enc))\n    else:\n\n        for (i, l) in enumerate(open(file_enc, 'rb')):\n            if not l:\n                break\n            if len(l) == 0:\n                continue\n\n            l_split = l.decode('utf8').strip().split(' ')\n            if len(l_split) == 2:\n                continue\n            embs[l_split[0]] = [float(em) for em in l_split[1:]]\n        logger.info(\"Got {} decryption embeddings from {}\".format(len(embs),\n                                                                  file_enc))\n    return embs\n\n\ndef match_embeddings(vocab, emb, opt):\n    dim = len(six.next(six.itervalues(emb)))\n    filtered_embeddings = np.zeros((len(vocab), dim))\n    count = {\"match\": 0, \"miss\": 0}\n    for w, w_id in vocab.stoi.items():\n        if w in emb:\n            filtered_embeddings[w_id] = emb[w]\n            count['match'] += 1\n        else:\n            if opt.verbose:\n                logger.info(u\"not found:\\t{}\".format(w), file=sys.stderr)\n            count['miss'] += 1\n\n    return torch.Tensor(filtered_embeddings), count\n\n\nTYPES = [\"GloVe\", \"word2vec\"]\n\n\ndef main():\n\n    parser = argparse.ArgumentParser(description='embeddings_to_torch.py')\n    parser.add_argument('-emb_file_enc', required=True,\n                        help=\"source Embeddings from this file\")\n    parser.add_argument('-emb_file_dec', required=True,\n                        help=\"target Embeddings from this file\")\n    parser.add_argument('-output_file', required=True,\n                        help=\"Output file for the prepared data\")\n    parser.add_argument('-dict_file', required=True,\n                        help=\"Dictionary file\")\n    parser.add_argument('-verbose', action=\"store_true\", default=False)\n    parser.add_argument('-skip_lines', type=int, default=0,\n                        help=\"Skip first lines of the embedding file\")\n    parser.add_argument('-type', choices=TYPES, default=\"GloVe\")\n    opt = parser.parse_args()\n\n    enc_vocab, dec_vocab = get_vocabs(opt.dict_file)\n    if opt.type == \"word2vec\":\n        opt.skip_lines = 1\n\n    embeddings_enc = get_embeddings(opt.emb_file_enc, opt, flag='enc')\n    embeddings_dec = get_embeddings(opt.emb_file_dec, opt, flag='dec')\n\n    filtered_enc_embeddings, enc_count = match_embeddings(enc_vocab,\n                                                          embeddings_enc,\n                                                          opt)\n    filtered_dec_embeddings, dec_count = match_embeddings(dec_vocab,\n                                                          embeddings_dec,\n                                                          opt)\n    logger.info(\"\\nMatching: \")\n    match_percent = [_['match'] / (_['match'] + _['miss']) * 100\n                     for _ in [enc_count, dec_count]]\n    logger.info(\"\\t* enc: %d match, %d missing, (%.2f%%)\"\n                % (enc_count['match'],\n                   enc_count['miss'],\n                   match_percent[0]))\n    logger.info(\"\\t* dec: %d match, %d missing, (%.2f%%)\"\n                % (dec_count['match'],\n                   dec_count['miss'],\n                   match_percent[1]))\n\n    logger.info(\"\\nFiltered embeddings:\")\n    logger.info(\"\\t* enc: %s\" % str(filtered_enc_embeddings.size()))\n    logger.info(\"\\t* dec: %s\" % str(filtered_dec_embeddings.size()))\n\n    enc_output_file = opt.output_file + \".enc.pt\"\n    dec_output_file = opt.output_file + \".dec.pt\"\n    logger.info(\"\\nSaving embedding as:\\n\\t* enc: %s\\n\\t* dec: %s\"\n                % (enc_output_file, dec_output_file))\n    torch.save(filtered_enc_embeddings, enc_output_file)\n    torch.save(filtered_dec_embeddings, dec_output_file)\n    logger.info(\"\\nDone.\")\n\n\nif __name__ == \"__main__\":\n    init_logger('embeddings_to_torch.log')\n    main()\n", "path": "tools/embeddings_to_torch.py"}]}