We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Training on GPU failed with Torchtext when using include_lengths=True in torchtext.data.Field
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->
The issues raises in pytorch_lightning/utilities/apply_func.py which assumes that the attributes of a Batch from trochtext are Tensors, however if torchtext.data.Field is configured to include a length Tensor (include_lengths=True) the field is a tuple.

A bugfix is prepared and a PR can be submitted soon.

### To Reproduce

Steps to reproduce the behavior:

1. Use Torchtext Field with include_lengths=True on a GPU machine and fit model.
2. Training works on CPU but fails on GPU with: `TypeError: cannot unpack non-iterable NoneType object`

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

### Full Error Message
```
Traceback (most recent call last):
 File "debug_torchtext.py", line 105, in <module>
  trainer.fit(model)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1003, in fit
  results = self.single_gpu_train(model)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 186, in single_gpu_train
  results = self.run_pretrain_routine(model)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1213, in run_pretrain_routine
  self.train()
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 370, in train
  self.run_training_epoch()
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 452, in run_training_epoch
  batch_output = self.run_training_batch(batch, batch_idx)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 632, in run_training_batch
  self.hiddens
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 776, in optimizer_closure
  hiddens)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 944, in training_forward
  batch = self.transfer_batch_to_gpu(batch, gpu_id)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 159, in transfer_batch_to_gpu
  return self.__transfer_batch_to_device(batch, device)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 164, in __transfer_batch_to_device
  return model.transfer_batch_to_device(batch, device)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/core/hooks.py", line 242, in transfer_batch_to_device
  return move_data_to_device(batch, device)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/utilities/apply_func.py", line 128, in move_data_to_device
  return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/utilities/apply_func.py", line 35, in apply_to_collection
  return function(data, *args, **kwargs)
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/utilities/apply_func.py", line 103, in batch_to
  device_field = getattr(data, field).to(device, non_blocking=True)
AttributeError: 'tuple' object has no attribute 'to'
Exception ignored in: <function tqdm.__del__ at 0x7fcb5e0b2680>
Traceback (most recent call last):
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py", line 1086, in __del__
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py", line 1293, in close
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py", line 1471, in display
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py", line 1089, in __repr__
 File "/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py", line 1433, in format_dict
TypeError: cannot unpack non-iterable NoneType object
```

#### Code sample
<!-- Ideally attach a minimal code sample to reproduce the decried issue. 
Minimal means having the shortest code but still preserving the bug. -->
```
import torch
from torch import nn, Tensor
import pytorch_lightning as pl
from pytorch_lightning import Trainer, seed_everything
from torchtext import data
seed_everything(1234)
def get_debug_data_loader():
    text_field = data.Field(sequential=True, pad_first=False,
                            init_token="<s>", eos_token="</s>", include_lengths=True)
    example1 = data.example.Example.fromdict({"text": "a b c a c"}, {"text": ("text", text_field)})
    example2 = data.example.Example.fromdict({"text": "b c a a"}, {"text": ("text", text_field)})
    example3 = data.example.Example.fromdict({"text": "c b a"}, {"text": ("text", text_field)})
    dataset = data.Dataset([example1, example2, example3], {"text": text_field})
    text_field.build_vocab(dataset)
    iterator = data.Iterator(dataset, batch_size=3,
                             sort_key=None, device=None, batch_size_fn=None,
                             train=True, repeat=False, shuffle=None, sort=None, sort_within_batch=None)
    return iterator, text_field
class DebugModel(pl.LightningModule):
    def __init__(self):
        super(DebugModel, self).__init__()
        # setup data loader
        self.debug_data_loader, self.text_field = get_debug_data_loader()
        self.learning_rate = 0.001
        self.hid_dim = 4
        pad_idx = self.text_field.vocab.stoi['<pad>']
        self.criterion = nn.CrossEntropyLoss(ignore_index=pad_idx)
        self.INPUT_DIM = len(self.text_field.vocab)
        self.ENC_EMB_DIM = 4  # keep it small for debugging
        self.embedding = nn.Embedding(self.INPUT_DIM, self.ENC_EMB_DIM)
        self.rnn = nn.GRU(self.ENC_EMB_DIM, self.hid_dim, 1, bidirectional=False)
        self.out = nn.Linear(self.hid_dim, self.embedding.num_embeddings)
        self.OUTPUT_DIM = len(self.text_field.vocab)
    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=self.learning_rate)
    def forward(self, input_seq, length):
        embedded: Tensor = self.embedding(input_seq)
        packed_embedded: Tensor = torch.nn.utils.rnn.pack_padded_sequence(embedded, length, batch_first=False,
                                                                          enforce_sorted=False)
        packed_outputs, hidden = self.rnn(packed_embedded)  # [sent len, batch size, emb dim]
        outputs, length = torch.nn.utils.rnn.pad_packed_sequence(packed_outputs)
        # outputs -> [sent len, batch size, hid dim * n directions]
        # hidden -> [n layers * n directions, batch size, hid dim]
        output = outputs.squeeze(0)
        prediction = self.out(output)
        return prediction
    @staticmethod
    def _parse_batch(batch):
        source = batch.text[0]
        source_length = batch.text[1]
        return source, source_length
    def training_step(self, batch, batch_nb):
        x = self._parse_batch(batch)
        target, target_length = x
        output = self.forward(target, target_length)
        loss = self.criterion(output[:-1].view(-1, output.shape[2]), target[1:].view(-1))
        prefix = 'train'
        tensorboard_logs = {f'{prefix}_loss': loss.item()}
        result = {'loss': loss, 'log': tensorboard_logs}
        return result
    def train_dataloader(self):
        return self.debug_data_loader
model = DebugModel()
cuda_device_cnt = torch.cuda.device_count()
if cuda_device_cnt > 0:
    use_num_cuda_devices = 1
else:
    use_num_cuda_devices = None
trainer = Trainer(fast_dev_run=False, max_steps=None,
                  gradient_clip_val=10,
                  weights_summary='full', gpus=use_num_cuda_devices,
                  show_progress_bar=True)
trainer.fit(model)
```

### Expected behavior

<!-- A clear and concise description of what you expected to happen. -->
Should not raise an error :-) 

### Environment

```
 CUDA:
    - GPU:
        - TITAN X (Pascal)
    - available:     True
    - version:      10.2
* Packages:
    - numpy:       1.17.3
    - pyTorch_debug:   False
    - pyTorch_version:  1.5.1
    - pytorch-lightning: 0.8.5
    - tensorboard:    2.2.2
    - tqdm:       4.47.0
* System:
    - OS:        Linux
    - architecture:
        - 64bit
        - 
    - processor:     x86_64
    - python:      3.7.4
    - version:      #1 SMP Tue Mar 17 23:49:17 UTC 2020
```
### Additional context

<!-- Add any other context about the problem here. -->


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `pytorch_lightning/utilities/apply_func.py`
Content:
```
1 from abc import ABC
2 from collections.abc import Mapping, Sequence
3 from copy import copy
4 from typing import Any, Callable, Union
5 
6 import torch
7 
8 import importlib
9 TORCHTEXT_AVAILABLE = importlib.util.find_spec("torchtext") is not None
10 if TORCHTEXT_AVAILABLE:
11     from torchtext.data import Batch
12 else:
13     Batch = type(None)
14 
15 
16 def apply_to_collection(data: Any, dtype: Union[type, tuple], function: Callable, *args, **kwargs) -> Any:
17     """
18     Recursively applies a function to all elements of a certain dtype.
19 
20     Args:
21         data: the collection to apply the function to
22         dtype: the given function will be applied to all elements of this dtype
23         function: the function to apply
24         *args: positional arguments (will be forwarded to calls of ``function``)
25         **kwargs: keyword arguments (will be forwarded to calls of ``function``)
26 
27     Returns:
28         the resulting collection
29 
30     """
31     elem_type = type(data)
32 
33     # Breaking condition
34     if isinstance(data, dtype):
35         return function(data, *args, **kwargs)
36 
37     # Recursively apply to collection items
38     elif isinstance(data, Mapping):
39         return elem_type({k: apply_to_collection(v, dtype, function, *args, **kwargs)
40                           for k, v in data.items()})
41     elif isinstance(data, tuple) and hasattr(data, '_fields'):  # named tuple
42         return elem_type(*(apply_to_collection(d, dtype, function, *args, **kwargs) for d in data))
43     elif isinstance(data, Sequence) and not isinstance(data, str):
44         return elem_type([apply_to_collection(d, dtype, function, *args, **kwargs) for d in data])
45 
46     # data is neither of dtype, nor a collection
47     return data
48 
49 
50 class TransferableDataType(ABC):
51     """
52     A custom type for data that can be moved to a torch device via `.to(...)`.
53 
54     Example:
55 
56         >>> isinstance(dict, TransferableDataType)
57         False
58         >>> isinstance(torch.rand(2, 3), TransferableDataType)
59         True
60         >>> class CustomObject:
61         ...     def __init__(self):
62         ...         self.x = torch.rand(2, 2)
63         ...     def to(self, device):
64         ...         self.x = self.x.to(device)
65         ...         return self
66         >>> isinstance(CustomObject(), TransferableDataType)
67         True
68     """
69 
70     @classmethod
71     def __subclasshook__(cls, subclass):
72         if cls is TransferableDataType:
73             to = getattr(subclass, "to", None)
74             return callable(to)
75         return NotImplemented
76 
77 
78 def move_data_to_device(batch: Any, device: torch.device):
79     """
80     Transfers a collection of data to the given device. Any object that defines a method
81     ``to(device)`` will be moved and all other objects in the collection will be left untouched.
82 
83     Args:
84         batch: A tensor or collection of tensors or anything that has a method `.to(...)`.
85             See :func:`apply_to_collection` for a list of supported collection types.
86         device: The device to which the data should be moved
87 
88     Return:
89         the same collection but with all contained tensors residing on the new device.
90 
91     See Also:
92         - :meth:`torch.Tensor.to`
93         - :class:`torch.device`
94     """
95     def batch_to(data):
96         # try to move torchtext data first
97         if TORCHTEXT_AVAILABLE and isinstance(data, Batch):
98 
99             # Shallow copy because each Batch has a reference to Dataset which contains all examples
100             device_data = copy(data)
101             for field in data.fields:
102                 # Batch contains output of Field.process(...) which is tensor hence .to(...) exists
103                 device_field = getattr(data, field).to(device, non_blocking=True)
104                 setattr(device_data, field, device_field)
105             return device_data
106         else:
107             return data.to(device, non_blocking=True)
108 
109     return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)
110
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/pytorch_lightning/utilities/apply_func.py b/pytorch_lightning/utilities/apply_func.py
--- a/pytorch_lightning/utilities/apply_func.py
+++ b/pytorch_lightning/utilities/apply_func.py
@@ -6,6 +6,7 @@
 import torch
 
 import importlib
+
 TORCHTEXT_AVAILABLE = importlib.util.find_spec("torchtext") is not None
 if TORCHTEXT_AVAILABLE:
     from torchtext.data import Batch
@@ -92,6 +93,7 @@
         - :meth:`torch.Tensor.to`
         - :class:`torch.device`
     """
+
     def batch_to(data):
         # try to move torchtext data first
         if TORCHTEXT_AVAILABLE and isinstance(data, Batch):
@@ -99,11 +101,10 @@
             # Shallow copy because each Batch has a reference to Dataset which contains all examples
             device_data = copy(data)
             for field in data.fields:
-                # Batch contains output of Field.process(...) which is tensor hence .to(...) exists
-                device_field = getattr(data, field).to(device, non_blocking=True)
+                device_field = move_data_to_device(getattr(data, field), device)
                 setattr(device_data, field, device_field)
             return device_data
-        else:
-            return data.to(device, non_blocking=True)
+
+        return data.to(device, non_blocking=True)
 
     return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)

{"golden_diff": "diff --git a/pytorch_lightning/utilities/apply_func.py b/pytorch_lightning/utilities/apply_func.py\n--- a/pytorch_lightning/utilities/apply_func.py\n+++ b/pytorch_lightning/utilities/apply_func.py\n@@ -6,6 +6,7 @@\n import torch\n \n import importlib\n+\n TORCHTEXT_AVAILABLE = importlib.util.find_spec(\"torchtext\") is not None\n if TORCHTEXT_AVAILABLE:\n     from torchtext.data import Batch\n@@ -92,6 +93,7 @@\n         - :meth:`torch.Tensor.to`\n         - :class:`torch.device`\n     \"\"\"\n+\n     def batch_to(data):\n         # try to move torchtext data first\n         if TORCHTEXT_AVAILABLE and isinstance(data, Batch):\n@@ -99,11 +101,10 @@\n             # Shallow copy because each Batch has a reference to Dataset which contains all examples\n             device_data = copy(data)\n             for field in data.fields:\n-                # Batch contains output of Field.process(...) which is tensor hence .to(...) exists\n-                device_field = getattr(data, field).to(device, non_blocking=True)\n+                device_field = move_data_to_device(getattr(data, field), device)\n                 setattr(device_data, field, device_field)\n             return device_data\n-        else:\n-            return data.to(device, non_blocking=True)\n+\n+        return data.to(device, non_blocking=True)\n \n     return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)\n", "issue": "Training on GPU failed with Torchtext when using include_lengths=True in torchtext.data.Field\n<!-- \r\n### Common bugs:\r\n1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    \r\n2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    \r\n-->\r\n\r\n## \ud83d\udc1b Bug\r\n\r\n<!-- A clear and concise description of what the bug is. -->\r\nThe issues raises in pytorch_lightning/utilities/apply_func.py which assumes that the attributes of a Batch from trochtext are Tensors, however if torchtext.data.Field is configured to include a length Tensor (include_lengths=True) the field is a tuple.\r\n\r\nA bugfix is prepared and a PR can be submitted soon.\r\n\r\n### To Reproduce\r\n\r\nSteps to reproduce the behavior:\r\n\r\n1. Use Torchtext Field with include_lengths=True on a GPU machine and fit model.\r\n2. Training works on CPU but fails on GPU with: `TypeError: cannot unpack non-iterable NoneType object`\r\n\r\n<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->\r\n\r\n### Full Error Message\r\n```\r\nTraceback (most recent call last):\r\n File \"debug_torchtext.py\", line 105, in <module>\r\n  trainer.fit(model)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py\", line 1003, in fit\r\n  results = self.single_gpu_train(model)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py\", line 186, in single_gpu_train\r\n  results = self.run_pretrain_routine(model)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py\", line 1213, in run_pretrain_routine\r\n  self.train()\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py\", line 370, in train\r\n  self.run_training_epoch()\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py\", line 452, in run_training_epoch\r\n  batch_output = self.run_training_batch(batch, batch_idx)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py\", line 632, in run_training_batch\r\n  self.hiddens\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py\", line 776, in optimizer_closure\r\n  hiddens)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py\", line 944, in training_forward\r\n  batch = self.transfer_batch_to_gpu(batch, gpu_id)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py\", line 159, in transfer_batch_to_gpu\r\n  return self.__transfer_batch_to_device(batch, device)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py\", line 164, in __transfer_batch_to_device\r\n  return model.transfer_batch_to_device(batch, device)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/core/hooks.py\", line 242, in transfer_batch_to_device\r\n  return move_data_to_device(batch, device)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/utilities/apply_func.py\", line 128, in move_data_to_device\r\n  return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/utilities/apply_func.py\", line 35, in apply_to_collection\r\n  return function(data, *args, **kwargs)\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/pytorch_lightning/utilities/apply_func.py\", line 103, in batch_to\r\n  device_field = getattr(data, field).to(device, non_blocking=True)\r\nAttributeError: 'tuple' object has no attribute 'to'\r\nException ignored in: <function tqdm.__del__ at 0x7fcb5e0b2680>\r\nTraceback (most recent call last):\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py\", line 1086, in __del__\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py\", line 1293, in close\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py\", line 1471, in display\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py\", line 1089, in __repr__\r\n File \"/home1/thschaaf/miniconda3/envs/p37/lib/python3.7/site-packages/tqdm/std.py\", line 1433, in format_dict\r\nTypeError: cannot unpack non-iterable NoneType object\r\n```\r\n\r\n#### Code sample\r\n<!-- Ideally attach a minimal code sample to reproduce the decried issue. \r\nMinimal means having the shortest code but still preserving the bug. -->\r\n```\r\nimport torch\r\nfrom torch import nn, Tensor\r\nimport pytorch_lightning as pl\r\nfrom pytorch_lightning import Trainer, seed_everything\r\nfrom torchtext import data\r\nseed_everything(1234)\r\ndef get_debug_data_loader():\r\n    text_field = data.Field(sequential=True, pad_first=False,\r\n                            init_token=\"<s>\", eos_token=\"</s>\", include_lengths=True)\r\n    example1 = data.example.Example.fromdict({\"text\": \"a b c a c\"}, {\"text\": (\"text\", text_field)})\r\n    example2 = data.example.Example.fromdict({\"text\": \"b c a a\"}, {\"text\": (\"text\", text_field)})\r\n    example3 = data.example.Example.fromdict({\"text\": \"c b a\"}, {\"text\": (\"text\", text_field)})\r\n    dataset = data.Dataset([example1, example2, example3], {\"text\": text_field})\r\n    text_field.build_vocab(dataset)\r\n    iterator = data.Iterator(dataset, batch_size=3,\r\n                             sort_key=None, device=None, batch_size_fn=None,\r\n                             train=True, repeat=False, shuffle=None, sort=None, sort_within_batch=None)\r\n    return iterator, text_field\r\nclass DebugModel(pl.LightningModule):\r\n    def __init__(self):\r\n        super(DebugModel, self).__init__()\r\n        # setup data loader\r\n        self.debug_data_loader, self.text_field = get_debug_data_loader()\r\n        self.learning_rate = 0.001\r\n        self.hid_dim = 4\r\n        pad_idx = self.text_field.vocab.stoi['<pad>']\r\n        self.criterion = nn.CrossEntropyLoss(ignore_index=pad_idx)\r\n        self.INPUT_DIM = len(self.text_field.vocab)\r\n        self.ENC_EMB_DIM = 4  # keep it small for debugging\r\n        self.embedding = nn.Embedding(self.INPUT_DIM, self.ENC_EMB_DIM)\r\n        self.rnn = nn.GRU(self.ENC_EMB_DIM, self.hid_dim, 1, bidirectional=False)\r\n        self.out = nn.Linear(self.hid_dim, self.embedding.num_embeddings)\r\n        self.OUTPUT_DIM = len(self.text_field.vocab)\r\n    def configure_optimizers(self):\r\n        return torch.optim.Adam(self.parameters(), lr=self.learning_rate)\r\n    def forward(self, input_seq, length):\r\n        embedded: Tensor = self.embedding(input_seq)\r\n        packed_embedded: Tensor = torch.nn.utils.rnn.pack_padded_sequence(embedded, length, batch_first=False,\r\n                                                                          enforce_sorted=False)\r\n        packed_outputs, hidden = self.rnn(packed_embedded)  # [sent len, batch size, emb dim]\r\n        outputs, length = torch.nn.utils.rnn.pad_packed_sequence(packed_outputs)\r\n        # outputs -> [sent len, batch size, hid dim * n directions]\r\n        # hidden -> [n layers * n directions, batch size, hid dim]\r\n        output = outputs.squeeze(0)\r\n        prediction = self.out(output)\r\n        return prediction\r\n    @staticmethod\r\n    def _parse_batch(batch):\r\n        source = batch.text[0]\r\n        source_length = batch.text[1]\r\n        return source, source_length\r\n    def training_step(self, batch, batch_nb):\r\n        x = self._parse_batch(batch)\r\n        target, target_length = x\r\n        output = self.forward(target, target_length)\r\n        loss = self.criterion(output[:-1].view(-1, output.shape[2]), target[1:].view(-1))\r\n        prefix = 'train'\r\n        tensorboard_logs = {f'{prefix}_loss': loss.item()}\r\n        result = {'loss': loss, 'log': tensorboard_logs}\r\n        return result\r\n    def train_dataloader(self):\r\n        return self.debug_data_loader\r\nmodel = DebugModel()\r\ncuda_device_cnt = torch.cuda.device_count()\r\nif cuda_device_cnt > 0:\r\n    use_num_cuda_devices = 1\r\nelse:\r\n    use_num_cuda_devices = None\r\ntrainer = Trainer(fast_dev_run=False, max_steps=None,\r\n                  gradient_clip_val=10,\r\n                  weights_summary='full', gpus=use_num_cuda_devices,\r\n                  show_progress_bar=True)\r\ntrainer.fit(model)\r\n```\r\n\r\n### Expected behavior\r\n\r\n<!-- A clear and concise description of what you expected to happen. -->\r\nShould not raise an error :-) \r\n\r\n### Environment\r\n\r\n```\r\n CUDA:\r\n    - GPU:\r\n        - TITAN X (Pascal)\r\n    - available:     True\r\n    - version:      10.2\r\n* Packages:\r\n    - numpy:       1.17.3\r\n    - pyTorch_debug:   False\r\n    - pyTorch_version:  1.5.1\r\n    - pytorch-lightning: 0.8.5\r\n    - tensorboard:    2.2.2\r\n    - tqdm:       4.47.0\r\n* System:\r\n    - OS:        Linux\r\n    - architecture:\r\n        - 64bit\r\n        - \r\n    - processor:     x86_64\r\n    - python:      3.7.4\r\n    - version:      #1 SMP Tue Mar 17 23:49:17 UTC 2020\r\n```\r\n### Additional context\r\n\r\n<!-- Add any other context about the problem here. -->\r\n\n", "before_files": [{"content": "from abc import ABC\nfrom collections.abc import Mapping, Sequence\nfrom copy import copy\nfrom typing import Any, Callable, Union\n\nimport torch\n\nimport importlib\nTORCHTEXT_AVAILABLE = importlib.util.find_spec(\"torchtext\") is not None\nif TORCHTEXT_AVAILABLE:\n    from torchtext.data import Batch\nelse:\n    Batch = type(None)\n\n\ndef apply_to_collection(data: Any, dtype: Union[type, tuple], function: Callable, *args, **kwargs) -> Any:\n    \"\"\"\n    Recursively applies a function to all elements of a certain dtype.\n\n    Args:\n        data: the collection to apply the function to\n        dtype: the given function will be applied to all elements of this dtype\n        function: the function to apply\n        *args: positional arguments (will be forwarded to calls of ``function``)\n        **kwargs: keyword arguments (will be forwarded to calls of ``function``)\n\n    Returns:\n        the resulting collection\n\n    \"\"\"\n    elem_type = type(data)\n\n    # Breaking condition\n    if isinstance(data, dtype):\n        return function(data, *args, **kwargs)\n\n    # Recursively apply to collection items\n    elif isinstance(data, Mapping):\n        return elem_type({k: apply_to_collection(v, dtype, function, *args, **kwargs)\n                          for k, v in data.items()})\n    elif isinstance(data, tuple) and hasattr(data, '_fields'):  # named tuple\n        return elem_type(*(apply_to_collection(d, dtype, function, *args, **kwargs) for d in data))\n    elif isinstance(data, Sequence) and not isinstance(data, str):\n        return elem_type([apply_to_collection(d, dtype, function, *args, **kwargs) for d in data])\n\n    # data is neither of dtype, nor a collection\n    return data\n\n\nclass TransferableDataType(ABC):\n    \"\"\"\n    A custom type for data that can be moved to a torch device via `.to(...)`.\n\n    Example:\n\n        >>> isinstance(dict, TransferableDataType)\n        False\n        >>> isinstance(torch.rand(2, 3), TransferableDataType)\n        True\n        >>> class CustomObject:\n        ...     def __init__(self):\n        ...         self.x = torch.rand(2, 2)\n        ...     def to(self, device):\n        ...         self.x = self.x.to(device)\n        ...         return self\n        >>> isinstance(CustomObject(), TransferableDataType)\n        True\n    \"\"\"\n\n    @classmethod\n    def __subclasshook__(cls, subclass):\n        if cls is TransferableDataType:\n            to = getattr(subclass, \"to\", None)\n            return callable(to)\n        return NotImplemented\n\n\ndef move_data_to_device(batch: Any, device: torch.device):\n    \"\"\"\n    Transfers a collection of data to the given device. Any object that defines a method\n    ``to(device)`` will be moved and all other objects in the collection will be left untouched.\n\n    Args:\n        batch: A tensor or collection of tensors or anything that has a method `.to(...)`.\n            See :func:`apply_to_collection` for a list of supported collection types.\n        device: The device to which the data should be moved\n\n    Return:\n        the same collection but with all contained tensors residing on the new device.\n\n    See Also:\n        - :meth:`torch.Tensor.to`\n        - :class:`torch.device`\n    \"\"\"\n    def batch_to(data):\n        # try to move torchtext data first\n        if TORCHTEXT_AVAILABLE and isinstance(data, Batch):\n\n            # Shallow copy because each Batch has a reference to Dataset which contains all examples\n            device_data = copy(data)\n            for field in data.fields:\n                # Batch contains output of Field.process(...) which is tensor hence .to(...) exists\n                device_field = getattr(data, field).to(device, non_blocking=True)\n                setattr(device_data, field, device_field)\n            return device_data\n        else:\n            return data.to(device, non_blocking=True)\n\n    return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)\n", "path": "pytorch_lightning/utilities/apply_func.py"}], "after_files": [{"content": "from abc import ABC\nfrom collections.abc import Mapping, Sequence\nfrom copy import copy\nfrom typing import Any, Callable, Union\n\nimport torch\n\nimport importlib\n\nTORCHTEXT_AVAILABLE = importlib.util.find_spec(\"torchtext\") is not None\nif TORCHTEXT_AVAILABLE:\n    from torchtext.data import Batch\nelse:\n    Batch = type(None)\n\n\ndef apply_to_collection(data: Any, dtype: Union[type, tuple], function: Callable, *args, **kwargs) -> Any:\n    \"\"\"\n    Recursively applies a function to all elements of a certain dtype.\n\n    Args:\n        data: the collection to apply the function to\n        dtype: the given function will be applied to all elements of this dtype\n        function: the function to apply\n        *args: positional arguments (will be forwarded to calls of ``function``)\n        **kwargs: keyword arguments (will be forwarded to calls of ``function``)\n\n    Returns:\n        the resulting collection\n\n    \"\"\"\n    elem_type = type(data)\n\n    # Breaking condition\n    if isinstance(data, dtype):\n        return function(data, *args, **kwargs)\n\n    # Recursively apply to collection items\n    elif isinstance(data, Mapping):\n        return elem_type({k: apply_to_collection(v, dtype, function, *args, **kwargs)\n                          for k, v in data.items()})\n    elif isinstance(data, tuple) and hasattr(data, '_fields'):  # named tuple\n        return elem_type(*(apply_to_collection(d, dtype, function, *args, **kwargs) for d in data))\n    elif isinstance(data, Sequence) and not isinstance(data, str):\n        return elem_type([apply_to_collection(d, dtype, function, *args, **kwargs) for d in data])\n\n    # data is neither of dtype, nor a collection\n    return data\n\n\nclass TransferableDataType(ABC):\n    \"\"\"\n    A custom type for data that can be moved to a torch device via `.to(...)`.\n\n    Example:\n\n        >>> isinstance(dict, TransferableDataType)\n        False\n        >>> isinstance(torch.rand(2, 3), TransferableDataType)\n        True\n        >>> class CustomObject:\n        ...     def __init__(self):\n        ...         self.x = torch.rand(2, 2)\n        ...     def to(self, device):\n        ...         self.x = self.x.to(device)\n        ...         return self\n        >>> isinstance(CustomObject(), TransferableDataType)\n        True\n    \"\"\"\n\n    @classmethod\n    def __subclasshook__(cls, subclass):\n        if cls is TransferableDataType:\n            to = getattr(subclass, \"to\", None)\n            return callable(to)\n        return NotImplemented\n\n\ndef move_data_to_device(batch: Any, device: torch.device):\n    \"\"\"\n    Transfers a collection of data to the given device. Any object that defines a method\n    ``to(device)`` will be moved and all other objects in the collection will be left untouched.\n\n    Args:\n        batch: A tensor or collection of tensors or anything that has a method `.to(...)`.\n            See :func:`apply_to_collection` for a list of supported collection types.\n        device: The device to which the data should be moved\n\n    Return:\n        the same collection but with all contained tensors residing on the new device.\n\n    See Also:\n        - :meth:`torch.Tensor.to`\n        - :class:`torch.device`\n    \"\"\"\n\n    def batch_to(data):\n        # try to move torchtext data first\n        if TORCHTEXT_AVAILABLE and isinstance(data, Batch):\n\n            # Shallow copy because each Batch has a reference to Dataset which contains all examples\n            device_data = copy(data)\n            for field in data.fields:\n                device_field = move_data_to_device(getattr(data, field), device)\n                setattr(device_data, field, device_field)\n            return device_data\n\n        return data.to(device, non_blocking=True)\n\n    return apply_to_collection(batch, dtype=(TransferableDataType, Batch), function=batch_to)\n", "path": "pytorch_lightning/utilities/apply_func.py"}]}