<NME> util_test.py
<BEF> ADDFILE
<MSG> Mnist duet nn sequential (#4750)

* added request_block support

* ran black

* fixed return types

* Added loopback=True to launch_duet and join_duet

- loopback automatically writes and reads the IDs to a shared tmp file
- The purpose of this helper is to speed up development or local usage

* Converted prototype MNIST notebook into Duet Notebooks

- Added additional required ops for MNIST
- Added matplotlib when displaying images for inference

* ignoring bandit warning on /tmp folder

- How else can we get a deterministic tmp folder between processes?

* Fixed /tmp reference with tempfile.gettempdir()

- Unique path on macOS seems to be deterministic between procs
- This should now work on Windows as well
- Tested on MNIST notebooks

* Added tests for torch.Tensor.backward and torch.Tensor.__getitem__

- Added requires_grad to Tensor since its legal to use requires_grad
  without it being a nn.Parameter and required by the .backward test

* Rebase off syft_0.3.0

* Re-enabling Windows CI from syft_0.3.0

* Renamed return types of SyNone to new _SyNone class

- SyNone is the singleton and _SyNone is the constructor Class

* Made improvements to get / request blocking mode

- Added customizable timeout
- Tested on MNIST notebooks

* proto regen noise

* Updated MNIST duet notebooks with blocking get requests

- Added torchvision DataLoader pointers from remote side

* Override __module__ for syft types (#4724)

* Renamed examples/pytorch to examples/duet

- Updated .gitignore for data download dirs

* Added request_name and reason to blocking get request

- Added more verbose output and UID in sy.VERBOSE mode

* Added sy.Module which aims to provide similar functionality to nn.Module

* Updated MNIST notebooks to use sy.Module

* sequential support

* removing MNIST

* Changed train and test loops to use precalculated index

- Added some get blocking to inference
- Benchmarked Sequential vs sy.Module, 40 secs vs 90 secs per epoch

* Improved test iteration for sy.module

* expelliarmus verbosus 🧙🏽‍♂️ (expel the VERBOSE)

- sy.VERBOSE is gone
- import syft as sy # absolute import is eradicated (except for Pointer)
- new logging example code added to MNIST notebooks

* Fixes KeyError on print(duet.store) #4745

- fqn was searching ast for syft.lib.python.list.List
- code now removes extra .class.Class and correctly searches for and
  finds: syft.lib.python.List
- extra exception catching and logging added to the code path
- syrange removed
- fetching grad on torch.Tensor only if requires_grad is True
- fixed issue with blocking request where loop wasn't waiting 1 second

* Refactored obj_id -> id_at_location to avoid confusion during debugging

- Replaced: node.store.store(obj=result)
  with    : node.store[self.id_at_location] = result
  which forces only 1 entry point for __setitem__ making tracking easier
- remove unused method send_signed ReprMessage in duet

* Added lots of try except and logging to WebRTC to try and catch errors

- Added logging of return Exception Message

* Added get_copy() and get(delete_obj: bool=True)

* Added code for downloading remote modules and overwriting local net

- Added state_dict to allowlist
- Added upcast function to refactor upcast_args_and_kwargs
- Fixed issue with Tuple and __getitem__ returning Tensors
- Added upcast to Tuple deserialize
- Fixed issue with Dict and items / __getitem__ returning Tensors
- Removed Dict __str__ which causes an exception, now works like List
- Added Hack to GetObjectResponseMessage to convert OrderedDict to Dict
- Fixed issue with sy.Module modules not automatically getting added
  sometimes when imported the full_name_with_qualname is different
  if "torch.nn" in full_name_with_qualname(klass=type(value)):
- Added async logging with enqueue=True
- Added some model download helpers to sy.Module

* Fixed issues with local_model sy.Module

- Tested sending large tensor like fc1 from MNIST, FC 9216x128 too big?

* Fixing local test function

- Added model scaling factor to reduce fc1 so it can be downloaded

* seq update

* Fixed duet loopback only working when you start DO first

- You can now start the notebooks in either order
- Updated a few display strings

* Added Request Handlers

- Domain has a thread which checks the handlers
- Client can send a handler rule and update or remove them
- Accept and Deny are being dispatched directly to the self Domain
- Added timeout_secs to requests which now also deletes them server side

* Ooops... forgot to remove the un-needed recursion in the While loop 🤦‍♂️

* Changed domain threading to asyncio thanks to Ionesio

- Added ability to print or log output of rejected requests
- Moved copy_remote_state request params to the outside
- Lowered the cov numbers 👻 until next week 😬

* Caught a blocked async await with the logging

* protobuf serde

* cleanup

* rebuilding proto

* proto fix

* Fixing windows tests with @pytest.mark.asyncio

* Fixing Windows asyncio tests

* progress on slaying json

* Added more logging debugging to the WebRTC blocking issue

- Added PORT env variable for network.py grid code

* Removed JSON and Hex and replaced all with to_bytes from_bytes

- added Metadata class and proto
- disabled JSON based serde tests to be fixed later
- removed unused grid files

* Added a message rate limiter

- Added queue monitoring with water mark and heart beat

* Turned off message rate limiter and trained 1 epoch to 96% accuracy 🎉

* Added colab cell at the top and classify download image on DS notebook

* Disabled old non WebRTCDuet tests which was removed

* Update to the README.md based on the existing main branch copy

- Badges and URLs need to be updated

* PR Improvements

- Removed unused old network.py and moved PORT code to examples version
- renamed sum_layers to debug_sum_layers

* Increasing code coverage back to 82%

- Fixing tests and start.ipynb after JSON serialization removal
- Added some additional tests
- Deleting unused code

* Fixing Windows async tests that use Domain()

- Removing unused proto files

* Remove unused test for old non WebRTC version of Duet which was removed

* Fixed GC tests on Parameter including new get_copy

Co-authored-by: Anubhav Raj Singh <ae6067cb821ab45cee3e00f81cdb0d7465f59f18@gmail.com>
Co-authored-by: Jasopaum <6b052968b438065457086a196781936c8fffa76a@gmail.com>
Co-authored-by: Tudor Cebere <041a3c880d46f720c6c93403ce5f8bc370b5cad3@gmail.com>
<DFF> @@ -0,0 +1,32 @@
+# third party
+import pytest
+
+# syft absolute
+from syft.util import get_fully_qualified_name
+from syft.util import get_subclasses
+from syft.util import index_syft_by_module_name
+
+
+def test_get_fully_qualified_name_exception() -> None:
+    class Bad:
+        def __class__(self) -> None:  # type: ignore
+            return None
+
+    with pytest.raises(Exception):
+        get_fully_qualified_name(Bad())
+
+
+def test_get_subclasses() -> None:
+    class A:
+        pass
+
+    class B(A):
+        pass
+
+    subclasses = get_subclasses(obj_type=A)
+    assert subclasses == [B]
+
+
+def test_index_syft_by_module_name() -> None:
+    Int = index_syft_by_module_name(fully_qualified_name="syft.lib.python.Int")
+    assert Int(1) == 1

{'addition_count': 32, 'commit_subject': 'Mnist duet nn sequential (#4750)', 'deletion_count': 0, 'file_extension': '.py', 'lang': 'py', 'license': 'apache-2.0', 'repo_name': 'OpenMined/PySyft'}