At its 2022 Conference on Fairness, Accountability, and Transparency (ACM FAccT 2022), the
Association for Computing Machinery, in Seoul, South Korea, presented and published findings that
recommend that until AI and robotics systems are demonstrated to be free of bias mistakes, they are
unsafe, and the use of self-learning neural networks trained on vast, unregulated sources of flawed
internet data should be curtailed.[234]
Many AI systems are so complex that their designers cannot explain how they reach their decisions.[235]
Particularly with deep neural networks, in which there are a large amount of non-linear relationships
between inputs and outputs. But some popular explainability techniques exist.[236]
It is impossible to be certain that a program is operating correctly if no one knows how exactly it works.
There have been many cases where a machine learning program passed rigorous tests, but nevertheless
learned something different than what the programmers intended. For example, a system that could
identify skin diseases better than medical professionals was found to actually have a strong tendency to
classify images with a ruler as "cancerous", because pictures of malignancies typically include a ruler to
show the scale.[237] Another machine learning system designed to help effectively allocate medical
resources was found to classify patients with asthma as being at "low risk" of dying from pneumonia.
Having asthma is actually a severe risk factor, but since the patients having asthma would usually get
much more medical care, they were relatively unlikely to die according to the training data. The
correlation between asthma and low risk of dying from pneumonia was real, but misleading.[238]
People who have been harmed by an algorithm's decision have a right to an explanation.[239] Doctors, for
example, are expected to clearly and completely explain to their colleagues the reasoning behind any
decision they make. Early drafts of the European Union's General Data Protection Regulation in 2016
included an explicit statement that this right exists.[n] Industry experts noted that this is an unsolved
problem with no solution in sight. Regulators argued that nevertheless the harm is real: if the problem has
no solution, the tools should not be used.[240]
DARPA established the XAI ("Explainable Artificial Intelligence") program in 2014 to try to solve these
problems.[241]
Several approaches aim to address the transparency problem. SHAP enables to visualise the contribution
of each feature to the output.[242] LIME can locally approximate a model's outputs with a simpler,
interpretable model.[243] Multitask learning provides a large number of outputs in addition to the target
classification. These other outputs can help developers deduce what the network has learned.[244]
Deconvolution, DeepDream and other generative methods can allow developers to see what different
layers of a deep network for computer vision have learned, and produce output that can suggest what the
network is learning.[245] For generative pre-trained transformers, Anthropic developed a technique based
on dictionary learning that associates patterns of neuron activations with human-understandable
concepts.[246]
Artificial intelligence provides a number of tools that are useful to bad actors, such as authoritarian
governments, terrorists, criminals or rogue states.
Lack of transparency
Bad actors and weaponized AI