abstract: We present a shared control paradigm that improves a user's ability to operate complex, dynamic systems in potentially dangerous environments without *a priori* knowledge of the user's objective. In this paradigm, the role of the autonomous partner is to improve the general safety of the system without constraining the user's ability to achieve unspecified behaviors. Our approach relies on a data-driven, model-based representation of the joint human-machine system to evaluate, in parallel, a significant number of potential inputs that the user may wish to provide. These samples are used to (1) predict the safety of the system over a receding horizon, and (2) minimize the influence of the autonomous partner. The resulting shared control algorithm maximizes the authority allocated to the human partner to improve their sense of agency, while improving safety. We evaluate the efficacy of our shared control algorithm with a human subjects study (n=20) conducted in two simulated environments: a balance bot and a race car. During the experiment, users are free to operate each system however they would like (i.e., there is no specified task) and are only asked to try to avoid unsafe regions of the state space. Using modern computational resources (i.e., GPUs) our approach is able to consider more than 10,000 potential trajectories at each time step in a control loop running at 100Hz for the balance bot and 60Hz for the race car. The results of the study show that our shared control paradigm improves system safety without knowledge of the user's goal, while maintaining high-levels of user satisfaction and low-levels of frustration. Our code is available online at <https://github.com/asbroad/mpmi_shared_control>.
author:

bibliography: references.bib
title: Highly Parallelized Data-driven MPC for
  Minimal Intervention Shared Control

# Introduction

Shared control is a paradigm that incorporates an autonomous partner into the control loop of a robotic system to help a human partner achieve tasks they would otherwise be unable to on their own . This approach offers an alternative to fully autonomous robotic systems, and can be used to extend the efficacy of modern robots in human-oriented domains (e.g., surgery, assistance and rehabilitation, search and rescue, etc$\dots$) through collaboration. In these domains the two partners often need to communicate frequently and may even be co-located. The most important consideration of the autonomous partner is therefore the safety of the joint system, as unsafe behavior can lead to injury to the human operator. However, there are other features that may be equally important when we consider both the range of behaviors the human partner may wish to perform, and the user's acceptance of the assistance provided the autonomous partner.

In the majority of related work, the decision of which partner should be in control at a given time is made based on (1) the safety of the system and (2) an evaluation of who would provide better input to achieve a perceived, or known, task goal . These task-specific systems can frustrate the human partner when the user's intended goal is difficult to predict, and they fail out-right when there is no desired trajectory, task or goal the user is trying to achieve. Consider, for example, a person operating a lower-limb exoskeleton for rehabilitation purposes. In this scenario, task-level and performance-based metrics (e.g., the amount of area covered during a search mission), are not important in determining which partner should be in control at a given time, as there is often no explicit notion of a goal (i.e., the human operator may simply want to wander around aimlessly). Instead, the person's instantaneous desires are often the most relevant feature, conditioned on the general safety of the system.

The question we consider in this work then is, how does one define safety constraints that can enhance a user's ability to operate complex, dynamic machines without artificially constraining their capacity to achieve unspecified behaviors? We address this gap in the literature with a task-agnostic control allocation strategy that balances the human's desires to achieve a wide range of possible behaviors, while simultaneously improving the safety of the joint system. Our shared control paradigm therefore adheres to the following three ideals:

1.  safety is paramount,

2.  the user has no explicit task goal, and

3.  the autonomy should exert as little influence as possible.

In other words, the goal of our shared control paradigm is to allow the user to do whatever they would like, so long as the safety of the joint system is satisfied. Additionally, when the autonomous partner does intervene, it should only minimally modify the user's input (a.k.a., the minimal intervention principal ). By adhering to these ideals, we hope to increase the influence of the human partner and consequently improve their acceptance of the assistance provided by the autonomy.

In this work, we develop a shared control algorithm that uses a highly parallelizable sampling-based model predictive control (MPC) algorithm to generate the autonomous partner's policy. By sampling densely at uniform over the input space, we can evaluate a large set of potential actions that the user may wish to take, without *a priori* knowledge of a specific goal. We then use ideas from model-based reinforcement learning and model predictive control to generate predicted trajectories (or imagined rollouts) that represent the configuration (and safety) of the robot over a receding horizon. Conditioned on the predicted safety of the robotic system, we iteratively select the sampled action that most closely matches the human partner's input, allowing the user to more safely move around the environment without adhering to a single objective. Our approach relies on a representation of the human-machine system that is valid both with, and without, a known analytical model. We focus on the latter case and therefore learn a model of the joint system offline from data. We evaluate the efficacy of our approach with a human subjects study in two simulated environments. Additionally, we provide an open-source, scalable implementation of our algorithm in both environments that uses a GPU for real-time interaction.

The main contributions of this work are therefore:

-   A highly parallelizable sampling-based model predictive control algorithm for autonomous policy generation.

-   A predictive notion of safety that can evaluate the impact of a current action over a receding horizon.

-   A human-motivated cost function that only considers the instantaneous desires of the human-in-the-loop and requires no knowledge of an explicit goal.

-   Results of a human subjects study to evaluate the efficacy of, and user experience with, our shared control paradigm.

-   A GPU-implementation for real-time control of two simulated systems.

In Section  we provide background information and related work. In Section  we detail the theoretically exact solution to our problem. In Section  we describe our approximation, why it scales well to the majority of devices in our target domain (human-centered robotics), and provide analytical bounds on the sub-optimality of the applied control with respect to the human's desired motion. In Section  we describe the experiment we use to validate our shared control paradigm through a human subjects study consisting of 20 participants in two simulated environments. We also detail pertinent metrics to analyze the human partner's control skill and style, which are easily computable due to our sampling-based approach. In Section  we present the study results which we discuss in Section . We conclude in Section .

# Background and Related Work

In this section we discuss background and related literature in both shared and fully autonomous control, with a particular focus on human-oriented domains.

## Shared Control

The majority of the shared control literature focuses on assisting a human operator when a desired task is known *a priori*  or predicted based on a model of the operator's intent . For this reason, task success is often the primary metric of concern in analyzing shared control systems, while the user's desires relating to *how* a motion is achieved are often disregarded. Related literature that addresses this same limitation in shared control includes techniques that rely on model-free control policies , POMDPs , and control barrier functions . In some application domains there is a welcome trade-off between achieving the desired high-level goal and intervention from the autonomous partner (e.g., with ground vehicles on a roadway where a large amount of structure is enforced on the motion of the dynamic system). In more human-centered domains, such as assistive and rehabilitation robotics, there is often significantly less structure imposed on the motion of the machine and there may be no explicit goal. For this reason, the same trade-off in task success and autonomous intervention is not consistently accepted by users . In these domains, it is instead of utmost importance that the user retains a sense of personal agency and a feeling of control over the mechanical device.

Despite these differences, the most closely related work to our own (from a methodological standpoint) can be found in the semi-autonomous vehicle literature. The semi-autonomous vehicle paradigm is distinct from the concept of a fully autonomous self-driving cars as the autonomy's goal is not to take full control of the vehicle, but instead to act as a guardian or intelligent co-pilot , intervening on the person's control when deemed necessary to ensure safety. For this reason, there is a growing line of work that follows the minimum intervention principle in the semi-autonomous vehicle domain . For example, Schwarting et al.  describe a parallel autonomy framework that develops control trajectories for semi-autonomous vehicles that minimize deviation from user-input and achieve task-specific metrics like road following and contour tracking. Anderson et al.  describe a geometric, homotopy-based algorithm for computing *free space* in the environment. The human operator is then allocated full control of the dynamic system so long as their input will not violate the constraints defined by the geometric constructs. In this work we provide an alternative method (prediction) of computing safe space in the environment that is simpler (e.g., there is no need to integrate constraints defined by potentially complex geometries into the optimization problem), and acts over the entire control space (i.e., instead of only the steering angle ). Additionally, all prior work in this area assumes *a priori* knowledge of the system dynamics whereas our technique extends to models learned from data.

## Sampling-based and Stochastic Optimal Control

From a control-theoretic standpoint, related work includes shared control algorithms that build on ideas from sampling-based optimal control. For example, Carlson et al. have explored the idea of controlling a wheelchair using *safe mini-trajectories* , however, this work again relies on *a priori* knowledge of the human operator's goal (or predicted goals based on sensor information). Relatedly, Shia et al.  use a probabilistic model of the user's potential future inputs to achieve a pre-specified task, instead of allowing the operator the freedom to move however they would like at each instant.

Our approach is also related to fully autonomous control solutions that rely on sampling-based and stochastic optimal control methods. For example, Lavalle et al. propose Rapidly-expanding Random Trees (RRTs) , which develop random trajectories through the state space that are achievable as they are constrained by the system dynamics. More recently, Kousik et al. extend this idea through a model predictive control algorithm that is based on the notion of a forward reachability set  to ensure safety. Finally, Williams et al. have proposed Model Predictive Path Integral (MPPI)  control as a method of solving the optimization problem through path integrals. These ideas build on similar theory to our own (approximating an optimal solution from a nominally infinite set of trajectories), however, they are again all standardly defined in relation to a specified start and goal configuration. We instead consider an approximation to the infinite set of trajectories that stem from a single point and extend in *all* directions for a given time-horizon.

# Safe Minimal Intervention Shared Control

In this section, we describe a theoretically correct (though computationally infeasible) solution to the problem of safe minimal intervention shared control. We begin by defining the shared control problem mathematically. This problem can be posed in standard optimal control terms with an additional constraint to incorporate information from both partners. That is, our goal is to find an optimal action sequence $\bf{\bar{u}}$, and corresponding state sequence $\bf{\bar{x}}$, that minimizes the cost $$\begin{aligned}
& \underset{J}{\text{minimize}}
& J(x(t), u(t)) &= \int\limits_{t=0}^{T} l(x(t), u(t)) + l_{T}(x(t)) \\
& \text{subject to}
& \dot{x}(t) &= f(x(t), u(t)),\\
& & u(t) &= g(u_h(t), u_a(t)) \\
\end{aligned}
\label{eqn-mpc}$$

where $x(t)$ and $u(t)$ are the state and control trajectories, and $l$ and $l_{T}$ are the running and terminal costs. The optimization is subject to constraints $f$ representing the nonlinear system dynamics, and $g$ representing the control allocation between the human ($u_h$) and autonomous partners ($u_a$).

## Safe Control and Inevitable Collision States

The primary constraint in $g$ relates to the safety of the human-machine system. That is, the autonomous partner should only produce trajectories that remain safe over a receding horizon. This requires ensuring that the system does not enter an Inevitable Collision State (ICS) . ICSs refer to configurations from which it is impossible to safely recover, regardless of the control trajectory taken. So long as the system does not enter an ICS, it is possible to develop a control strategy that results in continued safe interaction.

## Minimum Intervention Principle

The second feature we embed in $g$ is known as the minimal intervention principle (MIP) which states that "an autonomous partner should only augment the human partner's control by the minimum amount necessary to achieve the desired result" . By developing a shared control algorithm that adheres to the MIP we maximize the influence of the human partner's control at each given moment.

## Minimal Intervention Shared Control

The described solution to minimal intervention shared control is outlined in Algorithm . Here $\Gamma$ is the set of all possible trajectories $\gamma$ that stem from the current state $x(t)$, and $\Gamma_{safe}$ is the subset of safe trajectories. The inputs $u_r$ and $u_h$ are the signal sent to the robot, and the signal provide by the human partner, respectively. The $cost$ function describes how desirable a trajectory is based on it's distance from the human operator's input. If the human's command does not lead to an ICS, their exact input will be passed to the system; otherwise, a perturbation (computed as the minimum deviation required to ensure safety) will be applied to the control signal.

This exact (full information) algorithm is computationally infeasible to compute online for any reasonably complex human-in-the-loop system. In particular, the solution requires exploring an infinite set of potential actions over a receding horizon to (1) ensure collision-free trajectories , and (2) select the input that most closely matches the human's desired action. In this work, we instead propose an approximation to this solution that can be computed in real-time.

# Model Predictive Minimal Intervention Shared Control

To compute an approximation to the optimal solution described in the prior section, we make a few key methodological choices. First, instead of considering *all* possible trajectories from the current sate, we only consider a representative set that we generate by sampling densely (from a uniform prior) over the input space and only at the present time. We then predict the motion of the system over a receding horizon and reject any inputs that generate a trajectory that violate the defined safety constraints. From the subset of inputs that do not produce unsafe trajectories, we select the control signal that is closest to the human partner's input. Notably, this approach does not require a model of the user's actions, or knowledge of a desired goal, as the user is free to dynamically adjust their objective at each timestep. We describe each step of this algorithm in detail in the following subsections.

## Model Representation and Data-driven Approximations

The majority of related work requires hand-written (and potentially complex) models of the system and control dynamics . In this work, we instead use a representation that is simple (i.e., linear) regardless of the underlying dynamics and can be learned from data when the model is not known *a priori*. This representation is known as the Koopman operator  and it can be used to model nonlinear dynamical systems as a linear operator because it *maps functions of state to functions of state* instead operating in the original state space. This representation is valid both when we know an analytical model of the system in the standard state space , and when we must learn the model from data .

In this work, we opt to learn the system model from data to demonstrate the efficacy of our approach when we have no prior knowledge of the robotic system. In particular, we use sparsity-promoting Dynamic Mode Decomposition  to select an appropriate basis and approximate the Koopman operator . Data-driven Koopman operators have recently been explored as a method of generating model-based control in robotics  and in shared control . Unlike in prior work, however, the choice of the Koopman operator representation is explicitly motivated by our sampling-based policy generation method (see Section ) and modern computational resources (e.g., multi-core CPUs and GPUs). That is, the Koopman operator representation is particularly well suited for sampling-based control as forward predicting the state of the system requires only a single matrix-vector multiplication, and generating a large set of potential trajectories requires only a single matrix-matrix multiplication. Both of these operations can be easily parallelized on a GPU . Related work in model-based control for dynamic systems has utilized linear representations (e.g., Bayesian linear regression ), however, to the best of our knowledge, ours is the first work to develop a model-based controller the integrates a Koopman operator representation with sampling-based optimal control.

## Sampling-based Optimal Control and Predictive Safety

To generate a set of *unconstrained* potential trajectories the user may wish to execute, we sample $N$ inputs from an equally-spaced discretization of the control space. By relying on a uniform prior, we make no assumptions about the user's desired action at the next step (i.e., *there is no model of the user*). These samples can also be generated stochastically. However, stochasticity has known downsides in human-in-the-loop systems. For example, inputs that generate motion directly along a single dimension—a common desire of human operators—are unlikely to be sampled as they exist in segments of the input space that have near zero probability mass when sampling from a continuous distribution.

To ensure that our shared control algorithm only considers trajectories that satisfy the safety constraint described in Section , we evaluate the configuration of the system at each time step over the receding horizon in each predicted trajectory. If the system violates geometric safety checks that are defined with respect to the environment, we reject the input that generated that trajectory. If, however, the system is safe over the entire course of the predicted trajectory, we consider that input as a viable solution. This can be seen as a *predictive notion of safety* as we evaluate the likelihood of a particular signal leading to a catastrophic failure by observing how we expect the controlled device to evolve over time.

## Model Predictive Minimal Intervention Shared Control

The full Model Predictive Minimal Intervention Shared Control (MPMI-SC) approach can be seen in Fig.  and is outlined in Algorithm . The inputs are the current time $t$, the current state $x_t$ and the human partner's input $u_h$. $\xi$ is the sampled control, $\mathbb{U}$ is the distribution the control is sampled from, $M$ is the dimensionality of the input space, $N$ is the number of samples, and $T$ is the prediction horizon. During forward prediction (Line 6), $z_i$ is sampled i.i.d. from a white noise Gaussian process to account for inaccuracies in the dynamics model. Notably, this computation is done *in parallel on a GPU*. The learned Koopman operator system model $f(x,u)$ predicts the state $x_p$ at the next timestep $t_p$, which is evaluated for safety. The $cost$ function minimizes the influence of the autonomous partner, and $u_r$ is the control signal sent to the robot.

## Minimal Intervention Principle and Expected Deviation

An additional benefit of our sampling-based approach is that we can provide an explicit bound on the sub-optimality of the applied control *with respect to the user's instantaneous desires*. In particular, the deviation between the user's input and the applied control signal (*when the user input is safe*) is upper bounded by half the distance between the sampled inputs. This can be computed based on the number of samples generated at each timestep and the Lebesgue measure  of the control space. This relationship is described in Equation .

$$\mathbb{E}[\lVert u_h, u_r \rVert] = \frac{\lambda^*(U)}{2N}
\label{eqn-exp-deviation}$$

where $u_h$ is the human partners input, $u_r$ is the signal sent to the robot, and $\lVert u, v \rVert$ is the Euclidean distance. $\lambda^*(U)$ is the n-dimensional volume (i.e., Lebesgue measure) of the bounded input space and $N$ is the number of samples. As the number of samples grows ($N \rightarrow \infty$), the maximum deviation between the user's input and the applied signal shrinks to 0. However, we also note that while the number of samples (denominator) grows linearly, the Lebesgue measure (numerator) grows exponentially with each additional control dimension (i.e., the measure is defined as the Cartesian product of the intervals of each dimension). This describes a potential issue in the scalability of our sampling based solution; however, this issue is mitigated in the majority (if not all) of our application domains as the dimensionality of the input generally remains low as the human operator must capable of providing input to the system (e.g., using a joystick). The interval of each control dimension also generally remains small (e.g., \[-1, 1\] or \[0, 1\]) so that it is understandable by the human partner. A more in-depth discussion of the scalability of our algorithm is presented in the supplementary material.

# Experimental Evaluation

We validate MPMI-SC with a human subjects study in two simulated environments which we describe below.

## Simulated Environments

The first dynamic system that users operate is a simulated balance bot (Fig. ). When controlling this system, users are told (1) that they are free to move around the environment however they would like, and (2) that they should try to make sure that body of the robot does not collide with the ground (i.e., the safety constraint). This can be challenging for a novice operator due to the stabilization requirements. The system is based on an open-source package  developed using the PyBullet physics engine. The observation space is a three-dimensional continuous vector that includes the angular position and velocity of the robot's body, and the linear velocity of the system. The input is a one dimensional continuous signal that sets a target velocity for both wheels.

The second dynamic system that users operate is a simulated race car (Fig. ). When controlling this system, users are told (1) that they are free to move around the environment however they would like, and (2) that they should take caution not to drive off of the road (i.e the safety constraint). This can be challenging for a novice operator due to the narrowness of the road and the fact that the car can go unstable (e.g., skid out) if the force applied to the system exceeds the friction limit of the ground surface. This system is based on an open-source package released by OpenAI  and developed using the Box2D physics engine. The observation space is a six dimensional continuous vector that includes the $(x,y,\theta)$ position of the car and the associated velocities $(\dot{x},\dot{y},\dot{\theta})$. The input to the system is a three dimensional continuous vector that defines the desired heading for the robot, the positive acceleration (gas) and negative acceleration (break).

The complexity of these two systems makes them useful as testbeds to validate the impact of our shared control algorithm. In particular, stabilization and environmental constraints are important challenges for robotic systems in human-centered domains where the human partner is often co-located with the mechanical system.

## Experimental Design

To evaluate the impact of our shared control algorithm we ran a human-subjects study (n=20) in which participants operated the system under two distinct interaction paradigms:

-   User-only control (No assistance)

-   Model Predictive Minimal Intervention Shared Control

The order in which the participants saw the two paradigms was randomized and counter-balanced to account for ordering effects. In each trial, participants were told to perform whatever action they would like so long as the system remained safe. They were also told that in some conditions an autonomous partner would help maintain safety, but they were not told *how* it would help. A trial would end when the system violated one of the defined safety constraints or after a maximum alloted time (balance bot: 20 seconds, race car: 30 seconds). Each participant interacted with the system 10 times in each environment under each control condition. In the race car environment, the morphology of the road was generated randomly at the start of each trial, however, the same random seeds were maintained across participants to ensure that each subject saw the same road configurations.

## Implementation Details

### Safety Computation

In both experimental environments, the system is considered unsafe when it violates physical barriers. For the balance bot this is defined as the robot body colliding with the ground. For the race car this is defined as driving off the track. To compute the safety of a predicted trajectory we evaluate the configuration of the system at each discrete timestep. Theoretically, these geometric checks provide strong safety guarantees as we always pick the applied control signal from the subset of sampled inputs that do not violate the defined bounds. In practice, the accuracy of this method relies both on precise system models and our ability to account for noise in the dynamics and/or sensors. In this work, we account for these errors by adding an inflated barrier beyond the natural collision points to reduce the impact of inaccurate predictions. Importantly, even with errors in the dynamics model, and noise in the sensor measurements, one can define an inflation radius that provides strong guarantees on the safety of the system . However, in this initial work, we simply rely on a hand-tuned inflation radius.

### Algorithm Parameters

The implementation of our algorithm requires defining a small set of parameters, the two most important of which are the number of control samples $N$, and the length of the receding horizon $T$. As these parameters increase the approximation to the true solution improves, however, this comes at a cost of increased computational complexity. To address this issue, we provide a highly parallelized implementation of our algorithm that evaluates each trajectory independently on an NVIDIA GeForce 860M GPU (details in supplementary material). To provide a good user experience and demonstrate the scalability of our algorithm, we select a large $N$: 10,000 for the balance bot and 10,120[^1] for the race car. The receding horizon $T$ ($T=30$ for the balance bot, $T=25$ for the race car) was chosen based on observations of each system under MPMI-SC with random inputs.

### Basis Selection

Koopman operator system identification requires defining a basis to approximate the nominally infinite dimensional Hilbert space that the Koopman operator acts on. We select this basis through data-driven methods (as described in Section ). In particular, we generate a set of 50 and 150 random basis functions for the balance bot and race car environments, respectively. The chosen learning algorithm  selects the most relevant basis functions for modeling each system, resulting in 6 and 26 basis functions for the balance bot and race car environments, respectively. All parameter choices are well documented in the open-source code.

# Results

We first explore the impact of our algorithm on the safety of the human-machine system, and the users' response to the intervention of the autonomy. We then detail the computational performance of our system using the defined parameter settings. Finally, we provide a secondary analysis of metrics that relate to the human operator's control skill and style.

## Impact of Shared Control on Safety

To evaluate the impact of MPMI-SC on the safety of the joint system we compare (1) the average fraction of safe interactions to unsafe interactions and (2) the average time it took for the system to enter an unsafe state under each control paradigm (Fig. ). To compare these values, we use a non-parametric Wilcoxon signed-rank test. In both experimental environments we find that MPMI-SC significantly improves the rate at which users are able to safely control the system when compared to a user-only control paradigm ($p < 0.005$). Similarly, we find that users are able to safely control the system for a significantly longer amount of time under shared control then when under user-only control ($p < 0.005$).

## User Acceptance of Shared Control Paradigm

We next evaluate the users' acceptance of the assistance provided by the autonomous system. As mentioned in Section , the majority of shared control systems assist a user in achieving a specific task. The assistance can therefore come at the expense of user satisfaction as the human partner often feels that they are fighting the autonomy . In contrast, we develop a task-agnostic shared control paradigm that adheres to the human partner's desires at each instant.

To evaluate the user experience under MPMI-SC we asked participants to fill out a post-experiment questionnaire (Fig. ). Statements were rated on a 5-point Likert-type scale where 1 represents strong disagreement and 5 represents strong agreement. Overall, user's felt the assistance provided by MPMI-SC helped them keep the robot safe and execute their indented commands. Perhaps most telling is that the participants strongly preferred operating the system with assistance from the computer, and did not feel frustrated by the assistance.

## System Performance

We now describe the computational efficiency of MPMI-SC using the parameters defined in Section . Our algorithm is capable of generating trajectories at $\sim$`<!-- -->`{=html}7000 Hz and $\sim$`<!-- -->`{=html}3500 Hz in the balance bot and race car environments. Incorporating the safety checks, the system runs at $\sim$`<!-- -->`{=html}100 Hz and $\sim$`<!-- -->`{=html}60 Hz, respectively (i.e., between 600,000 and 1,000,000 trajectories every second). MPMI-SC is faster in the balance bot as it relies on a smaller basis and more efficient safety checks. A detailed description of our GPU implementation and the scalability of MPMI-SC is provided in the supplementary material.

We also compute the maximum possible deviation between a user's input and the closest safe signal at each timestep. This value is determined by evaluating Equation  with a known n-dimensional input volume ($\lambda*$) and number of samples ($N$). If the user's input is considered safe over the receding horizon (i.e., through our prediction method described in Section ), this value represents *the maximum influence of the autonomous partner on the applied signal*. If the user's input is not safe over the predicted trajectory, this bound represents *the maximum possible difference between the true closest safe signal and the applied signal*. For the balance bot, the maximum deviation is $\pmb{\mathbb{E} = 0.0001}$ because $\lambda^*(U)$ = 2 (i.e., 1D input that spans $[-1, 1]$) and $N = 10,000$. For the race car, the maximum deviation is $\pmb{\mathbb{E} = 0.0002}$ because $\lambda^*(U)$ = 4 (i.e., 3D input where the heading spans $[-1,1]$, the gas spans $[0,1]$, and the break spans $[-1, 0]$) and $N = 10,120$.

How large an impact an intervention of the described magnitudes has on the dynamics of a dynamic system is dependent on each particular machine. However, we note that both values described in this work are *very* small. We therefore hypothesize that there is likely no discernible difference between the effect of the intended signal and the applied signal when the user's input is considered safe. If there were a large impact on the system dynamics due to an intervention of this magnitude, the system may be chaotic and therefore challenging to operate no matter what controller was used .

## User Control Skill and Style

Finally, we provide a secondary analysis of the data collected during our experiment to evaluate the users' control skill and style. The proposed metrics are easy to compute based on the users' input and the trajectories generated at each timestep.

### Operator Control Skill

We first calculate the average observed deviation between the user input and the *closest safe signal* as a proxy for the users' understanding of the system dynamics and their control skill. To illustrate this relationship consider a car making a tight turn. If the human partner understands that they need to change their heading and speed to stay on the road during the turn, this metric will remain low (Sec. ). If, however, the human partner does not understand this relationship and relies on the autonomous partner to maintain the safety of the system during the turn, this metric will increase. The average value of this metric will be within the described bound ($\pmb{\mathbb{E}}$, above) if, and only if, *all* of the user's controls are considered safe during the course of their interaction. Otherwise this value will increase according to the minimal amount required to maintain safety.

### Operator Control Style

We then calculate the average percentage of sampled rollouts that are safe at each timestep as a proxy for the user's control style. A lower percentage of safe trajectories suggests that the user is operating the system in a more *dangerous* manner (i.e., the system may be closer to entering an ICS). However, we note that this metric alone does not directly relate to the user's control skill as there are many cases in which one may trade off notions of safety for performance. To illustrate this relationship consider, again, a person in a car taking a tight turn to decrease the time of their drive. If this value correlates positively with safe control of the system, we can say that the operator likely has a high degree of skill and is explicitly trading off conventional notions of safety (e.g., distance from an obstacle) for improved performance. However, if this metric correlates positively with unsafe behavior, it is likely that the operator is unskilled and making poor control decisions.

|        **Metric**        | **Control** | **Balance Bot** |  **Race Car**   |
|:------------------------:|:-----------:|:---------------:|:---------------:|
|    **Avg. Deviation**    |  User-only  | 0.46 $\pm$ 0.37 | 0.05 $\pm$ 0.03 |
|                          |   MPMI-SC   | 0.21 $\pm$ 0.21 | 0.07 $\pm$ 0.06 |
| **Avg. % Safe Rollouts** |  User-only  | 0.28 $\pm$ 0.15 | 0.38 $\pm$ 0.09 |
|                          |   MPMI-SC   | 0.50 $\pm$ 0.10 | 0.35 $\pm$ 0.08 |

Average deviation and average percentage safe rollouts, broken down by environment and by control condition.

### Analysis

In the balance bot environment, we find that the user's input more closely aligns with the closest safe signal under MPMI-SC (Table ). We also observe that the system remains in a state where more potential actions remain safe over the horizon. We interpret these results as evidence that the users provided more competent control with assistance than without. In the race car environment, we find that, in both control conditions, the user's input is nearly equally aligned with the closest safe signal, and that there are similar percentages of potentially safe actions. We interpret these results as evidence that MPMI-SC had less impact in this environment then in the balance bot. Evidence of this interpretation can also be seen in Figure  where we find larger raw differences between the safety metrics in the balance bot environment than in the race car environment. However, the differences between each of these primary safety metrics are statistically significant suggesting that, even when MPMI-SC is less impactful, it still meaningfully improves the safety of the joint system. Both proposed metrics are indirect measures of the user's control skill and style, but the preliminary results (Table ) suggest they warrant further investigation. In future work we plan to evaluate how they evolve over time for an individual operator.

# Discussion

## Study Observations

One piece of information that is not reflected in our analysis is how people alter their *control strategy* when they are under different paradigms. For example, participants were observed *testing* the limits of the assistance. That is, users would intentionally operate the system at the boundary of safety, and even act in an adversarial manner to test the reliability of MPMI-SC. In contrast, under user-only control, participants were much more cautious. This is potentially a consequence of not explaining *how* the autonomous partner would provide assistance; however, we also believe this behavior aligns with human nature as people often *explore* while they learn.

Another observation relates to why MPPI-SC had a larger impact on the balance bot than on the race car. In particular, participants were more likely to have prior experience operating car-like systems then unstable machines like the balance bot, which could mean they were better able to provide safe control based solely on experience and intuition.

## Limitation in Prediction of Safety

As described in Section , the control signal sent to the robot is selected based on the user's desires at each instant and therefore does not require a model of the user. However, to compute the safety of a given control action, we implicitly embed a naïve model of the user that assumes the human partner will continue to apply nearly the same input over the time-horizon. The negative implication of this assumption is that we will occasionally reject user inputs that *seem* dangerous, but are not in reality if the user quickly adjusts their strategy. Therefore, there will be trajectories that the person would like to execute and are safe (by recovering at a later timestep) that MPMI-SC incorrectly rejects. Despite this limitation, it is possible that our iterative, receding horizon approach alleviates the impact on the user's acceptance by recomputing the set of dangerous actions at each timestep. We leave a deeper evaluation of this idea to future work.

# Conclusion

In conclusion, we described a shared control paradigm that enhances a human partner's ability to operate complex, dynamic machines by incorporating safety constraints without explicit knowledge of the user's long-term objective. Our approach relies on a simple representation of the joint system (i.e., the Koopman operator) which, in turn, means we can very quickly generate and evaluate the safety of a large number of potential trajectories through the parallelization capabilities of a GPU. Importantly, this representation can be learned from data and therefore generalizes to any pair of partners. Finally, our approach adheres to the minimal intervention principle to ensure that the human partner is allocated the majority of the decision making authority throughout the interaction.

We evaluated the efficacy of our Model Predictive Minimal Intervention Shared Control (MPMI-SC) paradigm with a human-subjects study consisting of 20 participants. The results demonstrated that our approach is able to improve the general safety of the joint system without *a priori* knowledge of the user's desires. Additionally, we found that the participants enjoyed the assistance provided by the autonomy (and reported low levels of frustration), a feature lacking in many shared control paradigms . We have released our code with an open-source license on GitHub at <https://github.com/asbroad/mpmi_shared_control>.

# Acknowledgments

This material is based upon work supported by the National Science Foundation under Grants CNS 1329891 & 1837515. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the aforementioned institutions. The authors would also like to thank Emily Bernstein for her help in the study design.

[^1]: Chosen to produce equally spaced samples in all 3 input dimensions.


author: F. Marin[^1]; M. Schartmann
bibliography: biblio.bib
date: Received May 18, 2017; Accepted August 21, 2017
title: Polarized radiative transfer modeling of warped
  and clumpy dusty tori

Active galactic nuclei (AGN) are anisotropic objects surrounded by an optically thick equatorial medium whose true geometry still defy observers. In this paper, we aim to explore the optical, scattering-induced, polarization emerging from clumpy and warped dusty tori to check whether they can fit with the unified model predictions. We run polarized radiative transfer simulations in a set of warped and non warped clumpy tori to explore the differences induced by distorted dust distributions. We then include warped tori in a more complex model representative of an AGN to check, using polarimetry and imaging methods, if warps can reproduce the expected polarization dichotomy between Seyfert-1 and Seyfert-2 AGN. The main results from our simulations highlight that isolated warped structures imprint the polarization degree and angle with distinctive signatures at Seyfert-1 orientations. Included in an AGN model, the signatures of warps are easily (but not always) washed out by multiple scattering in a clumpy environment. Imaging polarimetry may help to detect warped tori but we prove that warps can exist in AGN circumnuclear regions without contradicting observations. Two warped tori with a non significant difference in geometry in terms of photometry or spectroscopy can have totally different signatures in polarimetry. Testing the geometry of any alternative model to the usual dusty torus using polarized radiative transfer is a necessary approach to verify or reject a hypothesis.

# Introduction

It is now commonly accepted that the innermost regions of active galactic nuclei (AGN) are surrounded by a parsec-scale, optically thick, equatorial distribution of dust. found, in the polarized optical light of the Seyfert-2 galaxy NGC 1068, a Seyfert-1 spectrum with broad emission lines, indicating that a Seyfert-1 nucleus exists in all Seyfert-2 AGN. The observational characteristics of Seyfert-2s are in fact hidden from direct view by optically thick material. Radiation form the central engine is scattered onto the AGN polar outflows, resulting in the observed near-ultraviolet, optical and near-infrared polarization properties. The idea of a common origin for emission lines for Seyfert-1 and 2 galaxies, broad- and narrow-line radio galaxies, and quasars is supported by the work of . He found that H$\alpha$ and other optical emission lines appear to correlate over a range of 7 orders of magnitude with the continuum radiation. Subsequent recombinations that affect the emission lines might be uniform for almost all classes of AGN, but the presence of this hardly-resolved dusty component is responsible for the observed orientation-dependent properties . Fortunately, the outskirts of the equatorial dusty medium of AGN start to be revealed thanks to the highest possible spatial resolution offered by mid-infrared interferometry. spatially resolved a parsec-sized torus-shaped distribution of dust grains in the galaxy NGC 1068, where a small hot structure is embedded in a colder (320 K) dusty cocoon extending up to 3.4 pc in diameter. Further flux-limited interferometric studies of nearby quasars, using the MID-infrared Interferometric instrument (MIDI) at the Very Large Telescope Interferometer (VLTI; see, e.g., ) or the Keck interferometer , definitively confirmed the existence of the dusty torus albeit with large differences within the sample.

The morphology of this region is a poorly-constrained parameter of this multifaceted problem. The formation and hydrodynamic stability of a parsec-scale dust reservoir are not trivially explained by a homogeneous distribution of dust in a uniform toroidal arrangement. Detailed spatial studies revealed subtle problems with regards to smooth tori. Using Spitzer data, , , and investigated the 10 $\mu$m silicate feature in a large sample of AGN. They found that the occasional detection of the feature in emission in Seyfert-2 AGN, together with the absence of any deeply absorbed features in Seyfert-1s might rule out smooth density torus models. To suppress the emission feature clumpiness is invoked . It is also a simple and elegant way to allow for the dust to survive the expected local temperatures. postulated that the circumnuclear dusty material is likely clumpy, filled with a large number of individually optically thick clouds. Cloud merging and tidal shearing ensure a mixed environment that efficiently blocks radiation along the equatorial plane but allows a distant polar observer to see the central supermassive black hole (SMBH) through the dust-free funnel of the torus. Unfortunately, the exact geometry of the clumpy medium is beyond the resolution capabilities of current instruments. Infrared radiative transfer modeling has proven that multi-temperature, multi-phase clumpy distributions can reproduce most of the observed characteristics of nearby AGN . Similar results for the use of clumpy media instead of smooth density structures also emerged from polarized radiative transfer simulations . The near-infrared, optical and ultraviolet polarization signatures of a complex AGN model including a clumpy dusty torus were found to be in agreement with the predictions from the unified model .

If clumpiness is becoming more and more regarded as an important feature to add in (torus) simulations, there is another aspect of the unified model of Seyfert galaxies and quasars that is still barely investigated: the existence of warps. Using Very Long Baseline Array (VLBA) milliarcsecond-scale resolution observations of NGC 1068, revealed a slight misalignment between the line traced by the H$_2$O maser spots and the radio axis of the radio continuum source, believed to mark the location of the hidden nucleus. Very long baseline radio interferometry was also used to show that the accretion disk inside NGC 4258 may be a thin, subparsec-scale, differentially rotating warped disk .

Misaligned or warped structures have been detected in a variety of sources and can be imaged with great precision for disks around nearby stars, such as in the case of HD 142527. used polarized differential imaging to detect a warp in the outer disk of the HD 142527 system, thanks to the shadows cast by the inner disk. Secular perturbations of a planet can also affect the inclinations (i.e. orbital plane) of nearby planetesimals. Introducing a planet into the disk on an orbit inclined to the disk midplane causes a warp to propagate away from the planet . In the case of galaxies, and reported observational evidence of spin-induced disk/jet precession. This precession may lead to a warp of the innermost part of the AGN accretion disk with respect to the outer parts due to the Bardeen-Petterson effect . Warps can also be created at larger distances from the potential well by nonaxisymmetric magnetorotational instability as well as the Kelvin-Helmholtz instability . The presence of shear motion inside the torus works as an agent to drive Kelvin-Helmholtz instabilities, resulting in the generation of a strong wind and potential warps . Such warps would be difficult to detect in photometry, spectroscopy or imaging in galaxies that are not nearly edge-on , but polarimetry could be the answer.

The discovery of a polarimetric dichotomy between Seyfert-1 and Seyfert-2 was the most important proof for the establishment of the unified scheme . Optical polarization from Seyfert-1s is mostly $<$ 1% and parallel to the projected radio axis of the system (its polarization position angle $PPA$ is thus equal to 90$^\circ$), while Seyfert-2s are always showing a polarization degree $>$ 10% with polarization angle = 0$^\circ$ (perpendicular polarization). This dichotomy is due to the presence of the circumnuclear dust material that prevents direct radiation to escape from the equatorial plane in edge-on systems. Scattering inside the polar outflows results in higher polarization degrees $P$ and a perpendicular polarization, driven by the angle-dependence of Thomson scattering. This observational and theoretical model works very well for smooth density and clumpy tori but would warped dusty tori fit in this picture? Could they account for the high perpendicular polarization found in Seyfert-2s? And would they reproduce the observed polarization dichotomy? The latest papers on warped structures around the innermost regions of AGN do not answer these questions and it is the goal of our present work.

Accounting for both non-smooth density distributions and warps, we aim to test if warped media can affect the optical continuum polarization of a complex AGN model. We want to determine if warps and misaligned structures at the torus distances can fit the unified model, such as postulated by , or if caution must be taken when dealing with deformed tori. To do so, we construct a simple toy model for warped tori in Sect. , where we present the formalism and the Monte Carlo radiative transfer code used in our simulations. It is not our ambition to provide an accurate physical model for warped structures; this should be done together with hydrodynamical simulations . In Sect. , we test our clumpy and warped torus and compare the results to non-warped structures. We explore the space of parameters used to create the warp and show that warps have a profound impact on isolated tori. We include the warped structures in more complex AGN models in Sect. , taking into account polar outflows and equatorial inflows. We create imaging polarimetry maps of an AGN with a warped torus before exploring another model for a warped, uniform, dusty torus in Sect. . We discuss our results and further progress to be made in Sect.  before concluding in Sect. .

# Modeling warped and clumpy dusty tori

One of the most common geometries used to represent the circumnuclear dusty material in AGN is a torus (see the earliest representations in , where the outer boundary of the equatorial region was not drawn due to the lack of information on that point). Other morphologies, such as flared dusty disks, representative of outflowing material, are sometimes used in infrared studies (e.g. ) but give similar spectroscopic results to dusty tori[^2]. In this paper, we focus on a toroidal structure that will be warped at a given distance from the center of the model (where the SMBH lies).

## The clump distribution

We first consider a regular torus with an elliptical cross-section, ranging from an inner radius R$_{\rm in}$ = 0.1 pc to an outer radius R$_{\rm out}$ = 5 pc. The inner radius is fixed according to the dust sublimation radius for a $\sim$`<!-- -->`{=html}10$^7$ M$_\odot$ SMBH with sub-Eddington accretion rate. R$_{\rm sub}$ is correlated to the UV/optical continuum emission (L$_{\rm bol}$/L$_{\rm Edd}$ = 0.45, see ) and to the sublimation temperature T$_{\rm sub}$. The latter is fixed to 1500 K, a value that accounts for the temperature range of ambient gas pressures for both graphite and silicate grains . The outer torus radius corresponds to the typical extent of compact and optically thick tori in nearby AGN, ranging from 1 to 10 pc according to simulations and observations . The torus half-opening angle is fixed to 30$^\circ$ from the equatorial plane, a value chosen to agree with observed half-opening angles of the equatorial AGN region recently estimated by and . The toroidal structure is then filled with constant-density spheres with radius 0.11 pc . Each clump has an optical depth of 50 in the V-band, so a single cloud can efficiently obscure the central AGN engine in the optical band . The dust mixture has a composition similar to the Milky Way dust . Note that the choice of a Milky Way dust mixture instead of, e.g., a Small Magellanic Cloud like dust composition, where the dust is smaller than that in the Galactic diffuse ISM , does not change the outcomes of the simulations in the optical band as scattering in opaque dust clouds produces relatively grey scattering . We fix the number of clouds to 2000 in order to have a volume filling factor of 22–23% (the spherical clouds do not overlap, see also ). This model is the basis of all models we will explore in the forthcoming sections, and it will be studied in Sect.  as a test case.

To create warps in this structure, we define a radius R$_{\rm warp}$ at which the clumps will depart from the torus structure. When a clump is generated at a distance larger than R$_{\rm warp}$, its $z$-coordinate will be shifted upward or downward according to its position on the $xy$-plane:

$$z = \pm [z + (\sqrt{x^2+y^2}-R_{\rm warp})\tan(\theta)] \,,
  \label{eq1}$$

with

$$\theta = \frac{2\times\theta_{\rm warp}(\frac{\pi}{2}-\arctan(\frac{|y|}{|x|}))}{\pi}.
  \label{eq2}$$

$\theta_{\rm warp}$ is the maximum warping angle, i.e. the angle between the equatorial plane and the warped torus in the $xz$-plane. A torus warped through angle $\theta_{\rm warp}$ will intercept and reradiate a fraction close to $\theta_{\rm warp}$/3 of the luminosity of the central source . A visual representation of a warped torus with R$_{\rm warp}$ = 2 pc and $\theta_{\rm warp}$ = 30$^\circ$ is shown in Fig.. The top-left panel is a three-dimensional view of the model with 2000 clumps (clump sizes not to scale). The other three panels show the projection of the warped torus structure onto the $xz$, $yz$, and $xy$-planes. The distance at which the warping effect starts can be seen in Fig. (b), together with the warping angle $\theta_{\rm warp}$. On Fig. (c), one can discern a denser region close to the equator, which is the inner part of the torus that is unaffected by the warping effect. Finally, panel (d) is a view from the top and it is impossible to detect the warp.

This simple warping method has several advantages: the outer radius of the dusty torus is the same regardless of the azimuthal angle of the observer, so Seyfert-1s with low inclinations ($<$ 10$^\circ$) should not lead to excessively elongated dust lanes along any directions. Only the height of the structure is affected, resulting in possible polar dust signatures for intermediate and Seyfert-2s, way beyond the torus height defined by its half-opening angle . Polar mid-IR emission from this kind of warped dust distribution could contribute to the important polar infrared flux discovered by interferometry studies of Seyfert-2s with high position angle and baseline coverage . The antisymmetry between the upper and lower parts of the warped structure naturally arises even under these simple conditions. This could be due to a strong radiation field or an outflow dense enough to wipe the external surface of the circumnuclear dust, but they must sustain a small deflection with respect to the torus axis. This is the basic idea developed in , , and , see Fig. 5 and 6 in for illustrations.

There are other methods to warp tori. Recently, investigated uniform, geometrically-thin, warped dusty disks in the infrared band using a disk modified to have a concave shape at small radii which turns over at large radii. Their final structure is similar to what is being investigated here, with differences in the temperature profile and bulk shape of the disk at R $>$ R$_{\rm warp}$. Since we will explore the optical, scattering-induced, linear polarization of warped tori in this paper, the temperature profile has very little influence onto the scattered radiation. An important difference is that our modeling includes clumpiness, a critical aspect that would change the shape of the silicate feature and the spectral energy distribution in the infrared band ( and references therein). However, note that our model was not checked for consistency with the IR emission expected from Seyferts, e.g. the silicate feature strengths or interferometry results. Our goal is not to present a new warping method but to check whether warped tori can exist within the unified AGN picture, based on polarimetric observations.

## Radiative transfer with STOKES

Radiative transfer was achieved using the Monte Carlo code stokes, a simulation tool that was written to investigate the polarization signature of AGN in the optical and ultraviolet bands . The code was later improved to provide imaging analysis and clumpy reprocessing regions . A summary of the code performances can be found in . The three-dimensional capabilities of the code allow to explore an AGN model without any symmetries by virtually revolving around. For the remainder of this paper, we will investigate three different inclinations: 18$^\circ$ (Seyfert-1 view), 50$^\circ$ (intermediate inclination), and 87$^\circ$ (Seyfert-2 view).

The photon source used in the simulation is a point-like source of isotropic emission. The initial radiation is unpolarized and fixed at a monochromatic wavelength of 5000 Å(B/V-band). Around the central source are the reprocessing regions that can scatter, absorb or re-emit radiation. Mie and Thomson scattering are accounted for but the code does not handle temperature variation due to dust being heated or grain cooling by emission at infrared wavelengths. Re-emission from dust is thus not treated in theses simulations, which is a non-vital simplification as dust re-emits at much longer wavelengths than in the blue and visual bands. The four Stokes parameters of light are recorded but we will focus on linear polarization, since optical circular polarization from AGN was much less investigated by observers. As the Monte Carlo method employs a stochastic approach, the results obey Poisson statistics. We set the number of photons to be simulated in order to have small ($<$ 1%) statistical fluctuations per direction bin. Hence, each simulation sampled $\sim$ 10$^9$ photons for a computational time of 96 hours per model[^3].

## Test case: a clumpy dusty torus without warps

Before exploring the optical, scattering-induced, linear polarization from warped and clumpy dusty tori, the test model presented in Sect.  was investigated. This test case is a clumpy torus model without warp and is similar to what has been studied in . The effects of the clumpiness of the torus in a direct comparison with a continuous torus are already presented in and . The results are shown in Fig. : the top graph presents the polarization degree $P$ as a function of the azimuthal angle for three different inclinations $i$. The chaotic variations in $P$ are not due to insufficient Monte Carlo statistics. They are due to the clumpiness of the circumnuclear region: as the circumnuclear torus was randomly filled with clumps, each azimuthal angle sees the central source with a different covering factor. The photon trajectories and the absorption probabilities vary from two consecutive bins separated by an angle of 1 degree, resulting in the observed variations in $P$. The linear polarization is also inclination-dependent, with $P <$ 1% for Seyfert-1 inclinations and $P <$ 10% at Seyfert-2 inclinations. Compared to the pole-on and edge-on views, the intermediate inclination polarization curve appears smoother in azimuth dependence; this effect is due to a lesser influence of clumpiness onto the $i$ = 50$^\circ$ case. At the transition angle between Seyfert-1s and Seyfert-2s, back scattering onto the torus funnel is contributing the most to the polarization signal: in this case, the photon only "sees" a convex wall of dust and, if not absorbed, backscatters towards the observers, regardless of the existence of clumps behind the first cloud. An azimuthal-integration of $P$ gives the following average values: 0.04% at $i$ = 18$^\circ$, 0.19% at $i$ = 50$^\circ$, and 2.61% at $i$ = 87$^\circ$. Since the variations of the non-warped model are caused by the random clump distributions, we will use those integrated values for comparison purposes with our following models (see next section).

Similar conclusions can be drawn from the polarization position angle ($PPA$, bottom figure), which is mainly parallel to the symmetry axis of the model (hence yielding a value close to 90$^\circ$ when azimuthally-integrated). This is the averaged polarization angle already found in previous modeling with geometrically flat, clumpy, tori, see and Fig. 5 in . What is shown here and not in the previous publications is that the $PPA$ can rotate by 90$^\circ$ when the covering fraction of the source is insufficient at Seyfert-2 inclinations. This effect is strengthened by the fact that there is no diffuse inter-clump medium that would prevent or attenuate the propagation of photons in edge-on orientations. A second feature of interest for the remainder of the paper is that the $PPA$ can be offsetted by 0 to 30$^\circ$ from parallel polarization at a given phase, simply because of the random clump distribution.

The results from the test case seem to contradict the observations, where Seyfert-2s should have P $>$ 10% and PPA $\sim$ 0$^\circ$, but we remind the reader that it is only because the torus is isolated from the rest of the usual AGN components. We will see in the following sections that including polar winds will drastically change the picture.

# Optical polarization from clumpy and warped tori

## Results for a specific parametrization

In the previous section, we obtained the optical, scattering-induced, polarimetric signature of a clumpy toroidal structure and saw that chaotic variations in $P$ and $PPA$ naturally arise from the random distribution of dust clumps that only fill 22-23% of the total volume. We now add a warp that starts at a distance of 2 pc from the center of the model. The warp has an angle of 30$^\circ$ with respect to the equatorial plane, thus reaches a maximum height of 1.73 pc above the midplane (see Fig. b).

Compared to the test case, a warped clumpy torus (Fig. ) shows remarkable signatures in both $P$ and $PPA$ at a Seyfert-1 inclination. The $PPA$ presents sawtooth oscillations with sharp inversions around azimuthal angles -90$^\circ$ and +90$^\circ$, associated with local minima in $P$. The polarization angle modulation can be as high as 180$^\circ$ due to the maximum asymmetry seen by a distant observer[^4]. The bottom part of the warp being hidden by the equatorial dust content at $i$ = 18$^\circ$, only the side of the warp that faces the observer contributes to the net polarization, resulting in azimuthal-dependent $PPA$. This effect is much less pronounced at intermediate and equatorial inclinations: the inner parts of the clumpy structure efficiently block radiation and the specific sawtooth oscillations are smoothed out. Compared to the test model, the scatter in $PPA$ for the warped torus appears smaller at equatorial inclinations. Polarization mainly arises from backscattering from the torus funnel and the contribution of the warped and distant surfaces is negligible (as it will be confirmed by imaging polarimetry results, see Sect. ). At those inclinations, the averaged $P$ for a warped dusty torus is marginally higher than what has been found for a non-warped model. Even if $P$ shows more scatter around the mean for the warped model, we may estimate its averaged values: 0.06% at $i$ = 18$^\circ$, 0.34% at $i$ = 50$^\circ$, and 4.23% at $i$ = 87$^\circ$.

We further investigated the sharp variation in $PPA$ by looking at the model from a wider range of inclinations. In Fig. , the rotation of the polarization angle close to the minimum of the sawtooth ($\sim$ 90$^\circ$ phase, between points A and B in Fig. ) is shown in black. The code has difficulties to sample inclinations lower than 8$^\circ$, but it is clear that the largest $PPA$ variations occur at polar inclinations. The increasing importance of backscattering from the torus funnel at inclinations $i \ge$ 25$^\circ$ stabilizes the polarization angle at low polarization modulations; the $PPA$ values are consistent with clump-induced effects, as discussed in the last point in Sect. . The inclination at which the $PPA$ flattens is consistent with $\theta_{\rm warp}$ = 30$^\circ$, accounting for the effects of clumping. At $i$ = 60$^\circ$, torus obscuration starts but the gap between the clouds allows radiation to easily escape. At $i \ge$ 65$^\circ$, the $PPA$ starts again to oscillate due to the growing influence of equatorial obscuration. Only the fraction of the torus funnel that is opposite to the observer and the far end of the warped structure are scattering photon, but the resulting polarization angle is still dominated by multiple scattering effects. Thus, clumpiness has a much more important impact than warps on polarization at intermediate and edge-on orientations.

## Exploring the parameter space

There are two important parameters that drive the warp structure: the distance at which the warping effect starts (R$_{\rm warp}$) and the maximum angle between the equatorial plane and the warped structure ($\theta_{\rm warp}$). Both can be used as tracers of the initial non-axisymmetric phenomenon that leads to torus warping but the question is: can they be estimated from polarimetric measurements? In this subsection, we run a grid of warped tori varying R$_{\rm warp}$ and $\theta_{\rm warp}$, and we examine the resulting polarization as a function of azimuthal angle. The results are summarized in Fig.  and Fig.  for the polarization degree and the polarization angle, respectively.

The first and probably most important observation is that a warped dusty torus always drives 180$^\circ$ rotations of the polarization angle at Seyfert-1 orientations when revolving around the structure. The sawtooth polarization modulation in $PPA$ are associated with local minima in $P$, such as seen in Fig. , that can decrease $P$ by one order of magnitude in the most extreme case. The azimuthal angle at which the polarization modulation happens does not change gradually as a function of both R$_{\rm warp}$ and $\theta_{\rm warp}$; it cannot be easily predicted. The random sampling of the density distribution/clumpiness creates higher- and lower-density sub-regions, resulting in enhanced asymmetry, and partly offset from 90$^\circ$ in azimuthal angle the minima in $P$ and $PPA$. Regardless of the configuration of the warp, the sawtooth rotations of the $PPA$ is a clear and distinct signature of isolated warped tori. The averaged polarization degree for warped tori at Seyfert-1 inclinations ranges from 0.04% to 0.12%. These values are slightly higher than what was found for non-warped, non-clumpy torus models , as asymmetry is enhanced here. At other inclinations, $PPA$ flattens around 90$^\circ$ due to the randomization of the clump positions and multiple scattering that tend to hide potential $P$(R$_{\rm warp}$,$\theta_{\rm warp}$) or $PPA$(R$_{\rm warp}$,$\theta_{\rm warp}$) correlations. Variations in the covering fraction of the source at different azimuthal angles have a much deeper effect onto the continuum polarization than warps. The averaged polarization degree for warped tori at intermediate inclinations ranges from 0.13% to 0.62%; for edge-on systems, $P$ ranges from 3.41% to 5.61%, which is in agreement with past polarimetric modeling .

# Impact of warped dusty tori in a global AGN model

We can summarize our current results by stating that warps do not strongly affect the polarimetric signatures of clumpy dusty tori at intermediate and edge-on inclinations. The signatures of warps in the dust structure are washed out by the polarization emerging from reprocessing on the equatorial distribution of clumps. However, at Seyfert-1 inclinations, there is a tremendous impact of warps onto the polarization position angle. This 180$^\circ$ polarization modulation occurs in the case of isolated warped structures, but does this feature also impact a complex AGN model where inflowing material from the torus connects to the broad line region (BLR)? Where ionized polar outflows extend up to tens of parsecs? Moreover, can warped and clumpy dusty tori account for the high perpendicular polarization found in Seyfert-2s? It is the aim of the next section to answer those fundamental questions.

## Polarization from an AGN with a warped equatorial obscurer

To investigate a complex AGN model, we added to the previous warped and clumpy dusty tori two extra components: an equatorial, electron-dominated inflow and a polar, bi-conical structure representative of the ionized winds. The former region represents the accretion flow between the torus and the BLR, a necessary structure used to reproduce the observed polarization dichotomy in AGN . The equatorial region is a flared disk with half-opening angle 20$^\circ$ from the equatorial plane and models a radiation-supported disk structure. It spans from 0.03 to 0.05 pc and is filled with electrons, such that the V-band optical depth along the equator is 1 . This equatorial inflow should not be confused with the BLR, where most of the material is neutral (as evidenced by fast variations in the neutral absorbing column seen in X-ray observations, e.g. or for models). The second structure is an electron-filled, hourglass-shaped wind that is outflowing from 0.1 to 30 pc from the center of the model. The optical depth of the wind is fixed to 0.05, a representative value of the optically-thin polar regions observed in nearby Seyferts . Increasing the optical depth contributed by the narrow-line region to values higher than 0.05 (such as presented in Fig. 3 in , where a typical A$_{\rm V}$ of 1 – 2 mag is found) will not change the outcome of the simulation as long as the optical depth remains lower than unity. Otherwise multiple scattering in a Thomson-thick electron medium will increase the number of scatterings needed by radiation to escape the model along Seyfert-1 viewing angles, rotating the final polarization position angle by 90$^\circ$ (see for a grid of AGN polarized simulations with the polar winds V-band optical depth ranging from 0.01 to 3).

Our grid of results are presented in Fig.  and Fig.  for the polarization degree and the polarization position angle, respectively. Focusing on Seyfert-1 inclinations, we see that both $P$ and the $PPA$ are azimuthally-independent. The polarization degree resulting from scattering in a complex AGN model with a warped torus is of the order of 0.4%. Scattering off the radiation-supported, inner disk provides most of the polarized information and the amount of secondary or tertiary reprocessing onto the dusty equatorial structure is not sufficient to impact the net $P$, nor the $PPA$, which is fixed to 90$^\circ$ (parallel polarization). Electrons in the polar winds do not contribute to $P$ as the medium is optically-thin and forward scattering leads to small degrees of polarization. This type of signature is the same that was found by previous authors when modeling smooth-density or clumpy distributions of matter in complex AGN models . This is also the typical amount of polarization found in nearby, equatorial-scattering dominated Seyferts .

The case of an intermediate inclination is similar: the polarization position angle is almost always parallel to the symmetry axis of the model, with polarization degrees between 2 and 3%. However, in rare cases, the polarization angle rotates by $\sim$ 90$^\circ$, driving oscillations in the polarization degree (see, e.g., the case R$_{\rm warp}$ = 3 pc and $\theta_{\rm warp}$ = 30$^\circ$ in Fig.  and ). This phenomenon is due to the occultation of the central region by a single or a group of dusty clouds. Scattered radiation from the wind will then prevail and rotate the $PPA$, impacting $P$. The polarization degree first decreases when the $PPA$ rotation occurs, then reaches a maximum ($<$ 10%) then decreases again when the $PPA$ stabilizes again. Most of those $PPA$ variations occur at an azimuthal angle close to 0$^\circ$, when the upper part of the warp (with positive $z$ coordinates) can intercept the observer's viewing angle.

The case seems more complex for Seyfert-2 inclinations. If the polarization degree is indeed high due to Thomson scattering in the polar winds, leading to $PPA$ = 0$^\circ$ such as expected from Seyfert-2s , it may also show orthogonal rotations when the clump distribution is not dense enough to entirely cover the central region. Scattered radiation from the equatorial plane will dominate and rotate the polarization angle, decreasing the net polarization degree (the inverse phenomenon described for intermediate inclinations). However, variations in $P$ are not always related to $PPA$ oscillations: the warped structure, extending higher from the equatorial plane than a regular torus, may contribute to the global polarization by scattering photons that emerge from the dust funnel or from the radiation-supported disk. The mainly parallel polarization angle of those photons will mix with the perpendicular signature of photon scattered in the polar region and thus cancel $P$. The contribution of the radiation with parallel $PPA$ is weaker, thus $P$ does not completely cancel out, but might be significant enough to bring the polarization degree to a couple of percents. Coupled to the clumpiness effect and the partial covering of the nucleus, $P$ can be as low as 0.2%. This case is statistically very unlikely to happen, otherwise AGN with optical Seyfert-2 classifications but Seyfert-1 polarization degree and angle would have been observed yet .

## Imaging polarimetry

Revealing the polarization pattern around the hidden nuclei of Seyfert-2s was a major success when the polarimeter on-board of the *Hubble Space Telescope* was available and helped the community to better understand the morphology and composition of the resolvable structures surrounding the obscured SMBH. In particular, imaging polarimetry helped to localize the position of the hidden source of UV continuum, highlighted the shape of the ionized polar winds, detected dichroic signatures in linear polarization, estimated the amount of foreground extinction, examined the composition of the NLR and lead to refined AGN models by better constraining the distribution of the scattering medium (see , and more recently with the Spectro-Polarimetric High-contrast Exoplanet REsearch instrument (SPHERE) on the *Very Large Telescope*, VLT). Polarization induced by dust scattering onto the circumnuclear region has been undetectable so far and will stay impossible to resolve for many years due to the small amount of flux reprocessed by this region and due to the technical limitations of current facilities. As an example, under decent atmospheric conditions, a maximum of 60 milliarcsecond ($\sim$ 4 pc) resolution for imaging polarimetry is achievable on NGC 1068 ($d$ = 14.4 Mpc) in the H and K bands using adaptive optics . On the other hand, General Relativity Analysis via VLT InTerferometrY (GRAVITY) at the VLTI (in the K band) has an imaging resolution of about 2 mas and may be able to measure polarization in a few bright AGN thanks to its Wollaston prism . Yet, observations will be limited to very few sources and the polarized signal from the torus might be diluted by the emission from the cones. However, if the torus is warped and extended along the polar regions, it might be possible to detect some effects on the AGN polarization maps.

To investigate the impact of warped tori, we ran the AGN model presented in Sect.  using the following warp parametrization: R$_{\rm warp}$ = 2 pc and $\theta_{\rm warp}$ = 30$^\circ$. We created the polarization maps of the AGN at the same three viewing angles as shown in all the figures of this paper (18$^\circ$, 50$^\circ$ and 87$^\circ$), together with three azimuthal angles (0$^\circ$, 45$^\circ$ and 90$^\circ$). The polarized flux maps are presented in Fig. , the polarization fraction maps in Fig.  and the polarization position angle maps in Fig. . The spatial resolution of the images is 200$\times$`<!-- -->`{=html}200 pixels which, for a NGC 1068-like AGN distance would give a pixel size of 0.11$\times$`<!-- -->`{=html}0.11 pc (0.1 arcsec $\approx$ 11 $h_{50}^{-1}$ pc, see ). The (0,0) coordinates give the position of the central SMBH.

As we can see from imaging polarimetry (Fig. , and ), most of the polarized flux emerges from the circumnuclear dust funnel and propagates through the wind. Forward scattering inside the wind happens too rarely to impact the larger amount of flux that has scattered inside the inner dust radius or propagated into the torus then escaped along the polar direction. Equatorial scattering thus imposes its $PPA$, as it can been seen from Fig. . The net polarization degree is diluted by unpolarized radiation from the source, resulting in low values (Fig. ). The elongated shape of the spot of maximum flux is due to the inclination of the system in addition to backward scattering of photons from the torus to the observer. Thanks to the gaps between the clouds, photons can journey further into the dust medium than in the case of a smooth-density distribution but their polarization signature has no global impact . The warps are not easily detected from imaging polarimetry of Seyfert-1s. For intermediate inclinations, the line-of-sight of the observer is not obscured by the equatorial dust. The inner region shines and its polarization dominates over the polarization resulting from wind interactions. This is a visual confirmation of the results presented in Fig.  and . The warp can be distinguished in the maps (middle-right images) as an extension towards the left of the image. This extension is particularly visible in the polarization degree and polarization angle maps while, due to the small amount of scattered radiation, can be missed in the polarized flux maps. Without prior information, it is impossible to determine if this material is an isolated dust lane or if it is part of the dusty structure. The warp that is situated beneath the equatorial plane is invisible. It is at Seyfert-2 inclinations that the warp has a direct impact onto the polarization maps. At an azimuthal angle of 0$^\circ$, when the warp's maximum polar elongation faces the observer, an important fraction of the wind base is obscured and there is an asymmetry in the polarized flux detected from the northern AGN part with respect to the southern flux. This is not due to an orientation effect, the warp is directly impacting the observed distribution of fluxes, with the southern cone being much brighter than its northern counterpart. This naturally explains why most of the variations in $P$ and $PPA$ observed in Fig.  and occurs at an azimuthal angle of 0$^\circ$. Rotating around the AGN slowly uncovers the northern wind base and equilibrates the distribution of polarized fluxes. When the maximum asymmetry of the model is reached (azimuthal angle 90$^\circ$), both the upper and lower warps can be detected thanks to multiple scattering, but their relative importance is too weak to impact the integrated polarization properties (see the low polarization fractions on Fig. ). The ratio between the polar scattered flux and the flux from the warped torus surfaces tells us that warped tori might be difficult to detect even with current polarimetric observatories.

# Exploring the warp morphology

We have seen that warped dusty tori are acceptable within the unified model picture as long as they do not violate one of the most important rules of the paradigm: there must be a polarization dichotomy between Seyfert-1 and Seyfert-2 orientations. Due to the clumpiness of the circumnuclear structure and the parametrization of the warps, a limited number of exceptions can arise but the majority of the models have proven to pass the test when adding additional AGN components. Yet, the question of the importance of the warp morphology remains as the methodology to produce warps is not unique. To illustrate how a similar model can change the resulting polarization degree and angle, we took the dust distribution from and implemented it into stokes. We clumpy the structure such that its volume filling factor is the same as the models investigated so far. A sketch of the corresponding smooth-density distribution model is shown in Fig. .

Running our Monte Carlo code for an isolated, warped and clumpy dusty torus, we found several similarities and differences with respect to the baseline model presented in this paper. In Fig. , we present the polarization results for the Jud model (left hand side panels, in red) and for a warped torus (right hand side panels, in black) created with the formulas presented in this paper. We used R$_{\rm warp}$ = 0.5 pc and $\theta_{\rm warp}$ = 10$^\circ$ to be as close as possible to the former structure. It appears that the model from produces different amounts of polarization degree associated with sharp sawtooth variations of the $PPA$ when the observer is revolving around the structure. The polarization angle modulations happen at the same azimuthal angle, with minor variations due to the random distribution of clumps. However, the polarization degree in Jud models is always superior to 0.1%, while in the case of the warped tori discussed in the previous sections of this paper, $P$ was often lower than 0.1%. The sharp diminution in $P$ associated with the $PPA$ rotations are not seen here due to the puffed-up structure of the warped component that enhances the scattering probabilities. This peculiarity could be used to constrain the geometry of warped tori around point sources[^5]. In the cases of the intermediate and equatorial viewing angles, the observer's line-of-sight is passing through the bulk of the obscuring material but the deep $PPA$ polarization modulations still occur. Compared to the models of this paper, the geometry of the warped disk from presents high and rapid variations of the polarization degree and no stabilization of the polarization angle around 0$^\circ$.

Including the other two components (equatorial radiation-supported disk and ionization cones) to the warped torus allows us to check the polarization properties of an hypothetical AGN with a Jud's dusty disk (Fig. , left hand side panels, in red), compared to our warped toy-model (Fig. , right hand side panels, in black). The properties of the extra components are the same as before so differences in the AGN polarization signatures will reflect the impact of the warped structure. The most striking result of Fig.  is that the model violates the predictions from the unified model: for face-on and intermediate inclinations, the polarization position angle is neither parallel nor perpendicular to the projected polar axis of the system but has preserved its sawtooth oscillations despite the additional scattering components. The difference is due to the geometry of the torus: 1) the model from has a concave shape (e.g. a sphere cut out) whereas the warped torus model from this paper has a convex shape in the innermost region (an elliptical cross section, ), and 2) the width of the warped dusty medium from , extending towards the polar region, is twice as large as in the model from this paper. The change in morphology enhances the scattering probabilities for the Jud model and more scattered radiation bounces from the dust structure beyond R$_{\rm warp}$ to a polar observer. The polarization signatures from the warped torus are preserved and result in variable $PPA$. Together with the degree of polarization (1 to 2%), this is unfortunately in contradiction to observations. Those high degrees of $P$ are rare among Seyfert-1s and are most of the time associated with a perpendicular polarization angle (the so-called polar-scattering dominated AGN, see and ). According to this model, all pole-on AGN should present high degrees of polarization together with an apparent random distribution of $PPA$. The case is similar for intermediate inclinations but it is more aligned with the prediction of the AGN paradigm. Intermediate orientation are allowed to produce $P >$ 1% and the simulation shows that the polarization angle is plateauing at 0$^\circ$ or 90$^\circ$ according to the phase. For edge-on inclinations, the model is dominated by scattering inside the electron cone and is in agreement with polarimetric expectations of Seyferts ($P$ = 20–80% and perpendicular polarization angle, see ).

# Outlook

In this paper, we presented an optical investigation of the continuum polarization from warped tori. The time and resources required by the simulations to run polarized transfer in a clumpy medium are high enough to explain why such work was never undertaken previously. With the development of faster codes, a more complete view of the polarimetric signature of warped and clumpy media is in reach, especially extending the wavelength range of our work. In particular, two codes are perfectly suited in the infrared band. skirt, presented in , and , is a radiative transfer code that simulates the journey of photon packages in dusty astrophysical systems. Thanks to an adaptive grid, skirt handles any 3D geometry of the dust distribution and offers a full treatment of multiple anisotropic scattering, absorbing and (re-)emitting media, which makes it a perfect tool to extend the work accomplished in this paper. The polarization of radiation by spherical dust grains is included and soon should be released to the public (Peest et al, submitted). The second infrared tool, MontAGN, is dedicated to the production of high angular resolution polarization maps (Grosset et al., in prep). The code shares many common points with stokes and also includes re-emission by dust, computing temperature equilibrium adjustment at each absorption. MontAGN works by sending photon packets that travel in a user-defined grid of dusty cells, such as skirt, and overlaps the wavelengths band of stokes in the 0.8 – 1 $\mu$m regime. This allowed for a comparison between the codes , which gave satisfactory results. The use of a grid in both infrared codes makes the creation of a clumpy medium easy and computationally cheap. Looking at the infrared polarization, it might become easier to check whether a defined warped configuration can explain the observed infrared polarization of nearby Seyferts, intrinsically less polluted by starlight emission than in the optical band. The importance of re-emission and multiple scattering, depolarizing the escaping radiation, will probably be enhanced by clumpiness and may lead to different signatures in $PPA$ than in the V and B bands.

Another waveband of importance is the X-ray band. With the advance of modern polarimeters, either based on the photoelectric effect , gas-filled Time Projection Chambers , or Compton scattering , it will be soon possible to measure the X-ray polarization from nearby and bright AGN. The expected AGN polarization in the soft energy band (2 – 10 keV) should be of similar levels than in the optical band but due to Compton and Inverse-Compton scattering, the hard X-ray band might show a distinctive polarization. It is thus important to produce new simulations of AGN including clumpy tori based on contemporary observations . Preliminary results for smooth density warped disks were recently obtained by , who computed the X-ray polarimetric features induced by a warped disk around a stellar-mass black hole. Using the Bardeen-Petterson effect, the authors explored the possibility of identifying a warped disk with X-ray spectropolarimetric measurements of the thermal spectrum of the accretion disk. Among other findings, they showed that the warped structure can be identified by a transition in polarization in the 0.1 – 3 keV band. The larger the inclination between the inner and outer part of the warped accretion disk, the more prominent is the feature. By looking from a polar direction, it might be possible to detect this feature with a future mission but the surrounding of the accreting system must be taken into account first (polar outflows being among the most important regions to add). The detection of an X-ray polarization signal influenced by a warped accreting medium might even be enhanced by Lense–Thirring precession of the inner accretion flow. found that the polarization signature emitted from a truncated disk/precessing inner flow geometry in an X-ray binary oscillates at a specific frequency where quasi-periodic oscillations are detected. The maximum polarization degree due to the Lense–Thirring effect is of the order of 1.5 %, which is not negligible for a potential detection. The case for AGN remains to be explored but encouraging results are emerging for the detection and characterization of warped structures around compact objects.

# Conclusions

In this paper, we explored the optical, scattering-induced, continuum polarization emerging from clumpy and warped dusty tori. Significant features appear at Seyfert-1 inclinations, where the polarization position angle rotates by $\sim$ 180$^\circ$ in sawtooth oscillations, locally decreasing the degree of polarization. At intermediate and edge-on inclinations, both $P$ and the $PPA$ are similar to non-warped clumpy tori. Examining a large range of warp parameters, we found that the impact of clumpiness onto $P$ is overwhelming the signatures of warps and it is almost impossible to distinguish between two different warped structures.

Including a warped torus in a more complex AGN model, with equatorial inflows and polar outflows, we are able to recover the observed polarization properties of nearby Seyferts, together with a proper distinction in polarization angles between a Seyfert-1 and a Seyfert-2. This indicates that warped equatorial structures may exist in AGN, leaving very little imprints on the optical continuum. However, imaging polarimetry is able to reveal the warps in Seyfert-2s but the sensitivity, contrast and angular resolution needed to achieve a detection are beyond current observatories, despite the progresses made by direct imaging instruments equipped with extreme adaptive optics systems .

If warped equatorial tori may indeed exist in AGN and pass the polarimetric tests of the unified model, the exact geometry of the warped structure must be examined carefully. We have proven that not all geometries are able to reproduce the observed polarization dichotomy. In this case, any new model of dusty disk-born winds , warped disks or exotic self-gravitating tori must be investigated using polarimetry. Polarized radiative transfer simulations will then ensure that the structure is consistent with the most geometry-sensitive tool the unified model has to offer.

The authors would like to acknowledge the anonymous referee for her/his useful comments that improved the quality of the paper. The authors are also grateful to D. Gratadour and L. Grosset from LESIA; M. Stalevski from the University of Chile; R. W. Goosmann and B. Vollmer from the Observatory of Strasbourg; K. Tristram from the European Southern Observatory, L. Burtscher from the faculty of Sterrewacht Leiden; and R. Antonucci from the University of California for useful discussions and comments that greatly helped to ameliorate this manuscript.

[^1]:

[^2]: For a polarimetric investigation of the differences between a clumpy flared disk and a clumpy torus, see .

[^3]: Radiative transfer simulations are time-consuming, especially when clumpy media are involved. After emission or a scattering event, stokes checks if a clump is along its direction. As there are thousands of clumps in the model, the checking procedures drastically increases the duration of the simulations.

[^4]: The polarization position angle is periodic. Any polarization ellipse is indistinguishable from one rotated by 180$^\circ$, which is one of the fundamental advantages of radio antenna polarimetry. The sawtooth pattern is caused by this periodicity; an observer is expected to see a continuous increase of the PPA with a polarization modulation.

[^5]: Such as warps in proto-planetary and dusty debris disks .