Ranking and Unranking Restricted Permutations

Abstract: We discuss efficient methods for unranking derangements and ménage permutations. That is, we will provide an algorithm to efficiently extract the $k$-th earliest such permutation under the lexicographic ordering. We will show that this problem can be reduced to the problem of computing the number of restricted permutations with a given prefix, and then we will use rook theory to solve this counting problem. This has applications to combinatorics, probability, statistics, and modeling.

\section{Overview and preliminaries}
In January 2020, Richard Arratia sent out an email announcing a talk he was
going to give about what came to be named ``unranking derangements''.
Arratia noted that it is straightfoward to enumerate derangements
(i.e. permutations without fixed points) in both senses
of the word: both counting the number of derangements on $n$ letters and listing
them out one at a time.
He asked, however, what if we want to find the element at a particular index
somewhere in the middle of a lexicographically sorted list?
Can one do this efficiently---in particular, can this be done without writing out
the entire list?
This suggests a broader combinatorial question: when is it possible to
find the element at a particular index of a totally ordered set in some
computationally efficient manner?

To be explicit about what we wish to compute efficiently, we define the notion
of a \textit{unranking}.
\begin{definition}
  Let $\mathcal C$ be a totally ordered finite set, and let
  $\{c_i\}_{i=1}^{|\mathcal C|}$ be the unique sequence of elements in
  $\mathcal{C}$ such that $c_i < c_{i+1}$ for all $1 \leq i < |\mathcal{C}|$.
  Then a \textbf{unranking} is a map
  \[
    \operatorname{unrank}_{\mathcal{C}} \colon \{1, 2, \dots, |\mathcal C|\} \to \mathcal{C}
  \] that sends $i \mapsto c_i$.
\end{definition}

An efficient way to unrank a collection of objects implies an efficient
algorithm for sampling from the collection uniformly at random. This can
potentially be of use in the case of Monte Carlo simulations and other
instances where it is useful to be able to sample uniformly from a collection
of combinatorial objects.

As the name suggests, every unranking problem comes with a dual ranking
problem.
\begin{definition}
  A \textbf{ranking} of a totally ordered finite set $\mathcal C$ is
  a map
  \[
    \operatorname{rank}_{\mathcal{C}} \colon \mathcal{C} \to \{1, 2, \dots, |\mathcal C|\}
  \] that sends $c_i \mapsto i$.
\end{definition}

The existence of both efficient ranking and unranking maps implies the existence
of an efficient encoding for these objects, which may be of interest to computer
scientists. The encoding works by ranking an object to get its index, which then
can be stored in as a positive integer and unranked on retrieval to recover the
original object.

In January 2021, Richard Arratia announced a \$100 prize for developing an
implementing an efficient algorithm for computing the unranking map in
the context of \textit{m\'enage permutations}. In particular, he stated the
problem as follows:
\begin{problem}
  For $n=20$ there are $A000179(20) = 312\,400\,218\,671\,253\,762 > 3.1\cdot 10^{17}$
  m\'enage permutations \cite{oeis}.
  Determine the $10^{17}$-th such permutation when listed in lexicographic order.
\end{problem}

In the remaining sections, we are show how to resolve all of the above problems
in order to claim Richard Arratia's \$100 prize.
That is, we will construct an algorithm for ranking and unranking both
derangements and m\'enage permutations under the lexicographic ordering.
We will show that the existence of an efficient way to count
the number of such permutations with a given prefix implies that there is an
efficient way to compute the ranking and unranking maps.
Then we will develop some ideas from rook theory and apply them to the context
of derangements and m\'enage permutations.

\section{Prefix counting and word ranking}

In both the case of unranking derangements and menage permutations
(and in many other applications) our combinatorial objects are
words in lexicographic order, which is a generalization of alphabetical order.

We begin by developing a general theory for unranking collections of words in
lexicographic order by counting the number of words with a given prefix.
\subsection{Words with a given prefix}

We will start by introducing some basic definitions about words and prefixes,
and to formalize the notion of lexicographic order.

\begin{definition}
  A finite \textbf{word} $w$ over an alphabet $\mathcal A$ is a finite sequence
  $\{w_i \in \mathcal A\}_{i=1}^N$.

  The collection of finite words over the alphabet $\mathcal A$ is denoted by
  $\pazocal{W}_\mathcal{A}$, or just $\pazocal{W}$ when the alphabet is
  implicit from context.
\end{definition}

\begin{definition}
  A word $w = \{w_i \in \mathcal A\}_{i=1}^N$ is said to begin with a
  \textbf{prefix} $\alpha = \{\alpha_i \in \mathcal A\}_{i=1}^M$ if
  $M \leq N$ and $w_i = \alpha_i$ for all $i \leq M$.
\end{definition}

\begin{definition}
  A word $w$ is said to be before $w'$ in \textbf{lexicographic order}
  if either $w$ is a proper prefix of $w'$, or if at the first position, $i$,
  where $w$ and $w'$ differ, $w_i < w'_i$.
  \label{def:lexicographicOrder}
\end{definition}

With these definitions established, we can turn the problem of unranking words
into a problem about counting words with specified prefixes.

\begin{theorem}
  For $k > 0$, let $\pazocal{W}$ be a set of nonempty words on the alphabet
  $[n]$, and let $\mathcal C \subsetneq \pazocal{W}$ be a finite subset of words
  on this alphabet, with a total order given by its lexicographic order.

  Let
  $\#\operatorname{prefix}_{\mathcal C}\colon \pazocal{W} \rightarrow \mathcal{C}$
  be the function that counts the number of words in $\mathcal C$ that begin
  with a given prefix.

  Then the unranking function can be computed recursively by \begin{equation}
    \operatorname{unrank}_\mathcal{C}(i) = f^{\mathcal C}_i((1), 0)
  \end{equation} where
  \begin{numcases}{f^{\mathcal C}_i(\alpha, j) =}
    f^{\mathcal C}_i(\alpha', j + \#\operatorname{prefix}_\mathcal{C}(\alpha))
    & $i > j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$
  \label{case:unrankIncrement}
  \\
  f^{\mathcal C}_i(\alpha'', j)
    & $\alpha \not\in \mathcal{C}$ and $i \leq j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$
  \label{case:unrankAppend}
  \\
  f^{\mathcal C}_i(\alpha'', j + 1)
    & $\alpha \in \mathcal{C}$, $i \leq j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$, and $i \neq j + 1$
  \label{case:unrankPrefix}
  \\
  \alpha
    & $\alpha \in \mathcal{C}$ and $i = j + 1$,
  \label{case:unrankFinish}
  \end{numcases}
where
$\alpha = (\alpha_1, \alpha_2, \dots, \alpha_\ell)$,
$\alpha' = (\alpha_1, \alpha_2, \dots, \alpha_{\ell-1}, 1 + \alpha_\ell)$,
$\alpha'' = (\alpha_1, \alpha_2, \dots, \alpha_\ell, 1)$,
and $j$ denotes the number of words in $\mathcal{C}$ that occur strictly
before $\alpha$.
\label{theorem:unrankFromPrefix}
\end{theorem}

\begin{proof}
  We make four claims that we will prove using
  induction on the recursive applications of $f_i^\mathcal{C}(\alpha, j)$:
  that $j$ is the number of words in $\mathcal{C}$ that occur strictly before $\alpha$,
  that $\alpha \leq w_i$,
  and that the sequence of $\alpha$s is strictly increasing.

  This final claim (that the sequence of $\alpha$s is strictly increasing)
  follows from the observation that ${\alpha < \alpha'' < \alpha'}$ in
  lexicographic order.

  Because each iteration increases either $j$ or $\ell$ or both,
  the number of recursive applications of $f_i^\mathcal{C}$ required to
  determine $w_i$ is at most $i + \max_{w \in \pazocal W} |w|$, which is
  finite because $\pazocal W$ only contains of finite words.

  The base case is clear: We start with $f_i^\mathcal{C}((1), 0)$ because
  $(1)$ is the lexicographically earliest word, so $0$ nonempty words strictly
  precede it, and $(1) \leq w_i$.

  We will repeatedly use the observation that if $j$ words precede $\alpha$,
  then a word has prefix $\alpha$ if and only if its index is in
  $(j, j + \#\operatorname{prefix}_\mathcal{C}(\alpha)]$. Note that this
  range is empty whenever there are no words prefixed by $\alpha$.

  \textit{Case \eqref{case:unrankIncrement}.}
  Because $j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$ is the index
  of the last word that begins with $\alpha$,
  if $i > j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$, then $w_i$ must
  begin with a length-$\ell$ prefix that is lexicographically later than $\alpha$.

  By construction, $\alpha'$ is the lexicographically earliest word of length
  $\ell$ that comes after $\alpha$, therefore $\alpha' \leq w_i$.
  As such, the number of words that strictly precede
  $\alpha'$ is ${j + \#\operatorname{prefix}_\mathcal{C}(\alpha)}$,
  which is the sum of the number of words that occur strictly before $\alpha$
  and the number of words that have prefix $\alpha$.

  \textit{Case \eqref{case:unrankAppend}.}
  If $\alpha \not\in \mathcal C$ and $i \leq j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$,
  then $\alpha$ is a \textit{proper} prefix of $w_i$, and $w_i$ is of length at
  least $\ell + 1$.
  By construction, $\alpha''$ is the lexicographically earliest word of length
  $\ell + 1$ that has a prefix of $\alpha$,
  so $\alpha'' \leq w_i$,
  and the number of words in $\mathcal{C}$ that precede
  $\alpha''$ is equal to $j$, the number of words that precede $\alpha$.

  \textit{Case \eqref{case:unrankPrefix}.}
  If $\alpha \in \mathcal C$,
  $i \leq j + \#\operatorname{prefix}_\mathcal{C}(\alpha)$, and
  $i \neq j + 1$ then $\alpha$ must be the word at index $j + 1 < i$,
  because $\alpha$ itself is the lexicographically earliest word with the prefix
  $\alpha$. Because words cannot appear multiple times in $\mathcal{C}$, $w_i$
  must have $\alpha$ as a \textit{proper} prefix.
  Therefore $\alpha'' \leq w_i$ and
  the number of words that strictly precede $\alpha''$ is $j + 1$:
  the number of words that strictly precede $\alpha$ plus $\alpha$ itself.

  \textit{Case \eqref{case:unrankFinish}.}
  If $\alpha \in \mathcal{C}$ and $i = j + 1$, then
  $w_i = \alpha$ because $\alpha$ itself is the lexicographically earliest word
  with the prefix $\alpha$, so it must occur at index $i = j + 1$.
\end{proof}

Notice that each recursive call of $f_i^\mathcal{C}$ increases the sum of the
letters of $\alpha$. If we suppose that
$\mathcal{C}^{(\leq k)} \subsetneq \pazocal{W}_{[n]}$
is a finite set of words on the alphabet $[n]$ of length at most $k$, then unranking
a word from $\mathcal{C}^{(\leq k)}$ requires at most $nk$ recursive
applications of $f_i^{\mathcal{C}^{(\leq k)}}$.

Therefore
if there exists a polynomial time algorithm for computing
$\#\operatorname{prefix}_{\mathcal{C}}$, then there exists an unranking
algorithm that is polynomial in the size of the alphabet $\mathcal A$ and the
length of the longest word. In the case of restricted permutations, each of
these grow linearly with the number of letters in the m\'enage permutations.

\subsection{Ranking words}
Just as we can recursively find a word at a given index, we can also
recursively find a index corresponding to a given word.

\begin{claim} If $\mathcal{C}$ is a collection of words such that no word is a
  prefix of another, then the rank of the word $w \in \mathcal{C}$ can
  be computed as the sum \[
    1 + \sum_{i=1}^{|w|} \sum_{i = 1}^{w_i - 1} \#\operatorname{prefix}_\mathcal{C}(w_{(i)}^x)
  \]
  where $w = (w_1, w_2, \dots, w_{|w|})$ and $w_{(i)}^x = (w_1, w_2, \dots, w_{i-1}, x)$.
\end{claim}

\section{Basic notions of rook theory}
Now that we have shown that we can unrank words whenever we can compute the
number of words with a given prefix, we want to develop techniques for this new
counting problem.
In the case of unranking derangements and permutations, it is useful to use
ideas from rook theory, which provides a theory for understanding
position-restricted permutations.
Rook theory was introduced by Kaplansky and Riordan \cite{Kaplansky1946}
in their 1946 paper \textit{The Problem of the Rooks and its Applications}. In
it, they discuss problems of restricted permutations in the language of rooks
placed on a chessboard.

\input{assets/fig_permutationFromRooks.tex}

\subsection{Definitions in rook theory}
We begin by introducing some preliminary ideas from rook theory.

\begin{definition}
  A \textbf{board} $B$ is a subset of $[n] \times [n]$ which represents the
  squares of an $n \times n$ chessboard that rooks are allowed to be placed on.
  Every board $B$ has a \textit{complementary board}
  $B^c = ([n] \times [n]) \setminus B$, which consists of all of the
  squares of $B$ that a rook cannot be placed on.
\end{definition}

To each board, we can associate a generating polynomial that keeps track of the
number of ways to place a given number of rooks on the squares of $B$ in such a
way that no two rooks are in the same row or column.

\begin{definition}
  The \textbf{rook polynomial} associated with a board $B$,
  \begin{equation}
    p_B(x) = r_0 + r_1 x + r_2 x^2 + \dots + r_n x^n,
  \end{equation}
  is a generating polynomial where $r_k$ denotes the number of $k$-element subsets
  of $B$ such that no two elements share an $x$-coordinate or a $y$-coordinate.
\end{definition}

In the context of permutations, we're typically interested in $r_n$, the number
of ways to place $n$ rooks on a restricted $n \times n$ board.
However, it turns out that a naive application of the techniques from
rook theory does not immediately allow us to count the number of
restricted permutations with a given prefix.
Computing the number of such permutations is known to be \#P-hard
for a board with arbitrary restrictions.
We can see this by encoding a board $B$ as a $(0,1)$-matrix and computing the matrix
permanent, since there is a bijection between boards and $(0,1)$-matrices.
(In fact, Shevelev \cite{Shevelev1992} claims that
``the theory of enumerating the permutations with restricted positions
stimulated the development of the theory of the permanent.'')

\begin{lemma}
  Let $M_B = \{a_{ij}\}$ be an $n \times n$ matrix where \begin{equation}
    a_{ij} = \begin{cases}
      1 & (i,j) \in B \\
      0 & (i,j) \not\in B
    \end{cases}.
  \end{equation}
  Then the coefficient of $x^n$ in $p_B(x)$ is given by the matrix permanent
  \begin{equation}
    \operatorname{perm}(M_B) = \sum_{\sigma \in S_n} \prod_{i=1}^n a_{i\sigma(i)}.
  \end{equation}
\end{lemma}

Now is an appropriate time to recall Valiant's Theorem.

\begin{theorem}[Valiant's Theorem \cite{Valiant1979}]
  The counting problem of computing the permanent of a (0,1)-matrix is \#P-complete.
\end{theorem}

\begin{corollary}
  Computing the number of rook placements on an arbitrary $n \times n$ board is
  \#P-hard.
\end{corollary}

Therefore, in order to compute the number of permutations, we must exploit some
additional structure of the restrictions.

\subsection{Techniques of rook theory}
Rook polynomials can be computed recursively. The base case is that
for an empty board $B = \emptyset$, the corresponding rook polynomial is
$p_\emptyset(x) = 1$, because there is one way to place no rooks, and no way
to place one or more rooks.
\begin{lemma}[\cite{Riordan1980}]
  Given a board $B$ and a square $(x,y) \in B$, we can define
  two resulting boards from including or excluding the given square:
  \begin{align}
    B_i &= \{(x',y') \in B : x \neq x' \text{ and } y \neq y'\} \\
    B_e &= B \setminus {(x,y)}.
  \end{align}
  Then we can write the rook polynomial for $B$ in terms of this decomposition.
  \begin{equation}
    p_B(x) = xp_{B_i}(x) + p_{B_e}(x).
  \end{equation}
  \label{lemma:rookPolynomialRecursion}
\end{lemma}

If we want to compute a rook polynomial using this construction, we can end
up adding up lots of smaller rook polynomials---a number that is exponential in
the size of $B$.
When the number of squares that are missing from $B$ is small,
it can be easier to compute the rook polynomial of the complementary board,
$p_{B^c}$, and use the principle of
inclusion/exclusion on its coefficients to determine the rook polynomial for
the original board, $B$.

In the case of derangements and m\'enage permutations, this is the strategy
we'll use.
We will start by finding the resulting board from a given prefix,
find the rook polynomial of the complementary board, and
use the principle of inclusion/exclusion to determine the number of ways to
place rooks in the resulting board.

\section{Unranking derangements}

\subsection{The answer to a \$100 question about derangements}
In January 2020, Richard Arratia sent out an email proposing a seminar talk.
The title describes his first ``\$100 problem'':
\begin{problem}
``For $100$ dollars, what is the $500$ quadrillion-th derangement on $n=20$?''
\end{problem}

\begin{answer}
The author's computer program was able to compute the answer in
less than twenty milliseconds. When written as words in lexicographic order, the
derangement in $S_{20}$ with rank $5 \times 10^{17}$ is \begin{equation}
  12\ 14\ 2\ 9\ 13\ 20\ 6\ 3\ 1\ 17\ 5\ 11\ 19\ 15\ 10\ 18\ 8\ 7\ 4\ 16.
\end{equation}
\end{answer}

\subsection{Overview for unranking derangements}
Arratia's question focused on unranking derangements written as words in
lexicographic order.
Other authors have looked at unranking derangements based on other total
orderings. In particular, Mikawa and Tanaka \cite{Mikawa2014} give an algorithm
to rank/unrank derangements
with respect to \textit{lexicographic ordering in cycle notation}.

In this section we will develop an algorithm for ranking and unranking
derangements with respect to their lexicographic ordering as words. The
technique that we use will broadly be re-used in the next section.
It is worthwhile to begin by recalling the definition of a derangement.
\begin{definition}
  A \textbf{derangement} is a permutation $\pi \in S_n$ such that $\pi$ has no
  fixed points. That is, the set of derangements on $n$ letters is \begin{equation}
    \mathcal{D}_n = \{\pi \in S_n : \pi(i) \neq i\ \forall i \in [n]\}.
  \end{equation}
\end{definition}

\subsection{The complementary board}
In order to compute the number of derangements with a given prefix, it is
useful to look at the board that results after placing $\ell$ rooks according to
these positions, as illustrated in Figure \ref{fig:derangementPrefix}.

\input{assets/fig_derangementPrefix.tex}

\begin{definition}
  If $B$ is an $n \times n$ board, and
  $\alpha = (\alpha_1, \alpha_2, \dots, \alpha_\ell)$ is a valid prefix of length
  $\ell$, the \textbf{derived board} of $B$ from $\alpha$,
  denoted $B_\alpha$,
  is constructed by removing
  rows $1, 2, \dots, \ell$ and
  columns $\alpha_1, \alpha_2, \dots, \alpha_\ell$ from $B$,
  reindexing in such a way that both the row and column indexes are in
  $[n - \ell]$.

  The \textbf{derived complementary board} $B_\alpha^c$ is the complement of
  $B_\alpha$ with respect to $[n - \ell] \times [n - \ell]$.
\end{definition}

Given a prefix of length $\ell$, the number of ways of placing $n - \ell$ rooks
on the derived board $B_\alpha$ is, by construction,
equal to the number of words in $\mathcal{C}$ with prefix $\alpha$
\begin{lemma}
  Given a valid $\ell$-letter prefix $(\alpha_1, \alpha_2, \dots, \alpha_\ell)$
  of a word on $n$ letters,
  the number of squares in the derived complementary board is \begin{equation}
    |B_\alpha^c| = n - \ell - |\{\ell+1, \ell+2, \dots, n\} \cap \{\alpha_1, \alpha_2, \dots, \alpha_\ell\}|,
  \end{equation} and no two of these squares are in the same row or column.
  \label{lemma:derangementComplementSize}
\end{lemma}
\begin{proof}
  Notice that the derived complementary board can be constructed in a different
  order: by first taking the complement, then deleting rows and columns, and
  finally reindexing the squares.
  Because the complementary board has no two squares in the same row or column,
  deleting and reindexing results in a derived complementary board with the same
  property.

  Thus, we only need to classify which squares in the complementary board are
  deleted to make the derived complementary board.
  We start by deleting $\ell$ squares corresponding to the deletion of the first
  $\ell$ rows, namely $(1, 1), (2, 2), \dots, (\ell, \ell)$.

  Some of these squares may also be in columns
  $\alpha_1, \alpha_2, \dots, \alpha_\ell$, but to avoid double-counting, we
  only consider those letters that are greater than $\ell$. These are
  $|\{\ell+1, \ell+2, \dots, n\} \cap \{\alpha_1, \alpha_2, \dots, \alpha_\ell\}|$,
  as desired.
\end{proof}

\subsection{Derangements with a given prefix}
Now that we have a way of quickly computing $|B_\alpha^c|$, we can compute the
number of ways to place a given number of rooks on the complementary board.
We can use this
to compute the rook polynomial for the derived complementary board
$p_{B_\alpha^c}(x)$.
We will see later that we can use the coefficients of this polynomial to compute
the number of ways of placing $n - \ell$ rooks on the derived board $B_\alpha$.

\begin{lemma}
  The rook polynomial for the complementary board $B_\alpha^c$ is \begin{equation}
    p_{B_\alpha^c}(x) = \sum_{j = 0}^{|B_\alpha^c|} \binom{|B_\alpha^c|}{j}x^j.
  \end{equation}
\end{lemma}
\begin{proof}
  Recall that no two squares of $B^c_\alpha$ are in the same row or column.
  Thus the number of ways to place $j$ rooks is equivalent to selecting any
  $j$ squares from the collection of $|B^c_\alpha|$ squares.

  Therefore the coefficient of $x^j$ in the rook polynomial is
  $\binom{|B_\alpha^c|}{j}$.
\end{proof}

Now we introduce a lemma of Stanley \cite{Stanley2011EC1} to compute the number
of ways of placing $n - \ell$ rooks in the derived board
$B_\alpha \subseteq [n - \ell] \times [n - \ell]$.

\begin{lemma}[\cite{Stanley2011EC1}]
  Let $B \subseteq [n] \times [n]$ be a board with complementary board $B^c$,
  and denote the rook polynomial of $B^c$ by
  $P_{B^c}(x) = \sum_{k=0}^n r^c_k x^k$.

  Then the number of ways, $N_0$, of placing $n$ nonattacking rooks on $B$
  is given by the principle of inclusion/exclusion
  \begin{equation}
    N_0 = \sum_{k=0}^n (-1)^k r^c_k (n-k)!.
  \end{equation}
  \label{lemma:CountsFromComplementaryPolynomials}
\end{lemma}

This lemma allows us to compute the number of rook placements on the derived
board $B_\alpha$, which is the number of derangements in $\mathcal{D}$ that
begin with the prefix $\alpha$.
\begin{corollary}
  The number of derangements with prefix
  $\alpha = (\alpha_1, \alpha_2, \dots, \alpha_\ell)$
  is given by \begin{equation}
    \#\operatorname{prefix}_\mathcal{D}(\alpha)
    = \sum_{j=0}^{|B_\alpha^c|} (-1)^j \binom{|B_\alpha^c|}{j}(n-\ell-j)!,
  \end{equation} which is $\operatorname{A047920}(n-\ell, |B_\alpha^c|)$ in
  the On-Line Encyclopedia of Integer Sequences \cite{oeis}.
\label{cor:derangementsWithPrefix}
\end{corollary}

Because we can compute $|B_\alpha^c|$ from $\alpha$ in linear time
(see Lemma \ref{lemma:derangementComplementSize}), if we use a computational
model where factorials are given by an oracle and arithmetic can be computed
in constant time, then $\#\operatorname{prefix}_\mathcal{D}$ can be computed
in linear time with respect to $\ell$, the length of the prefix.

\begin{example}
  For $n = 12$, we wish to count the number of derangements that
  start with the prefix $\alpha = (6,1)$, as illustrated in Figure
  \ref{fig:derangementPrefix}.
  Since the prefix has two letters, $\ell = 2$ and $n - \ell = 12 - 2 = 10$.
  The number of squares in $B_\alpha^c$ is \begin{equation}
    |B_\alpha^c| = 12 - 2 - \underbrace{|\{3,4,\dots,12\} \cap \{6, 1\}|}_1 = 9.
  \end{equation}
  Thus there are $A047920(10,9) = 1\,468\,457$ derangements in $S_{12}$
  that start with the prefix $\alpha = (6,1)$.
\end{example}

Now that we have an efficient algorithm for computing
$\#\operatorname{prefix}_\mathcal{D} \colon \pazocal{W} \rightarrow \mathbb{N}_{\geq 0}$,
we can invoke the recursive formula in Theorem \ref{theorem:unrankFromPrefix} to
compute
$\operatorname{unrank}_\mathcal{D} \colon \mathbb{N}_{\geq 0} \rightarrow \mathcal{D}$
and unrank derangements.
The sequence of recursive steps is illustrated in
Table \ref{table:unrankDerangement}.
\begin{table}
\center
\begin{tabular}{|l|r|l|c|l|}
  \hline
  $\alpha$ (prefix)
    & $\operatorname{\#prefix}_\mathcal{D}(\alpha)$
    & index range
    & $|B_\alpha^c|$
    & $\operatorname{unrank}_\mathcal{D}(1000)$
  \\ \hline
  $1       $ & $0$    & $(0,0]$           & $-$ & $f^{\mathcal{D}}_{1000}(1, 0)$          \\
  $2       $ & $2119$ & $(0,2119]$        & $6$ & $f^{\mathcal{D}}_{1000}(2, 0)$          \\
  \hline
  $21      $ & $265$  & $(0, 265]$        & $6$ & $f^{\mathcal{D}}_{1000}(21, 0)$         \\
  $22      $ & $0$    & $(265, 265]$      & $-$ & $f^{\mathcal{D}}_{1000}(22, 265)$       \\
  $23      $ & $309$  & $(265, 574]$      & $5$ & $f^{\mathcal{D}}_{1000}(23, 265)$       \\
  $24      $ & $309$  & $(574, 883]$      & $5$ & $f^{\mathcal{D}}_{1000}(24, 574)$       \\
  $25      $ & $309$  & $(883, 1192]$     & $5$ & $f^{\mathcal{D}}_{1000}(25, 883)$       \\
  \hline
  $251     $ & $53$   & $(883, 936]$      & $4$ & $f^{\mathcal{D}}_{1000}(251, 883)$      \\
  $253     $ & $0$    & $(936, 936]$      & $-$ & $f^{\mathcal{D}}_{1000}(253, 936)$      \\
  $254     $ & $64$   & $(936, 1000]$     & $3$ & $f^{\mathcal{D}}_{1000}(254, 936)$      \\
  \hline
  $2541    $ & $11$   & $(936, 947]$      & $3$ & $f^{\mathcal{D}}_{1000}(2541, 936)$     \\
  $2543    $ & $11$   & $(947, 958]$      & $3$ & $f^{\mathcal{D}}_{1000}(2543, 947)$     \\
  $2546    $ & $14$   & $(958, 972]$      & $2$ & $f^{\mathcal{D}}_{1000}(2546, 958)$     \\
  $2547    $ & $14$   & $(972, 986]$      & $2$ & $f^{\mathcal{D}}_{1000}(2547, 972)$     \\
  $2548    $ & $14$   & $(986, 1000]$     & $2$ & $f^{\mathcal{D}}_{1000}(2548, 986)$     \\
  \hline
  $25481   $ & $3$    & $(986, 989]$      & $2$ & $f^{\mathcal{D}}_{1000}(25481, 986)$    \\
  $25483   $ & $3$    & $(989, 992]$      & $2$ & $f^{\mathcal{D}}_{1000}(25483, 989)$    \\
  $25486   $ & $4$    & $(992, 996]$      & $1$ & $f^{\mathcal{D}}_{1000}(25486, 992)$    \\
  $25487   $ & $4$    & $(996, 1000]$     & $1$ & $f^{\mathcal{D}}_{1000}(25487, 996)$    \\
  \hline
  $254871   $ & $2$   & $(996, 998]$      & $0$ & $f^{\mathcal{D}}_{1000}(254871, 996)$   \\
  $254873   $ & $2$   & $(998, 1000]$     & $0$ & $f^{\mathcal{D}}_{1000}(254873, 998)$   \\
  \hline
  $2548731  $ & $1$   & $(998, 999]$      & $0$ & $f^{\mathcal{D}}_{1000}(2548731, 998)$  \\
  $2548736  $ & $1$   & $(999, 1000]$     & $0$ & $f^{\mathcal{D}}_{1000}(2548736, 999)$  \\
  \hline
  $25487361 $ & $1$   & $(999, 1000]$     & $0$ & $f^{\mathcal{D}}_{1000}(25487361, 999)$ \\
  \hline
\end{tabular}
\caption[Steps for computing the $1000$th derangement in $S_8$]{
  There are $A000166(8) = 14833$ derangements on $8$ letters.
  The table shows the recursive steps to find that the derangement at index
  $1000$ is $25487361$.
}
\label{table:unrankDerangement}
\end{table}

\section{Unranking m\'enage permutations}
\label{sec:unrankingMenage}
After claiming Richard's prize for unranking derangements, the conversation shifted to
how this technique could be extended. A natural next step seemed to be to look
at another family of restricted permutations, namely m\'enage permutations.

A m\'enage permutation comes from the \textit{probl\`eme des m\'enages},
introduced by \'Edouard Lucas in 1891.
There are a few choices of how to define these permutations, but we will
use the following definition for simplicity.
\begin{definition}
  A \textbf{m\'enage permutation} is a permutation $\pi \in S_n$ such that for
  all $i \in [n]$,
  $\pi(i) \neq i$ and
  $\pi(i) + 1 \not\equiv i \bmod n$.
  The set of m\'enage permutations of length $n$ is denoted by $\pazocal{M}_n$.
\end{definition}

\subsection{The answer to a \$100 question about m\'enage permutations}
By February 2020, it appeared that the techniques for unranking derangements
would not directly translate to the context of m\'enage permutations. In
response, Richard Arratia upped the stakes by offering another prize for
unranking m\'enage permutations. Specifically, he posed the following problem.
\begin{problem}
  For $n=20$ there are $A000179(20) = 312\,400\,218\,671\,253\,762 > 3.1\cdot 10^{17}$
  m\'enage permutations.
  Determine the $10^{17}$-th such permutation when listed in lexicographic order.
\end{problem}
Using the techniques described in this section, we developed a computer program that
computed the answer in less than $30$ milliseconds.
(By comparison, unranking the $10^{157}$-th m\'enage permutation of the more
than $1.25 \times 10^{157}$ m\'enage permutations in $S_{100}$
with the same program takes about $7$ seconds.)
\begin{answer}
  The desired permutation is \begin{equation}
    7\ 16\ 19\ 12\ 2\ 8\ 15\ 1\ 18\ 14\ 3\ 9\ 20\ 10\ 5\ 17\ 13\ 4\ 11\ 6.
  \end{equation}
\end{answer}

\subsection{Overview for unranking m\'enage permutations}

As in the section about unranking derangements, we will use the insight from
Theorem \ref{theorem:unrankFromPrefix} that if we can efficiently count the number
of words with a given prefix, then we can efficiently unrank the words.

The technique exploits the following observations:
after placing rooks on a board corresponding to our
prefix, the remaining board has the property that its complement
can be partitioned into sub-boards that do not share rows or columns.
These sub-boards have a structure that we can understand,
and we can leverage that understanding to compute the rook polynomials of these
sub-boards and consequently of the complementary board itself. Once we have computed
the rook polynomial of the complementary board, we can again use
Lemma \ref{lemma:CountsFromComplementaryPolynomials}
to compute the number of full rook placements on the original board.
This gives us the number of m\'enage permutations with a given prefix.

\subsection{Disjoint board decomposition}
Figure \ref{fig:menageMultiplePlacement}
suggestively shows a placement of rooks according to a prefix that
results in a board whose complement can be partitioned into
sub-boards whose squares don't share any rows or columns.
We will see that this property indeed holds in general,
and we can exploit this in order to count the m\'enage permutations
with a given prefix.

\input{assets/fig_menageMultiplePlacement.tex}

The property of complements that can be partitioned into sub-boards
whose squares don't share rows or columns is useful because it provides
a way of factoring the rook polynomial of the bigger board into the rook
polynomials of the sub-boards.
\begin{definition}
  Two sub-boards $B$ and $B'$ are called \textbf{disjoint} if no squares of $B$ are
  in the same row or column as any square in $B'$.
\end{definition}

Kaplansky gives a way of computing the rook polynomial of a board in terms of
its disjoint boards.
\begin{theorem}[\cite{Kaplansky1946}]
  If $B$ can be partitioned into disjoint boards $b_1, b_2, \dots, b_m$,
  then the rook polynomial of $B$ is the product of the rook polynomials of
  each sub-board \begin{equation}
    p_B(x) = \prod_{i=1}^m p_{b_i}(x).
  \end{equation}
\label{thm:productOfDisjointBoards}
\end{theorem}

We will use this disjoint board decomposition repeatedly, because the boards
that result
after placing a prefix can be partitioned into disjoint sub-boards whose
structure is well understood. Now we will give a name to these blocks,
which are illustrated in Figure \ref{fig:blockShape}.

\input{assets/fig_blockShape.tex}

\begin{definition}
  A board is called \textbf{staircase-shaped} if it matches one of the
  following four shapes:
  \begin{alignat*}{2}
    \mathcal{O}_{2n-1}           &= \{(i,i) : i \in [n]\}    &&\cup\ \{(i,i+1) : i \in [n-1]\} \\
    \mathcal{O}_{2n-1}^\intercal &= \{(i,i) : i \in [n]\}    &&\cup\ \{(i+1,i) : i \in [n-1]\} \\
    \mathcal{E}_{2n-2}           &= \{(i,i) : i \in [n-1]\}\ &&\cup\ \{(i+1,i) : i \in [n-1]\} \\
    \mathcal{E}_{2n-2}^\intercal &= \{(i,i) : i \in [n-1]\}\ &&\cup\ \{(i,i+1) : i \in [n-1]\}.
  \end{alignat*}
  The subscripts represent the number of squares, and the names represent their
  parity.
  \label{def:staircaseShaped}
\end{definition}

We now show that our resulting boards can be partitioned into boards of these shapes.

\begin{lemma}
  For $\ell \geq 1$, and prefix $\alpha = (\alpha_1, \alpha_2, \dots, \alpha_\ell)$
  the derived complementary board $B_\alpha^c$ can be partitioned into
  disjoint staircase-shaped boards.
  \label{lemma:boardShape}
\end{lemma}
\begin{proof}
  The proof proceeds by induction on the length of the prefix.

  To establish the base case, consider a prefix of length $\ell = 1$.
  Because of the m\'enage restriction,
  $\alpha_1 \in \{2, 3, \dots, n-1\}$, and
  the derived complementary board $B_{(\alpha_1)}^c$
  can be partitioned into two disjoint sub-boards with shapes
  $\mathcal{O}_{2\alpha_1 - 3}$ and
  $\mathcal{O}^\intercal_{2n - 2\alpha_1 - 1}$.
  (This is illustrated for the case of $n = 7$ and $\alpha_1$ in
  Figure \ref{fig:menageSinglePlacement}.)

  The inductive hypothesis is that the derived complementary board for
  a prefix of length ${\ell - 1}$ consists of sub-boards with shape
  $\mathcal{O}_{2m - 1}$,
  $\mathcal{O}^\intercal_{2m - 1}$,
  $\mathcal{E}_{2m - 2}$, or
  $\mathcal{E}^\intercal_{2m - 2}$.
  Placing a rook in row $\ell$ can remove a top row or a column or both in a
  given sub-board.
  Table \ref{table:rooksInBlocks} below
  shows the resulting sub-boards after placing a rook in
  $\ell$-th row of $B$,
  which may be in the top row, the $i$-th column, or both.
  \\ ~ \\
  \captionsetup{type=table}
  \begin{tabular}{|l|l|l|l|l|}
  \hline
  Rook placement
    & $\mathcal{O}_{2m-1}$
    & $\mathcal{O}_{2m-1}^\intercal$
    & $\mathcal{E}_{2m-2}$
    & $\mathcal{E}_{2m-2}$
  \\ \hline
  Row $1$
    & $\mathcal{O}_{2m-3}$
    & $\mathcal{E}_{2m-2}^\intercal$
    & $\mathcal{O}_{2m-3}$
    & $\mathcal{E}_{2m-4}^\intercal$
  \\
  Column $i$
    & $\mathcal{O}_{2i-3}$, $\mathcal{E}_{2m-2i}$
    & $\mathcal{E}_{2i-2}$, $\mathcal{O}_{2m-2i-1}^\intercal$
    & $\mathcal{E}_{2i-2}$, $\mathcal{E}_{2m-2i-2}$
    & $\mathcal{O}_{2i-3}$, $\mathcal{O}_{2m-2i-1}^\intercal$
  \\
  Row $1$, column $i$
    & $\mathcal{O}_{2i-5}$, $\mathcal{E}_{2m-2i}$
    & $\mathcal{O}_{2i-3}$, $\mathcal{O}_{2m-2i-1}^\intercal$
    & $\mathcal{O}_{2i-3}$, $\mathcal{E}_{2m-2i-2}$
    & $\mathcal{O}_{2i-5}$, $\mathcal{O}_{2m-2i-1}^\intercal$
  \\ \hline
  \end{tabular}
  \captionof{table}[Rook placements on staircase-shaped boards]{
    The results of placing a rook in the first row, $i$-th column, or both
    for all staircase-shaped boards.
  }
  \label{table:rooksInBlocks}
  Therefore placing any number of rooks in the first $\ell$ rows results in a
  board whose complementary derived board is composed of disjoint
  staircase-shaped sub-boards.
\end{proof}

\input{assets/fig_menageSinglePlacement.tex}

\subsection{Rook polynomials of blocks}
Recall that the goal of partitioning $B$ into disjoint sub-boards
$b_1, b_2, \dots, b_m$ is so that we can
factor $p_B(x)$ in terms of $p_{b_i}(x)$.
Of course, this is only useful if we can describe $p_{b_i}(x)$,
which is the goal of this subsection.
Conveniently, the rook polynomial of each $b_i$ will turn out to depend only on the
number of squares, $|b_i|$, which can be computed recursively because of its
staircase shape.

We will begin by defining a family of polynomials that, suggestively, will turn
out to be the rook polynomials that we are looking for.
(The coefficients of these polynomials are described by OEIS sequence A011973 \cite{oeis}.)
\begin{definition}
  For $j \geq 0$, the $j$-th \textbf{Fibonacci polynomial} $F_{j}(x)$ is defined recursively as:
  \begin{align}
    F_0(x) &= 1 \\
    F_1(x) &= 1 + x \\
    F_n(x) &= xF_{n-2}(x) + F_{n-1}(x).
  \end{align}
\label{def:FibonacciPolynomial}
\end{definition}

The rook polynomials of the staircase-shaped boards agree with these Fibonacci
polynomials.

\begin{lemma}
  If $B$ is a staircase-shaped board with $k$ squares, then
  $B$ has rook polynomial ${p_{B}(x) = F_{k}(x)}$, equal to the $k$-th
  Fibonacci polynomial.
  \label{lemma:staircaseIsFibonacci}
\end{lemma}
\begin{proof}
  We will recall the recursive construction of rook polynomials from
  Lemma \ref{lemma:rookPolynomialRecursion}, and proceed by
  induction on the number of squares, always choosing to include or exclude
  the upper-left square.

  Since the reflections of board have the same rook polynomial as the
  unreflected board, without loss of generality, we
  will compute the rook polynomials for
  $\mathcal{O}_{2m-1}$ and $\mathcal{E}_{2m-2}$, respectively.

  To establish a base case, consider the rook polynomials when $n = 1$, so
  the even board has $|\mathcal{E}_{0}| = 0$ squares and
  the odd board has $|\mathcal{O}_{1}| = 1$ square.
  We can see the corresponding rook polynomials directly. There is $1$ way to
  place $0$ rooks on $\mathcal{E}_{0}$ and no ways to place more rooks;
  similarly there is
  $1$ way to place $0$ rooks on $\mathcal{O}_{1}$,
  $1$ way to place $1$ rook on $\mathcal{O}_{1}$, and
  no way to place more than one rook. Thus \begin{alignat}{2}
    p_{\mathcal{E}_{0}}(x) &= 1     &&= F_0(x), \text{ and} \\
    p_{\mathcal{O}_{1}}(x) &= 1 + x &&= F_1(x).
  \end{alignat}

  With the base case established, our inductive hypothesis is that
  $p_{B}(x) = F_{h}(x)$ whenever $B$ is a
  staircase-shaped board with $h < k$ squares.

  Assume that we have $k$ squares where $k$ is even, so our board looks like
  $\mathcal{E}_{k}$. We can either place a rook or not in the upper-left square.
  If we include the square, then $(\mathcal{E}_{k})_i \cong \mathcal{E}_{k-2}$,
  if we exclude the square, then $(\mathcal{E}_{k})_e \cong \mathcal{O}_{k-1}$.
  Thus by Lemma \ref{lemma:rookPolynomialRecursion}, the rook polynomial of
  $\mathcal{E}_{k}$ is
  \begin{align}
    p_{\mathcal{E}_{k}}(x)
    &= xp_{\mathcal{E}_{k-2}}(x) + p_{\mathcal{O}_{k-1}}(x) \\
    &= xF_{k-2}(x) + F_{k-1}(x) \\
    &= F_{k}(x).
  \end{align}

  The case where $k$ is odd proceeds in almost the same way.
  Here our board looks like $\mathcal{O}_{k}$.
  We can either place a rook or not in the upper-left square.
  If we include the square, then $(\mathcal{O}_{k})_i \cong \mathcal{O}_{k-2}$,
  if we exclude the square, then $(\mathcal{O}_{k})_e \cong \mathcal{E}_{k-1}$.
  Again by Lemma \ref{lemma:rookPolynomialRecursion}, the rook polynomial
  of $\mathcal{O}_{k}$ is
  \begin{align}
    p_{\mathcal{O}_{k}}(x)
    &= xp_{\mathcal{O}_{k-2}}(x) + p_{\mathcal{E}_{k-1}}(x) \\
    &= xF_{k-2}(x) + F_{k-1}(x) \\
    &= F_{k}(x).
  \end{align}
\end{proof}

Therefore, we now have the ingredients to describe the rook polynomial of
a derived complementary board.

\begin{corollary}
  Suppose that $B_\alpha^c$ can be partitioned into $m$ disjoint
  staircase-shaped sub-boards of sizes $b_1, b_2, \dots, b_m$.
  Then the rook polynomial of
  $B_\alpha^c$ is \begin{equation}
    p_{B_\alpha^c}(x) = \prod_{i=1}^m F_{b_i},
  \end{equation} where $F_j$ is the $j$-th Fibonacci polynomial.
  \label{cor:derivedComplementaryRookPolynomial}
\end{corollary}
\begin{proof}
  This follows directly from Theorem \ref{thm:productOfDisjointBoards} together
  with Lemma \ref{lemma:staircaseIsFibonacci}.
\end{proof}

\subsection{Sub-boards from prefix}

In this part, we discuss how to algorithmically compute the size of the
sub-boards of the partition of the derived complementary board $B_\alpha^c$
for a given prefix $\alpha$.

\begin{lemma}
  Given a nonempty prefix $\alpha = (\alpha_1, \alpha_2, \dots, \alpha_\ell)$
  and $i \not\in \alpha$,
  the number of squares of $B^c$ in column $i$ that do not have
  a first coordinate in $[\ell]$
  is given by the rule:
  \begin{singlespace}
  \begin{equation}
    c_i = \begin{cases}
      0 & i < \ell \\
      1 & i = \ell \text{ or } i = n \\
      2 & \ell < i < n
    \end{cases}
  \end{equation}
  \end{singlespace}
\end{lemma}

\begin{proof}
  It is helpful to recall that the complementary board $B^c$ consists of squares
  on the diagonal, squares on the subdiagonal, and the square $(1, n)$:
  \begin{equation}
    B^c = \{(i, i) : i \in [n]\} \cup \{(i + 1, i) : i \in [n-1]\} \cup \{(1,n)\}.
  \end{equation}

  Now if $i < \ell$, then $(i,i)$ and $(i + 1, i)$ both have a first coordinate
  less than or equal to $\ell$.

  If $i = \ell$, then $(i, i)$ has a first coordinate in $[\ell]$, but
  $(i + 1, i) = (\ell + 1, \ell)$ does not have its first coordinate in $[\ell]$.

  If $i = n$, there are two squares of $B^c$ in column $i$: $(n, n)$ and $(1, n)$.
  Only $(1,n)$ has its first coordinate in $[\ell]$.

  If $\ell < i < n$, then neither the square $(i, i)$ nor $(i+1, i)$ has
  its first coordinate in $[\ell]$.
\end{proof}

Now we will go through each contiguous section of columns, and count the number
of squares in each to build up the size of each of the blocks.

\begin{lemma}
  Partition $[n] \setminus \alpha$ into contiguous parts, $\pazocal{P}$.
  Each part $P_i \in \pazocal{P}$ of the partition corresponds to a staircase-shaped
  sub-board of size $\sum_{p \in P_i} c_p$.

  Therefore the size of the disjoint sub-boards in the derived complementary
  board $B_\alpha^c$ is given by the multiset \begin{equation}
    \pazocal{P}_\alpha = \left\{ \sum_{p \in P_i} c_p : P_i \in \pazocal{P} \right\}.
  \end{equation}
\label{lem:subBoardSizes}
\end{lemma}

\begin{proof}
  Once the first row of a complementary m\'enage board has been deleted,
  the resulting board has the property that any two nonadjacent columns
  do not have any squares in the same row, because column $i$ has
  squares in $(i, i)$ and $(i+1, i)$.

  Within each contiguous interval between the letters of $\alpha$,
  the columns form a staircase-shaped sub-board because
  each column with a square in position $(i + 1, i)$ has a square to its right,
  in position $(i + 1, i + 1)$ whenever $i + 1 \not\in \alpha$.
\end{proof}

\begin{example}
  As illustrated in Figure \ref{fig:menageMultiplePlacement}, if $n = 12$
  and $\alpha = (3,6,1,8)$, then the contiguous partition of \begin{equation}
    [12] \setminus \{3,6,1,8\} = \{
      \underbrace{2\vphantom{,}}_{P_1},
      \underbrace{4, 5}_{P_2},
      \underbrace{7\vphantom{,}}_{P_3},
      \underbrace{9, 10, 11, 12}_{P_4}
    \}
  \end{equation} is $\{P_1, P_2, P_3, P_4\}$. The corresponding staircase-shaped sub-boards
  have sizes
  \begin{alignat*}{3}
    k_1 &= c_2                            &&= 0             &&= 0 \\
    k_2 &= c_4 + c_5                      &&= 1 + 2         &&= 3 \\
    k_3 &= c_7                            &&= 2             &&= 2 \\
    k_4 &= c_9 + c_{10} + c_{11} + c_{12} &&= 2 + 2 + 2 + 1 &&= 7,
  \end{alignat*}
  which matches what we observe in the illustration:
  \begin{equation}
    B_\alpha^c = \mathcal{E}_0 \sqcup \mathcal{O}_3 \sqcup \mathcal{E}_2 \sqcup \mathcal{O}_7^\intercal,
  \end{equation}
  \label{ex:blocksFromPrefix}
\end{example}

\subsection{Complementary polynomials}
We have now established a method taking a prefix $\alpha$
and partitioning $B_\alpha^c$ into disjoint staircase-shaped sub-boards,
which allow us to determine the rook polynomial of $B_\alpha^c$.
Using Lemma \ref{lemma:CountsFromComplementaryPolynomials}, this allows us
to finally compute the number of ways of placing $n - \ell$ rooks on
$B_\alpha$, thus determining the number of derangements that begin with
$\alpha$.

\begin{theorem}
  The number of m\'enage permutations that begin with a valid, nonempty
  prefix $\alpha$ is
  \begin{equation}
    \#\operatorname{prefix}_\pazocal{M}(\alpha) = \sum_{k=0}^{n} (-1)^k r_k^c (n-k)!
  \end{equation} where
  $\sum_{k=0}^n r_k^c x^k = \prod_{p \in \pazocal{P}_\alpha} F_p$,
  $F_k$ is the $k$-th Fibonacci polynomial, and
  $\pazocal{P}_\alpha$ is the multiset corresponding to the size of the staircase-shaped
  sub-boards in the disjoint partition of $B_\alpha^c$.
\end{theorem}
\begin{proof}
  This follows directly from Corollary \ref{cor:derivedComplementaryRookPolynomial}
  together with Lemma \ref{lem:subBoardSizes}
\end{proof}

Now that we have computed the number of m\'enage permutations,
Theorem \ref{theorem:unrankFromPrefix} provides an efficient
unranking algorithm for $\pazocal{M}$.

We will illustrate this with a specific example computing the number of m\'enage
permutations with a given prefix.
\begin{example}
  We will continue with the running example illustrated in
  Figure \ref{fig:menageMultiplePlacement} and expounded on
  in Example \ref{ex:blocksFromPrefix}.

  We've already seen that the for $n = 12$, the prefix $\alpha = (3,6,1,8)$
  partitions the derived complementary board into three nonempty sub-boards: \begin{equation}
    B_\alpha^c = \mathcal{E}_0 \sqcup \mathcal{O}_3 \sqcup \mathcal{E}_2 \sqcup \mathcal{O}_7^\intercal.
  \end{equation} Lemma \ref{lemma:staircaseIsFibonacci} tells us that the rook polynomial of
  $B_\alpha^c$ is \begin{align}
    p_{B_\alpha^c}(x)
    &= F_3(x)F_2(x)F_7(x) \\
    &= (1 + 3x + x^2)(1 + 2x)(1 + 7x + 15x^2 + 10x^3 + x^4) \\
    &= 1 + 12 x + 57 x^2 + 136 x^3 + 170 x^4 + 105 x^5 + 27 x^6 + 2 x^7 \\
    &= \sum_{k=0}^7 r_k^c x^k.
  \end{align}

  By Lemma \ref{lemma:CountsFromComplementaryPolynomials},
  the number of ways to place eight rooks on $B_\alpha$ is
  is \begin{align}
    N_0
      &= \sum_{k=0}^{7} (-1)^k r_k^c (8-k)! \\
      &= 1(8!) - 12(7!) + 57(6!) - 136(5!) + 170(4!) - 105(3!) + 27(2!) - 2(1!) \\
      &= 8062.
  \end{align}
  Therefore there are $8062$ m\'enage permutations in $S_{12}$ that start with
  the prefix $(3,6,1,8)$.
\end{example}

We can now repeatedly use the above counting technique in conjunction with
Theorem \ref{theorem:unrankFromPrefix} to unrank derangements.

\begin{example}
  There are $A000179(8) = 4738$ m\'enage permutations on $8$ letters.
  Table \ref{table:unrankMenage} shows the steps of the algorithm that
  determines that the $1000$th m\'enage permutation in lexicographic order is
  \begin{equation}
    w_{1000} = 3 \ 5 \ 4 \ 8 \ 2 \ 7 \ 1 \ 6.
  \end{equation}
\end{example}

\input{assets/table_menage1000}

\section{Generalizations and open questions}

In this final section we explore several possible future directions for
applying these ideas in new contexts. We can potentially apply these unranking
techniques to position-restricted permutations, permutations that satisfy
certain inequalities with respect to a permutation statistic, or words that
avoid or match certain patterns.

\subsection{Other restricted permutations}
In a 2014 paper about finding linear recurrences for derangements, m\'enage
permutations and other restricted permutations, Doron Zeilberger
introduces a more general family of restricted permutations.
\begin{definition}[\cite{Zeilberger2014}]
  Let $S \subset \mathbb Z$ be a finite collection of integers.
  An $S$-\textbf{avoiding permutation} is a permutation $\pi \in S_n$ such that
  \begin{equation}
    \pi(i) - i - s \not\equiv 0 \bmod n
    \hspace{0.25cm}\text{for all}\hspace{0.25cm}
    i \in [n]
    \hspace{0.25cm}\text{and}\hspace{0.25cm}
    s \in S.
  \end{equation}
\end{definition}

\begin{example}
  In terms of $S$-avoiding permutations, \begin{itemize}
    \item ordinary permutations are $\emptyset$-avoiding permutations,
    \item derangements are $\{0\}$-avoiding permutations, and
    \item m\'enage permutations are $\{-1,0\}$-avoiding permutations.
  \end{itemize}
\end{example}

The results in the previous sections straightforwardly adapt to the cases of
unranking $\{i\}$-avoiding and $\{i, i+1\}$ avoiding permutations.

\begin{openquestion}
  For arbitrary finite subsets $S \subset \mathbb Z$,
  do there exist efficient unranking algorithms on $S$-avoiding permutations?
\end{openquestion}

The techniques used to unrank derangements and m\'enage permutations do
not appear to generalize even to superficially similar domains. So, in the
spirit of Richard Arratia's bounties, it is only fair to offer one of my own.

\begin{problem}
  Do there exist efficient unranking algorithms on $\{-1, 1\}$-avoiding
  permutations?
\end{problem}

The main obstruction to using the techniques from
Section \ref{sec:unrankingMenage} to resolve this question is that placing
a rook and deleting a column does not necessarily cause the left and right
sides of that column to be disjoint. As such, unranking $\{-1,1\}$-avoiding
permutations appears to require a genuinely novel insight.

\subsection{Permutation statistics}
Another area for exploration is unranking permutations with a given permutation
statistic.

\begin{openquestion}
  Let $\operatorname{inv}\colon S_n \rightarrow \mathbb N_{\geq 0}$ be the map
  that counts inversions of a permutation. Since $\operatorname{inv}$ is a
  Mahonian statistic, the generating function for the number of permutations
  $\pi \in S_n$ such that $\operatorname{inv}(\pi) = k$ is given by the
  $q$ analog of $n!$, $n!_q$.

  Does there exist an efficient unranking function on the set \begin{equation}
    \mathcal{I}_n^k = \{\pi \in S_n | \operatorname{inv}(\pi) = k\},
  \end{equation} and if so, how does one construct it?
\end{openquestion}

We can, of course, substitute $\operatorname{inv}$ with any other permutation
statistic of interest.

\subsection{Pattern avoidance}
In the field of combinatorics on words, there exists a notion of patterns and
instances of a pattern. At this level of informality, this is probably best
illustrated with an example (with undefined words in bold).

\begin{example}
  The word $1100010110$ is an \textbf{instance} of the \textbf{pattern} $ABA$ with $A = 110$ and
  $B = 0010$.
  The word $32123213212$ is said to \textbf{match} the pattern $CC$ with $C = 321$ because
  it contains a substring of the form $321321$.
\end{example}

\begin{openquestion}
  Given a pattern $P$, is it possible to unrank words of length $n$ over an
  alphabet $\mathcal A$ that are not instances of the pattern $P$?
  That match the pattern $P$? That don't match the pattern $P$?
\end{openquestion}

\subsection{Prefixes of Lyndon words}
There are other collections of finite words that might be amenable to some of
the above techniques. In particular, Kociumaka, Radoszewski, and Rytter
\cite{Kociumaka2014} give polynomial time algorithms for unranking Lyndon
words. We have some conjectures about prefixes of Lyndon words and open
questions about other restricted words.
\begin{definition}
  A \textbf{Lyndon word} is a string over an alphabet of letters
  that is the unique minimum with respect to all of its rotations.
\end{definition}
\begin{example}
  $00101$ is a Lyndon word because \begin{equation}
    00101 = \min\{00101, 01010, 10100, 01001, 10010\}
  \end{equation} is the unique minimum of all of its rotations.

  $011011$ is not a Lyndon word because while \begin{equation}
    011011 = \min\{011011, 110110, 101101, 011011, 110110, 101101\},
  \end{equation}
  it is not the \textbf{unique} minimum.
  (That is, rotating it three positions returns it to itself.)
\end{example}

\begin{definition}[\cite{Sloane1995}]
  Suppose that $\{a_i\}_{i=1}^\infty$ and $\{b_i\}_{i=1}^\infty$ are integer sequences
  related by \begin{equation}
    1 + \sum_{n=1}^\infty b_n x^n = \prod_{i=1}^\infty \frac{1}{(1-x^i)^{a_i}}.
  \end{equation} Then $\{b_i\}_{i=1}^\infty$ is said to be the
  \textbf{Euler transform} of $\{a_i\}_{i=1}^\infty$, denoted
  $\pazocal{E}(\{a_i\}_{i=1}^\infty) = \{b_i\}_{i=1}^\infty$.
\end{definition}

\begin{definition}
  Let $\mathcal{L}_\alpha = \{\ell_n^\alpha\}_{n=1}^\infty$ where
  $\ell_n^\alpha$ is the number of Lyndon words with prefix $\alpha$ and length $n$
  over the alphabet $\{0,1\}$.
\end{definition}

\begin{conjecture}
  The Euler transform of the number of Lyndon words with prefix $\alpha$ and
  length $n$ over the alphabet $\{0,1\}$,
  $\pazocal{E}(\mathcal{L_\alpha}) = \{t_{n}^\alpha\}_{n=1}^\infty$,
  follows a linear recurrence for all $n \geq N_\alpha$.
\end{conjecture}

This conjecture and the following conjectures are based on the data
in Table \ref{table:lyndonConjectures}.

\include{assets/table_lyndonConjectures.tex}

We start with two specific conjectures about two families of prefixes.
\begin{conjecture}
  For $k \geq 1$,
  let $\mathcal{L}_\alpha$ be the sequence of the
  number of Lyndon words of length $n$ with prefix
  ${\alpha = (0, 0, \dots, 0)}$ of length $k$ over the alphabet $\{0,1\}$.
  Then the Euler transform
  $\pazocal{E}(\mathcal{L}_\alpha) = \{t_{n}^\alpha\}_{n=1}^\infty$
  follows the linear recurrence $t_{n+1}^\alpha = 2t_{n}^\alpha$ for all
  $n \geq k + 2$.
\end{conjecture}

\begin{conjecture}
  For $k \geq 2$
  let $\mathcal{L}_\alpha$ be the sequence of the
  number of Lyndon words of length $n$ with prefix
  ${\alpha = (1, 0, 0, \dots, 0)}$ of length $k$ over the alphabet $\{0,1\}$.
  Then the Euler transform
  $\pazocal{E}(\mathcal{L}_\alpha) = \{t_{n}^\alpha\}_{n=1}^\infty$
  follows the linear recurrence $t_{n+k}^\alpha = \sum_{i=0}^{k-1} t_{n+i}^\alpha$
  for all $n \geq 1$.
\end{conjecture}

And more broadly, we have a conjecture in the case that the prefix is not the
zero sequence.
\begin{conjecture}
  Let $\mathcal{L}_\alpha$ be the sequence of the
  number of Lyndon words of length $n$ with prefix $\alpha$ over the alphabet
  $\{0,1\}$ such that $\alpha$ contains at least one $1$.
  Then the Euler transform of the sequence,
  $\pazocal{E}(\mathcal{L_\alpha}) = \{t_{n}^\alpha\}_{n=1}^\infty$,
  follows the linear recurrence where all terms have coefficients of $0$ or $1$.
\end{conjecture}

\begin{openquestion}
  If, as the evidence suggests, $\pazocal{E}(\mathcal{L_\alpha})$ follows a
  linear recurrence,
  what is the length of the recurrence and
  what are the coefficients of the recurrence
  as a function of $\alpha$?
\end{openquestion}

\printbibliography
\end{document}


Chern class of Schubert cells in the flag manifold and related algebras

Abstract: We discuss a relationship between Chern-Schwartz-MacPherson classes for Schubert cells in flag manifolds, Fomin-Kirillov algebra, and the generalized nil-Hecke algebra. We show that nonnegativity conjecture in Fomin-Kirillov algebra implies the nonnegativity of the Chern-Schwartz-MacPherson classes for Schubert cells in flag manifolds for type A. Motivated by this connection, we also prove that the (equivariant) Chern-Schwartz-MacPherson classes for Schubert cells in flag manifolds are certain summations of the structure constants of the equivariant cohomology of the Bott-Samelson varieties. We also discuss the refined positivity conjectures of the Chern-Schwartz-MacPherson classes for Schubert cells motivated by the nonnegativity conjecture in Fomin-Kirillov algebra.

\section{introduction}
The Chern classes are characteristic classes associated with vector bundles on a smooth variety. A functorial theory of Chern classes for possibly singular variety was conjectured by Grothendieck and Deligne, and established by MacPherson \cite{Mac74}. For a constructible function $\phi$ on a complex variety $X$, the theory associates a class $c_*(\phi) \in H_*(X)$ such that $c_*(\mathbbm{1}_X)= c(X) \cap [X]$ when $X$ is a smooth compact complex variety. The functoriality properties satisfied by theses classes determine them uniquely. If $X$ is a compact complex variety, then the class $c_*(\mathbbm{1}_X)$ coincides with a class defined by Schwartz \cite{Sch65a,Sch65b}, so the class is commonly called the Chern-Schwartz-MacPherson (CSM) class of $X$.\\

Aluffi and Mihalcea studied the CSM classes of Schubert cells in Grassmannians and flag varieties \cite{AM09,AM15}. For an element $w$ in the Weyl group, the CSM class of the Schubert cell $X(w)^\circ$ is $c_*(\mathbbm{1}_{X(w)^\circ})$, which we denote by $\csm(X(w)^\circ)$. In \cite{AM15}, Aluffi and Mihalcea provided a beautiful reculsive formula to compute the CSM classes of Schubert cells $X(w)^\circ$ in the flag variety by investigating the relationship between $X(w)^\circ$ and $X(ws_k)^\circ$ where $s_k$ is a simple reflection. They also conjectured that the CSM classes of Schubert cells are nonegative, i.e., when we write
$$\csm(X(w)^\circ)= \sum_u c(w;u) [X(u)]$$ in terms of Schubert basis $[X(u)]$ in the homology of the flag variety, $c(w;u)$ is nonnegative. Huh \cite{Huh} provides a proof of the positivity of the CSM classes of Schubert cells in the Grassmannian, but his technique does not seem to apply in the flag variety. \\

In this paper, we connect the theory of CSM classes of Schubert cells in a flag variety for type A with the seemingly unrelated algebras, called the Fomin-Kirillov algebra and the generalized nil-Hecke algebra. Fomin and Kirillov \cite{FK99} studied a certain quadratic algebra, also called the Fomin-Kirillov algebra, to understand the combinatorics of the cohomology of the flag variety. There is a commutative subalgebra of the Fomin-Kirillov algebra generated by Dunkl elements, which is isomorphic to the cohomology of the flag variety. They conjectured that Schubert polynomials evaluated at Dunkl elements can be written as a nonnegeative linear combination of non-commutative monomials, which is called the nonnegativity conjecture.\\

The first main theorem in this paper is the following:

\begin{thm} \label{main1} The nonnegativity conjecture implies the nonnegativity of the CSM classes of Schubert cells in the flag variety for type A. \end{thm}

 It turns out that the nonnegativity conjecture for the Fomin-Kirillov algebra is much stronger than the positivity conjecture for CSM classes conjectured by Aluffi and Mihalcea for type A. Motivated by this fact, we investigate the relation between the equivariant CSM classes of Schubert cells in the flag variety of all types and the structure constants of the equivariant cohomology of the Bott-Salemson varieties. Ohmoto proves \cite{Ohm} that there is an equivariant version of $c_*$, denoted by $c^T_*$ such that for a constructible set $U$ in a complex variety $X$, $c^T_*(\mathbbm{1}_U)$ is an element in the equivariant homology $H^T_*(X)$ of $X$ that satisfies $c^T_*(\mathbbm{1}_Y)= c^T(T_Y) \cap [Y]_T$ if $Y$ is a projective, non-singular variety. Denote the CSM class of the Schubert cell $c^T_*(\mathbbm{1}_{X(w)^\circ})$ by $\csmt(X(w)^\circ)$. Aluffi and Mihalcea \cite{AM15} shows that the their reculsive formula for $\csmt(X(w)^\circ)$ is essentially the same as those for the ordinary CSM classes (See Section 2 for details). \\

Allufi and Mihalcea \cite{AM15} also conjectured that $\csmt(X(w)^\circ)$ also possess positivity. More precisely, when we write
$$\csmt(X(w)^\circ)=\sum_u c_T(w;u) [X(u)]_T,$$
 they conjectured that $c_T(w;v)$ is a nonnegative polynomial in simple roots. The second main theorem is the following.

\begin{thm}\label{main2} For elements $w,v$ in the Weyl group, $c_T(w;v)$ is a certain sum of the structure constants of the equivariant Bott-Samelson variety (See Section 6 for the precise statement.)
\end{thm}
This result together with Theorem \ref{main1} suggests interesting conjectures about the positivity of refined sum of the constants of the equivariant Bott-Samelson variety strictly stronger than Aluffi and Mihalcea's conjectures. One of refined conjectures is the following.

 \begin{conj}\label{refined} For Weyl group elements $w,u,v$ with $\ell(w)=\ell(u)+\ell(v)$, let $p^w_{u,v}$ be the Littlewood-Richardson coefficients, the structure constants of the cohomology of the flag variety. Then
 $$c(w;v) \geq \sum_{\substack{u\\ \ell(w)=\ell(u)+\ell(v)}} p^w_{u,v}.$$
 \end{conj}

We use the generalized nil-Hecke algebras studied by Berenstein and Richmond \cite{BR15} for studying the equivariant structure constants of the cohomology of the Bott-Samelson varieties and relate the generalized nil-Hecke algebra with the Fomin-Kirillov algebra and the operator $T_k$ studied by Aluffi and Mihalcea (See Section 2 for the definition of $T_k$).\\

 The structure of the paper is as follows: In Section 2, we discuss Allufi and Mihalcea's reculsive formula about the CSM class of the Schubert cells in the flag variety. In Section 3, we introduce the Fomin-Kirillov algebra and describe the nonnegativity conjecture. In Section 4, we state Theorem \ref{main1} precisely without the proof and discuss its implications. In Section 5, we define the generalized nil-Hecke algebra and connect the algebra with the structure constants of the equivariant cohomology of the Bott-Samelson variety. In Section 6, we interpret $T_k$ in terms of the generalized nil-Hecke algebra and show Theorem \ref{main2}. In Section 7, we prove Theorem \ref{main1} by relating Section 6 with the Fomin-Kirillov algebra. In Section 8, we discuss refined positivity conjectures and other concluding remarks.

\section{CSM classes for Schubert cells in flag varieties}
 In this section, we give a brief introduction to the CSM class and the summary of results in \cite{AM15} by Allufi and Mihalcea.\\

Let $G$ be a complex simple Lie group, $B$ be a Borel subgroup, and $T$ a maximal torus in $B$. Let $\mathfrak{g},\mathfrak{h}$ be the Lie algebras of $G$ and $T$, and let $R\subset \mathfrak{h}^*$ be the associated root system with the set of positive roots $R^+$ determined by $B$. Denote by $\{\alpha_1,\ldots,\alpha_r\} \subset R^+$ the set of simple roots. Let $R^\vee$ denote the set of coroots $\alpha^\vee \in \mathfrak{h}$ and $\langle \cdot,\cdot\rangle : R\otimes R^\vee \rightarrow \mathbb{Z}$ the pairing.\\

Let $W$ be the Weyl group of $G$ generated by simple reflections $s_1,\ldots,s_r$ corresponding respectively to the simple roots $\alpha_1,\ldots,\alpha_r$ of $G$. Let $X:=G/B$ be the flag variety and $X(w)^\circ:=BwB/B$ the Schbuert cell for $w$ in the flag variety $X$. Each cell $X(w)^\circ$ is isomorphic to $\mathbb{C}^{\ell(w)}$. Let $X(w):= \overline{BwB/B}$ be the Schubert variety for $w$. Each Schubert variety $X(w)$ has a fundamental class $[X(w)] \in H_*(X,\mathbb{Q})$, and these classes form a basis for the (co)homology of the flag variety. It may be verified that $X(v) \subset X(w)$ if and only if $v\leq w$ in the Bruhat order (See \cite{Ful97} for instance). For type $A$ case, we have $G=SL(n)$ and $B$ the set of upper triangular matrices in $G$. The Weyl group $W$ is the symmetric group $S_n$ and the for elements $w,v \in S_n$, the Bruhat order $v \leq w$ is defined by $v=wt_{ij}$ for some reflection $t_{ij} \in S_n$ with $\ell(v)=\ell(w)-1$.\\

Let $\partial_k$ be the BGG operator \cite{BGG73} from $H_*(X) \rightarrow H_{*+2}(X)$ for each $1\leq k \leq r$, which sends the Schubert class $[X(w)]$ to $[X(ws_k)]$ if $\ell(ws_k)>\ell(w)$ and to 0 otherwise. The Weyl group admits a right action on $H_*(X)$, and one can write the action of $s_k$ on $H_*(X)$ in terms of $\partial_k$ and the first Chern class of the line bundle $\mathcal L_{\alpha_k}$ indexed by $\alpha_k$. More precisely, we have $s_k=id-c_1(\mathcal L_{\alpha_k})\partial_k$ where the action of the first Chern class of line bundle $c_1(\mathcal L_\lambda)$ indexed by an integral weight $\lambda$ on $H_*(X)$ is given by the Chevalley formula:
$$c_1(\mathcal L_{\lambda})\cdot [X(w)]= \sum \langle \lambda, \beta^\vee\rangle [X(ws_\beta)]$$
where the sum is over all positive roots $\beta$ such that $\ell(ws_\beta)=\ell(w)-1$. See e.g. \cite{Ful97} for more details. \\

For $1\leq k \leq r$, define the non-homogeneous operator
$$T_k:=\partial_k-s_k : H_*(X)\rightarrow H_*(X).$$

The CSM class of the Schubert cell $X(w)^\circ$ is defined by $c_*(\mathbbm{1}_{X(w)^\circ}) \in H_*(G/B)$, and we denote by $\csm(X(w)^\circ)$. Although one needs to describe functorial properties of $c_*$ \cite{Mac74} to define the CSM class of the Schubert cells, Aluffi and Mihalcea \cite{AM15} provided the following recursive formula for $\csm(X(w)^\circ)$ which is purely combinatorial.

\begin{thm}  \label{rec} \cite{AM15}
 Let $w \in W$ and $1\leq k \leq r$. Then
$$ T_k(\csm(X(w)^\circ))= \csm(X(ws_k)^\circ)).$$
\end{thm}

For $w \in W$, one can write $\csm(X(w)^\circ)=\sum_{v \in W} c(w;v) [X(v)]$. It is not hard to show from Theorem \ref{rec} that $c(w;v)=0$ unless $v \leq w$. Aluffi and Mihalcea proposed the following positivity conjecture.

\begin{conj}\cite[Conjecture 1]{AM15}\label{poscsm}
For $v\leq w$, we have $c(w;v)>0$.
\end{conj}
Note that the $T_k$ action does not preserve the positivity. \\

Let $P\in G$ be a parabolic subgroup containing $B$, and let  $p: G/B \rightarrow G/P$ be the natural projection. Then $p(X(w))=X(w W_p)$, where $W_P$ is the subgroup of $W$ generated by the reflections in $P$. The functoriality of CSM classes can be used to show that
$$p_*(\csm(X(w)^\circ))= \csm (X(wW_p)^\circ).$$
Therefore, Theorem \ref{poscsm} implies the nonnegativity of CSM classes of any $G/P$. When $G/B$ is a Grassmannian manifold, positivity of special cases of CSM classes in $G/P$ are proved by several authors \cite{AM09,Mih15,Jon10,Str11}, and it was settled for all cases by Huh \cite{Huh}.\\

Aluffi and Leonardo also shows that Theorem \ref{rec} works in the equivariant setting. Let $T\subset B$ be the maximal torus in $B$. For a variety $X$ with a $T$-action, the equivariant cohomology $H^*_T(X)$ is the ordinary cohomology of $(ET \times X)/T$, where $ET$ is the universal $T$-bundle. The equivariant cohomology is an algebra over $H^*_T(pt)$, a polynomial ring in generators for the weight lattice of $T$. Since $G/B$ is smooth, we identify the equivariant homology $H^T_*(G/B)$ with the equivariant cohomology $H^*_T(G/B)$. For a subvariety $Y\subset X$ invariant under the $T$ action, one can associate its equivariant fundamental class $[Y]_T \in H^T_*(X)$. See e.g. \cite{Ful07} for basics of the equivariant cohomology. \\

Ohmoto \cite{Ohm} shows that one can define the functor $c_*^T$ from the group of equivariant constructible functions to the equivariant homology $H_*^T(X)$ satisfying certain functorial properties and $c^T_*(\mathbbm{1}_Y)= c^T(T_Y) \cap [Y]_T$ if $Y$ is a projective, non-singular variety. The image of $c^T_*(\mathbbm{1}_Y)$ via the map $H_*^T(X)\rightarrow H_*(X)$ is the CSM class $c_*(\mathbbm{1}_Y)$. Let $i$ be the map $H_*^T(G/B)\rightarrow H_*(G/B)$ induced by the projection $H^T_*(pt) \rightarrow H_*(pt)\cong\mathbb{Q}$ sending all generators in weight lattice to 0. Then we have $i(\csmt(X(w)^\circ))=\csm(X(w)^\circ)$ and $i(c_T(w;v))=c(w;v)$.

\section{Fomin-Kirillov algebra}

Fomin and Kirillov defined a certain quadratic algebra $\fk$, also called the Fomin-Kirillov algebra, to better understand the combinatorics of the cohomology ring of the flag variety. They show that the commutative subalgebra generated by Dunkl elements of degree 1 is isomorphic to the cohomology of the flag variety. Since then, a lot of variations for the quadratic algebra has been studied \cite{Kir15,KM03,KM04,KM05,KM10,KM12}. For example, there are generalizations of the Fomin-Kirillov algebra for K-theory, quantum, equivariant cohomology and for other finite types.\\

We review some facts on the Fomin-Kirillov algebra mostly proved in \cite{FK99}.
\begin{definition}
For a fixed positive integer $n$, let $\fk$ be the the free algebra generated by $\{[ij]: i,j\in \mathbb{Z},1\leq i<j \leq n\}$ with the following relations:
\begin{align*}
[ij]^2&=0.\\
[ij][kl]&=[kl][ij] \text{ for distinct } i, j, k ,l. \\
[ij][jk]&=[jk][ik]+[ik][ij] \text{ and } [jk][ij]=[ik][jk]+[ik][ij] \text{ for distinct } i, j, k.
\end{align*}
\end{definition}
There are two representations of $\fk$ on $\zsn$. Define the \emph{Bruhat action} of $[ij]$ on $\zsn$ by
\begin{equation}\label{bruhat}
w\cdot[ij] =
\begin{cases}
wt_{i,j} & \text{if}\quad \ell(wt_{i,j})=\ell(w)-1 \\
0 & \text{otherwise.}
\end{cases}
\end{equation}
where the $\ell(w)$ is the lenght of $w$ in $S_n$. The Bruhat action is dual to the action described in \cite{FK99}. Note that Fomin and Kirillov \cite{FK99} defined the action such that $w[ij]=wt_{ij}$ if $\ell(wt_{i,j})=\ell(w)+1$ and $0$ otherwise.\\

The following action is less known but closely related to the CSM classes of the Schubert cells in the flag variety. Define the \emph{extended Bruhat action} of $[ij]$ on $\zsn$ by
\begin{equation}\label{ebruhat}
w*[ij] =
\begin{cases}
wt_{i,j} & \text{if}\quad \ell(wt_{i,j})<\ell(w) \\
0 & \text{otherwise.}
\end{cases}
\end{equation}
Note that the Bruhat action is the degree $-1$ part of the extended Bruhat action. The proof that the extended Bruhat action of $[ij]$ provides a representation of $\fk$ on $\zsn$ is straightforward calculation.\\

\begin{rmk} The $*$-action can be interpreted as edges in the Bruhat graph, although author is not aware that $*$-action is used in past literatures.
\end{rmk}

Let $\theta_i$ be the Dunkl elements defined by $\sum_{j\neq i}[ij]$. Here for $i<j$, $[ji]$ is the same as $-[ij]$. Then $\theta_i$ commutes pairwise for all $i$. Moreover, all symmetric functions in Dunkl elements vanish in $\fk$ and these are all relations between $\theta_i$. Therefore, the commutative subalgebra $\mathcal C$ generated by Dunkl elements is isomorphic to the cohomology of the flag variety. See \cite{FK99} for details.\\

Let $\mathfrak{S}_w$ be the Schubert polynomial in $x_1,\ldots,x_n$ for $w \in S_n$. It is defined recursively by
$$\partial_k \mathfrak{S}_w = \mathfrak{S}_{ws_k},$$
if $\ell(w)>\ell(ws_k)$, where $\partial_k(f)= { f- s_k(f) \over x_i-x_{i+1}}$. For the longest element $w_0$ in $S_n$, $\mathfrak{S}_w=x_1^{n-1}x_2^{n-2}\ldots x_{n-1}$.\\

In \cite{FK99}, Fomin and Kirillov claimed the following nonnegativity conjecture.
\begin{conj}[Nonnegativity conjecture \cite{FK99}] \label{posfk}
Schubert polynomials $\mathfrak{S}_w(\theta)$ evaluated at Dunkl elements (by replacing $x_i$ by $\theta_i$) can be written as a nonnegative linear combination of noncommutative monomials $[i_1,j_1][i_2,j_2]\cdots [i_m j_m]$.
\end{conj}

Note that the nonnegative formula for $\mathfrak{S}_w(\theta)$ implies the combinatorial formula of the Littlewood-Richardson coefficients for the flag variety for type A from the following theorem:
\begin{thm}\cite{FK99}\label{lr}
For $w,u,v \in S_n$ with $\ell(w)=\ell(u)+\ell(v)$, the Littlewood-Richardson coefficients $p^w_{u,v}$ is the same as the coefficient of $v$ in $w \cdot \mathfrak{S}_u$.
\end{thm}

\begin{rmk}\label{cases}
The nonnegativity conjecture is known for few $w$. Pieri rule, the case $w_{a,b}:=s_a s_{a+1}\ldots s_b$ for $1\leq a\leq b \leq n$, is conjectured in \cite{FK99} and proved by Postnikov \cite{Pos99}. The dual case $w=w_{a,b}^{-1}$ follows from Pieri rule as well. Recently, the nonnegative formula for the hook-shape case $w_{b+1,c}^{-1}w_{a,b}$ for $a\leq b\leq c$ is given by M\'{e}sz\'{a}ros, Panova, and Postnikov \cite{MPP14}. Positivity is also known for hook-shape with one box added at (2,2) in \cite{MPP14}.
\end{rmk}

\section{Application of the nonnegativity conjecture}
Let $\psi: \zsn \rightarrow \mathbb{Z}$ be the ring homomorphism defined by $w\mapsto 1$ for all $w \in S_n$. Then the following is the relation between the Fomin-Kirillov algebra and the CSM classes.

\begin{thm}\label{main}
For $w,v \in S_n$, we have
$$c(w;v)= \psi( w * \mathfrak{S}_v).$$
\end{thm}
Note that assuming Conjecture \ref{posfk}, we have
$$c(w;v)= \psi( w * \mathfrak{S}_v) \geq \psi(w \cdot\mathfrak{S}_v) =\sum_u p^w_{u,v}$$
by Theorem \ref{lr} and the fact that $w \cdot\mathfrak{S}_v$ is the degree $\ell(w)-\ell(v)$ part of $w * \mathfrak{S}_v$. Therefore, Conjecture \ref{posfk} guarantees Conjecture \ref{refined} for type A. Since the term $\sum_u p^w_{u,v}$ is strictly positive when $v\leq w$, Conjecture \ref{poscsm} also follows. In fact, the nonnegativity conjecture is much stronger than positivity of the CSM classes of Schubert cells. Note that it is well-known that for $v \leq w$, there exists $u$ such that $p^w_{u,v}$ is positive. \\

The proof of Theorem \ref{main} will be given in Section 7, after describing the relation between the CSM classes and the generalized nil-Hecke algebra. We discuss applications of Theorem \ref{main} in this section. \\

\begin{example}
For $\ell(w)=\ell(v)+1$, we have
$$c(w;v)= \psi( w * \mathfrak{S}_v)=\psi( w \cdot \mathfrak{S}_v)=\sum_u p^w_{u,v}>0.$$
\end{example}
\begin{example}
Consider $v=s_{i_1}\ldots s_{i_k}$ when $s_{i_j}$ pairwise commutes. Then
$$\mathfrak{S}_v=\prod_{j=1}^k \mathfrak{S}_{s_k} = \prod_{j=1}^k \sum_{a=1}^j \theta_a,$$
which is the same as
$$\prod_{j=1}^k \left(\sum_{x\leq j <y} [xy]\right).$$
This provides a positive formula for $c(w;v)$ for any $w$ and $v=s_{i_1}\ldots s_{i_k}$ such that $s_{i_j}$ pairwise commutes.
\end{example}
\begin{example} \cite{Pos99} For $1\leq a\leq b\leq n$, we have
$$\mathfrak{S}_{w_{a,b}}= \sum [x_1, y_1] \ldots [x_{b-a+1} y_{b-a+1}]$$
where the sum runs over all sequences $x_1,\ldots x_{b-a+1},y_1,\ldots,y_{b-a+1}$ satisfying (i) $x_i\leq b <y_i$; (2) $x_i$ are all distinct; (3) $y_1\leq \cdots \leq y_{b-a+1}$. It provides a positive formula for $c(w;v)$ for any $w$ and $v=w_{a,b}$.
\end{example}

See Remark \ref{cases} for $v \in S_n$ such that the nonnegativity conjecture of $\mathfrak{S}_v(\theta)$ is known. \\

It seems that conjecturally, we have a positive refinement of $c(w;v)$: for $w,v,u \in W$ there are nonnegative integers
$$f(w,v,u)= \text{ coefficient of }u \text{ in } w*\mathfrak{S}_v,$$
such that $f(w,u,v)=p^w_{u,v}$ when $\ell(w)=\ell(u)+\ell(v)$, and $c(w;v)=\sum_u f(w,v,u)$. The nonnegativity suggests that $f(w,v,u)$ are nonnegative. One natural question is whether the constants $f(w,v,u)$ have a meaning in algebraic geometry or reperesentation theory. In the next two sections, we show that $f(w,v,u)$ is a certain sum of structure constants of the cohomology of the Bott-Samelson variety.

\section{Generalized nil-Hecke algebra}
In this section, we discuss nil-Hecke algebra studied by Kostant and Kumar \cite{KK86} and generalized nil-Hecke algebra studied by Berenstein and Richmond \cite{BR15}. \\

\subsection{The nil-Hecke algebra}

Recall that $\alpha_1,\ldots,\alpha_r$ are the positive simple roots of $G$. Let $A=(a_{ij})_{i,j \in I}$, where $I$ is the set $\{1,2,\ldots,r\}$ and $a_{ij}=\langle \alpha_i , \alpha^\vee_j\rangle$. Let $S$ be the symmetric algebra of the root lattice, which is isomorphic to $H^*_T(X)$. Let $F=Frac(S)$ be the fraction field of $S$. There is a Weyl group action on $F$ given by $s_i(\alpha_j)=\alpha_j- a_{ij} \alpha_i$ and $w(q_1 q_2)=w(q_1) w(q_2)$ for $w \in W, q_1,q_2 \in S$. Let $S_W$ be the smash product $F\rtimes \mathbb{Q}[W]$ subject to the relation:
\begin{equation} \label{wq} wq= w(q) \cdot w\end{equation}
for all $q\in S, w\in W$. There is a coproduct structure in $S_W$ given by
\begin{equation}\label{coproduct}
\Delta(qw)= qw \otimes w.
\end{equation}
The coproduct $\Delta: S_W \rightarrow S_W \otimes_S S_W$ is also a homomorphism of algebras. Note that the multiplication on $S_W \otimes_S S_W$ is component-wise product: for $a,b,c,d \in S_W$, we have $(a\otimes b)\cdot(c\otimes d)=ac \otimes bd$. \\

For $i\in I$, define an element $x_i \in S_W$ by $x_i:= \alpha_i^{-1} (s_i-1)$. The nilHecke algebra $\mathbb{A}$ is a subalgebra of $S_W$ generated by $x_i$ for all $i\in I$ and $S$. Let $w \in W$ and let $w=s_{i_1}\ldots s_{i_m}$ be a reduced expression of $w$. Then the element $x_{i_1}\ldots x_{i_m}$ does not depend upon the choice of reduced expression of $w$ by \cite[Prop 4.2]{KK86}. We define $x_w$ as $x_{i_1}\ldots x_{i_m}$. Then $\{x_w\}_{w \in W}$ is a left (as well as right) $S$-basis of $\mathbb{A}$.\\

There is an induced coproduct on $\mathbb{A}$ given by
\begin{equation}\label{coproduct}
\Delta(x_i)= x_i \otimes 1 + s_i \otimes x_i = x_i \otimes 1+ (1+\alpha_i x_i) \otimes x_i.
\end{equation}

It turns out that structure constants of the coproduct is the same as the structure constants of the equivariant cohomology of the flag variety $X=G/B$.

\begin{thm} \cite{KK86} For $w,u,v \in W$, Let $\sigma_w$ be the Schubert basis in $H^*_T(X)$ Poincare dual to $[X(w)]$. Let $p^{w}_{u,v}$ be the (equivariant) Littlewood Richardson coefficients, i.e., constansts satisfying $\sigma_u \sigma_v = \sum_w p^w_{u,v} \sigma_w$. Then we have
$$\Delta(x_w)= \sum_{u,v \in W} p^{w}_{u,v} x_u \otimes x_v.$$
\end{thm}

The follwing lemma will be useful when computing $\Delta(x_w)$.
\begin{lemma}\label{xi} For $i \in I$ and $\lambda \in \mathfrak{h}^*$, we have
$$x_i \lambda = s_i(\lambda) x_i - \langle \lambda, \alpha^\vee \rangle$$
\end{lemma}
\begin{proof} By Equation (\ref{wq}) for $w=s_i$ and $q=\lambda \in \mathfrak{h}^* \subset S$, we have
$$(1+\alpha_ix_i) \lambda= s_i(\lambda) (1+\alpha_i x_i).$$
By using $s_i(\lambda)=\lambda-\langle \lambda, \alpha^\vee \rangle \alpha_i$ and simplyfing, the lemma follows. \end{proof}
\subsection{The generalized nil-Hecke algebra}

Let $\hat W$ be the free monoid generated by $s_1,\ldots, s_r \in W$. There is an surjective homomorphism of monoids $\mu: \hat W \rightarrow W$ sending $s_i$ to $s_i$. The action of $\hat W$ on $S$ is induced by $\mu$. The generalized nil-Hecke algebra $\hata$ is the smash product $S \rtimes \mathbb{Q}[\hat W]$. For $\bfi=({i_1},\ldots, {i_m}) \in I^m$, define $x_\bfi$ by $x_{i_1}\ldots x_{i_m}$. Then the set $\{ x_\bfi \mid \bfi \in I^m \text{ for } m\geq 0 \}$ forms a $S$-basis of $\hata$. Note that $\mu$ induces the surjection $\hata \rightarrow \mathbb{A}$, denoted by the same $\mu$. Let $w_\bfi$ be the element $s_{i_1}\ldots s_{i_m}$ in $W$ and let $\ell(\bfi)$ be $m$ for $\bfi \in I^m$. It is obvious from the definition that $\ell(\bfi)\geq \ell(w_\bfi)$, and equality holds if and only if $s_{i_1}\ldots s_{i_m}$ is a reduced word of $w_\bfi$, and we call $\bfi$ reduced. The surjection $\mu$ sends $x_\bfi$ to $x_{w_\bfi}$ if $\bfi$ is reduced, 0 otherwise.\\

The generalized nil-Hecke algebra also has a coproduct structure induced by (\ref{coproduct}), and the structure constants is the same as the structure constants of the $T$-equivariant cohomology of the Bott-Samelson varieties. We follow the notations in \cite{BR15} to review some results about Bott-Samelson variety. \\

 For $\bfi=(i_1,\ldots,i_m) \in I^m$ and a subset $K=\{m_1<\ldots<m_k\}$ of $[m]$, denote $\bfi_K$ the subsequence $(i_{m_1},\ldots, i_{m_k}) \in I^k$ of $\bfi$. For each $\bfi=(i_1,\ldots,i_m) \in I^m$, the $\bfi$-th Bott-Samelson variety $\Gamma_\bfi=\Gamma_\bfi(G)$ of $G$ is defined by:
$$\Gamma_\bfi = \big( P_{i_1} \times_B P_{i_2} \times_B \ldots \times_B P_{i_m} \big) /B $$
where $P_i, i\in I$ stands for the $i$-th minimal parabolic subgroup. It is well-known that $T$-equivariant cohomology $H^*_T(\Gamma_\bfi)$ has a $S$-basis $\{\sigma^T_K\}$, where $K$ runs over all subsets of $[m]$. The equivariant Bott-Samelson coefficients $p^{\bfi,K}_{K',K''} \in S$ are defined by
$$\sigma^T_{K'} \sigma^T_{K''}= \sum p^{\bfi,K}_{K',K''} \sigma^T_K,$$
where the summation runs over all subset $K\subset [m]$ such that $K' \cup K'' \subset K$ and $|K| \leq |K'|+|K''|$. The structure constants of the ordinary cohomology of the Bott-Samelson varieties are $p^{\bfi,K}_{K',K''}$ with $|K|=|K'|+|K''|$ and these constants are integers, but not necessarily nonnegative integers.
\\

The following result was proved by H. Duan \cite{Duan} for the ordinary cohomology and by M. Willems \cite{Wil04} in the equivariant setting.

\begin{thm} \label{mu} Let $G$ be a Kac-Moody group and $W$ be its Weyl group. Then for any sequence $\bfi=(i_1,\ldots,i_m) \in I^m$ one has:
\begin{enumerate}
\item The pullback of the canonical projection $\mu_\bfi : \Gamma_\bfi \rightarrow G/B$ is an algebra homomorphism $\mu^*_\bfi: H^*_T(G/B)\rightarrow H^*_T(\Gamma_\bfi)$ given by
$$\mu^*_\bfi(\sigma^T_w)= \sum_{K \subset [m], \bfi_K \in R(w)} \sigma^T_K$$
for all $w\in W$ (If there is no $K$ satisfying $\bfi_K\in R(w)$, $\mu^*_\bfi(\sigma^T_w)=0$). Here $R(w)$ is the set consisting of all reduced words of $w$.
\item If $\bfi \in R(w)$ for some $w\in W$, then for any $u,v \in W$, the equivariant Littlewood-Richardson and Bott-Samelson coefficients are related by:
$$ p^w_{u,v} = \sum p^{\bfi,[m]}_{K',K''},$$
where the summation runs over all subsets $K',K'' \subset [m]$ such that $\bfi_{K'} \in R(u), \bfi_{K''} \in R(v)$.
\end{enumerate}
\end{thm}

Let $\mu : \hata \rightarrow \mathbb{A}$ be the map sending $A_i$ to $A_i$. In other words, $\mu$ sends $A_\bfi$ to $A_{w_\bfi}$ if $\ell(\bfi)=\ell(w_\bfi)$, and 0 otherwise. One can define the coproduct $\Delta : \hata \rightarrow \hata \otimes_S \hata$ defined by the equation $(4)$ and $\Delta(ab)=\Delta(a)\Delta(b)$. Note that the definition is the same as the coproduct on $\mathbb{A}$, so that we have $\Delta\circ \mu = (\mu\otimes \mu)\circ \Delta$.
For words $\bfi,\bfi', \bfi''$, let $p^{\bfi}_{\bfi', \bfi''}$ be an element in $S$ satisfying
\begin{align}\label{delta} \Delta(x_\bfi)= \sum p^{\bfi}_{\bfi', \bfi''} x_{\bfi'} \otimes x_{\bfi''}.\end{align}
 In \cite{BR15}, those elements are called \emph{relative} (equivariant) Littlewood-Richardson coefficients. The relative equivariant Littlewood-Richardon coefficients and the equivariant Bott-Samelson coefficients are related by the following theorem.

\begin{thm} \label{relative} \cite{BR15} For words $\bfi,\bfi', \bfi''$, we have
$$p^{\bfi}_{\bfi', \bfi''}=\sum p^{\bfi,[m]}_{K',K''}$$
where the sum runs over all pairs $K',K'' \subset [m]$ such that $\bfi_{K'}=\bfi',\bfi_{K''}=\bfi''$.
\end{thm}

Note that one can compute $p^{\bfi,[m]}_{\bfi', \bfi''}$ by using (\ref{coproduct}).
\begin{thm}\label{compute}
For $K' \subset [m]$ and $j \in [m]$, let $E_{\bfi,K'}(j)$ be $s_{i_j}$ if $j \in K'$ and $A_{i_j}$ otherwise. Then $p^{\bfi}_{\bfi', \bfi''}$ is the coeeficient of $x_{\bfi''}$ in $\sum_{K'}\prod_{j=1}^m E_{\bfi,K'}(j)$,
where the sum runs over all $K'$ satisfying $\bfi_{K'}=\bfi'$.
\end{thm}
\begin{proof} It directly follows from (\ref{coproduct}) and the fact that $\Delta$ is an algebra homomorphism. \end{proof}

\begin{example}
Let $\bfi=(1,2,1)$ and $\bfi' =(1,1)$. so that there is only one $K'=\{1,3\}$ satisfying $\bfi_{K'}=\bfi'$. Consider $\prod_{j=1}^3 E_{\bfi,K'}(j)=s_1A_2s_1$.\\
\begin{align*}
s_1A_2s_1&=(1+\alpha_1 A_1) A_2 (1+\alpha_1 A_1)\\
&=A_2+\alpha_1 A_{1,2}+ A_2 \alpha_1 A_1+ \alpha_1 A_{1,2} \alpha_1 A_1 \\
&=  A_2+\alpha_1 A_{1,2}+A_1+ (\alpha_1+\alpha_2) A_{2,1}+ \alpha_1 A_{1,1} - \alpha_1 A_{2,1} + \alpha_1\alpha_2 A_{1,2,1}\\
&= A_2+\alpha_1 A_{1,2}+A_1+\alpha_2 A_{2,1}+ \alpha_1 A_{1,1}+ \alpha_1\alpha_2 A_{1,2,1}
\end{align*}
Note that we frequently used Lemma \ref{xi}. The above computation shows that, for example, $p^{121}_{11,21}=\alpha_2$ by Theorem \ref{compute}.

\end{example}
\section{Relationship between the CSM classes and the equivariant Bott-Samelson coefficients}
The goal of this section is to relate $c_T(w;v)$ with the equivariant Bott-Samelson coefficients $p^{\bfi,[m]}_{K',K''}$. \\

Let $\phi$ the isomorphism between the equivariant homology of the flag variety $H^T_*(X)$ and the nilHecke algebra $\mathbb{A}$ as $S$-modules sending $[X_w]^T$ to $x_i \in \mathbb{A}$. Then the Weyl group action and the divided difference operator $\partial_k$ can be purely written in the nilHecke algebra.

\begin{lem} \label{translate} For $k \in I$ and $\xi \in H^T_*(X)$, we have
\begin{align*}
\phi( \partial_k \xi )&= \phi(\xi) x_k,\\
\phi( s_k \xi )&= -\phi(\xi)s_k,\\
\phi( T_k \xi )&= \phi(\xi)(x_k+s_k).
\end{align*}
In the last two equations, the $s_k$ on the left-hand side is the Weyl group action on $H^T_*(X)$, but the $s_k$'s on the right-hand side are an element in $\mathbb{A}$.
\end{lem}
\begin{proof} It is enough to show for $\xi=[X_w]^T$. Then the first equation in the lemma is obvious, and the proof of the second equiation can be checked by comparing the Weyl group action and $\partial_k$ used in Allufi and Mihalcea's paper \cite{AM15} and Kumar's book \cite[(11.1.2)]{Kum02}. The last equation follows by the first two equations and Theorem \ref{rec}.
\end{proof}\\

 By Theorem \ref{rec} and Lemma \ref{translate}, we have the following result.
\begin{thm}\label{cwv}
Fix a reduced word $s_{i_1}\ldots s_{i_m}$ of $w$. Then $c_T(w;v)$'s are the coefficients of $x_v$ in the product $\prod (s_{i_j}+x_{i_j}) \in \mathbb{A}$:
$$\prod_{i=1}^m (s_{i_j}+x_{i_j})= \sum_v c_T(w;v) x_v$$
where the notation $\prod_{i=1}^m a_i$ means $a_1 a_2 \ldots a_m$.
\end{thm}

Fix a word $\bfi=(i_1,\ldots,i_m) \in R(w)$ and consider the same product $\prod (s_{i_j}+x_{i_j})$ as an element in $\hata$. Define the map $pr: \hata \otimes_S \hata \rightarrow \hata$ defined by $pr(\sum f_{\bfi,\bfi'} x_\bfi \otimes x_{\bfi'})= \sum f_{ij}x_\bfi$. Note that the map $pr$ is an algbra homomorphism. Indeed, we have
\begin{align*} pr\big((\sum_{\bfi,\bfi'} f_{\bfi,\bfi'} x_\bfi \otimes x_{\bfi'})(\sum_{\bfj,\bfj'} g_{\bfj,\bfj'} x_\bfj \otimes x_{\bfj'})\big)&= \sum_{\bfi,\bfi',\bfj,\bfj'} f_{\bfi,\bfi'}g_{\bfj,\bfj'} x_{\bfi \cup \bfj}\\&= pr\big(\sum_{\bfi,\bfi',\bfj,\bfj'}  f_{\bfi,\bfi'}g_{\bfj,\bfj'} x_{\bfi \cup \bfj}\otimes x_{\bfi' \cup \bfj'}\big),\end{align*} where $\bfi \cup \bfj$ is the word joining two words $\bfi, \bfj$. \\

Since $\Delta(x_i)= x_i \otimes 1 + s_i \otimes x_i$, we have $pr\circ \Delta(x_i)=x_i+s_i$. Therefore, we have the identity
\begin{align} \label{sx}\prod (s_{i_j}+x_{i_j})= pr\circ \Delta(x_\bfi)\end{align}
in $\hata$. Now we are ready to prove the following theorem.

\begin{thm} \label{ct} For $w \in W$ with $\ell(w)=m$, fix $\bfi \in R(w)$. Then for $v\in W$,
$$c_T(w;v)= \sum_{K,K' \subset [m], \bfi_K \in R(v)} p^{\bfi,[m]}_{K,K'}.$$
\end{thm}
\begin{proof}
By Theorem \ref{cwv} and (\ref{sx}), we have
$$\sum_v c_T(w;v) x_v= \mu\circ pr \circ \Delta(x_\bfi).$$
By (\ref{delta}), we have
$$\mu\circ pr \circ \Delta(x_\bfi)= \mu (\sum_{\bfi',\bfi''} p^{\bfi}_{\bfi',\bfi''} x_{\bfi'}).$$
Therefore, $c(w,v)$ is the same as $\sum p^{\bfi}_{\bfi',\bfi''}$ where the sun runs over $\bfi' \in R(v)$ and any words $\bfi''$. By Theorem \ref{relative}, the theorem follows.
\end{proof}

\begin{rmk}Note that the same map $pr$ is not an algebra homomorphism if we used the nil-Hecke algebra instead of $\hata$. For instance, $(1\otimes x_1)(1\otimes x_1)$ is 0 in the usual nil-Hecke algebra so that $$0=pr((1\otimes x_1)(1\otimes x_1)) \neq pr(1\otimes x_1)\cdot pr(1\otimes x_1)=1.$$
In $\hata$, they are all equal to 1.
\end{rmk}

\section{Relation with Fomin-Kirillov algebra.}
In this section, we define a certain algebra using Fomin-Kirillov algebra $\fk$ and $\mathbb{Q}[\hat W]$ containing the generalized nilHecke algebra for type $A$. Let $r=n-1$, i.e., $\hat W$ is the free monoid generated by $s_1,\ldots, s_{n-1}$. We relate this with the extended Bruhat action on $\fk$ to prove a stronger version of Theorem \ref{main}. Note that we will be working on type $A$ non-equivariantly and $W$ is the symmetric group $S_n$.\\

Let $\mathcal C$ be the subalgebra of the Fomin-Kirillov algebra generated by Dunkl elements $\theta_i$ for $i=1,\ldots,n-1$. Let $\mathcal D$ be the subalgebra of $\mathcal C$ generated by $\theta_i-\theta_{i+1}$ for $i=1,\ldots,n-1$.
Let $\mathcal B$ be the smash product $\fk \rtimes \mathbb{Q}[\hat W]$ subject to the relation:
\begin{equation} \label{kl} x_i [kl]= s_i([kl]) x_i + \delta'_{(i,i+1),(k,l)},\end{equation}
where $\delta'_{(i,i+1),(k,l)}$ is 1 if $i=k,i+1=l$, -1 if $i=l,i+1=k$, 0 otherwise. \\

Fomin and Kirillov \cite{FK99} defined an operator $\Delta_{ij}$ on $\fk$ defined by $\Delta_{ij}([kl])=\delta'_{(i,j),(k,l)}$ and
\begin{align*}
\Delta_{ij}([x_1y_1][x_2y_2] \ldots [x_m y_m])&= \Delta_{ij}([x_1 y_1]) [x_2y_2] \ldots [x_m y_m]\\
&+ s_{ij}([x_1y_1]) \Delta_{ij}([x_2 y_2]) [x_3 y_3] \ldots [x_m y_m]\\
&+ \ldots\\
&+ s_{ij} ([x_1y_1] \ldots [x_{k-1} y_{k-1}]) \Delta_{ij}([x_m y_m]).
\end{align*}
By Equation (\ref{kl}), for $k \in I$ and an element $p$ in $\fk$ we have
\begin{equation}\label{xkp} x_k p= s_k(p) x_k + \Delta_{k,k+1}(p).\end{equation}
Indeed, it is enough to show the identity for $p=[x_1y_1][x_2y_2] \ldots [x_m y_m]$ and in this case the identity follows from the definition.\\

It is proved by Fomin and Kirillov that the action $\Delta_{k,k+1}$ is compatible with the divided difference operator $\partial_k$ acting on $H^*(X)$.
\begin{thm} \cite{FK99}
Let $i_{\mathcal C}$ be the isomorphism between $\mathcal C$ and $H^*(X)$. For $1\leq k \leq n$ and $w \in S_n$, we have
$$ i_{\mathcal C} (\partial_k \mathfrak{S}_w(\theta) )= \Delta_{k,k+1}\circ i_{\mathcal C}( \mathfrak{S}_w(\theta)).$$
\end{thm}

Define an algebra map $d: \mathbb{Q}[\hat W] \rightarrow \mathbb{Q}[W]$ defined by
$$d(x_\bfi)= w_\bfi.$$

The algebra $\mathcal B$ and the extended Bruhat action on the Fomin-Kirillov algebra is related by the following lemma.
\begin{lemma}\label{dx}
Let $\bfi$ be a word with $w_\bfi=w \in S_n$. Consider the expansion
$$ x_\bfi [kl] - w_\bfi ([kl]) x_\bfi = \sum_{\bfi'} c_{\bfi'}x_{\bfi'}$$
by successively applying (\ref{kl}) to $x_\bfi [kl]$, where $c_{\bfi'} \in \mathbb{Z}$ and $k<l$. Then
$$d( \sum_{\bfi'} c_{\bfi'}x_{\bfi'})=  \begin{cases} w_\bfi t_{kl}, & \text{if } \ell(w_\bfi t_{kl})<\ell(w_\bfi) \\ 0, & \text{otherwise.} \end{cases}$$
\end{lemma}
\begin{proof}
Let $\bfi = (i_1,\ldots,i_m)$ and let $t_j$ an element in $S_n$ defined by $s_{i_{j+1}}s_{i_{j+2}}\ldots s_{i_m}$. By using (\ref{kl}), we have
\begin{align*}
 x_\bfi [kl] - w([kl]) x_\bfi = \sum_{j=1}^m \delta'_{(i_j,i_j+1), (t_j(k),t_j(l))} x_{i_1}\ldots x_{i_{j-1}}\widehat{x_{i_j}}x_{i_{j+1}} \ldots x_{i_m},
\end{align*}
where $j$-th summand omits $x_{i_j}$. Denote $\bfi'_j$ by $x_{i_1}\ldots x_{i_{j-1}}\hat{x_{i_j}}x_{i_{j+1}} \ldots x_{i_m}$.\\

Note that $\delta'_{(i_j,i_j+1), (t_j(k),t_j(l))}$ is nonzero if and only if $w_{\bfi'_j}=wt_{kl}$ since we have $s_{i_j}t_j=t_j t_{t_j(i_j),t_j(i_j +1)}$. Let $V$ denote the set of all $j$ such that $\delta'_{(i_j,i_j+1), (t_j(k),t_j(l))}$ is nonzero. One can show inductively that $\delta'_{(i_j,i_j+1), (t_j(k),t_j(l))}=1$ (resp. $=-1$) if and only if there are even (resp. odd) numbers in $V$ greater than $j$. Therefore, $d(x_\bfi [kl] - w([kl]))= w_\bfi t_{kl}$ if the cardinarity of $V$ is odd, 0 otherwise. Whether $|V|$ is odd or even can be checked by considering $w([kl])$. Indeed, $w([kl])=[ab]$ (resp. $=-[ab]$) for some $a<b$ if and only if $|V|$ is even (resp. odd). Since it is well-known that $w(k)>w(l)$ if and only if $\ell(w)>\ell(wt_{kl})$, we are done (Such pair $(k,l)$ is called an inversion of $w$. See \cite{BB05} for instance).
\end{proof}\\

Since we are working non-equivariantly, define the map $ev: \fk \rightarrow \mathbb{Q}$ by sending all $[ij]$ to zero, and 1 to 1. This induces a map $ev: \fk \rtimes \mathbb{Q}[W]\rightarrow  \mathbb{Q}[ {W}]$. Then Lemma \ref{dx} implies that
\begin{equation} \label{dev1} ev(d(x_\bfi [kl]))= w* [kl]. \end{equation}

\medskip

We consider subalgebras of $\mathcal B$, $\mathcal C \rtimes \mathbb{Q}[\hat W]$ and $\mathcal D \rtimes \mathbb{Q}[\hat W]$. From (\ref{kl}), we have
\begin{equation}\label{theta} x_i \theta_j = \begin{cases} \theta_j x_i & \text{ if } j\neq i,i+1 \\ \theta_{i+1} x_i + 1 & \text{ if } j=i \\ \theta_i x_i - 1 & \text{ if} j=i+1 \end{cases}\end{equation}
By (\ref{theta}), Any element in $\mathcal C \rtimes \mathbb{Q}[\hat W]$ can be written as $\sum_\bfi p_\bfi(\theta) x_\bfi$ where $p_\bfi(\theta)$ is a polynomial in Dunkl elements $\theta_i$. Similarly, any element in $\mathcal D \rtimes \mathbb{Q}[\hat W]$ can be written as $\sum_\bfi q_\bfi x_\bfi$ where $q_\bfi$ is a polynomial in $\theta_i-\theta_{i+1}$ for $1\leq i <n$.\\

Let $g$ be an surjective algebra map from $S$ to $\fk$ defined by $g(\alpha_i)=\theta_{i+1}-\theta_{i}$ so that the image of $g$ is $\mathcal D$. Recall that the symmetric functions in $\theta_i$ for $1\leq i \leq n$ is 0 in the Fomin-Kirillov algebra so that $g$ is not injective. The map $g$ induces a map from the generalized nil-Hecke ring $S \rtimes \mathbb{Q}[\hat W]$ to $\fk \rtimes \mathbb{Q}[\hat W]$, denoted by $g$ by abusing the notation. Indeed, it is enough to show that relations defining two smash products are compatible with the map $g$: $x_\bfi g(\alpha_i) = g(x_\bfi \alpha_i)$ for a word $\bfi$.\\

For a root $\lambda$ in $S$, we have by definition $x_i \lambda = s_i(\lambda) x_i$ as an identity in $S \rtimes \mathbb{Q}[\hat W]$. By using $s_i= 1+\alpha_i x_i$, we have
$$x_i \lambda = {s_i(\lambda)-\lambda \over \alpha_i}+ s_i(\lambda)x_i=-\langle \lambda, \alpha_i^\vee \rangle+ s_i(\lambda)x_i.$$
By setting $\lambda =\alpha_i$ and comparing with (\ref{theta}), we get the desired equality $x_\bfi g(\alpha_i) = g(x_\bfi \alpha_i)$.\\

The following theorem shows the relationship between the Fomin-Kirillov algebra and generalized nil-Hecke algebra.\

\begin{thm}\label{compute}
For $K' \subset [m]$ and $j \in [m]$, recall that $E_{\bfi,K'}(j)$ is $s_{i_j}$ if $j \in K'$ and $A_{i_j}$ otherwise. Then for a word $\bfi$ and $v\in W$, we have
$$x_\bfi \mathfrak {S}_v(\theta)- \mathfrak {S}_v(\theta) x_\bfi= g(\sum_{K'} \prod_{j=1}^m E_{\bfi,K'}(j))$$
where the sum runs over all $K'$ with $i_{K'}\in R(u)$.

\end{thm}
\begin{proof} We use an induction on $\ell(\bfi)$. Let $\bfi= \bfi' \cup \{k\}$, where $k$ is the last element in $\bfi$. Then one can write
$$x_\bfi \mathfrak {S}_v(\theta)=x_{\bfi'} x_i \mathfrak {S}_v(\theta).$$
By Equation (\ref{xkp}), we have
\begin{align*}
x_{\bfi'} x_k \mathfrak {S}_v(\theta)&= x_{\bfi'} s_k(\mathfrak {S}_v(\theta))+ x_{\bfi'}(\partial_k\mathfrak {S}_v(\theta))\\
&=x_{\bfi'} \mathfrak {S}_v(\theta) x_k + x_{\bfi'}(\partial_k\mathfrak {S}_v(\theta))(1+ (\theta_{k+1}-\theta_k) x_k)\\
&=x_{\bfi'} \mathfrak {S}_v(\theta) x_k + x_{\bfi'}(\partial_k\mathfrak {S}_v(\theta))(1+ g(\alpha_i) x_k).
\end{align*}
From the first line to the second line, we used the identity $f-s_k(f)= \partial_k(f) (\theta_k-\theta_{k+1})$ for a polynomial $f$ in $\theta_1,\ldots,\theta_n$ by the definition of $\partial_k$. By applying the induction hypothesis on $x_{\bfi'} \mathfrak {S}_v(\theta)$ and $ x_{\bfi'}(\partial_k\mathfrak {S}_v(\theta))$, the theorem follows.

\end{proof}\\

By Theorem \ref{compute}, we have the following corollary.

\begin{cor}\label{gx2} For a word $\bfi$ and $v\in W$, we have

$$x_\bfi \mathfrak {S}_v(\theta)- \mathfrak {S}_v(\theta) x_\bfi= g(\sum p^\bfi_{\bfi',\bfi''}x_{\bfi''})$$
where the sum runs over all $\bfi',\bfi''$ such that $\bfi' \in R(v)$.
\end{cor}

By applying $ev\circ d$ on Theorem \ref{gx2}, we have
\begin{align*}
w*\mathfrak {S}_v(\theta)&= ev(d(x_\bfi\mathfrak {S}_v(\theta)) \quad \text{(By Equation (\ref{dev1}))}\\
&= ev(d(x_\bfi\mathfrak {S}_v(\theta)-\mathfrak {S}_v(\theta) x_\bfi)\\
&= ev(d(g(\sum_{\substack{\bfi',\bfi'' \\ \bfi' \in R(v)}} p^\bfi_{\bfi',\bfi''}x_{\bfi''}))).
\end{align*}
Note that the composition $ev\circ d \circ g$ is the map from $\hata= S\rtimes \mathbb{Q}[\hat W]$ to $\mathbb{Q}[W]$, sending $\sum_\bfi  f_\bfi x_\bfi$ to $\sum_{\substack{ \bfi \text{ reduced} }} i(f_\bfi) w_\bfi$ where $i$ is the projection map from $S$ to $\mathbb{Q}$. Note that $p^\bfi_{\bfi',\bfi''}$ is a homogeneous polynomial in $\alpha_i$'s of degree $\ell(\bfi')+\ell(\bfi'')-m$ so that $p^\bfi_{\bfi',\bfi''}$ is an integer if and only if $\ell(\bfi')+\ell(\bfi'')=m$ (See \cite{BR15,Wil04} for instance). Therefore, we proved the following theorem.

\begin{thm} For $w,v \in W$, we have
$$w*\mathfrak {S}_v(\theta)=\sum_{\substack{\bfi',\bfi'' \\ \bfi' \in R(v), w_{\bfi''}=u\\\ell(\bfi')+\ell(\bfi'')=m}} p^{\bfi}_{\bfi',\bfi''} u.$$
\end{thm}
As a corollary, we obtain the following interpretation of $f(w,v,u)$.
\begin{cor}\label{fwvu}
For $w \in W$ with $\ell(w)=m$, fix $\bfi \in R(w)$. Then for $u,v\in W$, we have
$$f(w,v,u)= \sum_{\substack{K,K' \subset [m] \\ |K|+|K'|=m\\ \bfi_K \in R(v), w_{\bfi_K'} = u} } p^{\bfi,[m]}_{K,K'}.$$
\end{cor}
We end the section by proving Theorem \ref{main}.\\
{\it Proof of Theorem \ref{main}.} Recall that by Theorem \ref{ct}, for $w,v \in W$ with $\ell(w)=m$ with a fixed $\bfi \in R(w)$, we have
$$c_T(w;v)= \sum_{K,K' \subset [m], \bfi_K \in R(v)} p^{\bfi,[m]}_{K,K'}.$$
By applying the map $i:S\rightarrow \mathbb{Q}$, we have
$$c(w;v)= \sum_{\substack{K,K' \subset [m], \bfi_K \in R(v)\\|K|+|K'|=m}} p^{\bfi,[m]}_{K,K'}.$$
Since $c(w,v)=\sum_u f(w,v,u)$ by the definition of $f(w,v,u)$, Theorem follows by Corollary \ref{fwvu}. \qed

\section{Concluding remark}
Although we provided an intepretation of $f(w,v,u)$ in terms of equivariant Bott-Samelson constants, it is not clear why $f(w,v,u)$ or $c(w;v)$ is nonnegative. Note that the Littlewood-Richardson coefficients $p^w_{u,v}$ follows from Kleinman's transversality theorem \cite{Kle74}, and the equivariant version was established by Graham \cite{Gra01}. It would be very interesting to relate the work in this paper and Huh's proof \cite{Huh} of the positivity of $c(w,v)$ when $w,v$ are elements in the $W/W_P$, where $G/P$ is the Grassmannian.\\

It would be also interesting to look for the representation theoritic meaning of $c_T(w;v)$. For $w,v \in W$, the degree of $c_T(w;v)$ in $\alpha_i$'s is $\ell(v)$. Denote the homogeneous polynomial of degree $\ell(v)$ in $c_T(w;v)$ by $a(w,v)$. We prove the following theorem in this section.

\begin{thm} \label{top} For $w,v \in W$, $a(w,v)$ is the same as the localization of the Schubert class $\sigma^w$ at the point $v$, denoted by $\sigma^w(v)$. \end{thm}
$\sigma^w(v)$ has a representation meaning by Kumar \cite{Kum95} and it has a positive formula given by Billey \cite{Bil99}. It would be interesting to extend Kumar's work \cite{Kum95} to find a reperesentational meaning of $c_T(w;v)$.\\

{\it Proof of Theorem \ref{top}.} Introduce the $\mathbb{Z}$-degree $z$ on $\mathbb{A}$ by $z(x_i)=-1$ and $z(\alpha_i)=1$. Note that the degree map $z$ is well-defined since the equation in Lemma \ref{xi} is homogeneous with respect to the degree map $z$. Then the degree $0$ component of $\sum_v c_T(w,v) x_v $ with respect to $z$ is $\sum_v a(w,v) x_v$. On the other hand, for $w=s_{i_1}\ldots s_{i_m}$, the sum $\sum_v a(w,v) x_v$ is the degree $0$ part of the product $\prod_{i=1}^m (s_{i_j}+x_{i_j})$ where $w=s_{i_1}\ldots s_{i_m}$ by Theorem \ref{cwv}. Since $z(s_i)=0$ and $z(x_i)=-1$, the degree $0$ part is $\prod_{i=1}^m s_{i_j}=w$ in $\mathbb{A}$. Since it is well-known that $w= \sum_v \sigma^w(v) x_v$ (See \cite{Kum02} for example), we are done.\qed

\section*{acknowledgment}
I thank Leonardo Mihalcea for helpful discussions.

Combinatorics (math.CO); Algebraic Geometry (math.AG)