\section{Introduction}

In the last years advanced laser facilities have achieved intensities of
the order of $10^{15} W/cm^2$ and pulse lengths of the order of 10 fs., which corresponds
to few cycles of an electrical field of 800 nm wavelength \cite{nisoli, brabec00}. The process of interaction of such short and strong pulses with matter is a topic which has attracted much interest recently.

In the multiphoton regime, many experimental \cite{agostini, paulus02, hansch, nandor, wiehle} and theoretical studies \cite{mainfray, parker, proto, radha} have
been performed. In the tunneling regime, on the other hand, recent experiments \cite{rudenko, maharjan} with
linearly polarized lasers have shown structures in the momentum distribution of the
photoionized electrons in rare gases which have not been fully understood so far. The role of interference in few-cycle pulses is actually been both theoretically and
experimentally investigated \cite{paulus01, lindner, arbo1}.

Moreover, the combined laser and charged particle induced electronic
processes either in ion-atom or ion-surface collisions are of considerable interest
both for basic and applied sciences \cite{kirchner}. From the fundamental point of view
they might broaden our general understanding of the dynamics of atomic processes
for laser-matter interactions and field-free collisions.
As for the applications these studies can help us to
find the way for the control of ultrashort quantum processes
which are important in a number of applications, like in laser-driven fusion, in plasma heating, or in the
development of fast optical electronic devices.
The dynamics of atomic processes for the above mentioned interactions are not fully understood due to the lack of the exact and efficient theoretical models. For a detailed understanding of processes involved in the laser-matter interaction one needs to solve the time dependent Schr\"odinger equation (TDSE) for an atomic system in the radiation field, but its exact analytical solution is not known. Several numerical solutions of the Schr\"odinger equation for these kinds of systems are known \cite{cormier, chen, hansen, awasthi, chu}, but they are very time consuming for large systems and at high radiation intensities they converge slowly.
To overcome this problem there are several theoretical approaches, which are based on the simplification of the TDSE using different approximations depending on the laser field parameters.

At low and moderate laser field intensities the time dependent perturbation theory (TDPT) is a well suited approximation for single- and multi-photon processes and even for the above threshold ionization \cite{mainfray}. In this case the TDSE is solved by considering the interaction between the laser field and the studied atomic system as a perturbation. The TDPT approach breaks down at higher laser intensities when it fails to describe the "peak suppression" in the above threshold ionization spectra \cite{kruit, xiong}.

At higher laser field intensities other non-perturbative processes emerge like high harmonics generation, tunneling ionization (TI), over the barrier ionization (OBI), which can not be described using TDPT and other approaches are necessary.

Time independent, stationary approaches are developed based on the Floquet theory \cite{chu01}. In the framework of the generalized Floquet theory a stationary treatment of a periodic or quasiperiodic TDSE is possible. The TDSE is reduced to a set of time-independent coupled equations or into a Floquet matrix eigenvalue problem. Approaches involving the Floquet theory are applied successfully to describe multiphoton excitation, above threshold ionization and dissociation, but their main disadvantage is that in most applications the analytical form of the involved Floquet states is not known and their numerical calculation is also time consuming.

The most frequently used approaches are based on the Keldysh theory \cite{keldysh, reiss}. The Keldysh theory is based on the assumption that on the final state wavefunction only the external laser field has a dominant influence and it can be considered as a momentum eigenstate. The main shortcoming of the Keldys theory is that it completely neglects the long ranged Coulomb interaction between the ionized electron and remaining target ion. The Coulomb interaction leads to phenomena like sub-peaks in the above threshold ionization spectra \cite{freeman}, asymmetry in the spatial distribution of the ejected electron even for a symmetric few-cycle laser pulses \cite{chelk}, which can not be explained in the simplified Keldysh formalism. There are two possible ways of including the Coulomb interaction in the Keldysh formalism.  The first one is by making corrections in the transition matrix, and the second one is by making corrections in the Volkov wavefunction. Approaches based on the Keldysh theory using these corrections are applied with considerable success to study multiphoton and tunneling ionization of atomic systems \cite{reiss, popov}.

In the recent years the Coulomb-Volkov (CV) wavefunctions were used to describe processes in the presence of intense ultrashort laser field \cite{ducha01, ducha02, gayet}. The Coulomb-Volkov wavefunctions were introduced by Jain and Tzoar in 1978 \cite{tzoar} to describe laser assisted collisions. Later on they were successfully applied to describe multiphoton and above threshold ionization \cite{rodriguez, gayet}. The Coulomb-Volkov wavefunctions are also used to study ionization in tunneling and in over the barrier regime in the framework of sudden approximation \cite{ducha01, ducha02}. This model is called Coulomb-Volkov model

The accuracy of the results provided by the CV model is limited by the sudden approximation (it provides accurate results only if the pulse duration is less then two orbital period of the active electron \cite{ducha02}) and by the fact that the CV wavefunctions are only an approximate solution of the TDSE for charged particle in the presence of external radiation field.
From these two main limitations the first one is more restrictive, because it is limiting to the pulse duration in the attosecond region.
Our goal is to construct a theoretical model, which provides results as good as or better than the CV model, but which does not have the limitation in pulse duration.

Our approach is based on the approximate solution of the TDSE for quantum systems with one active electron, where the Coulomb interaction between the electron and the remaining target ion can be considered as a perturbation during the external laser pulse. The time dependent wavefunction of the active electron is expanded in terms of Volkov wavefunctions. The equation for the expansion coefficients obtained from the TDSE is solved with this approximation.

The ionization process in the over the barrier regime is considered to be a classical one and it is believed that it can be described very well by classical models like classical trajectory Monte Carlo (CTMC) \cite{olson, ducha02, hansen}.

We present double differential ionization cross sections for different laser pulses using the present, CTMC and Volkov models. The ionization probability densities are presented as a function of the electron energy and ejection angle.

In several works \cite{mu, mishima, zhang} the effect of the Coulomb interaction on the ionization of the hydrogen atom by ultrashort laser pulses at different field intensities using different approaches is studied. In these studies the effect of the Coulomb potential is studied by employing different types of final state wavefunctions like the Coulomb-Volkov wavefunction \cite{zhang}, or first and second order Coulomb corrected Volkov wavefunctions \cite{mishima}, or by employing Coulomb corrections on the Keldysh-Faisal-Reiss theory \cite{mu}. Moreover, the effect of the Coulom potential on the overall
ionization process is also investigated.

In the present paper we  study the effect of the Coulomb potential during and after the laser pulse by analyzing the angular distribution of the electrons at given energies (extracted from the double differential cross section) and the ionization probability density (calculated from the double differential cross section by integration over the ejection angles) using various model calculations.
The dependence of the angular distribution of photoelectrons on the ejection energy allow us to investigate the transition between a quantum type and a classical type ionization also studied by Dimitrovski and Solov'ev \cite{dimitrov} in the framework of the first Magnus approximation.
Atomic units are used throughout the calculations.

\section{Theory}

\subsection{Characterization of the model}
The time evolution of atomic systems in the presence of one intense ultrashort laser pulse is investigated. The laser pulse, an external electromagnetic field from the point of view of the studied system, is defined by its electric component
\begin{equation}
\vec E= \left \{
\begin{array}{ll}
 \hat\varepsilon E_0 \sin(\omega t +\phi_0)\sin^2(\frac{\pi t}{\tau})&\  \mathrm{if}\ t\in[0,\tau] \\
 0 &\ \mathrm{elsewhere}
\end{array}
 \right.,
 \label{field}
\end{equation}
where $\hat\varepsilon$ is the polarization vector along which the laser pulse is linearly polarized, $\omega$ is the frequency of the carrier wave, $\phi_0$ is the carrier envelope phase and $\tau$ is the pulse duration. The carrier envelope phase is set as follows:
\begin{equation}
\phi_0=-\frac{\omega\tau}{2}-\frac{\pi}{2},
\end{equation}
leading to a time symmetric laser pulse. The laser pulse is composed of a carrier wave $\sin(\omega t -\frac{\omega\tau}{2}-\frac{\pi}{2}),$ which is modulated by a sine-square envelope function $\sin^2(\frac{\pi t}{\tau})$. Figure \ref{fig:1} shows the shape of the laser pulses used in our calculations.
\begin{figure}
\includegraphics[width=8.5cm]{fig1.eps}
\caption{\label{fig:1} (Color online) The electric components of the laser pulses used in our calculations. Solid line - $\tau=3$ a.u.; Dotted line - $\tau=5$ a.u.; Dashed line - $\tau=10$ a.u. }
\end{figure}

In the time evolution of the studied system one may distinguish three main time intervals.

In the first time interval ($t < 0$) the laser field is not yet switched on and the studied system is in a field free eigenstate. The Hamilton operator of the system in these conditions is
\begin{equation}
\hat H_I=\frac{\hat p^2}{2}+V(\vec r),
\label{h1}
\end{equation}
where $V(\vec r)$ is the Coulomb potential between the active electron and the rest of the system.
The eigenvectors and eigenfunctions of this Hamilton operator are considered to be known
\begin{equation}
\hat H_I \psi_j=E_j\psi_j
\end{equation}
and the initial state of our system coincides with eigenstate $\psi_i$.
Under these conditions the time dependent wavefunction of the system is
\begin{equation}
\Psi_I(t)=\psi_ie^{-iE_it}.
\end{equation}

During the second time interval ($0\le t\le\tau$) the laser field is switched on and the Hamilton operator of the system becomes
\begin{equation}
\hat H_{II}=\frac{\hat p^2}{2}+\vec r\vec E+V(\vec r),
\end{equation}
where $\vec r\vec E$ is the interaction term between the laser field and the active electron expressed in $E$ gauge (Electric field gauge, length gauge). The dipole approximation was implicitly applied when the spatial dependence of $\vec E$ was neglected in Eq. (\ref{field}).
The time dependent Schr\"odinger equation
\begin{equation}
i\frac{\partial }{\partial t}\Psi_{II}(t)=\hat H_{II}\Psi_{II}(t),
\label{tdse2}
\end{equation}
with $\Psi_{II}(t)$ time dependent wavefunction of the active electron in the presence of the laser pulse, does not have known analytical solution. In order to determine the time evolution of the studied system one needs to know the $\Psi_{II}(t)$ wavefunction. In the present approach the wavefunction is considered in the following form
\begin{equation}
\Psi_{II}(t)=\int d\vec k c(\vec k, t)\Psi_V(\vec k, t),
\label{expansion}
\end{equation}
where $\Psi_V(\vec k, t)$ are the Volkov wavefunctions. The Volkov wavefunctions are the solutions of the TDSE in dipole approximation for a charged particle in radiation field
\begin{equation}
i\frac{\partial}{\partial t}\Psi_V(\vec k, t)=\left(\frac{\hat p^2}{2}+\vec r\vec E\right)\Psi_V(\vec k,t),
\label{tdsevolkov}
\end{equation}
and they can be expressed as
\begin{equation}
\Psi_V(\vec k, t)=e^{-\frac{i}{2}\int_0^tdt'(\vec k+\vec A(t'))^2}e^{i(\vec k+\vec A(t))\vec r}.
\label{volkov}
\end{equation}
In the above expression
\begin{equation}
\vec A(t)=-\int_0^t\vec E (t')dt'
\end{equation}
is the vector potential of the electromagnetic field.

The time dependent wavefunction is well defined by the expansion coefficients $c(\vec k, t)$, so if one is interested in the time evolution of the system is enough to calculate these coefficients.

By substituting the time dependent wavefunction of Eq. (\ref{expansion}) into the TDSE given by Eq. (\ref{tdse2}) can be obtained
\begin{equation}
i\frac{\partial}{\partial t}\int d\vec k c(\vec k,t) \Psi_V(\vec k,t)=\left(\frac{\hat p^2}{2}+V(\vec r)+\vec r\vec E \right)\int d\vec k c(\vec k,t)\Psi_V(\vec k, t).
\label{tdse21}
\end{equation}
Using Eq. (\ref{tdsevolkov}) it is easy to prove that
\begin{equation}
\int d\vec k c(\vec k,t)i\frac{\partial}{\partial t}\Psi_V(\vec k, t)=\left(\frac{\hat p^2}{2}+\vec r\vec E \right)\int d\vec k c(\vec k,t)\Psi_V(\vec k, t),
\end{equation}
which can be used to simplify Eq. (\ref{tdse21})
\begin{equation}
i\int d\vec k \Psi_V(\vec k,t)\frac{\partial}{\partial t}c(\vec k,t)=\int d\vec k c(\vec k,t) V(\vec r)\Psi_V(\vec k, t).
\label{koz}
\end{equation}
Eq. (\ref{koz}) can be converted into a more favorable form by transforming it into momentum space
\[
i\int d\vec k \left(\frac{\partial}{\partial t}c(\vec k,t)\right)e^{-\frac{i}{2}\int_0^tdt'(\vec k+\vec A(t'))^2}\int d\vec r e^{i(\vec k-\vec p+\vec A(t))\vec r}=
\]
\begin{equation}
\int d\vec k c(\vec k,t)e^{-\frac{i}{2}\int_0^tdt'(\vec k+\vec A(t'))^2}\int d\vec r V(\vec r) e^{i(\vec k-\vec p+\vec A(t))\vec r}.
\end{equation}
After basic mathematical operations and the substitution $\vec q = \vec p -\vec A(t)$, this equation becomes
\begin{equation}
i(2\pi)^3\frac{\partial}{\partial t}c(\vec q,t)e^{-\frac{i}{2}\int_0^tdt'(\vec q+\vec A(t'))^2}=\int d\vec k c(\vec k,t)e^{-\frac{i}{2}\int_0^tdt'(\vec k+\vec A(t'))^2}\int d\vec r V(\vec r) e^{i(\vec k-\vec p+\vec A(t))\vec r}.
\label{diff1}
\end{equation}
By introducing the notation $\vec s=\vec k-\vec q$ and by restructuring Eq. (\ref{diff1}) for the expansion coefficients $c(\vec k,t)$ can be obtained the following integro-differential equation
\begin{equation}
\frac{\partial}{\partial t} c(\vec q,t)=-\frac{i}{(2\pi)^3}\int d\vec s c(\vec s+\vec q,t)e^{-\frac{i}{2}\int_0^tdt'\vec s\left(\vec s+2\vec q+2\vec A(t')\right)}\int\vec drV(\vec r)e^{i\vec s\vec r}.
\label{diff2}
\end{equation}

We note, however, that Eq. (\ref{diff2}) is equivalent with the Schr\"odinger equation given by Eq. (\ref{tdse2}). By solving Eq. (\ref{diff2}) one obtains directly the time dependent wavefunction in momentum space, which carries all the information about the studied system. Eq. (\ref{diff2}) can be solved numerically, but this direct approach needs large computational resources, and in several cases it is not worth to do it, because by introducing some approximations one may keep almost the same accuracy.

In the third time interval ($t > \tau$) the laser field is switched off and the Hamilton operator of the system can be expressed as
\begin{equation}
\hat H_{III}=\frac{\hat p^2}{2}+V(\vec r),
\end{equation}
which is identical with the one given in the first time interval.\\
The time dependent wavefunction of the system in this time interval can be given as a linear combination of stationary-state wavefunctions
\begin{equation}
\Psi_{III}(\vec r,t)=\sum_b\psi_be^{-iE_bt}+\int d\vec k c_f(\vec k)\psi_fe^{-iE_ft},
\end{equation}
where $\psi_b$ represents bound, while $\psi_f$ represents free states. For convenience the free states are represented by plane waves, so we have
\begin{equation}
\psi_f=e^{i\vec k\vec r}
\end{equation}
and
\begin{equation}
E_f=\frac{k^2}{2}.
\end{equation}

One of the basic properties of the time dependent wavefunction, which describes the evolution of a real system is that it is continuous over time.

From the continuity condition at time $t=0$ one obtains
\begin{equation}
\Psi_{I}(t=0)=\Psi_{II}(t=0),
\end{equation}
which has the following explicit form
\begin{equation}
\psi_i=\int d\vec k c(\vec k,t=0)e^{i\vec k\vec r}.
\end{equation}
From this expression one may get the initial condition for Eq. (\ref{diff2})
\begin{equation}
c(\vec q,t=0)=\frac{1}{(2\pi)^3}\langle e^{i\vec q\vec r}|\psi_i\rangle
\label{initcond}
\end{equation}
It is worth to notice that the above initial condition is in fact the initial state wavefunction in the momentum space, because at $t=0$ the Volkov wavefunctions are reduced to plane waves.

From continuity condition at $t=\tau$ we can write:
\begin{equation}
\Psi_{II}(t=\tau)=\Psi_{III}(t=\tau),
\end{equation}
which can be given explicitly as
\begin{equation}
\int d\vec k c(\vec k,\tau)e^{-\frac{i}{2}\int_0^\tau dt'(\vec k+\vec A(t'))^2}e^{i\vec r(\vec k+\vec A(\tau))}=\sum_b\psi_be^{-iE_b\tau}+\int d\vec kc_f(\vec k)e^{i(\vec k\vec r-E_f\tau)}.
\end{equation}

We are interested in the transition probability to a free final state, which means that we need to know $c_f(\vec k)$. This can be obtained by transforming the above equation into momentum space and by neglecting the contribution of bond states
\begin{equation}
c_f(\vec p)=c(\vec p-\vec A(\tau),\tau)e^{-i[E(\vec p-\vec A(\tau),\tau)-E_f\tau]},
\label{final}
\end{equation}
where we used the notation
\begin{equation}
E(\vec k,T)=\frac{1}{2}\int_0^Tdt'(\vec k+\vec A(t'))^2.
\end{equation}
The $c_f(\vec p)$ given by Eq. (\ref{final}) is the final-state wavefunction in momentum space.\\
The transition probability from the initial state $\psi_i$ to a free final state $\psi_f$, with a well defined momentum $\vec p$, is given as
\begin{equation}
P_{i\rightarrow f}= (2\pi)^3\left|c_f(\vec p) \right|^2=(2\pi)^3\left|c(\vec p-\vec A(\tau),\tau) \right|^2.
\label{ionprob}
\end{equation}

\subsection{Volkov model as a zero order approximation }

The critical point of our model is how to solve Eq. (\ref{diff2}) or in other words how to propagate our system in the presence of external laser field in the second time interval. The simplest way is by neglecting completely the Coulomb potential ($V(\vec r)=0$). This approximation provides good results only for high laser field intensities. In this framework Eq. (\ref{diff2}) becomes
\begin{equation}
\frac{\partial}{\partial t} c(\vec q,t)=0,
\end{equation}
which has the following analytical solution using the initial condition (\ref{initcond})
\begin{equation}
c(\vec q,t)\equiv c^{(0)}(\vec q)=\frac{1}{(2\pi)^3}\langle e^{i\vec q\vec r}|\psi_i\rangle.
\end{equation}
From Eq. (\ref{final}) the final state wavefunction in momentum space can be written as:
\begin{equation}
c_f(\vec p)=c^0(\vec p-\vec A(\tau))e^{-i[E(\vec p-\vec A(\tau),\tau)-E_f\tau]},
\label{volkovmodel}
\end{equation}
which is the initial state wavefunction in the momentum space shifted by the momentum transfer $\vec A(\tau)$ gained from the external laser field.

\subsection{First order approximation }
In most cases the Volkov model (see Eq. (\ref{volkovmodel})) does not provide accurate results, because the Coulomb interaction at moderate intensities can not be totally neglected. At moderate laser intensities one can assume that the influence of the Coulomb interaction on the evolution of the system in the second time interval is small and the expansion coefficients $c(\vec k, t)$ are close to that ones provided by the Volkov model (\ref{volkovmodel}). Based on this arguments Eq. (\ref{diff2}) can be simplified by replacing $c(\vec k, t)$ in the right hand side by $c^{(0)}(\vec k)$ as follows
\begin{equation}
\frac{\partial}{\partial t} c^{(1)}(\vec q,t)=-\frac{i}{(2\pi)^3}\int d\vec s c^{(0)}(\vec s+\vec q)e^{-\frac{i}{2}\int_0^tdt'\vec s\left(\vec s+2\vec q+2\vec A(t')\right)}\int\vec drV(\vec r)e^{i\vec s\vec r}.
\label{firstorder}
\end{equation}
The same equation can be obtained using a different approach by considering the Coulomb interaction as a small perturbation and by keeping only the first order terms in $V(\vec r)$.
The advantage of this approximation is that it eliminates the direct coupling between the expansion  coefficients $c(\vec k, t)$, which makes easier and faster the solution of Eq. (\ref{diff2}).\\
Eq. (\ref{firstorder}) can be simplified and its solution can be given as simple integral
\begin{equation}
c^{(1)}(\vec q,t)=c^{(0)}(\vec q)-\frac{i}{(2\pi)^3}\int_0^t dt' I(\vec q, t'),
\label{diff3}
\end{equation}
where
\begin{equation}
I(\vec q, t)=\int d\vec s c^{(0)}(\vec s+\vec q)e^{-\frac{i}{2}\int_0^tdt'\vec s\left(\vec s+2\vec q+2\vec A(t')\right)}\int\vec drV(\vec r)e^{i\vec s\vec r}.
\label{inteq}
\end{equation}

\subsection{The Classical Trajectory Monte Carlo simulation}

In the last two decades there was a great revival of the classical trajectory Monte Carlo (CTMC) calculations applied in atomic collisions involving three or more particles  \cite{Olson91}. This approximation seems to be useful in treating atomic collisions where the quantum mechanical ones become very complicated or unfeasible. This is the case usually when higher order perturbations should be applied or many particles take part in the processes. The CTMC method has been quite successful in dealing with the ionization process in laser-atom collisions, when, instead of the charged particles, electromagnetic fields are used for excitation
of the target. The CTMC method is a nonperturbativ method, where classical equations of motions are solved numerically.
A micro-canonical ensemble characterizes the initial state of the
target. In this work, the initial conditions of the target are taken from this ensemble, which is constrained to an initial binding energy of H(1s) (0.5 a.u.).

In the present CTMC approach, Newton's classical nonrelativistic equations of motions are solved \cite{Abrines66,Olson77,Tokesi94} numerically when an external laser field given by Eq. (\ref{field}) is included.
For the given initial parameters, Newton's equations of motion were integrated with
respect to time as an independent variable by
the standard Runge-Kutta method until the real exit channels are obtained.
For the ionization channel the final energy and the scattering angles
(polar and azimuth) of the projectile and the ionized electron were recorded.
These parameters were calculated at large separation of the ionized electron
and the target nucleus, where the Coulomb interaction is negligible.

The single and double differential ionization probabilities ($P_i$) were computed
with the following formulas:

\begin{equation}
{{dP_i}\over{dE}}={{N_i}\over{N \Delta E
}},
\label{eq:engdiff}
\end{equation}

\begin{equation}
{{dP_i}\over{d\Omega}}={{N_i}\over{N
\Delta \Omega}},
\label{eq:angdiff}
\end{equation}

\begin{equation}
{{d^2P_i}\over{dEd\Omega}}={{N_i}\over{N
\Delta E \Delta \Omega}}.
\label{eq:doublediff}
\end{equation}

\noindent The standard deviation for a differential probabilities is defined through:

\begin{equation}
\Delta P_i= P_i \left [{{N-N_i}\over{N-N_i}}
\right ] ^{1/2}.
\label{eq:err}
\end{equation}

In Eqs. (\ref{eq:engdiff}- \ref{eq:err}) $N$ is the total number of trajectories calculated for the given collision system, $N_{i}$  is the number of trajectories that satisfy the criteria for ionization under consideration in the energy interval $\Delta E$ and the emission angle interval $\Delta \Omega$ of the electron.

\section{Ionization of the hydrogen atom}

The theoretical approaches presented above are applied to describe the ionization of the hydrogen atom in over the barrier regime. We choose this system because the calculations are easy to perform and because there are several theoretical studies on this system \cite{hansen, ducha01, ducha02, mishima, zhang} and it is easy to compare our results with others found in the literature \cite{ducha02}.\\
Using the 1s orbital of the hydrogen atom as initial state wavefunction
\begin{equation}
\psi_i=\frac{1}{\sqrt{\pi}}e^{-r}
\end{equation}
one obtains the following initial condition for Eq. (\ref{diff2})
\begin{equation}
c^{(0)}(\vec q) = \frac{1}{\pi^2\sqrt \pi (1+\vec q^2)^2}.
\label{inithyd}
\end{equation}
Using Eq. (\ref{inithyd}) the ionization probability in Volkov approximation can be expressed as
\begin{equation}
P_{i\rightarrow f}(\vec p)=\frac{16}{\pi(1+(\vec p-\vec A(\tau))^2)^4}.
\end{equation}
The first order model can also be adapted very easily by using the Coulomb potential
\begin{equation}
V(\vec r)=-\frac{1}{r}
\end{equation}
in Eq.(\ref{inteq}), which can be significantly simplified by performing some of the involved integrals
\[
I(\vec q, t)=\lim_{r_{max}\rightarrow +\infty}\frac{8}{\sqrt \pi}\int_0^\infty\int_0^\pi\frac{\sin(\theta_s)d\theta_sds}{(1+s^2+q^2+2sq\cos(\theta_s))^2}\quad\times
\]
\begin{equation}
e^{-\frac{i}{2}(s^2t+2sq\cos(\theta_s)t+2f(t)s\cos(\theta_s)\cos(\theta_\varepsilon))}J_0(2sf(t)\sin(\theta_s)\sin(\theta_\varepsilon))(\cos(sr_{max})-1),
\label{firstsol}
\end{equation}
where $J_0$ is the Bessel function of the first kind.
The angle between $\vec s$ and $\vec q$ is $\theta_s$, while the angle between $\hat \varepsilon$ and $\vec q$ is $\theta_\varepsilon$ and
\begin{equation}
f(t)=\int_0^{t}A(t')dt'.
\end{equation}
By substituting  Eq.(\ref{firstsol}) into Eq.(\ref{diff3}) and by performing the remaining integrals numerically one obtains the expansion coefficient in first order approximation. From the obtained expansion coefficients the ionization probability is calculated by using Eq.(\ref{ionprob}).

\section{Results and discussion}

Calculations are performed using laser pulses with duration $\tau$ of $3$ a.u., $5$ a.u. and $10$ a.u. at two different field intensities ($E_0=1$ a.u. and $E_0=10$ a.u) for each pulse duration. The shape of the pulses can be seen on FIG. \ref{fig:1}.\\
\begin{figure}
\includegraphics[width=16cm]{fig2.eps}
\caption{\label{fig:2} (Color online) Two-dimensional ionization probability density in the yOz positive semi-plane as a function of the  electron energy and the ejection angle. First column - Volkov results; Second column - first order results; Third column - CTMC results; First row - $\tau = 3$ a.u. and $E_0 = 1$ a.u.; Second row - $\tau = 3$ a.u. and $E_0 = 10$ a.u.; Third row - $\tau = 5$ a.u. and $E_0 = 1$ a.u.; Fourth row - $\tau = 10$ a.u. and $E_0 = 1$ a.u.}
\end{figure}
The double differential ionization cross sections calculated using the Volkov, the first order and the CTMC model are presented on Figure \ref{fig:2}, where the ionization probability density is plotted as a function of the electron energy and the ejection angle. The ejection angle is measured from the polarization vector $\hat\varepsilon$, which is in the direction of z axis. Due to the spherical symmetry of the ground state wavefunction and of the Coulomb potential the double differential ionization cross section has a rotational symmetry, where the rotational axis is in the direction of the polarization vector $\hat\varepsilon$ of the laser field. Due to this symmetry the yOz positive semi-plane carries all the information about the ionization cross section. At first sight one can observe that at a large scale all three models predicts the same cross sections. In each approach the electrons are ejected with maximum probability along the polarization vector $\hat\varepsilon$ with energy around the value $\frac{A(\tau)^2}{2},$ which is gained by the momentum transfer $\vec A(\tau)$ from the external laser field.

After detailed analysis, important differences can be observed. In the case of the first order and CTMC model the maxima of the predicted cross sections are shifted toward smaller energies. This shift is caused by the Coulomb attraction during the ionization between the active electron and the rest of the system, in a classical picture the ejected electrons are decelerated by the Coulomb attraction.
\begin{figure}
\includegraphics[width=14cm]{fig3.eps}
\caption{\label{fig:3} (Color online) Angular distribution of the photoelectrons for given electron energies ($W_e$). Solid line - first order; Dotted line - Volkov; Squares - CTMC; First column - $\tau = 5$ a.u. and $E_0 = 1$ a.u.; Second column - $\tau = 10$ a.u. and $E_0 = 1$ a.u.; (a) $W_e = 0.02$ a.u.; (b) $W_e = 0.125$ a.u.; (c) $W_e = 3.5 $ a.u.; (d)  $W_e = 4.5$ a.u.; (e) $W_e = 6$ a.u.; (f)  $W_e = 10$ a.u.}
\end{figure}

Other important differences can be identified at the angular distribution of the ejected electrons at fixed energies ($W_e$) as shown in Figure \ref{fig:3}. The maximum value of each angular distribution is normalized to unity to allow an easier comparison.

At the low part of the energy spectrum (see FIG. \ref{fig:3} (a) and (d), with $W_e$ below the peak observable on FIG. 2) significant differences are observed between the predicted first order and CTMC angular distributions. The observed differences implies the quantum nature of the ionization for low energy electrons, which can not be described correctly by classical calculations. These differences start to disappear at higher electron energies (see FIG. \ref{fig:3} (b) and (e)), where both distributions roughly are the same. However some minor differences still exist showing a transition between the quantum and classical nature of the ionization. At high energies, the differences observed at lower energies completely disappear (see FIG. \ref{fig:3} (e) and (f)) and the distributions predicted by first order and CTMC model are in good agreement indicating that the ionization mechanism for electrons ejected at high energies behave classically. This evolution of the cross section and the transition between the quantum and classical type ionization was also studied by Dimitrovski and Solov'ev \cite{dimitrov} in the framework of first Magnus approximation. They obtained the same transition between quantum and classical type ionization by varying the net momentum transfer.

At the low part of the energy spectrum (see FIG. \ref{fig:3} (a) and (d)) large discrepancies are also observed between the predicted first order and Volkov angular distributions. These discrepancies show the significant influence of the Coulomb interaction on the ionization of the low energy electrons. This disagreement between the discussed models diminishes and completely disappears at higher electron energies (see FIG. \ref{fig:3} (b, c, e, f)), which indicates that the Coulomb interaction has less influence on the ionization of the high energy electrons.
\begin{figure}
\includegraphics[width=8.5cm]{fig4.eps}
\caption{\label{fig:4} (Color online) The net angular distribution of the photoelectrons. Solid line - first order; Dotted line - Volkov; Squares - CTMC; (a) - $\tau = 5$ a.u. and $E_0 = 1$ a.u.; (b) - $\tau = 10$ a.u. and $E_0 = 1$ a.u.}
\end{figure}
Figure \ref{fig:4} shows the net angular distributions of the ejected electrons obtained from the double differential cross section by integrating over the electron energies for different pulse durations. The angular distributions predicted by different models are  in good agreement with each other, because the contribution of the low energy electrons in the net angular distribution is negligible, so the influence of the Coulomb potential and the quantum nature of the ionization can be  also neglected.
\begin{figure}
\includegraphics[width=15cm]{fig5.eps}
\caption{\label{fig:5} (Color online) Ionization probability density as function of the electron energy. Solid line - first order; Dotted line - Volkov; Squares - CTMC; Dashed line - TDSE \cite{ducha02}; Dash-dotted line - CV \cite{ducha02}  First column - $E_0 = 1$ a.u.; Second column - $E_0 = 10$ a.u.; First row - $\tau = 3$ a.u.; Second row - $\tau = 5$ a.u.; Third row - $\tau = 10$ a.u.}
\end{figure}
On Figure \ref{fig:5} we have represented the ionization probability densities obtained from the double differential ionization cross sections by integrating over the ejection angle. The  probability densities are calculated using the first order, the Volkov and CTMC models. Figure \ref{fig:5} also shows the TDSE and CV results calculated by Duchateau et. al. \cite{ducha02}.
At high laser field intensity ($E_0=10$ a.u.) a very good agreement is observed between TDSE and CTMC results on Figure \ref{fig:5} (b). At lower laser intensity ($E_0=1$ a.u.) the agreement is acceptable (see FIG. \ref{fig:5} (b) and (c) ), but not as good as in the previous case, because at these intensities tunneling ionization also takes place, which is not considered in our CTMC model. This good agreement between classical and quantum approaches at high laser field intensities was also confirmed by several other studies \cite{hansen, ducha02}, which proves the classical nature of the over the barrier ionization.

The main difference between the Volkov and the first order model is that the Volkov model completely neglects the Coulomb attraction during the ionization process. The effect of the Coulomb interaction on the photoelectrons during the ionization can be studied by comparing the Volkov and first order ionization probability densities. At the high energy part of each photoelectron spectrum presented on Figure \ref{fig:5}, a good agreement can be observed between the Volkov and first order model. This agreement breaks down at the lower energy part of the spectra, where the first order spectrum is shifted toward the lower energies. The above behavior was also observed in the angular distribution of ionized electrons and it can be explained using a very simple intuitive picture presented on Figure \ref{fig:6}. The electrons with high ejection energy have their momentum in the initial state in the same direction as the net momentum transfer. In this way the trajectory of the high energy electrons leads directly away from the core, with a very small portion close to the core where the Coulomb potential has a significant influence. On the other hand the electrons with lower ejection energies have a momentum in the initial state, which leads in the opposite direction of the net impulse transfer. In this way the low energy electrons need to "go around" the core, leading to a trajectory, which has a long portion close to the core, where they can be influenced significantly by the Coulomb interaction.

\begin{figure}
\includegraphics[width=8.5cm]{fig6.eps}
\caption{\label{fig:6}(Color online) Two possible electron trajectories during the ionization corresponding to photoelectrons with high (Solid line) and low (Dotted line) ejection energies. The black dot is the residual proton and the net momentum transfer is along the z axis.}
\end{figure}

Results obtained by Duchateau et. al. \cite{ducha02} using Coulomb-Volkov wavefunctions and the sudden
approximation is also included in our analysis. In the CV model the influence of the Coulomb interaction
on the ionization process is not included, it is taken into account only in the final state wavefunction.
The comparison of CTMC, CV and first order results allow us to study in detail the influence of the Coulomb
interaction during and after the laser pulse.

At low intensities for short pulse, the agreement between the CV and CTMC results is better than between the CTMC and first order results (see FIG. \ref{fig:5} (a)). For this pulse parameters the net momentum transfer is small leading to photoelectrons with small momentum, which can not be described correctly by the plane waves employed by the first order model.

At the same intensity but for longer pulses, major discrepancies can be observed between CV and CTMC results (see FIG. \ref{fig:5} (c, e)), because for these pulse durations the sudden approximation no longer applies. The same discrepancies are observed between first order and CTMC results, because the final state wavefunction is still inaccurate.

At high laser field intensities the agreement between the first order and CTMC results is better than the agreement between CV and CTMC results (see FIG. \ref{fig:5} (b)). In this case the momentum transfer is high leading to photoelectrons with high energies, where the plane wave approximation for the final state is accurate. At the low energy part of the spectra presented on Figure \ref{fig:5} (b, d, f) the first order results show good agreement with the CTMC results (considered identical with the TDSE results at this high intensity), but discrepancies exists at the high energy part of the spectra. Both first order and CV models predict a higher ionization probability than the CTMC model at the high energy part of the spectra, which in the case of the first order model can be corrected by taking into account the Coulomb interaction after the laser pulse.

\section{Conclusions and Outlook}

In the present work a general approach for the ionization of atomic systems by ultrashort laser pulses in one active electron approximation was presented. The time dependent Schr\"odinger equation of the studied system was transformed into a pseudo-momentum space, where it was solved in an iterative way. In the zero order solution the Coulomb potential was neglected (Volkov model), while in the first order solution it was taken into account during the ionization as a perturbation.

Calculations were performed for the ionization of the hydrogen atom in over the barrier regime using the Volkov and the first order solution of the TDSE. Classical trajectory Monte-Carlo calculations were also performed. The obtained results were analyzed and compared with results obtained by Duchateau and coworkers \cite{ducha02} using TDSE and CV model.

The double differential ionization cross sections were calculated using Volkov, first order and CTMC models for different laser pulses. A good agreement was found between the results using these three models at high laser field intensities. This good agreement between classical and  quantum calculations was also confirmed by other groups \cite{hansen, ducha02}. At lower intensities, however, small discrepancies appeared due to the tunneling ionization.

The first order and CTMC results were shifted toward smaller energies due to the Coulomb attraction of the remaining target ion.

More differences were identified by analyzing the angular distribution of the ejected electrons.
For low energy electrons major discrepancies have been found between CTMC and first order results, because the ionization of this low energy electrons is a quantum process, which can not be described in a classical approach.
At higher electron energies, the observed discrepancies disappear leading to the conclusion that the ionization process of the high energy electrons can be described classically.
At low energies, differences were also observed between the first order and Volkov results, which disappeared at higher electron energies indicating that during the ionization electrons with higher energies are less influenced by the Coulomb interaction. The same observation was made by analyzing the ionization probability densities, where significant differences between the Volkov and first order results were observed for low energy photoelectrons. A simple explanation for this behavior was found by considering the possible classical trajectories of the electrons during the ionization. Similar conclusion was also drawn by Zhang et. al. \cite{zhang} in their study, where they found that the influence of the Coulomb interaction depends mainly on the energy of the photoelectrons rather than on the laser field intensities.

The influence of the Coulomb potential during and after the laser pulse was studied. For short laser pulses with low momentum transfer we found that the Coulomb potential has an important role after the laser pulse is switched off. In the case of longer pulses, with low momentum transfer, the Coulomb potential is equally important during and after the pulse, while in the case of high momentum transfer the Coulomb interaction has a greater influence during the laser pulse.

A good agreement between first order and CTMC results was found at high laser field intensities, where the momentum was high, while at lower intensities with low momentum the agreement was acceptable, comparable with the agreement between the CV and CTMC results.

Our model can be further improved by considering the influence of the Coulomb potential on the final state wavefunction, by using Coulomb wavefunctions instead of simple plane waves for the final state, which will be the subject of our future investigations.

\section{Acknowledgment}

The work was supported by the Romanian Academy of Sciences (grant No. 35/3.09.2007), the Romanian National Plan for Research (PN II) under contract No. ID$\_539$, the grant ``Bolyai'' from the Hungarian Academy of Sciences, the Hungarian National Office for Research and Technology.


\section{Appendix}

\begin{table}[b!]
  \caption{Fields in a SNORT alert.}
  \label{tab:snort-alert-fields}
  \centering
  \begin{tabular}[t]{ccc}
    \begin{minipage}[t]{.33\linewidth}
      \begin{titemize}
      \item timestamp
      \item sig\_generator
      \item sig\_id
      \item sig\_rev
      \item msg
      \item proto
      \item src
      \item srcport
      \item dst
      \item dstport
    \end{titemize}
  \end{minipage} &
  \begin{minipage}[t]{.33\linewidth}
    \begin{titemize}
      \item ethsrc
      \item ethdst
      \item ethlen
      \item tcpflags
      \item tcpseq
      \item tcpack
      \item tcplen
      \item tcpwindow
      \item ttl
    \end{titemize}
  \end{minipage} &
  \begin{minipage}[t]{.33\linewidth}
    \begin{titemize}
      \item tos
      \item id
      \item dgmlen
      \item iplen
      \item icmptype
      \item icmpcoe
      \item icmpid
      \item icmpseq
    \end{titemize}
  \end{minipage}
\end{tabular}
\end{table}

\begin{figure*}[t!]
  \includegraphics[width=7.0in]{./img/PCAPFieldAnony.pdf}
  \caption{PCAP Fields and Anonymization Algorithms. Each section contains the fields (on the left) on which any of the anonymization algorithms on the right can be applied. For instance, Only the anonymization algorithms BinaryBlackMarker and Annihilation can be applied to the ICMP\_ORIG\_DATA field (the bottom left section)}
\label{fig:pcapanoy}
\end{figure*}

\input{./AnonyParameters.tex}

\begin{table*}[t!]
  \caption{Anonymization algorithms with applicable data types.}
  \label{tab:anony-algorithms}
  \begin{tabular}[t]{p{1.5in}|l|p{3in}}
    {\bf Anonymization Alg.} & {\bf Data Type(s)} & {\bf Description} \\
    Prefix-preserving & binary & Implements prefix-preserving permuta-\\
     & & tion described in \cite{xu02prefixpreserving}\\
    Truncation & binary & Removes suffix or prefix of data by\\
     & string & specified number of units.\\
     & numeric & \\
    Hash & binary & Outputs cryptographic has of data.\\
     & string & \\
     & hostname & \\
    Black Marker & binary & Overwrites specified number of units \\
     & string & with specified constant.\\
     & hostname & \\
    Time Unit Annihilation & timestamp & Annihilates particular time units\\
     & & (e.g., hour and minute units).\\
    Random Time Shift & timestamp & Randomly shifts timestamps within\\
     & & given window by same amount.\\
    Enumeration & timestamp & Preserves order, but not distance \\
     & & between elements.\\
    Random Permutation & binary & Creates random 1-to-1 mapping.\\
    Annihilation & binary & Replaces field with NULL value.\\
     & string & \\
    Classify & numeric & Partitions data into multiple non-\\
     & & overlapping subsets.\\
    Substitution & binary & Replaces all instances with a particu-\\
     & numeric & lar constant value.
  \end{tabular}
\end{table*}

\section{Conclusions}
\label{sec:conclusions}
Anonymization can be a powerful tool to allow greater cooperation between organizations. The need for cooperation is strikingly clear. However, a clear understanding of the needs of the data provider and the client is necessary before flexible, effective sharing between organizations can occur.

The objective of this paper has been to begin to formally evaluate the
utility vs. security trade off. The \emph{IDS metric} is simple yet
effective in evaluating the difference in utility when anonymizing
different fields in different ways.

In this paper, we have focused on answering three questions: whether
the field affects anonymization more than the algorithm; which fields
have a larger impact on utility; and which anonymization algorithms
have a larger impact on utility.

We have provided a thorough evaluation of single field anonymization
polices upon pcap formatted network traces. We found that the primary
impact on the utility of a log is not the particular anonymization
algorithm, but rather the field that was anonymized.

In addition, we were able to empirically show a range of utilities for
a log based on the field that was anonymized. The loss of utility was
largest for ports and IP addresses. There was some loss of utility for
the fields of ID, sequence number, flags, timestamp, and ACK
number. However, for many of the fields there was no change in utility
when anonymized.

This empirical evaluation provides the basis for further work on
studying the impact of more complex anonymization schemes on the
utility of a log.

\section{Future Work}
\label{sec:future-work}
There are numerous ways in which this work can be extended. First of
all, it is clear that evaluating utility via Snort generated IDS
alerts will cause the utility to depend upon the rule set. In fact, it
is as of yet unclear exactly how the rule set impacts the utility
measure. Though, we suspect there is a significant effect since we
found the specific fields anonymized has more effect than how it was
anonymized, and Snort rules usually focus on a one or two fields.

While we used Snort for all of our experiments, it is just one IDS and
only one type. It would be very interesting to investigate whether
anomaly-based IDSs are affected in similar ways.

The strength of an anonymization algorithm is a measure of how difficult it is to ``break'' or ``de-anonymize'' a log that has been anonymized via the algorithm. It is clear that we want strong anonymization algorithms so that attackers will have a difficult time to break the algorithm. As \cite{slagell06flaim} points out, there is a trade off between the security of an anonymization algorithm and the utility of the log. We have not discussed the strength of an anonymization algorithm in this work.

Our work is currently limited to anonymization policies for just one
field. Our next step would be to extend this work to multiple field
anonymization policies and to work with more realistic data. The
DARPA evaluation data set is useful because it is supervised, however
it is still synthetic. In the future, we will be gaining access to
other large {\em unanonymized} data sets that we can use instead of
the DARPA data. However, it is important for this research to have
unanonymized baseline sets.

Finally, we have still focused on utility for just one task, attack
detection. An incident responder does more than just detect attacks,
and in the future, we could look at how anonymizing logs affects other
important security related tasks---such as alert correlation.

\section{FLAIM: Framework for Log Anonymization and Information Management}
\label{sec:flaim:-framework-log}

We chose to use FLAIM \cite{slagell06flaim} as our anonymization
engine for several reasons. First, we could easily script its
execution for a multitude of tests. Second, it has a very flexible XML
policy language that made it simple to generate hundreds of unique
anonymization policies. Third, it can anonymize as many or more fields
in PCAP traces as any other anonymization tool. Lastly, FLAIM has a
very rich set of anonymization algorithms that can be applied to all
these fields. With these properties, it was the ideal tool for our
experiments.

\subsection{Anonymization Algorithms}
FLAIM implements a plethora of anonymization algorithms for several
data types. The three basic data types available to most anonymization
algorithms are \emph{binary, string,} and
\emph{numeric}. Additionally, there are a few special data types like
{\em timestamps}. Binary data is just treated as a string of bits
with no special structure. Algorithms anonymizing binary data
output binary data of the same length. String data are variable
length, terminated by a null character. Anonymization algorithms
that take in strings will also output stings. However, the
length may change. Numeric data is interpreted as a number of a
given base, specified in the anonymization policy. This is
useful, for example, when working with decimal numbers like a
port number. There, one may want to act on individual digits,
rather than bits (e.g., replacing the last 3 digits with 0's).

Table~\ref{tab:anony-algorithms} lists the different anonymization
algorithms in FLAIM along with the data types they operate
upon. Further information on these algorithms can be found in
\cite{flaim-manual}.

\subsection{Anonymization Policies in FLAIM}
FLAIM provides an expressive and powerful method for specifying
anonymization policies that can be modified at run time, thus
enabling efficient automation. An anonymization policy is an XML file
that specifies the anonymization algorithms that should be applied to
the various fields in the log, along with any special parameters to be
passed to those algorithms.

\section{Introduction}\label{intro}
The ability to safely share log files and network traces has become
increasing important to several communities: networking research,
computer security research, incident response, and education
\cite{slagell05sharingcomputer}. Synthetically generated data is
abundant, but has been highly criticized for many uses
\cite{mchugh00testing}. Honeynets can be useful in generating
exercises for students and help meet the needs of educators
\cite{honeynetChallenge}, but very few honeynets have been setup on a
scale to generate some of the large, cross-sectional data sets needed
by computer security researchers
\cite{vrable05scalability}. Furthermore, using a honeynet does not
necessarily release the owner from all legal responsibility when
sharing data \cite{sicker07legalissues}. Lastly, nothing but real data
will suit the needs of the incident responder who must share data
about specific attacks under investigation of real
machines. Therefore, there is a high demand for methods to share {\em
  real} log files and network traces within several communities.

At the same time as there is an increased need to share these data
sets, there is increased reluctance. First, many data owners recognize
the inherent security risks of releasing detailed information that
could be used to map out their own networks, services or sensor
locations
\cite{bethencourt05mappinginternet,coull07playing,kohno05remotephysical,lincoln04privacypreserving,pang06devil,Pang03,xu02prefixpreserving,zalewskip0f}. Secondly,
there are serious privacy concerns that companies have about releasing
customer data, especially in light of recent incidents where some
publicly released data---believed to be anonymous---leaked information about
specific users \cite{aol06nytimes,narayanan06howtobreak}. Lastly,
recent research has questioned the legality of releasing much of the
data as has been done up until now \cite{sicker07legalissues}.

The burgeoning field of data sanitization has helped address this
tension by providing organizations, who wish to share their data, with
new tools to anonymize computer and network logs
\cite{coull07playing,foukarakis07flexible,lundin99privacy,pang06devil,seeberg07anewclassification,slagell06flaim,slagell05sharingnetworklogs,slagell05networkloganonymization,slagell05networkloganonymization,xu02prefixpreserving,yurcik07anonyids}. However,
little has been done to help users negotiate the difficult trade-off
between the data owner's need for security and privacy, and the data
analyst's need for high quality data---what is called the {\em
  utility~vs.~security~trade-off} \cite{slagell06flaim}. As
anonymization is an inherently lossy process, and the data analyst
wants information as close to the original as possible, there is
always this tension and a need to negotiate policies that meet the
needs of both parties.

Necessary to solving this problem of negotiating policies for data
sanitization is the ability to analyze the effects of anonymization on
both the security of the sanitized data and the utility left after
anonymization. In this paper, we focus upon the latter problem of
evaluating the effects of anonymization on the utility of the data
sets to be shared. Of course, utility is subjective since it depends
upon who is using the data, or more specifically, for what purpose it
is being used. Hence, what is important to a researcher in the network
measurements community may be completely irrelevant to the incident
responder. Therefore, we have scoped this work to evaluating utility
for attack detection.

The task of attack detection is an important part of the incident
responder's daily job. When investigating broad attacks, of which
their organization is only a part, they may have to settle for
anonymized logs from the other sites involved. It would be a similar
case when using a distributed or collaborative intrusion detection
system that crosses organizational boundaries. Output from the sensors
may need to be anonymized. Not only is attack detection important in
these ``real world'' applications, but it is important to the
intrusion detection research community, as well. This community has
often complained that their only good data sets to test new
technologies against are synthetic. However, if they can still do their
analysis with anonymized data, then they are more likely to obtain
large, real data sets.

The utility of a data set is not only constrained by the type of
analysis being done with it but also the type of data being shared. We
have chosen to look at the effects of anonymization on one of the most
commonly shared and general types of data, the pcap formatted network
trace. From this type of data, many others can be derived (e.g.,
NetFlows) \cite{argus08website}.

To quantitatively measure the ability to identify attacks in
anonymized data, we developed what we call the {\em IDS Utility
  Metric}. This measurement evaluates and compares the false positive
and negative rates of a baseline {\em unanonymized} data set with that
of an {\em anonymized} data set. By doing this, we can automate an
objective process to help us answer several questions: (1) How does
anonymization of a particular field affect the ability to detect an
attack, (2) are there unique effects when certain pairs or triplets of
fields are anonymized together, and (3) how does the use of different
types of anonymization algorithms affect attack identification? In
this paper, we present the results of anonymizing a portion of the
1999 MIT/Lincoln Labs DARPA data set---by using the FLAIM
\cite{slagell06flaim} anonymization framework---with over 150 separate
anonymization policies to help us answer these questions.

The rest of the paper is organized as
follows. Section~\ref{sec:flaim:-framework-log} describes the
anonymization algorithms used by FLAIM in our experiments, while
section~\ref{sec:methodology} describes the methodology and
setup of our experiments. In section~\ref{sec:results-analysis}, we
present our results and analysis. We survey the related work in
section~\ref{related}, and state our conclusions, as well as scope out
future work to be done, in sections~\ref{sec:conclusions}
and~\ref{sec:future-work}.

\section{Methodology}\label{sec:methodology}

\begin{figure}[tp]
  \includegraphics[width=2in]{./img/methodology.pdf}
  \caption{This diagram illustrates the process by which we compare anonymized logs versus non-anonymized logs.}
  \label{fig:methodology}
\end{figure}

For all of our experiments, we used a subset of the 1999 DARPA
evaluation data set. The Defense Advanced Research Projects Office
(DARPA) created an Intrusion Detection Evaluation testbed in 1998 and
1999. Data was captured from a simulated network that was subjected to
various attacks. This data set has been frequently used in evaluating
intrusion detection systems since its creation
\cite{lippmann00darpa1999}. Thus, we found it appropriate to use in
evaluating the effects anonymization of data can have on intrusion
detection.

As we mention, we used but a portion of the 1999 data set. Specifically, we
used the {\em inside} tcpdump data from Wednesday of the second week of the
evaluation. Since FLAIM currently just anonymizes TCP, UDP and ICMP,
we filtered out all other network protocols before running our
experiments, whose methodology is depicted in Figure~\ref{fig:methodology}.

The {\em IDS Utility Metric} is constructed as follows. First, the
unanonymized data set is processed by Snort to produce the
``baseline'' set of alerts, using the default Snort rule sets. We
consider this the ideal set of alerts for the data set. Next, we take
the same data set, and anonymize it---in our case, with FLAIM. We then
take the anonymized data set, and we run Snort against it with the
same rule set as before. The alerts generated from the unanonymized
file are used as a baseline against which the alerts generated by the
anonymized file are compared. The difference between the alerts in the
anonymized file, versus the unanonymized file, is used as a measure of
the loss of utility in the log. The larger the difference, the more
information was not available in the logs in order for the IDS to
correctly identify an attack. This process is depicted in
Figure~\ref{fig:methodology}.

FLAIM schemas specify a set of anonymization algorithms that are
appropriate for each field in a pcap log. These are summarized in
Figure~\ref{fig:pcapanoy}. In this evaluation, we only consider
anonymization policies that transform single fields. Before we can
evaluate the affect of multi-field policies, we must come to an
understanding of single field policies. There are 152 single field
policies that can be generated for pcap data
(Figure~\ref{fig:pcapanoy} in the Appendix lists all the fields and
anonymization algorithms used for testing). Each anonymization
algorithm also has parameters that affect how it anonymizes the
field. We will not go over the parameters in detail, but instead refer
readers to the FLAIM manual
\cite{flaim-manual}. Table~\ref{tab:anonyparam} in the Appendix
summarizes the parameter settings for the anonymization algorithms.

We iterate the process described above over each of the 152
anonymization policies, comparing the results pre- and
post-anonymization. We describe how we compare these data sets in the
next section, while the section after discuses the actual metric in
more detail.

\subsection{Comparing Snort Alerts}
Alerts generated by Snort are defined by several properties (a full
list is included in the appendix as
table~\ref{tab:snort-alert-fields}). Each alert is associated with a
specific packet. The relevant alert fields---for our purposes---are
listed below:

\begin{description}
  \item[timestamp]: the timestamp from the offending packet.
  \item [sig\_generator]: the part of Snort generating alert.
  \item[sig\_id]: the Id. number of the signature that was fired.
  \item[msg]:  description of the alert.
  \item[proto]: the protocol of the offending packet.
  \item[src]: the source IP address of the offending packet.
  \item[srcport]: The source port of the offending packet.
  \item[dst]: the destination IP address of the offending packet.
  \item[dstport]: the destination port of the offending packet.
  \item[id]: Packet Id.
\end{description}

To determine whether two alert sets are equal we need a way of
determining if two alerts are equal. Normally this can be done by
comparing each field of the alerts. However, in this case the alerts
generated from the anonymized log will cause alerts that are actually
equal to appear unequal. To overcome this, we compare alerts on fields
which will not change due to anonymization. This leads to two distinct
field sets that must be used when comparing alerts. They are shown in
Table~\ref{tab:alert-field-sets}. Field Set 1 is used when the
timestamp field has not been anonymized. Field Set 2 is used when the
timestamp field has been anonymized.

\begin{table}
  \centering
  \begin{tabular}[t]{c|c}
    Field Set 1 & Field Set 2 \\ \hline
    timestamp & sig\_id \\
    sig\_id & src \\
    id & srcport \\
    & dst \\
    & dstport \\
    & id \\
    & tcpseq \\
  \end{tabular}
  \caption{The two field sets that are used to compare alerts.}
  \label{tab:alert-field-sets}
\end{table}

\subsection{Metrics for evaluating utility}

The purpose of anonymization is to share logs while hiding sensitive
information. Anonymization, while inherently an information reducing
procedure, must be measured in terms of the amount of information that
is lost in the file.  However, by anonymizing we can introduce new
false patterns into the data. The ``best'' anonymization policy should
minimize information loss, while not adding any new false patterns to
the log.

We can consider the IDS process as a pattern classification
process. The data set is input, and the IDS classifies each packet as
malicious or not. The alerts generated from the unanonymized data
(which we call the baseline data) are considered to be the correct
analysis of the data set. We then compare the alerts generated by the
anonymized data to the baseline data alert set.

Let the set of alerts generated from the baseline file be called
\ensuremath{\mathcal{A}_{baseline}} ~and the set of alerts generated from the anonymized file be
called \ensuremath{\mathcal{A}_{anony}}. In terms of alerts we should compare \ensuremath{\mathcal{A}_{anony}}
vs. \ensuremath{\mathcal{A}_{baseline}}. The best result would be for the anonymized alerts to
match, exactly, the alerts generated in the baseline set. Let us
consider the baseline alerts to be the target set. The alerts from the
anonymized file will be the generated set. Then we can define several
metrics:

\begin{description}
\item[True Positive] $TP = \ensuremath{\mathcal{A}_{anony}} \cap \ensuremath{\mathcal{A}_{baseline}}|$ The number of alerts in \ensuremath{\mathcal{A}_{anony}} that are also in \ensuremath{\mathcal{A}_{baseline}}.
\item[False Positive] $FP = |\ensuremath{\mathcal{A}_{anony}} - (\ensuremath{\mathcal{A}_{anony}} \cap \ensuremath{\mathcal{A}_{baseline}})|$ The number of alerts that were generated by the anonymized file, but were not in the baseline file.
\item[False Negative] $FN = |\ensuremath{\mathcal{A}_{baseline}} - (\ensuremath{\mathcal{A}_{anony}} \cap \ensuremath{\mathcal{A}_{baseline}})|$ The number of alerts that were not caught by the anonymized file.
\end{description}

The True Positive rate indicates how much of the information was
preserved in the anonymized log. The False Positive rate indicates how
many additional patterns were added to the log through
anonymization. The False Negative rate indicate the amount of
information that was removed from the log. A good anonymization policy
should make sure both False Positive and False Negative are low while
maximizing the True Positive rate.

While the false positive rate is an important factor of primary importance is the false negative rate, as it indicates the loss of information through anonymization. For the remainder of this paper, we use both False Positive and False Negative rates as a measure of the utility of a log post-anonymization, but focus more on the false negative rate.

\section{Multi-Field Policy Analysis}
\label{sec:multi-field-policy}
By multi-field anonymization policies, we are referring to
anonymization schemes that transform two or more fields in a log. The
bulk of this work has focused on single field policy
analysis. However, single field policy analysis will lay the
groundwork for understanding more complex multi-field policies.

\begin{table}
  \caption{This table shows that a multi-field policy has uncertain affects on the utility of a log}
  \label{tab:multifield}
  \begin{tabular}{ll}
    \textbf{Policy} & \textbf{Alerts} \\
    IPV4\_SRC\_IP-IPv4PrefixPreserving	& 1014 \\
    TCP\_SRC\_PORT-RandomPermutation & 7 \\
    TS\_SEC-TimeEnumeration & 399 \\
    IPV4\_SRC\_IP,TCP\_SRC\_PORT & 1016 \\
    IPV4\_SRC\_IP,TCP\_SRC\_PORT and TS\_SEC & 1010 \\
  \end{tabular}
\end{table}

Multi-field policies are difficult to analyze because the fields are
often related in subtle ways, not just very direct ways, such as
between the ACK and SEQ numbers. The anonymization of additional
fields certainly does not affect the total number of false positive
alerts in a linear way. For example, examine
Table~\ref{tab:multifield}.

Here, we see that anonymizing 3 fields separately produces 1014, 7 and
399 false alerts, respectively. If we anonymize the first 2 fields, we
get $1016\neq 1014+7$ alerts. This may not be too surprising. However,
if we do all 3 fields together, we get only $1010$ alerts---rather
than something around $1420=1014+7+399$. This is actually fewer false
alerts than any one field being anonymized in isolation.

So while there is always more information loss when anonymizing more
fields, there may be a critical point at which one starts getting
fewer false positives as they do additional anonymization. We suspect
that this is always the case since complete anonymization will leave
nothing to alert upon. But clearly, more analysis must be done on
multi-field anonymization, and our future work will be focused on
evaluating multi-field anonymization policies.

\section{Related Work}\label{related}
Work in data sanitization for computer and network logs to date has
focused almost entirely upon development of tools
\cite{slagell06flaim,li05canine,yurcik07anonyids,luo06scrubpa,foukarakis07flexible,Pang03}
and anonymization algorithms \cite{xu02prefixpreserving}, with little
to no work doing any formal analysis of the effects of
anonymization. This is an important aspect missing from the research
body since without it, we just have a lot of tools to haphazardly
anonymize data without knowing how to do it wisely or
effectively. This is also in stark contrast to k-anonymity
\cite{sweeney02kanonymity} which is usually applied to medical and
census type data and has received much more attention from the
research community.

While writing this paper, a new piece of work closely related to ours
appeared \cite{yurcik07anonyids}. Though this is mostly a paper
presenting yet another anonymization tool for pcap logs, at the end
they introduce a similar method of analyzing the effects of
anonymization of pcap traces by use of an IDS. Theirs is a cursory
analysis, only considering anonymization of one field at a time and
for only a few different fields. Also, their analysis only considered
the number of alerts, where the more alerts are generated the more
``security analysis'' is provided. Clearly, more than just the number
of alerts needs to be considered when evaluating utility, such as the
false positive and negative rates. We were unable to reproduce their
results (which may be because they used an unspecified subset of the
LBNL data set\footnote{http://www.icir.org/enterprise-tracing/}), but
more troubling is the use of this data set in the first place. To
evaluate the effects of anonymization, one must start with
unanonymized data. However, this LBNL data set is already
anonymized. So one has no clean baseline with which to compare the
anonymized data in this case.

\section{Results and Analysis}
\label{sec:results-analysis}
Our objective was to begin to understand the affect of anonymization on the the utility of a log. There are two factors that affect the utility of a log, the anonymization algorithm and the field which is anonymized. We first look at how the field that is anonymized affects utility.

We evaluated all 152 pairs of fields and anonymization algorithms. For each pair the number of alerts generated by Snort was calculated. The alerts generated for each pair was compared with the baseline alerts. False Positives/False negatives were calculated based on the definitions above. Table~\ref{table:raw-anony1} and Table~\ref{table:raw-anony2} list the results from most of the pairs of anonymization algorithms and fields.

ICMP related fields (such as ICMP\_CHECKSUM, ICMP\_COE etc) were excluded since we filtered ICMP packets from the log. There remained a total of 81 field-anonymization pairs.

The unanonymized file produced 81 alerts. They number and types of alerts produced are summarized in Table~\ref{table:baseline-alerts}. There are a total of 81 alerts generated in the baseline file, but only 19 unique types of alerts.

\begin{table}[t]
  \begin{tabular}[t]{llp{2in}}
    {\textbf{AlertID}} & {\textbf{Num}} & \textbf{Desc} \\
    1 & 1 & (portscan) TCP Portscan \footnote{This alert was generated from generator 122} \\
    323 & 1 & FINGER root query \\
    330 & 1 & FINGER redirection attempt \\
    332 & 1 & FINGER 0 query \\
    356 & 1 &  FTP passwd retrieval attempt \\
    359 & 1 & FTP satan scan \\
    503 & 4 & MISC Source Port 20 to <1024 \\
    1200 & 9 & ATTACK-RESPONSES Invalid UR \\
    1201 & 12 & ATTACK-RESPONSES 403 Forbidden \\
    1288 & 4 &  WEB-FRONTPAGE /\_vti\_bin/ access \\
    1292 & 30 &  ATTACK-RESPONSES directory listing \\
    1418 & 2 & SNMP request tcp \\
    1420 & 2 &  SNMP trap tcp \\
    1421 & 1& SNMP AgentX/tcp request\\
    2467 & 1 & NETBIOS SMB D\$ unicode share access \\
    2470 & 1 & NETBIOS SMB C\$ unicode share access \\
    2473 & 1 & NETBIOS SMB ADMIN\$ unicode share access \\
    3151 & 6 &  FINGER / execution attempt \\
    3441 & 2 & FTP PORT bounce attempt \\
  \end{tabular}
  \caption{Alerts generated in the baseline unanonymized file. \textbf{AlertID} is the id of the alert; \textbf{Num} is the number of that type of alert generated; \textbf{Desc} is a description of the alert}
  \label{table:baseline-alerts}
\end{table}

Table~\ref{table:raw-anony1} and Table~\ref{table:raw-anony2} provided in the Appendix, contain the number of alerts generated for each pair of field and anonymization algorithm. However, we will be focusing on the summarized results here.

To evaluate the affect of anonymizing a particular field, we combine the alerts generated for a particular field over all the anonymization algorithms applied to it. Table~\ref{table:marginal-field-alerts} shows the number of alerts generated for a field, averaged over all the anonymization algorithms. Table~\ref{table:marginal-field-alerts} also shows the number of false positives, false negatives, and error.

We can see that several of the fields resulted, in, on average, only 81 alerts. These fields (shown in italics in the table) had 0 error. Judging from these results, we can see that this set of fields did not affect the utility of the log as measured by the IDS metric.

TCP\_DST\_PORT and TCP\_SRC\_PORT generated the most alerts on average, as well as the most error. Table~\ref{table:raw-anony1} lists the number of alerts for each anonymization algorithm-field pair. We can see that most of the pairs generated only 2 types of alerts. In the case of BinaryBlackMarker there were 520286 alerts of type 524 generated. Alert 524 is ``BAD-TRAFFIC tcp port 0 traffic''. The other anonymization algorithms produced the same pattern --- the majority of alerts were of type 524.

The reason for this is due to what we substitute for the port field. The BinaryBlackMarker, Substitution, NumericTruncation, and Annihilation algorithms replaced the port field with $0$. This resulted in the alert for nearly all the packets in the log (see Table~\ref{tab:anonyparam} for the parameter settings of the anonymization algorithms). the same problem occurs for the TCP\_SRC\_PORT field.

The RandomPermutation algorithm replaced the port number with another, random port number, thus not causing the generation of the BAD-TRAFFIC alert. Clearly the choice of parameter setting can cause a large effect on the utility of a log.

Table~\ref{table:marginal-field-alerts} indicates the impact on utility for anonymizing particular fields. From this data, we can determine a rough ordering for fields in terms of their impact on utility. The field that can affect utility most would be the source and destination port fields. However, much of this is due to the parameter settings. Given a ``correct'' parameter setting the alerts might be reduced.

Following the port field are the source and destination ip address fields. It is surprising that the IP address impacts the utility of the log.  The false positive and false negative scores indicate that anonymizing the fields resulted in the generation of alerts outside of the baseline. Table~\ref{table:raw-anony1} indicates some interesting differences between anonymization algorithms. Further investigation is warranted to decide why the IP address causes such an impact.

We can clearly see that the choice of field impacts utility. However, we do not know how the anonymization algorithm affects the utility. We have seen some instances of fields where the choice of anonymization algorithm (and even parameters of the algorithm) can cause large differences. Given this, we would expect significant differences between the utility of two fields anonymized with different algorithms. To detect this difference we calculated the number of alerts generated by a particular anonymization algorithm, over all the fields. The results of this are shown in Table~\ref{table:marginal-anony-alerts}.

However, Table~\ref{table:marginal-anony-alerts} will be skewed by the effect of the field. As can be seen, the five algorithms used to anonymize the port fields have a large number of alerts which overshadow the affect of the algorithms on other fields.

To get a better look, we looked at the variance of alerts field by field. For each field we calculated the standard deviation of alerts over anonymization algorithms. Table~\ref{table:marginal-field-alerts-std} contains this data. We can see that the choice of anonymization algorithm does affect the utility of a log. The impact of a particular anonymization algorithm needs to be studied further, but it is clear that it does affect the utility.

\begin{figure*}[t]
  \includegraphics[width=3in]{./img/Field-AvgError.pdf}
  \caption{Graph of error averaged over anonymization algorithm. For every policy take the average error generated over all anonymization algorithms. Note that only the fields TCP\_SRC\_PORT and TCP\_DST\_PORT provide large error. }
\end{figure*}

\begin{figure*}[t]
  \includegraphics[width=3in]{./img/AnonyA-AvgError.pdf}
  \caption{Error averaged over fields. For every policy take the average error generated over all fields. Note that substitution has significantly higer error than the other anonymization algorithms.}
\end{figure*}
\input{rawAlertData.tex}

\section{Results and Analysis}
\label{sec:results-analysis}

In this section, we describe the results of our experiments with single
field anonymization policies. These experiments provide a substantive
start to answering these questions:

\begin{itemize}
\item What affects utility more, the fields that are anonymized or the anonymization algorithm?
\item Which fields have a larger impact on the utility of a log?
\item Which anonymization algorithms have a larger impact on the utility of a log?
\end{itemize}

To answer these questions, we evaluated all 152 pairs of fields and
anonymization algorithms. For each pair, the number of alerts
generated by Snort was calculated. The alerts generated for each pair
were compared with the baseline alerts (See
Table~\ref{table:raw-anony1} and Table~\ref{table:raw-anony2}). False
Positives/False negatives were calculated based on the definitions
above, whose detailed results we discuss later.

Although we evaluated all 152 field-anonymization algorithm pairs, we did not include ICMP related fields (such as ICMP\_CHECKSUM, ICMP\_CODE etc) in the results presentend below, since we filtered ICMP packets from the log. There remains a total of 81 field-anonymization pairs.

The unanonymized file produced 81 alerts. The number and types of
alerts produced are summarized in
Table~\ref{table:baseline-alerts}. There are a total of 81 alerts
generated in the baseline file, but only 19 unique types of alerts.

\input{rawAlertData.tex}

\begin{table}[htp]
  \caption{Alerts generated in the baseline unanonymized file. \textbf{AlertID} is the id of the alert; \textbf{Num} is the number of that type of alert generated; \textbf{Desc} is a description of the alert}
  \label{table:baseline-alerts}
  \begin{tabular}[t]{llp{2in}}
    {\textbf{AlertID}} & {\textbf{Num}} & \textbf{Desc} \\
    1 & 1 & (portscan) TCP Portscan \footnote{This alert was generated from generator 122} \\
    323 & 1 & FINGER root query \\
    330 & 1 & FINGER redirection attempt \\
    332 & 1 & FINGER 0 query \\
    356 & 1 &  FTP passwd retrieval attempt \\
    359 & 1 & FTP satan scan \\
    503 & 4 & MISC Source Port 20 to <1024 \\
    1200 & 9 & ATTACK-RESPONSES Invalid UR \\
    1201 & 12 & ATTACK-RESPONSES 403 Forbidden \\
    1288 & 4 &  WEB-FRONTPAGE /\_vti\_bin/ access \\
    1292 & 30 &  ATTACK-RESPONSES directory listing \\
    1418 & 2 & SNMP request tcp \\
    1420 & 2 &  SNMP trap tcp \\
    1421 & 1& SNMP AgentX/tcp request\\
    2467 & 1 & NETBIOS SMB D\$ unicode share access \\
    2470 & 1 & NETBIOS SMB C\$ unicode share access \\
    2473 & 1 & NETBIOS SMB ADMIN\$ unicode share access \\
    3151 & 6 &  FINGER / execution attempt \\
    3441 & 2 & FTP PORT bounce attempt \\
  \end{tabular}
\end{table}

\subsection{Fields or Anonymization Algorithm?}
It is important to understand which causes a greater impact on utility; the field that is being anonymized or the anonymization algorithm that is being applied.

To evaluate the effects of anonymizing a field we calculate the \emph{marginal} of a field. The marginal of a field is the average number of false positive/false negatives over all anonymization algorithms. Similarly the marginal of an anonymization algorithm is the false positive/false negative rate averaged over all fields. The marginal provides a concise summary of the effect of anonymizing a field.

Figure~\ref{fig:marg-fieldvsfpfn} shows the false positive and false negative marginals of the anonymization algorithms. The full data for this graph is in Table~\ref{tab:marg-fieldFNFP}.

\begin{figure*}[htp]
  \begin{tabular}{cc}
    \includegraphics[width=3in]{./img/marg-fieldvsfp.pdf} & \includegraphics[width=3in]{./img/marg-fieldvsfn.pdf} \\
  \end{tabular}
  \caption{The left hand chart shows the average number of false positive alerts generated for each field. The right hand chart shows the average number of false negative alerts generated for each field.}
  \label{fig:marg-fieldvsfpfn}
\end{figure*}

Figure~\ref{fig:marg-fieldvsfpfn} indicates that the majority of fields generate no false positives or false negatives on average. The fields for which this is true (such as DST\_MAC) did not affect the utility of the log under any anonymization algorithm. We can conclude that anonymizing these fields has no impact on the utility of a log, with respect to the IDS metric.

The fact that the majority of fields have no impact on utility, even when anonymized in numerous ways, indicates that fields are more important than the anonymization algorithm in determining the utility of an anonymized log. If it was the other way around, we would expect false positives and false negatives to be distributed over all the fields.

In contrast, if we do the same with anonymization algorithms we see that the majority of anonymization algorithms generate false positives and false negatives, suggesting again that the field has a greater impact on the utility of the log than the anonymization algorithm. Figure~\ref{fig:marg-anonyvsfpfn} shows marginals for anonymization algorithms (averaged over fields). Table~\ref{tab:marg-anonyFNFP} contains the data for the graph.

We can see in Figure~\ref{fig:marg-anonyvsfpfn} that in terms of false negatives, there are more alerts spread over the set of anonymization algorithms.

\subsection{Which fields have high impact on utility?}
As Figure~\ref{fig:marg-fieldvsfpfn} and Table~\ref{tab:marg-fieldFNFP} clearly show, the majority of fields do not generate false positives (on average). The fields that do are TCP\_DST\_PORT, TCP\_SRC\_PORT,\\ IPV4\_SRC\_IP, IPV4\_DST\_IP, TS\_SEC, \\ and TCP\_ACK\_NO.

We can see that several of the fields resulted, in, on average, only 81 alerts. These fields (shown in italics in the table) had 0 error. Judging from these results, we can see that this set of fields did not affect the utility of the log as measured by the IDS metric.

TCP\_DST\_PORT and TCP\_SRC\_PORT generated the most alerts on
average. Upon inspection of the generated alert files, we can see that
most of the pairs generated only 2 types of alerts. In the case of
BinaryBlackMarker there were 520286 alerts of type 524
generated. Alert 524 is ``BAD-TRAFFIC tcp port 0 traffic''. The other
anonymization algorithms produced the same pattern---the majority of
alerts were of type 524.

The reason for this is due to what we substitute for the port
field. The BinaryBlackMarker, Substitution, NumericTruncation, and
Annihilation algorithms replaced the port field with $0$. This
resulted in the alert for nearly all the packets in the log (See
Table~\ref{tab:anonyparam} for the parameter settings of the
anonymization algorithms). the same problem occurs for the
TCP\_SRC\_PORT field.

The RandomPermutation algorithm replaced the port number with another,
random port number, thus not causing the generation of the BAD-TRAFFIC
alert. It is clear from this, that the false positive rate was greatly
affected by the choice of the substitution port, and another useful
metric may be to count types of new alerts rather than the raw alerts.

Arguably, it is the false negative count that is most important in determining the utility of a log. A low false negative count indicates that no information was lost in the process of anonymization. In terms of false negatives, we find that there are $8$ fields that have an impact on the average false negative rate: TCP\_DST\_PORT,TCP\_FLAGS,TCP\_SRC\_PORT, \\ IPV4\_DST\_IP,IPV4\_SRC\_IP,TCP\_ACK\_NO, \\ TCP\_SEQUENCE,TS\_SEC

\subsection{What anonymization algorithms have high impact on utility?}

Figure~\ref{fig:marg-anonyvsfpfn} summarizes the false positive and
false negative rates with respect to anonymization algorithm. For each
anonymization algorithm, the average false positive/false negative rate
is calculated over all the fields. Table~\ref{tab:marg-anonyFNFP}
contains the data for the graphs.

\begin{figure*}[htp]
  \begin{tabular}{cc}
  \includegraphics[width=3in]{./img/marg-anonyvsfp.pdf} & \includegraphics[width=3in]{./img/marg-anonyvsfn.pdf} \\
  \end{tabular}

  \caption{The left hand chart shows the average number of false positive alerts generated for each anonymization algorithm. The right hand chart shows the average number of false negative alerts generated for each anonymization algorithm.}
  \label{fig:marg-anonyvsfpfn}
\end{figure*}

We can see from these that most anonymization algorithms have an impact on the utility of a log. In contrast, the field data that we saw before showed strong structure in what fields affected utility. When veiwed from the perspective of anonymization algorithms, there is no single anonymization algorithm that stands out.

It might seem like the substitution algorithm has the largest number
of alerts. However, this is because of the parameter setting used. By
looking at the false negative rate we can see that while substitution
still has a large affect on utility, most of the other algorithms have an
affect as well.

\input{AnonyMarginal.tex} \input{FieldMarginal.tex}

\section{Results and Analysis}
\label{sec:results-analysis}

In this section, we describe the results of our experiments with single
field anonymization policies. These experiments provide a substantive
start to answering these questions:

\begin{itemize}
\item What affects utility more, the fields that are anonymized or the anonymization algorithm?
\item Which fields have a larger impact on the utility of a log?
\item Which anonymization algorithms have a larger impact on the utility of a log?
\end{itemize}

To answer these questions, we evaluated all 152 pairs of fields and
anonymization algorithms. For each pair, the number of alerts
generated by Snort was calculated. The alerts generated for each pair
were compared with the baseline alerts (See
Table~\ref{table:raw-anony1} and Table~\ref{table:raw-anony2}). False
Positives/False Negatives were calculated based on the definitions
above, whose detailed results we discuss later.

The unanonymized file produced 81 alerts. The number and types of
alerts produced are summarized in
Table~\ref{table:baseline-alerts}. There are a total of 81 alerts
generated in the baseline file, but only 19 unique types of alerts.

\input{rawAlertData.tex}

\begin{table}[htp]
 \caption{Alerts generated in the baseline unanonymized file. \textbf{AlertID} is the id of the alert; \textbf{Num} is the number of that type of alert generated; \textbf{Desc} is a description of the alert}
  \label{table:baseline-alerts}
  \begin{tabular}[t]{llp{6in}}
    {\textbf{AlertID}} & {\textbf{Num}} & \textbf{Desc} \\
    1 & 1 & (portscan) TCP Portscan \footnote{This alert was generated from generator 122} \\
    323 & 1 & FINGER root query \\
    330 & 1 & FINGER redirection attempt \\
    332 & 1 & FINGER 0 query \\
    356 & 1 &  FTP passwd retrieval attempt \\
    359 & 1 & FTP satan scan \\
    503 & 4 & MISC Source Port 20 to <1024 \\
    1200 & 9 & ATTACK-RESPONSES Invalid URL \\
    1201 & 12 & ATTACK-RESPONSES 403 Forbidden \\
    1288 & 4 &  WEB-FRONTPAGE /\_vti\_bin/ access \\
    1292 & 30 &  ATTACK-RESPONSES directory listing \\
    1418 & 2 & SNMP request tcp \\
    1420 & 2 &  SNMP trap tcp \\
    1421 & 1& SNMP AgentX/tcp request\\
    2467 & 1 & NETBIOS SMB D\$ unicode share access \\
    2470 & 1 & NETBIOS SMB C\$ unicode share access \\
    2473 & 1 & NETBIOS SMB ADMIN\$ unicode share access \\
    3151 & 6 &  FINGER / execution attempt \\
    3441 & 2 & FTP PORT bounce attempt \\
  \end{tabular}
\end{table}

\subsection{Fields or Anonymization Algorithms?}
It is important to understand which causes a greater impact on utility; the field that is being anonymized or the anonymization algorithm that is being applied.

To evaluate the effects of anonymizing a field we calculate the
\emph{marginal} of a field. The marginal of a field is the average
number of false positive/false negatives over all anonymization
algorithms. Similarly, the marginal of an anonymization algorithm is
the false positive/false negative rate averaged over all fields. The
marginal provides a concise summary of the effect of anonymizing a
particular field or using a particular anonymization algorithm.

Figure~\ref{fig:marg-fieldvsfpfn} shows the false positive and false negative marginals of the fields. The full data for these graphs is in Table~\ref{tab:marg-fieldFNFP}. Figure~\ref{fig:marg-anonyvsfpfn} shows the false positive and false negative marginals of the anonymization algorithms. The full data for these graphs is in Table~\ref{tab:marg-anonyFNFP}.

\begin{figure*}[htp]
  \begin{tabular}{cc}
    \includegraphics[width=3in]{./img/FieldVsFalsePos.pdf} & \includegraphics[width=3in]{./img/marg-fieldvsfn.pdf} \\
  \end{tabular}
  \caption{The left hand chart shows the marginal of each field with respect to false positives (i.e, the average number of false positives for a field, averaged over all anonymization algorithms). The right hand chart shows the marginal of each field with respect to false negatives.}
  \label{fig:marg-fieldvsfpfn}
\end{figure*}

\begin{figure*}[htp]
  \begin{tabular}{cc}
  \includegraphics[width=3in]{./img/marg-anonyvsfp.pdf} & \includegraphics[width=3in]{./img/marg-anonyvsfn.pdf} \\
  \end{tabular}
  \caption{The left hand chart shows the marginal of all the anonymization algorithms, with respect to false positives. The right hand chart shows the marginal of all anonymization algorithms with respect to false negatives.}
  \label{fig:marg-anonyvsfpfn}
\end{figure*}

The fact that the majority of fields have no impact on utility, even
when anonymized in numerous ways, indicates that fields are more
important than the anonymization algorithm in determining the utility
of an anonymized log. If it was the other way around, we would expect
false positives and false negatives to be more evenly distributed over
all the fields.

Figure~\ref{fig:marg-fieldvsfpfn} indicates that the majority of
fields generate no false positives or false negatives. The fields for
which this is true (such as DST\_MAC) did not affect the utility of
the log under any anonymization algorithm. We can conclude that
anonymizing these fields has no impact on the utility of a log, with
respect to the IDS metric.

Consider the false positive rate first. We can see that very few
fields generated any false positives. This indicates that
anonymization, usually, does not add new patterns.

\subsection{Which fields have higher impact on utility?}
As Figure~\ref{fig:marg-fieldvsfpfn} and
Table~\ref{tab:marg-fieldFNFP} clearly show, the majority of fields do
not generate false positives. The fields that do are TCP\_DST\_PORT,
TCP\_SRC\_PORT, IPV4\_SRC\_IP, IPV4\_DST\_IP, TS\_SEC, and
TCP\_ACK\_NO.

We can see that several of the fields resulted in an average of 81
alerts. These fields (shown in italics in the table) had 0
error. Judging from these results, we can see that this set of fields
did not affect the utility of the log as measured by the IDS metric.

TCP\_DST\_PORT and TCP\_SRC\_PORT generated the most false alerts on
average. Upon inspection of the generated alert files, we can see that
most of the pairs generated only 2 types of alerts. In the case of
BinaryBlackMarker there were 520286 alerts of type 524. Alert 524 is ``BAD-TRAFFIC tcp port 0 traffic''. The other
anonymization algorithms produced the same pattern---the majority of
alerts were of type 524.

The reason for this is the value we substitute for the port field. The
BinaryBlackMarker, Substitution, NumericTruncation, and Annihilation
algorithms all replaced the port field with $0$. This resulted in the
alert being triggered for nearly all the packets in the log (See
Table~\ref{tab:anonyparam} for the parameter settings of the
anonymization algorithms). The same problem occurs for the
TCP\_SRC\_PORT field.

The RandomPermutation algorithm replaced the port number with another,
random port number, thus infrequently causing the generation of the BAD-TRAFFIC
alert. It is clear from this that the false positive rate was greatly
affected by the choice of the substitution port. However, the effect
would have been less pronounced had we counted the types of new alerts
rather than the raw alerts, themselves.

Arguably, it is the false negative count that is most important in
determining the utility of a log. A low false negative count indicates
that little information was lost in the process of anonymization. In terms
of false negatives, we find that there are $8$ fields that have an
impact on the average false negative rate:
TCP\_DST\_PORT,TCP\_FLAGS,TCP\_SRC\_PORT, IPV4\_DST\_IP, IPV4\_SRC\_IP,TCP\_ACK\_NO, TCP\_SEQUENCE, and TS\_SEC.

\subsection{What anonymization algorithms have higher impact on utility?}

Figure~\ref{fig:marg-anonyvsfpfn} summarizes the false positive and
false negative rates with respect to anonymization algorithm. For each
anonymization algorithm, the average false positive/false negative rate
is calculated over all the fields. Table~\ref{tab:marg-anonyFNFP}
contains the data for the graphs.

We can see from these that most anonymization algorithms have an
impact on the utility of a log. In contrast, the field data that we
saw before showed strong structure in what fields affected
utility. When viewed from the perspective of anonymization algorithms,
there is no single anonymization algorithm that stands out.

We can see from these that most anonymization algorithms have an
impact on the utility of a log. In contrast, the field data that we
saw before showed strong structure in what fields affected
utility. When viewed from the perspective of anonymization algorithms,
there is no single anonymization algorithm that stands out.

It might seem like the substitution algorithm has the largest effect
with a huge number of alerts. However, this is because of the
parameter setting used. By looking at the false negative rate we can
see that while substitution still has a large effect on utility, most
of the other algorithms have an effect as well.

\input{AnonyMarginal.tex} \input{FieldMarginal.tex}