\section{Introduction}
\label{intro}
The rare event search experiments, such as dark matter direct detections and neutrinoless double beta decay experiments, are operated in deep underground laboratories with passive and active shields as well as selecting radiopure materials to effectively reduce their intrinsic background \cite{Heusser1995, Abgrall2016, Akerib2020, Ma2021}.
Under the ultra-low background achieved by substantial efforts, the radioactive impurities in the selected materials induced by cosmogenic activation on the ground could become even more prominent than the residual contamination of the primordial nuclides.
\par
With high radiopurity and attractive mechanical properties, copper is widely innermost used as shields or parts of the detectors in rare event search experiments \cite{Jiang2018,Agostini2017,Abgrall2014}.
However, radioisotopes produced by the cosmogenic activation during manufacture, transport, and storage, disrupt the background suppression in copper.
Among these cosmogenic nuclides, short-lived nuclides like ${^{59}}$Fe and ${^{48}}$V decay rapidly when stored in the underground laboratories.
In contrast, long-lived nuclides like ${^{60}}$Co and ${^{54}}$Mn remain and continue to contribute to the detector's background level \cite{Mei2009, Ma2018}.
\par
The China dark matter experiment (CDEX) aiming at direct dark matter detection and neutrinoless double beta decay of ${^{76}}$Ge operates p-type point-contact germanium detectors (PPCGe) at the China Jinping underground Laboratory (CJPL) \cite{Jiang2018, Liu2019, Yang2019, She2020, Wang2020}.
The copper used in the PPCGe detectors of the CDEX experiment leads to a crucial background contribution.
It is necessary to assess the background due to cosmogenic activations in copper to establish the background model for the CDEX experiment.
However, the production rates of cosmogenic activations differs significantly among different simulation packages according to previous reseraches \cite{Cebri2010, Wei2017}
\par
In this study, we present the measurements of the cosmogenic activation in the copper samples before and after exposure to the cosmic rays.
In comparison with the measured results, the corresponding radionuclide production rates are also calculated by Monte Carlo simulations.
Finally, we simulate the background spectra of cosmogenic radionuclides in the copper components of the PPCGe detector array used in the future CDEX experiment under the expected exposure history of detectors above the ground.
\par

\section{Experiment and methods}
The oxygen-free high thermal conductivity (OFHC) copper bricks (their chemical purities are more than 99.995\% with natural isotope abundance), 20 ${\times}$ 10 ${\times}$ 5 cm in size, were exposed to cosmic rays and measured in this study.
The cosmic-ray muon flux in the CJPL is suppressed by 8 orders of magnitude compared to surface laboratories due to the 2400 m rock overburden \cite{Wu2013, Guo2021}.
These OFHC copper bricks have been housed in the CJPL for more than 4 years.
These OFHC copper samples were measured to evaluate the residual cosmogenic radioactivity in advance with a low background germanium gamma-ray spectrometer GeTHU at CJPL, whose background rates between 100 and 2700 keV is 732 counts/day \cite{Zeng2014}.
To enhance the detection efficiency, the germanium detector of GeTHU were surrounded by these copper sample bricks as shown in Figure~\ref{shielding}.
The measured spectrum is depicted in Figure~\ref{exposure} and the characteristic gamma-ray peaks from cosmogenic radionuclides are not statistically significant and lower than the minimum detectable activities (MDA) of GeTHU.
\par
\begin{figure}[htb]
    \centering
	\includegraphics[width=\columnwidth]{./CopperBricks.pdf}
    \caption{\textrm{ The schematic diagram of the palcement of the OFHC copper bricks in the spectrometer: right view (left) and top view (right).}}
    \label{shielding}
\end{figure}
Sixteen copper bricks with a total mass of 142.50 kg were exposed to cosmic-rays for 504 days, from Aug. 31, 2019 to Jan. 16, 2021, at an altitude of 2469.4 m in the vinicity of CJPL.
The specific activities of cosmogenic radionuclides inside these copper bricks are calculated by spectral analysis.
In spectral analysis, the net area of a peak is determined under the assumption of a linear background \cite{Gilmore2008}.
Besides, the detection efficiencies for concerned radionuclides are simulated by Geant4.10.06 \cite{Agostinelli2003, Allison2006, Allison2016}.
\par
The specific activities of cosmogenic radionuclides can be deduced by combining the production rates and the decay rates as shown in Figure~\ref{yield}.
\par
\begin{equation}
\label{yield}
  A = R(1-e^{-\lambda t})+A_0e^{-\lambda t},
\end{equation}
Where ${A}$ is the specific activity in the unit of Bq/kg and ${R}$ denotes the production rate of a particular cosmogenic radionuclide in unit of Bq/kg.
${\lambda}$ represents the decay constant and ${t}$ is the time during cosmogenic activation.
${A_0}$ stands for specific activities before exposure.
The specific activities of the cosmogenic radionuclides are unsaturated after 504 days of exposure for radionuclides with long half-lives, such as ${^{60}}$Co.
We have applied the exponential hypothesis as shown in (\ref{yield}) with zero ${A_0}$ to calculate the production rates due to the copper sample measurement mentioned above.
The decay of cosmogenic radionuclides during the measurements was also considered, especially for those with short half-lives, such as ${^{58}}$Co.
\par
The cosmogenic production rate ${R}$ can be calculated and simplified as shown in (\ref{activation}).
\begin{equation}
    \label{activation}
    R = \int \frac{d\Phi(E)}{dE}\sigma(E) dE \approx \Phi_{tot}\int \frac{d\phi(E)}{dE}\sigma(E) dE \propto \Phi_{tot},
\end{equation}
${\frac{d\Phi(E)}{dE}}$ is the energy spectrum of cosmic rays while the ${\frac{d\phi(E)}{dE}}$ is the normalized energy spectrum.
${\Phi_{tot}}$ is the flux of cosmic rays.
As the energy spectra differ slightly at different altitudes within 20 km \cite{Goldhagen2002}, the normalized energy spectra at different altitudes can be simplified as a constant.
The
efore, the cosmogenic production rates are directly proportional to the strength of the cosmic-ray flux.
The cosmic-ray flux at a given altitude can be estimated by exponential correction \cite{Lifton2014}.
The altitude corrections on the cosmic-ray flux according to (\ref{Intensity}) is applied to calculate the cosmic-ray flux at different altitudes.
\begin{equation}
    \label{Intensity}
    \Phi_{tot, H} = \Phi_{tot, 0}e^{\frac{p(H)-p(0)}{L}},
\end{equation}
where ${\Phi_{tot, H}}$ and ${\Phi_{tot, 0}}$ indicate the flux of certain particles in the cosmic ray at altitudes ${H}$ and sea level in unit of particles/cm${^2}$/s, respectively.
${p(H)}$ is the altitude pressure at altitude ${H}$ \cite{Ziegler1996, Lifton2014}and ${L}$ is the typical absorption length and specified for a certain cosmic-ray component.
It is convinced that the production rates of the cosmogenic activations are dominated by neutrons so that the absorption length of the neutron is a suitable choice \cite{Breier2020, Cebri2020}.
The cosmogenic production rates are proptional to the neutron flux leading to the same exponential correction can be applied to get the corresponding production rates at sea level as shown in (\ref{production})
\begin{equation}
    \label{production}
    R(H) = R(0)e^{\frac{p(H)-p(0)}{L}},
\end{equation}
For comparison with measurements, the cosmogenic production rates are also simulated by Geant4 with the physical list QGSP\_INCLXX \cite{Allison2006,Allison2016,Apostolakis2009}.
CRY-1.7 \cite{Hagmann2007} provides the energy spectra of different cosmic-ray components (neutron, proton, muon, gamma, electron and pion) at sea level for the Geant4 simulation.
In the simulation, we count each product after the interactions with the copper sample for the production rate calculation.

\begin{figure}[htb]
    \centering
    \includegraphics[width=\columnwidth]{./afterExposure.pdf}
    \caption{The measured spectra of the OFHC copper samples pre- (in blue) and post-exposure (in black). The gamma-ray peaks induced by cosmogenic activations and primordal radionuclides are also indicated. }
    \label{exposure}
\end{figure}

\section{Cosmic ray activation in copper}
\label{Activation}
These copper bricks were measured from Jan 16th to 30th, 2021 and the spectrum after exposure is also shown in Figure~\ref{exposure}, in which the selected gamma-ray peaks of cosmogenic radionuclides are marked.
In addition to the characteristic gamma-ray peaks from cosmogenic radionuclides, there also exist several gamma-ray peaks related to the primordial radionuclides and the 511 keV annihilation peak.
Compared with pre-expoure spectrum, the background level gets worse due to the break-down of the low-radon air system of the laboratory when measuring these bricks after exposure.
GeTHU has a
efilling system flushing its sample space with the low-radon air (with boil-off nitrogen gas previously) to mitigate radon and its daughters \cite{Zeng2014}.
Without flushing, the concentrations of radon and its progenies in the sample space get higher and contribute to the excessive background rates and higher gamma-ray peak rates at a lower energy region (from 238 keV to 609 keV) as shown in the Figure~\ref{exposure}.
However, it is evident that these cosmogenic gamma-ray peaks are still significant enough to calculate their specific activities.
Following the aforementioned gamma-ray spectral analysis method, the specific activities of these cosmogenic radionuclides are calculated and listed in Table~\ref{nuclides}.
\par
\begin{table}
    \centering
    \caption{The measured specific activities of cosmogenic radionuclides after exposure (with 1 ${\sigma}$ uncertainty) in copper samples. The gamma-ray peaks with better statistics are selected to determine the specific activities when radionuclides have two or more gamma-ray peaks.}
    \label{nuclides}
    \begin{threeparttable}
        \begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}}llll@{}}
        \toprule
        Radionuclides& Half-lives / (d)& Specific activity / (mBq/kg)\\
        \hline
            $^{54}$Mn&  312.20&  ${0.871 \pm 0.095}$\\
            $^{56}$Co&   77.24&  ${0.649 \pm 0.084}$\\
            $^{57}$Co&  271.74&  ${2.423 \pm 0.275}$\\
            $^{58}$Co&   70.86&  ${3.394 \pm 0.166}$\\
            $^{60}$Co& 1925.28&  ${0.464 \pm 0.070}$\\
        \bottomrule
        \end{tabular*}
    \end{threeparttable}
\end{table}
After the altitude correction, the cosmogenic production rates are reported in \c
ef{research}, compared with results from previous studies.
The results of our measurements are generally consistent with those by the simulation with Geant4 except for ${^{56}}$Co.
The specific activity of ${^{57}}$Co is about 3\% smaller than simulation while that of ${^{56}}$Co is 49\% of the simulation result.
In addition, Geant4 simulation result in \cite{Zhang2016} is only half of our simulated result for ${^{56}}$Co mainly due to the differences of the physical lists between 'shielding' and 'QGSP\_INCLXX'and copper samples used in these two simulations.
Due to its short half-life, 77.2 days, the ${^{56}}$Co will decay rapidly when stored underground resulting in a negiligible background contribution.
Thus, the difference between simualtion and measurement is not the prominent obstacle to assess the cosmogenic background of the underground experiments.
Our measurements are relatively close to those measured by Ref. \cite{Baudis2015}.
The discrepancies of the production rates in the literatures could be due to the different cross-section databases, uncertainties of altitude correction, and the fluxes and energy spectra of cosmic rays in different experimental latitudes.
\par
\begin{table*}[htb]
    \begin{threeparttable}
        \centering
        \caption{Production rates of cosmogenic radionuclides in the copper bricks at sea level (Unit: kg${^{-1}}$ d${^{-1}}$).}
        \label{research}
        \begin{tabular*}{\textwidth}{@{\extracolsep{\fill}}lrrrrrl@{}}
            \toprule
            & \multirow{2}*{Method}& \multicolumn{5}{c}{Nuclides} \\
            \cmidrule{3-7} & & ${^{54}}$Mn& ${^{56}}$Co& ${^{57}}$Co& ${^{58}}$Co& ${^{60}}$Co\\
            \hline
            \multirow{2}*{This work}& Measurement& ${18.6 \pm 2.0}$& ${9.9 \pm 1.3}$& ${48.3 \pm 5.5}$&	${51.8 \pm 2.5}$& ${39.7 \pm 5.7}$\\
            ~& Geant4& 21.1& 20.4& 49.6& 70.9& 44.1\\[5pt]
            R. Breier et al \cite{Breier2020}& CONUS \cite{Masarik1999}& 14& 10& 50& 76& 92\\[5pt]
            \multirow{2}*{C. Zhang et al\cite{Zhang2016}}& Geant4& 12.31& 10.32& 67.15& 57.26& 64.63\\
            ~& ACTIVIA\tnote{1}& 14.32/30.00& 8.74/20.13& 32.44/77.45& 56.61/138.06& 26.28/66.12\\[5pt]
            S. Cebrian et al \cite{Cebri2010}& MENDL+YIELDX\tnote{2}& 32.5/27.7& 22.9/20.0& 88.3/74.1& 159.6/123.0& 97.4/55.4\\[5pt]
            \multirow{2}*{L. Baudis et al\cite{Baudis2015}}& Measurement& ${13.3^{+3.0}_{-2.9}}$& ${9.3^{+1.2}_{-1.4}}$& ${44.8^{+8.6}_{-8.2}}$& ${68.9^{+5.4}_{-5.0}}$& ${29.4^{+7.1}_{-5.9}}$\\
            ~& COSMO & 13.5& 7.0& 30.2& 54.6&25.7\\[5pt]
            M. Laubenstein et al\cite{Laubenstein2009}& Measurement& ${8.85\pm0.86}$& ${9.5\pm1.2}$& ${73.8\pm16.7}$& ${67.9\pm3.7}$& ${86.2\pm7.6}$\\[5pt]
        \bottomrule
        \end{tabular*}
        \begin{tablenotes}
            \item[1]{These two calculations rely on two cosmic neutron spectra from Gordan and ACTIVIA \cite{Gordan2004}}
            \item[2]{These two calculations rely on two cosmic neutron spectra from Ziegler and Gordan, respectively \cite{Gordan2004,Ziegler1998}}
    \end{tablenotes}
    \end{threeparttable}
\end{table*}
\par

\section{Background assessment in PPCGe detector}
The next phase of CDEX experiment, named CDEX-100, will operate about 100 kg germanium detectors immersed in the liquid nitrogen in a 1725-m$^{3}$ cryotank at the Hall-C of the second phase of CJPL.
The specific activities of cosmogenic radionuclides are calculated based on their production rates and the expected exposure hitory.
The total exposure period is a combination of manufacture, transportation above ground and storage underground.
Their specific activities can be calculated as (\ref{activity})
\begin{align}
    A = & R(0)e^{p(H_1)/L}(1-e^{-\lambda t_{manu}})e^{-\lambda(t_{trans}+t_{cool})} \nonumber \\
	&+ F \times R(0)e^{p(H_2)/L}(1-e^{-\lambda t_{trans}})e^{-\lambda t_{cool}},
    \label{activity}
\end{align}
The ${R(0)}$ is the measured production rates at sea level shown in the Table~\ref{research}.
The ${H_1}$ and ${H_2}$ are the altitudes of the manufacture factory and the transportation routine above ground.
The time of manufacture, transportation above ground and storage underground are represented with ${t_{manu}}$, ${t_{trans}}$ and ${t_{cool}}$, respectively.
The factor ${F}$ is the factor describing the shielding effect when transportation with a steel or lead surroudings.
\par
For simplicity, we assume that the copper components follow the same exposure history of the detectors.
In the expected exposure history, the manufecture time, transportation time and underground storage time are supposed to be 3 months, half a month and one year.
After 3 month fabrication, the detectors are transported with a shielding, in which the cosmogenic activations can be reduced by a factor of 10 \cite{Ma2018,Abgrall2014}, to the underground laboratory in half a month.
The altitude of manufacture factory is selected to be at sea level while that of the whole transportation routine is selected to be 1500m, which is the average altitude of Liangshan P
efecture where CJPL is located.
Then, the cupreous components will be stored in CJPL for preparation work with the decay of cosmogenic nuclides, so-called cooling time, lasting about 1 year.
Importing the measured production rates of the cosmogenic activations, the background contributions from cupreous components were estimated with a Geant4-based Monte Carlo framework called Simulation and Analysis of Germanium Experiment (SAGE) developed for the CDEX-100 experiment \cite{She2021}.
In the simulation, these cosmogenic radionuclides are distributed uniformly across all cupreous components as shown in Figure~\ref{Component}.
The energy spectra of the different cosmogenic nuclides and the total background contributions are simulated as shown in Figure~\ref{SimulationSpec}.
\par
\begin{figure}[htb]
    \centering
    \includegraphics[width=\columnwidth]{./detectorSchema.png}
    \caption{The sketch of the PPCGe detector array (left) and the PPCGe detector unit (right) implemented in SAGE together with its cupreous components shown in orange. The PPCGe detector array is immersed in the liquid nitrogen when simulation.}
    \label{Component}
\end{figure}
Under above assumptions, the simulated background rates in the low energy range (2-4 keV) and around 2039 keV (energy of neutrinoless double beta decay of ${^{76}}$Ge) are ${7.17\times10^{-2}}$ counts per keV per tonne per year (cpkty) and ${1.82\times10^{-3}}$ cpkty, respectively.
The contribution of ${^{57}}$Co dominates the background contributions lower than 122 keV and ${^{54}}$Mn is dominant when energy is lower than 830 keV.
${^{60}}$Co is prevailing in the higher energy region while there only exists the background from ${^{56}}$Co over 2.5 MeV.
Among these cosmogenic nuclides, only ${^{60}}$Co has a half-life over one year and it is hard to be significantly suppressed by underground cooling rather than other cosmogenic nuclides do.
The feasible method to mitigate the contribution from ${^{60}}$Co is to reduce the exposure time above ground or electroform the copper underground.
\begin{figure}
    \centering
    \includegraphics[width=\columnwidth]{./EnergySpectrum.pdf}
    \caption{The simulated cosmogenic background spectra of 0-2.8MeV in copper components of germanium detectors for CDEX-100 experiment.The energy resolution is determined by the germanium detector used in CDEX-1B \cite{Yang2019}.}
    \label{SimulationSpec}
\end{figure}
\par

\section{Summary}
We investigate the cosmogenic activation in OFHC copper bricks at the altitude of 2469.4m.
The specific activities of cosmogenic radionuclides in the exposed copper bricks are measured with a low background germanium gamma-ray spectrometer.
The production rates of several cosmogenic radionuclides are calculated and compared with previous studies.
The production rates at sea level in units of nuclei/kg/day are ${18.6 \pm 2.0}$ for ${^{54}}$Mn, ${9.9 \pm 1.3}$ for ${^{56}}$Co, ${48.3 \pm 5.5}$ for ${^{57}}$Co, ${51.8 \pm 2.5}$ for ${^{58}}$Co and ${39.7 \pm 5.7}$ for ${^{60}}$Co, respectively.
For comparison, we simulate their cosmogenic activation with Geant4, showing that the agreement between measurement and simulation is generally satisfactory.
The discrepancies with other published results are also presented.
\par
Based on the results of the production rates of cosmogenic radionuclides in copper, the cosmogenic background from copper components of detectors is simulated for CDEX-100 experiment.
The total background rates from cosmogenic nuclides are ${7.2\times10^{-2}}$ cpkty and ${1.8\times10^{-3}}$ cpkty for the 2-4 keV and around 2039 keV energy regions, respectively.
The cosmogenic background is dominated by ${^{57}}$Co, ${^{54}}$Mn and ${^{60}}$Co after one-year cooling underground.
To mitigate the cosmogenic activation above ground, it is necessary to control the total exposure time including manufacture and transportation with an appropriate shield.
Moreover, underground electroforming of copper is another option to dramatically reduce the cosmogenic background from copper components of germanium detectors.
\par

\begin{acknowledgements}
This work was supported by the National Key Research and Development Program of China (No. 2017YFA0402201), National Natural Science Foundation of China (No.11675088, 11725522, U1865205), and Tsinghua University Initiative Scientific Research Program. Thanks to colleagues of CJPL for their help in copper sample measurements.
\end{acknowledgements}
\textbf{Conflicts of interest}: The authors have no relevant financial or non-financial interests to disclose.

\bibliographystyle{unsrt}


\section*{Discussions and Conclusions}

In this study, we presented a machine learning method based on textural features of patella to detect PFOA from lateral view knee x-rays.  Texture features were extracted both by handcrafted descriptor (LBP) and also learned directly from texture patches using CNNs.  We compared-texture based models with more conventional models that use clinical assessments and participant characteristics (age, sex, BMI, WOMAC,KL).
The model that uses only texture features obtained from superior ROI from patellar region yielded statistically significant improvement over the clinical features (ROC AUC of 0.884 vs. 0.817). This finding suggests that patellar texture features can provide useful imaging biomarkers for OA diagnostics.

The highest ROC AUC and AP for classifying knees without and with PFOA were obtained when model predictions based on texture features and clinical data were combined into a second level machine learning model.
This can be explained by the fact that the textural features of patellar bone and the clinical features are complementing each other.
Model stacking outperformed early feature fusion because the former method allow
each classifier to include their own benefits.

We tested three different ROIs for texture analysis.
Texture features from the superior ROI had the highest classification performance to distinguish between knees with and without PFOA.
We observed a performance decrease when we used the whole patellar region for predicting PFOA compared to superior and inferior ROI.
These findings suggest that the area closest to the inferior margin of the patella, where osteophytes typically occur, experience the most significant bony changes in PFOA.

This study has also limitations.
The most important one is the lack of external data for validating the performances of the models.
Therefore, we adopted the cross fold validation strategy to evaluate them.
However, the actual generalizability of the models could only be understood on a separate test data that comes from different sources (population, hospital, device, etc.).
However, because the patellofemoral OA is not studied as much as tibiofemoral OA, the number of PFOA-focused dataset is limited.
Second, PFOA is not only altering the bone structure but at the same time changes the morphology and the alignment of patella \cite{macri2017patellofemoral}.
In future studies, models that consider patella morphology and alignment should be explored and the additional value of textural features on the diagnosis of PFOA should be studied.
Third, other views of patella such as skyline view should be analysed to confirm the location of the bone alterations.
Finally, in order to extract texture features, we employed rectangular ROIs. In some cases such ROI could overlap with the background that might slightly affect the results.
In principle, adopting ROIs that adhere to the boundaries would be better, but this requires exact segmentation of the patellar region, and it might also lead to a non-constant ROI size further affecting the standardized comparisons between different subjects.

In conclusion, we present the first study that evaluates patellar bone texture for detecting PFOA. Our results show that texture features of patellar bone are different between knees with and without PFOA. Good classification performance values indicate that analyzed texture features contain useful information of patellar bone structure, which seems to change in PFOA. These texture features may be used in the future as novel imaging biomarkers in OA diagnostics.

\section*{Results}

\begin{table}[]
\centerline{
\begin{tabular}{@{}lllllll@{}}
\toprule
                            & Input                                                                                & Method                   & AUC                                           & AP                                            & \begin{tabular}[c]{@{}l@{}}Brier \\ Score\end{tabular} &                \\ \midrule
\multicolumn{1}{l|}{Model1} & \multicolumn{1}{l|}{\begin{tabular}[c]{@{}l@{}}LBP  Texture Features \\ (extracted from superior ROI)\end{tabular}} & \multicolumn{1}{l|}{GBM} & \multicolumn{1}{l|}{0.884 {[}0.871, 0.895{]}} & \multicolumn{1}{l|}{0.697 {[}0.669, 0.722{]}} & \multicolumn{1}{l|}{0.087}                             & Texture Model  \\ \midrule
\multicolumn{1}{l|}{Model2} & \multicolumn{1}{l|}{Age, Sex, BMI}                                                                                & \multicolumn{1}{l|}{GBM} & \multicolumn{1}{l|}{0.647 {[}0.626, 0.666{]}} & \multicolumn{1}{l|}{0.294 {[}0.273, 0.317{]}} & \multicolumn{1}{l|}{0.136}                             & Clinical Model \\ \midrule
\multicolumn{1}{l|}{Model3} & \multicolumn{1}{l|}{Age, Sex, BMI, WOMAC}                                                                           & \multicolumn{1}{l|}{GBM} & \multicolumn{1}{l|}{0.715 {[}0.696, 0.732{]}} & \multicolumn{1}{l|}{0.351 {[}0.325, 0.378{]}} & \multicolumn{1}{l|}{0.130}                             & Clinical Model \\ \midrule
\multicolumn{1}{l|}{Model4} & \multicolumn{1}{l|}{Age, Sex, BMI, KL}                                                                              & \multicolumn{1}{l|}{GBM} & \multicolumn{1}{l|}{0.812 {[}0.798, 0.826{]}} & \multicolumn{1}{l|}{0.47 {[}0.441, 0.5{]}}    & \multicolumn{1}{l|}{0.114}                             & Clinical Model \\ \midrule
\multicolumn{1}{l|}{Model5} & \multicolumn{1}{l|}{Age, Sex, BMI, WOMAC, KL}                                                                       & \multicolumn{1}{l|}{GBM} & \multicolumn{1}{l|}{0.817 {[}0.802, 0.831{]}} & \multicolumn{1}{l|}{0.487 {[}0.457, 0.517{]}} & \multicolumn{1}{l|}{0.113}                             & Clinical Model \\ \midrule

\multicolumn{1}{l|}{Model6} & \multicolumn{1}{l|}{\begin{tabular}[c]{@{}l@{}}Superior ROI\\ (image patch)\end{tabular}}                           & \multicolumn{1}{l|}{CNN} & \multicolumn{1}{l|}{0.889 {[}0.875, 0.9}      & \multicolumn{1}{l|}{0.714 {[}0.687, 0.739{]}} & \multicolumn{1}{l|}{0.084}                             & CNN model      \\ \midrule

\multicolumn{1}{l|}{Model7}                     & \multicolumn{1}{l|} {\begin{tabular}[c]{@{}l@{}} LBP, Age, Sex, BMI \\WOMAC, KL\end{tabular} }                                      &  \multicolumn{1}{l|} {GBM}                      & \multicolumn{1}{l|}{{0.904 [0.892, 0.913]}   }          & \multicolumn{1}{l|}{{0.719 (0.693, 0.743)} }            & \multicolumn{1}{l|}{0.083}                                         & Fused Model  \\ \midrule

\multicolumn{1}{l|}{Model8}                     & \multicolumn{1}{l|} {\begin{tabular}[c]{@{}l@{}}Predictions from \\ Model5 and Model6\end{tabular} }                                      &  \multicolumn{1}{l|} {GBM}                      & \multicolumn{1}{l|}{\textbf{0.937 {[}0.929, 0.945{]}}   }          & \multicolumn{1}{l|}{\textbf{0.791 {[}0.768, 0.813{]}} }            & \multicolumn{1}{l|}{\textbf{0.069}}                                         & Stacked Model  \\ \bottomrule
\end{tabular}
}
\caption{Comparison of the developed models}
\label{tab:comparison}
\end{table}

\subsection*{Comparison of Region of Interest (ROI)}
Firstly, we compared the performance of the texture descriptor (LBP) on the superior ROI, inferior ROI, and also on the whole patella in subject-wise stratified 5-fold cross validation setting (Figure \ref{fig:roi_comparison}).
Here, we used GBM models to predict PFOA.
From Figure \ref{fig:roi_comparison}, it can be seen that the model employing superior ROI features performs best yielding an AUC of 0.884 (0871-0.895) and AP of 0.697 (0.669-0.722).
We chose the superior ROI in our further comparisons because it performs with higher precision at most recall levels.
We observed a lower performance when we used texture features extracted from the whole patella.

\begin{figure} [!htb]
    \centerline{

    [width=0.6\textwidth]{images/baseline_visit_pfoa_roc_curves_roi_comparison.pdf}
    [width=0.6\textwidth]{images/baseline_vist_pfoa_pr_curves_roi_comparison.pdf}\hfill}
    \caption{ (a) ROC and (b) PR curves demonstrating the performance of the models operating on different region of interests (ROIs) - superior ROI, inferior ROI, and whole patella.
    Area under the curves and 95\% confidence intervals in parentheses were given based on a 5-fold cross validation setting.
    Dashed lines in ROC indicate the performance of a random classifier and in case of PR it indicates the distributions of the labels of the dataset (PFOA vs non-PFOA).}
    \label{fig:roi_comparison}
\end{figure}

\begin{figure}[!htb]
    \centerline{

    [width=0.6\textwidth]{images/baseline_visit_pfoa_roc_curves_feature_comparison.pdf}
    [width=0.6\textwidth]{images/baseline_visit_pfoa_pr_curves_feature_comparison.pdf}
    \hfill}
    \caption{ Comparison of texture features and clinical data. (a) ROC and (b) PR curves demonstrating the performance of the gradient boosting machine (GBM) models.
    Superior region of interest(ROI) with Local Binary Pattern (LBP) features were employed to train the texture model.
    Area under the curves and 95\% confidence intervals in parentheses were given based on a 5-fold cross validation setting.
    Dashed lines in ROC indicate the performance of a random classifier and in case of PR it indicates the distributions of the labels of the dataset (PFOA vs non-PFOA). }
    \label{fig:features_comparison}
\end{figure}

\subsection*{{Comparison of Texture Features and Clinical Variables}}

After testing the ROIs, we developed machine learning models based on clinical assessments and subject characteristics.
In order to understand the value of texture features, we performed a thorough evaluation of age, sex, body-mass index (BMI), WOMAC and TFOA KL scores (Figure \ref{fig:features_comparison}).
Here, we utilised GBM models and trained them to predict the probability of PFOA using different combinations of the risk factors mentioned above.
Figure \ref{fig:features_comparison} demonstrates that the best performed reference model is based on age, sex, body-mass index, WOMAC, and TFOA KL scores (Model5).
It reached the AUC of 0.817 (0.802–0.831) and AP of 0.487 (0.457–0.517).
When we compared our texture model (Model1) to the strongest reference method (Model5),
we obtained a statistically significant difference between the AUC values (DeLong's p-value$<1e-5$).

\subsection*{{Comparison of Texture Features and Deep Convolutional Neural Network Features}}
Subsequently, we developed a CNN model, which directly works on texture patches in an automatic manner.
Compared to the handcrafted texture model (Model1), our CNN model(Model6) yielded slightly higher performance with an AUC of 0.889(0.875-0.9) and AP of 0.714(0.687-0.739), however the  performance difference in AUC was not statistically significant (DeLong's p-value=0.25) (Table \ref{tab:comparison}).
ROC and PR curves are shown in the Supplementary Material (Figure \ref{fig:texture_cnn_comparison}).

\subsection*{{Model Stacking and Early Feature Fusion}}

Finally, in order to evaluate whether a combination of texture features and clinical assessments and participant characteristics would further increase the predictive accuracy, we utilised a second layer GBM model that fuses model predictions (model stacking).
Here, we used predictions of our CNN model (Model6) and predictions of the strongest reference model (Model5)  as input features for the second level GBM model (Figure \ref{fig:model_stacking}). Same 5-fold stratified cross validation setup was used.
This model (Model8) yielded the best AUC of 0.937 (0.929, 0.945) and AP of 0.791 (0.768, 0.813) and the Brier score of 0.069.
This increase in AUC was also statistically significant (DeLong's p-value$<1e-5$) between the stacked model (Model8) and the CNN model (Model6).

\begin{figure} [!htb]
    \centerline{

    [width=0.8\textwidth]{images/stacked.pdf}
    }
    \caption{ Schematic representation of model stacking.}
    \label{fig:model_stacking}
\end{figure}

For comprehensiveness, we also tested fusion of texture features (LBP) and clinical assessments and participant characteristics (age, sex, BMI, WOMAC, KL) in the same GBM model (See Supplementary Figure \ref{fig:fused_model}).
ROC and PR curves of the fused model are shown in the Supplementary Material in Figure \ref{fig:fused_comparison}.
Fused model yielded AUC of 0.904 (0.892, 0.913) and AP of 0.719 (0.693, 0.743).
While fused model (Model7) performed significantly better than the texture model (Model1) in terms of AUC (DeLong's p-value$<1e-5$), it is still not better than the stacked model (Model8).

\section*{Introduction}
\subfile{introduction.tex}
\subfile{proposal.tex}
\subfile{experiments.tex}
\subfile{Conclusion.tex}

\newpage
\pagebreak
\begin{backmatter}

\section*{Acknowledgements

Multicenter Osteoarthritis Study (MOST) Funding Acknowledgment. MOST is comprised of four cooperative grants (Felson – AG18820; Torner – AG18832, Lewis – AG18947, and Nevitt – AG19069) funded by the National Institutes of Health, a branch
of the Department of Health and Human Services, and conducted by MOST study investigators. This manuscript was prepared using MOST data and does not necessarily reflect the opinions or views of MOST investigators.

We would like to acknowledge the strategic funding of the University of Oulu, Infotech Oulu.

We gratefully acknowledge Claudia Lindner for providing the BoneFinder\textsuperscript{\textregistered} tool and lateral knee active shape model, Aleksei Tiulpin for providing an interface to BoneFinder to fully leverage multiple processors, and the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used in this research.

\section*{Funding
Funding sources are not associated with the scientific contents of the study.

\section*{Competing interests}
The authors declare that they have no competing interests.

\section*{Authors' contributions}
 N.B. originated the idea of the study. N.B. performed
the experiments and took major part in writing of the manuscript. M.T.N, and S.S. supervised the project. All  authors participated in producing the final manuscript draft and approved the final submitted version.

\section*{Summary Table}
\begin{itemize}
\item We  present  the  first  study  that  evaluates  patellar  bone  texture for  detecting  patellofemoral osteoarthritis (PFOA).
\item We proposed a framework based on machine learning and performed extensive experiments to demonstrate the influence of texture features on the diagnostic performance.

\item Different region of interests (ROIs) from patellar region from lateral view radiographs (X-rays) were studied whether texture features can be used to distinguish the knees with definite radiographic PFOA from healthy knee radiographs.

\item We compared the performances of hand-crafted features, deep convolutional neural network features,  and
clinical variables including age, sex, body mass index (BMI), the total Western Ontario and McMaster Universities ArthritisIndex (WOMAC) score, and Kellgren and Lawrence (KL) score of the tibiofemoral joint.

\item Finally,  we  propose  a  stacked  model  where  both patellar texture and clinical feature predictions are combined with a second level machine learning model - Gradient Boosting Machine (GBM).

\item Our  results  show  that  texture  features  of  patellar  bone  are different between knees with and without PFOA.

\item Good classification performance values indicate that analyzed texture features contain useful information of patellar bone structure, which seems to change in PFOA.

\item Patellar bone texture features may be used in the future as novel imaging biomarkers in OA diagnostics.

\end{itemize}

\bibliographystyle{bmc-mathphys}

\section*{Materials and Methods}
The overall pipeline of our study is shown in Figure \ref{fig:flowchart}.
In order to pre-process the data, we extracted anatomical landmark points \cite{lindner2013fully} and applied intensity normalization using global contrast normalisation and a histogram truncation between the $5^{th}$ and $99^{th}$ percentiles.
We then resampled the data to have a 0.2mm pixel spacing.
Subsequently, we located ROIs using landmark points.
Right knee images were then horizontally flipped to have a similar view with left knee images.
Finally, we predicted PFOA using both handcrafted texture features using a machine learning-based approach and a deep CNN.
We also trained a machine learning model (GBM \cite{friedman2001greedy}) on clinical features as a reference  method  to  compare  with  the  proposed  approach.

\subsection*{\textbf{Data}}
In the study, we used data from the Multicenter Osteoarthritis Study public use datasets (MOST, \href{http://most.ucsf.edu}{http://most.ucsf.edu}).
The MOST study is a longitudinal observational study of adults who have or are at high risk
for knee OA.
At baseline, there were 3,026 individuals aged 50–79 years who either had radiographic knee OA or were at high risk for developing the disease.
In MOST, semiflexed lateral view radiographs were acquired according to a standardized protocol.
Knee radiographs were read from the baseline to 15, 30, 60 and 84-month follow-up visits.
We employed radiographs taken at the baseline visit that includes  5507 knees after removing data which have missing information. Out of 5507 knees, 953 had PFOA(17.3\%).

In the MOST public use datasets, radiographic PFOA is defined from lateral view radiographs as follows: Osteophyte score $\geq$ 2 or the joint space narrowing (JSN) score is $\geq1$ plus any osteophyte, sclerosis or cysts $\geq1$ in the PF joint (grades 0–3; 0=normal, 1=mild, 2=moderate, 3=severe) (Figure \ref{fig:assessment}).

\begin{figure}[!ht]
\centering
[width = 1\linewidth, ]{images/assessment.pdf}
\caption{Assessment of patellofemoral osteoarthritis in the MOST public use datasets. }
\label{fig:assessment}
\end{figure}

Individual radiographic features in the MOST dataset were  graded by two independent expert readers based on the atlases from the Osteoarthritis Research Society International (OARSI) \cite{altman2007atlas} which refers to the previous OARSI atlas for the patellofemoral joint \cite{altman1995atlas} and Framingham Osteoarthritis Study \cite{chaisson2000detecting}.
When there was a disagreement in film readings, a panel of three adjudicators resolved the discrepancies\cite{roemer2009association}.

\subsection*{\textbf{Patellar Landmark Localization}}
In this study, we utilized BoneFinder\textsuperscript{\textregistered} software \cite{lindner2013fully} in order to locate the landmark points along the contours of the patella (Figure \ref{fig:landmarks}).
BoneFinder\textsuperscript{\textregistered} uses random forest regression voting with constrained local model approach to automatically locate 21 anatomical landmarks in patellar region from lateral knee X-rays.
These points enabled us to locate ROIs within the patella with precision.
Moreover, we used two marginal landmarks in order to align patella to obtain the same anatomical region among the knees in the ROI localization step (Figure \ref{fig:landmarks}).

\begin{figure}[!tb]

\centerline{
\begin{adjustbox}{color = bmcblue, varwidth=1\textwidth,margin=3pt 1pt 3pt 5,frame=0.5pt }
\centering
    [height=0.6\textwidth]{images/lateral_with_landmarks.png}
    [height=0.6\textwidth]{images/alignment.png}

    \centering
    \caption{Anatomical patellar landmarks were detected from lateral view knee radiographs. BoneFinder\textsuperscript{\textregistered} software \cite{lindner2013fully} were used to locate 21 landmark points along the contours of the patella.
    Marginal landmarks were employed in order to align the patellar region. The line on the right between the marginal points were draw in order to illustrate the final alignment of patella. }
    \label{fig:landmarks}

\end{adjustbox}
}
\end{figure}

\subsection*{\textbf{Texture Descriptor - Local Binary Patterns (LBP) }}
Previously in OA research, multiple texture descriptors have been applied to plain knee radiographs for several years,
Fractal Signature Analysis (FSA) or fractal dimension (FD) being the most widely used ones \cite{lynch1991analysis, janvier2017subchondral,hirvasniemi2017differences,kraus2018predictive, bayramoglu2019adaptive}.
However, FD method is susceptible to limited discrimination power due to its sensitivity to image artifacts and noise \cite{veenland1996estimation}.
It was shown in \cite{bayramoglu2019adaptive} that Local Binary Patterns (LBP)  \cite{ojala2002multiresolution} yields  better  performance in describing bone texture to detect OA for tibiofemoral joint.
Therefore, in our study, we selected LBP as a handcrafted texture descriptor as well.

LBPs captures a local representation of texture.
In order to compute LBP value of a pixel \textit{(c)}, \textit{p} neighboring pixels  that are evenly distributed in angle on a circle of radius \textit{r} are sampled first.
Then the LBP pattern is constructed by comparing the gray value of the center pixel (c) with its surrounding neighborhood of \textit{p} pixels and a binary vector of \textit{p} bits is extracted from this comparison.
The decimal value of LBP pattern is then obtained from the binary sequence.
For an $N\times M$ texture image, a LBP pattern can be the computed at each pixel -($N\times M$) LBP patterns-, then the image texture can be represented by the distribution of LBP values, by a LBP histogram vector.
In this study, we used $r=2$, $p=8\times r$ and a 256 bins histogram to obtain LBP descriptions of our texture patches.

\subsection*{\textbf{Selection of Region of Interest (ROI)}}
We explored three different ROIs within patellar region for texture analysis (Figure \ref{fig:patellar_roi}).
It is  known that along with the progression of OA, bone is subject to changes in its structure and composition \cite{buckland2004subchondral,kamibayashi1995trabecular}.
The radiographically most distinctive bony changes occur at the margins where osteophytes typically occur\cite{bayramoglu2019adaptive}.
Therefore, we selected two ROIs from the inferior and superior region of patella .
Their sizes are proportional to the height of the patella measured from the outermost points ($20\%$ of the patella height).
To locate those ROIs, we used the marginal landmarks which were also utilized previously for alignment (Figure \ref{fig:landmarks}).
As the third ROI, we utilized the whole patellar region.
For segmenting patella from the background, we employed landmark points and obtained a smoothed spline that roughly approximates the contour  (Figure \ref{fig:patellar_roi}).

\begin{figure}[!h]
    \centerline{

\begin{adjustbox}{color = bmcblue, varwidth=1\textwidth,margin=3pt 1pt 3pt 5,frame=0.5pt }
\centering

    \subfloat[Superior ROI]{[height=0.4\textwidth]{images/roi1.png}}
    \subfloat[Inferior ROI]{[height=0.4\textwidth]{images/roi2.png}}
    \subfloat[Whole Patella]{[height=0.4\textwidth]{images/whole_roi.png}}

    \caption{ Examples  of  region-of-interest  (ROI)  placement  for  texture  analysis. Here,  a) shows an example of superior ROI, b) shows inferior ROI,  and  c)  whole patellar region. }

    \label{fig:patellar_roi}
    \end{adjustbox}
    }
\end{figure}

In order to detect the most informative (optimal) ROI, we employed LBP descriptor.
Here, we defined the most informative region as the subregion where the texture classifier (based on LBP features) performs best to distinguish OA samples from non-OA.

We used Gradient Boosting Machine (GBM) classifier based on decision tree algorithms \cite{friedman2001greedy} to predict PFOA using LBP features that were extracted from ROIs defined previously (Figure \ref{fig:patellar_roi}).
We observed that superior ROI features yields better classification performance (See Results section).
Therefore, we utilized superior ROI ( Figure \ref{fig:patellar_roi}a) in the subsequent experiments.

\subsection*{\textbf{Machine Learning Models}}
We employed both GBM and deep CNN methods to predict PFOA from the texture patches and clinical assessments and participant characteristics.
GBM is a powerful decision trees based machine learning algorithm that rely on the concept of boosting ``weak learners”\cite{friedman2001greedy}.
GBM is an iterative process - in each iteration training set is re-weighted such that it compensates for the weaknesses of the previous model.
In this study, we used an efficient implementation of GBM called LightGBM \cite{ke2017lightgbm}.
Table \ref{tab:comparison} summarises all the models developed in this study.

First, we extracted LBP texture features from superior ROI texture patches and trained a GBM model based on these features (Model1).
Then, we used a deep CNN architecture that was trained from end-to-end to classify superior ROI texture  (Model6).
In this way, texture features were learned from the imaging data itself.
CNNs captures both fine-level high spatial-frequency details such as edges, lines, texture, corners, etc and recognize more complex features as it gets deeper (as the number of convolutional layer increases).
We refer reader to \cite{Goodfellow-et-al-2016} and \cite{nielsen2015neural} for more information on the deep neural networks.
We used a three layers' CNN model where the input patches are scaled to $64\times64$ images.
Details of the CNN architecture and our training strategy can be found in the Supplementary Material.
In this study, we also explored more conventional machine learning based prediction models using the clinical data and risk factors.
These include age, sex, body-mass index (BMI), the total Western Ontario and McMaster Universities Arthritis Index (WOMAC) score, and the KL grade of the tibiofemoral joint (Model2,3,4,5).
These reference methods were also built using the GBM classifier.

In all models, we employed subject-wise stratified 5-fold cross validation.
Subject-wise splitting is used to eliminate the subject-dependent bias between training and validation.
That is, all the data (imaging and/or clinical) coming from a particular subject is either put in the training or the test set.
Moreover, we used stratified folds where each fold represents the actual class distribution of the data.
The class distribution is the ratio between the positive and negative samples.
The same folds were used for all the models to have fair comparisons.
All the models were trained separately, thus reported performances were derived from the separate models.

\subsection*{\textbf{Statistical Analyses}}

The models were assessed using Receiver Operating Characteristics (ROC) curves, Precision-Recall (PR) curves, and Brier score \cite{brier1950verification}.
In ROC curves, the true positive rate (TPR) against the false positive rate (FPR) are plotted, whereas Precision-Recall (PR) curves are composed of the precision and the true positive rate.
Therefore, PR analysis is more focused on the positive class.
When data is imbalanced (i.e. more negative samples than positive samples), the ROC curve might not reflect the true performance of the classifier as false positive rate increases more slowly because of the large numbers of negatives.
We used the area under the  ROC  curves  (ROC  AUC) and similarly area under the PR curves (Average Precision; AP) to summarize model performances.
Brier score equals to the mean squared error of the prediction.
In order to compare the differences between model AUCs, we applied DeLong's test \cite{delong1988comparing}.
\subsection*{Deep Convolutional Neural Network Architecture and Training Strategy}

\begin{figure}[!ht]
\centering
[width = 0.5\linewidth, ]{images/network.pdf}
\caption{Network Architecture }
\label{fig:network}
\end{figure}

The CNN model consists of 3 convolutional layers dedicated to texture feature extraction.
Each  convolution layer (stride=1, padding =1) is followed by Batch normalization (BN), max pooling ($2\times2$) and ReLU.
After feature extraction, we used two fully connected layers to make the prediction.
A dropout of 0.5 is inserted after the first fully connected layer.

In all the CNN based experiments, we used the same training strategy.
We trained the models from scratch (end-to-end) using the random weight initialization.
We adopted stochastic gradient descent training on a GPU.
A mini-batch of 64 images were employed, and a momentum of 0.9 was used and trained without weight decay.
We used a starting learning rate of 0.01 and decreased it by 10 every 8 epochs.
The models were trained for 40 epochs and we selected the best performed model.

\begin{figure} [!h]
    \centerline{

    [width=0.6\textwidth]{images/baseline_visit_pfoa_roc_curves_texture_cnn_comparison.pdf}
    [width=0.6\textwidth]{images/baseline_vist_pfoa_pr_curves_texture_cnn_comparison.pdf}\hfill}
    \caption{ (a) ROC and (b) PR curves demonstrating the performance of the texture models- \textbf{CNN vs LBP}.
    Area under the curves and 95\% confidence intervals in parentheses were given based on a 5-fold cross validation setting.
    Dashed lines in ROC indicate the performance of a random classifier and in case of PR it indicates the distributions of the labels of the dataset (PFOA vs non-PFOA).}
    \label{fig:texture_cnn_comparison}
\end{figure}

\begin{figure} [!htb]
    \centerline{

    [width=0.8\textwidth]{images/fused_model.pdf}
    }
    \caption{ Schematic representation of feature fusion where texture features (LBP) and clinical data were combined in the same model. Performance of the fused model is shown in Figure \ref{fig:fused_comparison}. }
    \label{fig:fused_model}
\end{figure}

\begin{figure} [!htb]
    \centerline{

    [width=0.6\textwidth]{images/baseline_visit_pfoa_roc_curves_texture_contrib_comparison.pdf}
    [width=0.6\textwidth]{images/baseline_visit_pfoa_pr_curves_texture_contrib_comparison.pdf}\hfill}
    \caption{ (a) ROC and (b) PR curves demonstrating the performance of the texture (LBP) model, fused model (that combines texture features and clinical data in the same model as shown in Figure \ref{fig:fused_model}) and stacked model (Figure \ref{fig:model_stacking}).
    Area under the curves and 95\% confidence intervals in parentheses were given based on a 5-fold cross validation setting.
    Dashed lines in ROC indicate the performance of a random classifier and in case of PR it indicates the distributions of the labels of the dataset (PFOA vs non-PFOA).}
    \label{fig:fused_comparison}
\end{figure}