Machine Learning Techniques with Fairness for Prediction of Completion of Drug and Alcohol Rehabilitation
Karen Roberts-Licklider Theodore Trafalis











K. Roberts-Licklider University of Oklahoma
              School of Industrial and Systems Engineering

              
              
              Karen.R.RobertsLicklider-1@ou.edu  
              

           T. Trafalis University of Oklahoma
              School of Industrial and Systems Engineering
              ttrafalis@ou.edu
              






Machine Learning Techniques with Fairness for Prediction of Completion of Drug and Alcohol Rehabilitation
    Karen Roberts-Licklider         Theodore Trafalis

    Received: date / Accepted: date
=========================================================================================================




The aim of this study is to look at predicting whether a person will complete a drug and alcohol rehabilitation program and the number of times a person attends.  The study is based on demographic data obtained from Substance Abuse and Mental Health Services Administration (SAMHSA) from both admissions and discharge data from drug and alcohol rehabilitation centers in Oklahoma.  Demographic data is highly categorical which led to binary encoding being used and various fairness measures being utilized to mitigate bias of nine demographic variables.  Kernel methods such as linear, polynomial, sigmoid, and radial basis functions were compared using support vector machines at various parameter ranges to find the optimal values.  These were then compared to methods such as decision trees, random forests, and neural networks. Synthetic Minority Oversampling Technique Nominal (SMOTEN) for categorical data was used to balance the data with imputation for missing data. The nine bias variables were then intersectionalized to mitigate bias and the dual and triple interactions were integrated to use the probabilities to look at worst case ratio fairness mitigation.  Disparate Impact, Statistical Parity difference, Conditional Statistical Parity Ratio, Demographic Parity, Demographic Parity Ratio, Equalized Odds, Equalized Odds Ratio, Equal Opportunity, and Equalized Opportunity Ratio were all explored at both the binary and multiclass scenarios.  







§ INTRODUCTION


Substance abuse is one of the leading causes for mental illness and these issues are dealt with in the Diagnostic and Statistical Manual of Mental Disorders (DSM) and the International Classification of Diseases (ICD) <cit.>. 36.2% of adults age 18-25 had a mental illness, 29.4% for age 26-49 and 13.9% 50 or older with 11.6% being serious mental illness for age 18-25, 7.6% for 26-49 years old and 3.0% for 50 years or older <cit.>. The COMPAS assessment has been used in criminal sentencing to predict whether an offender will reoffend, however it has proven to be racially biased. COMPAS is the Correctional Offender Management Profiling for Alternative Sanctions. It was developed in 1998 and uses a recidivism risk scale and has been used since 2000. It predicts a defendant’s risk of committing a misdemeanor or felony within 2 years of assessment for 137 features about an individual and the individual’s past criminal record <cit.>. The COMPAS assessment incorrectly predicted that whites would reoffend at a rate of 47.7% which was twice the rate of blacks at 28.0%. It favored white defendants over blacks. Its accuracy for white defendants was 67% and 63.8% for black defendants <cit.>. For this reason, considering fairness in these types of assessments is important. Also looking at whether an offender would complete rehab instead of going to prison is important as an alternative to sending them to prison due to overcrowding in prisons especially in Oklahoma. Oklahoma itself ranks 3rd if looking at the world’s incarceration rate considering every state as a country <cit.>. For the scope of this paper, treatment episode data from the Substance Abuse and Mental Health Services website <cit.> is used to look at predicting whether a person completed a drug and alcohol rehabilitation program, the number of prior treatments a person has been to when entering treatment, and then the concatenation of both variables.<cit.> thoroughly reviewed 
 SMOTE: Synthetic  Minority Over-Sampling Technique for both the continuous and Nominal case. The nominal case is what we will use from this work. The algorithms and use of the value distance metric were described for SMOTE-N and SMOTE-NC and how these relate to the ROC curve. Dual and three-way interactions of both the additive and multiplicative type are discussed in <cit.> and the intersecting of bias terms in shown in this work. Demographic Parity, Disparate Impact, Statistical Parity Difference, Equal Opportunity, Equalized Odds, and Min-Max Fairness were all discussed in length in <cit.>. They all compared models such as Support Vector Classifiers, Gaussian Process Classifiers, Gaussian Naïve Bayesian, and Linear Discriminant Analysis. Worst-case fairness metrics such as Demographic Parity Ratio, Disparate Impact Ratio, Conditional Statistical Parity Ratio, Equal Opportunity Ratio, and Equal Odds Ratio as well as a multiclass example was shown in <cit.>. These fairness measures can be used when there are multiple classes or when the reweighting or intersectionality has been done and there are more than two ratios to evaluate between.
This paper is organized as follows: in section 2 we discuss the methodology. In sections 3, 4 and 5 we discuss issues of data cleaning, encoding, balancing and data sensitivities and distributions. Section 6 discusses fairness measures applied to our data. In sections 7 and 8 we explore several machine learning models and interpretation of the results. Sections 9 and 10 discusses reweighting and new fairness calculations. Finally sections 11 and 12 discuss the conclusions and future work.



§ METHODOLOGY

Various machine learning algorithms were used on this data with a focus on kernel methods for support vector machines. With this data being highly categorical in nature, encoding techniques were used to transform the data to make it more manageable.  The data was balanced to make predictions more accurate, and the missing data was imputed.  Fairness measures were compared before and after weighing the variables using two- and three-way interactions with chi squared significance and intersectionalizing the nine bias variables.  The nine bias variables explored were gender, veteran status, marital status, education, age, employment, pregnancy status, race, and ethnicity.  Four kernels were compared; linear, polynomial, radial basis function, and sigmoid.  These were tried at a range of degrees, c values, gammas, and r values. Decision trees, random forests, and neural networks were compared. The models were then compared to each other to see which models outperformed each other.  



§ DATA CLEAN UP PROCESS

The data set was filtered down to Oklahoma and down to 30-day rehabilitation centers only.   A calculated column was created called COMPLETED in which a value of ‘COMPLETE’ was taken if the REASON variable took on a value of 1 and ‘INCOMPLETE’ if REASON took on a value of anything else.  The reason code translations can be seen in figure 1 below.  





Some user defined functions were created to clean up the data. One removes unneeded columns that just have a constant value in them or had an identification number for the CASEID, admit year (ADMYR), REGION and STFIPS could be removed since the data was filtered for Oklahoma.  STFIPS was the FIPS code for each state.  This was filtered for 40 for Oklahoma as mentioned above. 




§ ENCODING AND BALANCING DATA

Next, three data sets were created.  One was for the completed predictions, one was for the reasons predictions, and one was for the no-priors (number of times in treatment) predictions.  Each data set went through an encoding and data balancing processes.  One hot encoding was used to encode the categorical variables into new binary variables.  Each class within that variable is created into a new column which takes on the value of 1 if it appears for that record and 0 otherwise. An Example of how the services classes in figure 2 are translated to binary variables in figure 3 can be seen below. A description of what each service class is, is included.


Next, we look at how the data is imbalanced in all three data sets.  We use the SMOTEN (Synthetic Minority Over-Sampling Technique Nominal) package in Python which uses K-Nearest neighbors <cit.> algorithm to balance the data.  See the figures below. This is done for the three data frames used in prediction.






Typically SMOTE is used which helps to improve the prediction accuracy. It distributes the instances of the majority class and the minority class equally. SMOTE technique increases the predictive accuracy over the minority class by creating synthetic instances of that minority class, <cit.>. SMOTEN extends SMOTE for nominal features by getting the nearest neighbors by using the modified version of the Value Difference Metric which looks at the overlap of feature values over all feature vectors.  A matrix of features for all feature vectors is created and the distance between those features is defined in the following equation:


    δ(V_1,V_2) = ∑_i=1^n| C_1i/C_1 - C_2i/C_2|^k


Where V_1 and V_2 are the corresponding feature values and C_1 is the total number of times V_1 occurs and C_1i is the total occurrences of feature V_1 for class i. The same holds for V_2, C_2, and C_2i. k is some constant that is typically set to 1. This gives the matrix of value differences for each nominal value in the set of feature vectors<cit.>. An example given can be seen on the next page.






§ DATA SENSITIVITIES/DISTRIBUTIONS



 §.§ Pareto Charts

Nine variables were chosen and distributed by Pareto charts to see where bias occurred. These variables were gender, race, age, ethnic, veteran status, education status, marital status, employment status, and pregnancy status. These are shown for completion rehab outcomes in the figures below.









 §.§ Bucketized Categories

Based on the Pareto charts above, each variable was bucketized into dichotomous variables.  Variables such as gender, pregnancy, ethnicity, and veteran status were already dichotomous so they did not change.  Race became either white or non-white, employment status became employed or not employed, age became under 40 or 40 plus, marital status became never married or married/previously married, and education status became college or no college.


§ FAIRNESS MEASURES



 §.§ Disparate Impact

To find where the discrimination occurs, we first look at disparate impact in the dichotomous variables.  

    DI = P(Ŷ=1 | A=0)/P(Ŷ=1 | A=1)


This compares the proportion of individuals receiving a favorable outcome for a privileged and underprivileged group. The closer to 1 it is, the more fair it is. Ŷ is the model predictions and A is the protected attribute, with 0 being the underprivileged class and 1 being the privileged class <cit.>. The resulting disparate impact charts for the completed model can be seen in the figures below. Typically the 80% rule is followed for the threshold to be considered discriminatory <cit.>.












 §.§ Disparate Impact – Multiclass

Next we look at the multiclass case in which we find the disparate impact for each class and divide the minimum conditional probability when the target matches the outcome by the maximum conditional probability when the target matches the outcome.    


    DI_Multiclass = min( P(Ŷ = 1 | A = 1) )/max( P(Ŷ = 1 | A = 1) )


The outputs for noprior model for each of the variables can be seen in the table below.  






 §.§ Statistical Parity Difference

For the completed model we next look at Statistical Parity Difference (SPD).  The closer the result is to 0, the more fair it is.


    SPD = P(Ŷ = 1 | A = 0) - P(Ŷ = 1 | A = 1),


where Ŷ is the models predictions, A=0 is the protected attribute for the unprivileged class and A=1 is the protected attribute for the privileged class. <cit.>.  The results can be seen for each variable in the table below.





 §.§ Statistical Parity Difference – Multiclass

This was done again for each class of noprior and the results can be seen in the table below.  The max threshold is the maximum threshold for which a fairness is achieved for the spd value for each class.  If at least one of the classes has an SPD of zero, then SPD is satisfied for that variable, otherwise it is not.




 §.§ Equal Opportunity

When the true positive rate (TPR) is the same for both the privileged and unprivileged groups, it is considered fair <cit.>.


    P(Ŷ=1 | Y=1, A=0) = P(Ŷ=1 | Y=1, A=1)



    TPR_i = TP_i/TP_i + FN_i



    EqOpp_diff = max_i(TPR_i) - min_i(TPR_i)


This fairness measure was ran at four different C values with the default values for gamma and r, where gamma = scale = 1/n and r =0.  C was tried at .1, 1, 10, and 100.  The optimal results are shown in the table below. Notice again that this is tried at different thresholds.  The difference is found and then the maximum threshold value for which the EqOpp_diff<threshhold is recorded in the table for each model.








Next we do the same thing but for each class and take the max difference and the minimum difference to calculate the fairness in each class where c is in each class.  The results can be seen in the table below.  The  TPR difference is shown in the table and this is the maximum value that the threshold can be taken at.  Again this is shown for the optimal C value with the default values of gamma, r, and degree.


    TPR_sc = TP_sc/TP_sc + FN_sc



    TPR_diff = max(TPR_sc) - min(TPR_sc)



    TPR_sc = TP_sc/TP_sc + FN_sc



    FPR_sc = FP_sc/FP_sc + TN_sc



    Max_TPR_diff = max_c ( max_s (TPR_sc) )



    Min_TPR_diff = min_c ( min_s (TPR_sc) )



    EqOpp_diff = | Max_TPR_diff - Min_TPR_diff|







 §.§ Equalized Odds

Let A=1 and A= 0 be the privileged and unprivileged groups respectively.  When Y=1, the equations shows the TPR and when Y=0 it shows the FPR.  This shows the ROC where TPR and FPR are equal for the privileged and unprivileged demographics.
<cit.>.


    P[Ŷ=1 | Y=1, A=0] = P[Ŷ=1|Y=1, A=1]


The table below shows the TPR and FPR difference as well as the Equalized odds difference and whether it is fair or unfair.  The differences are calculated by subtracting the two values from each class from each other for the TPF and FPR respectively for each variable.  Fairness is determined based on a .2 threshold.  Any threshold could be chosen.  Since equality determines fairness, a threshold closer to zero is wanted.







 §.§ Equalized Odds - Multiclass

We do the same thing for the multiclass except now we take the maximum and minumum of each class to get the TPR difference and FPR difference for each class where s∈ sense variable, c∈ class label.


    TPR_sc = TP_sc/TP_sc + FN_sc



    FPR_sc = FP_sc/FP_sc + TN_sc

    

    TPR_diff = max(TPR_sc) - min(TPR_sc)

    

    FPR_diff = max(FPR_sc) - min(FPR_sc)



    EqOdds_diff =    | max(TPR_sc) - min(TPR_sc) | 
        - | max(FPR_sc) - min(FPR_sc) |


The table with the TPR difference, FPR difference, Equalized Odds difference and whether it is fair or unfair for each variable can be seen below.  Again this was done at a .2 threshold and the default values for gamma, degree, and r with optimal C values for the multiclass noprior model.  Note that closer to zero for an EqOdds_diff is more fair.





§ THE MODEL



 §.§ Support Vector Machines

The data sets are split 70/30 training and test sets with shuffle set at true which splits the data randomly.  Support Vector Machines were run with four kernels.  Linear, Polynomial, Radial Basis Function, and Sigmoid.  The formulas for these kernels can be seen below.


    Linear Kernel:    
        K(X,Y) = X^T Y 
    Polynomial Kernel:    
        K(X,Y) = (X^T Y + C)^d 
    Radial Basis Function Kernel:    
        K(X,Y) = e^-γX - Y^2, 
       γ > 0 
    Sigmoid Kernel:    
        K(X,Y) = tanh(γ X^T Y + C)

    

    where γ = auto      = 1/n
    where γ = scale      = 1/n ×Var(X)
    where Var(x)       = 1/n∑_i=1^n (x_i - μ)^2


After further research, it was determined that with algorithms such as SVM, feature importance is not that useful.  It is more useful to select an appropriate C value instead.  The C value is a parameter that sets how much you want to penalize your model for each misclassified point.  This means that the larger the C value, the smaller the margin, the smaller the C value, the larger the margin.  The goal is to maximize the margin while minimizing the classification error.  Next, we looked at different C values.  In the models above the C value was set to four different C-values, .1, 1, 10, and 100 and the optimal values were chosen for each model. Note in the figure below <cit.> the difference between large and small C values.
    


The results of using the different C-Values with the default values of gamma, r, and degree can be seen on the COMPLETED and NOPRIOR datasets for all four kernels below. The best performing model was the polynomial model with a C value of 10 having a prediction accuracy of 63.70% for the COMPLETED data set. The confusion matrix can be seen in the figure below. The radial basis function kernel model was a close second with an accuracy of 61.73% with a C=10. C=10 performed better in all models except in the sigmoid and linear kernel models.





For the multiclass model, the accuracy was much higher with RBF coming in first with an accuracy of 91.83% with a C-value of 10 and polynomial coming in second with an accuracy of 90.89% and a C-value of 1.  The linear model was also high at 88.09% with a C-value of 10.  The confusion matrix can be seen below.










 §.§ Decision Trees

A grid search approach was used for decision trees with the following parameters:


    
  * Max Depth: [None, 2, 4, 6, 8, 10]
    
  * Min Samples Split: [2, 5, 10]
    
  * Min Samples Leaf: [1, 2, 4]


The optimal results for each data set can be seen below:









 §.§ Random Forests

A parameter grid approach was used for random forests with the following parameters:


    
  * Number of Estimators: [10, 50, 100, 200]
    
  * Max Features: ['auto', 'sqrt', 'log2']
    
  * Max Depth: [None, 5, 10, 20]
    
  * Min Samples Split: [2, 5, 10]
    
  * Min Samples Leaf: [1, 2, 4]













 §.§ Neural Networks

The parameter grid for the neural network model is as follows:


    
  * Units in Layer 1: [8, 10, 20, 30]
    
  * Units in Layer 2: [8, 10, 20, 30]
    
  * Activation Function in Input Layers: ['relu', 'tanh', 'sigmoid']
    
  * Optimizer: ['adam', 'sgd']
    
  * Loss Function: ['categorical_crossentropy', 'mean_squared_error']


The resulting optimal parameters along with their confusion matrices for each data set can be seen below:












§ INTERPRETATION OF RESULTS


The NOPRIOR variable translation can be seen below.  This is the translation to what is shown in the x-axis of the confusion matrix above. 





The encoded variables used in the data sets and their translated meanings can be seen in the following tables.  




 













§ REWEIGHTING



 §.§ Interactions


The Chi Squared test was used to test significant interactions between bias variables.


    χ^2 = (E - O)^2/E,


where E is the expected value and O is referring to the observed frequencies in the contingency table respectively.   We looked at both dual and three-way interactions and then combined the datasets and calculated the probability that at least one of these interactions occurred as our new reweighted weight.  If there is a significant interaction, then the interaction counts for each x1, x2 for the dual interaction or x1, x2, and x3 for the three-way interaction are then divided by the total value counts for x1, x2, with the target or x1, x2, and x3 with the target. This gives the probability.  These would be multiplicative models <cit.>.



  §.§.§ Dual Interaction


    S_1    = {x_1, x_2, target}
    
        S_2    = {x_1, x_2}
    DualInteraction_Probability   = |S_1|/|S_2|
    Example:
       P(Complete∩White∩Male)/P(White∩Male)





  §.§.§ Three Way Interaction


    S_3    = {x_1, x_2, x_3, target}
    
        S_4    = {x_1, x_2, x_3}
    3wayInteraction_Probability   = |S_3|/|S_4|
    Example:   
       P(Complete∩White∩Female∩Pregnant)/P(White∩Female∩Pregnant)




 §.§ Intersectionalization

By taking the case for which at least one interaction occurs, we are reweighting the fairness.  Then we combined the two-way and three-way interaction datasets together to get our new df dataset.  This allows for the case when a person can be in two groups at once.  An example would be veteran and male or veteran and male under 40.


    FinalProbability   = 1 - (1 - ∏probability_interactions) 
    probability_interactions   = (DualInteraction_Probability) 
         ∪(3wayInteraction_Probability)




§ NEW FAIRNESS CALCULATIONS




 §.§ Worst Case Demographic Parity

Demographic parity compares the pass rate of (positive outcome) of two groups. It is satisfied if:


    P(Ŷ| A ∈sg_i) = P(Ŷ| A ∈sg_i) ∀ i, j ∈ℕ, i ≠ j


Where N is the total number of subgroups.  When using the worst-case ratio formula, it is also known as demographic parity ratio, <cit.>:


    DPR = min{ P(Ŷ| A ∈sg_i) ∀ i ∈ℕ}/max{ P(Ŷ| A ∈sg_i) ∀ i ∈ℕ}











 §.§ Worst Case Demographic Parity – Multiclass


    DPR_multiclass = min{min{ P(Ŷ| A ∈sg_i) ∀ i ∈ N }/max{ P(Ŷ| A ∈sg_i) ∀ i ∈ N }}


The same is done for the multiclass DPR except the overall minimum is taken to get the minimum over all classes. [Gosh et al., 2022]<cit.>.  The results can be seen in the figures below for COMPLETED, NORPIOR, and COMPLETED_NORPIOR merged on two-way and three-way interactions.







 §.§ Worst Case Disparate Impact 


    DI = min{P(Ŷ | A ∈sg_i)/P(Ŷ | A ∈sg_i); ∀ i, j ∈ N, i ≠ j },


where in this case i and j are unique pairs in the series.  The data is grouped by number of terms (2- or 3-way interactions), The probabilities are calculated between the pairs of ratios for each group of target class and pair of probabilities. The minimum ratios are all aggregated across all pairs and then minimum amongst these aggregated ratios is the worst-case scenario for disparate impact. The 80% rule has to be passed for fairness to be achieved.  This is also known as the four fifths rule.  The pass rate of group 1 to group 2 has to be greater than 80% with group 1 and 2 being interchangeable <cit.>.  Again, the results are shown below.







Note that, COMPLETED is the fairest, NOPRIOR is second most fair, and COMPLETED_NOPRIOR is the least fair.  It is fair individually, however in a worst-case scenario it has the lowest score.



 §.§ Conditional Statistical Parity Ratio

This uses the worst-case min-max ratio, with predictor Ŷ, member A, and legitimate attribute L <cit.>.


    P(Ŷ| L=1, A ∈sg_i) = 
    P(Ŷ| L=1, A ∈sg_j)   ∀ i, j ∈ N, i ≠ j




    CSPR    = 
    min{P(Ŷ| L=1, A ∈sg_i)  ∀ i ∈ N}/max{P(Ŷ| L=1, A ∈sg_i)  ∀ i ∈ N}




    CSPR_Multiclass   = 
    min{min{P(Ŷ| L=1, A ∈sg_i)  ∀ i ∈ N}/max{P(Ŷ| L=1, A ∈sg_i)  ∀ i ∈ N}}



This uses the worst-case min-max ratio, with predictor Ŷ, member A, and legitimate attribute L <cit.>.










 §.§ Equal Opportunity Ratio - Multiclass

The TPR is compared for the protected and unprotected group.  The minimum of these overall is taken for the multiclass case <cit.>.

    P(   Ŷ=1 | A ∈sg_i, Y=1) = 
       P(Ŷ=1 | A ∈sg_j, Y=1) 
       ∀ i, j ∈ N, i ≠ j 
    EOppR   = min{ P(Ŷ=1 | A ∈sg_i, Y=1) ∀ i ∈ N }/max{ P(Ŷ=1 | A ∈sg_i, Y=1) ∀ i ∈ N }


    EOppR_Multi   = 
    min{min{ P(Ŷ=y_k | A ∈sg_i, Y=y_k), ∀ i ∈ N, ∀ k ∈ K }/max{ P(Ŷ=y_k | A ∈sg_i, Y=y_k), ∀ i ∈ N, ∀ k ∈ K }}

















 §.§ Equal Odds Ratio

K is the set of all possible output classes.  The closer the value is to 1, the lower the disparity is.   For this case we calculate the worst-case odds for each output class y and then take the minimum of all those values <cit.>. This is the multiclass case for equalized odds ratio.


    P(Ŷ=y_k | A ∈sg_i) = 
    P(Ŷ=y_k | A ∈sg_j)   ∀ i, j ∈ N, i ≠ j, ∀ k ∈ K



    EOddR_Multiclass = 
       min{min{P(Ŷ=y_k | A ∈sg_i)}/max{P(Ŷ=y_k | A ∈sg_i)}}∀ i ∈ N, ∀ k ∈ K






































The Rectified Linear Unit activation function is 0 when x is negative and x when x is positive f(x)=max(x,0), <cit.>.  This was the optimal activation function in all three models.






§ CONCLUSIONS

Initially this research set out to predict whether a person completed treatment or not and it proved difficult due to the imbalance of the data with roughly 32% of people completing treatment. When this proved to have poor accuracy with SVMs, other algorithms such as decision trees and random forest approaches were utilized, A different approach was taken to look at predicting how many prior times a person had been in treatment as well as how the combination of completed and NOPRIOR.  Fairness was explored with 9 variables and after reweighting the variables, a higher fairness was achieved in all the models with only a small decrease in accuracy.  The highest model for the completed data set was the poly-wgt at 86.14%. Note that the reweighted COMPLETED model is fair when it comes to all fairness measures with a fairness of 95% for demographic parity, 98% for CSPR, equal opportunity of 95% and equal odds of 98%.  For the NOPRIOR data set, RBF-wgt had the best accuracy with the best fairness.  It had an accuracy of 91.83% and a fairness of 72% for all fairness except equal odds which was 80%.   COMPLETED_NOPRIOR was fair looking at individual fairness but not at worse case fairness, due to some classes having poor fairness. 

The accuracy of this model was also 91.83% for RBF-wgt.  Based on fairness and accuracy, the NOPRIOR model predicts the best and COMPLETED is the next best. An RBF or poly kernel would be the best to use when it comes to SVM, c value of 10, r=-0.1, gamma=scale.  This provides the best results.  In comparison, decision trees and random forests performed much better overall with decision trees having an accuracy of 97.98% for COMPLETED, 96.07% for NOPRIOR and 87.97% for COMPLETED_NOPRIOR.  Random forests had an accuracy of 88.56% for COMPLETED, 90.24% for NOPRIOR, and 80% for COMPLETED_NOPRIOR.  Neural Networks performed the worst with an accuracy of 77.52% for COMPLETED, 85.09% for NORPIOR, and 73.65% for COMPLETED_NOPRIOR.  Overall decision trees performed the best with the COMPLETED data set being the best predicting data set, NOPRIOR being next and COMPLETED_NOPRIOR coming in last in accuracy.  For this reason, we would choose to use the decision trees model to predict all three data sets.



§ FUTURE WORK

For future work, it would be beneficial to look at linear discriminant analysis for dimensionality reduction.  Also, MCA or Multiple Correspondence Analysis would be good to look at for reducing dimensionality of categorical data.  Researching kernelized MCA would be interesting to see how that affects the accuracy of the model.  Other methods of balancing the data that are more sophisticated than SMOTE using K-nearest neighbors or deep learning methods would be interesting to investigate.  Looking deeper into the data to see what other information could be extracted.  Perhaps a clustering analysis of what drugs/substances occur in treatment together in a patient and how those ties to age and demographics.






apalike







































































