# Tutorial: optimal piecewise binning with binary target

## Basic

To get us started, let's load a well-known dataset from the UCI repository and transform the data into a ``pandas.DataFrame``.


```python
import numpy as np
import pandas as pd
from sklearn.datasets import load_breast_cancer
```


```python
data = load_breast_cancer()
df = pd.DataFrame(data.data, columns=data.feature_names)
```

We choose a variable to discretize and the binary target.


```python
variable = "mean radius"
x = df[variable].values
y = data.target
```

Import and instantiate an ``OptimalPWBinning`` object class and we pass the variable name. The ``OptimalPWBinning`` can **ONLY** handle numerical variables. This differs from the ``OptimalBinning`` object class.


```python
from optbinning import OptimalPWBinning
```


```python
optb = OptimalPWBinning(name=variable)
```

We fit the optimal binning object with arrays ``x`` and ``y``. We set a minimum (lower bound) and maximum (upper bound) probability using arguments ``lb`` and ``ub``. Note that these bounds are optional, but guarantee an event rate between $[0, 1]$ as any probability distribution. As shown later, settings these bounds will increase the problem size thus leading to higher solution times. An alternative approach to bypass these bounds is implemented when transforming to event rate or WoE using functions ``fit`` and ``fit_transform``, where values outside the interval $[0, 1]$ are clipped.

Note that the Weight-of-Evidence is not defined for event rate equal 0 or 1. Therefore, for small datasets, we recommend modifying the interval $[0, 1]$ with a small value, thus using $[\epsilon, 1 - \epsilon]$.


```python
optb.fit(x, y, lb=0.001, ub=0.999)
```




    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     min_n_bins=None, min_prebin_size=0.05, monotonic_trend='auto',
                     n_subsamples=None, name='mean radius', objective='l2',
                     outlier_detector=None, outlier_params=None,
                     prebinning_method='cart', quantile=0.5, random_state=None,
                     reg_l1=1.0, reg_l2=1.0, regularization=None, solver='auto',
                     special_codes=None, split_digits=None, user_splits=None,
                     user_splits_fixed=None, verbose=False)



You can check if an optimal solution has been found via the ``status`` attribute:


```python
optb.status
```




    'OPTIMAL'



You can also retrieve the optimal split points via the ``splits`` attribute:


```python
optb.splits
```




    array([11.42500019, 12.32999992, 13.09499979, 13.70499992, 15.04500008,
           16.92500019])



#### The binning table

The optimal binning algorithms return a binning table; a binning table displays the binned data and several metrics for each bin. Class ``OptimalPWBinning`` returns an object ``PWBinningTable`` via the ``binning_table`` attribute.


```python
binning_table = optb.binning_table
```


```python
type(binning_table)
```




    optbinning.binning.piecewise.binning_statistics.PWBinningTable



The `binning_table` is instantiated, but not built. Therefore, the first step is to call the method `build`, which returns a ``pandas.DataFrame``.


```python
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Non-event</th>
      <th>Event</th>
      <th>c0</th>
      <th>c1</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 11.43)</td>
      <td>118</td>
      <td>0.207381</td>
      <td>3</td>
      <td>115</td>
      <td>1.03265</td>
      <td>-0.0048199</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[11.43, 12.33)</td>
      <td>79</td>
      <td>0.138840</td>
      <td>3</td>
      <td>76</td>
      <td>1.64484</td>
      <td>-0.0584032</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[12.33, 13.09)</td>
      <td>68</td>
      <td>0.119508</td>
      <td>7</td>
      <td>61</td>
      <td>2.14057</td>
      <td>-0.0986085</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[13.09, 13.70)</td>
      <td>49</td>
      <td>0.086116</td>
      <td>10</td>
      <td>39</td>
      <td>2.91496</td>
      <td>-0.157745</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[13.70, 15.05)</td>
      <td>83</td>
      <td>0.145870</td>
      <td>28</td>
      <td>55</td>
      <td>4.10527</td>
      <td>-0.244597</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[15.05, 16.93)</td>
      <td>54</td>
      <td>0.094903</td>
      <td>44</td>
      <td>10</td>
      <td>3.56249</td>
      <td>-0.20852</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[16.93, inf)</td>
      <td>118</td>
      <td>0.207381</td>
      <td>117</td>
      <td>1</td>
      <td>0.0821419</td>
      <td>-0.00288659</td>
    </tr>
    <tr>
      <th>7</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>569</td>
      <td>1.000000</td>
      <td>212</td>
      <td>357</td>
      <td>-</td>
      <td>-</td>
    </tr>
  </tbody>
</table>
</div>



Let's describe the columns of this binning table:

- Bin: the intervals delimited by the optimal split points.
- Count: the number of records for each bin.
- Count (%): the percentage of records for each bin.
- Non-event: the number of non-event records $(y = 0)$ for each bin.
- Event: the number of event records $(y = 1)$ for each bin.
- $c_0$: the first coefficient of the event rate polynomial.
- $c_1$: the second coefficient of the event rate polynomial.

The event rate for bin $i$ is defined as $ER_i = c_0 + c_1 x_i$, where $x_i \in \text{Bin}_{i}$. In general,
\begin{equation}
ER_i = \sum_{j=0}^d c_j x_i^j,
\end{equation}
where $d$ is the degree of the event rate polynomial.

The last row shows the total number of records, non-event records, event records.

You can use the method ``plot`` to visualize the number of non-event and event records by bin, and WoE or event rate curve.


```python
binning_table.plot(metric="woe")
```


```python
binning_table.plot(metric="event_rate")
```

##### Event rate / WoE transformation

Now that we have checked the binned data, we can transform our original data into WoE or event rate values.


```python
x_transform_woe = optb.transform(x, metric="woe")
```


```python
x_transform_event_rate = optb.transform(x, metric="event_rate")
```

## Advanced

#### Optimal binning Information

The ``OptimalPWBinning`` can print overview information about the options settings, problem statistics, and the solution of the computation. By default, ``print_level=1``.


```python
optb = OptimalPWBinning(name=variable)
optb.fit(x, y)
```




    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     min_n_bins=None, min_prebin_size=0.05, monotonic_trend='auto',
                     n_subsamples=None, name='mean radius', objective='l2',
                     outlier_detector=None, outlier_params=None,
                     prebinning_method='cart', quantile=0.5, random_state=None,
                     reg_l1=1.0, reg_l2=1.0, regularization=None, solver='auto',
                     special_codes=None, split_digits=None, user_splits=None,
                     user_splits_fixed=None, verbose=False)



If ``print_level=0``, a minimal output including the header, variable name, status, and total time are printed.


```python
optb.information(print_level=0)
```

    optbinning (Version 0.9.0)
    Copyright (c) 2019-2021 Guillermo Navas-Palencia, Apache License 2.0
    
      Name    : mean radius                     
      Status  : OPTIMAL                         
    
      Time    : 0.0717  sec
    


If ``print_level>=1``, statistics on the pre-binning phase and the solver are printed. More detailed timing statistics are also included.


```python
optb.information(print_level=1)
```

    optbinning (Version 0.9.0)
    Copyright (c) 2019-2021 Guillermo Navas-Palencia, Apache License 2.0
    
      Name    : mean radius                     
      Status  : OPTIMAL                         
    
      Pre-binning statistics
        Number of bins                         7
    
      Solver statistics
        Type                                auto
        Number of variables                   14
        Number of constraints                 20
    
      Timing
        Total time                          0.07 sec
        Pre-processing                      0.00 sec   (  0.29%)
        Estimator                           0.01 sec   ( 12.23%)
        Pre-binning                         0.04 sec   ( 55.33%)
        Solver                              0.02 sec   ( 21.76%)
        Post-processing                     0.01 sec   (  9.47%)
    


If ``print_level=2``, the list of all options of the ``OptimalBinning`` are displayed.


```python
optb.information(print_level=2)
```

    optbinning (Version 0.9.0)
    Copyright (c) 2019-2021 Guillermo Navas-Palencia, Apache License 2.0
    
      Begin options
        name                         mean radius   * U
        estimator                            yes   * U
        objective                             l2   * d
        degree                                 1   * d
        continuous                          True   * d
        prebinning_method                   cart   * d
        max_n_prebins                         20   * d
        min_prebin_size                     0.05   * d
        min_n_bins                            no   * d
        max_n_bins                            no   * d
        min_bin_size                          no   * d
        max_bin_size                          no   * d
        monotonic_trend                     auto   * d
        n_subsamples                          no   * d
        max_pvalue                            no   * d
        max_pvalue_policy            consecutive   * d
        outlier_detector                      no   * d
        outlier_params                        no   * d
        user_splits                           no   * d
        special_codes                         no   * d
        split_digits                          no   * d
        solver                              auto   * d
        h_epsilon                           1.35   * d
        quantile                             0.5   * d
        regularization                        no   * d
        reg_l1                               1.0   * d
        reg_l2                               1.0   * d
        random_state                          no   * d
        verbose                            False   * d
      End options
    
      Name    : mean radius                     
      Status  : OPTIMAL                         
    
      Pre-binning statistics
        Number of bins                         7
    
      Solver statistics
        Type                                auto
        Number of variables                   14
        Number of constraints                 20
    
      Timing
        Total time                          0.07 sec
        Pre-processing                      0.00 sec   (  0.29%)
        Estimator                           0.01 sec   ( 12.23%)
        Pre-binning                         0.04 sec   ( 55.33%)
        Solver                              0.02 sec   ( 21.76%)
        Post-processing                     0.01 sec   (  9.47%)
    


#### Binning table statistical analysis

The ``analysis`` method performs a statistical analysis of the binning table, computing the statistics Gini index, Information Value (IV), Jensen-Shannon divergence, and the quality score. Additionally, several statistical significance tests between consecutive bins of the contingency table are performed: a frequentist test using the Chi-square test or the Fisher's exact test, and a Bayesian A/B test using the beta distribution as a conjugate prior of the Bernoulli distribution. The piecewise binning also includes two performance metrics: the average precision and the Brier score.


```python
binning_table.analysis(pvalue_test="chi2")
```

    ---------------------------------------------
    OptimalBinning: Binary Binning Table Analysis
    ---------------------------------------------
    
      General metrics
    
        Gini index               0.87503303
        IV (Jeffrey)             4.47808263
        JS (Jensen-Shannon)      0.37566002
        Hellinger                0.44267970
        Triangular               1.20862637
        KS                       0.72390466
        Avg precision            0.95577179
        Brier score              0.08956351
        HHI                      0.15727342
        HHI (normalized)         0.05193260
        Cramer's V               0.80066760
        Quality score            0.00000000
    
      Significance tests
    
        Bin A  Bin B  t-statistic       p-value  P[A > B]      P[B > A]
            0      1     0.289789  5.903557e-01  0.697563  3.024368e-01
            1      2     2.285355  1.306002e-01  0.941469  5.853092e-02
            2      3     2.317546  1.279217e-01  0.936661  6.333919e-02
            3      4     2.539025  1.110634e-01  0.950869  4.913141e-02
            4      5    29.418844  5.830790e-08  1.000000  1.898159e-11
            5      6    18.624518  1.591604e-05  0.999997  3.106835e-06
    



```python
binning_table.analysis(pvalue_test="fisher")
```

    ---------------------------------------------
    OptimalBinning: Binary Binning Table Analysis
    ---------------------------------------------
    
      General metrics
    
        Gini index               0.87503303
        IV (Jeffrey)             4.47808263
        JS (Jensen-Shannon)      0.37566002
        Hellinger                0.44267970
        Triangular               1.20862637
        KS                       0.72390466
        Avg precision            0.95577179
        Brier score              0.08956351
        HHI                      0.15727342
        HHI (normalized)         0.05193260
        Cramer's V               0.80066760
        Quality score            0.00000000
    
      Significance tests
    
        Bin A  Bin B  odd ratio       p-value  P[A > B]      P[B > A]
            0      1   0.660870  6.855009e-01  0.697563  3.024368e-01
            1      2   0.343985  1.879769e-01  0.941469  5.853092e-02
            2      3   0.447541  1.829850e-01  0.936661  6.333919e-02
            3      4   0.503663  1.153088e-01  0.950869  4.913141e-02
            4      5   0.115702  3.464724e-08  1.000000  1.898159e-11
            5      6   0.037607  4.144258e-05  0.999997  3.106835e-06
    


#### Event rate / WoE monotonicity

The ``monotonic_trend`` option permits forcing a monotonic trend to the event rate curve. The default setting "auto" should be the preferred option, however, some business constraints might require to impose different trends. The default setting "auto" chooses the monotonic trend most likely to maximize the information value from the options "ascending", "descending", "peak" and "valley" using a machine-learning-based classifier.


```python
variable = "mean texture"
x = df[variable].values
y = data.target
```


```python
optb = OptimalPWBinning(name=variable)
optb.fit(x, y, lb=0.001, ub=0.999)
```




    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     min_n_bins=None, min_prebin_size=0.05, monotonic_trend='auto',
                     n_subsamples=None, name='mean texture', objective='l2',
                     outlier_detector=None, outlier_params=None,
                     prebinning_method='cart', quantile=0.5, random_state=None,
                     reg_l1=1.0, reg_l2=1.0, regularization=None, solver='auto',
                     special_codes=None, split_digits=None, user_splits=None,
                     user_splits_fixed=None, verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Non-event</th>
      <th>Event</th>
      <th>c0</th>
      <th>c1</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 15.05)</td>
      <td>92</td>
      <td>0.161687</td>
      <td>4</td>
      <td>88</td>
      <td>1.19518</td>
      <td>-0.0239015</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[15.05, 16.39)</td>
      <td>61</td>
      <td>0.107206</td>
      <td>8</td>
      <td>53</td>
      <td>1.43854</td>
      <td>-0.0400774</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[16.39, 17.03)</td>
      <td>29</td>
      <td>0.050967</td>
      <td>6</td>
      <td>23</td>
      <td>1.42601</td>
      <td>-0.0393128</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[17.03, 18.46)</td>
      <td>79</td>
      <td>0.138840</td>
      <td>17</td>
      <td>62</td>
      <td>1.55674</td>
      <td>-0.0469873</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[18.46, 19.47)</td>
      <td>55</td>
      <td>0.096661</td>
      <td>20</td>
      <td>35</td>
      <td>1.66523</td>
      <td>-0.0528638</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[19.47, 20.20)</td>
      <td>36</td>
      <td>0.063269</td>
      <td>18</td>
      <td>18</td>
      <td>1.71402</td>
      <td>-0.0553698</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[20.20, 21.71)</td>
      <td>72</td>
      <td>0.126538</td>
      <td>43</td>
      <td>29</td>
      <td>1.76253</td>
      <td>-0.0577723</td>
    </tr>
    <tr>
      <th>7</th>
      <td>[21.71, 22.74)</td>
      <td>40</td>
      <td>0.070299</td>
      <td>27</td>
      <td>13</td>
      <td>1.80404</td>
      <td>-0.059684</td>
    </tr>
    <tr>
      <th>8</th>
      <td>[22.74, 24.00)</td>
      <td>29</td>
      <td>0.050967</td>
      <td>24</td>
      <td>5</td>
      <td>1.67099</td>
      <td>-0.0538332</td>
    </tr>
    <tr>
      <th>9</th>
      <td>[24.00, 26.98)</td>
      <td>43</td>
      <td>0.075571</td>
      <td>30</td>
      <td>13</td>
      <td>1.75293</td>
      <td>-0.0572474</td>
    </tr>
    <tr>
      <th>10</th>
      <td>[26.98, inf)</td>
      <td>33</td>
      <td>0.057996</td>
      <td>15</td>
      <td>18</td>
      <td>0.663321</td>
      <td>-0.0168615</td>
    </tr>
    <tr>
      <th>11</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>12</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>569</td>
      <td>1.000000</td>
      <td>212</td>
      <td>357</td>
      <td>-</td>
      <td>-</td>
    </tr>
  </tbody>
</table>
</div>




```python
binning_table.plot(metric="event_rate")
```

For example, we can force the variable mean texture to be convex with respect to the probability of having breast cancer.


```python
optb = OptimalPWBinning(name=variable, monotonic_trend="convex")
optb.fit(x, y, lb=0.001, ub=0.999)
```




    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     min_n_bins=None, min_prebin_size=0.05,
                     monotonic_trend='convex', n_subsamples=None,
                     name='mean texture', objective='l2', outlier_detector=None,
                     outlier_params=None, prebinning_method='cart', quantile=0.5,
                     random_state=None, reg_l1=1.0, reg_l2=1.0, regularization=None,
                     solver='auto', special_codes=None, split_digits=None,
                     user_splits=None, user_splits_fixed=None, verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
binning_table.plot(metric="event_rate")
```

#### User-defined split points

In some situations, we have defined split points or bins required to satisfy a priori belief,  knowledge or business constraint. The ``OptimalPWBinning`` permits to pass user-defined split points. The supplied information is used as a pre-binning, disallowing any pre-binning method set by the user. Furthermore, version 0.5.0 introduces ``user_splits_fixed`` parameter, to allow the user to fix some user-defined splits, so these must appear in the solution.

Example numerical variable:


```python
user_splits =       [  14,    15,    16,    17,    20,    21,    22,    27]
user_splits_fixed = [False, True,  True, False, False, False, False, False]
```


```python
optb = OptimalPWBinning(name=variable, user_splits=user_splits,
                        user_splits_fixed=user_splits_fixed)

optb.fit(x, y)
```




    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     n_subsamples=None, name='mean texture', objective='l2',
                     outlier_detector=None, outlier_params=None,
                     prebinning_method='cart', quantile=0.5, random_state=None,
                     reg_l1=1.0, reg_l2=1.0, regularization=None, solver='auto',
                     special_codes=None, split_digits=None,
                     user_splits=[14, 15, 16, 17, 20, 21, 22, 27],
                     user_splits_fixed=[False, True, True, False, False, False,
                                        False, False],
                     verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Non-event</th>
      <th>Event</th>
      <th>c0</th>
      <th>c1</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 14.00)</td>
      <td>54</td>
      <td>0.094903</td>
      <td>2</td>
      <td>52</td>
      <td>1.15397</td>
      <td>-0.0205654</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[14.00, 15.00)</td>
      <td>37</td>
      <td>0.065026</td>
      <td>2</td>
      <td>35</td>
      <td>1.35451</td>
      <td>-0.0348897</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[15.00, 16.00)</td>
      <td>43</td>
      <td>0.075571</td>
      <td>7</td>
      <td>36</td>
      <td>1.24446</td>
      <td>-0.0275524</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[16.00, 20.00)</td>
      <td>210</td>
      <td>0.369069</td>
      <td>59</td>
      <td>151</td>
      <td>1.5709</td>
      <td>-0.0479552</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[20.00, 21.00)</td>
      <td>45</td>
      <td>0.079086</td>
      <td>26</td>
      <td>19</td>
      <td>1.87464</td>
      <td>-0.063142</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[21.00, 22.00)</td>
      <td>49</td>
      <td>0.086116</td>
      <td>30</td>
      <td>19</td>
      <td>1.78837</td>
      <td>-0.0590341</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[22.00, 27.00)</td>
      <td>99</td>
      <td>0.173989</td>
      <td>71</td>
      <td>28</td>
      <td>1.70844</td>
      <td>-0.0554011</td>
    </tr>
    <tr>
      <th>7</th>
      <td>[27.00, inf)</td>
      <td>32</td>
      <td>0.056239</td>
      <td>15</td>
      <td>17</td>
      <td>0.789092</td>
      <td>-0.021351</td>
    </tr>
    <tr>
      <th>8</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>9</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>569</td>
      <td>1.000000</td>
      <td>212</td>
      <td>357</td>
      <td>-</td>
      <td>-</td>
    </tr>
  </tbody>
</table>
</div>




```python
optb.information()
```

    optbinning (Version 0.9.0)
    Copyright (c) 2019-2021 Guillermo Navas-Palencia, Apache License 2.0
    
      Name    : mean texture                    
      Status  : OPTIMAL                         
    
      Pre-binning statistics
        Number of bins                         8
    
      Solver statistics
        Type                                auto
        Number of variables                   16
        Number of constraints                 23
    
      Timing
        Total time                          0.15 sec
        Pre-processing                      0.00 sec   (  0.18%)
        Estimator                           0.01 sec   (  7.64%)
        Pre-binning                         0.09 sec   ( 61.96%)
        Solver                              0.03 sec   ( 21.76%)
        Post-processing                     0.01 sec   (  7.82%)
    


#### Performance: choosing a solver

``OptimalPWBinning`` uses the RoPWR library to solving the piecewise regression problem. See https://github.com/guillermo-navas-palencia/ropwr for details.

#### Robustness: choosing an objective function and regularization

``OptimalPWBinning`` uses the RoPWR library to solving the piecewise regression problem. See https://github.com/guillermo-navas-palencia/ropwr for details.

#### Missing data and special codes

For this example, let's load data from the FICO Explainable Machine Learning Challenge: https://community.fico.com/s/explainable-machine-learning-challenge


```python
df = pd.read_csv("data/FICO_challenge/heloc_dataset_v1.csv", sep=",")
```

The data dictionary of this challenge includes three special values/codes:

* -9 No Bureau Record or No Investigation
* -8 No Usable/Valid Trades or Inquiries
* -7 Condition not Met (e.g. No Inquiries, No Delinquencies) 


```python
special_codes = [-9, -8, -7]
```


```python
variable = "AverageMInFile"
x = df[variable].values
y = df.RiskPerformance.values
```


```python
df.RiskPerformance.unique()
```




    array(['Bad', 'Good'], dtype=object)



Target is a categorical dichotomic variable, which can be easily transform into numerical.


```python
mask = y == "Bad"
y[mask] = 1
y[~mask] = 0
y = y.astype(np.int)
```

For the sake of completeness, we include a few missing values


```python
idx = np.random.randint(0, len(x), 500)
x = x.astype(np.float)
x[idx] = np.nan
```


```python
optb = OptimalPWBinning(name=variable, special_codes=special_codes, solver="osqp")

optb.fit(x, y, lb=0, ub=1)
```




    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     min_n_bins=None, min_prebin_size=0.05, monotonic_trend='auto',
                     n_subsamples=None, name='AverageMInFile', objective='l2',
                     outlier_detector=None, outlier_params=None,
                     prebinning_method='cart', quantile=0.5, random_state=None,
                     reg_l1=1.0, reg_l2=1.0, regularization=None, solver='osqp',
                     special_codes=[-9, -8, -7], split_digits=None,
                     user_splits=None, user_splits_fixed=None, verbose=False)




```python
optb.information(print_level=1)
```

    optbinning (Version 0.9.0)
    Copyright (c) 2019-2021 Guillermo Navas-Palencia, Apache License 2.0
    
      Name    : AverageMInFile                  
      Status  : OPTIMAL                         
    
      Pre-binning statistics
        Number of bins                        12
    
      Solver statistics
        Type                                osqp
        Number of variables                   24
        Number of constraints              18847
    
      Timing
        Total time                          0.39 sec
        Pre-processing                      0.00 sec   (  0.44%)
        Estimator                           0.04 sec   (  9.37%)
        Pre-binning                         0.17 sec   ( 43.37%)
        Solver                              0.16 sec   ( 41.45%)
        Post-processing                     0.02 sec   (  4.96%)
    



```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Non-event</th>
      <th>Event</th>
      <th>c0</th>
      <th>c1</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 30.50)</td>
      <td>543</td>
      <td>0.051917</td>
      <td>97</td>
      <td>446</td>
      <td>0.788301</td>
      <td>-0.00304249</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[30.50, 41.50)</td>
      <td>554</td>
      <td>0.052969</td>
      <td>147</td>
      <td>407</td>
      <td>0.799788</td>
      <td>-0.00341911</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[41.50, 48.50)</td>
      <td>516</td>
      <td>0.049336</td>
      <td>145</td>
      <td>371</td>
      <td>0.803716</td>
      <td>-0.00351377</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[48.50, 54.50)</td>
      <td>528</td>
      <td>0.050483</td>
      <td>188</td>
      <td>340</td>
      <td>0.808907</td>
      <td>-0.0036208</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[54.50, 59.50)</td>
      <td>494</td>
      <td>0.047232</td>
      <td>195</td>
      <td>299</td>
      <td>0.812732</td>
      <td>-0.00369099</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[59.50, 64.50)</td>
      <td>600</td>
      <td>0.057367</td>
      <td>263</td>
      <td>337</td>
      <td>0.814625</td>
      <td>-0.0037228</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[64.50, 70.50)</td>
      <td>781</td>
      <td>0.074673</td>
      <td>361</td>
      <td>420</td>
      <td>0.821307</td>
      <td>-0.00382639</td>
    </tr>
    <tr>
      <th>7</th>
      <td>[70.50, 74.50)</td>
      <td>534</td>
      <td>0.051057</td>
      <td>260</td>
      <td>274</td>
      <td>0.802862</td>
      <td>-0.00356477</td>
    </tr>
    <tr>
      <th>8</th>
      <td>[74.50, 81.50)</td>
      <td>904</td>
      <td>0.086433</td>
      <td>476</td>
      <td>428</td>
      <td>0.858438</td>
      <td>-0.00431075</td>
    </tr>
    <tr>
      <th>9</th>
      <td>[81.50, 97.50)</td>
      <td>1693</td>
      <td>0.161870</td>
      <td>961</td>
      <td>732</td>
      <td>0.77016</td>
      <td>-0.00322759</td>
    </tr>
    <tr>
      <th>10</th>
      <td>[97.50, 116.50)</td>
      <td>1169</td>
      <td>0.111770</td>
      <td>711</td>
      <td>458</td>
      <td>1.03335</td>
      <td>-0.00592698</td>
    </tr>
    <tr>
      <th>11</th>
      <td>[116.50, inf)</td>
      <td>1090</td>
      <td>0.104216</td>
      <td>703</td>
      <td>387</td>
      <td>0.492737</td>
      <td>-0.00128652</td>
    </tr>
    <tr>
      <th>12</th>
      <td>Special</td>
      <td>566</td>
      <td>0.054116</td>
      <td>255</td>
      <td>311</td>
      <td>0.54947</td>
      <td>0</td>
    </tr>
    <tr>
      <th>13</th>
      <td>Missing</td>
      <td>487</td>
      <td>0.046563</td>
      <td>238</td>
      <td>249</td>
      <td>0.511294</td>
      <td>0</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>10459</td>
      <td>1.000000</td>
      <td>5000</td>
      <td>5459</td>
      <td>-</td>
      <td>-</td>
    </tr>
  </tbody>
</table>
</div>



Note the dashed bins 10 and 11, corresponding to the special codes bin and the missing bin, respectively.


```python
binning_table.plot(metric="event_rate")
```

#### Verbosity option

For debugging purposes, we can print information on each step of the computation by triggering the ``verbose`` option.


```python
optb = OptimalPWBinning(name=variable, solver="osqp", verbose=True)
optb.fit(x, y, lb=0, ub=1)
```

    2021-01-12 20:10:32,053 | INFO : Optimal piecewise binning started.
    2021-01-12 20:10:32,054 | INFO : Options: check parameters.
    2021-01-12 20:10:32,055 | INFO : Pre-processing started.
    2021-01-12 20:10:32,056 | INFO : Pre-processing: number of samples: 10459
    2021-01-12 20:10:32,059 | INFO : Pre-processing: number of clean samples: 9972
    2021-01-12 20:10:32,060 | INFO : Pre-processing: number of missing samples: 487
    2021-01-12 20:10:32,061 | INFO : Pre-processing: number of special samples: 0
    2021-01-12 20:10:32,062 | INFO : Pre-processing terminated. Time: 0.0019s
    2021-01-12 20:10:32,062 | INFO : Pre-binning: set logistic regression as an estimator.
    2021-01-12 20:10:32,063 | INFO : Pre-binning: estimator fitting started.
    2021-01-12 20:10:32,107 | INFO : Pre-binning: estimator terminated. Time 0.0448s.
    2021-01-12 20:10:32,108 | INFO : Pre-binning: optimal binning started.
    2021-01-12 20:10:32,287 | INFO : Pre-binning: number of splits: 9.
    2021-01-12 20:10:32,289 | INFO : Pre-binning: no need for subsamples.
    2021-01-12 20:10:32,292 | INFO : Pre-binning: optimal binning terminated. Time 0.183s.
    2021-01-12 20:10:32,294 | INFO : Optimizer started.
    2021-01-12 20:10:32,297 | INFO : Optimizer: descending monotonic trend.
    -----------------------------------------------------------------
               OSQP v0.6.0  -  Operator Splitting QP Solver
                  (c) Bartolomeo Stellato,  Goran Banjac
            University of Oxford  -  Stanford University 2019
    -----------------------------------------------------------------
    problem:  variables n = 20, constraints m = 19973
              nnz(P) + nnz(A) = 39964
    settings: linear system solver = qdldl,
              eps_abs = 1.0e-05, eps_rel = 1.0e-05,
              eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,
              rho = 1.00e-01 (adaptive),
              sigma = 1.00e-06, alpha = 1.60, max_iter = 10000
              check_termination: on (interval 25),
              scaling: on, scaled_termination: off
              warm start: on, polish: on, time_limit: off
    
    iter   objective    pri res    dua res    rho        time
       1  -4.3051e+01   9.52e-04   1.61e+05   1.00e-01   1.64e-02s
     200  -2.5013e+03   8.64e-03   1.16e+04   1.86e-02   1.20e-01s
     400  -2.7932e+03   3.69e-04   3.01e+03   1.86e-02   2.22e-01s
     525  -2.8264e+03   4.36e-06   1.38e-02   4.19e-03   2.81e-01s
    plsh  -2.8264e+03   1.17e-16   7.61e-11   --------   2.82e-01s
    
    status:               solved
    solution polish:      successful
    number of iterations: 525
    optimal objective:    -2826.4429
    run time:             2.82e-01s
    optimal rho estimate: 1.17e-02
    
    2021-01-12 20:10:32,624 | INFO : Optimizer terminated. Time: 0.3253s
    2021-01-12 20:10:32,625 | INFO : Post-processing started.
    2021-01-12 20:10:32,626 | INFO : Post-processing: compute binning information.
    2021-01-12 20:10:32,635 | INFO : Post-processing: compute performance metrics.
    2021-01-12 20:10:32,650 | INFO : Post-processing terminated. Time: 0.0227s
    2021-01-12 20:10:32,650 | INFO : Optimal piecewise binning terminated. Status: OPTIMAL. Time: 0.5978s





    OptimalPWBinning(continuous=True, degree=1,
                     estimator=LogisticRegression(C=1.0, class_weight=None,
                                                  dual=False, fit_intercept=True,
                                                  intercept_scaling=1,
                                                  l1_ratio=None, max_iter=100,
                                                  multi_class='auto', n_jobs=None,
                                                  penalty='l2', random_state=None,
                                                  solver='lbfgs', tol=0.0001,
                                                  verbose=0, warm_start=False),
                     h_epsilon=1.35, max_bin_size=None, max_n_bins=None,
                     max_n_prebins...
                     min_n_bins=None, min_prebin_size=0.05, monotonic_trend='auto',
                     n_subsamples=None, name='AverageMInFile', objective='l2',
                     outlier_detector=None, outlier_params=None,
                     prebinning_method='cart', quantile=0.5, random_state=None,
                     reg_l1=1.0, reg_l2=1.0, regularization=None, solver='osqp',
                     special_codes=None, split_digits=None, user_splits=None,
                     user_splits_fixed=None, verbose=True)



3bfe902bb1fc5c89cf3f9c69ba8d840b79cc4c55
ipynb
Jupyter Notebook
doc/source/tutorials/tutorial_piecewise_binary.ipynb
enriczhang/optbinning
918b4b0e0f417edfaf9dd5fc80b3021a355fc3dd
Apache-2.0
2021-02-09T02:49:32.000Z
2021-02-09T02:49:32.000Z
doc/source/tutorials/tutorial_piecewise_binary.ipynb
enriczhang/optbinning
918b4b0e0f417edfaf9dd5fc80b3021a355fc3dd
Apache-2.0
doc/source/tutorials/tutorial_piecewise_binary.ipynb
enriczhang/optbinning
918b4b0e0f417edfaf9dd5fc80b3021a355fc3dd
Apache-2.0
Qwen/Qwen-72B
1. YES
2. YES