# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/PennNGG/Quantitative-Neuroscience/blob/master/Hypothesis%20Testing/Python/Simple%20Non%2dParametric%20Tests.ipynb)

# + [markdown] id="pKIiY6p3GRFq"
# # Definitions

# + [markdown] id="x7VmLUr5GTNw"
# When comparing the central tendencies (e.g., means) of two samples, if you know how the samples are distributed and/or *n* is large enough in each sample so that you can assume that their means are normally distributed via the Central Limit Theorem, then it is reasonable to use parametric hypothesis tests like a [*t*-test](https://colab.research.google.com/drive/1M7xjaMwJUEyULPHfXc3tWG6-WVjCl-uQ?usp=sharing). Otherwise, nonparametric tests should be used. 
#
# There is an increasing understanding by neuroscientists and others that parametric tests, while they tend to be simple and convenient, often have assumptions that are not well justified and thus should be used only when appropriate. See, for example:
#
# [Running the Numbers](https://www.nature.com/articles/nn0205-123) from *Nature Neuroscience*, which discusses using nonparametric approaches when appropriate.
#
# [An evaluation of nonparametric approaches in clinical trials](https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-5-35), which covers several of the tests described below.
#
# Here we will not dive deep into the math but instead will provide a more practical guide for when to use three common forms of nonparametric one- and two-sample tests.

# + [markdown] id="5ULZKAuOBol4"
#
# # Getting started with code

# + [markdown] id="S57swuS-BwVO"
# Matlab code is found in the [NGG Statistics GitHub Repository](https://github.com/PennNGG/Statistics.git) under "Hypothesis Testing/SimpleNonParametricTests.m".
#
# Python code is included below. First run the code cell just below to make sure all of the required Python modules are loaded, then you can run the other cell(s).

# + id="CjqTNogqBtUX"
import numpy as np
import scipy.stats as st
from statsmodels.stats.descriptivestats import sign_test

# + [markdown] id="nvmWeSSHSF95"
# # Sign test: one sample, skewed distribution

# + [markdown] id="bki88QRVSaj8"
# This test is applied to a single sample but is typically used for paired measurements to test the hypothesis that there is no systematic direction of a treatment effect (positive=the treatment resulted in a bigger value, negative=the treatment resulted in a smaller value), regardless of magnitude. The Null hypothesis is an equal probability of an effect either direction, so the data are treated as a [binomial distribution](https://colab.research.google.com/drive/1q1KaEjkAzUKRFSLPQ0SFdqU_byc70Oi2?usp=sharing) with *p*=0.5

# + id="NDfcV21FJ7v0"
# Make some paired data
a = [3,10,4,20,4,7,50,3,5,5,7]
b = [5,9,10,15,6,5,43,6,2,1,0]
diff = [bi-ai for ai, bi in zip(a,b)]
_, p = sign_test(diff)
print(f'p={p:.2f}')

# + [markdown] id="To3hA26GKjv0"
# # Wilcoxon signed-rank test: one sample or paired samples, symmetric distribution(s)

# + [markdown] id="QtK7ZyfjKojz"
# This test is typically used as a substitute for a [one-sample *t*-test](https://colab.research.google.com/drive/1M7xjaMwJUEyULPHfXc3tWG6-WVjCl-uQ?usp=sharing) and can be used to test the (null) hypothesis that a sample came from a population with a particular median value, or that the median values of two paired samples are equal to each other (i.e., the median of the difference distribution is zero)
#

# + colab={"base_uri": "https://localhost:8080/"} executionInfo={"elapsed": 97, "status": "ok", "timestamp": 1626630526050, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhY1fK-mWt81XFeafwTBs66lN9JXee76x713d4Y=s64", "userId": "06476168460066594437"}, "user_tz": 240} id="yjGigVEMLWMP" outputId="1aef3c87-7610-4f79-abf8-c01145928cd8"
samples = np.random.randint(0, high=51, size=200)
null_hypothesis_median = 24

# Unlike in Matlab, the scipy implementation does not handle the case of comparing
#  to a median other than zero, so we make this a (fake) paired two-sample test
#  by subtracting the median from each value
_, p = st.wilcoxon(samples-null_hypothesis_median)
print(f'p = {p:.2f}')


# + [markdown] id="k_iBh6ZsLnX7"
# # Mann-Whitney: unpaired, two sample
#

# + [markdown] id="clnBO4FU28El"
# This is the test that is typically used as a substitute for a [two-sample *t*-test](https://colab.research.google.com/drive/1M7xjaMwJUEyULPHfXc3tWG6-WVjCl-uQ?usp=sharing) and can be used to test the hypothesis that the two unpaired samples come from distributions that differ by a particular median value.
#
#

# + colab={"base_uri": "https://localhost:8080/"} executionInfo={"elapsed": 121, "status": "ok", "timestamp": 1626630827326, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhY1fK-mWt81XFeafwTBs66lN9JXee76x713d4Y=s64", "userId": "06476168460066594437"}, "user_tz": 240} id="vbhD1d9vOBCM" outputId="b29805f4-281f-4b5c-ce45-9dd09dab6c48"
X = np.random.randint(0, high=51, size=200)
Y = 2 + np.random.randint(0, high=51, size=200)
_, p = st.mannwhitneyu(X,Y)
print(f'p = {p:.2f}')

# + [markdown] id="UqoNXyuxP-go"
# # Additional Resources
#

# + [markdown] id="tteEm2Qlgbb3"
# # Credits
#
# Copyright 2021 by <NAME>, University of Pennsylvania

StarcoderdataJupyterScriptsDedupFiltered