Sample size, P-values (its relationship), and data visualization with plots. ggplot and T test We know that P-values (within t-test context as an example..) is highly sensitive to sample size. A larger sample will yield a smaller p-value remaining everything else constant. On the other hand, Cohen´s d effect size remains the same.



I'm inspired in this code here, but I´ve changed some parts to make the difference between means constant, instead of creating a random variable based on a normal distribution.

Although everything is working, I do imagine that some of the experts in this community could improve my syntax.

library(tidyverse)

ctrl_mean &lt;- 8
ctrl_sd &lt;- 1

treated_mean &lt;- 7.9
treated_sd &lt;- 1.2

sample &lt;- numeric() #criar vetor para grupar resultados
nsim &lt;- 1000 #criar variavel
t_result &lt;- numeric()

for (i in 1:nsim) { 
  set.seed(123) 
  t_result[i] &lt;- (mean(ctrl_mean)-mean(treated_mean))/sqrt((ctrl_sd^2/(i))+(treated_sd^2/(i))) #manual t test
  sample[i] &lt;- i # number of participants
}
ds &lt;- data.frame(
  sample = sample, #assign the sample size
  t_result = round(t_result,3), #get the t test result
  degrees = sample*2-2) #compute the degrees of freedom

ds %&gt;% 
  filter(sample&gt;1) %&gt;% 
  mutate(P_Value = 2*pt(abs(t_result), df=degrees,lower.tail=FALSE)) %&gt;% 
  left_join(ds,.) -&gt; ds

#plot 
ggplot(ds, aes(x=sample, y=P_Value)) +
  geom_line() +
  annotate("segment", x = 1, xend=sample, y = 0.05, yend = 0.05, colour = "purple", linetype = "dashed") +
  annotate("segment", x = 1, xend=sample, y = 0.01, yend = 0.01, colour = "red", linetype = "dashed") +
  annotate("text", x = c(1,1), y=c(.035,.001), label = c("p &lt; 0.05", "p &lt; 0.01"))
In this code you do not need the loop:

sample &lt;- 1:nsim
t_result &lt;- (mean(ctrl_mean)-mean(treated_mean)) /
  sqrt((ctrl_sd^2/(sample))+(treated_sd^2/(sample)))
# OR:
t_result &lt;- (mean(ctrl_mean) - mean(treated_mean)) /
  sqrt((ctrl_sd^2 + treated_sd^2) / sample)


why the set seed?