Unanswered Questions
5,699 questions with no upvoted or accepted answers
16
votes
0
answers
1k
views
Understanding Sequential Probability Ratio Test (SPRT) Likelihood Ratio
I am a software developer looking to develop an alternative for the simple hypothesis testing scheme described here. In short, the test works as follows:
Two URLs are compared for their ability to ...
CommunityBot
- 1
13
votes
0
answers
2k
views
Are these equivalent (for p-values): super-uniform, stochastically larger than / dominating the uniform, conservative?
In the literature and online, I have found three different wordings that I think refer to the same concept: stochastically larger than uniform (which I take is ...
CommunityBot
- 1
11
votes
1
answer
3k
views
How to test for the difference in skewness of two samples?
I have two samples. From looking at their densities, one appears symmetrical and the other from some right-tailed distribution. I would like to test that the two do not have the same skewness (...
CommunityBot
- 1
9
votes
0
answers
629
views
How is it possible to control false discovery rate (FDR) without knowing the power and the prevalence of the nulls?
If we have p-value 0.05, to calculate probability of our discovery to be false positive, we need to use complex formula with prevalence (prior probability) and statistical power.
If we have lots of ...
- 108k
9
votes
0
answers
4k
views
T-test for regression coefficients obtained from Ridge, LASSO etc
In ordinary least squares, for example in an experimental design case, I obtain the regression coefficents by:
$ \hat B = {({X^t}{X})}^{-1}X^ty$
Then, my null hypothesis for each coefficent is:
$...
CommunityBot
- 1
9
votes
0
answers
1k
views
Proof of Kolmogorov-Smirnov test
Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the $...
9
votes
0
answers
6k
views
One-Sample t Test and Likelihood Ratio Test
Concerning testing the hypotheses about the mean $\mu$ of a single population I know of things like the One-Sample $t$ test where there is a null hypothesis: $H_0: \mu = \mu_0$, and a test statistic $...
- 1,092
8
votes
1
answer
105
views
Testing a multifactor asset pricing model against another one
I have a sample of $(Y_{i,t},X_{1,i,t},X_{2,i,t})$ for $i=1,\dots,N$ and $t=1,\dots,T$. I want to figure out which data generating process (DGP) it comes from, DGP1 or DGP2.
DGP1:
$$
Y_{i,t}=\lambda_0+...
- 70.5k
8
votes
2
answers
5k
views
2-Sample Proportions z-test vs Fisher's Exact Test
I want to evaluate the difference in proportions in declined applications and exceptions from policy from one quarter of data to the next. My sample sizes are generally not small (several hundred to ...
CommunityBot
- 1
8
votes
0
answers
6k
views
Two-Sample Hypothesis Test for Difference Between Two Proportions: Pooled vs Unpooled? How to Choose?
When are you supposed to use the pooled formula vs the unpooled formula? My understanding is that you use the pooled formula when testing a hypothesis where the null hypothesis is $H_0 = 0$ and the ...
- 1
8
votes
0
answers
4k
views
Assumptions of correlation test vs regression slope test (significance testing)
If my understanding is correct, then
the test on a regression slope in a simple bivariate regression - i.e. the test of $\mathcal{H}_0$: $b = 0$ in $Y' = a + bX$
and
the test of a correlation, i.e. $\...
- 70.4k
8
votes
0
answers
690
views
Skew of p-value distribution under composite null hypotheses
On this page it says
...if HA holds, the p-values have a distribution for which values near 0 are more likely than values near 1. However the p-values may have a distribution that is not rectangular ...
- 12.9k
8
votes
1
answer
7k
views
How to analyze and interpret A/B test results via Bootstrap method?
We've run a split test of a new product feature and want to measure if the uplift on revenue is significant. Our observations are definitely not normally distributed (most of our users don't spend, ...
CommunityBot
- 1
8
votes
0
answers
652
views
Calculate goodness-of-fit (with deviance) to compare averaged models?
I need to compare the goodness of fit of several averaged logistic regression models by calculating the deviance explained. I'm using the MuMIn package in R to ...
CommunityBot
- 1
7
votes
1
answer
2k
views
Overfitting a neural network to a single batch as a sanity check - how small a loss value is small enough and long to run for?
I'm currently developing a neural network for a regression task. Following on from the advice given in places like here, here, and here I'm attempting to overfit my model to a single batch of 5 ...
CommunityBot
- 1