Current Slide

Small screen detected. You are viewing the mobile version of SlideWiki. If you wish to edit slides you will need to use a larger device.

Estimating Confidence Intervals: t-test

  • If only 1 test set available: pairwise comparison
    • For ith round of 10-fold cross-validation, the same cross partitioning is used to obtain err(M1)i and err(M2)i
    • Average over 10 rounds to get err'(M1) and err'(M2)
    • t-test computes t-statistic with k-1 degrees of freedom:

\[t=\frac{\bar{err}(M_{1})-\bar{err}(M_{2})}{\sqrt{var(M_{1}-M_{2})/k}}\]

\[var(M_{1}-M_{2})=\frac{1}{k}\sum_{i=1}^{k}\left [err(M_{1})_{i} - err(M_{2})_{i} -(\bar{err}(M_{1})-\bar{err}(M_{2}))\right ]^{2}\]

  • If two test sets available: use non-paired t-test

\[var(M_{1}-M_{2})=\sqrt{\frac{var(M_{1})}{k_{1}}+\frac{var(M_{2})}{k_{2}}}\]

where k1 & k2 are # of cross-validation samples used for M1 & M2, resp.


Speaker notes:

Content Tools

Sources

There are currently no sources for this slide.