What the ICC statistics calculator does
The intraclass correlation coefficient (ICC) measures how strongly units in the same group resemble each other. In practice it quantifies the reliability of ratings or repeated measurements: how much of the total variation is due to real differences between subjects rather than measurement noise. This tool computes the one-way random effects, single-measures form, \(\text{ICC}(1,1)\), from a simple ratings matrix.
How to use it
Enter one subject (target) per line, and separate each rater's score or repeated measurement with spaces or commas. Every row must contain the same number of columns \(k\). The calculator returns the ICC along with the number of subjects \(n\), the number of raters \(k\), and the between- and within-subjects mean squares.
The formula explained
The ICC is built from an analysis of variance. With \(n\) subjects and \(k\) measurements each, the display formula is:
$$\text{ICC}(1,1) = \frac{MS_B - MS_W}{MS_B + (k-1)\,MS_W}$$
Here \(MS_B = \frac{SS_B}{n-1}\) is the between-subjects mean square and \(MS_W = \frac{SS_W}{n(k-1)}\) is the within-subjects mean square. Values near \(1\) indicate excellent agreement, while values near \(0\) indicate poor reliability.
Worked example
Take three subjects rated by two raters: \((1,2)\), \((3,4)\), \((5,6)\). The grand mean is \(3.5\). This gives \(MS_B = 8\) and \(MS_W = 0.5\), so:
$$\text{ICC} = \frac{8 - 0.5}{8 + (2-1)\times 0.5} = \frac{7.5}{8.5} \approx 0.882$$
An ICC of \(0.88\) reflects good to excellent reliability between the two raters.
How to Calculate ICC by Hand
The formula below is the ICC(1,1) — the single-measures coefficient from a one-way random-effects model. It is a general statistical method (not tied to any country or national standard) and is widely used in reliability and agreement studies. We work through a small example with three subjects each rated by two raters: the ratings are \((1,2)\), \((3,4)\) and \((5,6)\), so \(n = 3\) subjects and \(k = 2\) raters.
-
Arrange the matrix and find the grand mean. Lay out the data with one subject per row and one rater per column, then average all \(n\cdot k\) values.
$$\bar{x} = \frac{1+2+3+4+5+6}{6} = \frac{21}{6} = 3.5$$ -
Compute each subject's row mean.
$$\bar{x}_1 = \frac{1+2}{2} = 1.5,\qquad \bar{x}_2 = \frac{3+4}{2} = 3.5,\qquad \bar{x}_3 = \frac{5+6}{2} = 5.5$$ -
Between-subjects sum of squares \(\text{SS}_B = k\sum(\bar{x}_i - \bar{x})^2\):
$$\text{SS}_B = 2\left[(1.5-3.5)^2 + (3.5-3.5)^2 + (5.5-3.5)^2\right] = 2\left[4 + 0 + 4\right] = 16$$ -
Within-subjects sum of squares \(\text{SS}_W = \sum\sum (x_{ij} - \bar{x}_i)^2\), summing squared deviations of each value from its own row mean:
$$\text{SS}_W = \underbrace{(0.25+0.25)}_{\text{row }1} + \underbrace{(0.25+0.25)}_{\text{row }2} + \underbrace{(0.25+0.25)}_{\text{row }3} = 1.5$$ -
Divide by degrees of freedom to get the mean squares, using \(df_B = n-1\) and \(df_W = n(k-1)\):
$$\text{MS}_B = \frac{\text{SS}_B}{n-1} = \frac{16}{3-1} = 8,\qquad \text{MS}_W = \frac{\text{SS}_W}{n(k-1)} = \frac{1.5}{3(2-1)} = 0.5$$ -
Substitute into the ICC(1,1) formula:
$$\text{ICC}(1,1) = \frac{\text{MS}_B - \text{MS}_W}{\text{MS}_B + (k-1)\,\text{MS}_W} = \frac{8 - 0.5}{8 + (2-1)(0.5)} = \frac{7.5}{8.5} \approx 0.882$$
For this data set the intraclass correlation coefficient is 0.882, indicating strong agreement — the variability between subjects greatly exceeds the variability within a subject's repeated ratings.
Key Terms and Variables
- Intraclass correlation coefficient (ICC) — a value (typically between 0 and 1) that quantifies how strongly measurements within the same group resemble each other; used to assess agreement and reliability across raters or repeated measurements. Higher values mean greater consistency.
- Subject / target — the object being measured (a person, sample, or item). In the ratings matrix each subject is one row.
- Rater — the person, instrument, or measurement occasion that produces a rating. Each rater (or repeated measurement) is one column of the matrix.
- \(n\) (number of subjects) — the count of rows in the matrix, i.e. how many distinct targets were rated. Its degrees of freedom is \(n-1\).
- \(k\) (number of raters / measurements) — the count of columns, i.e. how many ratings each subject received.
- \(\text{SS}_B\) and \(\text{MS}_B\) (between-subjects) — \(\text{SS}_B = k\sum(\bar{x}_i-\bar{x})^2\) measures how much the subjects' row means differ from the grand mean; the between-subjects mean square \(\text{MS}_B = \text{SS}_B/(n-1)\) captures true differences among subjects.
- \(\text{SS}_W\) and \(\text{MS}_W\) (within-subjects) — \(\text{SS}_W = \sum\sum(x_{ij}-\bar{x}_i)^2\) measures spread of a subject's ratings around its own mean; the within-subjects mean square \(\text{MS}_W = \text{SS}_W/[n(k-1)]\) reflects measurement error or rater disagreement.
- One-way random-effects model — the model behind ICC(1,1): each subject is rated by a (possibly different) set of raters drawn at random, and only the subject effect is modeled. It answers "how reliable is a single rating from a randomly chosen rater?"
- Single-measures vs average-measures — single-measures ICC estimates the reliability of one individual rating, while average-measures ICC estimates the reliability of the mean of all \(k\) ratings. The average-measures value is always higher because averaging reduces error.
- Agreement / reliability — reliability is the proportion of total variance due to true differences between subjects; agreement additionally requires that raters give the same absolute values, not merely rank subjects consistently. ICC(1,1) reflects absolute agreement under the one-way model.
FAQ
What is a good ICC value? Common guidance treats values below \(0.5\) as poor, \(0.5\)–\(0.75\) as moderate, \(0.75\)–\(0.9\) as good, and above \(0.9\) as excellent.
Which ICC model is used here? This calculator reports the one-way random effects, single-measures model, \(\text{ICC}(1,1)\), suitable when each subject is rated by a different set of raters.
Can ICC be negative? Yes. When within-subject variation exceeds between-subject variation, the ICC can be slightly negative, which is usually interpreted as no agreement.