# Intraclass Correlation (ICC) Calculator

Calculate ICC (intraclass correlation coefficient) for inter-rater reliability. Get ICC(1,1), ICC(2,1), and ICC(3,1) from a rater-by-subject matrix.

## What this calculates

Enter ratings from multiple raters on multiple subjects to calculate three forms of the intraclass correlation coefficient. ICC measures how consistently raters assign similar scores to the same subjects.

## Inputs

- **Number of Subjects (n)** — min 2, max 50 — Number of subjects, items, or units being rated.
- **Number of Raters (k)** — min 2, max 20 — Number of raters or judges providing ratings.
- **Ratings (row per subject, comma-separated)** — Enter ratings row by row. Separate values with commas, rows with semicolons. Each row = one subject's ratings from all raters.

## Outputs

- **ICC(1,1) - One-Way Random** — Each subject rated by a different random set of raters.
- **ICC(2,1) - Two-Way Random, Single** — Random raters, single measure reliability.
- **ICC(3,1) - Two-Way Mixed, Single** — Fixed raters, single measure reliability (consistency).
- **Interpretation** — formatted as text — Qualitative assessment of reliability (based on ICC(2,1)).
- **MSR (Between Subjects)** — Mean square for rows (subjects).
- **MSC (Between Raters)** — Mean square for columns (raters).
- **MSE (Residual)** — Mean square error (residual).
- **Calculation** — formatted as text — ANOVA decomposition and ICC formulas.

## Details

**What is the ICC?**

The intraclass correlation coefficient (ICC) quantifies reliability or agreement between two or more raters. Unlike Pearson correlation, ICC accounts for both correlation and systematic differences between raters.

**ICC Types:**

- **ICC(1,1)** -- One-way random: Each subject is rated by a different random set of raters. Appropriate when raters are sampled from a larger population.
- **ICC(2,1)** -- Two-way random, single measures: Both subjects and raters are random effects. Measures absolute agreement.
- **ICC(3,1)** -- Two-way mixed, single measures: Subjects are random but raters are the only raters of interest. Measures consistency (systematic differences between raters are ignored).

**Formulas (from two-way ANOVA):**

- ICC(1,1) = (MSR - MSW) / (MSR + (k-1) x MSW)
- ICC(2,1) = (MSR - MSE) / (MSR + (k-1) x MSE + (k/n) x (MSC - MSE))
- ICC(3,1) = (MSR - MSE) / (MSR + (k-1) x MSE)

Where MSR = mean square between subjects, MSC = mean square between raters, MSE = residual mean square.

**Interpretation (Cicchetti, 1994):**

| ICC | Reliability |
|-----|-------------|
| < 0.40 | Poor |
| 0.40 - 0.59 | Fair |
| 0.60 - 0.74 | Good |
| 0.75 - 1.00 | Excellent |

## Frequently Asked Questions

**Q: Which ICC form should I use?**

A: Use ICC(1,1) when each subject is rated by a different random group of raters. Use ICC(2,1) when the same raters rate all subjects and you want to generalize to other raters (absolute agreement). Use ICC(3,1) when the same fixed set of raters rate all subjects and you care about consistency but not absolute agreement. In most research scenarios, ICC(2,1) or ICC(3,1) is appropriate.

**Q: What is the difference between agreement and consistency?**

A: Agreement (ICC(2,1)) requires raters to assign the same actual score to each subject. Consistency (ICC(3,1)) only requires raters to rank subjects in the same order, even if one rater systematically scores higher. If Rater A scores 2,4,6 and Rater B scores 5,7,9, consistency is perfect but agreement is poor because the values differ.

**Q: Can ICC be negative?**

A: Yes. A negative ICC means the variability within subjects (between raters) exceeds the variability between subjects. This indicates extremely poor reliability, where raters disagree more than random chance would predict. It can also happen with very small sample sizes or data entry errors.

---

Source: https://vastcalc.com/calculators/statistics/intraclass-correlation
Category: Statistics
Last updated: 2026-04-08