Concordance: the c-statistic, Somers' D, Gini and Kendall's tauOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Concordance: the c-statistic, Somers' D, Gini and Kendall's tau

A scorecard validation pack, a survival model's summary table and a scikit-learn roc_auc_score call are all reporting the same quantity under four different names. The quantity is concordance: given one case and one control, how often does the model score the case higher?

Concordance is worth deriving separately from the ROC curve because the pair-counting definition makes three things obvious that the curve hides: what happens to tied scores, why Somers' D and Gini are the same number, and why binning a score into grades destroys measurable discrimination.

Pairs, not thresholds

Take a binary outcome yi∈{0,1}y_i \in \{0, 1\} and a score sis_i. Form every pair consisting of one positive ii (with yi=1y_i = 1) and one negative jj. With n+n_+ positives and n−n_- negatives there are n+n−n_+ n_- such pairs, and each one is exactly one of three things:

PairConditionMeaning
Concordant (CC)si>sjs_i > s_jthe model ranked the positive above the negative, correctly
Discordant (DD)si<sjs_i < s_jthe model got this pair backwards
Tied (TT)si=sjs_i = s_jthe model expressed no preference

C+D+T=n+n−C + D + T = n_+ n_- by construction. Every statistic on this page is a different way of dividing those three counts.

The worked example

Eight loan applicants, scored and then observed. Three defaulted (the positives), five repaid.

ApplicantScoreOutcome
A0.82default
B0.71repaid
C0.65default
D0.65repaid
E0.50default
F0.44repaid
G0.30repaid
H0.12repaid

There are 3×5=153 \times 5 = 15 pairs. Going default by default:

  • A (0.82) beats B, D, F, G, H: 5 concordant.
  • C (0.65) loses to B (0.71), ties with D (0.65), beats F, G, H: 3 concordant, 1 discordant, 1 tied.
  • E (0.50) loses to B and D, beats F, G, H: 3 concordant, 2 discordant.
C=11,D=3,T=1,C+D+T=15C = 11, \qquad D = 3, \qquad T = 1, \qquad C + D + T = 15

Concordant pairs — the c-statistic, Somers' D and gamma from one double loop

1/7

Eight loan applicants, scored by a model and then observed: three defaulted, five repaid. A threshold metric would make you pick a cutoff before you can say anything at all. Concordance asks a question with no cutoff in it: take one default and one repayer, does the model score the default higher?

python
1def concordance(scores, labels):
2 pos = [s for s, y in zip(scores, labels) if y]
3 neg = [s for s, y in zip(scores, labels) if not y]
4 C = D = T = 0
5 for p in pos:
6 for q in neg:
7 if p > q:
8 C += 1 # concordant
9 elif p < q:
10 D += 1 # discordant
11 else:
12 T += 1 # tied
13 total = C + D + T
14 c = (C + 0.5 * T) / total
15 somers_d = (C - D) / total
16 gamma = (C - D) / (C + D)
17 return c, somers_d, gamma
state
all eight applicants, ranked by score
A 0.82default
B 0.71repaid
C 0.65default
D 0.65repaid
E 0.50default
F 0.44repaid
G 0.30repaid
H 0.12repaid
the two groups
defaults (n₊) → 3repaid (n₋) → 5pairs to judge → 3 × 5 = 15

Only the ordering matters. Any monotone transform of the scores — a log-odds shift, a 0–1000 rescale, a calibration map — leaves every verdict below unchanged.

Every (default, repayer) pair on the eight-applicant example judged one positive at a time: 11 concordant, 3 discordant, 1 tied. Watch the four statistics fall out of those three counts — c = 0.7667 is the AUC, Somers' D = 0.5333 is 2c − 1. The payoff collapses the same scores onto a three-grade scorecard: ties go 1 → 5, c and Somers' D fall, and gamma rises, because it deletes the ties instead of paying for them.

What to notice while stepping through it:

  • The counters only ever see comparisons, never the scores themselves. Replace every score with its rank and nothing changes.
  • The tie between C and D is the only pair the model refuses to judge, and the four statistics below disagree entirely about what to do with it.
  • The payoff frame changes nothing but the score's resolution and three of the four statistics move, one of them in the wrong direction.

The four statistics

The c-statistic (= AUC)

Score a tie as half a win, which is the natural convention if you imagine breaking ties by a coin flip:

c=C+12TC+D+T=11+0.515=0.7667c = \frac{C + \tfrac{1}{2} T}{C + D + T} = \frac{11 + 0.5}{15} = 0.7667

This is identically the area under the ROC curve. The ROC page derives the same number by sweeping a threshold; the two routes agree because the Wilcoxon–Mann–Whitney statistic is the trapezoid area. The advantage of the pair form is interpretive: cc is a probability, P(si>sj)+12P(si=sj)P(s_i > s_j) + \tfrac{1}{2}P(s_i = s_j) for a random case–control pair, with 0.5 meaning a coin flip and 1.0 meaning perfect separation.

Somers' D and the Gini coefficient

Dxy=C−DC+D+T=11−315=0.5333D_{xy} = \frac{C - D}{C + D + T} = \frac{11 - 3}{15} = 0.5333

Ties contribute nothing to the numerator and still count in the denominator, so a tie is scored as neither a win nor a loss. Substituting C+D+T=n+n−C + D + T = n_+n_- and T=n+n−−C−DT = n_+n_- - C - D into the definition of cc gives the identity every credit-risk report leans on:

Dxy=2c−1D_{xy} = 2c - 1

Check: 2×0.7667−1=0.53332 \times 0.7667 - 1 = 0.5333. The Gini coefficient quoted for a scorecard is this same 2⋅AUC−12 \cdot \text{AUC} - 1, so "Gini 53%", "Somers' D 0.53" and "AUC 0.767" are three ways of saying one thing. The rescaling is useful only because it puts a coin-flip model at 0 instead of 0.5.

Two Ginis, unrelated

The Gini coefficient of a classifier (2⋅AUC−12 \cdot \text{AUC} - 1) has nothing to do with Gini impurity, the splitting criterion in a decision tree. They share a name and an ancestor in the economics literature, nothing else. Say "Gini coefficient" or "Somers' D" when you mean rank quality.

Kendall's tau-a

Tau-a uses the same numerator but divides by all (n2)\binom{n}{2} pairs of observations, including the case–case and control–control pairs that a binary outcome cannot order:

τa=C−Dn(n−1)/2=828=0.2857\tau_a = \frac{C - D}{n(n-1)/2} = \frac{8}{28} = 0.2857

With binary labels, (n+2)+(n−2)\binom{n_+}{2} + \binom{n_-}{2} pairs are unorderable by construction, so τa\tau_a cannot reach 1 even for a perfect model. It is the right statistic for two continuous rankings (a model's score against an expert's ranking, say), and the wrong one for a binary outcome. Reporting it there understates the model dramatically.

Goodman–Kruskal gamma

γ=C−DC+D=814=0.5714\gamma = \frac{C - D}{C + D} = \frac{8}{14} = 0.5714

Gamma deletes tied pairs from the denominator entirely. On a continuous score with no ties, γ=Dxy\gamma = D_{xy} and the choice is irrelevant. On a coarse score they diverge sharply, and gamma flatters the model.

StatisticFormulaValueTies count asRange on binary labels
c-statistic (AUC)(C+T/2)/(C+D+T)(C + T/2)/(C+D+T)0.7667half a win0.5 to 1
Somers' D (Gini)(C−D)/(C+D+T)(C - D)/(C+D+T)0.5333nothing0 to 1
Kendall's tau-a(C−D)/(n2)(C - D)/\binom{n}{2}0.2857nothing0 to <1<1
Goodman–Kruskal γ(C−D)/(C+D)(C - D)/(C+D)0.5714deleted0 to 1

What binning costs

Put the same eight applicants through a three-grade scorecard: High (≥0.65\ge 0.65), Mid (≥0.44\ge 0.44), Low (below). The ordering between grades survives; the ordering within a grade is destroyed, so every within-grade pair becomes a tie. Tied pairs go from 1 to 5, and:

StatisticRaw scoresThree gradesDirection
C/D/TC / D / T11 / 3 / 18 / 2 / 5ties ×5
c-statistic0.76670.7000falls, correctly
Somers' D0.53330.4000falls, correctly
Goodman–Kruskal γ0.57140.6000rises

The c-statistic and Somers' D charge the model for information it threw away. Gamma rewards it, because a pair it refuses to judge is removed from the denominator rather than counted against it. This is not a contrived edge case: risk grades, rounded scores, decile buckets and any tree-based model with few distinct leaf values all produce mass ties.

Compute concordance on raw scores

Always compute cc on the finest score the model produces, before grading or rounding. Then, if you also report the graded version, the gap between the two is a direct measurement of what the grading cost you. Quoting gamma instead hides exactly that gap.

The O(n log n) shortcut

The double loop is O(n+n−)O(n_+ n_-), which is 25 billion comparisons on a 100k-row test set with 5% positives. The rank-sum identity gives the same answer after a sort. Assign midranks rir_i to the pooled scores (tied observations share the average of the ranks they span), then:

c=R+−n+(n++1)2n+n−,R+=∑i: yi=1ric = \frac{R_+ - \frac{n_+(n_+ + 1)}{2}}{n_+ n_-}, \qquad R_+ = \sum_{i:\, y_i = 1} r_i

Verifying on the worked example. Pooled ranks, lowest score first, with C and D sharing ranks 5 and 6 as midrank 5.5:

Score0.120.300.440.500.650.650.710.82
ApplicantHGFEC/DC/DBA
Rank12345.55.578

The positives are E (4), C (5.5) and A (8), so R+=17.5R_+ = 17.5 and

c=17.5−3⋅423⋅5=17.5−615=11.515=0.7667c = \frac{17.5 - \frac{3 \cdot 4}{2}}{3 \cdot 5} = \frac{17.5 - 6}{15} = \frac{11.5}{15} = 0.7667

which matches the pair count exactly. The midrank convention is what makes ties come out as half-wins automatically. Sorting dominates, so the whole computation is O(nlog⁡n)O(n \log n) with O(n)O(n) memory.

Concordance beyond binary outcomes

Harrell's c-index extends the same counting to survival data, where the outcome is a time and some observations are censored. A pair is comparable only if you can tell who failed first: if subject ii failed at tit_i and subject jj was still under observation at tit_i, the pair counts; if jj was censored before tit_i, it does not. Concordance is then the concordant share of comparable pairs. Everything else carries over, including the tie convention and the interpretation as a probability. Heavy censoring shrinks the comparable set, so a c-index on 20%-observed data rests on far fewer pairs than its confidence interval suggests unless you use an inverse-probability-weighted variant (Uno's c).

Common mistakes

  • Reporting gamma as "the concordance". It is not the c-statistic unless the score has no ties, and it moves the wrong way when they appear.
  • Computing concordance on binned scores and comparing it to a competitor's raw-score number. You are comparing resolutions, not models.
  • Reading a high cc as good calibration. Concordance is invariant to every monotone transform of the score, so a model that ranks perfectly and predicts probabilities 100× too large scores c=1c = 1. Discrimination and calibration are separate audits.
  • Quoting cc with no interval. A c-statistic from 40 positives has a bootstrap standard error around 0.04, so a 0.02 gap between two models is noise until a paired test says otherwise. See comparing model performance for the paired way to test a difference.
  • Using tau-a on a binary outcome and concluding the model is weak, when the unorderable same-class pairs in the denominator cap it well below 1.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
Concordance: the c-statistic, Som…ROC Curve and AUC: Deriva…Is That Difference Real? …Decile Analysis: Gains, L…Classification & Regressi…Rank Ordering & Model Com…Decision TreesCalibration & Uncertainty