Two sample homogeneity tests: tennis dataset

To go back to the Home page, click here.

1 Data Description

The dataset contains the results for the men’s ATP tour date back to January 2000, including Grand Slams, Masters Series, Masters Cup and International Series competitions. All head to head matches are provided with rank of both opponents, the name of the winner, and the final score. A more detailled description of the data can be found here.

The data can be found here.

In this notebook, we will only be focused on the so-called BIG 3 that corresponds to the three best tennis players over the last 20 years, namely Roger Federer, Rafael Nadal and Novak Djokovic.

2 Study of head to head in the BIG 3

Players in the BIG 3 have really different game styles. In this first section, we want to test for each member of the BIG 3 if he has the same chances to win against one or the other of its greatest rivals.

# Selecting Head to Head matches
big3 =  c('Federer R.','Nadal R.', 'Djokovic N.')
FvsN = ('Federer R.' == data$Winner[(data$Winner %in% big3[-3]) & (data$Loser %in% big3[-3])]);
FvsD = ('Federer R.' == data$Winner[(data$Winner %in% big3[-2]) & (data$Loser %in% big3[-2])]);
NvsD = ('Nadal R.' == data$Winner[(data$Winner %in% big3[-1]) & (data$Loser %in% big3[-1])]);

We will use a Z-Test to carry out our study.

2.1 Power comparison between the asymptotic Z-test and the exact Fisher test

The following functions compute the p-values resulting from the Fisher Test and the asymptotic Z-test.

# Exact Fisher Test
FisherTestStat <- function (seq1,seq2){
  contingency = matrix(ncol=2,nrow=2);
  contingency[1,1] = sum(seq1);
  contingency[2,1] = length(seq1)-contingency[1,1];
  contingency[1,2] = sum(seq2);
  contingency[2,2] = length(seq2)-contingency[1,2];
  Zstat <- fisher.test(contingency);
  return (Zstat$p.value);
}

# Asymptotic Z-Test
computeAsymptoticZstat <- function (seq1,seq2) {
  n1 = length(seq1);
  n2 = length(seq2);
  bar1 = mean(seq1);
  bar2 = mean(seq2);
  bar = (sum(seq1)+sum(seq2))/(n1+n2);
  Zstat = sqrt(n1*n2/(n1+n2))*(bar1-bar2)/sqrt(bar*(1-bar));
  pvalue = 2*pnorm(-abs(Zstat),mean=0,sd=1);
  return (data.frame(stat=Zstat, pval=pvalue));
}

In our case, we will deal with sample sizes around 40. We propose to study the power of both tests for sample size of this order.

Let us first check that the above functions are working correctly by plotting the proportion of Type I errors for tests with level 0.05.

# Proportion of Type I error for both tests depending on the sample size (using a level = 0.05)
level = 0.05;
i = 1;
list_n <- seq(20,200,by=10);
fisher <- rep(0,length(list_n));

asymptoticZtest <- rep(0,length(list_n));
for (n in list_n){
  for (k in 1:1000){
      seq1 = rbinom(n,1,0.5);
      seq2 = rbinom(n,1,0.5);
      asymptoticZtest[i] = asymptoticZtest[i] + 1*(computeAsymptoticZstat(seq1,seq2)$pval < level)/1000;
      fisher[i] = fisher[i] + 1*(FisherTestStat(seq1,seq2) < level)/1000;
    }
  i = i+1;
}
errorstype1 <- data.frame(list_n,fisher,asymptoticZtest);
ggplot(errorstype1, aes(list_n)) +                    # basic graphical object
  geom_line(aes(y=fisher,  colour="Fisher test")) +  # first layer
  geom_line(aes(y=asymptoticZtest,  colour = "Asymptotic Z test")) + # second layer
  scale_color_manual(name = "", values = c("Asymptotic Z test" = "green", "Fisher test" = "red")) +
  labs(x = "Sample size") +
  labs(y = "Proportion of Type I error")

As theoretically guaranteed, the Fisher leads to a proportion of Type I errors smaller than 0.05. Indeed, it is the opportunity to recall that Fisher test only provides an upper bound on the level of the test.

Regarding the asymptotic Z-test, the proportion of Type I errors is around 0.05 but may slightly above or below this value due to randomness but also due to the fact that the theoretical control of the level of the test holds only asymptotically.

Let us now dig into the power study. We consider two sequences of Bernoulli random variables with respective parameters p1 and p2 with 0.5-p1=p2-0.5.

level = 0.05;
list_n = seq(10,100,by=10);
list_gaps = seq(0.,0.2,by=0.02);
power_asymptoticZtest = matrix(0,ncol=length(list_n),nrow=length(list_gaps));
power_fisher = matrix(0,ncol=length(list_n),nrow=length(list_gaps));
i = 1;
for (n in list_n){
  j = 1;
  for (gap in list_gaps){
    p1 = 0.5-gap;
    p2 = 0.5+gap;
    for (k in 1:300){
      seq1 = rbinom(n,1,p1);
      seq2 = rbinom(n,1,p2);
      
      power_asymptoticZtest[j,i] = power_asymptoticZtest[j,i] + 1*(computeAsymptoticZstat(seq1,seq2)$pval < level)/300;
      power_fisher[j,i] = power_fisher[j,i] + 1*(FisherTestStat(seq1,seq2) < level)/300;
    }
    j = j+1;
  }
  i = i+1;
}

heatmap <- expand.grid(X=list_gaps, Y=list_n);
heatmap$Z <- c(power_asymptoticZtest - power_fisher);

## Try it out
levelplot(Z ~ X*Y, data=heatmap  ,xlab="p2-p1", ylab="Number of samples: n",
          main="Power(Asymptotic Z test) - Power(Fisher Test)")

The heatmap shows that the power of asymptotic Z test is greater than the one of the Fisher test in the range of sample size of interest. Hence, in the following, we will use the asymptotic Z test.

2.2 Study for Federer

# We compute the p-value of the T-Test for Federer
resZtest <- computeAsymptoticZstat(FvsN,FvsD);
resZtest$pval

## [1] 0.1994286

The p-value is equal to 0.2 meaning that we do not reject the null hypothesis. Hence, the probability of victory of Federer over Nadal is not statistically different from the one of Federer over Djokovic.

2.3 Study for Nadal

# We compute the p-value of the T-Test for Nadal
resZtest <- computeAsymptoticZstat(1-FvsN,NvsD);
resZtest$pval;

## [1] 0.04057708

The p-value is equal to 0.04 meaning that we would tend to reject the null hypothesis. Hence, the probability of victory of Nadal over Federer is statistically different (in this case: larger) from the one of Nadal over Djokovic.

2.4 Study for Djokovic

# We compute the p-value of the T-Test for Djokovic
resZtest <- computeAsymptoticZstat(1-FvsD,1-NvsD);
resZtest$pval;

## [1] 0.8484067

The p-value is equal to 0.85 meaning that we do not reject the null hypothesis. Hence, the probability of victory of Djokovic over Federer is not statistically different from the one of Djokovic over Nadal.

Conclusion While Nadal and Djokovic have a probability of victory against the other members of the Big 3 that does not depend on the opponent, we showed that Nadal has significantly less chance of winning against Djokovic than against Federer.

Now we want to understand why Nadal performs better when facing Federer than Djokovic. In the following, we will be focused on the following questions.

1. Does this fact holds for every surfaces?

2. Was it true during the whole Nadal’s career?

We first investigate the influence of the surface.

3 Influence of the Surface

We take a closer look to matches between Nadal and tho others members of the BIG 3 depending on the different surfaces. We only focus on Clay and Hard Court since we do not have enough data for Carpet and Grass.

The R function presented below returns:

1. the probability of victory of Nadal over Federer or Djokovic on both surfaces

2. the p-value of the T-test of the previous section conducted on both surfaces separately

Nadal_study <- function(df){
    list_surfaces = c("Clay","Hard");
    pvals = list();
    probasNadalvictory = data.frame(matrix(ncol=length(list_surfaces),nrow=2));
    colnames(probasNadalvictory) <- list_surfaces;
    rownames(probasNadalvictory) <- c('Federer R.', 'Djokovic N.');
    for (surface in list_surfaces){
      FvsNsurface = ('Federer R.' == df$Winner[(df$Winner %in% big3[-3]) & (df$Loser %in% big3[-3]) & (df$Surface == surface)]);
      NvsDsurface = ('Nadal R.' == df$Winner[(df$Winner %in% big3[-1]) & (df$Loser %in% big3[-1]) & (df$Surface == surface)]);
      resZtest <- computeAsymptoticZstat(1-FvsNsurface,NvsDsurface);
      pvals = append(pvals, resZtest$pval);
      probasNadalvictory['Federer R.',surface] <- mean(1-FvsNsurface);
      probasNadalvictory['Djokovic N.',surface] <- mean(NvsDsurface);
    }
    names(pvals) <- list_surfaces;
    return (list(pvals=pvals,probasvictory=probasNadalvictory));
}

Nadal_res <- Nadal_study(data);
Nadal_res$pvals

## $Clay
## [1] 0.1466762
## 
## $Hard
## [1] 0.04550026

Nadal_res$probasvictory

##                  Clay   Hard
## Federer R.  0.8666667 0.5625
## Djokovic N. 0.6500000 0.2500

With a p-value=0.15 for Clay, we conclude that the difference between the probabilities of victory of Nadal against Federer or Djokovic are not statistically different (Nadal is known to be the King of Clay ;) Hence he often comes out with a victory on this surface regardless of the opponent).

However, on Hard courts the p-value is 0.045 and we can conclude that Nadal has greater chances to win against Federer rather than Djokovic.

4 Studying Nadal matches over time

Based on the previous section, we will still consider separately the different surfaces: Clay and Hard courts. We do the same tests as previously but focusing on matches played before 2010 or after 2010.

4.1 a) Before 2010

dates <- apply(select(data, Date), 1, function(x){ as.integer(substr(x,nchar(x)-3,nchar(x)));});
Nadal_res <- Nadal_study(data[dates <= 2010,]);
Nadal_res$pvals

## $Clay
## [1] 0.2235429
## 
## $Hard
## [1] 0.7829351

Nadal_res$probasvictory

##                  Clay      Hard
## Federer R.  0.8333333 0.4285714
## Djokovic N. 1.0000000 0.3636364

We do not reject the null hypothesis for both Clay and Hard courts working with matches played before 2010. Hence, for matches played before 2010, the difference in probability of victory of Nadal over Federer or Djokovic is not statistically significant.

4.2 b) After 2010

Nadal_res <- Nadal_study(data[dates > 2010,]);
Nadal_res$pvals

## $Clay
## [1] 0.07007554
## 
## $Hard
## [1] 0.01395398

Nadal_res$probasvictory

##                  Clay      Hard
## Federer R.  1.0000000 0.6666667
## Djokovic N. 0.4166667 0.1538462

Contrary to the previous paragraph, we again reach the conclusion that Nadal has significantly more chances to win over Federer than Djokovic on Hard courts if we focus only on matches played after 2010. Moreover, the p-value on Clay for matches played after 2010 is equal to 0.07, which is smaller compared to the previous section. If this result may be too weak to reject the null with conviction, it could lead us to think that the older Nadal was getting, the better it is for him to face Federer rather than Djokovic (regardless of the surface).

5 Conclusion

A distracted reader could think that the main reason explaining the results of this study is the age of Federer. One could easily think that after 2010, Federer was still a high level player but found difficulties to compete with the other members of the BIG 3.

But if it was the case, we should also find a difference between the probability of victory of Djokovic over Nadal or Federer for matches after 2010, which is not the case has shown below.

Djokovic_study <- function(df){
    list_surfaces = c("Clay","Hard");
    pvals = list();
    probasDjokovicvictory = data.frame(matrix(ncol=length(list_surfaces),nrow=2));
    colnames(probasDjokovicvictory) <- list_surfaces;
    rownames(probasDjokovicvictory) <- c('Federer R.', 'Nadal R.');
    for (surface in list_surfaces){
      DvsNsurface = ('Djokovic N.' == df$Winner[(df$Winner %in% big3[-1]) & (df$Loser %in% big3[-1]) & (df$Surface == surface)]);
      DvsFsurface = ('Djokovic N.' == df$Winner[(df$Winner %in% big3[-2]) & (df$Loser %in% big3[-2]) & (df$Surface == surface)]);
      resZtest <- computeAsymptoticZstat(DvsNsurface,DvsFsurface);
      pvals = append(pvals, resZtest$pval);
      probasDjokovicvictory['Federer R.',surface] <- mean(DvsFsurface);
      probasDjokovicvictory['Nadal R.',surface] <- mean(DvsNsurface);
    }
    names(pvals) <- list_surfaces;
    return (list(pvals=pvals,probasvictory=probasDjokovicvictory));
}

Djokovic_res <- Djokovic_study(data[dates > 2010,]);
Djokovic_res$pvals

## $Clay
## [1] 0.9492721
## 
## $Hard
## [1] 0.2987821

Hence saying that Federer level decayed after 2010 would be a hasty conclusion.

From my personal point of view (and as a tennis fan), the fact that Nadal had better chances to win against Federer rather than against Djokovic was mainly due to a psychological ascendant he hed on the Swiss. But everyone will make his own opinion !