Introduction

This post is to summarize the 2016 results to date (round 05) and compare this to the bookmakers predictions. Considering that round 6 is now going to be my first betting round (in lui of round 5 - purely due to the minimal time I have had to work on this), I thought it important to track how we are going to date. I am also going to discuss the results of a model re-run since the results of round 4 and 5 were less than favorable.

Summary of results to date (r05)

We observe from the summary table/s and figures below that:

• overall model accuracy/precision to date is 56%

• overall bookmaker accuracy/precision to date is 66%

• the model had higher accuracy than the bookmakers in rounds 2 and 3

• the models round 4 and 5 accuracy was below 50%

• Bookmaker precision was very poor (<30%) in round 3

• In round 5, there were 6 away team wins vs 2 home team wins

• In round 4 there were and even number of home and away team wins (4 home and 4 away)

• In all other rounds the number of home team wins outweighed the number of away team wins

• poor accuracy in round 4 is largely attributed to predictions being based on round 2 results (without knowing round 3 results) and high away team wins (4)

• poor accuracy in round’s 5 is largely attributed to high number of away team wins (6)

• round 3 contained a drawn match which is historically rare (there have only been 8 drawn games since the inception of the golden point rule representing 0.5% of all games to date)

## [1] 0.564

## [1] 0.657

Discussion of results

We can see from the results that the model performed particularly poorly through rounds 4 and 5 as compared to the bookmaker. This dramatically affected overall precision which was 56% as compared to the bookmakers 66%. After reviewing the model, I determined that the main reason for poor performance through rounds 4 and 5 is due to the high weight of the importance of feature ‘home_away’ in the model. Because the model (in most circumstances) preferentially picks home teams to win, the model suffered from a dramatic increase in away teams winning through rounds 4-5.

We also observe poor accuracy from the bookmaker in round 3 where there were a number of very close games, and a draw. As the model was likely ignoring additional feature the bookmaker was relying on in round 3 (and using home/away as a highly weighted feature) the model performed well by comparison.

Generally, I would have expected model accuracy to be increasing as rounds progress (based on historic analysis), however the importance of home/away in the model was such that it overwhelmed the importance of other features such as summary of results to date, previous match ups etc. Re-running the model with less weight (less feature importance) on home/away confirms that accuracy would be higher by boosting the importance of other features. We have seen that in 2015, the importance of home away teams was much lower than historic (2009-2014) and 2016 may turn out to be similar in this regard.

Re-modelling

Since I didn’t get an opportunity to use round 3 results for round 4 predictions I firstly wanted to re-model this to determine if round 4 accuracy would have been improved. Secondly, because I am generally not satisfied with the average precision to date, and the degradation in precision (we should actually be improving in precision) I wanted to re-model all rounds with an improved predicative model. Specifically, I am concerned that the original model/s place too much significance on the home/away feature and that (like in 2015) we may see reduced significance of this feature in 2016. Therefore I chose to ‘de-weight’ this feature so that it was not an overwhelming predictive feature in the model/s. As you will see, the new model improved average precision to date, generally improved precision across all rounds as compared to the current model, and had slightly better overall precision than the bookmaker

Financial modelling

Finally, I wanted to determine how we would have went using various staking strategies to bet on predictions to date. As stated previously one of the main reasons for choosing to wait until round 5 or 6 to place bets was due to historically poor (50-60%) model performance in rounds 1-5. So it is worth finding out if we would have lost money betting in these rounds this year. I was also interested to know if my preferred staking strategy would have mitigated losses (avoided losing bets). i am going to keep it relatively simple and just compare a fixed staking strategy to my preferred staking strategy.

Conclusion

I have shown that my strategy to wait until round 6 to make a bet was a good choice, as the original model suffered from low overall accuracy (56% to date). Despite this I have also shown that my preferred staking strategy had high accuracy (71%) and was able to mitigate most losses, however would have only broke even in terms of profit. I have also shown that my latest model would have produced much better results through rounds 1-5 (67.5% accuracy) and we would have made a profit no matter which staking strategy used. I am going to utilize the latest model for round 6 predictions and at this stage, I will utilize the risk adverse preferred staking strategy.

Appendix - Summary of all predictions to date

Beginnings

Hi all my name is Matthew Dick and like Kane I have an interest in data science and machine learning. Kane has been kind enough to allow me to use maxwellAi as platform to share some interesting stuff about the NRL stats that he has put together. This is the first article, but i’ve had a few ideas about concepts/theories I’d like to look into regarding the historical NRL data.
Hopefully some of this can be used to help the prediction model…

I took an interest the National Rugby League (NRL) Experiment a couple weeks ago and have been putting together some player data to join up with all the round by round data Kane had already collected.

The first chapter of this data analysis vignette is a look at the data collected from the NRL website, as well as from NRLStats and AFL tables

The Data

The dataset I am working from is still a CSV (comma separated variable file) that was created by joining several thousand individual CSVs from each team and round, for seasons 2005-2015.

A quick look at the variables…

str(nrl)

## 'data.frame':    72664 obs. of  66 variables:
##  $ X.2          : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ X.1          : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ X            : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ player_code  : Factor w/ 1171 levels "AaronCannings",..: 297 167 595 1007 237 320 1107 135 189 298 ...
##  $ P            : int  12 5 4 4 4 0 0 0 0 0 ...
##  $ T            : int  1 1 1 1 1 0 0 0 0 0 ...
##  $ G            : int  4 0 0 0 0 0 0 0 0 0 ...
##  $ AR           : int  4 4 8 10 14 8 10 14 3 9 ...
##  $ ARM          : int  27 24 82 77 130 62 88 128 49 76 ...
##  $ LB           : int  0 1 1 0 0 0 1 0 0 0 ...
##  $ TA           : int  0 1 0 0 0 0 0 0 0 0 ...
##  $ OL           : int  0 0 2 1 2 0 1 1 0 0 ...
##  $ TK           : int  17 14 2 18 31 40 30 17 14 19 ...
##  $ MT           : int  10 8 0 1 2 3 3 2 2 1 ...
##  $ E            : int  3 0 1 2 0 0 1 0 1 0 ...
##  $ K            : int  8 7 1 0 0 0 0 0 1 0 ...
##  $ LBA          : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ KM           : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ KR           : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ CN           : int  4 0 0 0 0 0 0 0 0 0 ...
##  $ FG           : int  0 1 0 0 0 0 0 0 0 0 ...
##  $ X40          : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ DH           : int  0 0 0 2 0 2 0 0 0 0 ...
##  $ S            : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ SF           : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ PenF         : int  1 0 0 0 0 1 0 1 0 0 ...
##  $ CM           : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ season       : int  2005 2005 2005 2005 2005 2005 2005 2005 2005 2005 ...
##  $ round        : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ club_name    : Factor w/ 16 levels "Broncos","Bulldogs",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ round_name   : Factor w/ 31 levels "1","10","11",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ date         : Factor w/ 1171 levels "1-Apr-05","1-Apr-06",..: 182 182 182 182 182 182 182 182 182 182 ...
##  $ venue        : Factor w/ 41 levels "Adelaide","Barlow",..: 17 17 17 17 17 17 17 17 17 17 ...
##  $ attendance   : int  43488 43488 43488 43488 43488 43488 43488 43488 43488 43488 ...
##  $ home_team    : Factor w/ 16 levels "Broncos","Bulldogs",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ home_away    : Factor w/ 2 levels "away","home": 2 2 2 2 2 2 2 2 2 2 ...
##  $ home_score   : int  29 29 29 29 29 29 29 29 29 29 ...
##  $ home_tries   : int  5 5 5 5 5 5 5 5 5 5 ...
##  $ home_goals   : int  4 4 4 4 4 4 4 4 4 4 ...
##  $ home_fg      : int  1 1 1 1 1 1 1 1 1 1 ...
##  $ away_team    : Factor w/ 16 levels "Broncos","Bulldogs",..: 3 3 3 3 3 3 3 3 3 3 ...
##  $ away_score   : int  16 16 16 16 16 16 16 16 16 16 ...
##  $ away_tries   : int  3 3 3 3 3 3 3 3 3 3 ...
##  $ away_goals   : int  2 2 2 2 2 2 2 2 2 2 ...
##  $ away_fg      : int  0 0 0 0 0 0 0 0 0 0 ...
##  $ referee1     : Factor w/ 29 levels "Adam Devcich",..: 27 27 27 27 27 27 27 27 27 27 ...
##  $ referee2     : Factor w/ 30 levels "","Adam Devcich",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ home_scrum   : int  10 10 10 10 10 10 10 10 10 10 ...
##  $ home_penalty : int  7 7 7 7 7 7 7 7 7 7 ...
##  $ away_scrum   : int  7 7 7 7 7 7 7 7 7 7 ...
##  $ away_penalty : int  4 4 4 4 4 4 4 4 4 4 ...
##  $ position     : Factor w/ 10 levels "BE","CN","FB",..: 4 5 3 2 9 9 7 1 1 1 ...
##  $ player_name  : Factor w/ 1170 levels "Aaron Cannings",..: 298 168 595 1006 238 321 1105 135 190 299 ...
##  $ player_code.y: Factor w/ 1170 levels "AaronCannings",..: 298 168 595 1006 238 321 1105 135 190 299 ...
##  $ first_name   : Factor w/ 441 levels "Aaron ","Adam ",..: 93 46 210 362 77 96 405 37 59 93 ...
##  $ surname      : Factor w/ 944 levels "Abraham","Achurch",..: 443 743 347 51 624 779 100 839 512 481 ...
##  $ team_code    : Factor w/ 16 levels "BR","CB","CN",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ tries        : int  1 1 1 1 1 0 0 0 0 0 ...
##  $ goals        : int  4 0 0 0 0 0 0 0 0 0 ...
##  $ goal_attempts: int  6 0 0 0 0 0 0 0 0 0 ...
##  $ field_goals  : int  0 1 0 0 0 0 0 0 0 0 ...
##  $ FPP          : num  35.7 33.4 26.2 35.7 56 46.2 44.8 31.8 18.9 26.6 ...
##  $ FPN          : int  28 16 2 6 4 8 8 6 6 2 ...
##  $ home_win     : logi  TRUE TRUE TRUE TRUE TRUE TRUE ...
##  $ FP           : num  7.7 17.4 24.2 29.7 52 38.2 36.8 25.8 12.9 24.6 ...
##  $ win          : logi  TRUE TRUE TRUE TRUE TRUE TRUE ...

Part 2: Penalties

One of the first stats that I thought might be interesting were the penalties against each player; recorded as the “PenF” variable.

Lets see how many penalties per year there were.

Woops; looks like something is up. Either the referees weren’t looking in 2013 and 2014, or there was an issue with the data source.

Unfortunately it looks like the original data on the NRL website shows the same information. Whilst there are other sources for the penalty data, most of this is on a by game basis…and I’d like to look a little deeper.

In the meantime we’ll just have to exclude 2013 and 2014 from our analyses, until someone can help us out with the missing data (any volunteers?).

Who is the most penalised player in our data set?

And on an average penalties per game basis.

Not very insightful…players with low game counts are going to potentially skew our numbers.

Lets filter by player with at least 10 games.

And last year?

The data seems to match some quick googling.

This article mentions the Gold Coast Titans as having a reputation as the “bad-boys” of the NRL.

“It is a concern,” Henry said of the Gold Coast’s discipline.

“We were down 6-3 (in the penalty count) at half-time (against Parramatta) and talked about avoidable penalties around the ruck.”

“We’ll be looking to reduce that.”

Let's have a look at the overall average penalties per game for each club in the 2015 season, to see where the Titans sit.

Hmmm - nothing so far

They don’t seem to stand out here.

The article is dated April, which is fairly early in the season. It is possible that the Titans had a higher penalty count in the early rounds of the season. Assuming that Neil Henry had a chat to his players after round 8, we’ll look at the average penalties in rounds 1-8, and 9-26 as two separate samples.

We’ll have to exclude the byes from rounds 8 to 26 so as to not unfairly lower the average penalties for the “post chewing out” period.

Looking promising…

The mean penalties for the two parts of the season certain appear to correlate to our assumption

We can do a t-test to determine if the difference in the means is statistically significant. To do this, we have to rearrange the data a little bit.

tmp = nrl %>% group_by(season, round, club_name) %>%
  summarise(games = length(unique(round)), pen_per_game = sum(PenF)/games) %>%
  filter(season == 2015, club_name == "Titans", pen_per_game != 0) %>%
  mutate(before_after = round > 8)

t.test(pen_per_game ~ before_after, data = tmp, var.equal=T)

## 
##  Two Sample t-test
## 
## data:  pen_per_game by before_after
## t = 2.491, df = 22, p-value = 0.02077
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2302531 2.5197469
## sample estimates:
## mean in group FALSE  mean in group TRUE 
##               8.000               6.625

T-test interpretation

The t-test p-value of 0.02 is less than 0.05, so we reject the NULL hypothesis and conclude that the difference the means is significant.

The Titans lowered their average penalties per game after round 8 of 2015. Whether this is due to Neil Henry’s coaching, I’ll leave to the reader.