I have no opener here. No paragraph where I mention some music I like to listen to and then go on to make a connection between that piece of music and my thoughts. Such as why the Koscielny-Mustafi partnership is like Philip Glass’ Akhnaten.
I went and did something I have been avoiding all season: I watched the Trump-Clinton debate. The only stat I have from that debate is that we didn’t have a policy question until the 30th minute. By that time I was chugging wine. By the end of the debate I was clutching my debate partner like Arsene Wenger hugging Pat Rice. I think she was sitting there staring at the TV, her mouth agog. I was burying my head. I can’t take this election. I can’t.
I would love it if the next debate had the following rules: you get 30 seconds to answer the question and you cannot mention the opponent. It would be like couples therapy where both debaters can only use “I phrases.” For example, “I will solve the problem with Social Security by eliminating the $110k cap” and “I will build a glorious wall, the best wall and I will make Mexico pay for it.” It would be like couples therapy for our nation.
I guess this is my opener. I guess this is my culturally relevant segue into talking about Arsenal.
I had a look at the stats for Arsenal this season. I have been avoiding this because I was traumatized by the stats from last season. If you recall, I was big on Arsenal’s chances to win the League last season because all of the stats seemed to indicate that Arsenal should win the League.
For the first few months, Arsenal were creating so many golden chances, more than any other team, and we had the best keeper in the League, saving more big chances than any other. I kept telling you, dear reader, that Arsenal were just on the verge of breaking out. That Arsenal were doing everything right, that Arsenal’s problem wasn’t chance creation, it was finishing, and that finishing is temperamental, it swings with peaks and valleys. Just as soon as Arsenal start finishing all these chances, man, then we are really going to win the League.
You may also remember that I often pointed to Leicester as pretenders to the throne. Their finishing was simply ridiculous and the number of shots they were allowing was among the worst in the League. They were relying on blocking shots and hustling defense. They also needed teams to attack them, to create space in behind so that Vardy and Mahrez could counter. It seemed unsustainable.
It turned out that Arsenal’s attack was unsustainable and Leicester’s counter-intuitive approach was enough to win the League. Enough… They won the League by 10 points. Arsenal finished the season scoring four goals below their “expected goals” and Leicester, thanks largely to 10 penalties, finished the season scoring eight goals above their expected goals.
So, I went and looked at the stats for this year (thanks to blog reader Scott Willis for compiling this data) and sure enough, it looks like Arsenal are the ones now who are distinctly average in terms of shot creation but are the ones finishing at the highest rate in the League.
Scott calls it Chance Quality, which I prefer to expected goals, because that is what we are actually measuring, the quality of a shot and not our expectation of a goal. But regardless of what you call it, Man City, no surprise, are the best in the League. They may even be the best team in Europe at the moment, in terms of Chance Quality. Man U is right there behind them, buoyed by the Zlatan. And Arsenal are all the way down below Everton in 7th place in terms of Chance Quality.
In real terms, Arsenal have scored five more goals than their Chance Quality would indicate and Arsenal are doing that because they are converting 14% of their shots. 14% conversion isn’t unheard of under Arsene Wenger. His golden era team (the Invincibles) converted 18% of their shots in their prime (04/05). And Arsenal converted 13% in each of the three seasons between 2011 and 2014. Only Watford surpasses Arsenal in terms of goal efficiency.
But what does this mean? It just confirms what some of have observed: that Arsenal are clicking sometimes and other times not. Arsenal are just inconsistent at the moment. It also may indicate that only a certain level of chance creation is needed to be successful. Though that’s a bigger topic than I have energy to address today.
It also means that I’m not predicting ANYTHING. Certainly not the presidential election.
Just because the results didn’t match the predictions doesn’t mean your methods were wrong. Probability is just that, and not a prophecy that must come true. Most predictions tools, whether xG based or otherwise, saw what you saw last year predicted Arsenal to win the league due to their supremacy in the early season. It was a perfectly logical conclusion that this would eventually win out over Leicester’s flukey high finishing statistics. We could not have foreseen that Arsenal’s midfield would be so crucially awful after the injuries to CoqZorla, nor that Leicester would close the year with equally flukey high defensive statistics. Basically everything broke for them all in one season, and yes that includes Arsenal’s (and City’s) second half demise just as much as Chelsea’s unexpected total collapse, Man United’s ineptitude and Tottenham’s hilarious end of season collapse. There probably will never be another premier league season like that. You should not abandon statistical methods for the fear of being wrong. Simply acknowledge the limitations of the methods and that football is not played on spreadsheets… that’s the greatest allure of the sport after all: that despite what we all know SHOULD happen, anything CAN happen on any given match day.
I’m not abandoning my methods. I’m questioning them. That’s normal.
Sorry but if you say you’ve been avoiding stats because you were traumatized by last season’s outcomes, that makes it sound like much more than that.
That season was traumatic for me: I had such huge faith in the Arsenal to come good and then the wheels came off. Of course I have been looking at the stats but not in any depth. Don’t read too much into it.
All right, just don’t scare me like that 🙂
Also just want to point out that Trump is here because the American people put him here. He may be a rotten scoundrel but he won the primary fair and square. We The People could’ve chosen any number of more reasonable candidates. Democracy is great and fine but its constituents are under no moral obligations to make good choices. The voters decided that his unapologetic individualism, grandiosity and selfishness trumped the need for basic decency. They would rather see a candidate try to strong arm the democratic system into giving them the changes they want than work to build bridges with constituents on the other side of their ideologic divide. And that’s the sad demise of our democracy.
Because you brought it up; are we watching the demise of democracy in the US or a well-functioning oligarchy of the rich and powerful? I have a hard time feeling shocked at what we are seeing in this election (albeit from a Canadian perspective); if a democracy depends on rational citizens making informed decisions then I don’t think we’ve seen that for some time. These debates seem to suggest the campaigners are assuming their constituents are irrational citizens making uninformed decisions.
It may be an oligarchy but it’s not well functioning for anyone except maybe the oligarchs. I can’t speak for them.
Even the Oligarchy is scared of Trump because Trump reveals their secrets. Trump is pulling back the curtain on American politics. He told people that he gave donations in order to win influence! Which is what rich people have been doing for decades but no one seemed to know or care. He gleefully doesn’t pay taxes, which is what rich people have been doing for decades, there is even an entire “policy” based on this called “trickle down economics”. His racially charged language is stuff that the right in America has been saying, but in code words, for decades. He didn’t come up with a hatred of “political correctness”, that’s something that has been bubbling for decades. He is in favor of punishing women who get a legal abortion, something the really crazy right wing have wanted a politician to say for decades. His rhetoric that Mexican immigrants are rapists and murderers is a paraphrasing of Ann Coulter’s book “Adios”. And of course, the Millennials would point out that Donald’s boasting of sexual assault is just another example of a pervasive “rape culture” in America.
Is there enough data out there to show that there is a direct relationship between Chance Quality and titles? I mean.. I am sure good quality chances improve your goals scored which leads to more points but I am not sure if you can necessarily predict your chances of winning the league from that stat in itself. Also, while it’s true that a team returns to its mean over the long-term, it seems that outperformance/underperformance over, let’s say a 6 month period, could basically set the tone of that team’s title chances that season which makes it even more difficult to predict.
On another note, our Coq’s back in training. I’m really curious to see how many people think he should be in the starting XI instead of Xhaka. Xhaka was excellent against Basel but I feel Coq’s performance against Chelsea warrants him a place in the starting XI.
Prediction is hard, especially for a sport with so few scoring data points. Using chance quality or xG as it is more commonly known is better than simply goals scored/conceded but it isn’t some magic bullet. It helps give more context to the information that we have.
The data from last season seems to prove what we were mostly all vocalising, that if arsenal had a proper out and out striker we’d have stormed the league. No?
It’s funny that this year we have Alexis, and even though the # if chances created is down, we’re converting at a high rate. Is Alexis a big move in the direction we needed for striker? It’s seems so. I certainly love seeing him buzzing about up there. Is Iwobi the missing piece that allowed Wenger to experiment (again) with Alexis up front? It looks that way to me.
Just a note, the stats are from open play only. This season Arsenal have actually got penalties given to them! but that is not shown here.
Arsenal’s offense is pretty much average this season (but improving). Arsenal have 101 shots compared to 93 for league average. For Chance quality they are at 9.2 compared to 8.3 for league average. 11 Big Chances (penalties excluded) compared to 9.5 for league average. Average chance quality per shot 0.09 compared to 0.09 for the league.
For more visual types this chart shows the quantity of chances with the quality.
Top right is lots of good chances, top left lots of low quality chances, bottom right few but good chances, and bottom left few shots that aren’t good and prepare for a regulation battle.
And Arsenal are the most middle team of all 20.
We are average, on average. I think a few things contribute to this. We had an early season slump where we created very little against Liverpool, Leicester and Southampton. This was down to rotation in midfield and integrating a new style of play with Sanchez as the striker. This was followed by several games which we won in the first half: Watford, Hull, Chelsea. That’s roughly half the season. Shot prediction models don’t know when teams are just sitting on a lead and passing it around the back rather than trying to push the tempo. I’d bet there aren’t too many teams in the PL who won three games in the first half like we did. Even City, for all their marvelous pressing, have been in a lot of close games. I bet that’s what drives their chance creation numbers. I’ll also bet that’s what drove Arsenal’s stellar xG numbers last year, because the ball wasn’t going into the back of the net, so we had to keep trying. Thoughts on this, Scott and Tim?
Score effects are definitely a factor. When Arsenal go up more than a couple goals they really tend to shut up shop and play on the counter especially after the 60 minute mark (this is from watching not from the stats).
Is anyone working on sorting through addressing the confounding from score effect for shot quality metrics? You could stratify your data into three broad categories: leading, drawing, trailing. Would be interesting.
That is something that I have thought about, along with a way to figure out how to get the minutes played by the players. Right now I am focusing on getting as many of the manual processes out of pulling the data, I’ve got it down from 40 minutes per game week to 20-25 and I’d love to get it down to 10-15 before adding new stuff.
Sounds like you need more grant money 😉
Out of curiousity as a rank ignoramus on statistical analysis; how do penalty shots affect the xG vs. actual goals? If Iwobi shoots from the edge of the box and it hits a hand, becomes a penalty shot that we convert is that a conversion of a poor or high quality chance?
How would Koscielny’s bicycle kick and handball goal factor in? Didn’t both come off corner kicks? Just wondering if corner kicks are assigned an automatic rating for quality of the chance created.
With the play by play data that I get, a shot that has a hand ball penalty is not counted as a shot and instead considered a foul and no xG is assigned for that. The information that I publish does not have penalties included.
The goal that Arsenal scored against Burnley, the information that I have for that simply says that it was a right footed shot from inside the 6 yard box, it was assisted by a headed pass, following a corner and it was deemed a big chance by Opta. Given this information it registered as 0.63 on the chance quality scale.
As for the overhead kick against Southampton, that one the play by play data doesn’t talk about the awesomeness. It gives me Right footed shot from the center of the box, following a corner. Given this information it registered at 0.16 on the chance quality scale. This is probably way too high considering the difficulty of doing the bicycle kick.
When thinking about last season I don’t think the lack of a better striker was what cost us the league. Ollie was one of the few players to perform up to the standards you’d expect from him. The problem was that so many other players fell way below expectations all at the same time for a prolonged period.
After all, wasn’t a lot of the disappointment with last season based on the idea that we had a team, on paper, that should have been more than capable of finishing above a team like Leicester, even without an ungrade on the HFB?
“It also means that I’m not predicting ANYTHING. Certainly not the presidential election.”
Please tell me you’re joking. Is it really that close?
You know the thing that scares Trump more than anything, right? Losing to a woman. God, I want Clinton to win just for that reality to sink in for him. That he lost an election to a person whose gender, according to him, has no value outside a sexual currency.
I saw some polls after I wrote this that have Clinton ahead by a decent margin now.
The problem is that I think a lot of “undecided” voters are just people who are ashamed to admit that they are going to vote for Trump.
Plus, Hillary has been targeted by the right and Russia to be brought down. I don’t trust anything at this point, man.
I wish Warren had run. I’d say she would’ve had like a 40% chance to beat Hillary given how well Bernie did and she’d probably be even more favored against Trump and probably also help congressional races more. And of course she’d be a much better president.
If you think this election is hard to take , you better brace yourself for the next 4 years of the Hillary presidency.
If you think Obama was unfairly scrutinized and thrown roadblocks by the republicans at every turn , wait and see what the right wing media , including the new outfit -” Trump News”-, yea, it’s coming,will do during the next four years to the political discourse in this country.
If the democrats don’t win the senat and the house , the gridlock will be worse than ever.
He basically laid out the plan in the last debate: investigate Hillary with a special prosecutor every day for the next four years.
He laid out the plan from a position he isn’t going to be in , as this election is effectively over.
What he is doing though, is positioning himself to cash in on a tremendous hatred of Clinton by the extreme right, and is getting ready to form his own ultra right media outlet headed by Ailes, Kushner and Bannon or any combination of the three.
Trump has a devoted 15-16 million followers. Bannon & Ailes will start a Breitbart News Network post-election to compete with Fox and of which Trump will get a piece of the action. 15-16 million is a very strong base for any fledgling network.
The extreme right has rebranded itself as the alt-right. Hatred of the Clintons is only a small part of it. Their larger concern is apparently the “white genocide” that is occurring in the west. It’s pretty incredible really but it seems to me that a lot of it has come about from lies perpetuated by the Republicans themselves via Fox News and other media outlets. The Guardian had a pretty good interview with some of the men behind the scenes. If you haven’t read it already, it is worth a read:
His hyperventilating aside, you don’t think she’s guilty? From everything I’ve read of the FBI report and how the FBI and DOJ conducted their investigation it sounds like they did everything in their power to let her escape formal charges.
Hillary 14 points ahead in the polls as of the 9th. Looking good
Is Hillary that much better? It’s lose/lose for humanity in this one. Hilary is a war mongering puppet of the corporate elite.
I wouldn’t be celebrating either candidate. In fact trump winning might just shock American into not letting the system corruptly weed out decent politicians like sanders.
From what i’ve read, the idea that Trump might actually be the better choice because he’ll be so bad change will have to come from it, is something easier to say from a position of privilege (as the millennials might also say).
As a middle class straight white male, Trumps policies would probably only have a negligible difference on how I’d be treated compared to a Hillary win, but people not in that demographic would probably suffer a lot more than they do now. Even if he’s not responsible for specific legislation, a win for Trump would be a win for fundamentalists, racists and bigots everywhere. As a straight white guy I’m not on their radar, but some of the people who need the most protection are, and they are the ones that will suffer the most.
All I’m saying is that being a straight white guy it’s a lot easier for me to say ‘sure let him win, it might be better and shake things up in the long run’ but if I was gay, or black, or Hispanic or an immigrant, and poor, in America I don’t think I’d be as willing to make that gamble.
The democratic primaries had to be a wake-up call for the democrats (as much as the republican primaries were for the republicans). Washington isn’t working for the people anymore. It works for donors, corporations and lobbyists. This has to change. If it was a younger version of Bernie or a slightly less repugnant version of Trump, Hillary would have no chance to win. Surely both parties recognize this and will make some changes for the better. I don’t think we need to elect a jackass like Trump to see these changes. If nothing changes, I truly have no hope for the future.
I just can’t see Arsenal winning the title under Arsene in the presence of more ruthless and pragmatic coaches like Pep, Klopp, Mourinhio and Pochetino.
I still love Arsene and his methods that make him look stupid infront of millions of football Einsteins.
I much prefer seeing mourinho looking stupid but to each his own I guess
There’s a couple of famous quotes that seem appropriate to this discussion. The first is along the lines of “…some individuals use statistics as a drunk man uses lamp-posts — for support rather than for illumination”. The second is by the famous physicist Niels Bohr, “… prediction is very difficult, especially about the future …”.
Tim’s stats are ones I tend to find illuminating, they ask you to think when you make use of them. Anybody who thought xG was helpful last season, instead of watching what went on, espacially in the treatment room, was I would suggest, the proverbial drunk!
Anyone who’s not excited by the trajectory of Arsenal’s season so far should, in my opinion, give up watching football. We’ve seen a difficult first few games, followed by a period of bedding in a new central defensive partnership, followed by a couple of exciting performances, followed by a slightly fortuitous win against one of the most obdurately defenisve sides as home that we will encounter.
Will it all pan out well? Who knows but we are in a very strong position with great options.
Finally, I can’t resist saying that to me Kostafi resembles ‘Solo Piano’ – harmonious, calm and beautiful. Whether it stays that way or descends towards ‘Music in Contrary Motion’ we will see 🙂
Should have been Koscielny-Mustafi partnership in the final paragraph
What we’re seeing is the rise of a nationalist party in the US, much like the Front Nationale headed by Le Pen in France or any other European country. An isolationist, anti-trade, anti-immigrant party organized primarily around a single ethnic group or identity.
Most true conservatives who believe in free trade, reduced entitlements and balanced budgets, constitutional limitations on government powers and American defense of the free world against aggressive tyranny will split off from this new nationalist Republican party.
Similarly I wouldn’t be surprised to see a more socialist party split off from the Democrats, but winning will probably put that off for a few more years yet.
Look at it this way, Tim. You’re in good company; Wenger was once a season early with one of his predictions, too. 😉
How are predictive stats supposed to cope with a season where leicester finish 25 points above chelsea.
Last season was weird.
Great blog as usual. Thanks for the many though provoking comments as well
The day citizens united was passed was the end of hope of a reasonable democratic government. Blame bush and Roberts for that one. This election is a question of lesser of the two evils
Comments are closed.