Can MMA statistics help us predict what will happen in important fights?
Maybe. If they can’t outright predict a single outcome, they can at least give us a sense of how the fight is likely to play out or maybe even look. That’s the premise behind FightMetric’s latest innovation: the MATch Up Analysis, or MATUA simulation engine, which according to FightMetric is “not a true prognostication tool” but “produces a statistical view of upcoming fights that may see things that our eyes do not.”
“The MATUA model harnesses the power of FightMetric’s deep database of statistics to simulate a match based on the two fighters’ past statistical performance. The simulation is run 10,000 times to reduce random chance and then produces the number of simulations in which each fighter won, by what method, and in which round.”
MATUA debuted just prior UFC 146. The first and only bout it has run publicly was for that night’s main event: Junior dos Santos vs. Frank Mir. Ultimately, MATUA gave UFC heavyweight champion Junior dos Santos a winning percentage of 66% against Mir in 10,000 simulated fights. And as we know, the champ eventually dispatched with Mir in the second round via TKO stoppage, something MATUA had as the second most-likely outcome in a dos Santos victory.
For now, FightMetric isn’t running MATUA on theoretical match-ups. If you’re curious about it says for a Cain Velasquez vs. Alistair Overeem fight, you’ll have to wait until that one is signed and ready to happen. But I was still curious about the numbers and what they can actually tell us fight predictions and even MMA itself. I caught up with the creator of MATUA, John Candido, to find out more.
Below is an excerpted portion of our conversation.
Luke Thomas: How did the idea to create this come about?
John Candido: I had initially talked with Rami [Genauer], the owner of FightMetric, and I’d always been interested in looking under the hood of different sports and things like that to sort of understand the moving parts behind it.
I wanted to see what really mattered in a fight. That’s how my relationship with FightMetric started. Once I got access to all the statistics I started to basically breakdown and model what exactly goes on in a fight; sort of looking at the mechanics behind fights.
Once we were able to do that, then we were able to develop this model which basically takes all those mechanics and is able to use them to simulate exactly how fights play out and what goes down in fights. In doing so, we’re able to sort of see a good percentage of which rounds likely lead to the fight ending in, and also what the typical outcomes are.
Luke Thomas: The description of MATUA states “the simulation is run 10,000 times to reduce random chance”. Why 10,000?
John Candido: To be honest with you, it seems like pretty much the accepted industry number. It really doesn’t have that much statistical significance, specifically the number 10,000. But since that’s what most other people who run these type of simulations are doing, I figured we might as well make it even across the board. It’s easier for the general public to understand it and compare it with some other simulations going on in other sports.
Luke Thomas: The description also says “It is built upon a statistical analysis of every UFC fight of the modern era”. What does that mean, post-UFC 32?
John Candido: I think it was post-UFC 24, if I remember correctly. Whenever the rule change happened. Whenever that happened, it’s all the fights after that.*
Luke Thomas: What’s the extra value add of a simulation engine that we don’t get from other forms of qualitative analysis?
John Candido: Specifically, the engine is driven by a machine running algorithms. A lot of people will look at statistics for upcoming fights and do intuition in their own head about weighting which factors are more valuable, which factors are more predictive of outcomes, and things like that.
What this does is actually put some science to those intuitions. It puts an exact weight based on a lot of different simulations and algorithms and analysis, and actually figures out scientifically what the exact weight of these things are.
So people might think dos Santos has a great chance of knocking Mir out and we know this because a lot of his career has been knockouts, and he’s a very good boxer and the numbers show that. We’re actually able tell you why statistically dos Santos has a larger chance of having an outcome of a knockout. It’s putting exact science to all those guesses that you would typically make when you’re breaking down a fight intuitively or looking over the basic statistics for it.
Luke Thomas: Critics of MMA statistics often say there isn’t enough sample size to make them meaningful. That criticism would apply to MATUA, too. How would you respond to that?
John Candido: It’s not like we’re basing this whole simulation model on dos Santos’ career. Because we’re basically basing it on every fight that’s happened in UFC since the rule change, we’re able to look at the mechanics instead of just an individual fighter’s record.
From the perspective of ‘this guy’s had a pretty good run in the last ten fights’, whether that’s legit or not, yes, and there’s a question there.
But as far as understanding the basics of how MMA works, that’s something I think we’ve pretty solidly nailed. Even though we only have ten fights worth of information, let’s say on a particular fighter to base that off because the sample size is pretty small, the common fan is still doing the same math in their head based on those ten fights anyway.
Yes, because we have smaller sample sizes it’s a little more difficult, but the accuracies at least I’ve run into doing the analysis have been pretty good. It is a smaller sample size, but there is a lot of significance and predictive value to the small sample size because MMA – as a sport to model – lends itself very well to breaking it down statistically and being able to put weights on all the different parts of it.
Luke Thomas: You say you have a sense of how MMA works. How did you come to that position where you have a feeling for the complexion of the sport such that you can run a simulation engine?
John Candido: It’s just basically developing a lot of different variables that reflect or represent a lot of the different types of aspects of the Fight Metric stats. Once we were able to do that, we can put them in a horse race of which variables will perform the best when it comes to predicting different outcomes.
For instance, predicting knockouts for a fight. There’s going to be certain variables that lend themselves more or more heavily weighted that will predict that outcome more often that not. Different variables obviously will be more predictive of different outcomes.
You wrote an article about age. That would be another factor I would put in a horse race with a lot of other factors. Once I run the analysis and the algorithms on those, I’m able to see exactly which ones come out with more predictive value than others do. Once I’m able to do that, I can compare how all the variables stack up against each other. The model is built around that principle: weighing heavily the favorable variables that have a lot of predictive value to them and then ignoring some of the other variables that don’t exactly give us a good idea about who is going to win.
Luke Thomas: Did you learn anything in putting together this model together and figuring out what the favorable predictive variables were? Did you learn something about MMA in the process?
John Candido: Oh, absolutely. Plenty of things jumped out. There were a lot of notions I had going in of what I thought I would find that I was surprised by or taught to look a different way at it.
Just specifically something off the top of my head was finding out how important wrestling was in MMA. That was probably one of the bigger takeaways of developing the model, just breaking down the sport statistically in general. Wrestling is a huge, huge factor and I didn’t expect that as much. I didn’t expect that striking would be such a significant second to wrestling when it came to a lot of the variables.
Luke Thomas: In terms of Mir’s 34% chance of winning, do you believe this is a better reflection of how he’ll do or just that it’s in contrast to what the oddsmakers are suggesting?
| Method of Victory | Round of Stoppage | ||||||||
| KO/TKO | Submission | Decision | Total Wins | 1 | 2 | 3 | 4 | 5 | |
| dos Santos | 45% | 6% | 15% | 66% | 32% | 18% | 11% | 8% | 6% |
| Mir | 9% | 14% | 11% | 34% | 25% | 14% | 11% | 7% | 5% |
John Candido: When it comes to odds, that’s a different type of analysis. I wouldn’t necessarily say that reflects something in the odds themselves. I think what the simulation does more is allow us to see more of how the fight plays out and the different outcomes, methods – more how the fight is going to go down than necessarily the absolute outcome.
I wouldn’t say that’s the most effective use of the model. I have a separate model that I use when I write my ESPN Insider articles and what that model does is actually – it’s more based on predicting inefficiencies in the market than it is predicting what’s going to go down in an actual fight.
When you’re looking at that you’re asking two different questions statistically, and because of that you have to break it down a little bit differently and analyze it a little bit differently. But I would put more stock in the method outcomes and the round outcomes because those are generated off of the 10,000 simulations and those are aggregated or compiled based on what’s most likely to occur, what’s most likely to happen.
You can definitely take from the fact that dos Santos is a 66% favorite in a sport with a lot of parity that 66% is a pretty significant advantage over his opponent. You can take that as dos Santos having a pretty big edge over Mir.
Luke Thomas: Is there a way to run this in reverse? That is, is it a fair way to gauge the accuracy of this engine to marry what actually happened in fights after the fact with what the simulation engine says about what would happen?
John Candido: Yeah, sure. When all the algorithms are built, they’re built off of historical data, but in constructing them they’re definitely tested against data that is not used in constructing the model. In a sense, in the construction of it it’s already done that.
The relevance of the algorithms that it comes up with, the relevance of the model itself is tested out beforehand to see whether or not it performs well or holds its weight against new data in anticipation of it receiving new data in the future.
So yeah, you can definitely go back and validate its performance and see how it would’ve done because its already done that in the construction of the model itself.
Luke Thomas: How often do you believe you’ll have to go back and update the algorithm and the model here as the game itself changes?
John Candido: That’s an interesting point because obviously mixed martial arts is a very evolutionary sport and things are constantly changing.
I don’t see it being a fight-to-fight basis or an event-to-event basis, but if there are new patterns that emerge and things that come up, then I’ll definitely be able to see that in the change of how new models asses weights different variables and how that kind of changes over time.
The more data we have, the better the model will be and the more accurate it will be. It will pick up on these new patterns as they arise and as new data is set in to it to train it on.
* = UFC 28 was the first UFC event to use the Unified Rules of MMA.
In the wake of Kenny Florian’s retirement, I felt it necessary to give praise to one of MMA’s improperly viewed fighters. That is, nearly everyone agrees Florian was a supremely talented fighter, but failing to win titles in three attempts across two weight classes are the true defining moments of his career.
It’s true it’s impossible to ignore those shortcomings. They are part of his history. But undo focus on them really shortchanges a fighter of pretty remarkable accomplishment.
More than almost anyone in the modern era, Florian worked with an unparalleled diligence to consistently improve his skill set. It’s true Florian’s athleticism was always a touch underrated and his improvement is partly of function of what a good athlete he actually was. But there are very few fighters one can point to from Florian’s generation or ‘class’ who developed into the final product he became given his starting point.
Every aspect of his game was sharpened. Those dimensions where he lacked severely in the early stages of his professional career ultimately became his strengths. Florian had weaknesses, too. No fighter is an android of perfect technique and execution, but the journey he traveled to position himself to win a title must’ve taken a will few among us can summon in any professional endeavor.
That he did what he did in MMA through the sweat of his brow and unfailing belief in himself is worthy of our highest praise and admiration. He unquestionably gave MMA everything he had and he did it year after year, fight after fight. We can ask for not one thing more from anyone who competes.
Fortunately, there’s more to the story than my qualitative analysis. FightMetric breaks down the numbers behind his very real accomplishments:
– Florian is the first and only fighter to compete in four different weight classes: middleweight, welterweight, lightweight, and featherweight.
– He retires with 8 wins in the UFC by submission. That ties him with the most among fighters in the modern era. Royce Gracie has more all-time submission victories.
– Florian earned 7 tapouts on 13 submission attempts, giving him a submission accuracy of 54%. That makes him the only fighter in the modern era with a submission accuracy greater than 40% (a minimum of 10 sub. attempts is required to be considered).
– He holds a perfect 7-for-7 record on rear naked choke attempts.Florian landed more significant strikes than his opponent in every one of his UFC victories. 10 of his 12 wins inside the Octagon came by way of stoppage.
– He retires having accumulate 3:07:38 of fight time. That’s the 13th highest career length in UFC history, putting him directly behind Chuck Liddell.
All quantitative data provided by FightMetric except where otherwise noted.