Analysis: The Numbers Bankability Index Under the Covers

November 13, 2013

On Tuesday we launched The Numbers Bankability Index, a new service to help assess the value that different people bring to the industry, from actors and actresses to directors, screenwriters, producers, and anyone else involved in the creative process of making a movie. In this article, I'll look more deeply into how the Index is compiled, and how we use the tools behind it to analyze questions about people in movies.

If you haven't already, now's a good time to look at our announcement video, and the November Edition of the Worldwide Bankability Index.

The original idea for the Index came from the challenges of using traditional methods to measure the top people in the industry. Regular visitors to The Numbers will be familiar with our People Records section, which lists people ranked by the total box office of their films. Over the years, we've added various refinements to these lists, most recently adding analysis of technical credits, people in leading roles, live action films, cameos, supporting roles, and so on.

These lists are quite instructive, but they run into a number of limitations when one tries to apply them to, for example, estimate how much someone is worth in a particular role. For example, the top 100 stars in leading roles gives you a good sense of who the top all-time stars are, but doesn't give you much of an idea who you should cast in the next Star Wars movie.

One can attempt to better find the next Star Wars lead by adding rules on top of a list like this. For example, one could discount historical box office performance, filter out roles in animated movies, and add in information about Oscars, or People's Choice awards. However, you soon run into a problem I discussed in my recent article on Medium Data: the more complexity you add to the model, the higher the chance that you will find "signals" in the data that are the result of randomness, not a real underlying phenomenon. Anyone who does statistical analysis is familiar with this problem: every time you add another variable, you increase the noise. Eventually you end up with a kind of data porridge that doesn't tell you anything.

The other problem with an approach that looks at the movies someone has contributed to, assigns values to them, and adds the results together to come up with a score, is that it doesn't take into account how movie making actually works.

If you left Steven Spielberg alone on a desert island with film-making equipment he might make something amazing, but it wouldn't be what we think of as a "Steven Spielberg movie". To make a "Steven Spielberg movie" you need Steven Spielberg, of course, but you also need an actor like Tom Hanks, a composer like John Williams, an editor like Michael Kahn, a cinematographer like Janusz Kaminski, and so on.

In fact, I would go further than that and say that some relationships are critical to a certain type of film: Janusz Kaminski's cinematography is one of the things that makes Steven Spielberg's films so recognizable.

These two issues -- avoiding the data porridge caused by using too many variables, and recognizing the networks of people that make films -- were what drove the design of the Bankability Index.

In fact, realizing that films are made by networks of people was the big clue to the creation of the Bankability Index methodology. One of the areas of very active development in analytical models in recent years has been graph analysis. It's a technique that allows Google to rank web pages, Facebook to decide who you might want to connect to as a friend, and credit card companies to detect fraud, among many other things. Perhaps analyzing the network of people in the movie business would allow us to evaluate which films might be successful, and who has the biggest influence on their financial success.

This led us to develop the Hollywood Creative Graph™, which is a network of everyone who has worked together in the film industry, and then to develop ways to apply graph analysis to the Creative Graph to determine who brings the greatest value to a particular area of the industry.

If you want to skip the technical details of this, begin reading again after the box. If you want to dive in, keep reading!

In summary, the process of determining the value people will likely bring to a particular project consists of four steps:

• First, build a graph that links all people who have worked together. This is the relationship graph.
• Second, assign an economic value to each relationship between two people. This is the metric we use in the analysis.
• Third, use a graph analysis algorithm to estimate the relative contribution of each person in the network. This is the algorithm.
• Finally, multiply the relative contribution of a person by the value of the graph to derive an estimate of the contribution of each person in dollar terms. This is the valuation step.

To answer different questions about people in the industry, we use different metrics in the second step, and different valuations in the final step. For example, if we're looking only at the domestic box office for adventure films, the metric will be the domestic box office for each adventure film, and, in the final step, the total domestic box office for all adventure films combined would be the value of the network. The result of the analysis for each person is their contribution to the domestic box office market for adventure films.

For our monthly chart, The Numbers Worldwide Bankability Index, we use worldwide box office as the metric in step 2, with two additional factors. The first discounts box office over time so the chart roughly reflects someone's contribution over the last ten years, and the second discounts the performance of sequels, so we're looking at the value that someone brings to non-franchise films. The value of the graph is an estimate of the share of the global movie industry ascribable to its creative talent (basically, 25% of global film revenues, on the assumption that half of the revenue stays with the retailer and 25% is spent on non-film-making sectors such as marketing, distribution, and so on).

The Index also contains a "value per movie" measurement. This is roughly how much I would pay someone to be involved in the making of an average non-franchise film. (To be more precise, it's how much I'd pay them to work on their average non-franchise film — people who choose their projects more carefully tend to have higher averages in the Index.)

The Bankability Index has generated some interesting and lively discussion. Some see the Bankability Index as suggesting that A-List actors are overpaid. Is that likely to be the case, based on the data?

No, not really. The Worldwide Bankability Index assesses the value someone would bring to their average, non-franchise, film — regardless of whether they are in a lead role, or even an acting role. It looks at the value each person brings to a film as part of a team.

This can be very different from deciding how much they should be paid for a particular role, particularly in established franchises. Johnny Depp, for example is measured as being worth about $9.3 million to an average movie (like, for example, The Rum Diary), but is clearly worth far more playing Captain Jack Sparrow in Pirates of the Caribbean.

To illustrate this fact, I ran the algorithm to focus specifically on lead actors and actresses in blockbuster franchises. This analysis assumes that someone is appearing in a role for which they are already well known — Robert Downey Jr. as Iron Man for example, or Daniel Radcliffe as Harry Potter. It also heavily discounts older films, so we're looking mainly at performance over the past 5 years. Finally, I used a value for the graph as a whole based on an estimate of the revenues generated by blockbuster franchises over the past 5 years. Here are the estimated values for each person in each role:

I should note that I made a fairly rough estimate of the value of blockbuster franchise films, so the table should be read more as a comparison of the relative value of each role. The order certainly seems about right to me though, and the dollar figures at least in the right ballpark.

The important thing to take away from this is that context matters, a lot. Johnny Depp will be great in pretty much whichever role he takes on, but he really generates the big bucks when he's working with the creative team behind Pirates of the Caribbean, and playing Jack Sparrow. Outside of that franchise, he's still one of the highest-performing people in the industry, but his value in a particular film comes much more down to earth. The same can be said of anyone in the list above.

Now that The Numbers Bankability Index is up and running, we'll be doing a lot more of this kind of analysis, and incorporating more of the features of the Index into the web site. Look for more on that shortly. In the meantime, as always, I welcome feedback and questions (my email address is below), and if you're interested in using the Bankability Index tools in your project, please email us at research@the-numbers.com

Filed under: Analysis, Pirates of the Caribbean, Johnny Depp, Tom Hanks, Christian Bale, Robert Downey, Jr., Rupert Grint, Shia LaBeouf, Taylor Lautner, Robert Pattinson, Daniel Radcliffe, Kristen Stewart, Emma Watson, John Williams, Janusz Kaminski, Michael Kahn