Moneyball for Legal Hiring
October 9, 2018
You’ve probably heard of Moneyball, the 2011 movie starring Brad Pitt, based on the book by Michael Lewis. "It's the story of Billy Bean, the Oakland Athletics' General Manager, and his use of sabermetrics to win a divisional championship with one of the lowest payroll budgets in baseball."
Michael Lewis—who clocked hundreds of hours with Billy Beane to write the book—called Beane’s method Moneyball.
The Story of Moneyball
2002 was a tough year for the Oakland Athletics Team. They lost three key players to larger market teams, including 2000 MVP Jason Giambi to the Yankees. Frustrated with his inability to compete with larger market teams. —the A’s had one of the smallest budgets for player salaries that year—Beane sought the help of Paul DePodesta, a statistician from Harvard, to seek out players that were being undervalued by the market. Together, they looked at statistics of in-game activity by players – called sabermetrics – to evaluate how the A’s were scouting talent.
Their analysis revealed that a player’s on-base percentage is a better indicator of winning baseball games than batting average or home runs. Players who scored high on this were undervalued by the market. Beane began recruiting these overlooked players.
The other scouts were using their intuition and were deeply skeptical of Bean’s statistical approach. Nevertheless, he persisted.
And his analytical recruiting strategy quickly paid off—the A’s started winning, even against baseball teams with much larger budgets. The team famously became the first American League team in over 100 years to win 20 consecutive games. Today, sabermetricians are routinely hired by every major league front office.
Law firms and legal departments can leverage a similar statistical edge.
We know, because we do it every day at Hire an Esquire. We call it Moneyball for Legal Hiring.
How Soft Skills Predict Job Performance
Like sabermetricians, Industrial and Organizational (I/O) Psychologists study human behavior in organizations and the workplace. What they’ve found, and studies show, is that two personality traits are highly predictive of future performance on the job for all industries and roles:
-
Conscientiousness—refers to a person's ability to be organized, disciplined, achievement-oriented, and dependable
-
Emotional resilience—refers to a person's ability to remain calm under pressure and be adaptive to stressful situations and people
In short, candidates who score high here are most likely to be top performers.
Yet, not all candidates are conscientiousness rockstars, and that doesn’t necessarily preclude them from doing a great job. Not surprisingly, the traits of a successful doctor are different than those of a successful artist.
These more nuanced and contextual elements to specific jobs are the focus of our industry- tailored approach. All of our candidates take a proprietary psychometric assessment based on the five-factor model of personality characteristics aka “the big five.” By measuring these traits, we can then match their competencies to the roles where they will best succeed.
The other side of this match is an analysis of the job being hired for based on a set of knowledge, skills, and abilities (KSAs) that we’ve tailored to specific practice areas. We require each law firm or legal department to complete our job analysis questionnaire which further determines what qualities are important for a specific job.
Of course, a candidate assessment alone is only one data point. We triangulate our assessments with a behavioral interview, another proven psychometric tool, and feedback from references, peers and reviews from clients whom they’ve worked for through Hire an Esquire. This triangulation also helps to identify when a candidate is attempting to “game” an assessment or where there is a large gap between self-perception and real-world performance.
To walk through how this works in practice - a Hire an Esquire law firm clients posts an associate attorney role where the KSAs and results of the questionnaire indicate that leadership is an essential competency to this particular job. The candidate assessments of the 10 attorneys in the Hire an Esquire network who have the specific prerequisite experience and background would consider if they are assertive and competent, which are specific facets of extraversion and conscientiousness that have been linked to leadership performance.
Their assessments would be scored in tandem with how these candidates answered the behavioral interview questions assessing the same traits such as— “describe a time that you took charge of a situation at work?” These results would be combined with responses from peer and reference surveys designed to assess these same traits.
Together these results would help to predict how a candidate would perform and their “fit” score for the role.
A Significant Data Set is Essential
The third component, crucial to assessment accuracy, is having a substantial pool of candidates and data to compare against each other. Our network of over 9,000 candidates coupled with the thousands of job postings, applicants, and placements that have gone through our platform have provided a wealth of data points and outcomes for comparison.
This data is essential to making accurate comparisons and to weighing one candidate’s scores and hiring outcomes against another. This also further differentiates Hire an Esquire from brick and mortar agencies (even if some are now attempting to implement some form of assessment or structured interview) or other hiring processes where there is limited candidate information and/ or the data is siloed in spreadsheets or basic CRMs.
Playing Moneyball
Measuring future human behavior in high correlation is impossible. These methods have no absolutes or guarantees. But like Moneyball, we can see correlations, and offer better chances of finding the best talent.
And we have at Hire an Esquire.
Temporary-to-permanent employee conversions increased from 5% to 17% using the Moneyball method.
Since we began developing and fine-tuning these assessments in the fall of 2016, we’ve seen our candidate incident rate fall from 4% to 1%. Our rate of clients requesting the same temporary candidate again jumped ~67%. Additionally, our temp to perm conversion rate tripled across the board and specifically, our rate of conversions for AMLaw 200 and Fortune 500 placements jumped 240%.
As we approach a world where careers are becoming more transient and on-demand, assessments like ours are crucial to finding the right talent, quickly —even more so for legal talent, where bad hires are expensive not just in terms of payroll indirect impact to client outcomes. Assessments like these are the best tools we have to predict how an attorney or paralegal will perform on the job. These increased odds for finding the right talent can position your practice for higher productivity and lower turnover.
The Moneyball Edge
A common objection to Beane’s Moneyball method: if the Athletics became so good at predicting future success, why did they have so many losing seasons since 2002?
“Once all teams are playing Moneyball, then playing Moneyball no longer gives you an edge,” writes Simon Kuper for Slate. In other words, to get the Moneyball edge, your team has to be the only one doing it.
Hire An Esquire is the only legal staffing agency measuring the indicators of high performance against a wide pool of candidates. Our team of I/O psychologists have nailed down psychometric assessments to give law firms and legal departments an edge over bigger players in the market.
And it works because we’re the only one doing it.
Learn more about Hire an Esquire’s Moneyball Method and our statistical edge over traditional legal staffing agencies and recruiting methods, or sign up here speak to one of our experts.
References and further reading:
Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job performance: a meta‐analysis. Personnel psychology, 44(1), 1-26.
Judge, T. A., & Zapata, C. P. (2015). The person–situation debate revisited: Effect of situation strength and trait activation on the validity of the Big Five personality traits in predicting job performance. Academy of Management Journal, 58(4), 1149-1179.
Shultz, M. M., & Zedeck, S. (2012). Admission to law school: New measures. Educational Psychologist, 47(1), 51-65.