On charisma and knowing founders

More questions than answers

Every now and then, I walk away from a pitch in awe. All venture capitalist goes through a similar experience. Great pitches are the subject of venture lore that echoes, "I knew I had to invest within 10 minutes..." But what makes a great pitch? There are several elements but one is charisma. It is often mixed up with passion. But charisma is social by definition but passion is not. A concept difficult to define clearly, charisma is one of those you-just-know-it-when-you-see-it. But new machine learning research was able to measure it and show it can increase a startup's chances of getting into selective accelerators like Y Combinator by 35%.

The charisma effect on fundraising

Song Ma and Allen Hu studied what it took to persuade investors in a video-based world. It is an ingenious study that machine learning and venture geeks should read. They collected publicly-available pitch videos (63% of pitches are for YC), calculated a pitch score for each, and compared them against funding outcomes. The one to three minute-long videos are part of the applications submitted to accelerators. The videos are fed to machine learning models to measure visual, vocal, and verbal scores, scaling them from positive (e.g., smiling) to negative (e.g., frowning) sentiments. These measurements are then combined to calculate a pitch score, standardized to zero-mean with a standard deviation of one (if you know how to express this in simpler words, please comment). Simply put, the pitch score measures charisma. He found that pitches with a score of one increase the probability of getting funded by an accelerator by 3.0% points from 8.5% to 11.5%, a 35% increase. The charisma effect is more than twice as large, 6.6% points, between the best and worst pitches. The results hold after considering founders’ experiences, education, video quality, and other factors. The startling insight is that the pitch score's primary driver is the vocal valence, which measures positivity vs. negativity, and arousal, which measures excitement vs. calmness, measured from sound waves. Secondary drivers are whether founders are frowning or use frown or use melancholic vocabulary.

Example of high arousal pitch. From "Persuading Investors" by Song Ma and Allen Hu

Example of low arousal pitch. From "Persuading Investors" by Song Ma and Allen Hu

I asked a YC alumna how he prepared and what advice he got. No one coached him to change the way he talks, not that he needed it, nor has he heard of the advice circling in YC prep groups. Maybe it is obvious that founders need to be captivating so no one talks about it. Maybe it is also not socially acceptable to tell them they are not. But telling their business idea sucks is.

Why investors like charismatic founders and maybe why they should not

We are universally attracted to charismatic individuals, from Barack Obama to Steve Jobs, even if we find them disagreeable. For decades, psychologists have debated how to measure charisma, but they broadly agree that we are all enchanted by it. Even then, as professional capital allocators, venture capitalists should not invest on charisma alone. You cannot tell LPs that is your investment strategy. (Or maybe you can?) Charisma should increase the probability and size of DVPIs. The belief is that charismatic founders can hire better talent, sell to more customers, raise more capital, and run their business better. I remember asking one of my mentors what I should focus on looking for in founders. He said,

"After being in the industry for decades, what I have learned is that all you need to look for [in founders] are two things: the ability to hire people and raise money.”

To investigate this, Song & Allen experimented with measuring whether investors liked charismatic pitches because of their personal taste or belief that it leads to success. He found that belief generates 82% of the charisma effect. Not surprising. Decades of research and business school education permeated it to the public consciousness as common knowledge. Researchers repeatedly found that having charismatic leaders led to better profits and motivated employees, which should result in better returns for investors.

But do they actually? The study’s conclusion was no. He discovered that higher pitch scores led to worse outcomes. Startups with a pitch score of one are 9% less likely to raise follow on funding and 4% likelier to close shop after two years. Maybe this is one reason why 65% of venture investments lose money. When I interned for Hustle Fund, I met a founder building a platform for sales engineers to deliver interactive demos. It was a great pitch. He was eloquent, energetic, and "passionate". While we did not end up investing, I thought he would be off to do great with his startup. A few months later, he shut the company down and took a corporate job instead.

Conundrum of knowing founders beyond the pitch

Why is there a disconnect between the literature exhibiting the cloud of charismatic leaders and poorer investment results? It is because founders can behave differently while pitching and running their companies. The cynical view is that founders put on a different face while fundraising. But that is normal. Everyone does it, investors included. It takes time to know someone. Years of going through thick and thin. But all accelerators have to evaluate a company are an application form, a recorded video, and a short interview. So, pitches, whether recorded or live, play a key role. There is not enough time to do more diligence than that. Accelerators run multiple programs a year and receive several thousand applications. YC runs their program twice a year, every cycle receiving more than 10,000 applications - each has a 1-minute pitch video. Watching all the videos is equivalent to watching the entire catalog of The Office four times in a year. At peak review time, a partner will review over 100 applications a day. A few minutes of attention is all an application gets.

Contrast this to the diligence process of institutional venture capital funds like Shasta Ventures. Diligence usually takes weeks. Investors meet founders multiple times and interview several references before making a decision. With more information about how founders are beyond the pitch, there should (theoretically) be less of this charismatic bias. The investor-founder relationship is often likened to a marriage. So both parties want to know each other well before committing. This is why investors like to back the repeat entrepreneurs, especially ones they have backed before.

Like most investors, the majority of founders I meet are first-time founders I have not met before. I needed a systematic way of knowing founders better. Learning from seasoned investors is valuable and the fastest way but each investor will have his-her own way of doing so. And it is difficult to get perspective from more than a handful of people. So I looked for research distilling different investors' approaches and found one, luckily. Geoffrey Smart collected 84 case studies from 48 venture capital funds to study how GPs evaluate startups and how accurately they assessed the team. Accuracy is measured by GPs self-reporting whether they got their assessments of the team right. It is a fascinating study that you should read if you are a junior investor. Since the median venture capitalist makes two investments a year, the study distills 42 years of experience.

Geoffrey created typologies of how investors approach team diligence and categorized their diligence methods (e.g., interviews, references). Some investors were systematic in their diligence but most assessed the startup team based on gut. It is no surprise then that he found systematic investors to be more accurate and have better returns. The insight is that numerous structured discussions with the founders about their past and future are the most accurate methods to assess teams (called past-oriented interview and work sample, respectively, in the exhibit below). Discussing the future is something all venture capitalists do. It consists of the usual questions on the use of funds, competitive differentiation, growth targets, etc. But structured discussions about their past are uncommon. These are similar to standard job interview questions probing into past achievements, failures, and behaviors. Venture capitalists just do it casually via family dinners and hikes, which are reckless to do now, instead of back-to-back 45-minute interviews in a conference room. The value of probing the past is finding actual behavioral patterns that persist. By focusing on the product roadmap and sales forecasts, early-stage investors misjudge a team's technical and selling skills only 10% of the time but misjudge managerial skills 30% of the time. The practical lesson here is that investors should have processes and questions to understand founders’ managerial skills, whether they are students, serial entrepreneurs, or executives who have run $100 million divisions.

Drivers of accuracy. From “Management Assessment Methods” by Geoffrey Smart

More questions than answers

The challenge is contextualizing the learnings to today's market. Deal time has shortened, and it is a founders' market. Geoffrey completed the study in 1998 when investors had more time. The investors he interviewed spent 120 hours over months on average to assess the team for each deal. First Round's average first contact-to-term sheet time in 2004 was 90 days. In 2019, it was nine days. In comparison, US couples date for 2-5 years before tying the knot. So I asked some seasoned investors how they do it. How do they get comfortable committing to a decade-long relationship in days? One answer I got summarized the overall sentiment. In a bit of a sour tone, he said,

"It is not ideal but you just have to take the risk. You also can't ask hard questions. You have to be nice to founders or else you might lose the deal. Of course it is different if you have the Sequoia brand."

I'm ending this essay with more questions in my mind. I should spend more time with founders learning about their past, but what should I focus on given the frenzied pace of deal making? What are the best questions and hacks? How to do it best in a remote world where we're all just boxes on digital screens? But the questions also go the other way. How can founders do the same?

— Kenn

Preview of the next essay: The number of unprofitable companies going public today is as high as it was in the dot-com bubble.

Other notes from writing this essay

• The other significant finding from Song & Allen's research is evidence of gender bias. The gender stereotypes are that males are judged based on their ability while women are judged based on their personality. The study showed that pitch score matters more for women than men by a factor of 4-7x. Women are penalized more if they not charismatic. More on this in the future.

• I want to build an app based on this study. What are the best tools to build this quickly and host it cheaply? I prototyped an ML app before but cluelessly racked up a $3,000 cloud bill - which AWS graciously offered to waive. I want to avoid that from happening again.