- OptimistiCallie
- Posts
- 🧠 The best analysts chew gum
|
I have a theory about AI that I want to share with you.
A lot of smart people think AI is going to displace a lot of jobs.
But they don’t think AI will replace their job.
Why is that?
From my perspective, it’s not because we all think we are special, irreplaceable snowflakes (OK, maybe a little).
Instead, it’s because we intimately understand all the human touch and qualitative decisions needed for our own job, while we reduce other people’s roles to a few tasks in our mind.
I am guilty of this. You probably are too.
Still, in the conversation around AI and the future of work, I think it’s important we talk about what exactly we’re claiming when we think AI will replace X or Y job.
And what better way to do it than to share the intricate details around my own line of work?
Yes, I am among the people who think AI will never replace the best investment analysts, even though I think AI will supplement our jobs in interesting ways.
If you’re new here – a little about me:
I’m the chief market strategist at a $7.6 billion RIA, responsible for helping craft and communicate the firm’s views on markets, the economy and investing.
I talk and write about these topics all the time, but what you see in public only scratches the surface of my day job. My most important role is leading the vision on how the firm’s investment committee thinks about our portfolios through thoughtful analysis (and then telling that story to advisors, clients and prospects).
I’ve been in research for 13 years now. First as a data-driven journalist, now as an analyst. I’ve worked at banks, brokerages and market structure research firms. Companies big and small.
The stakes are high in this role. Honestly, about the highest they’ve ever been in my career. And it’s not because of my title – it’s because I am directly involved with investment decisions. I have a huge responsibility to all the clients we serve, and I take that responsibility seriously. Data integrity is my number one priority, because cutting corners can lead to financial consequences.
Also, as an analyst involved in money management, my most precious commodity is my reputation. I protect it fiercely, and that requires me to put weight behind everything I post and say.
Research is messy.
Data quality is a big, big deal.
After all – garbage in, garbage out. And whew, there’s a lot of garbage.
A big part of my analysis is understanding what data is trusted, useful and correct. If the sample size is big enough, if the data provider is sound, and if the data itself passes the BS sniffer.
Analyzing data requires an intimate knowledge of each space, too. The data series you use can matter a lot. Choose the wrong cut of data and suddenly you’re looking at an entirely different trend.
Retail sales are a great example. The Census Bureau releases a monthly report on sales across different goods and services, and it’s a decent way to judge total spending by category.
That is, if you look at it through the right lens.
Many economists skip past the main numbers you see in headlines because they include categories like food and gas, which can be subject to volatile price fluctuations from supply constraints, weather, and geopolitical conflict. There’s a special cut of data called control group sales, which are adjusted to align with GDP data collection methods.
From there, you need to adjust for prices – because a high-inflation environment can trick you into thinking demand is strong. See 2022 in this chart.
How can you possibly judge revenue growth without understanding if sales are higher due to price or units sold?
You can’t. Not without lying to yourself and your stakeholders.
Research is expensive.
Bloomberg terminals cost $30,000 a year. They’re one of the most eye-watering expenses analysts have to lobby for. My nerd friends and I share horror stories over having to ask the powers that be for a data budget.
But here’s the truth: market data is expensive. Index providers like S&P and MSCI make a killing on selling their proprietary data. And while the Bloomberg terminal is extremely hard to disrupt for many reasons, the universe of quality data is near the top of the list.
You will always pay through the nose for the best clean, trusted, robust raw data. And clean, trusted, robust data matters when client money is on the line. AI can’t guarantee this, at least with publicly available data.
Also, even the best data has its flaws that only eagle eyes can pick up on. Recently, I pulled historical volume data for a universe of 500-odd leveraged funds for this research piece, and found that a ticker of an inactive leveraged fund was actually the existing ticker for a new ETF. The historical volume data included both funds. I threw it out.
Don’t get me wrong: data is becoming more accessible, and the advent of AI will help tear those walls down.
I do think, however, that some walls will grow higher, and mainstream access to cheap quality data will be fragmented and spotty. It’ll be extra important to provide your own (expensive) data to an LLM or know its sources in and out.
Not only that, but data can be precious. Shout-out to my business analysts who handle sensitive customer data and wouldn’t dream of feeding it through a public LLM. Data protection issues are a big deal, and I can only imagine there are unintended consequences ahead of us who share too much with an LLM.
Research is a process.
Remember when your fourth-grade math teacher asked you to show your work?
They wanted to see how you got your answer. But more importantly, they wanted you to think about each step of the equation.
Research works the same way. There’s value in working through every step.
Questions lead to more questions. You poke holes and uncover blind spots. You adjust your focus.
The refining isn’t just a phase. It’s the whole point.
Speed through the process, and you miss the story.
When you use AI, you skip the process. And while you can ask the LLM to show its work, the burden of proof is on you. You may never know what corners were cut or which loose ends were never tied up.
A colleague of mine ran the SpaceX IPO through Claude for a summary of the high points. A good idea, because who has time to read through 300 pages of 8-point type?
Claude spit out some decent takeaways, but ultimately admitted (after some grilling from him) that it only analyzed a tenth of the document because we hit a data cap.
C’mon, Claude.
Research is squishy.
Research isn’t just crunching numbers.
Research is about encountering squishy situations with viable solutions. And to do that, you have to make qualitative decisions to answer quantitative questions. Left brain and right brain, hand in hand.
Should I use total return or price return data? It depends. Is this a goods or services sector? It depends.
AI can pull in expert opinions to answer these questions, but often, the choice is based on the context you’re working in.
Let’s say a worried client comes to an advisor and asks if they can trust bond ETFs during a crisis. Sure, I have my opinions, but I need to show my work and quantify the risks. Bond ETFs haven’t melted down en masse before, but they have acted weird because of market and fund-specific quirks.
So if I’m looking through history at bond ETFs’ stability in past crises, I have to define the bond ETF universe, what a crisis is (a recession, a jump in interest rates, all of the above?), and what stability would look like (in this case, I’ll say the average intraday price of the ETF relative to its net asset value). Also, good luck getting historical NAVs for bond funds for free.
A good analyst works through all of these definitions thoughtfully, because they matter for the output and the conclusion. And they understand that the qualitative parameters they’re defining must be consistent and sound.
If you’re putting your name on a piece of research, do you feel comfortable leaving these squishy judgment calls up to AI?
Research is personal.
The best analysts chew gum.
They go to Tuesday morning yoga. Sip a latte at the local coffee shop. Walk their dogs, chat with neighbors, grab a beer with friends.
They move around the world like every other human, except they’re constantly taking notes and connecting dots along the way.
I don’t see the world like you do. Because while I’m not sure who’s reading this, I am pretty sure they’re not a 34-year-old mom of two who grew up in Burlington, N.C., studied journalism at UNC, lived in NYC for a spell, and runs/cooks/wrestles with a pug for fun.
People don’t pay for my Excel tricks. They pay for my perspective, and only I can bring that to the table.
Investing isn’t scientific, and if it were we’d bang it out on a spreadsheet and call it a day. A chart of inflation rarely calms the rattled nerves of someone who watched their retirement portfolio get cut in half.
The synthesis is where your humanity shines.
I don’t think AI will ever replace a good human analyst.
But I do think AI can help in human-driven research.
Use it for simple questions. Editing a newsletter you wrote. A gut check on a calculation. A friend when you’re writing by candlelight. I kid, but I kind of don’t?
Here’s a great use case: searching expense ratios for funds by ticker. Ask for the source to make sure it’s reputable (Morningstar, the issuer’s website).
Train your own LLMs. Input your own data. Make sure you have sole control over what’s going in it, mainly so you can have a handle on data quality and approach. This matters a lot for compliance and data protection.
Work on prompting. If you’re willing to rely on broader market AIs, be very specific in what you ask. Ask the LLM to show its work. Assumptions lead to misunderstandings.
Learn to write. Or speak. Or paint. Feed the expressive beast inside of you. You need to harness your relational tendencies if you want to survive in this brave new world of AI. If you’re an analyst, storytelling has never been more important.
Thanks for reading!
Callie
Like what you just read? Share it with a friend, pretty please 😊