You’re reading AI at the Top - Our Sunday series on how Fortune 500 enterprises use AI in business.
1. The short version
The company: Morgan Stanley, No. 39 on the 2026 Fortune 500, reported third-quarter 2026 net revenues of $20.3 billion, including record Wealth Management revenue of $9.1 billion.
2. Morgan Stanley tested its AI against its own experts first
Kaitlin Elliott, head of firmwide generative AI solutions, told Scale AI’s podcast how Morgan Stanley started: It raced its AI against a subject matter expert, with about 25 questions and one hour each. “The AI was able to answer all 25 questions in an hour, and the humans weren’t.” That convinced the bank’s control partners.
Next came a 500-question regression suite, rerun after every change to the assistant. After launch, annotators grade a daily sample of real questions (Elliott’s example: 100) against a rubric to catch dips. The setup took nine months and became the template for later use cases.
Advisers and prompt engineers graded answers for accuracy and coherence, OpenAI says. Access to documents rose from 20% to 80%.
Not in the 3Q26 earnings release, OpenAI’s case study or the Distinguished Engineers release: The assistant’s accuracy scores or total AI spend.
Built by Neatprompts from Scale AI’s podcast and the Debrief launch release.
3. How to run Morgan Stanley’s test on your own AI this week
Race first, then grade and diagnose.
Morgan Stanley’s expert race sets the baseline.
Try it: Pick 25 real questions your team gets, then give your AI and an expert one hour each.Label Studio, open source with ~29K GitHub stars, lets your experts grade answers against a rubric.
Try it: Ask your AI lead to set up its template gallery with labels for accurate, coherent and complete.Ragas, open source with ~16K GitHub stars, shows whether a bad answer came from search or the model.
Try it: Ask your AI lead to run its context precision check on your AI’s wrong answers.
Ask your AI lead this: For every AI tool going live, the number of test questions, who graded them and the pass mark.
🌟 Want to be featured in the next issue? Reach out with your best AI use case and we’ll spotlight it.
4. Advisers now build their own agents
Advisers have made nearly 50,000 attempts at building their own AI agents. “Sometimes people build an agent, they use it twice, and then they never touch it again,” said Koren Maranca, AI head for wealth management. The bank’s own agents now take about half of adviser questions first, after 1,000+ experts reviewed their answers for completeness.
DevGen.AI had saved developers 280,000+ hours by June 2025. The bank now says it has saved more than 1.4 million hours of manual coding, translation and mapping, with more than 950 use cases onboarded. We found no dollar value on those hours in what we read.
In October’s Evident AI Index, Morgan Stanley fell four places to ninth of 50 banks.
See how other banks govern AI: Goldman Sachs, Citigroup, Bank of America and Wells Fargo.
Built by Neatprompts from Morgan Stanley, Business Insider, Financial Planning and OpenAI.
𝕏 Know a CIO about to roll out an AI assistant? Send them this before launch day.
Click to Share
5. The system prompt that runs Morgan Stanley’s test on your AI
Paste in your test results. This prompt judges them the Morgan Stanley way.
Use it to check whether an internal AI tool is ready before it reaches more people. It works for any team: Sales, support, engineering, finance, operations or HR.
Role: You are a head of AI quality deciding whether one internal AI tool is ready for a wider rollout.
You only use the test results provided. You never guess a score.
1. Describe the test set: How many questions, who wrote them and whether they came from real users. If there are fewer than 25, write TOO SMALL.
[Click below to open full prompt]Get more such prompts in the Prompting Playbook (free for you)
Whose internal AI should we break down next?
6. Advisers stay, and AI takes the routine work
In March, Morgan Stanley cut about 2,500 jobs, roughly 3% of staff. Advisers in field offices were spared.
Wealth head Jed Finn said in February that AI will help advisers serve more clients, with an agent handling routine service tasks. Trevor Brosnan, who led the DevGen.AI build, told Business Insider the bank has “ongoing demand from all of our businesses to deliver more functionality.” CEO Ted Pick told analysts in October: “It’s a people business.”
7. What to take to your board
Every company we've tracked, in one place. Our enterprise AI database lists the in-house tool, the use case and the measured impact for each Fortune 500 company we've covered.
8. Tools and Resources
Tools
Arize Phoenix: Logs live answers for people to grade, free to self-host.
Try it: Ask your AI lead to set up answer annotations so reviewers grade 100 a day.
Resources
A Field Guide to Rapidly Improving AI Products: Hamel Husain on sorting AI mistakes into buckets, as Morgan Stanley did.
Try it: Run its error analysis process on 50 wrong answers.Task-Specific LLM Evals that Do & Don’t Work: Eugene Yan on grading summaries, which Morgan Stanley tested.
Try it: Use the summarization checks on your meeting-notes tool.Who Validates the Validators?: A 2024 paper on checking AI graders against people. Morgan Stanley leaned on human graders.
Try it: Have one expert regrade 50 answers your AI grader scored.
Sources
Morgan Stanley reporting and releases
Outside sources
✨ Was this issue valuable for you?
Stay curious, {{first_name | readers}}.
PS. Missed the last issue? Read how Cisco uses AI internally
