Security, meet CX: Why we invested in Strivacity

These days, consumers are more likely to enter their favorite businesses through a digital front door than a physical one. This creates a unique challenge for companies: how can they ensure a seamless and easy online experience for their customers while also keeping their customers’ personal information secure? It’s a delicate balancing act between providing convenience and safety.

On one hand, the CMO wants to make the registration and sign-in process as easy as possible, removing friction and allowing customers to create accounts, sign in, and use their services in novel ways. On the other hand, the CISO’s responsibility is to ensure there are no data breaches or hacked accounts. This tension creates an extremely difficult business problem in today’s digital-first world. Companies have two options:

• Tighten the system too much and you see new customer registrations, usage, and conversion rates plummet.
  • One leading national hotel chain implemented one-size-fits-all multi-factor authentication (MFA), increasing friction for their users: revenue fell 11%.
• Relax the constraints and your brand risks being a front-page headline and losing customer confidence.
  • Account takeover (ATO) has become an increasingly prevalent issue in recent years. Account takeover fraud losses reached $3 billion in 2022, according to Javelin Research—and that doesn’t include reputational damage.

Enter Strivacity—a new kind of customer identity and access management (CIAM) software provider, designed around a cloud-based “configuration-as-code” offering. Because of its architecture, business-critical changes can be made without needing expensive and time-consuming engineering resources. This allows the CMO or Head of Digital to experiment and iterate quickly on customer experience (CX) workflows, while the CISO can rest assured that the underlying authentication system is secure and automatically kept current with associated enterprise infrastructure. Due to this flexibility, Strivacity can ensure its CIAM platform will meet the needs of the enterprise C-suite and revenue-function owners. This makes it unlike most existing solutions, which are tailored almost exclusively to the technical teams inside a large organization.

That’s why SignalFire is excited to lead Strivacity’s latest $20M investment round.

One size doesn’t fit all

 

Strivacity understands that workforce identity and customer identity solutions have very different security and user experience needs. Instead of trying to cram these solutions together, Strivacity focuses solely on the needs of customer identity management and has been deliberate in its approach to building a seamless CIAM platform. From a single console, an administrator is able to control and orchestrate registration, sign-in and authentication, identity verification, consent management (for data privacy), and ongoing fraud prevention. Other providers have stitched these capabilities together via M&A alongside an alphabet soup of modules for workforce identity management (PAM, IGA, IDTR, etc.), resulting in disjointed architectures and significant engineering and consulting maintenance costs. In fact, during our due diligence, one enterprise customer estimated that Strivacity provides a 30% annual total cost of ownership (TCO) reduction versus their legacy vendor.

“We started Strivacity because we saw a familiar and frustrating story replaying itself over and

over at Fortune 1000 companies. Workforce-centric identity solution providers were force-fitting their products to serve customer use cases. The results were invariably monthslong rollouts that ended with organizations settling for poor customer experiences and writing big checks for after-market services,” said Keith Graham, Strivacity’s co-founder and CEO.

An all-star team for identity security

Strivacity’s founders, Keith Graham (CEO) and Stephen Cox (CTO), have been working together on enterprise cybersecurity solutions for more than ten years, most recently leading teams at SecureAuth and Mandiant. In fact, Kevin Mandia (founder and CEO of Mandiant) was so impressed with the product vision and early customer traction, he invested and joined their board of directors. 

Jack Huffard (co-founder and former COO of Tenable) invested alongside SignalFire, and joined Strivacity as Executive Chairman. We have been working with Jack as part of our XIR program for about a year to find an innovative cybersecurity company that was re-imagining “identity” solutions for the enterprise—and to our good fortune, we found Strivacity.

“With Strivacity, improving security with customers doesn’t require increasing friction for users. They’re pioneering a new approach to customer identity and access management (CIAM) that’s synchronized with marketing to let enterprises optimize both in tandem. Between SignalFire’s recruiting and growth help, my experience scaling Tenable into a public company, and the Strivacity founders’ vision for the future of CIAM, we can vastly improve the security and experience customers have with the brands they trust.”
—Jack Huffard, Chairman of Strivacity, Co-Founder of Tenable (NASDAQ:TENB), SignalFire XIR

As they were looking for their next investor, Keith and Stephen wanted a partner who would actively help them scale the business. Keith shared:

We are thrilled to be partnering with this team and co-investing with Todd Weber (former CTO of Optiv), partner at TenEleven Ventures. With this unparalleled group of cybersecurity experts around the table, and an innovative approach to orchestration, we believe Strivacity is the future of CIAM.

The market is voting…

 

We’re not alone in our opinion of this product and team—industry analysts and early customers are raving about them as well. Strivacity is the only startup recognized as a Leader in The Forrester Wave™: Customer Identity and Access Management, Q2 2022. And on the customer side, leading enterprises across gaming, online education, financial services, and consumer software rely on Strivacity to manage their customer registration, sign-in, and privacy workflows.

Strivacity is faster to deploy, easier to support, and more comprehensive than any other CIAM solution in the market. The balance of simple and customizable CX with strong cybersecurity is a delicate one. With Strivacity, that balancing act becomes cheaper, faster, and easier to achieve. Finally: something that both the business side and the IT side of an enterprise can agree on.

---

*SignalFire may engage Affiliate Advisors, Retained Advisors, and other consultants as listed above to provide their expertise on a formal or ad hoc basis. They are not employed by SignalFire and do not provide investment advisory services to clients on behalf of SignalFire. For more information on their specific roles, please contact us. Portfolio Company Endorsements: Certain portfolio company founders or Affiliate Advisors listed above may or may not be current investors in a SF fund in which they receive a fee reduction. Such fee reductions were not provided in exchange for or an incentive for their feedback, nor contingent upon the individual’s approval for SignalFire’s continued use. Please refer to our website for additional disclosures.

The Patient Engagement Revolution: Why we led Wellth’s $20M Series B

Healthcare costs Americans a staggering $4.1 trillion dollars each year. But what actually drives all that cost? One glaring, seemingly avoidable factor is patient behavior. Six in ten adults in the U.S. have a chronic disease— the conditions that lead to 86% of our healthcare costs. Individual behaviors—attending doctor’s appointments, taking medications as prescribed, and “knowing your numbers” are crucial in managing these chronic conditions. In fact, $300–500 billion each year is wasted just as a result of prescription medication non-adherence and mismanagement, according to the National Institutes of Health.

That’s why we’re leading a $20M Series B round into Wellth, the leader in leveraging behavioral economics to drive adherence to prescribed care and treatment plans. You can read more about the news from Wellth and Fierce Healthcare.

As mentioned above, six in ten Americans suffer from a chronic health issue—including both authors of this piece. That’s more than 200 million of us dealing with ongoing treatment programs. To reduce the waste from non-adherence, Wellth is pioneering better ways to incentivize this group to remain steadfast in their treatment plans, focusing on a massive needle mover: motivation. Specifically, engaging and motivating the population that is most at risk of going astray from healthy decision-making—the 25% of Medicare beneficiaries that incur 96% of the costs(!).

Behavior: The true unlock

 

Forty percent of health outcomes can be linked directly to personal decisions—not genetics, not quality of care, and not social determinants like income, according to the Kaiser Family Foundation. We all have friends or family who know they need to get to the gym to be healthier but can never quite stick to it. There’s a big difference between knowing what we need to do to take care of our health and actually doing it. Especially for senior citizens and lower-income Americans—people who may not have the time, energy or motivation to keep up with daily health choices—leading patients to disengage from their care plans.

Disengaged behavior takes many forms: skipping appointments, forgetting medication, not checking key health signs frequently, etc. These may sound like minor slip-ups, but they can often cause extremely harmful (and expensive) effects down the line. Consider:

• Heart attack: Patients who don’t take their heart medications after a heart attack are 2x more likely to have a second heart attack in the next three months.
• Kidney disease: Patients with end-stage renal disease (ESRD) who miss one dialysis appointment are 3x as likely to end up in the hospital (and have 50% higher mortality).
• Diabetes: Patients with diabetes who don’t check their glucometers or miss oral insulin prescriptions can be unaware of their blood sugar being either far too high or far too low. There were more than two million diabetes-related emergency room visits annually in the U.S. in 2021.

How Wellth helps

 

Wellth uses behavioral economics to drive better outcomes, primarily through a monthly rewards system. Patients use the Wellth app to document daily healthy behaviors (like taking their medication as directed)—and depending on the activity, a fraction is deducted from their reward if it’s not completed. 

Wellth’s programs are delivered by leveraging key behavioral psychology principles such as:

• Loss aversion: A $10 loss feels worse to us than how good a $10 gain feels
• The intent-behavior gap: What we plan to do doesn’t translate to what we will do
• The endowment effect: We value things more when we own them

Wellth partners with some of the nation’s largest health insurance companies (e.g., United Healthcare, Centene) to motivate their most vulnerable members. Wellth reaches out to these people, gets them to download the app, and motivates daily action with their treatment plan.

The rewards members receive are subsidized by those health insurers, and depending on the program, they are triggered by different user inputs—it may be snapping a picture of pills in your hand, showing a glucometer reading, or confirming presence at appointments. In each case, it’s as easy as texting a photo. That’s why Wellth has engagement rates that are completely unheard of in healthcare. 91% of Wellth’s users use the app every day. That’s 3x the engagement rate of TikTok’s user base.

In short, Wellth uses modern technology to get people to actually make the decisions that they know they ought to make, benefitting both their physical health, and the health of the U.S. healthcare system as a whole. Wellth’s co-founders, CEO Matt Loper and CTO Alec Zopf, have kept their mission of a healthier America front-and-center as they built this company together, leveraging expertise from prior careers in healthcare advisory and data engineering.

The proof that Wellth works

 

Great, so people log in and use the app. But that sounds like we’re spending more money, doesn’t it? Where’s the proof that people engaging with the app actually saves money downstream, by avoiding ER visits and the like?

Having served more than 30,000 patients, the outcomes are nothing short of a revelation:

• Missed appointments: Decreased 15% (keeping the system moving efficiently)
• Emergency room visits: Decreased 29% (ER visits are incredibly costly)
• Days spent in the hospital: Decreased 42% (hospitalizations are even more costly)
• Annual savings per patient: Over $2,500
• Total savings to the U.S. healthcare system: $50+ million to date

Across the estimated 30 million polychronic and disengaged individuals in the U.S., Wellth could eventually reduce the cost of the U.S. health system by over $75 billion.

Why Wellth chose SignalFire

 

Before investing in Wellth, SignalFire spoke to more than 50 companies in the patient engagement space. We determined that patient behavior change around medication non-adherence was the most important problem to solve. When we saw the engagement rate, the increased adherence, and the savings to health plans Wellth was generating, we knew we’d found a winner.

Every investment is a two-way street. So once we decided to work with Wellth, they had to decide to work with us. When asked why they did, Wellth’s CEO Matt Loper told us:

“Every venture fund I’ve ever met has promised to add value beyond capital—but SignalFire is the first I’ve known to exceed all expectations. They uniquely understand the challenges founders face and have purpose-built their team to help them overcome those challenges. Every team member is an expert in their field that goes above and beyond to help their portfolio companies. Chris Farmer and Chris Scoggins have seen firsthand how companies define industries and scale. Tawni and Heather help us think through how to best grow our team. YY and Tony immediately became a key extension of our executive team. And of course, there’s SignalFire’s XIR Frank Williams, who is a force of nature and would be a first-ballot inductee into the Healthcare Hall of Fame. Working with Frank and the whole SignalFire team has already proved to be a dream come true.” —Matt Loper, CEO of Wellth

Matt is referring to the unique Executive-in-Residence (XIR) program that SignalFire designed for situations just like this one: pairing A) startups serving a clear market need and with the potential to be generation-defining companies, with B) deeply experienced industry leaders in their field who have founded and scaled massively influential companies. 

In this case, to help find a company working on a solution to the patient motivation problem, SignalFire partnered with XIR Frank Williams, former CEO of the Advisory Board Company (acquired by Optum for $2.5B) and founder/CEO of Evolent Health (NYSE:EVH, $3B+ market cap).

Williams, who is joining as the chairman of Wellth’s board of directors, shared, “I haven’t seen a venture firm that has made such a comprehensive investment in supporting entrepreneurs in scaling their businesses successfully.” He added:

What’s next?

 

Health outcomes are driven by behaviors, and Wellth is fundamentally evolving the way we think about healthcare behavior change. We are incredibly excited to be a part of the journey alongside Matt, Alec, and the entire L.A.-based Wellth team, as well as their new chairman Frank Williams. There are so many systemic issues in today’s healthcare system that it can feel impossible to change, but Wellth is showing us that with technology and an innovative approach to patient motivation, we can make a real difference.

And they’re hiring! If you want to help make a change in the way we take care of ourselves in this country, reach out!

–

*SignalFire may engage Affiliate Advisors, Retained Advisors, and other consultants as listed above to provide their expertise on a formal or ad hoc basis. They are not employed by SignalFire and do not provide investment advisory services to clients on behalf of SignalFire. For more information on their specific roles, please contact us. Portfolio Company Endorsements: Certain portfolio company founders or Affiliate Advisors listed above may or may not be current investors in a SF fund in which they receive a fee reduction. Such fee reductions were not provided in exchange for or an incentive for their feedback, nor contingent upon the individual’s approval for SignalFire’s continued use. Please refer to our website for additional disclosures.

Funding Fixie’s LLMOps to unlock AI for enterprise

We’re missing the true value of large language models by keeping them stuck in a chat box. LLMs will be a staple of tomorrow’s machine learning startup opportunities, but only if their impact can be felt where we already work today.

Not long after ChatGPT became a thing, people discovered LLMs could do more than just predict the next word for a sequence of text. Given instructions or when asked to solve a problem, the language model would iterate step-by-step to solve that problem. People started to build on top of LLMs. While the models began with text autocomplete, they’re now used to break down problems into subtasks and execute them one-by-one. 

Yet what the language models lack today is the ability to connect with external systems. The genie is out of the bottle. The models are out there. The question is how to use them in the context of an enterprise while not violating standards for privacy and safety.

Large model APIs help engineers get started faster. They take them for some part of the journey but don’t carry them the last mile to the destination. What corporations want to accomplish is usually quite specific, and building a custom solution often makes the most sense. As Ines Montani, the CEO of Explosion, a SignalFire portfolio company, says: “Eventually, the large model will be one part of the toolbox, and ‘surprisingly good’ won’t be good enough; you’ll want something ‘better’.” Her co-founder, Matthew Honnibal, adds that “ultimately users care about how high the ceiling is, not how the high floor is.”

Tomorrow’s machine learning startup opportunities

Building powerful apps on top of large language models with Fixie

 

Currently, there is a flurry of LLM startups, many of which are using LLM models without any fine-tuning: they just stuff as much info into the prompt and allow the model to take it and use it to refine itself. While this is the quickest way to make use of this space, we believe it won’t scale very well. Differentiation comes from the data moat, the extended capabilities of the product, and the tastes and elbow grease of the humans building these products. For enterprise, we believe that customers want flexibility when using any ML model and any framework. They want the experience to be provider-agnostic, hosted wherever they are, and without vendor lock-in. This is why we invested in Ivy to unify all ML tooling, spaCy/Explosion for providing an Open Source NLP toolbox and annotation system, and most recently we invested in Fixie as the pioneers of LLM adoption for enterprises. 

Fixie provides extensions for language models to access external systems, allowing people to ask questions, get responses, and take action. With Fixie, customers can build natural language agents that connect to their data, talk to APIs, and solve complex problems. It’s designed from the ground up with enterprise customers in mind, offering them maximum flexibility and robustness that is necessary to serve enterprise use cases. If you are a company who wants to add LLMs to your application, reach out to learn more or play with the product https://www.fixie.ai/. 

We at SignalFire are excited to partner with the founders of Fixie, backing its $5M pre-seed round ahead of its new $12M seed round. Read more about it from Fixie, TechCrunch, and GeekWire.

The team has a strong background building large-scale systems and AI-powered products for billions of users. Matt Welsh, the CEO, was a professor of computer science at Harvard (one of his students was Mark Zuckerberg). After nearly killing Facebook, Matt spent time at Google, Xnor.ai, Apple, and OctoML. Zach Koch, the CPO, is a former product director at Shopify, and was previously a product lead at Google on the Chrome and Android teams. CTO Justin Uberti was the head of the Stadia, Duo, and Hangouts Video teams at Google, and was one of the inventors of WebRTC. Hessam Bagherinezhad is the chief AI officer, and he was an AI/ML leader at Apple and the first employee at Xnor.ai.

SignalFire loves working with experienced AI teams because we build AI ourselves. For the past decade we’ve had a half-dozen engineers working on our Beacon AI data platform, which helps us source investments and assist our portfolio companies with hiring. With all the new AI companies popping up, recruiting top talent in the space can be a challenge. If you’re building an AI company that wants to use AI to find AI engineers, come talk to us at SignalFire or email me at [email protected]. I’m a machine learning engineer, too, who’s willing do anything to help founders succeed.

Why founders and VC Journal call Oana Olteanu a Rising Star

Why founders and VC Journal call Oana Olteanu a Rising Star

Oana Olteanu’s childhood playground was a TR-85 battle tank. While growing up in Romania, her father was a tank mechanic, teaching her how the smallest fix could make a hulking machine come to life. To this day, she loves the feeling of driving a tank. “It makes you feel unstoppable.”

That relentless spirit and willingness to get her hands dirty are why Venture Capital Journal chose Oana for its 2023 Rising Star award. She helps portfolio founders spot opportunities and stays up late on recruiting calls bringing in talent that changes the trajectory of their company.

“Oana is a powerhouse—an absolute force of nature. She has been essential to getting us up and running, putting in a tremendous amount of energy helping us find our first hires, connecting us with customers, and providing intel on the market,” says Matt Welsh, founder of SignalFire portfolio company Fixie. “Everyone who has worked with Oana has similar things to say. We’re thrilled to be working with her.”

But Oana never intended to be a VC. In fact, she still cringes when people call her one.

When Oana hears someone say they’re a venture capitalist, the first thing that comes to her mind is a loud blowhard in a Patagonia vest who talks a big game about providing value but never really shows up. Oana was determined to be different. So she structured her career around a single ethic that SignalFire shares: help founders.

Earlier in her career, a seed founder she’d backed was lagging in their go-to-market motion and was in danger of failing to raise a Series A. Oana compiled a list of 50 potential customers, collected feedback on the product from them, and then found 15 other companies in the sector that were consistently hitting their revenue targets and might have strategies to share. She presented these contacts to the founder alongside an offer to make introductions, noting that founder-to-founder talks always beat out founder-to-investor talks. Rather than ghosting a portfolio company because they were in a rough spot, Oana stepped up to help them help themselves. The founder was able to build a network of peer advisors and capitalize on the new customer leads. Soon they were receiving preemptive interest for their series A.

“It was the founder who moved the chess pieces and got the go-to-market strategy to work, not me. I only advised,” Oana insists.

“Oana goes to bat for her founders more than anyone else I know. She’s put her own blood, sweat, and tears into our company, from sourcing leads for an important position to evangelizing our product at conferences—even connecting us to expert advisors who have built companies facing similar problems we have. Oana is always willing to go the extra mile to help us out and it really makes a difference, especially for us early-stage and first-time founders,” says Galen Marchetti, the CEO of Kurtosis.

Her path to this point was as unorthodox as the results that followed. It’s a journey that started in high school when she helped create a space station.

After falling in love with Star Wars at a young age, Oana was browsing NASA’s site in her computer science class when she saw that they were hosting a design contest to reimagine the International Space Station. She entered and put forward an idea unlike anything the judges had seen before. Inspired by the bees her family kept at home, she suggested they make the space station like a honeycomb. It was modular, yet able to maintain the doughnut shape needed to keep the station spinning to generate artificial gravity.

She won the competition. Oana’s first time on a plane was when she was flown to visit a NASA site as part of her prize. Impressed by their intelligent systems work, she knew she wanted to be in software.

After moving herself to Germany to study computer science and machine learning, she ended up in Silicon Valley while working for SAP. That’s where she was first exposed to the VC profession in SAP’s corporate venture arm. After a successful stint at Scale Ventures, Oana came to SignalFire, where she’s now a partner leading the fund’s developer tool and AI infrastructure investment practice.

Providing measurable value is a simple philosophy and one that’s earned Oana a reputation as one of the most supportive venture capitalists in the industry. When The Twenty Minute VC podcast’s host, Harry Stebbings, asked who the best young partners were in venture, Oana was overwhelmingly recommended—becoming the most suggested by a factor of four. You can hear Oana on the 20VC podcast here.

What makes Oana different isn’t just her rapid rise, but her willingness to roll up her sleeves and build beside the founders she funds. Rather than having a background in consulting or finance, Oana’s an engineer. Calling her a VC undersells her deep technical skills.

“Her expertise and understanding of emerging tech are obvious and she is one of the few investors I’m able to give deep technical details/updates to without further explanation,” says the CEO of a stealth company Oana works with.

“I treat the founders’ time as more valuable than my own because it’s the founders who have the impossible job of building these companies against all odds, so if I can do anything that makes a difference for them, I will do it,” Oana explains. “My job is the easy one.“

It’s this perspective that founders deserve more from their VCs that attracted her to SignalFire. The early-stage fund was the first VC built like a tech company, spending six years in stealth making its Beacon AI system for sourcing investments and helping founders hire before SignalFire launched. SignalFire’s other value-adds include its advisor network, data science team, and in-house experts on growth and PR. It was a natural fit for Oana’s founder-focused mindset.

It all comes back to Oana’s drive to see everyone get their fair shake in the tech ecosystem. Beyond the work she does with founders, she’s also helping more women get jobs in tech or start their own companies with the ShesReady2Dev community she started. 

“Women’s career paths are steeper than men’s, no doubt about it. Talks about influence and growth in the startup world are received with skepticism by many women since too many have faced resistance to their advancement. After working at an all-male startup, I had a firsthand experience with that problem. That’s why I want to even out the playing field in the startup world, so women feel more comfortable entering and succeeding in this space,” Oana writes.

Oana remains steadfast that VCs should never treat capital as their only real value-add and should instead actually work alongside their portfolio companies. The help should always be optional, but founders shouldn’t have to beg for it or hear crickets if everything isn’t up and to the right. You’re more likely to see Oana digging through her portfolio companies’ engineering role candidates than talking herself up on stage. That’s why being named Venture Capital Journal’s Rising Star is both a little ironic…and also deeply fitting.

If you want startup trend reports, founder guides, or invites to tech events, sign up here for SignalFire updates.

Tomorrow’s machine learning startup opportunities

Fads fade. We believe machine learning (ML) is a classic piece of building software. Classics have an undeniable air of timelessness. They are hard to ignore and impossible to miss. Classics are forever. We believe ML is forever for software. In fact, at SignalFire, ML is woven into the software we build. We develop our own ML models in-house for our Beacon AI data platform, which we use for sourcing investments and helping portfolio companies with recruiting. That’s given us a deep appreciation for how ML tooling can improve a business, so we’re eager to invest in and support founders building the future of ML infrastructure.

Hot takes on machine learning and LLMs

1. For the time being, large companies will own model training for large language models (LLMs) and foundation models (FMs). Training requires a combination of proprietary/web-scale datasets and costly infrastructure, making them too big a challenge to be adequately addressed by smaller companies.
2. Companies can open-source models and methodology without fear of giving away their secret sauce since data is often proprietary and infrastructure is tough to build. The secret sauce is not primarily the recipe, but the rare ingredients and huge kitchen required. Facebook’s pytorch may be open-source, but you can’t build Facebook models on your own.
3. Foundation models unlock bigger TAMs for enterprises via additional services. Incumbents can charge higher annual contract value for premium AI features, and there is a high interest in companies like Fixie helping enterprises build with LLMs.
4. Foundation models are industry agnostic, making specialization a growth opportunity. To scale in the commercial medium, one has to fine-tune on industry-specific data, as well as have more control and visibility—for example, the ability to filter for brand safety and content moderation standards.
5. MLOps will be a critical component in the industry for the foreseeable future. For several years to come MLOps will be important for tasks where LLMs don’t perform reliably. And once large models become more efficient, MLOps will be able to fine-tune the models on a task, get data, manage artifacts, monitor, update a model to address failures, and more.

ML is a core component of the software we build and our portfolio

SignalFire truly understands ML because we’ve spent 10 years building our own ML models for many uses across venture: sourcing companies, completing due diligence, winning competitive deals, and supporting our portfolio with recruiting data and customer lead lists. We do not build a “robot general partner,” but rather utilize data to empower human decision-makers. For example, we use natural language processing (NLP) models to classify companies and surface them to the appropriate investors, classify potential recruiting leads for our portfolio companies to match their hiring needs, and use graph ML algorithms on the open-source community to find compelling technical projects, many of which are ML projects themselves!

MLOps or MLOops?

Machine learning infrastructure went from Matlab produced by Mathworks in 1984 to an explosion of tooling to operationalize models, referred to in the industry as MLOps. Up to 2021, $3.8B was invested in MLOps. The category has existed for a decade, and while there are a few companies valued at $1B+—such as Scale.ai, Weights & Biases, and DataRobot—there is no IPO yet. Which begs the question: MLOps or MLOops? Here’s how we think the timeline will play out.

ML infrastructure timeline

We believe that MLOps aren’t disappearing anytime soon. LLMs have raised the awareness of AI in our collective mindshare. While they can be useful, LLMs are not (yet) suitable for all tasks, such as time series forecasting, or where end users can’t afford to be wrong in even one percent of cases, such as self-driving cars. While the latest GPT-4 LLM has expanded to accept audio and image inputs—and other foundation models tackle image (DALL-E 2) and audio (AudioLM) generation—the high cost of inference and model fine-tuning are a barrier for high-volume, low-margin business applications.

For several years to come, MLOps will be needed for tasks where LLMs don’t perform or are cost prohibitive. And even once large models become more efficient, there will still be a need for MLOps to fine-tune the models on your task, get data, manage artifacts, monitor data, update the model when you see failure cases, and update the simpler models that manage spam filtering and toxicity detection, among other things. 

In the category of MLOps, we invested in annotation (Explosion), testing (Kolena), and compute (SaturnCloud). While there is no common tooling yet, there is a common workflow to build models. In the image below, you can see an incomplete sketch of companies operating in each of the categories, illustrating in one snapshot the common workflow for building ML models. It captures notable companies for each step, the different teams involved, and some of their publicly announced total funding. 

This graphic shows how operationalizing an ML model requires a wide variety of different tools and teams with deep expertise. Enterprise companies with the requisite data may still find it hard to staff all these teams with top AI talent—but they have the funds to purchase MLOps.

MLOps startup opportunities

We want to meet founders to solve common pain points for commercial customers who want to add AI skills. If you are building tooling that helps companies use ML to benefit the bottom line, please email [email protected]. 

Three of the areas we are especially interested to meet more MLOps startups are:

1. Visibility across the stack: Customers tell us their ML stacks are heterogeneous and hard to review and audit end-to-end. A head of AI at a large retail chain told us, “We have to work with many companies to get to 75% of what Google has internally as their ML platform.” He would pay top dollar for tooling that can provide the enterprise level of standardization and visibility end-to-end, vs. opinionated ML tooling for a point solution.   
2. Closing the skills gap: As the diagram makes clear, a multitude of different builders have to interact throughout the ML lifecycle. We are interested in startups that bridge the gap in skills between these different organizations as well as tooling that helps the engineers who build models to be more productive. 
3. Data engineering: We have more data than ever, and we are interested in tooling that makes the collection, transformation, and usage of data more efficient.

Native LLMOps 

Large language models are bringing faster time-to-value to enterprises. SignalFire recently invested in Fixie.ai in this space, a cloud-hosted Platform-as-a-Service that enables anyone to build and integrate smart agents that leverage the power of LLMs to solve problems based on natural language. We’re looking to meet more founders building native tooling to use LLMs for commercial use cases, particularly around these three areas:

1. Inference: Inference (applying an ML model to new data) is expensive, and optimizing it would be beneficial to many companies. Businesses need help with inference and also need help understanding the cost. Algorithmia pioneered MLOps; we would love to speak with teams building the next Algorithmia and startups helping build industrial-strength inference.
2. Business-specific context: Foundation models are industry agnostic; we are interested in tooling to verticalize a model, providing the vertical-specific context to the model, such as vector databases. 
3. Commercial control: To scale in the commercial medium, one has to have more control and visibility—for example, a tool to filter for brand safety. ChatGPT is cool but cannot be adopted and used in production if it uses profanity or risks saying something against brand standards that could be screenshotted and shared with everyone.

A VC engineered to help you scale

At SignalFire, we like to say “think of us as an extension of your team that scales with you.” Beyond our in-house Beacon AI, we built our full-time Portfolio Experience team with world-class operators across a variety of functions including the former Chief People Officer at Netflix for developing an engineer hiring strategy, the Chief Marketing Officer at Stripe to optimize your sales process, and the former Editor-At-Large at TechCrunch to help you convert the value you deliver into a persuasive story. Our approach was built around providing value to founders, leading to our net promoter score of 85 among founders, with 85% saying we are the most valuable investor on their cap table.

We are investors, and we don’t have a magic crystal ball; we are open to being proven wrong and we keep an open mind as we’re meeting founders. If you are building in the LLM space, come talk with us—we will share our full research, and we’d like to learn more about what you’re building and earn the option to be at the table when you’re raising. We’re also co-hosting plenty of AI events at our San Francisco office if you have ideas. Cold emails welcomed at [email protected].

Why Signalfire funded CodaMetrix to fix medical billing

Healthcare is under threat of an enormous labor shortage, caused in large part by an aging workforce. Across the industry—physicians, nurses, administrators—many workers are expected to retire in the next decade, with not enough young people entering the profession to replace them. According to the American Journal of Nursing, 4 million nurses are expected to retire by 2030. Add to that an increasing rate of burnout due to COVID. 

One particularly interesting, relatively unknown healthcare profession that’s also been affected is the medical coder. It’s a niche role—there are approximately 140,000 people in the U.S. and an additional 150,000 globally, in the trade. A coder’s job is to take doctors’ notes and come up with a combination of one or more Current Procedural Terminology (CPT) codes, and one or more ICD-10 codes (International Classification of Diseases, 10th Revision) that are used to express the type of care that was delivered to meet payers’ reimbursement guidelines and requirements. There are literally billions of combinations.

Depending on the type of care provided, this could quickly become a very complex task. For example, if you go to your primary care doctor for a routine appointment, it’s usually a highly structured visit where several standard checks are performed, maybe a blood test is ordered, and you’re good to go. But let’s think about something like an inpatient visit: you’re staying in a hospital for a week because you just had surgery; you have multiple, varied interactions with your surgeon, the anesthesiologist, other specialists, and multiple nurses; all over the course of those seven days. The work to record this activity accurately is enormous. The “doctors’ notes” from your weeklong stay can become the length of a term paper. Coders then have to extract the key information to generate dozens and dozens of billing codes, or worse yet in some cases the physicians have to do that themselves, which takes them away from patient care.

Across the U.S., health systems spend a collective total of $9 billion annually on medical coding. And despite the large spend, the current state of coding is challenging—it’s highly manual, and so errors happen— in fact the recent Becker’s Hospital CFO Report shows 42% of denials are due to coding inaccuracies.  Currently, miscoding can lead to costly back-and-forth communications between Providers and Payers, and undercoding can lead to Providers being underpaid for necessary services they delivered to patients.

Medical coding should be largely automated—enter “Applied AI”

Given SignalFire’s strength in data and analytics, AI, and specifically machine learning (ML) and natural language processing (NLP), we were particularly excited about how automation can play an outsized role in the medical coding space for four key reasons:

1. Coders are quickly aging out of the workforce. The average medical coder is approximately 50 years old, and fewer young people have been entering the field. Providers must find a way to scale their coding capacity, while making their existing employees more productive.
2. Coders are humans and humans make mistakes. We’ve all heard hospitals complain about Payers denying some portion of their claims. On the other hand, Payers complain that hospitals try to squeeze more reimbursement from them by billing for more services than were performed—a concept called “upcoding.” In reality, both sides are right, but very rarely do claims get denied because of fraud (intentional upcoding). Instead, the problem is usually human error. In the weeklong inpatient visit example above, it wouldn’t be unusual for a coder to make mistakes and input incorrect codes. Also, the list of codes changes and new codes are released annually or quarterly. Coders need training to keep up with the latest and they have to keep up with their CEU (Continuing Education Units) to maintain their certifications.
3. ML and NLP technology advancements have now made coding automation feasible with high accuracy. Natural language processing technology was not mature enough until recently to tackle challenges for automated coding. Many of these NLP systems required hand-annotated training data, which had poorer quality results. It wasn’t until 2013 that there was a big breakthrough in deep-learning–based methods (starting with a language model called word2vec), which created a step function improvement in accuracy across all NLP tasks. For example, Google Translate was first publicly released in 2006 but was able to improve accuracy by 60% when they switched to a deep learning model in 2016. Furthermore, NLP applied to the medical field saw exponential improvement in 2020 as researchers released deep learning language models focused specifically on medical notes (called BioBERT).
4. More compliant adoption of Electronic Health Record systems (“EHRs”) and highly scalable and affordable cloud computing. US healthcare Providers and Payers alike have made significant investments in electronification of both clinical and billing data. Now, this large corpus of data can be used by fairly reliable, secure, and affordable cloud and AI infrastructure to both clean the data to develop the ground truth data needed to train Machine Learning (ML) models that not only take advantage of more advanced NLP, but also add the pattern recognition and statistical analysis dimension to produce even better results.

Why now?

Between 2014 and 2018 many startups began working to apply the more mature NLP systems to medical coding. Fast forward to 2023 and we now finally see a small cohort of autonomous medical coding companies that have achieved product-market fit but are still in the early stages of crossing the chasm to achieve adoption widely. It’s not a surprise that companies have taken several years to get to this point. It requires an enormous amount of time to obtain a large training data set, train and backtest the models until they work, convince the typical hyper-conservative-unwilling-to-try-any-new-technology healthcare organization to use their solution, and finally integrate with all of their existing workflow solutions—all before any revenue is generated. But we believe the market is finally ready for these solutions, because ML and NLP technology advancements have enabled these solutions to actually deliver the results they claim.

Why CodaMetrix?

When we built a market map of all the medical coding automation companies that have product-market fit (defined as having at least a handful of customers and a minimum of several million dollars of revenue) there were only a few that made the list. At SignalFire, we believe the companies that will win over time are those with products built on a true machine learning approach and with teams that have experience working inside US-based health systems. CodaMetrix was the only company with both. 

ML is the right platform approach to this technology problem. We think it’s best positioned to extend across specialities over time. The alternative to using an ML model is a simpler rules-based NLP system—the concept that if a doctor wrote X, Y and Z keywords and phrases, then output would be specific billing (CPT and ICD) codes. However, the pure rules-based NLP systems do not scale well because changes in documentation style and every subsequent speciality product developed will require a new set of rules to be programmed. When a system relies on a large set of rules, that’s when it becomes “brittle” and will require lots of caring and feeding to function properly. Currently, companies in this space begin with one or two specialities (for instance, CodaMetrix started with radiology and pathology, and has now expanded to offer surgery, inpatient coding, etc.). But as the autonomous coding platforms need to handle a larger number of specialties and subspecialties, the ML approach, vs. NLP-centric, promises to have the chops to handle the complexities.    

However, ML platforms are useless without an abundance of accurate training data. Because CodaMetrix was incubated and launched inside Mass General Brigham (MGB)—one of the largest and most innovative health systems in the country—it has had unparalleled access to large volumes of training data on any specialty. The company’s cutting edge deep learning models can quickly “learn” the terminology and coding guidelines (or “patterns”) of a new speciality, enabling a rapid product development cycle. This allows CodaMetrix to get its new, tested products into the hands of customers faster than the competition.

One important, yet subtle, feature of its platform is “code transparency.” Because the company started from day one as a product built for a health system (MGB), the tech team also creatively designed the architecture to solve for a key challenge in the industry—showing the reasoning behind its AI decisions. Most machine learning models are black boxes, but CodaMetrix has a built-in audit trail feature in their product so one can see what information the system specifically used to reach its conclusion. This is incredibly rare, yet valuable, in machine learning solutions and a must-have in healthcare, where Providers are frequently audited by Payers to determine if they are coding appropriately.

CodaMetrix’s ability to build products tailored for health systems is driven by an experienced team who deeply understand the ecosystem. The executive team, which has decades of experience working inside and with health systems, is led by CEO Hamid Tabatabaie who was previously an Entrepreneur-in-Residence at MGB, and formerly the CEO and Founder of LifeIMAGE. 

The early results are impressive. Today, CodaMetrix is a multi-specialty platform that classifies codes across radiology, pathology, surgery, gastroenterology, and inpatient professional coding with customers across 10 health systems and major academic universities, representing 111 hospitals in 40 states, including Mass General Brigham, University of Colorado Medicine, Yale Medicine, and Henry Ford Health Systems. Customers using CodaMetrix have seen significant increase in cost savings and cash acceleration—from 70% reduction in manual labor, 59% reduction in denials, and improved cash collection by as many as 47 days.

Why CodaMetrix and MGB chose SignalFire

SignalFire is honored to be leading CodaMetrix’s $55M Series A alongside Frist Cressey Ventures, Martin Ventures, Yale Medicine, CU Healthcare Innovation Fund, and Mass General Brigham physician organizations. SignalFire’s proprietary AI data platform Beacon—which tracks 495 million employees and 80 million companies—stood out to CodaMetrix’s CEO Hamid Tabatabaie as a demonstration of our depth in AI and data as one of the key reason why he chose us to lead the Series A, amongst many others.

“SignalFire’s mantra of adding value beyond the check was evident from the onset and further pronounced with every step along the process. The impeccable combination of Yuanling Yuan’’s in-depth industry research, Tom Peterson’s continuous involvement, SignalFire’s in-house AI and data expertise, Chris Scoggins’ infectious drive for scale, Chris Farmer’s refreshing healthcare thesis, and the brilliant team of experts in talent, operations, and go-to-market strategists are clear reasons for us to have selected them as the lead investor.’ – Hamid Tabatabaie, CodaMetrix CEO

We are also thrilled to have the support of Mass General Brigham in selecting SignalFire as a long-term partner for CodaMetrix.

“Mass General Brigham is delighted to have SignalFire lead the Series A and attract a strong syndicate. SignalFire brings to CodaMetrix deep AI/ML and go-to-market commercialization expertise that will ensure CodaMetrix continues to build on those capabilities developed at Mass General Brigham and lead the market in autonomous medical coding to help drive down healthcare administrative costs and reduce physician burnout. In the short time that SignalFire has been involved with CodaMetrix, they have already demonstrated how their focus on data and GTM expertise will add value.” — Gaye Bok, Partner of AI and Digital Innovation Fund, Mass General Brigham

The team at CodaMetrix was excited about SignalFire’s unique XIR program, which pairs deeply experienced tech industry leaders from the firm’s Advisor Network with high-potential portfolio companies to help accelerate their growth. Tom Peterson, SignalFire’s XIR who will be very actively involved at the CodaMetrix board level and advising on day-to-day operations, boasts 20+ years of deep healthcare experience as the co-founder and former COO of Evolent Health (NYSE:EVH) and former executive director of the Advisory Board Company (acquired by United Health and Vista Equity Partners for $2.6B). 

“Given the strength of the leadership team and the connection to MGB, I was really impressed with the depth of CodaMetrix’s understanding of health systems and how that was combined with best-in-class technology to develop industry leading coding automation products. CodaMetrix is the only coding automation company out there that is built for the health system ecosystem by people who deeply understand it.” — Tom Peterson, SignalFire XIR and co-founder and former COO of Evolent Health (NYSE:EVH)

We are incredibly excited to see how the medical coding automation space evolves over the next decade. This solution is coming just in time, as hiring managers in the industry are likely to face a steep challenge replacing a soon-to-be retiring workforce. With investments like CodaMetrix, we see a future where healthcare Providers can save on labor costs, ensure proper billing for services rendered and achieve faster pay reimbursement thanks to a much lower rate of claim denials. All of these will help our overburdened health systems to remain financially solvent as they try to keep up with the rapidly increasing needs of U.S. patients.

 

---

 

*SignalFire may engage Affiliate Advisors, Retained Advisors, and other consultants as listed above to provide their expertise on a formal or ad hoc basis. They are not employed by SignalFire and do not provide investment advisory services to clients on behalf of SignalFire. For more information on their specific roles, please contact us. Portfolio Company Endorsements: Certain portfolio company founders listed above have not received any compensation for this feedback and did not invest in a SignalFire fund. Please refer to our website for additional disclosures.