Upwork is a global, two-sided platform that connects clients and independent freelancers across thousands of skill sets, such as software and web development.

For this 0-to-1 MVP, our goal was to improve client satisfaction and reduce negative subscription cancellations. And here’s why: 26% of clients thought freelancers require excessive oversight after hire. Meanwhile, 58% of clients had 10+ contracts at a time, further compounding the issue when freelancers missed deadlines or didn’t meet quality expectations. Year over year, we saw increased client dissatisfaction, an uptick in NPS detractors, and high subscription cancellations.

In the initial discovery process—including a listening tour, internal research analysis, and external data from McKinsey—the team found that more than 90% of organizations wanted a human in the loop for AI solutions. This data informed our strategy to combine an AI agent with an internal Upwork recruiter, helping us balance the needs of top-tier Business Plus clients with the internal cost of human recruiters.

So how did this thing work, exactly? Well, when a back-end signal indicated that a project was at risk—a missed deadline, for example—Upwork’s LLM Uma crawled the existing project metadata to find comparable freelancers who were available immediately. Uma created a shortlist for the client to choose the best freelancers, then generated bulk messages to remove some of the time-consuming admin work. The majority of clients self-serviced via the happy path and replaced their freelancers with Uma’s help; for the remaining 10% or so, a human Upwork recruiter stepped in with the same offering. This way, we improved client satisfaction without significantly increasing business costs.