A Co-pilot with the right stuff
How does a Customer Service Agent give their undivided attention to 4 people at the same time? With the support from a Co-pilot that scours years of conversations to find the best answers to customer questions.
Through job shadowing and research we found that AI training was failing to scale, and that a different kind of experience was needed. Our insights led to an innovative new AI product that puts the agent back in the captain’s seat.
Sneak peek – Co-pilot, an AI powered console.
It’s AI learning engine offers answers to agents so they can focus on great service.
Integrating into systems of record, Co-pilot’s Learning Engine can scan 500,000 documents in under 2 seconds, to find the right answers for any context.
Amelia can take over conversations to free agents to work on the more challenging requests.
A collaborative platform for human & digital service agents.
Co-pilot
Watching corporations struggling across the board to successfully implement AI is a challenge for the entire industry.
I kept thinking ‘How can we make this easier?’
Then I had a conversation with a client who really knew her business inside out, and a broader strategy started to reveal itself.
This is a “Zero to One” case study - As characterized by Peter Thiel - Zero to One describes the process of creating something radically new and taking it to the first step.
How do you identify the best experts to train Amelia?
Training AI in a vacuum simply does not work. Companies need to shorten the distance between accurate subject matter expertise, and automation at scale.
Having founded the Linguistic team at IPsoft, we soon found that their job tasks needed to be revised because the pace of change in technology was increasing faster than they could keep up with.
New product enhancements introduced a slew of technical hurdles that delivery teams had to navigate quickly. We created new roles like Conversational Experience Designers to focus on the ‘UX’ of the conversation, while Intent Recognition Designers would build out the architecture of the intents and entities (how the AI understands intentions), to scale the roles of the digital human over time.
Clients time to value still takes too long.
All these changes still weren’t enough. We decided to conduct research into our delivery operations and see if there was a more efficient way to train Amelia. We focused on knowledge workers and subject matter experts, working backwards from customer service agents.
Documenting AI training methods was critical for knowledge sharing across teams in other geographies, and to enable client centers of excellence. (It also highlighted the laborious nature of the work).
Even with vast improvements to Amelia’s language usage, consistency and intent recognition, our approach had room for improvement. We needed a technical solution.
Humans + Avatars
What’s it like to be a customer service agent?
I conducted field research to really get to understand both the agents perspective and job role, and listen in on the customer calls to get a handle on their needs.
It was fascinating watching the agents at work. While the company had done their best to set them up for success, most of their job was improvisation, real-time problem solving, and figuring out how to hack the corporate system to get the right answer.
Agents juggle 3 or 4 people at time. They talk with more customers than anyone in the organization.
Great ones are masters of multi-tasking, yet make you feel like you really matter to them.
They are overwhelmed, and under appreciated. It’s a thankless job.
Many agents suffer from undue stress, and depression. The role has an extremely high turnover rate.
Research objectives
Observe the day to day job of Customer Service Agents.
Assess usage of the Amelia platform, (how customers were escalated to the agent).
Discover measures of success for agents, supervisors and the organization.
Discover areas of improvement and delight.
51 participants
Agent, Supervisor, Cognitive Designer, Program Manager, Digital Coach, Process Analyst, SME, Workforce Manager, UX Researcher, Data Scientist, Director of Workplace technology, VP of IT, Manager of Call center, Insurance Service Analyst
8 countries
USA, India, Romania, Egypt, Spain, Philippines, Costa Rica, UK
5 industries
IT, HR, Banking, Insurance, Teleco
A supervisor in a large banking firm. They track hundreds of conversations a day, and often need to help deal with critical issues, VIP customers, and system wide failures – its a stressful environment.
AI training tools
Active conversations dashboard.
Agents were struggling.
Which customer in this active conversations dashboard is the most important one to address right now?
R&D’s strategy was focused on 'complete automation,' which meant the onus was on solutions teams to perfectively train Amelia on every scenario, (an impossible task). Little thought had been put into the whole customer journey, including how a human customer agent would accept escalations from Amelia when she couldn’t deliver the service required.
The active conversations dashboard was difficult to read, ran on the entire core SaaS platform (loaded the entire brain) so performance was very poor, and wasn’t easy to navigate or understand which customers had been waiting the longest.
Admin tool - agent chat.
Find ways to infuse customer centricity into the tools.
The chat on the left is the only part of the agent chat tool that customer service agents use. What could we do with the righthand side of the screen?
The original tooling had been designed with demo’s of Amelia in mind. The 2/3 section on the right side of the chat screen showed how Amelia’s brain was making connections between language – this was great for client presentations, but not so helpful for agents juggling 3 to 4 customers at a time with limited screen space.
We started to think about how we could repurpose this area to give more value to the agent. Our competitive research gave us some good pointers. We also started to think about how could we leverage NLP capabilities.
We really needed to understand the role of the agent at a deeper level.
Existing agent tools
Incompatible features to the job task.
Whole sections of the current solution were not used by the agents. Recommend to repurposing screen real-estate.
Wild west on consistency.
Agents cut & paste responses from their own hand written answers in 3rd party Notepad applications. Recommend a personalized Notepad to store own & team scripts/phrases, etc.
Hacked in functionality.
Self-made timer hacks don’t deliver desired transparency. Recommend customizable timers for agents & supervisors.
3rd party apps don’t integrate.
Most agents use 3rd party apps to view or gain control of users’ screen, which meant the chat platform was sidelined . Recommend enabling co-browsing/remote control.
Existing managers tools
Hacking 3rd party software to identify insights
Reporting is manual and hard to update. Have to use 3rd party software to ‘hack’ the data into visualizations.
Out of date reporting the moment it’s sent
Supervisors have to generate daily CSV data dumps, pivot the data in Excel, then cut & paste daily emails to their team to communicate performance goals.
Time-intensive analysis with little ROI
Reading past chat transcripts is nearly impossible, as it’s a line inside a CSV file. These CSV files often have over thousand lines.
Data-viz does not deliver value
Existing data-viz was seldom used for actual monitoring individual performance because of irrelevant stats. Team performance is not available. Have to hunt through several screens to get a better picture.
Journey mapping of Agents, Supervisors, and Coaches
After conducting interviews, numerous job shadowing exercises, and follow up reviews with SME’s, we formulated three journey maps: 1) Customer agents, 2) Supervisors, 3) Digital coaches. This exercise helped us see the breadth of the solution we needed to cover.
Initial sketch of a modular approach to UI. Each column is customizable, allowing clients to install their own features.
Customer service agents talk to customers more than anyone else in an organization. Their untapped insights can be transformative.
Many of the seasoned agents we interviewed had cobbled together their own cheat sheets of answers to typical questions. These ‘scratch pads’ were in many cases a simple text edit file full of hundreds of answers the agents had figured out over time; they were not connected to anything, not saved in a shared data base, did not conform to any official rules or procedures, nor were they approved or managed by the company.
Co-pilot aggregates and surfaces wisdom allowing the enterprise to benefit from customer knowledge in new and desirable ways.
Agents collaborate with the AI in new ways. Co-pilots Learning Engine can scan over 500,000 conversations, documents, and business processes in under 2 seconds to offer answers to the agent so they can respond faster and more efficiently. As they agents complete chats with Co-pilot they automatically train the AI to handle new use cases. With this combined approach companies can deliver exceptional customer service at scale.
Initial results
Usability testing of the new design against the existing core platform.
Design principles
Inclusion, craft and attention to detail are so important when you’re making anything great.
Put people first – the user, the client, and your team.
Consider use cases beyond your comfort zone – thinking ahead can help those sprints.
Pay close attention to every detail, it matters – functional as well as visual.
Some tools we used – Sketch, Zeplin, Invision, Principle, Storybook, Confluence, JIRA.
Strategic for the future
The goal - Build a full suite of tools to help the organization create a fluid and dynamic means to measure and adjust customer service, sharing insights and knowledge from across the enterprise.
Every member of the company can tap into the knowledge base or add to it. Over time the it becomes a single source of truth helping to retain institutional knowledge, optimize best practices, invent and test new services, and measure overall performance,
Platform workflow
Amelia - End User
Scalable Al agent
Omni-channel availability 24/7
Integrated overlay UI that manages both
Web & Native mobile apps dataRich interactive components that enhance the user experience
Agents
Agents receive contextual recommendations on best actions to perform (from coaches and the Al)
Messaging between agents/supervisors
Al intelligence integrated into the core of the platform (Al can manipulate the data & functions)
Al learns from every interaction feeding
Amelia new trainingSupervisors can message the entire group, or an individual agent
Coaches learn which behaviors deliver the best outcomes
Supervisors
Supervisors can track agent/team pertormance in real-time
Can see trending topics as they happen aggregating language into intuitive visualizations
Al Intelligence recommends best actions for agents, supplying notifications
Can easily generate reports
Can make recommendations on improvements in agent/team performance
Coaches
Analyzes content & usage of the Al/ agent generated content, annotating and modifying utterances, recommending improvements in conversational approach for both human and digital agents
Process design & governance
Service growth planning
Launching Co-pilot
This is what a fantastic team looks like! 😊
We launched a Beta version of Co-pilot at the Digital Workforce Summit, an AI conference held annually in New York city. We bootstrapped the entire experience.
The team did an amazing job building out a live use case that demonstrated the power of Co-pilot and it’s learning engine.
The day was a lot of fun, and some of the team who’d never spoken with clients before really learned a lot, and made some new connections. We received a surprisingly large share of the overall attendee traffic, landing 61 prospects many of whom were Fortune 500 companies.
The interest helped us generate additional funding, but most importantly more great conversations with clients. The feedback from prospects was encouraging – they particularly liked the simplicity of the product, and the overall positioning.
