Product Design

Conversation Design

Walk through my design process to redesign the landing page for a

Bot Optimization product

dashboard to redesign

Approach to redesign to meet targeted persona needs

1. Why change?

1. What’s better?

• At a quick glance, the user doesn’t learn if the metric performance is above or below target. The number has to be read to get an indication on performance.
• The donut chart colours are more dominant, which do not contribute to the perception of performance being good/bad.
• With the donut chart, it’s not obvious what the metric ranges are. Here, the metrics being displayed are custom metrics. To reduce the learning curve, readability is important.
• UI for current value displayed (in red/green) is not accessible.
  

• In one take, the user can gauge the bot's health.
• Using colour psychology principles, semantic colours like red/yellow/green give a clear indication of the performance being bad/acceptable/good.
• Clear display of what the metric ranges are and where the bot’s experience & automation performance stands against the target and maximum value.
• Ability to customize gauge charts.
• Use of icons to make current metric value accessible.
  

2. Why change?

• Trend analysis for metrics on experience and automation need not be seen together.
• The only value of looking at experience and automation metrics at the same level is to know how they’re faring against each other, which is met by the dial changes.
• Both trend lines are solid lines, making this graph inaccessible.
  

2. What’s better?

• Helps focus on Experience and Automation metrics separately, which is important to get a single view of the KPI that matters.
• It’s more important to visualize how the metric is trending against the target instead of looking at the metrics trending against each other.
• Added pattern distinction to increase it’s accessibility.
  

3. Why change?

• The Conversational Volume is the most important piece of information here.
• How the conversations are handled is useful, but the restriction with showing “contained” and “handed off” metrics in the dashboard are:
• “Contained” is a sub-set of Automation score, so showing this detail here is redundant.
  
• “Handed off” was not a valuable metric here because for some clients, there was no live chat data in the product, so there was no way of knowing whether the handoff successfully took place.
• The page right below the dashboard focuses on “Conversation Distribution”, which has a more accurate and valuable funnel of conversation handling.
  

3. What’s better?

• The most important bits for conversation volume are covered:
• Volume #
  
• % change in volume
• Volume Trend
  

4. What’s missing?

• No way to know RoI on bot program cost.
• No way to know what’s causing the gaps in metrics.
  

4. What’s better?

• Introduced a metric for Cost per Automated Chat, through which savings provided by the bot program can be known.
• Introduced a way to visualize what’s causing gaps in automation and experience, and what the impact of each of those signals are.
• Palette of decorative colours for waterfall chart carefully chosen to meet different colour deficiency needs.
  

Having decided what pieces of information need to go into the redesigned dashboard, I did some sketches to think through UI layout.

What I finally decided on was...

redesigned dashboard v1

This was the first version we published.

What aspects were well received?

• The metric dials serve as an anchor, setting the reference point for the bot’s performance.
  
• Progressive disclosure of information.
  
• When you click on a metric from the top panel, more details about the metric are populated in the second panel.
  
• With this approach, users can focus on KPIs that matter most to them.
  
• At a quick glance, users can note if the metric is performing well or not, which will help them decide whether that metric needs to be looked into further. Users also get a quick view on what’s affecting the performance.
  
• More inclusive design.
• Easier to read left to right meant it was easier to capture metric readings.
  

Demo’ing this version helped gather valuable feedback on what’s not working well with this design. Based on usability feedback:

Why change?

• Executives would get thrown off by the waterfall chart.
• If the metric value is high, the signals impact would be low, because of which the signal bars appear too small and the “Total” bar appears to be overwhelming.
  
• There were a lot of questions around how the metric relates to the other signals.
  

What’s better?

• The entire chart focuses on just the gaps affecting metric performance.
• Even if the metric value is high, the visualization would not appear disproportionate, and when it does, it means one signal has such a high impact that it’s good if it catches the most attention.
  

Why change?

• The page was designed to be responsive. For smaller screen sizes, the horizontally laid out cards would stack up vertically.
• With this layout, it’s tough to see what the card interactions are because the second panel of metric details wouldn’t be in immediate view.
  

What’s better?

• Making the top panel a carousel when all 4 metric cards don’t fit the screen horizontally helps take care of the issue being caused with the second panel for metric details getting hidden.
  

redesigned dashboard v2

The final solution.

TELECOMMUNICATIONS

A Fortune Global 500 Information and Communication Technology firm

DISCOVERY PROCESS

An example from start to finish

THE BRIEF

DISCOVERY ACTIVITIES

The client approached us with a clear objective — they wanted to automate customer interaction. The motives were simple: improve engagement, push down costs, and increase revenue.

The challenge was clear. I had to demonstrate value in a month! We committed to delivering 2 use-cases (from a total of 40) on the platform in 30 days, developed and ready to play with.

A key stakeholder summarized the business goals well for us:

I was able to gain empathy for users, become aware of the business problem (and opportunity space), and define project milestones & metrics of success.

I arrived at an understanding by conducting workshops, interviews, and activities inc. surveys, assumption mapping, affinity diagramming, etc.) with client stakeholders and a representative group of users.

"We want to automate customer-facing touchpoints to deliver a personalized experience 24×7 so customers can feel valued."

The platform allowed us to combine complex NLP, cognitive learning abilities, emotional intelligence, and autonomic task management, helping the virtual agent learn from and respond to text inputs in an engaging, personalized, and emotionally cognizant manner.

With that...

Kicking off the project, I took charge of the experience strategy & design.

The efforts of collaboration with a UX designer, two project managers, and a team of developers, resulted in deployment of a virtual assistant on the client’s website. I worked on this project for a year. This is before the focus shifted on introducing other channels for interaction (SMS to begin with and voice later).

KEY INSIGHTS

Outcome of Discovery

In trying to uncover why people seek customer support from their internet service provider, the process they go through, and what works or does not work in that experience, I made some interesting discoveries. 32% of existing users would reach out to customer support for modem troubleshooting. Let’s focus on that in this case study.

DESIGN PROCESS

Outcome of Discovery Insights

1. DEFINING THE PROBLEM

3. CUSTOMER JOURNEY MAPPING

With the goal to define and align on the problem to solve, formulating the “user need statements” helped organize learnings about the problem space, using what was learnt in Discovery about users and what’s important to them. The statement acted as a condensed explanation of who the problem affects and why we’re solving it (e.g. How might we free up customer service representatives for complex cases?).

With the goal to experience the product from a user’s point of view, map the entry point, frustrations, etc., I built user persona specific customer journey maps. Taking the series of user actions in a timeline with user thoughts and emotions (thinking, feeling & doing representations), helps with storytelling and visualization.

2. CRAFTING PERSONAS

With the goal to build empathy using representative symbols, I built personas to guide experience choices and decisions. Building personas helped focus design efforts on a common goal, understand user needs & expectations, and align stakeholders accordingly. The user persona acted as a fictional archetype of the most strategic (here the largest) focal point of the end user population. I also designed a persona for the virtual agent. This involved identifying the qualities and adjectives that would characterize the personality of the virtual agent, setting the tone and language the agent would use, and ensuring that the persona aligns with the brand being represented.

4. PERSONA INTERACTION PATH AND SIGNPOSTING

With the goal to build a foundation for the full conversational architecture, I designed the conversational pathway detailing the persona’s journey. The persona interaction path is the baseline conversation by which a user accomplishes the goal of a use case. I also used signposting to formulate conversational junctions, through which language cues are given to prompt the user & elicit responses important to take the conversation forward.

5. CONVERSATIONAL DIAGRAMMING

With the goal to cover all possible user journeys, I designed the conversational flow, which offers:

– The full mapping of all conversational pathways

– Language signposts

– Notes on leveraging UI elements

– Conditional logic for pathway triaging

– Database or API references dependencies and

– Fallback routes.

Technical detail is added to the final iteration of the conversational diagram and is then used to develop the use cases.

ADDITIONAL ACTIVITIES CARRIED OUT

RETAIL

North American Multinational Retail Corporation

Experience Design Audit Approach

Within journeys contributing to the overall bot volume, which journeys have the highest negative impact on performance metrics that matter?

Top Metrics

Analyzed

Deflection Rate

(Users contained by bot to deflect from live agent)

Abandonment Rate

(Users dropping off before completing the bot journey)

1. Check Return Policy

From the custom anaytics report, it was found that 29% users who trigger the return policy journey abandon the conversation at the first step (i.e. the bot response is read but no associated action is taken). With this journey structure, only if users click on a menu option will they reach the “end response” flag that marks completion.

However, since the journey’s objective was covered in the first bot response itself (i.e. sharing the return policy), it was important to restructure the journey to reduce preceived abandonment and find a place for the highest clicked menu option with it’s logical corollary (i.e. clubbing policy and exceptions).

2. Get Product Information

The Product Info journey would trigger a business queue clarifier. Queues were structured by a combination of product type and location of order placement. Due to the overlap, the clarifier required to be explained to the user and it’s language was confusing enough for 22% requiring queue transfer (the journey was designed to collect info and auto-escalate).

The responses following queue selection weren’t unique. In all queue paths, almost the same menu options (with a few exceptions) were presented at different steps of the journey. The steps were consolidated and decision points were identified to mark out points within the journey that would require business queue selection to proceed further. Decision flow attached below.

Business Queue Clarifier - Decision Flowchart

3. Track Order

Almost two-thirds of all users who checked the status of their order via the VA would request to be escalated to a live agent. With API limitations, the bot couldn’t do more for this journey than share the order tracking link. Transcript reviews showed that once the user got escalated, some issues they brought up were:

• Status not changed for days/ link not providing enough info
• Backordered without informing
• Users don’t have the tracking number required for order tracking, etc.

Apart from the first one that was blocked by technical limitations, the other user needs were built into the bot.

Demonstrated Impact - KPI Improvements

FINANCE

Direct bank in North America - Personal banking Division

Findings

The lack of targeted personalization, contextual understanding, and backend integration to fetch user-specific info or perform actions for the user seen as a “Bad VA Experience”

Enhancements implemented

Conversation Design

• High-impact use cases prioritized. These are cases with high volume, low effectiveness (aka containment)
  
• For design overhauls, self-serve use-cases that took more than 4 turns to achieve resolution were identified

NLU Design

• Increased clarification question threshold
  
• Improved entity training
• Stored entities in context to allow entity recall within journey
  

Experiences recreated using a Conversational Interface Prototyping tool

Previous experience

Enhanced experience

Say hi!

Looking to hire? Up for a coffee chat?

Interested in discussing everything about AI disruptions, great products and innovative solutions?

Let’s start a conversation.

chadharaagini@gmail.com

Toronto, Ontario

Canada