The Auto Navigator product and marketing teams had been challenged to provide a more personalized experience to customers to increase engagement and conversion. There were multiple streams of experimentation work by multiple teams happening at once to determine how to best take on this challenge. Over a few months, the teams had individually gained learnings about what was working. We needed to better align and unify the work.
I led the effort to bring product, marketing, data science and the business analyst teams together, determine what a unified data model and experience would look like, and provide clarity on the stages of customer actions to ensure all work moving forward would be cohesive.
Problem to Solve
Multiple teams were working independently to determine how to increase engagement and conversion by leveraging data science and AI implementation. Each team had gained learnings about what was working and what wasn’t working in a silo. Each team had different data models, definitions of the customer, messaging, and outcomes they were looking to drive for each stage for the customer. The value proposition for how Capital One was uniquely positioned to best support the customer was either unaligned or didn’t exist at all.
Discovery
Gather and Synthesize
I gathered the learnings and the materials created by each of the individual teams. I synthesized the learnings, definitions of the customer, outcomes and unknowns. In addition, I gathered inputs from each team via survey to see what they thought the value proposition was.
Gain Alignment
I brought the synthesis, summary of the value proposition survey and my recommendations to executive leadership to provide visibility to the learnings and to gain alignment on the approach I was recommending.
The outcome:
The results of the value proposition survey made it very clear that everyone had a different definition of how the product was uniquely positioned to provide value to customers. In addition, the customer stages and the definition of the customer varied by team, which was problematic for implementing an AI model that would personalize messaging to customers.
I needed to lead an effort to define the customer stages, definitions of each stage, and the outcomes we were looking to drive, which ensuring I consistently had alignment from senior leaders and was bringing the collective team along with me on the journey.
The recommendation:
I recommended that I lead a workshop that included the executive team along with the team on the ground with the objective of aligning on:
- the customer stages that would need a personalized experience and unique data model
- the outcome we were looking to drive for each customer stage
- the unique offering we could deliver for each stage
Next, we would have a working group that didn’t include executive leadership that would define:
- the messaging
- what needs to exist or be elevated for these customers within the product experience
- what do we not know that we need to know before we go to market
Definition
During the initial 4 hour workshop, I led the group of executive leadership and the leads from the working teams to align on 5 customers stages, the definitions, the outcomes and the unique product offering we could provide.
Undecided
Definition: customers that are casually exploring and have not decided if they want to buy a car.
Outcome: make it easier to start their search and get answers to their questions.
Unique product offering: establish Capital One as an authority that can help them on their car buying journey.
Exploration
Definition: car buyers who have started their search but don’t know what car they want yet.
Outcome: make it easy for them to narrow down cars they could potentially buy.
Unique product offering: provide smart suggestions, recommendations and make it easy to compare.
Conversion
Definition: car buyers who know the make and model of the car they want to buy or have a short list of vehicles they like.
Outcome: make it easy for them to determine which car and which dealership is the best fit for them.
Unique product offering: Capital One’s ability to elevate quality dealers who are great to work with.
Next Steps
Definition: car buyers who have determined the exact car they want to buy and the exact dealership they want to buy from.
Outcome: make it easy for the customer to connect with the dealership
Unique product offering: schedule a test drive, check availability, anonymous connect
Managing Money
Definition: car buyers who need to determine how they will finance their car.
Outcome: make it easy for the customer to understand their financing options.
Unique product offering: trade in evaluation, pre-qualification, shop by monthly payments
From here, the working group determined the messaging, what the experience would elevate for each customer stage and established a learning plan that outlined what we needed to learn before launch and what we wanted to learn along the way. I presented the options back to the leadership team to obtain their alignment.
Build
I worked closely with data science team leadership to ensure that they were in the loop from the very beginning of this initiative. This enabled their team to concurrently build the data models while we obtained alignment on the customer stages and what the experience would look like for each stage.
Once the data models and designs were approved, we were able to quickly go to market leveraging a throttled approach, starting with 10% of customers seeing the new experience options. We were testing up to 20 iterations for each customer stage leveraging an AI testing model to quickly learn what was resonating with the customer and determine a champion design.
Measure
Leveraging the AI testing model, we were able to determine a winning design for each customer stage within a few days time. The criteria for the winner differed for each customer stage. For example, the criteria for an undecided customer would be that the customer moved forward from the home page to either start a car search or start a pre-qualification application.
Iterate
As the in market tests were running, the team conducted desirability testing via usertesting.com to gather qualitative data in addition to the quantitative data. We leveraged the results from both tests to consistently feed the AI data model new prompts and new creatives. This enabled the AI model to consistently learn, adjust and provide data back.
The Results
Since the initial test and launch, the team has been able to vet the customer stages, add additional stages, and create relevant, personalized content for customers. Auto Navigator has evolved from having two primary experience functions of gaining financing or starting a car search to being a hub for car research, a platform to sell your car, and has become a significantly more valuable lead source for dealerships.