Case Study 05 | Tactical product Decisions

Using usability testing and behavioral analytics to guide build, remove and continue decisions.

Across three product questions, I combined usability testing, qualitative desirability research and behavioral analytics to help teams decide whether to improve a redesign, remove a low-value feature, or continue investing in a beta.

Tennis ChannelRead AloudVideo Shorts

Project overview: evidence for build, remove and continue decisions

Three product questionsShould the team build a redesign, remove a feature, or continue a beta?

Match evidence to the decisionBenchmark usability, qualitative desirability and product analytics.

Build, remove, continueEach study translated evidence into a clear recommendation.

Roadmap direction changedTeams moved forward with a redesign, removal, and continued beta investment.

58 to 84UMUX in redesign benchmark

80+ / ~$35Kstations removed / annual savings

5.9% / 49%adoption / engagement-time lift

58 to 84

UMUX improvement in the Tennis Channel benchmark, supporting the redesign direction before build

$35K per year

saved after Read Aloud research supported removing a low-value shipped feature across 80+ stations

49%

engagement-time lift per session in the Video Shorts beta, supporting continued investment

Influence

How my work influenced the decision

Define the decision

Clarify whether the team needs evidence to build, remove or continue.

Choose the right evidence

Use usability tests, qualitative studies or analytics based on the question.

Expose limitations

Make small samples, internal recruiting and launch effects visible.

Translate into a recommendation

Give teams a clear decision with the evidence and caveats attached.

Change roadmap direction

Each study supported a build, remove or continue decision.

Strategic tension

The tradeoffs that shaped the work

Speed vs confidence

Move quickly without overreading weak evidence.

Local findings vs scale signals

Use qualitative nuance and analytics scale for different parts of the decision.

Launch spike vs stable trend

Separate temporary launch effects from durable behavior.

Overview

Three product questions needed evidence strong enough to support a decision

Each example started with a different business decision: should we build this redesign, keep this feature, or invest further in this beta? By combining user behavior with observed usability issues, I helped teams make faster decisions with more confidence.

Build

Tennis Channel usability testing showed whether the redesign was ready to move forward and where one workflow still needed attention.

Remove

Read Aloud research showed that the shipped feature had limited customer value and supported a removal decision that reduced annual cost.

Continue

Video Shorts telemetry showed stronger engagement among adopters while also identifying the exit behavior and launch effects that needed to be monitored.

Study 01

Tennis Channel: validate the redesign before production

A fast benchmark compared the live experience with the proposed Figma redesign. The redesign improved four of five tasks and moved UMUX from 58 to 84; the schedule/EPG task remained the key regression to address.

Benchmark report comparing the live site and proposed redesign.
View full visual

Decision

Greenlight the redesign direction before a line of production code was written, while scoping the schedule page as a follow-up.

Study 02

Read Aloud: know when to remove a shipped feature

Two qualitative studies with 21 participants tested discoverability and desirability. Nineteen of 21 participants preferred reading for local-news stories; interest in audio rose only slightly for multitasking contexts.

Desirability study results.
View full visual

Decision

Recommend removal rather than further investment. Read Aloud was removed across 80+ stations and vendor terms were renegotiated, saving roughly $35K annually.

Study 03

Video Shorts: measure the beta without overreading launch data

A five-week analytics readout across three markets showed 5.9% adoption and a 49% engagement-time lift per session. I caveated week-one adoption because the launch tooltip inflated the initial spike and anchored the recommendation on the stabilized trend.

Video Shorts beta readout across three local news markets.
View full visual

Decision

Continue investment and increase publishing cadence to support engagement rather than treating the launch spike as proof on its own.

Pattern

What connects these three studies

Match method to decision

Benchmark usability, qualitative desirability and product analytics each answered a different question.

State the limitation

Small samples, internal recruiting and launch effects were made visible rather than hidden.

End with a recommendation

Each study changed a roadmap direction: build, remove or continue investment.

PreviousDriving consistent persona adoption with AI NextCollaborative vendor selection that saved $10K

All metrics and roadmap decisions shown on this page are taken from the supplied case-study source.