← All Examples
📊SaaS / ProductQuick

Beta Feature Feedback

Collect structured user impressions on a feature before GA launch.

Sender: Product Researcher at Collaboration tool startup
Participant: Power User — Beta program participant
This is an illustrative example showing how Willit could be used in saas / product. All names, quotes, and data are fictional. We never use real customer interviews for marketing purposes.
1

The Brief

The sender described what they wanted to learn. Willit's AI refined these instructions into a natural interview flow.

We shipped a new async video messaging feature to our beta group two weeks ago and I need to collect impressions before we decide whether to go GA or iterate first. This respondent is in our power user cohort — they use the product daily and have given feedback before. Keep the conversation focused and snappy. I don't need their full life story, I need their honest take on this specific feature. **First impression:** What was their reaction when they first discovered it? Did they try it immediately or ignore it for a while? **Comparison to expectations:** We've been talking about this feature in our changelog and community posts for a few months. Did it meet what they expected? Better or worse? **Workflow changes:** Have they actually changed how they work because of it? Even small changes count — switching from a screen recording tool they used before, reducing the number of meetings they schedule, that kind of thing. **Bugs or glitches:** Anything broken, awkward, or confusing? I want specifics — button placements, error states, loading times, anything. **Would they miss it:** If we turned it off tomorrow, would they notice? This is the real adoption test. **One thing to change:** If they could only fix or improve one thing, what is it? Force a choice — don't let them give me a list. Keep it punchy. This doesn't need to be a long conversation — I'd rather have sharp, honest answers than polished ones.
2

The Interview

Willit's AI detective conducted a quick interview with a Power User. The conversation explored 5 topic areas through natural follow-up questions, adapting in real-time based on the participant's responses.

Capture first impression and initial adoption behaviorAssess whether it met pre-launch expectationsUnderstand workflow changesSurface bugs and UX frictionTest stickiness — would they miss it
3

The Report

Willit automatically extracted structured insights from the conversation — scores, goal coverage, key quotes, and red flags.

Interview Scorecard

EngagementSentimentDepthQualityCoverageCoherence

Metric Averages

Engagement
91
Sentiment
78
Depth / Accuracy
72
Info Quality
76
Goal Coverage
83
Coherence
88

Summary

The beta user has adopted the async video feature meaningfully — it has replaced roughly 3 short Loom recordings per week. First impressions were positive but tempered by a clunky recording start flow. The biggest outstanding issue is the lack of playback speed controls. The user would notice if the feature disappeared, which is a strong adoption signal, but the feature is not yet sticky enough to be a purchase driver.

Goal Coverage

Covered

Capture first impression and initial adoption behavior

  • Tried the feature the same day it appeared in the product — actively looking for it after seeing the changelog post
  • First recording attempt failed because the countdown timer wasn't visible — had to retry twice
Partial

Assess whether it met pre-launch expectations

  • Video quality exceeded expectations — expected it to look like a screen capture, got broadcast quality
  • Expected threading/reply capability that wasn't there — was disappointed by this gap

Gap: Did not explore what specific marketing language created the threading expectation

Covered

Understand workflow changes

  • Has replaced Loom for internal async updates — estimates 3 fewer Loom recordings per week
  • Has not reduced number of meetings yet — still defaults to live calls for anything requiring back-and-forth
Covered

Surface bugs and UX friction

  • Countdown timer before recording starts is too small and easy to miss — caused confusion on first use
  • No playback speed control — forced to watch at 1x which feels slow
Covered

Test stickiness — would they miss it

  • Would notice if it disappeared — would go back to Loom but would be annoyed at the context switch

Key Quotes

“I thought it was going to look like a Zoom screenshot. It actually looks good. That surprised me.”
“Playback speed. That's the one thing. I cannot watch myself talk at 1x, it's excruciating.”
“I'd be annoyed if it went away. Not devastated — annoyed. It's become a thing I reach for.”

Red Flags

  • Threading expectation created by community posts that doesn't match current feature — risk of GA disappointment if not addressed in messaging
  • Countdown timer UX causing failed first recordings — likely a common drop-off point for new users

Follow-up Suggestions

  • Add playback speed controls (1.25x, 1.5x, 2x) before GA — this is table stakes for async video
  • Audit changelog and community post language for any references to threading or reply functionality
  • Instrument recording start flow to measure how many users fail their first recording attempt

Ready to run your own AI interviews?

Set up your first interview in under 5 minutes.

Get StartedMore Examples