Building AI Features Users Actually Want

Bottom Line First

Most AI features that fail were built without understanding what job users were hiring them for. Not because the teams lacked engineering skill or a good model. Because they started from “what could AI do here?” instead of “what is the user trying to get done, and where does that fall apart?” The graveyard of unused AI features is full of technically impressive things nobody asked for: AI-generated onboarding copy, smart inbox sorting that users turned off after two days, and “AI-powered insights” dashboards that got one click at launch and never opened again.

The fix is not a better model. It is a better question asked before the first line of code is written.

The Jobs-to-Be-Done Lens for AI Features

Clayton Christensen’s jobs-to-be-done framework is more useful for AI product decisions than almost any other lens, because it forces you off the feature plane and onto the outcome plane. Users do not hire AI because they want AI. They hire it because they have a job they are struggling to complete, and AI is a candidate that might do it better than their current approach.

The jobs that AI can actually fill break into three categories:

Saving time on work that is tedious but well-defined. The user knows how to do the task. They just do not want to spend 45 minutes doing it. GitHub Copilot autocompletes boilerplate. Otter.ai transcribes meetings. Notion AI drafts a first-pass outline. The job is clear, the AI’s role is clear, and the value is legible in minutes of recovered time.

Reducing cognitive load on work that is complex but learnable. The user could figure it out eventually, but it takes too long or requires context they do not have. Cursor explains unfamiliar code inline. Perplexity synthesizes research from multiple sources so the user does not have to read ten tabs. The job involves reducing the gap between what the user knows and what the task requires.

Unlocking capability the user does not have at all. The user cannot do this job without AI — not slowly, not inefficiently, just not at all. ElevenLabs lets non-audio engineers produce studio-quality voiceovers. Midjourney lets designers prototype visual concepts without illustrators. The AI is not a faster version of an existing tool; it is the only tool.

When you cannot clearly place an AI feature into one of these three categories, that is a signal you do not understand the job yet. Features built without a clear job to fill tend to get used once, out of curiosity, and then abandoned.

Finding Real Demand vs. Imagined Demand

The fastest way to build an AI feature nobody uses is to ask users “would you use an AI that does X?” in a survey or user interview. They will say yes. People say yes to hypothetical features all the time, especially AI features, because they sound impressive and agreeable in the abstract.

Real demand shows up in behavior, not stated preferences. Three places to look:

Usage patterns in existing tools. Where do users spend the most time? Where do they drop off? If you run a data analysis product and you see users spending 40% of their session time writing the same four types of SQL queries, that is a scoped, high-frequency job. Aggregate query templates with one-click parameterization, or generate them from natural language — either way, you have found a real job. The signal is already in your logs.

Support tickets as demand signal. Support tickets are compressed expressions of unmet needs. If users are writing in to ask “can I do X?” — search across bulk conversations, export to CSV, reformat this output — they are telling you what jobs they cannot complete. Linear, Intercom, and Zendesk all have enough volume that keyword clustering on tickets reliably surfaces the top ten unmet jobs in any mature product.

The shadow workflow. This is the most underused signal in product development. A shadow workflow is something users are doing manually, outside your product, to accomplish a goal your product was supposed to handle. They export data to a spreadsheet and write formulas. They copy-paste content into ChatGPT. They maintain a personal Notion doc that duplicates data from your app. These are not signs of lazy users — they are signs that your product has a job gap. Wherever users have built a shadow workflow, there is usually a strong candidate for an AI feature.

Finding shadow workflows requires asking “walk me through everything you do after you leave this screen” in interviews, and watching session recordings for clipboard activity, app switching, and manual re-entry of data that your product already has.

What Users Say They Want vs. What They Actually Use

The most consistent pattern in failed AI features is the gap between stated intent and actual behavior. Users want the feature that sounds impressive. They use the feature that reduces friction on something they do every day.

AI writing assistants are the canonical example. In user research, people consistently say they want help writing emails, blog posts, and reports. In practice, AI-generated long-form writing gets used for rough drafts that get deleted and rewritten, if it gets used at all. The job users think they have — “I need someone to write this for me” — is not the actual job. The actual job is “I need to get past the blank page so I can write this myself.” Tools like Lex nail this because they do not try to finish writing for you — they unstick you.

Email AI is the starkest split. Every email app has launched some version of “AI compose” or “smart reply.” Usage rates on smart reply are consistently single-digit percentages after the first week. Meanwhile, Superhuman’s AI triage — one-click to route, mute, or archive an email — has high daily engagement because the job it fills (reducing inbox anxiety for high-volume email users) is both real and recurring. Writing an email from scratch is a once-a-day job for most people. Triaging thirty emails is a ten-times-a-day job. Frequency matters.

The other pattern: users will use AI features that fit inside their existing workflow and reject features that require them to change their workflow to use the AI. Copilot’s in-editor autocomplete succeeded partly because it required no workflow change — you kept coding and suggestions appeared. Products that require users to go to a separate “AI mode” or fill out a form before the AI will help see much lower adoption, because the context switch has a cost that the feature has to overcome.

How to Run User Research for AI Features

Standard user research methods work for AI features, but some techniques are particularly high-leverage.

Contextual inquiry over retrospective interviews. Watch users do the actual task in their real environment, rather than asking them to describe it. You are looking for moments of friction: the pause before a Google search, the copy-paste into a different tool, the “ugh” before a manual process. These are candidate jobs. Retrospective interviews produce cleaned-up narratives. Contextual inquiry produces raw friction.

Session recordings filtered for hesitation signals. Tools like FullStory and PostHog let you filter recordings for rage clicks, long pauses, and repeated actions. A user who clicks the same button three times is confused. A user who pauses for 20 seconds on a screen is stuck. A user who copy-pastes content to their clipboard and then immediately closes your app is performing a shadow workflow. These signals are the behavioral equivalent of the “ugh” in a contextual interview.

Wizard of Oz prototyping for AI features specifically. Before building any AI feature, run a Wizard of Oz test: simulate the AI behavior manually. If you are considering building a “summarize this customer conversation” feature, have a human analyst summarize a sample of conversations and show the output to users as if it were AI-generated. Measure whether they find it useful, how they use the output, and whether it actually changes their workflow. This tells you whether the feature has value independent of the AI implementation, and it costs nothing to run.

The Wizard of Oz method is especially useful for AI features because it separates the product question (does this output add value?) from the engineering question (can we generate this output reliably?). Most teams conflate the two and end up building impressive technical demos that answer a question nobody asked.

Signals That an AI Feature Has Real Pull

You ship the feature. How do you know if it worked?

Daily active use from users who had no prompting. Any feature will get used in the first week because you told people about it. Real pull shows up in week 3 and week 6, from users who are coming back to the feature on their own without a push notification or an email. Track feature-level retention, not just product-level retention.

Workflow integration, not one-off use. A feature that users run once and then move on from is a novelty. A feature that becomes a regular step in how users do their job is a product. The clearest sign of workflow integration: users start treating the AI output as an input to the next step, not as an end in itself. They take the AI-generated outline and start writing. They use the AI triage category to decide their next action. The AI output is upstream of something else they do.

Users are upset when you remove it. This is the bluntest test. In beta and early access, periodically roll back AI features for a small cohort and watch the support volume. Copilot famously gets negative reviews and support tickets within hours of an outage — not because it is perfect, but because developers have reorganized their flow around it. If you remove a feature and nobody notices, the feature did not have pull. If users write in within 24 hours asking where it went, it did.

Features That Work vs. Features That Disappoint

Features that consistently work share a profile: they target a high-frequency job, they fit inside the user’s existing workflow, and the quality bar is achievable with current model capabilities.

• Code autocomplete (Copilot, Cursor, Tabnine): high frequency, zero workflow change, quality good enough at p50
• Meeting transcription and summaries (Otter.ai, Fireflies, Notion AI Meeting Notes): high frequency, passive capture requires no extra work, output is immediately useful
• Search over personal or organizational content (Notion AI search, Glean, Guru): replaces a painful manual job, quality bar is “better than Ctrl+F,” which is achievable
• One-click triage actions (Superhuman): very high frequency, very low friction, clear job

Features that consistently disappoint share a different profile: low-frequency job, requires workflow change, and quality variance is high enough that users cannot rely on it.

• AI-generated long-form content: low frequency, quality unreliable enough that users spend as much time editing as they would writing, and users often feel they cannot put their name on AI prose without rewriting it
• Predictive analytics / AI insights dashboards: vague job definition, high variance in output quality, output requires significant interpretation before it is actionable
• AI chatbots for customer support (first-generation, rule-based): job is real, but quality variance means users lose trust quickly and escalate anyway
• Personalized AI recommendations in domains where the signal is weak: recommendation quality is only as good as the behavioral signal, and for new users or low-engagement users there is not enough signal to produce useful recommendations

Cheap Ways to Test AI Feature Ideas Before Building

The fake door. Add the feature to your UI — a button, a menu item, a surface in the product — and log clicks before building it. Users who click are expressing real intent. A click rate under 2% on a prominent surface is a strong signal to deprioritize. A click rate over 10% justifies a Wizard of Oz or a small build.

The manual version test. As described above: do the job manually for a small cohort and measure value before automating it. If the manual version does not change user behavior, the automated version will not either.

Prompt testing without a product. For features that are essentially “send this user data to an LLM and surface the output,” you can test model output quality in two days with a spreadsheet and direct API calls. Sample 50 real cases, generate output, have users rate it. This tells you whether the model can do the job before you build the product wrapper around it.

Usage-gated rollout. Ship the feature to 5% of users and run it for four weeks before rolling out broadly. Track weekly retention on the feature specifically. If week-4 retention on the feature is under 20% of week-1 usage, you have a novelty feature, not a sticky one. Invest in understanding why before expanding.

The Question Worth Asking Before Every AI Feature

Before you start scoping, ask one question: what is the user doing right now to accomplish this job, and why is that worse than what the AI would do?

If you cannot answer that question specifically — not “it takes a long time” (everything takes a long time) but “they spend 30 minutes a day writing status updates by pulling from three different sources because there is no aggregated view” — you do not have enough signal to build. Go find that specificity first. It will save you a sprint, and it will tell you exactly what the AI feature needs to do to win.

The teams shipping AI features with real retention are not the ones with the best models. They are the ones who found a real job, verified it was painful enough to justify a solution, and built something that fit inside the workflow users already had. The model is almost always good enough. The job definition is almost always the gap.