How to Identify the Right Product in Research and Development
Identifying the right product in research and development means filtering many ideas through evidence gates before spending the development budget. Teams screen ideas against strategy, validate the customer problem through interviews, test feasibility with technical and channel experts, test concepts with target buyers, model economics, and confirm product-market fit after launch.
What does product selection in research and development mean?
Product selection in research and development is the evidence-based process of deciding which product ideas earn development budget and which are stopped before costs become hard to reverse. Product selection sits at the front of the product development process, before engineering, formulation or software work scales up.
Research and development (R&D) covers two different activities. Research generates new knowledge, such as a material or an algorithm. Development turns that knowledge into a product someone will buy.
New product development (NPD) is the managed process that carries an idea from concept to launch. Product development research is the market-facing evidence that feeds NPD decisions: customer needs, feasibility, concept appeal, pricing and fit.
For a consulting or marketing firm, product selection is often the deliverable the client is paying for: a defensible recommendation on which one or two ideas to fund
Why do new products fail, and how often?
Peer-reviewed evidence puts the new product failure rate at roughly 40%, and Robert G. Cooper’s research identifies weak up-front market homework as the leading cause. George Castellion and Stephen Markham reviewed the evidence in the Journal of Product Innovation Management (2013) and called the popular 80% figure a myth. The PDMA studies they cite found failure rates of 42% (1985–1989), 40% (1995) and 46% (2003–2004).
The higher numbers persist because people confuse idea failure with product failure. Most ideas die before launch, which is the screening system working as intended.
| Commonly quoted claim | What the traceable source says |
|---|---|
| “80–90% of new products fail” | Castellion and Markham (2013): empirical studies since 1977 find a failure rate of 40% or less. |
| “95% of new products fail” (attributed to Clayton Christensen) | Circulates through blog posts; no peer-reviewed study located. |
| “3,000 ideas for one success” | Stevens and Burley, Research-Technology Management (1997): applies to substantially new industrial products; far fewer ideas are needed for line extensions. |
| “Seven ideas for one success” | Booz Allen Hamilton, New Products Management for the 1980s (1982), as cited in later academic reviews. |
Skipping homework has a measurable cost. In Cooper’s NewProd III study, published in Research-Technology Management (1990), the typical failed project spent about five person-days on a detailed market study, roughly 0.5% of total project effort.
What are the stages of product development?
The stages of product development run from idea generation through screening, business analysis, development, testing and launch, with formal go/kill decisions between them. Three widely used models, Booz Allen Hamilton’s seven stages, Cooper’s Stage-Gate and Eric Ries’s Lean Startup, each place research at a different point.
| Model | Origin | Stages | Where product research sits |
|---|---|---|---|
| Booz Allen Hamilton NPD process | New Products Management for the 1980s (1982) | New product strategy, idea generation, screening and evaluation, business analysis, development, testing, commercialization | Screening, business analysis and testing |
| Stage-Gate® | Robert G. Cooper, Business Horizons (1990) | Discovery, Scoping, Build the Business Case, Development, Testing and Validation, Launch, with a gate before each stage | Scoping is largely desk research; Build the Business Case requires primary market and technical research |
| Lean Startup | Eric Ries, The Lean Startup (2011) | Build–Measure–Learn loop around a minimum viable product (MVP) | Continuous in-market experiments |
Stage-Gate suits capital-intensive, regulated or B2B products where building the wrong thing is expensive. Lean Startup suits software and digital products, where Ries’s MVP, the version that “allows a team to collect the maximum amount of validated learning about customers with the least effort,” is cheap to ship.
Hybrids are now well documented: Cooper and Sommer’s “The Agile–Stage-Gate Hybrid Model” (Journal of Product Innovation Management, 2016) describes gates controlling funding while teams work in sprints inside each stage, with LEGO as a case example.
How do you identify the right product to develop? The Nexus Evidence Ladder
The most reliable way to choose the right product is to make each idea climb six rungs of evidence, from strategic fit to market fit, and to stop funding any idea that fails a rung. Nexus Expert Research calls this the Evidence Ladder. Each rung has one question, a primary method and a kill signal.
- Strategic fit: Does the idea belong in this company’s portfolio?
- Problem evidence: Do target buyers have this problem badly enough to pay for a solution?
- Feasibility: Can the product be built, supplied, approved and sold?
- Concept appeal: Does the specific concept beat the buyer’s current alternative?
- Economics: Will price, cost and volume produce an acceptable return?
- Market fit: After launch, do users depend on the product?
Rung 1: How to screen product ideas
Idea screening is the first gate, where a cross-functional team scores each idea against must-meet and should-meet criteria and drops the weakest without commissioning new research. Must-meet criteria are yes/no knock-outs, such as strategic alignment, legal acceptability and access to the required technology.
Should-meet criteria are scored, typically:
- Market size and growth
- Product advantage over current alternatives
- Fit with existing channels and sales capabilities
- Technical and manufacturing synergy
Use desk research and internal data only at this rung. The goal is to cut a long list to three to five candidates cheaply. Kill signal: the idea fails any must-meet criterion.
Rung 2: How to validate the customer problem
Problem validation confirms that a defined buyer group experiences the problem often and painfully enough to change behavior. One-on-one interviews work best at this rung. Griffin and Hauser’s 1993 study in Marketing Science found that 20–30 interviews per segment surface about 90–95% of customer needs. Hauser’s MIT note on the study reports that a single one-on-one interview identified 33% of the 230 needs found, and two interviews identified 51%.
Frame interviews around the job the customer is trying to get done, the approach Clayton Christensen and colleagues set out in Harvard Business Review (2016). Ask about recent behavior, workarounds and money already spent instead of hypothetical purchase intent. Kill signal: buyers describe the problem as minor or already solved.
Rung 3: How to run a product feasibility analysis
Product feasibility analysis tests whether the product can be built, made, approved, distributed and supported at the target cost. It covers four areas: technical, regulatory, operational (supply and manufacturing) and commercial (channel and sales motion). End customers rarely know the answers, so feasibility depends on practitioners who have done the work.
Useful practitioners include a former plant manager on tooling costs, a regulatory affairs lead on approval pathways and a distributor on shelf economics. Expert network interviews are most valuable at this rung. Kill signal: a hard constraint, such as an approval timeline or unit cost, that the business case cannot absorb
Rung 4: How to run concept testing in product development
Concept testing measures how target buyers react to a described or visualized product before it is built. Common designs include monadic tests (each respondent sees one concept), sequential monadic tests (each respondent sees several), and conjoint or MaxDiff exercises that rank features by trade-off.
Concept tests need volume, so surveys lead at this rung, with qualitative follow-up to explain the scores. Test the concept against the buyer’s current alternative. Kill signal: the concept does not outscore the incumbent alternative among the core segment.
Rung 5: How to test pricing and economics
Economic validation checks that a realistic price, unit cost and adoption rate produce a return the business will accept. Common price research methods include Van Westendorp price sensitivity analysis, Gabor-Granger testing and conjoint analysis.
In B2B markets, procurement leaders and channel partners can explain budget cycles, approval thresholds and switching costs that a survey will miss. Kill signal: the price buyers accept does not cover target margin at a plausible volume.
Rung 6: How to confirm product-market fit
Product-market fit research confirms after launch that a defined group of users would genuinely miss the product. The most widely used benchmark comes from Sean Ellis, who wrote in 2009 that fit “requires at least 40% of users saying they would be ‘very disappointed'” without the product.
Ellis himself called the threshold “a bit arbitrary” and based it on comparing nearly 100 startups. Treat the 40% figure as a directional signal and pair it with retention and repeat-purchase data. Kill or pivot signal: the share stays well below 40% after iterating on the core segment.
Which product development research methods fit each stage?
No single method answers every product selection question: desk research narrows the field, interviews explain, surveys quantify and in-market tests prove. The table below matches common product development research methods to the decisions they support.
| Method | Best for | Typical scale | Main limitation |
|---|---|---|---|
| Desk research | Idea screening, market sizing, competitor scans | Existing reports and data | Lags new markets; thin on niche B2B segments |
| Expert interviews | Feasibility, regulation, channels, buyer process | One-on-one calls, commonly 30–60 minutes | Individual judgment, not market proportions |
| Customer in-depth interviews | Problem validation, jobs to be done | 20–30 per segment (Griffin and Hauser, 1993) | Stated attitudes can differ from behavior |
| Focus groups | Reactions to early concepts and buyer language | Small moderated groups | Group influence; hard to convene senior B2B buyers |
| Surveys | Concept testing, pricing, segment sizing | – | Only answers questions you already know to ask |
| MVP or in-market test | Final proof of demand | Real users and transactions | Slowest and most expensive to set up |
Expert interviews vs surveys vs focus groups: which should you use for product validation?
Use expert interviews when the question needs judgment from people who have run the operation, surveys when the question needs a percentage, and focus groups when you need to hear how buyers talk about a concept. Each method answers a different question, and the common mistake is using one method to answer another method’s question.
| Question type | Best method | Reason |
|---|---|---|
| “Can this be manufactured at the target unit cost?” | Expert interview | Requires operator knowledge |
| “What share of buyers prefer concept A?” | Survey | Requires a representative sample |
| “How do buyers describe this problem?” | Customer IDIs or focus groups | Requires open-ended language |
| “How does procurement approve a new vendor?” | Expert interview | Process knowledge held by few people |
| “Which features drive willingness to pay?” | Conjoint survey | Requires trade-off data at scale |
For B2B and technical products, start with a short round of expert and buyer interviews to frame hypotheses, then quantify the strongest hypotheses with a survey. For consumer products, lead with surveys and usage tests, and use experts for supply-side questions. Expert calls have limits. Expert calls cannot measure consumer preference at scale, and compliance rules bar seeking material non-public information from competitors’ current employees.
How many interviews and how long does product validation take?
Problem validation typically needs 20–30 customer interviews per segment, and a full idea-to-business-case cycle runs from weeks to months depending on product complexity. Griffin and Hauser’s 1993 research remains the strongest public benchmark for interview counts.
Hauser’s MIT teaching note on the study reports that one major US firm estimated out-of-pocket costs of $10,000–20,000 for 30 interviews, plus over 250 person-hours of team time. Those figures date from the early 1990s and should be read as a historical reference, not a current price.
Customer interviews for problem validation: 20–30 per segment (Griffin and Hauser, 1993).
Pre-development effort: In Cooper’s 1990 NewProd III data, the typical project used about 1,000 person-days, and successful products had about 75% more person-days devoted to pre-development activities than failures.
Expert calls for feasibility:
Time to first expert: current site pages state under 24 hours, 24–48 hours and 24–72 hours in different places
Stage durations.
Worked example: screening three product ideas for a client
A worked scoring example shows how the Evidence Ladder turns a long list into one fundable recommendation. The scenario below is illustrative, not a client case: an industrial equipment client asks a consulting team to choose among three ideas.
| Criterion | Idea A: IoT sensor retrofit | Idea B: Subscription maintenance service | Idea C: Premium pump line |
|---|---|---|---|
| Rung 1: strategic fit (must-meet) | Pass | Pass | Pass |
| Rung 2: problem evidence (customer IDIs) | Strong: unplanned downtime cited repeatedly | Moderate | Weak: buyers satisfied with current pumps |
| Rung 3: feasibility (expert calls) | Integration with legacy controllers is difficult | Field-service capacity available | Not reached |
| Rung 4: concept test vs current alternative | Not yet run | Outscores in-house maintenance | Not reached |
| Decision | Hold: fund a technical spike | Advance to business case | Kill |
Every decision in this example traces to a named evidence source, which makes the recommendation defensible in a client steering meeting. Idea C was stopped cheaply at Rung 2, before any concept-test budget was spent.
When should you not run more product research?
Stop researching when the decision is cheap to reverse, when the remaining uncertainty can only be resolved in the market, or when the evidence already clears every gate. More research does not automatically mean more rigor.
- Line extensions: Stevens and Burley noted that far fewer ideas are needed per success for line extensions than for substantially new products, so lighter screening fits.
- Low-cost digital features: Shipping an MVP and measuring behavior is often faster than running a concept test.
- New-to-the-world products: Buyers struggle to evaluate products they have never seen, so weight problem evidence and prototypes above stated purchase intent.
- Regulated categories (medical, financial, food): Never skip feasibility. Regulatory experts belong at Rung 3, and formal legal review sits outside market research.
- Compliance limits: Expert interviews must not seek confidential or material non-public information from competitors’ current employees.
Frequently asked questions
Is Stage-Gate still relevant for Agile teams?
Yes. Robert G. Cooper’s “Agile–Stage-Gate Hybrids: The Next Stage for Product Development” (Research-Technology Management 59(1), 2016) describes a model where gates still govern funding decisions while development work inside each stage runs in short sprints.
What is the difference between concept testing and product testing?
Concept testing measures reaction to a described or visualized idea before it is built. Product testing measures reaction to a working prototype or finished product, including usability and performance in real use.
How do you validate a product idea without building it?
Validate the problem through customer interviews, test feasibility with operators and technical experts, then run a concept test or a landing-page or pre-order test that measures behavior rather than stated intent.
Can expert networks support consumer product research?
Expert networks help with the supply side of consumer products, including retail buyers, category managers, formulators and distributors. Consumer preference itself is better measured with surveys and in-home usage tests.
How much does an expert call cost for product research?
Expert call pricing depends on expert seniority, geography and the network’s model, such as per-call fees or annual subscriptions.
What to do next
Take your client’s current idea list and mark, for each idea, the highest rung of the Evidence Ladder it has cleared with real evidence. Any idea scheduled for development that has not cleared Rung 3 carries untested feasibility risk.
If closing that gap needs operator, regulatory or buyer input, scope a short round of expert interviews and consult with Nexus Expert Research before the next gate meeting.
