Discover what's possible with Go Testify & unlock in-depth insights!
With thousands of games being released daily across platforms, investing time and energy in a title that transcends a genre and ensures you stand out from the crowded market has never been more critical.
The following playbook has been designed to help you start off on the right foot and set up effective appeal or concept testing. This guide will ensure people provide you with comprehensive feedback that helps to align the early decisions you make around your concept.
WHAT IS AN APPEAL TEST or CONCEPT TEST?
Well, an appeal test (often called a concept test) is typically conducted in the early phases of development to evaluate the market desirability of a product or feature; however, it can also be used during the Alpha & beta stages by marketing teams to evaluate everything from store pages to advertising material.
Now, a team can evaluate a product's appeal at scale in various ways to build confidence in their direction; however, for this playbook, we'll focus on our own unmoderated method.
This approach is a cross between conducting a focus group or 1-1 interview with a survey; however, within the survey, people will complete various tasks while providing verbal feedback in their own time.
We've found this method provides companies with:
- 80% of the in-depth comprehension that would traditionally come from focus groups or 1 to 1 interviews.
- market feedback that is less biased due to moderator or social dynamics.
- the ability to keep their marketing spend down whilst also ensuring insight hits their team at pace.
Examples of participants providing comprehensive feedback using this method.
COMMON APPEAL TEST GOALS
These can vary greatly depending on your individual needs; however, in general, stakeholders wish to:
- Evaluate what aspects of the concept resonate with the market and why.
- Determine if their in-game or marketing material communicates the title's vision effectively.
- Evaluate if their world-building or character design communicates the right message
- Inform the development of which features motivate/excite the market to explore in the future.
- Identify if their concept & USPs will stand out from their competitors in the market & why
CREATING AN EFFECTIVE APPEAL OR CONCEPT TEST
Appeal or concept testing is easy to conduct but hard to ensure comprehensive feedback that helps make meaningful decisions.
The best results come by:
- Dialling into what you need to learn in detail.
- Ensuring participants understand the goal for every requested task/activity.
- Preparing assets (Artwork, concept summary, feature summaries, videos, etc.) that allow people to paint a picture in their minds to provide constructive insight.
- Utilizing open, closed & follow-up questions at the right time to ensure people provide you with an in-depth understanding.
- Gurellia testing with people unfamiliar with the concept(s)/terminology to ensure that every question is understood before launching your test on Go Testify.
The following step-by-step guide will assist you whether you're doing it for the first time or looking for advice on how to get in-depth insight from everyone who participates.
Need help with writing effective surveys?
Take the time to explore our Survey 101 Playbook.
In particular, check out the 'Writing a Question for Everyone' section, which will help you avoid some of the common pitfalls that affect comprehensive feedback.
Define what you want to learn
There are different reasons why you may conduct an Appeal or Concept test; however, below are some of the critical questions you need to ask yourself first and some advice to help you decide.
| Decisions to make | Best Practices for Appeal Testing |
|---|---|
| What will I be validating with the market & why? |
|
| Do I want to validate with a broad or target audience? |
|
| Do I want to have people rate my concept against other competitors or not? |
|
| What questions will be open & what should be closed? |
|
| How many participants do I need to make confident decisions? |
|
| Do I intend to use the same Appeal test or Concept test with a larger sample (100+ participants) in the future? |
|
Decide how you will present everything
There are hundreds of tools in the market that offer you the capability to present an effective Appeal test; however, below are some of the key features we look for when assessing a tool to use with the Go Testify platform.
The tool must provide you with:
- Open & follow-up question capabilities.
- Skip logic (when someone responds to a question) isn't always required but can be helpful for A/B testing
- Scale-based & matrix questions for those times when you want to pulse the market.
- Card sorting capabilities on a per-participant basis, so any changes by one person don't affect the others.
- The ability to add images or videos.
- The ability to have different pages or sections to group similar questions together or keep conversational topics separate.
- The ability to review, share and analyse the data afterwards at pace.
We understand everyone has their tool of choice to present their thoughts; however, we find survey tools are best equipped to deliver the results we need when using this method.
Below are some of the tools we & our customers have used in the past.
- Alchemer
- Survey Monkey
- Zoho Survey
- Qualtrics
- Google Forms (FREE, but lacks certain capabilities)
- Google Slides
- Powerpoint
Structure effective appeal questions
Now that you've chosen your tool, you must consider the best way to take a person through your appeal test.
Below is the journey/flow we like to take people on:
(1) Outline clear expectations
At the start of the test, ensure people understand what you expect from them and outline the activities they will complete.
(2) Identify the participant
To cross-reference the feedback on the Go Testify platform we recommend capturing a participant's first name, and surname initial.
(3) Organise your questions into phases
Helping people build a picture of the concept over time, allows you uncover high-level reactions before participants provide comprehensive insight.
(4) Outline expectations for every task
As people take part, it's easy to forget what's required, so consistently remind people what you expect from every task/question.
(5) Begin with open ended questions
As in a interview, starting the conversation of in a broad sense allows people to easily answer & feel comfortable when providing feedback.
(6) Outline probing questions to dive deeper
Provide people specific talking points to ensure people provide you with an in-depth understanding without you having to be there.
(7) Mix verbal & scale based feedback
Throughout key phases/milestones, consider adding closed questions to gauge appeal, interest levels, and average scores.
(8) Benchmark interest levels over time
As you provide more context, pulse interest in the concept or feature multiple times to assess whether their viewpoint changes over time and why.
Prepare & implement informative assets
It's always hard to provide advice in this area, as concept or appeal tests can use a wide array of assets to communicate their game; however, below are some common things to consider when talking to the design team.
| Do | Don't |
|---|---|
| Take the time to prepare assets (Artwork, concept summaries, feature summaries, videos, etc.) that allow people to visualize a picture in their minds and provide constructive insight into the topics in your probing questions. | Don't use images or videos of different sizes when requesting people to A/B test design elements/characters, as this will bias people towards assets with more prominence. |
| If using images or videos, ensure the quality is high so they can be enlarged for people to inspect closely and explore whilst giving feedback. | If comparing more than three images, don't have people scroll through a long list one after the other, as they may forget/miss the talking points for the activity. Aim to combine those images into one larger image, and for reference, label each design with a letter rather than a number. |
| Have people provide their thoughts whilst ranking design concepts using card-sorting activities or scale-based appeal questions. This will provide you with their subjective opinions and a score to evaluate the concepts. | |
| Use a landscape format for images & videos where possible to help with testing with mobile or desktop participants. |
Set up to test via Go Testify
Once you have the Appeal or Concept test ready, follow the steps below to add it to the platform and launch it with either your own community or the Go Testify Gamer Network.
- Create a new test
- Choose Self-Guided
- Begin filling in your test draft
- Choose a platform for people to view the appeal test; you can have people take part on a PC (desktop/laptop) or their mobile phones via Android.
- Choose 'Web URL' as the destination
- Add your link to a survey or a URL of your choice
- Finalise your test plan
Don't forget you can choose to add an additional post survey via the platform if you want to.
Validate your appeal or concept test structure before launch
As with all market validation, no matter how well it's written or constructed, a survey or presentation is only as good as the feedback it captures.
Before placing it in the wild, we recommend:
- [a] Conducting a pass yourself to ensure the logic/structure works as you expected.
- [b] Testing it with at least one person who hasn't seen it before to ensure there is no confusion & someone can provide you with the necessary information.
- [c] Refining the survey or presentation based on external feedback ensures it's iron-tight for confident results.
In Conclusion
We find most people struggle to get started when creating their first Appeal test or Concept test; however, as you can see from the Unmoderated Appeal & Concept Testing playbook, it's not much different from building a survey and by following some basic steps, you can ensure comprehensive feedback which will effectively validate your roadmap.
We hope you found the playbook informative; however, if you have any questions, want to speak to someone, or want to see a demo of our platform, reach out!
