Survey 101 Playbook

Discover the power of exceptional surveys and unlock more in-depth insights!

TABLE OF CONTENTS

• Introduction
• Getting Started & Establishing Key Questions
• Choosing the Right Approach
• Writing a Question for Everyone
• FREE Game Focused Question Bank & Survey Templates
• Structure Surveys For More Comprehensive Feedback
• Testing Your Survey
• Analysing Your Survey Data
• In Conclusion

INTRODUCTION

Whether you're just starting out or well into the process, our comprehensive guide is crafted to help put a stop to procrastination, simplify decision-making, and provide you with the essential questions to ask yourself while minimizing mental clutter.

By continuing to read, you'll never wonder what to ask or be concerned about missing critical information again!

GETTING STARTED & ESTABLISHING KEY QUESTIONS

Sometimes we all find the hardest part is getting started 😀

Below is a step-by-step guide to get the ball rolling and make your life easier as you design a rockstar survey!

Step 1 - Spend time reviewing what you're about to test

Doing this activity helps to...

• Ensure you have a full understanding of what is being tested.
• Put yourself in the shoes of your market.
• Understand everything people will experience & may be able to give feedback on.
• Check if anything may prevent someone from giving in-depth insight.

Step 2 - Ask yourself what you wish to validate at this stage & note those key points/questions down

Below are some of the common questions to ask yourself

• Do you want to assess the enjoyment of the game or a feature?
• Do you want to understand where people are confused?
• Do you want to judge frustration or progression over time?
• Do you want to determine if people will spend money with you or not?
• Do you want to know what people like or dislike?
• Do you want to know if someone would talk about their experience with others?

Once you've done those first few steps, then you should have a general list of learning objectives/questions. If...

• [a] You're still struggling to think about things you'd like to learn, chat with your team or ask yourself if this is the right time to test.
  
  
• [b] You've got 100 questions already; prepare yourself to prioritise what you can learn at this stage, as people would likely leave your survey before they answer your critical questions.

CHOOSING THE RIGHT APPROACH

The next place people tend to get stuck is in what type of question will help get comprehensive feedback & allow them to make decisions at pace.

Below is a guide to the two common question approaches and when to use them:

Now, It's easy to grasp how to set up open-ended questions; however, when considering closed questions, you still have one critical question to ask yourself:

• Do I want people only to select one option, or can they select multiple options from the list?
  

At this stage in the process, you will have a list of questions, and each one should have a chosen type(open-ended or closed question) to gather the information you need.

WRITING A QUESTION FOR EVERYONE

Now that we've got a general picture let's not waste any more time and dive straight into how to write a good question.

To ensure your survey questions work for all, we live and die by the following five rules

Rule 1 - Don't overcomplicate it.

Having an impressive vocabulary is great if you want to do a speech and showcase your prowess with the spoken word; however, your audience will have varying IQs and will likely experience issues with your terminology.

For instance, which question would you find easier to answer

"Considering the various game mechanics, storyline, player progression, graphics, sound effects, user interface, level design, social interaction, monetization strategies, and overall game balance, what specific aspects of this game do you find most enjoyable, and how do these aspects contribute to your overall enjoyment of the game?"

or

"When playing today, what aspect(s) did you find enjoyable?"

The first approach has too many components/room for misinterpretation, and a participant will likely forget to include something; however, the latter question doesn't leave room for misinterpretation and is straightforward to answer.

Keeping it simple will help to:

• Reduce confusion & make it easier for anyone to complete successfully
• Improve the amount of detail/depth provided
• Reduce the time it takes for you to write it :)

Rule 2 - Don't write a confusing question.

You'll get much better information from having a simple question that a person from any walk of life can answer easily.

For instance, using complex terminology that exists within the game could be confusing, as someone may find it hard to respond.

For example, say you asked the following

• "When using the 'Trident' force, what effect did it have on the game?"

In this instance, an individual might not have realised that the 'Trident' Force was the name of the special ability; however, if you alter the framing to the following,

• "When using your special ability today, did it make the game easier or harder?"

Making it clearer will:

• Reduce the likelihood of people assuming what it means and providing the wrong feedback
• Improve the amount of detail/depth provided because they understand the context of the question
• Ensure any time you spend reviewing open-ended responses is valuable.

Rule 3 - When asking closed questions in your survey, always make sure people can answer correctly

This rule builds on the second and is one of the biggest reasons for inaccurate survey feedback.

For instance, when asking

• "How impactful was the 'Trident' force in your success?"

And providing the following options to choose

• Negligible
• Mildly noticeable
• Significant
• Highly significant
• Utterly significant
  

People might not understand what the Trident force is and also have no option to choose if they've never experienced it within the game.

Altering the question to have more clarity and providing people with the following extra options when answering will ensure:

• Participants will find it easier to recall the experience, as they can relate to the question in multiple ways.
• You can make more confident decisions, as people will answer correctly.

Updated Question

• "How impactful was the 'Trident'(Blue energy bolt) force when used in combat with an end-of-level boss?"

Example options someone could choose to answer

• I never used the 'Trident' (Blue energy bolt) force today
• I'm not sure what the 'Trident' (Blue energy bolt) force is
• Negligible
• Mildly noticeable
• Significant
• Highly significant
• Utterly significant

Rule 4 - Aim to use the same style of closed question throughout.

This approach has helped studios of all sizes, from small indies to massive publishers, get comprehensive feedback every time!

For instance, you could use Agree/Disagree statement

• "Agree or Disagree: When playing today, I was confused"
• "Agree or Disagree: When playing today, I was frustrated"
• "Agree or Disagree: There were symbols/Icons in the game that I did not understand"

Using similar scales/choices will:

• Reduce fatigue when completing the survey/question
• Reduce the likelihood of someone choosing the wrong option
• Provide more accurate results
• Make it easier for you to analyse/report afterwards

Rule 5 - When using a Likert scale, keep to five unless it's critical that you need more granular insight

Every company has their own approach; however, we find that a 5-point scale is the perfect middle ground:

• for participants to reduce mental fatigue
• to help provide confident signals
• to analyse & report the data at pace

If you're still with us and enjoying the read below is a bonus, as to be honest, why create a question if someone has already done it for you!

STRUCTURE SURVEYS FOR MORE COMPREHENSIVE FEEDBACK

Once you master the perfect survey question, people often miss the important step to ensure someone is primed to provide you with the desired details.

With all your questions ready, you must ensure an ideal structure to help your participants feel relaxed and provide comprehensive open feedback when completing the survey.

Below is a fail-safe checklist to produce quality insight:

• [1] If the survey follows someone experiencing a product, move the first impression questions to the start to capture the feelings before they change. For example, your first question might be, "How fun did you find this experience?"

• [2] Make sure all questions around the same subject matter are grouped together to help people provide more depth as they recall and think about the topic more.

• [3] Assess if the start of the survey settles people into providing broad general answers before becoming focused on critical questions

• [4] Evaluate/Remove any questions that may provide the same information/insight and increase the analysis time.
  
  
• [5] Aim to have no more than 15 questions in your survey, as it's natural for people to get bored and fatigued with long surveys.

TESTING YOUR SURVEY

As with all market validation, no matter how well it's written or constructed, a survey is only as good as the feedback it captures.

Before placing it in the wild, we recommend:

• [a] Conducting a pass yourself to make sure the logic works as you expected.
  
  
• [b] Testing it with at least one person who hasn't seen it before to ensure there is no confusion & someone can provide you with the necessary information.

• [c] Refining the survey based on external feedback to ensure it's iron tight for confident results.

ANALYSING YOUR SURVEY DATA

Achieving the task at hand can be accomplished quickly or slowly, depending on the type of questions you used. Nonetheless, the following are essential techniques to ensure prompt and precise completion.

Closed questions are graphed and fast to evaluate; however, if you have any open-ended (qualitative) questions below are a few best practice steps to follow:

• Keep the analysis organised by creating a reference spreadsheet.
  
  
• To cross-reference the responses at pace, structure the spreadsheet with questions on individual rows and participants on individual columns across the top.
  
  
• To make life easy when reviewing the responses, label the themes/trends you find around a question and add those to a summary column beside the question with how often it was raised.
  
  
• To bring sentiment into the equation, group any trends/themes found into positive, negative & neutral, and keep in mind when this trend/theme was appearing if testing over a long period of time with the market.

IN CONCLUSION

Most people struggle with surveys; however, as you can see from the Survey 101 playbook, it's not rocket science to make sure you ask appropriate questions focused on what you wish to learn and analyse if you follow a trusted process.

If you run into challenges with any of the steps, though, we're only a call away, and our team is always here to help!