Quick & Dirty guide to Surveys
Before running the survey capture your assumptions about the user and about the problem you are solving.
Generate a list of things you want to learn in this process, ideally these are things that will help you refute, or confirm assumptions AND that will specifically empower you to make a decision of some kind.
Now, generate the actual questions, these will be different to the things you want to learn. You can’t always ask people directly.
Some basic categories of questions surveys should have are:
1. Demographic
2. Behavioural
3. Interactions with things and tools
(this links nicely to the next question about tools and things)
4. Tools/artefacts/things etc.
(this is /should be mostly quantitative factual reporting)
5. Feelings and choices, comparison etc.
(this provides space for the respondent to capture reflections and attribute meaning to the above self reporting)
Format tips:
If using a scale, or providing multiple options on a scale, always use an even number so they are forced to make a decision, and are not able to take the easy middle option.
Consider using Triads for responses, which is hard in a digital setting, but Dave Snowden and the Cynefin Sensemaker folk have a product that will do this, and the concept is open source so you can use the model anyway you want. Check it out here: https://thecynefin.co/sensemaker-2/
Use Design Thinking to get the questions right
If you have time, or need help getting the structure right (we always do) try doing two rounds.
The first is about understanding the key aspects and mental models of this space held by your target group. That initial sample will tell you how to structure the main one. In this case the first round would be quite open, using open ended questions, and allow free flowing responses, this can be managed at small scale with only a few participants.
Then you structure the main survey based on what you learnt, this will allow your scaled research to be of much better quality because you have anticipated the most likely and common ways of thinking about the problem (by your user).
This approach is the essence of design thinking, which can be executed in many ways digitally and in face to face settings. My short explanation of design thinking is that you do something, based on what you know, then check with the user/customer and learn about the problem by doing so. You’re not really checking the answer, but checking your understanding of the problem.
Card Sorting as an option
And another way to think about surveys, in a way that allows for emergent and new information, rather than locking down the user to your own biases and assumptions is Card Sorting. You may only be familiar with card sorting for when you are trying organise a website with lots of content, so you get users to sort it and you see how your navigation should be structured.
You can run a card sort in two ways, open, or closed. Closed assumes you know all the answers and provide the user/ customer with buckets to sort the content into, and they can’t add or change those buckets. There is also an open sort, in this case you provide the content and they can create their own buckets. In the case of trying to use a card sort to run a survey, there may be elements that benefit from an open card sort style approach. You may even run your ‘survey’ on a platform normally used for card sorting.
For card sorting I have always liked: https://www.optimalworkshop.com/optimalsort/
My good friend and colleague Dr. Sakib Jalil contributed to this article. He’s a very proper researcher, whilst I am a consultant.
