Hi Alin,

Thanks for your thoughtful reply and for bearing with me – I didn't see your comment until just now.

You're right in saying that paying people can change what people say or do. There are two things we do to get around this:

1. We don't handle the money directly. We use a third party recruiter in most instances to pay the participant. This mean that there's distance between us and the incentive. Their agreement is with the recruiter, and the recruiter just needs to make sure they turn up.

2. We brief the participant very thoroughly at the start of the interview and make it clear that there are no right or wrong answers. We remind the participant that we're running sessions like theirs all day, and that it's our job to test the prototype, not them. We ask them to behave naturally.

What alternative would you suggest, to not pay people? Unfortunately that's not a solution.

To not pay a participant would mean we have to rely on people who have spare time to take part in the study, and that immediately rewards only the most privileged people. It would also automatically target people who are more generous with their time, and would increase dramatically the rate of drop-outs, which would waste our time.

On your point about the variables like context, personality and motivation – those are recruitment factors, not things to test in a matrix. When we recruit people, we recruit carefully to a profile. Things like motivations and demographics would come into play then.

Thanks for raising the points. Let me know if you have any more questions.