PDA

View Full Version : RPG Survey (Please help me)



Jadev
2013-08-26, 01:23 PM
Hello. I'm currently enrolled in a Statistics course at my college and our teacher has told us to gather data about a subject of our choice, and me being the huge nerd that I am, I decided to focus on tabletop RPGs. I set up a survey (link below) and I'd really appreciate it if you all could go through it. It's only ten questions long. I'll be posting the data here after it's done as well. I need around 20-30 people to take it, but the more people answer the more accurate the data becomes.

Thanks in advance!

https://docs.google.com/forms/d/17k04HiFqMRlkeN4-EatSfcxvzIxaOdQT-NSaPJlKamo/viewform

JFahy
2013-08-26, 01:29 PM
Hey,

Just trying to do this, and a couple of the questions are ambiguous to the point where I think you're going to have headaches with the data.

In the last year I've had 21 Saturdays where I got together with my regular group - it was all part of the same Pathfinder campaign.

Now, pretend I've also done 4 sessions of a Vampire campaign that died out, and then 10 sessions in another Vampire campaign.

Have I played 35 games this year? Or 3 (3 campaigns)? Or 2 (Pathfinder and Vampire)?


And for the 'how many players' question, do I count the DM?


Sorry to be fussy, but I do contract research projects sometimes and this stuff jumps out at me. Trying to help!

(As long as I'm nagging you, one more - your age intervals overlap. If you've got 18-25, your next one should probably start at 26 or the question will play havok with the way people normally think of their age.)

Craft (Cheese)
2013-08-26, 01:31 PM
Question: When you ask "how many games" do you mean "how many sessions" or "how many campaigns?"

JFahy
2013-08-26, 01:36 PM
By the way, Jadev, if there's an analysis/reflection component to the assignment, you can write about this and how you had to refine the questions. It's a great illustration of how the assignment smoothed out some of your rough edges; your instructor will probably enjoy hearing about that.

JusticeZero
2013-08-26, 01:39 PM
You might want to better operationalize or choose a less loaded term for "Min-maxer".

JFahy
2013-08-26, 01:56 PM
You might want to better operationalize or choose a less loaded term for "Min-maxer".

Also a good point - and solved the same way the 'game' issue is. "For purposes of this question, a game is considered to be ________." "For purposes of this question, a 'min-maxer' is a player who _________________."

Similar thing for the question about 'new gamers'. Does that mean a total RPG virgin, or someone who's played less than x sessions, or someone who's played less than I have?

Jadev
2013-08-26, 02:12 PM
Wow. That was fast.

Thanks for all the feedback. I'm going to go edit a couple of the items that you all pointed out.

I've swapped out Min-maxer for powergamer. I was just trying to pick a more common term, but I should probably pick one that doesn't slant the question one way or another.

I've clarified most of the questions-- swapped out game for campaign, and added the bit about a DM not being considered a player for the purposes of the count.

Unfortunately, I had to delete the responses since I changed the questions somewhat.

Thanks for all the help guys!

JFahy
2013-08-26, 02:16 PM
Something like 'highly interested in optimization' might be best. With me and some of my players, even 'powergamer' is a pejorative. Welcome to the world of survey design. ;)

(Actually, there's something I'm uncertain about - are powergamers/minmaxers people who care about optimization? Or are they people who care only about optimization? It might be too inflammatory a topic for people to even know for sure. :smalltongue:)

You're quite welcome - I'll watch for the new version to show up.

Craft (Cheese)
2013-08-26, 02:24 PM
Similar thing for the question about 'new gamers'. Does that mean a total RPG virgin, or someone who's played less than x sessions, or someone who's played less than I have?

Or someone with more experience than myself, but I've never personally played with before?

JFahy
2013-08-26, 02:26 PM
Yeah, good catch - new to gaming, new to the game, or new to me?

(We better stop, or we'll end up with Zeno's paradox where he never finishes the survey because there's always one more suggestion that takes him a little bit closer to the perfect survey.)

CarpeGuitarrem
2013-08-26, 02:48 PM
If I can suggest one thing--less binary questions? I answered, for instance, in favor of "GM bends the rules occasionally", but I'm a strong supporter of the idea that the GM needs to adhere to the rules, only bending them when absolutely necessary. I'm not a fan of fudging. It didn't feel like my answer represented that well. A four-point or five-point scale on some of these questions would give you better data, I think.

Also, Jadev, love your avatar.

Slipperychicken
2013-08-26, 03:19 PM
I took it myself. One thing I've heard about survey-writing is to always have an "I don't know" response, so that people who aren't sure can complete the survey without skewing your data with false answers.

Terraoblivion
2013-08-26, 03:25 PM
Another problem with the survey is that it presumes that games are sequential and you only play in one at a time. I don't have a last campaign as I am currently in three, all online, and this is not too unusual among the people I know.

InQbait
2013-08-26, 03:48 PM
I rather liked your survey, Jadev. It wasn't perfect, but it was pretty good.
I am no expert when it comes to surveys, but, being a person who struggles with anxiety issues, I don't think you should obsess or lose sleep over making this survey perfect. I think there is no such thing as a perfect survey. You can't please everyone. I don't know your instructor personally, but I think he or she will find your survey at least adequate.

Joe the Rat
2013-08-26, 08:34 PM
Yeah, this iteration is looking much cleaner. And done.

Jay R
2013-08-26, 11:47 PM
Here's my critique of the survey, as I would grade it in Stat class.

It is crucial that the answers to a question be both mutually exclusive and exhaustive. In several cases, yours are not.

Questions on preference need to include an option for "no preference". For instance, while I care very much about the gamers I play with, I don't care if they are serious or joking.

Similarly, many games include, as part of the rules, the rule that DMs have the authority to change the rules when he or she deems it necessary. In such a game, a DM who "bends the rules" is in fact sticking to the rules.

In my experience, some powergamers make the game better; some make it worse. The primary issue is whether they are team players or self-centered. You question's wording makes it impossible for me to give either answer.

What is a role-playing based game rather than an action-based one? Are you asking about rules systems, GMs, players, or specific scenarios? Some people will say that, since they refer D&D to video games, the prefer role-playing games. Others will say that they prefer D&D to role-heavy systems, so they prefer action-based games. The question doesn't make a clear distinction between answers, because it can be read more than one way.

Most people create surveys with these kinds of problems. When I give this kind of assignment, at the end I point out that one of the purposes is to show people how hard it is to write unambiguous survey questions.

tasw
2013-08-27, 12:14 AM
Hey,

Just trying to do this, and a couple of the questions are ambiguous to the point where I think you're going to have headaches with the data.

In the last year I've had 21 Saturdays where I got together with my regular group - it was all part of the same Pathfinder campaign.

Now, pretend I've also done 4 sessions of a Vampire campaign that died out, and then 10 sessions in another Vampire campaign.

Have I played 35 games this year? Or 3 (3 campaigns)? Or 2 (Pathfinder and Vampire)?


And for the 'how many players' question, do I count the DM?


Sorry to be fussy, but I do contract research projects sometimes and this stuff jumps out at me. Trying to help!

(As long as I'm nagging you, one more - your age intervals overlap. If you've got 18-25, your next one should probably start at 26 or the question will play havok with the way people normally think of their age.)

I didnt pretend anything. Just answered the questions. made it easier.

Asmodai
2013-08-27, 05:24 AM
Would you mind sharing the data once you're done?

Rondodu
2013-08-27, 07:53 AM
Questions on preference need to include an option for "no preference". For instance, while I care very much about the gamers I play with, I don't care if they are serious or joking.That’s where I’m stuck right now.

I’m also not sure in person is the best way to express on a table with the other players. I don’t play online, but I’m pretty sure that people who do do it in person too. Or have the spambots really improved?


D&D SurveyBy the way, why that name? Should I read the whole survey as about D&D?

Joe the Rat
2013-08-27, 08:25 AM
This is half the reason so many surveys look so strange - filling out the corners of interpretation.

Personally I like having a no opinion option, but forced choice dichotomy is a thing. You usually see it in preference rankings though - where you have multiple pairings of different elements. Working some of these into Agree/Disagree statements (and a few "generally"s) would help avoid concerns about non-exclusive or non-opposing anchors. Some folks may not like that sort of monopole scale, but hey. (The other way you do it is present a 4-point scale, then reduce down to 2. It gives us fence-pole sitters a little room to lean without making us jump, and still gives you usable data for your 2x2s.)

Taking everything said here: Don't worry. This is the sort of feedback that will be useful when you get to the analysis stage. If you don't see patterns, this may indicate a lack of association between elements, or that the grumblings on specificity may be muddying the results. Making large scale changes to the wording and design of your survey at this point will also muddy the results - or else you have to throw out everything to date. If you are getting more responses than design complaints, leave it for now. Save the design issues for the discussion.

Winds
2013-08-27, 08:33 AM
If I can suggest one thing--less binary questions? I answered, for instance, in favor of "GM bends the rules occasionally", but I'm a strong supporter of the idea that the GM needs to adhere to the rules, only bending them when absolutely necessary. I'm not a fan of fudging. It didn't feel like my answer represented that well. A four-point or five-point scale on some of these questions would give you better data, I think.

Also, Jadev, love your avatar.


I had a similar problem with that question. I think the GM should know when to bend the rules-however, a GM bending the rules when it wasn't necessary led to some of my worst games. (Not saying that it needs to be changed again, I understand that that would mess up your gathering at this point.) You may need to bear in mind that question will likely reflect which style the answerer had a problem with most recently.