Unit 11

Unit 11:
How to Design and Administer Online Surveys
Unit 11: Assignment #1 (due before 11:59 pm Central on THU JUL 26):

  1. In this Unit, you will be constructing and administering online surveys. To learn why it’s important that you learn how to design and administer online surveys:
    1. Review from Unit 1: Assignment #6 Brooklyn College’s summary of Coplin’s (2012) book, 10 Things Employers Want You to Learn in College. Note that on p. 12, under the heading “Gathering Information,” the skill of “Construct Surveys” is listed.
    2. Read Tague’s (2004a) brief introduction, “When to Use a Survey,” which addresses the use of surveys outside of psychological science (i.e., beyond the purposes of basic scientific research).
  2. To learn whether your administering surveys as a class assignment requires IRB (Institutional Review Board) ethical approval for protection of human research participants:
    1. Read the University of Michigan’s (2004) “Research Ethics and Compliance Policy.” Pay attention to the “Student Class Assignment Definition.”
    2. Read the University of Wisconsin-Madison’s (2015) “Student Research Policy.” Pay attention to Statement II.B (which is highlighted in yellow).
  3. To consider topics for your surveys:
    1. Read closely through the PSY 225: Online Survey Topics handout.
    2. Be sure to read the two important notes on the right-hand side of the first page (one about assessing your access to research participants and the other about not soliciting students in this class or other classes who you don’t already know).
  4. Select two survey topics (from the PSY 225: Online Survey Topics handout). Be sure to select two topics
    1. that you’re interested in investigating, and
    2. for which you have access to the appropriate survey participants.
  5. Read and synthesize previously published psychological science on each of the two topics you selected.
    1. For each of the two survey topics you’ve selected, search Google Scholar for relevant scientific articles (using the procedure you learned in Unit 5).
      • Remember you can also find related articles by examining the “Cited By” tool and by examining an article’s reference list.
    2. For each of the two survey topics you’ve selected, find three scientific articles (via Google Scholar).
      • The three articles can be at any level in the Hierarchy of Scientific Evidence, and
      • all three articles can be at the same level in the Hierarchy of Scientific Evidence.
    3. For each of the two survey topics you’ve selected:
      • Read and analyze the three scientific articles you’ve found (using the procedure you learned in Unit 5).
      • Write one paragraph synthesizing the three articles you’ve found (using the procedure you learned in Unit 6).
      • Remember to synthesize the findings and not Mad Lib them; to write about behavior and phenomena, not researchers and their studies; to place each article’s in-text citation at the end of the sentence (in parentheses); and to synthesize conflicting results (e.g., using “However, ….”).
      • Lastly, include the three scientific articles’ full APA-style citations after each of your two synthesis paragraphs (using the procedure you learned in Unit 5).
  6. Go to the Unit 11: Assignment #1 Discussion Board and make a new post that includes the following five parts:
    1. In at least 50 words,
      • explain why you think constructing and administering surveys is a skill employers want you to learn in college, and
      • state at least two uses of online surveys for purposes other than basic scientific research. (These uses can come from Tague’s article or from surveys you’ve seen yourself.)
    2. In at least 50 words,
      • explain whether your designing and administering a survey as a class assignment requires IRB (Institutional Review Board) approval.
      • Do you agree with this policy? Why or why not?
    3. Post the statements:
      • I understand that the two surveys I will be constructing and administering in this course are solely for the purpose of a class assignment.
      • I understand that I am not allowed to solicit as research participants people I do not know.
      • I understand that I am not allowed to ask instructors (including the instructor of this class) if I can solicit research participants from their class.
      • I understand that my research participants can be other members of this class, but I must already know the students, and I cannot ‘cold-call’ other class members.
    4. State the two topics you’ve selected for your two surveys.
    5. Post the two paragraphs you’ve written that synthesize previous research on your two survey topics.
    6. Include the APA-style full citations at the end of both paragraphs.

Unit 11: Assignment #2 (due before 11:59 pm Central on THU JUL 26):

  1. To get a basic introduction to writing survey items, read Science Buddies’ (no date) article, “Designing a Survey.” This article provides a basic introduction; if you already have some experience writing survey items, you may skim (though not skip) this article.
  2. To become more informed about writing survey items:
    1. Read Beretta’s (2014) article, “Top Ten Common Problems in Designing Effective Survey Questions.” Make sure you understand all ten problems Beretta describes – and you know how to avoid all ten problems.
    2. Read Pew Research Center’s (no date) article, “Questionnaire Design.” Make sure you understand
      • open- versus closed-ended questions (the latter are what Science Buddies refers to as ‘structured questions’);
      • the importance of asking questions that are clear and well-specified;
      • what double-barreled questions and double-negatives are (and how to avoid them);
      • what acquiescence bias and social desirability bias are (and how to avoid them);
      • factors to consider in ordering your questions; and
      • the importance of placing demographic items last.
    3. To appreciate the power of question wording, look at Britain Elects’ (2017) pair of tweets.
    4. Read Harvard University’s Program on Survey Research (2007) “Tip Sheet on Question Wording.” Make sure you understand
      • how to avoid technical jargon, vague or imprecise terms;
      • how to avoid double-barreled questions (again!);
      • how to avoid leading, emotional, or evocative language; and
      • how to use ordinal scales, reference frames, and unique answer choices that cover all response options.
    5. Read Peters’ (no date) article, “How to Design a Survey.” Make sure you understand the differences between
      • categorical (also known as nominal) and
      • ordinal survey items (and survey responses).
  3. Make a teaching document that captures all the information you learned in steps a. and b. above.
    1. Your audience for your teaching document needs to be either other college students or people in industry (not psychological scientists).
    2. Your medium for your teaching document needs to be either a PPT or an Infographic.
    3. Save your teaching document as a PDF, named YourLastname_SurveyDesign.pdf.
  4. Test yourself on Professor Rennison’s (no date), “Examples of Bad Questions & How to Fix Them.”
  5. Go to the Unit 11: Assignment #2 Discussion Board and make a new post in which you
    1. attach your teaching document;
    2. tell us the intended audience of your teaching document and why you chose that intended audience; and
    3. tell us how well you did identifying the examples of bad questions and fixing them (on Professor Rennison’s quiz).

Unit 11: Assignment #3 (due before 11:59 pm Central on FRI JUL 27):

  1. Now it’s time to apply everything you’ve learned about writing survey items and write the items for your two surveys.
    1. For each of your two surveys, you must write no fewer than 5 and no more than 10 items.
    2. For each of your two surveys, you must include at least one and no more than two open-ended items (which count toward your total 5 to 10 total items per survey).
    3. For each of your two surveys, you must include
      • at least one categorical/nominal survey item and
      • at least one ordinal survey item.
    4. For each of your two surveys, if you include demographic items, you must place them at the end (unless there’s a good reason to place them earlier), and your demographic items count toward your total 5 to 10 total items per survey.
  2. After you’ve written a draft of your items for each of your two surveys:
    1. Check your items against the teaching document you made in Unit 11: Assignment #2.
    2. Be sure to follow the beneficial procedures you told others to follow.
    3. Be sure you don’t commit any errors you told others not to commit.
  3. Save your two sets of items in one PDF, making it clear in your PDF which survey topic each set of items was written to assess. Name your PDF YourLastname_SurveyItems.pdf.
  4. Go to the Unit 11: Assignment #3 Discussion Board and attach your PDF that includes your two sets of items.

Unit 11: Assignment #4 (due before 11:59 pm Central on FRI JUL 27):

  1. Explore and then select one of the following (free) online survey platforms:
    1. SurveyMonkey;
    2. Qualtrics (which is free to UW-Madison students);
    3. Google Forms;
    4. LimeSurvey; or
    5. SurveyGizmo.
  2. Using the online survey platform you’ve selected, create each of your two surveys. Be sure to create a meaningful (and inviting) title for each of your two surveys.
  3. Read Tague’s (2004b) steps for pilot-testing a survey, highlighted in yellow in the brief article, “How to Administer a Survey.”
  4. Collect pilot data from three participants on each of your two surveys.
    1. Your three pilot participants for each of your two surveys can be the same three people.
    2. Or your pilot participants can be two different sets of three people or a mixture of same and different.
    3. But you must collect pilot data from three participants on each of your two surveys.
  5. Go to the Unit 11: Assignment #4 Discussion Board and make a new post in which you
    1. provide a link to each of your two surveys (using the title of each of your surveys for the words that are linked); and
    2. write at least 200 words about what you learned from collecting pilot data about each of your two surveys, including what changes you need to make to each survey before you administer the survey to non-pilot participants.

Unit 11: Assignment #5 (due before 11:59 pm Central on SUN JUL 29):

  1. Read Boynton and Greenhalgh’s (2004) article, “Hands-on Guide to Questionnaire Research: Selecting, Designing, and Developing your Questionnaire.”
  2. Pay specific attention to the sections titled “Could you use an existing instrument?” and “Is the questionnaire valid and reliable?”
  3. Find one existing instrument (survey) that you might want to use someday in another psychological science research project.
    1. The survey must be available via Open Access (not paywalled and not licensed).
    2. The survey must have been used in previously published psychological science research.
    3. The survey must have some data previously reported about its validity and reliability.
    4. The survey does not need to be relevant to the two surveys you constructed in this Unit (but if the existing survey is relevant, that’s all the better).
  4. Go to the Unit 11: Assignment #5 Discussion Board and make a new post in which you
    1. identify and explain, in at least 200 words, the existing survey, using as a rough model the identification and explanation of the Subjective Happiness Scale (note, however, this explanation is unfortunately fewer than 200 words);
      • your explanation should also include information about the survey’s validity and reliability; and
      • your explanation should have appropriate in-text citations (at the ends of sentences in parentheses, although such in-text citations are unfortunately not illustrated in the model).
    2. list at least three “key references” (in full APA-style citation) for the instrument (the “key references” can include scientific articles that have used the instrument); and
    3. attach (in PDF) a copy of the instrument (for example, this PDF of the Subjective Happiness Scale).

Unit 11: Assignment #6 (due before 11:59 pm Central on SUN JUL 29):

  1. Meet online with your Chat Group for a one-hour text-based Chat.
  2. Prior to your Chat Group meeting, all members of your Chat Group must have completed Assignments #1, #2, #3, and #4 in this Unit.
  3. During the one-hour text-based Chat, each Chat Group member will provide the links to each of their two surveys, and the other Chat Group members will first take each survey and then provide peer feedback on each survey.
  4. Use Beretta’s (2014) article, “Top Ten Common Problems in Designing Effective Survey Questions,” as your peer-review guidelines.
    1. Assess each survey according to each of Beretta’s ten common problems.
    2. If the survey contains any of Beretta’s ten common problems, Chat Group members should recommend how each problem can be fixed.
    3. Chat Group members should also recommend other suggestions to improve each survey.
  5. At the end of your one-hour Chat:
    1. Nominate one member of your Chat Group (who participated in the Chat) to make a post on the Unit 11: Assignment #6 Discussion Board that summarizes your Group Chat in at least 200 words.
    2. Nominate another member of your Chat Group (who also participated in the Chat) to save the Chat transcript as a .htm or .html file and attach the file to a Unit 11: Assignment #6 Discussion Board post.
    3. Nominate another member of your Chat Group (who also participated in the Chat) to make another post on the Unit 11: Assignment #6 Discussion Board that states the name of your Chat Group, the names of the Chat Group members who participated the Chat, the date of your Chat, and the start and stop time of your Chat.
    4. If only two students participated in the Chat, then one of those two students needs to do two of the above three tasks.
    5. Before ending the Group Chat, your Chat Group might want to arrange the time for the Group Chat you will need to hold during the next Unit (Unit 12: Assignment #6).
  6. All members of the Chat Group:
    1. must use the peer-feedback from their Chat Group to improve their two surveys.
    2. must then proceed to collecting data using their two surveys. Remember:
      • 10 participants per survey, and the 10 participants can be the same participants for both surveys or some combination of same and different;
      • however, none of the 10 participants on either survey can be the same as your pilot-participants for that survey; AND
      • none of the 10 participants on either survey can be members of your current Chat Group, but they can be members of your previous Chat Group if those students are not also in members of your current Chat Group.
    3. must record a typical Unit entry in their own Course Journal for Unit 11.

Congratulations, you have finished Unit 11! Onward to Unit 12!