Unit 11

Unit 11:
How to Design and Administer Online Surveys

Unit 11: Assignment #1 (due before 11:59 pm Central on MON JUL 22):

In this Unit, you will be constructing and administering online surveys. To learn why it’s important that you learn how to design and administer online surveys:
1. First, review from Unit 1: Assignment #6 Brooklyn College’s summary of Coplin’s (2012) book, 10 Things Employers Want You to Learn in College. Note that on p. 12, under the heading “Gathering Information,” the skill of “Construct Surveys” is listed.
2. Second, read Tague’s (2004a) brief introduction, “When to Use a Survey,” which addresses the use of surveys outside of psychological science (i.e., beyond the purposes of basic scientific research).
3. Third, read cognitive psychologist J. P. de Ruiter’s (2017) tweet about the need to be knowledgeable when constructing surveys.
To learn whether your administering surveys as a class assignment requires IRB (Institutional Review Board) ethical approval for protection of human research participants:
1. First, read the University of Michigan’s (2004) “Research Ethics and Compliance Policy.” Pay attention to the “Student Class Assignment Definition.”
2. Second, read the University of Wisconsin-Madison’s (2015) “Student Research Policy.” Pay attention to Statement II.B (which is highlighted in yellow).
To consider topics for your surveys:
1. First, read closely through the PSY 225: Online Survey Topics handout.
2. Second, be sure to read the two important notes on the right-hand side of the first page of the handout (one about assessing your access to research participants and the other about not soliciting students in this class or other classes who you don’t already know).
Select two survey topics (from the PSY 225: Online Survey Topics handout). Be sure to select two topics
1. that you’re interested in investigating, and
2. for which you have access to the appropriate survey participants.
Find, read, and synthesize previously published psychological science on each of the two topics you selected.
1. First, for each of the two survey topics you’ve selected, search Google Scholar for relevant scientific articles (using the procedure you learned in Unit 5).
  - Remember you can also find related articles by examining the “Cited By” tool and by examining an article’s reference list.
2. Second, for each of the two survey topics you’ve selected, identify three scientific articles (via Google Scholar) that are most relevant.
  - The three articles can be at any level in the Hierarchy of Scientific Evidence, and
  - all three articles can be at the same level in the Hierarchy of Scientific Evidence.
3. Third, for EACH of the TWO survey topics you’ve selected:
  - Read and analyze the three scientific articles you’ve found (using the procedure you learned in Unit 5).
  - Write one paragraph synthesizing the three articles you’ve found (using the procedure you learned in Unit 6).
  - Remember to synthesize the findings and not Mad Lib them; to write about behavior and phenomena, not researchers and their studies; to place each article’s in-text citation at the end of the sentence (in parentheses); and to synthesize conflicting results (e.g., using “However, ….”).
  - Remember that all paragraphs need a topic sentence, three supporting sentences, and a conclusion sentence (i.e., five sentences).
  - Lastly, include the three scientific articles’ full APA-style citations after each of your two synthesis paragraphs (using the procedure you learned in Unit 5 for APA-style citations).
Go to the Unit 11: Assignment #1 Discussion Board and make a new post that includes the following six parts:
1. First, in at least 50 words,
  - explain why you think constructing and administering surveys is a skill employers want you to learn in college, and
  - state at least two uses of online surveys for purposes other than basic scientific research. (These uses can come from Tague’s article or from surveys you’ve seen yourself.)
2. Second, in at least 50 words,
  - explain whether your designing and administering a survey as a class assignment requires IRB (Institutional Review Board) approval.
  - Do you agree with this policy? Why or why not?
3. Third, post the following four statements:
  - I understand that the two surveys I will be constructing and administering in this course are solely for the purpose of a class assignment.
  - I understand that I am not allowed to solicit as research participants people I do not know.
  - I understand that I am not allowed to ask instructors (including the instructor of this class) if I can solicit research participants from their class.
  - I understand that my research participants can be other members of this class, but I must already know the students, and I cannot ‘cold-call’ other class members.
4. Fourth, state the two topics you’ve selected for your two surveys.
5. Fifth, post the two paragraphs you’ve written that synthesize previous research on your two survey topics.
  - Be sure to include the APA-style full citations at the end of both paragraphs.

Unit 11: Assignment #2 (due before 11:59 pm Central on MON JUL 22):

To get a basic introduction to writing survey items, read Science Buddies’ (no date) article, “Designing a Survey.”
1. Make sure you understand the difference between open- versus close-ended items:
  - Close-ended items (also called structured items) offer the respondent a set of response options from which to choose.
  - Open-ended items (also called non-structured items) are items for which there is no list of response options from which to choose.
2. Also, make sure you understand when to use open-ended items: Open-ended items “are useful when you … have no basis to develop an all inclusive set of structured responses.”
  - “It is best to use [open-ended items] when you … don’t really know what to expect from the respondents”
To become more informed about writing survey items:
1. First, read Beretta’s (2014) article, “Top Ten Common Problems in Designing Effective Survey Questions.” Make sure you understand all ten problems Beretta describes – and you know how to avoid all ten problems.
2. Second, read Pew Research Center’s (no date) article, “Questionnaire Design.” Make sure you understand
  - open- versus closed-ended questions (the latter are what Science Buddies refers to as ‘structured questions’);
  - the importance of asking questions that are clear and well-specified;
  - what double-barreled questions and double-negatives are (and how to avoid them);
  - what acquiescence bias and social desirability bias are (and how to avoid them);
  - factors to consider in ordering your questions; and
  - the importance of placing demographic items last.
3. Third, to appreciate the power of wording, look at Britain Elects’ (2017) pair of tweets.
4. Fourth, to appreciate the importance of including the least number of survey items possible, look at cognitive psychologist, J. P. de Ruiter’s (2018) tweet.
5. Fifth, read Harvard University’s Program on Survey Research (2007) “Tip Sheet on Question Wording.” Make sure you understand
  - how to avoid technical jargon, vague or imprecise terms;
  - how to avoid double-barreled questions (again!);
  - how to avoid leading, emotional, or evocative language; and
  - how to use ordinal scales, reference frames, and unique answer choices that cover all response options.
6. Sixth, read Peters’ (no date) article, “How to Design a Survey.” Make sure you understand the differences between
  - categorical (also known as nominal) and
  - ordinal survey items (and survey responses).
Make a teaching document that captures information that you learned from each of the sources above.
1. The audience for your teaching document needs to be either other college students or people in industry (not psychological scientists).
2. The medium for your teaching document can be made from PPT, Google Slides, Keynote, or other presentation apps OR you can make an Infographic.
3. Your teaching document needs to be saved as a PDF, named YourLastname_PSY-225_SurveyDesign.pdf
4. Test the size of your PDF using this handout. If your PDF is too large, reduce its size using this handout.
Test yourself on Professor Rennison’s (no date), “Examples of Bad Questions & How to Fix Them.”
Go to the Unit 11: Assignment #2 Discussion Board and make a new post in which you do the following:
1. First, attach your teaching document.
2. Second, tell us the intended audience of your teaching document and why you chose that intended audience.
3. Third, tell us how well you did identifying the examples of bad questions and fixing them (on Professor Rennison’s quiz).

Unit 11: Assignment #3 (due before 11:59 pm Central on TUE JUL 23):

Now it’s time to apply everything you’ve learned about writing survey items and write the items for your two surveys.
1. For each of your two surveys, you must write no fewer than 5 and no more than 10 items.
2. For each of your two surveys, you must include at least one and no more than two open-ended items (which count toward your total 5 to 10 items per survey).
3. For each of your two surveys, you must include
  - at least one categorical/nominal survey item and
  - at least one ordinal survey item.
4. For each of your two surveys, if you include demographic items, you must place them at the end (unless there’s a good reason to place them earlier), and your demographic items count toward your total 5 to 10 items per survey.
After you’ve written a draft of your items for each of your two surveys:
1. First, check your items against the teaching document you made in Unit 11: Assignment #2.
2. Second, be sure to follow the beneficial procedures you told others to follow.
3. Third, be sure you don’t commit any errors you told others not to commit.
Save your two sets of items in one PDF, making it clear in your PDF which survey topic each set of items was written to assess. Name your PDF YourLastname_PSY-225_SurveyItems.pdf.
Go to the Unit 11: Assignment #3 Discussion Board and attach your PDF that includes your two sets of items.

Unit 11: Assignment #4 (due before 11:59 pm Central on TUE JUL 23):

Explore and then select one of the following (free) online survey platforms:
1. SurveyMonkey;
2. Qualtrics (which is free to UW-Madison students);
3. Google Forms;
4. LimeSurvey; or
5. Alchemer (formerly named SurveyGizmo).
Using the online survey platform you’ve selected, create each of your two surveys. For the title of each of your surveys, use your research topic (e.g., Multi-Tasking While Studying; Student Debt; Texting while Driving; Food Speed versus Food Quality).
Read Tague’s (2004b) steps for pilot-testing a survey, highlighted in yellow in the brief article, “How to Administer a Survey.”
Collect pilot data from three participants on each of your two surveys.
1. Your three pilot participants for each of your two surveys can be the same three people.
2. Or your pilot participants can be two different sets of three people or a mixture of same and different.
3. But you must collect pilot data from three pilot participants on each of your two surveys.
Go to the Unit 11: Assignment #4 Discussion Board and make a new post in which you do all of the following:
1. First, provide a link to each of your two surveys using the title of each of your surveys for the words that are linked.
  - Remember you can test your links before you click “Reply” by right-clicking and selecting “Open in New Tab” or something similar on your browser.
2. Second, make sure the links you provide for each of your two surveys do not require someone else to log in to that app or program.
  - You can test this by opening your links in a different browser than the one you used to set up your surveys (be sure to use a different browser, not a different tab in the same browser) or better yet try to open each link on a different device (e.g., your mobile phone or tablet if you set up your surveys on your laptop)
3. Third, write at least 200 words about what you learned from collecting pilot data about each of your two surveys, including what changes you need to make to each survey before you administer the survey to non-pilot participants
4. IMPORTANT: After completing Unit 11: Assignment #4, begin collecting your actual (not pilot) data!

Unit 11: Assignment #5 (due before 11:59 pm Central on WED JUL 24):

Read Boynton and Greenhalgh’s (2004) article, “Hands-on Guide to Questionnaire Research: Selecting, Designing, and Developing your Questionnaire.”
Pay specific attention to the sections titled “Could you use an existing instrument?” and “Is the questionnaire valid and reliable.”
NOTE: An “instrument” is a questionnaire, survey, scale, or other set of items and their response options.
Find (using Google Scholar or any other search mechanism) one existing instrument (questionnaire, survey, scale, or other set of items and their responses) that you might want to use someday in another psychological science research project.
1. The instrument must be available via Open Access (not paywalled and not licensed).
2. The instrument must have been used in previously published psychological science research.
3. The instrument must have some data previously reported about its validity and reliability.
4. The instrument does not need to be relevant to the two surveys you constructed in this Unit (but if the existing instrument is relevant to one or both of your surveys, that’s all the better).
Go to the Unit 11: Assignment #5 Discussion Board and make a new post in which you do the following:
1. First, identify and explain, in at least 200 words, the existing instrument, using as a rough model the identification and explanation of the Subjective Happiness Scale (note, however, this explanation is, unfortunately, fewer than 200 words).
  - your explanation should also include information about the instrument’s validity and reliability, and
  - your explanation should have appropriate in-text citations (at the ends of sentences in parentheses).
2. Second, list at least three “key references” (in full APA-style citation) for the instrument (the “key references” can include scientific articles that have used the instrument).
3. Third, attach (in PDF) a copy of the instrument (its items and response options) to your Discussion Board post (for example, if you had found the Subjective Happiness Scale, you would attach a PDF of that scale to your Discussion Board post).

Unit 11: Assignment #6 (due before 11:59 pm Central on WED JUL 24):

Meet online with your Chat Group for a one-hour text-based Chat.
Prior to your Chat Group meeting, all members of your Chat Group MUST have completed Assignments #1, #2, #3, and #4 in this Unit.
Begin your one-hour Group Chat by doing the following: Each Chat Group member needs to indicate ONE of the nine “How Are You Feeling At the START of Today’s Group Chat?” images. More than one Chat Group member can indicate the same image if that’s how they are feeling, and please refer to each image by its number
During the one-hour Group Chat, each Chat Group member will develop and write two Research Proposals, one for each of their two surveys, using this Research Proposal Outline.
- NOTE: The Research Proposal Outline is purposely not a fillable PDF.
- It is not a fillable PDF because you will need to copy the contents from the Research Proposal Outline and paste that content into a Word doc, Google doc, or another doc, which is where you will fill in the content you supply.
For your first survey:
1. Fill in your Research Proposal’s Title with the title of your survey, which will be your research topic (e.g., Multi-Tasking While Studying; Student Debt; Texting while Driving; Food Speed versus Food Quality).
  - Bold and center your Research Proposal’s Title.
2. With the help of your Chat Group, write the introduction to your Research Proposal by doing the following:
  - Write a Hook Paragraph that is an engaging introduction to your survey’s topic. (Note that this is a Hook Paragraph rather than simply a Hook Sentence.)
    - For examples of Hook Paragraphs, read this excerpt from Professor Gernsbacher’s (2018) article.
  - Add in the Literature Review that you wrote in Unit 11: Assignment #1.
  - Complete the Overview Paragraph by filling in the blanks that are in the Research Proposal Outline.
  - Your Hook Paragraph, your Literature Review, and your Overview Paragraph form the Introduction to your Research Proposal. However, none of these paragraphs will have sub-headings. Neither does your Introduction have a sub-heading.
3. With the help of your Chat Group, write the Method section of your Research Proposal by doing the following:
  - Complete your Participants paragraph by filling in the blanks that are in the Research Proposal Outline.
  - Complete your Materials paragraph by filling in the blanks that are in the Research Proposal Outline.
  - Complete your Procedure paragraph by filling in the blanks that are in the Research Proposal Outline.
  - Copy (without having to fill in any blanks) the Analyses paragraph of the Research Proposal Outline.
  - Make sure you understand why you will not be using inferential statistics to analyze your survey data (because inferential statistics are inappropriate with the size samples you are collecting and because inferential statistics are unwarranted without a priori hypotheses); if you don’t understand these limitations, discuss these issues with your Chat Group.
  - Your Participants paragraph, your Materials paragraph, your Procedures paragraph, and your Analyses paragraph form the Method section of your Research Proposal. All of these paragraphs DO have sub-headings, and the Method section has a centered and bolded sub-heading.
4. With the help of your Chat Group, complete your References section by filling in the APA-style references you wrote in Unit 11: Assignment #1. (Your References section also has a sub-heading that is centered and bolded.)
5. Save your first survey’s Research Proposal as a PDF that is named YourLastName_PSY-225_ResearchProposal_One.pdf and that
  - is double-spaced throughout;
  - has one-inch margins on the top, bottom, left, and right;
  - has page numbers on the bottom; and
  - has a title page that looks like this.
6. Repeat steps 1 through 5 above for your second survey, except this time, save your second Research Proposal as a PDF that is named YourLastName_PSY-225_ResearchProposal_Two.pdf.
At the end of your one-hour Group Chat:
1. Nominate one member of your Chat Group (who participated in the Chat) to make a post on the Unit 11: Assignment #6 Discussion Board, after the Group Chat is over, that summarizes your Group Chat in at least 200 words.
  - Attach the Research Proposal for your first survey to this Discussion Board Post.
  - Because the Discussion Board will allow only one file to be attached to each post, make a reply post to your own post.
  - Use your reply post to attach the Research Proposal for your second survey.
2. Nominate another member of your Chat Group (who participated in the Group Chat using the browser Chrome on their laptop, rather than on their mobile device) to, after the Group Chat is over, save the Chat transcript, in PDF, as described in the Course How To (under the topic, “How To Save and Attach a Chat Transcript”), and attach the Chat transcript, in PDF, to a Unit 11: Assignment #6 Discussion Board post.
  - Because the Discussion Board allows only one file to be attached to each post, make a reply post to your own post.
  - Use your reply post to attach the Research Proposal for your first survey.
  - Make a second reply post and use that second reply post to attach your Research Proposal for your second survey.
3. Nominate another member of your Chat Group (who also participated in the Chat) to make another post on the Unit 11: Assignment #6 Discussion Board, after the Group Chat is over, that states the name of the assignment (Unit 11: Assignment #6), the full name of your Chat Group, the first and last names of each Chat Group member who participated in the Group Chat, the day (e.g., Sunday) and date of this Group Chat (e.g., June 13), the start and stop time of this Group Chat (e.g., 1pm to 2pm), AND the day, date, start and stop time of your NEXT Group Chat (e.g., Sunday, June 20 from 1pm to 2pm).
  - Attach the Research Proposal for your first survey to this Discussion Board Post.
  - Because the Discussion Board will allow only one file to be attached to each post, make a reply post to your own post.
  - Use your reply post to attach the Research Proposal for your second survey.
4. If only two students participated in the Chat, then one of those two students needs to do two of the above three tasks (in two separate Discussion Board posts).
5. Before ending the Group Chat, do the following:
  - Each Chat Group member needs to indicate ONE of the nine “How Are You Feeling at the END of Today’s Group Chat?” images. More than one Chat Group member can indicate the same image if that’s how they are feeling, and please refer to each image by its number.
  - NOTE: The “How Are You Feeling at the END of Today’s Group Chat” grid of images differs from the “How Are You Feeling at the START of Today’s Group Chat” grid of images.
  - As a group, the Chat Group needs to arrange the day, date, and time that your Chat Group will meet to hold the Group Chat for the next Unit (Unit 12: Assignment #6).
All members of the Chat Group:
1. must proceed to collect data using their two surveys. Remember:
  - at least 10 participants per survey, and the 10 participants can be the same participants for both surveys or some combination of the same and different participants;
  - however, none of the 10 participants on either survey can be the same as your pilot participants for that survey; AND
  - none of the 10 participants on either survey can be members of your current Chat Group, but they can be members of your previous Chat Group if those students are not also members of your current Chat Group.
2. must record a typical Unit entry in their own Course Journal for Unit 11.

Congratulations, you have finished Unit 11! Onward to Unit 12!