{"id":362,"date":"2021-03-16T20:23:03","date_gmt":"2021-03-16T20:23:03","guid":{"rendered":"https:\/\/connect.apsanet.org\/s42\/?p=362"},"modified":"2021-10-18T13:03:42","modified_gmt":"2021-10-18T13:03:42","slug":"do-your-qualitative-homework","status":"publish","type":"post","link":"https:\/\/connect.apsanet.org\/s42\/2021\/03\/16\/do-your-qualitative-homework\/","title":{"rendered":"Do Your (Qualitative) Homework"},"content":{"rendered":"\n

Melissa Michelson, Menlo College
for Lessons Learned the Hard Way<\/em> in The Experimental Political Scientist, Spring 2021<\/p>\n\n\n\n

True research requires staying open to the possibility of learning from our data and<\/em> our mistakes. Over the past two decades I have conducted hundreds of get-out-the-vote and persuasion experiments, most in cooperation with community organizations. Some of those efforts have included missteps and mistakes that figure prominently in my thinking when I launch newer projects.<\/p>\n\n\n\n

One of the biggest lessons I\u2019ve learned along the way is that it is important to do your (qualitative) homework prior to finalizing an experimental design. This decreases the likelihood of unintended missteps due to a lack of familiarity with a community. Teele (2019: 136) notes that \u201cknowing a good deal about the people and groups that are being studied\u201d can also help avoid potential harms to individuals and communities.<\/p>\n\n\n\n

In 2011, co-author Brian F. Harrison and I conducted an experiment in cooperation with a white-led advocacy organization seeking to increase support for marriage equality among Black Americans. We reached out to colleagues with relevant expertise in Black politics and survey experiments, Drs. Andra Gillespie and Tyson King-Meadows, for feedback on our preliminary research design. To my great embarrassment, they judged our instrument to be inappropriate and potentially counterproductive, in that the equating of the gay rights movement with the Black Civil Rights Movement might make recipients of the message less inclined to be open to receiving positive messages about gay rights. With their feedback in mind, we completely redesigned the experiment, generating successful (and useful, for our community partner) results (Harrison and Michelson 2017, chapter 5).<\/p>\n\n\n\n

Another example comes from a 2010 GOTV experiment conducted with a local registrar to test the effect of providing postage-paid return envelopes with vote-by-mail ballots, an experiment conducted in cooperation with Neil Malhotra, Andrew Healy, Don Green, Allison Carnegie, and Ali Valenzuela for the 2010 gubernatorial election in California (Michelson et al. 2012). 10,000 randomly chosen voters in San Mateo County received the envelopes with their usual ballot, while other registered voters receiving vote-by-mail ballots were sent the same (postage required) return envelope that had been used in previous elections. Then we waited for proof that reducing the cost of voting would increase participation.<\/p>\n\n\n\n

Spoiler alert: it didn\u2019t happen. Voters randomly assigned to treatment were no more likely to return their ballots. However, they were <\/em>more likely to have returned their ballot without using the envelope (e.g. by going to a polling place) rather than returning their ballot by mail. What could have gone wrong? Follow-up conversations with the registrar\u2019s office revealed that they had received a number of calls from concerned long-term vote-by-mail voters that the unusual envelope they had received was a mistake. Somewhat desperate to learn from an expensive failure, I sent invitations to registered voters in our experimental pool inviting them to come to my office for a face-to-face interview about vote-by-mail procedures. Those who showed up were shown and asked for feedback about the materials we had used for the treatment.<\/p>\n\n\n\n

References<\/strong><\/p>\n\n\n\n

Harrison, Brian F., and Melissa R. Michelson. 2017. Listen, We Need to Talk: How to Change Attitudes about LGBT Rights<\/em>. New York: Oxford University Press.<\/p>\n\n\n\n

Michelson, Melissa R et al. 2012. \u201cThe Effect of Prepaid Postage on Turnout: A Cautionary Tale for Election Administrators.\u201d Election Law Journal: Rules, Politics, and Policy<\/em> 11(3): 12.<\/p>\n","protected":false},"excerpt":{"rendered":"

Melissa Michelson, Menlo Collegefor Lessons Learned the Hard Way in The Experimental Political Scientist, Spring 2021 True research requires staying open to the possibility of learning from our data and our mistakes. Over the past two decades I have conducted hundreds of get-out-the-vote and persuasion experiments, most in cooperation with community organizations. Some of those … Read more<\/a><\/p>\n","protected":false},"author":23449,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4],"tags":[],"class_list":["post-362","post","type-post","status-publish","format-standard","hentry","category-newsletter"],"_links":{"self":[{"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/posts\/362","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/users\/23449"}],"replies":[{"embeddable":true,"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/comments?post=362"}],"version-history":[{"count":0,"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/posts\/362\/revisions"}],"wp:attachment":[{"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/media?parent=362"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/categories?post=362"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connect.apsanet.org\/s42\/wp-json\/wp\/v2\/tags?post=362"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}