Course Description

This course will give students an overview of introductory and intermediate mixed methods that are appropriate for planning different study designs. Students will develop an understanding of how to plan, design, analyze, present, and report a study. In addition, they will develop an understanding of how to: perform a scoping literature review, conduct basic qualitative and quantitative analyses, design surveys, choose validated instruments, operate within an ethical framework, and understand disciplinary norms, including peer review. Students will have the opportunity to hear from experts from a diverse professional environment on topics such as: as big data, participatory research, co-design, games user research, games for impact, content analysis, and more.

Students will get to plan, design, analyze, and report a research study for a digital media project or other types of research similar to what is covered in class.1

Planning includes 

• stating the purpose of the research, the question or hypothesis to be studied, the intended participants in the study (e.g., sample and population), the proposed method of data collection 

• defining the intended audience for the research results 

• choosing the approach to be taken for analysis and the final reporting

Design is:

• creating or choosing the data-gathering instruments and refining/practicing methods 

• developing the participant list (or other stakeholders)

• defining the timeline for the study, i.e., data collection, analysis, and write up 

Analysis consists of 

• exploration of the data collected 

• mapping that data to conclusions and/or recommendations using relevant statistical and/or qualitative methods 

The analysis should be valid and convincing to the audience specified and should fulfill the goal of the research project.

Reporting includes both 

• a presentation as a video poster or (research poster)

• a written report to an appropriate audience

Finally, students will be able to read research reports, review presentations and findings with respect to assessing their validity, their applicability, and gain insights for new hypotheses and new designs.

Learning Objectives

Upon completion, students should be able to:

1. Understand very broadly how to think about research methods across different disciplines.

2. Understand user research methods, their application, and purpose in developing digital media.

3. Design a research project including:

• Reviewing relevant literature.

• Stating a researchable question or testable hypothesis clearly.

• Defining the method and procedure.

• Creating and administering any research tools or instruments.

4. Analyze and report the results of the research, including the use of statistical tools.

5. Document all results for the appropriate audience.

6. Understand relevant research literature with respect to the validity, applicability, and possibilities for follow-on research.

7. Understand the mechanics of a peer review process and the steps to prepare a manuscript for submission to a journal or conference.

Prerequisite(s): None.

Co-Requisite(s): None

Concurrent Enrollment: None.

Recommended Preparation: Prior experience in qualitative or quantitative research.

Course Notes

The course is taken for a letter grade. The course is Hybrid and Online (Zoom).

Technological Proficiency and Hardware/Software Required

Students will be required to use Zoom, Slack, Miro, Google Drive, Docs, Sheets, and Slides, all of which are provided for free by USC and the instructor, and bibliographic reference software, such as Zotero (free). A working webcam, microphone, and headphones are required for Zoom.

Resources:

• USC Computing Center Laptop Loaner Program

• Zoom information for students

• Software available to USC Campus

• Google Apps at USC

Datasets, Instruments, Analysis Software & Training

We will be talking a lot about statistics in this course, although the class at its core is not a statistical training course. We will use software that is provided by USC free (https://itservices.usc.edu/) to the students (JMP, NVivo or Atlas.ti). If you already have a favorite software for doing your work, you can keep using it. Resources will be provided for getting basic training on this software. Still, it will be more critical to obtain confidence and competence in research design than advanced data analysis methods. More details about all this will be posted in Slack. Sample datasets and questionnaires will also be provided in the class Google Drive.

Class Schedule

The class schedule is available only to enrolled students via the class GDrive.

Sample schedule from Spring 2023

Description and Assessment of Assignments

Labs & Exercises

Details will be posted in Slack weekly.

The Research Project

This class provides the opportunity for students to prepare and conduct their own research in stages and to produce a report and video about it. Students enrolled in the class come from different disciplines and backgrounds and have different goals, thus, the research project will adapt to their needs. Leveraging work in other classes is recommended. For example, research on a topic related to another class, evaluating a game you are making with a team in another class or someone else’s game, preparing a proposal for funding that requires preliminary data results, etc. Your research project can be qualitative or quantitative or use mixed methods. While the class is more oriented to social science and human-computer interaction methods, arts and humanities research will also be briefly discussed. The research project expected in this course will be focused on methods that generate and evaluate data. Systematic reviews and meta-analyses are acceptable research projects, but other types of reviews will not fulfill the course research project criteria.

Besides systematic reviews, research projects undertaken in class cannot be presented or published at a conference without prior human subjects research approval. This is a lengthy process that can take up to 2 months to get completed. If this is something you are interested in doing, you will need to discuss it with the instructor early in the semester.

Here are some examples of research projects:

• A scoping/systematic/meta-analysis review. The depth and breadth of evidence tend to dictate the type of review you can undertake).

• RITE study of a game at the appropriate stage of development for this type of method.

• Single hypothesis pre/post-play experiment.

• Evaluation of entertainment efficacy post-play using standardized questionnaires.

• Post-play interviewing/ethnography/expert panel.

• An observational field study to pilot an observational protocol.

• A custom survey that can be piloted with class students or other willing/captive audiences. Unless you have a captive population, custom surveys are not recommended because finding enough participants is very hard and the internet is full of student surveys.

• Secondary data analysis from chosen or assigned dataset.

• Authoring and submission of a human subjects research protocol in preparation for a study.

• An autoethnographic study based on your own experience.

• A content analysis study of text, videos or other media.

The format of the report depends highly on the intended audience. In order to satisfy transparency and reproducibility requirements, you can use a checklist and guidelines from the Equator Network. For example, systematic reviews follow these guidelines. Nonrandomized designs that are observational or interventional can also be reported using the TREND checklist. You can combine these and reviewing them will help you think about your study design and also to reduce scope. Your research project can be lofty if you have the skills and time to complete it. If you’re a beginner, keep it simple and manageable. The goal is for you to accomplish one start-to-finish project on your own (in terms of leadership) that you feel you can control and still learn something. The instructor and your peers will give you scoping advice. Research is a process of revisions and failures. Relax, plan, and collaborate and you will do fine!

The report is written in stages to get you to accomplish the following by the time it is done.

Grading criteria:

• 50% for intellectual merit

  • Is the theory and prior research well presented and understood? Are ethical implications discussed?

  • Does the study describe a concept broad or personal impact adequately?

  • Does the student demonstrate epistemological (disciplinary) comfort with included theory, methods and applications? (I don’t expect mastery, but comfort means you have read ‘enough’ to get your two feet on the ground)

• 50% for clarity, style, and organization

  • Is the main research problem/question clear and explained by the evidence?

  • Is the formatting consistent? Are the references correct? Are data presented and explained adequately?

  • Is the flow of the paper appropriate?

Course Load + Grade Calculation

Experiences

• Lectures/Workshops by Instructor

• 5 self-directed labs

• 5 in-class exercises

• 2-3 peer review cycles for the research report

• Mentored student research reports

• 3-5 invited guests

Assignments (100 points = A)

Table 1. List of Assignments

Grading Scale

Course final grades will be determined using the following scale: 

Table 2. Course Grading Scale

Ungrading

I have been practicing ungrading (and here) since before the pandemic. You have to try pretty hard to fail this class, and it is not because it is easy. There is no reason to fail it or get a bad grade. There is so much flexibility with lecture recording, scheduling, deadlines, and more. The only deadline I cannot change is the university’s end grading deadline. I cannot give everyone that date for submitting everything, but some people need an extension. Before you stress out, just reach out!

READINGS/RESOURCES

Weekly recommended readings are included in the CTIN 510 Weekly Readings and Library and can be read before or after class.  You will benefit from perusing the Supplemental Bibliography, as well as the CTIN 503 Readings & Resources Library, and doing your own research that should be reflected in your Research Journal.

You are not expected to read every listed resource within a given topic/theme. Some are more specialized and will appeal to you depending on your interests and plans. Try to read outside your discipline even if you do not fully understand everything. At a minimum, read all the abstracts/summaries/tables of content to be aware of the literature, even if you do not have time to read everything. 

As you read, make a note, even if tiny, about something in your Research Journal so that the memory stays.

The Research Journal

Similar to a scholarly journal, this will contain your notes on readings, prior art reflections, media resource impressions, and anything new that you find of interest. It will also include your notes of decisions and observations from in-class exercises, self-directed labs, and your research study.

The instructor has curated a collection of readings and media resources from various disciplines to help with this immersion. Some of the readings may intimidate or confuse you, but that should not stop you from trying to read, listen, and ask questions. Keep the journal in the medium of your preference, including paper, if that is your primary mode of note-taking. 

Use Slack to share anything you find that may be of interest to others, and let us know what you think about that resource. You will be asked to submit the research journal for grading at the end of the semester. For self-accountability, you can place these weekly or as they accumulate in your Google Drive folder. You will not be graded for “regularity.” You should aim for a minimum of 15 items in your journal from the weekly readings and also add notes from your own research toward your paper.

If you journal it regularly, your practice will improve with time so when you start do not obsess with perfection. Obsess with practice!

TIPS

• Article - How to (seriously) read a scientific paper

• Article - Keep a research journal that works for you

• Video - How To read a scholarly journal article

Grading Criteria:

• 50% for note-taking quality

  • Do the notes improve over time? Are they already fairly good in quality?

  • Is there variety in the note-taking style? (reflective, critical, and/or reactive)

  • Are there takeaways?

• 50% for content diversity

  • Did the student look at multiple thematic areas?

  • Did the student read/view beyond their discipline/interest area?

  • Did the student look/view beyond the syllabus?

GROUPWARE & ACCESS

We will use USC Google Drive to store class resources for depositing assignments, Slack to chat with each other and share links and extra credit assignments, Zoom for online sessions, and Blackboard only for grades and official announcements.

For the Zoom sessions, students should plan to be somewhere with good internet access, have headphones, and be in a quiet environment, ideally on their laptop. Some readings will be on ARES, but others can be requested via USC interlibrary loan ILLIAD (pdf or ebook delivery service).

All readings are available to students for free, and they will learn how to access them via the USC Library and Google Scholar connected to the USC Library. There will be demonstrations of Zotero for the management of readings and outputting references for your papers.

Students should use a service like grammarly.com to proofread their papers, especially if English is not their native language.

The instructor will introduce other tools to help manage the class (Miro and more).

ChatGPT & ΑΙ Use Policy

The future is upon us, and we cannot stop it. You should use AI to help yourself, especially for improving your writing skills or learning how to structure writing. As you may know, AI is not great with facts and not great with the level of detail and originality a graduate course requires. Moreover, you are not expert enough in some areas to properly use it. If your language skills are not very high, it can be hard to know if the recommendations made are any good. Regardless, these are tools you must use, and you should also disclose how you used them and cite them.

AI can be a great starting point and time-saving tool, but it will not truly do your homework in helping you develop the skills you need to thrive. For example, you can use AI to read papers for you and summarize them. That can help if you are doing a first pass or cut on papers you want to decide to read in-depth or maybe even a thematic analysis, but at some point, you have to get your hands dirty and read and make decisions, analyze, and reflect.

Delegating everything to AI will deprive you of the brain skills you need to be employable. So leverage the tools, but do not cheat yourself of core aspects of an education. You don’t need to suffer to learn, but you need to spend some time actually challenging yourself: those neural synapses do not wire together without experiences :)

Grammarly Premium helps me edit everything, for example, including this syllabus, but I surely don’t agree with all its suggestions…

Collaboration Skills

Being a good collaborator requires a lot of different skills. While not all team experiences exercise these skills equally, here is an example of the checklist you can use for self-assessment in the course. 

CHECKLIST FOR SELF-ASSESSMENT OF STUDENTS’ COLLABORATIVE SKILLS (adopted) 

(Applies to: the student individually, yes/no statements)

• I know exactly what my tasks in this group are. 

• When I faced difficulties, I looked for help both inside and outside of the group.

• I presented to the group sources or documents relating to the subject (e.g. books, texts, web sites, videos).

• I have put forward to the group the issues which are relevant to our work.

• I helped my colleagues when they asked me for help.

• I helped my colleagues when I realized that they had problems, even without them asking me for help.

• I know perfectly well what are the roles and tasks of each colleague in my group.

• Usually I express my views and opinions clearly to my group.

• I feel able to assess the contributions of my peers’ in the work done by our group.

• I believe I would not be able to do a better job myself than what was achieved by my group.

CHECKLIST FOR PEER ASSESSMENT OF STUDENTS’ COLLABORATIVE SKILLS (adopted) 

(Applies to: the team/group, yes/no statements)

• All group members actively contributed to the final product.

• Group members gave each other support and feedback.

• When the group was having trouble, other groups spontaneously helped.

• When the group asked for help, other groups helped them immediately.

• In the end, everyone seemed satisfied with their group’s work.

• Group members exchanged and negotiated between them their ideas, strategies, tools and/or resources to carry out the activity.

• The group defined the tasks and the role of each member early on.

• The group asked the opinion and suggestions of other groups.

• The group assisted and gave advice to other groups.

• The group requested comments to other groups before finishing the activity.

• The group accepted critical comments from other groups.

Diversity of Human Experience

The definition of health and happiness varies greatly from individual to individual, family to family, and community to community. While considering media and the design and evaluation of interventions, you will be asked to consider many factors of the diversity of the human condition and human experience. This may mean age, gender, sexual orientation, ethnicity, religion, race, socioeconomic status, location, literacy, ability/disability, health status, access to services, and other variables. While proposing and critiquing scholarship, interventions, and evaluations, consider how these variables may affect the experience and impact.

Instructor’s Personal Statement on Safe/Brave Spaces

The spirit of a successful interventionist requires cognitive flexibility, imagination, curiosity, rigor, openness, compassion, honesty, and courage. There is no potential for transformative experiences if you are not personally pushed a bit past what you can tolerate, from boredom to excitement. You should feel free to express dissent in a civil manner and express your entire range of emotions in a controlled way. Try not to fight rhetorical or emotional battles you are unprepared to lose in class. We are all here to learn from each other, and the class can always benefit from being more inclusive. I am relying on you to point out the holes in the system. Be brave so we can all be safe. We will make mistakes and recover.

If something happened in class was deeply upsetting, bring it up to the instructor privately. We can all be fragile or imperfect at times, but resilience is a commitment to finding ways to move forward together. That is all that I ask of everyone. On a “casual” day in a class of 10, there are at least 2-3 people who have experienced sexual trauma and 3-4 people with a history of adverse childhood experiences. Your instructor is no exception. This class is not here to replace clinical therapy: a person who aspires to become an interventionist should aspire to tend to their own traumas and understand how to self-care. If you discover some loose ends in this class. Don’t stop at discovery: healing is the ultimate act of defiance in an unjust world.  

COMPLEMENTARY PERSPECTIVES ON SAFE SPACES

Byron, K. (2017). From infantilizing to world making: Safe spaces and trigger warnings on campus. Family Relations, 66(1), 116–125. https://doi.org/10.1111/fare.12233

Sanson, M., Strange, D., & Garry, M. (2019). Trigger warnings are trivially helpful at reducing negative affect, intrusive thoughts, and avoidance. Clinical Psychological Science, 7(4), 778–793. https://doi.org/10.1177/2167702619827018

Jones, P. J., Bellet, B. W., & McNally, R. J. (2020). Helping or harming? The effect of trigger warnings on individuals with trauma histories. Clinical Psychological Science, 8(5), 905–917. https://doi.org/10.1177/2167702620921341

RECOMMENDED SELF-CARE & PSYCHOLOGICAL FIRST AID COURSES (FREE)

Mental Health Self-Care: This presentation discusses the different types of stress and their sources. The speaker also provides seven methods for reducing stress.

https://www.jhsph.edu/research/centers-and-institutes/johns-hopkins-center-for-public-health-preparedness/training/online/self-care.html

Psychological First Aid (PFA): A full-scale public health response to disasters must attend to both the physical and mental health needs of affected groups. The latter set of needs is especially important because most authorities agree that far more individuals will report psychologically-related complaints than will report physical symptoms directly stemming from the injury-causing agent or event. Because a large-scale emergency will overwhelm existing mental health response resources, psychological first aid—the provision of basic psychological care in the short-term aftermath of a traumatic event—is an important skill set that all public health workers should possess.

https://www.jhsph.edu/research/centers-and-institutes/johns-hopkins-center-for-public-health-preparedness/training/PFA.html

Course Content Distribution and Synchronous Session Recordings Policies

USC has policies that prohibit recording and distribution of any synchronous and asynchronous course content outside of the learning environment.

Recording a university class without the express permission of the instructor and announcement to the class, or unless conducted pursuant to an Office of Student Accessibility Services (OSAS) accommodation. Recording can inhibit free discussion in the future, and thus infringe on the academic freedom of other students as well as the instructor. (Living our Unifying Values: The USC Student Handbook, page 13).

Distribution or use of notes, recordings, exams, or other intellectual property, based on university classes or lectures without the express permission of the instructor for purposes other than individual or group study. This includes but is not limited to providing materials for distribution by services publishing course materials. This restriction on unauthorized use also applies to all information, which had been distributed to students or in any way had been displayed for use in relationship to the class, whether obtained in class, via email, on the internet, or via any other media. (Living our Unifying Values: The USC Student Handbook, page 13).

Statement on Academic Conduct and Support Systems

Academic Integrity

The University of Southern California is a learning community committed to developing successful scholars and researchers dedicated to the pursuit of knowledge and the dissemination of ideas. Academic misconduct, which includes any act of dishonesty in the production or submission of academic work, comprises the integrity of the person who commits the act and can impugn the perceived integrity of the entire university community. It stands in opposition to the university’s mission to research, educate, and contribute productively to our community and the world.

All students are expected to submit assignments that represent their own original work, and that have been prepared specifically for the course or section for which they have been submitted. You may not submit work written by others or “recycle” work prepared for other courses without obtaining written permission from the instructor(s).

Other violations of academic integrity include, but are not limited to, cheating, plagiarism, fabrication (e.g., falsifying data), collusion, knowingly assisting others in acts of academic dishonesty, and any act that gains or is intended to gain an unfair academic advantage.

The impact of academic dishonesty is far-reaching and is considered a serious offense against the university. All incidences of academic misconduct will be reported to the Office of Academic Integrity and could result in outcomes such as failure on the assignment, failure in the course, suspension, or even expulsion from the university.

For more information about academic integrity see the student handbook or the Office of Academic Integrity’s website, and university policies on Research and Scholarship Misconduct.

Please ask your instructor if you are unsure what constitutes unauthorized assistance on an exam or assignment, or what information requires citation and/or attribution.

Students and Disability Accommodations

USC welcomes students with disabilities into all of the University’s educational programs. The Office of Student Accessibility Services (OSAS) is responsible for the determination of appropriate accommodations for students who encounter disability-related barriers. Once a student has completed the OSAS process (registration, initial appointment, and submitted documentation) and accommodations are determined to be reasonable and appropriate, a Letter of Accommodation (LOA) will be available to generate for each course. The LOA must be given to each course instructor by the student and followed up with a discussion. This should be done as early in the semester as possible as accommodations are not retroactive. More information can be found at osas.usc.edu. You may contact OSAS at (213) 740-0776 or via email at [email protected].

Support Systems

Counseling and Mental Health - (213) 740-9355 – 24/7 on call

Free and confidential mental health treatment for students, including short-term psychotherapy, group counseling, stress fitness workshops, and crisis intervention.

988 Suicide and Crisis Lifeline - 988 for both calls and text messages – 24/7 on call

The 988 Suicide and Crisis Lifeline (formerly known as the National Suicide Prevention Lifeline) provides free and confidential emotional support to people in suicidal crisis or emotional distress 24 hours a day, 7 days a week, across the United States. The Lifeline is comprised of a national network of over 200 local crisis centers, combining custom local care and resources with national standards and best practices. The new, shorter phone number makes it easier for people to remember and access mental health crisis services (though the previous 1 (800) 273-8255 number will continue to function indefinitely) and represents a continued commitment to those in crisis.

Relationship and Sexual Violence Prevention Services (RSVP) - (213) 740-9355(WELL) – 24/7 on call

Free and confidential therapy services, workshops, and training for situations related to gender- and power-based harm (including sexual assault, intimate partner violence, and stalking).

Office for Equity, Equal Opportunity, and Title IX (EEO-TIX) - (213) 740-5086

Information about how to get help or help someone affected by harassment or discrimination, rights of protected classes, reporting options, and additional resources for students, faculty, staff, visitors, and applicants.

Reporting Incidents of Bias or Harassment - (213) 740-5086 or (213) 821-8298

Avenue to report incidents of bias, hate crimes, and microaggressions to the Office for Equity, Equal Opportunity, and Title for appropriate investigation, supportive measures, and response.

The Office of Student Accessibility Services (OSAS) - (213) 740-0776

OSAS ensures equal access for students with disabilities through providing academic accommodations and auxiliary aids in accordance with federal laws and university policy.

USC Campus Support and Intervention - (213) 740-0411

Assists students and families in resolving complex personal, financial, and academic issues adversely affecting their success as a student.

Diversity, Equity and Inclusion - (213) 740-2101

Information on events, programs and training, the Provost’s Diversity and Inclusion Council, Diversity Liaisons for each academic school, chronology, participation, and various resources for students.

USC Emergency - UPC: (213) 740-4321, HSC: (323) 442-1000 – 24/7 on call

Emergency assistance and avenue to report a crime. Latest updates regarding safety, including ways in which instruction will be continued if an officially declared emergency makes travel to campus infeasible.

USC Department of Public Safety - UPC: (213) 740-6000, HSC: (323) 442-1200 – 24/7 on call

Non-emergency assistance or information.

Office of the Ombuds - (213) 821-9556 (UPC) / (323-442-0382 (HSC)

A safe and confidential place to share your USC-related issues with a University Ombuds who will work with you to explore options or paths to manage your concern.

Occupational Therapy Faculty Practice - (323) 442-2850 or [email protected]

​Confidential Lifestyle Redesign services for USC students to support health promoting habits and routines that enhance quality of life and academic performance.