Inclusivity Bugs in Online Courseware: A Field Study
Motivation: Although asynchronous online CS courses have enabled more diverse populations to access CS higher education, research shows that online CS-ed is far from inclusive, with women and other underrepresented groups continuing to face inclusion gaps. Worse, diversity/inclusion research in CS-ed has largely overlooked the online courseware—the web pages and course materials that populate the online learning platforms—that constitute asynchronous online CS-ed’s only mechanism of course delivery.
Objective: To investigate this aspect of CS-ed’s inclusivity, we conducted a three-phase field study with online CS faculty, with three research questions: (1) whether, how, and where online CS-ed’s courseware has inclusivity bugs; (2) whether an automated tool can detect them; and (3) how online CS faculty would make use of such a tool.
Method: In the study’s first phase, we facilitated online CS faculty members’ use of GenderMag (an inclusive design method) on two online CS courses to find their own courseware’s inclusivity bugs. In the second phase, we used a variant of the GenderMag Automated Inclusivity Detector (AID) tool to automatically locate a “vertical slice” of such courseware inclusivity bugs, and evaluated the tool’s accuracy. In the third phase, we investigated how online CS faculty used the tool to find inclusivity bugs in their own courseware.
Results: The results revealed 29 inclusivity bugs spanning 6 categories in the online courseware of 9 online CS courses; showed that the tool achieved an accuracy of 75% at finding such bugs; and revealed new insights into how a tool could help online CS faculty uncover assumptions about their own courseware to make it more inclusive.
Implications: As the first study to investigate the presence and types of cognitive- and gender-inclusivity bugs in online CS courseware and whether an automated tool can find them, our results reveal new possibilities for how to make online CS education a more inclusive virtual environment for gender-diverse students.
Wed 10 AugDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:30 - 14:30
Session 10: ResponsibilityResearch Papers at Aula Magna
Chair(s): Lisa Kaczmarczyk
|The Shortest Path to Ethics in AI: An Integrated Assignment Where Human Concerns Guide Technical Decisions|
Noelle Brown University of Utah, Koriann South University of Utah, Eliane Wiese University of UtahDOI
|Inclusivity Bugs in Online Courseware: A Field Study|
Amreeta Chatterjee Oregon State University, Lara Letaw Oregon State University, Rosalinda Garcia Oregon State University, Doshna Umma Reddy Oregon State University, Rudrajit Choudhuri Oregon State University, Sabyatha Sathish Kumar Oregon State University, Patricia Morreale Kean University, Anita Sarma Oregon State University, Margaret Burnett Oregon State UniversityDOI Pre-print