Topics
The handbook is also available as
- Print version (available from Verlag der TU Graz),
- free e-book (available via openlib.tugraz.at) and
- freely accessible prepared as a checklist in the TU Graz TeachCenter (currently only in German).
Contact
Advice and further information on diversity in research projects:
Armanda PILINGER
Phone: +43 316 873 6090
a.pilinger@tugraz.at
1. Focus
What is the focus of your research project?
Your research findings can be relevant for many groups of people.
Does your research relate to animals, tissue or cell material?
If humans, animals, tissue or cell material play a role in your research, diversity is relevant to your research project. In particular, this is obvious in applied technical research and development, if there are direct users, target groups or affected individuals.
Diversity aspects in research content are often less relevant to basic research.
Whether basic or applied research, it is always humans who do the research and who, at best, contribute to the results with a variety of knowledge and experience. The composition of the research team should therefore always be taken into account (cf. also the examples on "Research team" (e.g. 6.1 – 6.3).
Example 1.1. Publications and reports: Are your graphs readable for all target groups?
Publications and reports include graphs and tables to visualise results. Colours are also used. But if the colour codes are not chosen carefully, the desired differentiation is not perceptible to certain people. This is because 10% of the population is colour-blind or colour-faulty, with men being affected ten times more often than women. The colours red and green are the usual ones that some people find difficult or impossible to distinguish, difficulties with yellow and blue are much rarer.
This means if red and green colour codes are used for graphics and the associated lettering, then they are unusable or at least extremely difficult to interpret for those affected. Other colours or additional colour-independent coding can be used, for example different line types, shapes, and hatching.
The contact lens manufacturer Lenstore offers the experience of photographs seen as colour-blind people see them on the company website (https://www.lenstore.de/vc/farbwechsel/).
Literature for example 1.1.
- Colour Blind Awareness Organisation (2019). Website. http://www.colourblindawareness.org
- Shaffer, Jeffrey (2016). 5 tips on designing colorblind-friendly visualizations. Blogbeitrag Tableau Software. https://www.tableau.com/about/blog/2016/4/examining-data-viz-rules-dont-use-red-green-together-53463
Who will use and apply your research findings? Who is otherwise affected by your research?
In addition to direct users and target groups, funding agencies, public authorities or residents can also play an important role in your research. Analysing these different groups at the beginning of a research project and determining who should be considered and in what form can make a significant contribution to the success of the project.
Example 1.2. Software development: Who does the service reach via the smartphone?
For new technologies it is important to ensure acceptance and usability. Implicit assumptions can, however, lead to important target groups being neglected.
If, for example, services are now only offered via the smartphone, people with an affinity for technology, who may also be younger, are addressed. This is because access to technologies, interest in them and the necessary skills for their use are distributed differently in society. The scope of the application can be significantly improved if the needs and requirements of people who are remote from technology, critical of technology or older are taken into account (cf. Bath 2015, Marsden 2014).
Literature to example 1.2.
- Bath, Corinna (ed.) (2015). Gender, technology and mobility. Innovative, socio-technical solutions to social, scientific and economic challenges. Technical University of Braunschweig, Maria-Goeppert-Mayer-Professorship for Gender, Technology and Mobility. https://www.tu-braunschweig.de/index.php?eID=dumpFile&t=f&f=84793&token=473da4057d98ba748f783350531a84c42bba3279
- Marsden, Nicola (2014). Gender UseIt. HCI, usability and UX from a gender perspective. Guidelines for practice. https://www.gender-wissen-informatik.de/content/download/199/file/Gender-UseIT_20141105_final.pdf
What different needs do these groups of people have?
Various groups of people have different expectations from a technical product or the results of a research project (cf. Wilkowska, Gaul & Ziefle 2010). This may be related to different personal skills and abilities, working and living environment, physical characteristics, health, age, or gender. In addition, factors including diverse cultures and lifestyles, different economic and family circumstances such as care responsibilities are influential. These factors can also reinforce each other, e.g. if culturally women or men are more likely to use a device.
Sometimes there is also an important common basis, e.g. if a technical device is only used in certain professions or by people with a specific training.
Example 1.3. Methode Empathy Map: How can groups of people be analysed?
How can we determine which factors are relevant and how they are related? The best way is through a diverse group of people that reflects diversity of the users. In addition, tools such as Empathy Maps (Gray 2017) can be used to systematically describe representatives of a target group in terms of their environment, potential negative and positive attitudes and expectations or other aspects.
Literature for example 1.3.
- Wilkowska, Wiktoria; Gaul, Sylvia & Ziefle, Martina (2010). A Small but Significant Difference – The Role of Gender on Acceptance of Medical Assistive Technologies. In: G. Leitner, M. Hitz, and A. Holzinger (Eds.): HCI in Work and Learning, Life and Leisure. USAB 2010. Lecture Notes in Computer Science, vol 6389. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16607-5_6
- Gray, Dave (2017). Empathy Map - Blogeintrag. Gamestorming – A toolkit for innovators, rule-breakers and changemakers. https://gamestorming.com/empathy-mapping/
2. Literature
What insights into diversity aspects can be gained from the relevant literature?
Previous research may already have taken diversity aspects into account. A literature analysis also reveals gaps in existing research.
Which aspects of diversity have been examined to date and with what results?
When analysing the literature, pay attention to how diversity aspects are dealt with, even when these are not the focus of the respective publication.
Example 2.1. Machine learning: What are relevant variables for face recognition?
In the field of artificial intelligence, it is important to consider the type of data algorithms have been tested with. For example, a study (cf. Buolamwini & Gebru 2018) showed that commercial applications of facial recognition programs have very different marksmanship even in simple tasks such as automatically recognising the sex of a person in a photograph: Recognition was less frequently correct for women than for men, and the sex of darker skin types was classified much less accurately than that of lighter skin types. This is also related to the data used to train the programs. The over- or under-representation of certain groups can lead to unintended biases, e.g. with regard to gender, ethnicity or culture. For example, 45% of the images in the much-used "ImageNet" platform come from the USA, where only 4% of the world's population live. (cf. Zou & Schiebinger 2018)
Literatur for example 2.1.
Buolamwini, Joy & Gebru, Timnit (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, Proceedings of Machine Learning Research 81:1–15, 2018. http://proceedings.mlr.press/v81/buolamwini18a.html
Zou, James & Schiebinger, Londa (2018). AI can be sexist and racist – it’s time to make it fair, Nature. https://www.nature.com/articles/d41586-018-05707-8
Which methodologies can be found in the literature?
Pay attention to how the methodological approach is documented in literature (see also the examples on "Concepts & Models" (examples 3.1-3.3) and "Research Design" (examples 4.1-4.4)). Even seemingly neutral algorithms can reproduce prejudices when applied to information produced by humans. The unreflected use of algorithms assumed to be "neutral" ignores questions of morality, the desired handling of prejudices and social diversity.
Exmaple 2.2. Machine learning: What is to be questioned with the "word embedding" methodology?
“Word embeddings” are basic components of many machine learning applications in natural language processing. For the application in algorithms it is essential to represent words mathematically. Word embeddings are vectors that represent a word and its meaning in the context of other words. Words that are often used in similar contexts also have similar vectors. However, this also makes explicit any implicit biases and prejudices regarding sex / gender that are present in the training data. This is the case, for example, when "man" is associated with "computer programmer" in the same way as "woman" is associated with "homemaker" (cf. Bolukbasi et al. 2016). The widespread use of word embeddings bears the danger of further reinforcing these stereotypes. At the same time, there is also enough information available to reduce distortions and prejudices. Tolga Bolukbasi et al. (2016) propose a methodology to modify word embeddings so that gender stereotypes are removed but desired associations are preserved. For this purpose, gender-specific words are distinguished from gender-neutral words and the gender dimension in the vector is removed from the latter. (Bolukbasi et al. 2016; Becker 2018).
Literature for example 2.2.
Becker, Roland (2018). KI Basics – Word Embeddings – Methods for the representation of words in algorithms and neural networks. Website contribution, 21.5.2018. JUST ADD AI: https://jaai.de/word-embeddings-worteinbettung-word2vec-glove-bag-of-words-1872/
Bolukbasi, Tolga; Chang, Kai-Wei; Zou, James; Saligrama, Venkatesh; & Kalai, Adam (2016). Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain., 9p. https://papers.nips.cc/paper/6228-man-is-to-computer-programmer-as-woman-is-to-homemaker-debiasing-word-embeddings.pdf
What are the most important research gaps?
The following strategy regarding research gaps could be helpful (cf. Nieuwenhoven & Klinge 2010, Beery & Zucker 2011):
- If you have identified a research gap related to gender (no or doubtful knowledge about the existence of gender differences), you might consider biological gender as one of the priorities of your research.
- If gender differences are known, focus on overlapping (intersectional) aspects of diversity, but keep biological gender as an analytical unit where applicable. The investigation of the mechanisms and causes underlying the known differences could also be a priority.
- If previous work clearly indicates that there are no significant gender differences, check if there are other relevant dimensions of diversity.
Example 2.3. Automotive engineering: What has not yet been considered in the crash test?
Time and again, engineers and researchers have discovered gaps in research, for example in connection with crash test dummies, which are used for safety checks in the automotive industry. These dummies were first used in the late 1940s and were originally oriented towards the "young, white, strong, 70 kg man". Only later female models (from the late 1960s), children's models (1980s) and baby models (1990) were added (cf. Gendered Innovations – Pregnant Crash Test Dummies 2019). This served primarily to cover different body sizes. (Average body sizes vary not only between men and women, but also between different countries – for example, people from (northern) European countries such as Germany or the Netherlands tend to be tall, people from Asian countries tend to be short).
It was not until 1996 that a model was developed that took the special requirements of pregnancy into account. For example, it is difficult for pregnant women to buckle up with the three-point safety belt and in accidents it is even dangerous for the unborn child (cf. Bath 2015, Gendered Innovations – Pregnant Crash Test Dummies 2019, Linder & Svedberg 2019). Models are currently being developed in the USA that take into account the physique and special requirements of older and overweight people, who make up a growing proportion of the population (Kirkendoll 2017).
Literatur for example 2.3.
Bath, Corinna (ed.) (2015). Gender, technology and mobility. Innovative, socio-technical solutions to social, scientific and economic challenges. Technical University of Braunschweig, Maria-Goeppert-Mayer-Professorship for Gender, Technology and Mobility. https://www.tu-braunschweig.de/Medien-DB/gtm/bath/151002_tu-bs_imagebroschuere_gender_2-auflage_rz_k4.pdf
Beery, Annaliese K. & Zucker, Irving (2011). Sex Bias in Neuroscience and Biomedical Research. In: Neurosci Biobehav Rev. 2011 January ; 35(3): 565–572. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3008499/pdf/nihms230882.pdf
Gendered Innovations – Pregnant Crash Test Dummies (2019). Pregnant Crash Test Dummies: Rethinking Standards and Reference Models. https://genderedinnovations.stanford.edu/case-studies/crash.html#tabs-2
Kirkendoll, Shantell M. (2017). Car Crash-Test Dummies Move Beyond Young, Thin and Male. Artikel vom 11.1.2017, Michigan Health Lab, University of Michigan. https://labblog.uofmhealth.org/body-work/car-crash-test-dummies-move-beyond-young-thin-and-male
Linder, Astrid & Svedberg, Wanna (2019). Review of average sized male and female occupant models in European regulatory safety assessment tests and European laws: Gaps and bridging suggestions. Accident Analysis and Prevention 127:156-162. https://doi.org/10.1016/j.aap.2019.02.030
Niewenhoven, Linda & Klinge, Ineke (2010). Scientific Excellence in Applying Sex- and Gender-Sensitive Methods in Biomedical and Health Research. Journal of Women’s Health, Vol19, No. 2. DOI:10.1089/jwh.2008.1156
3. Concepts and Models
How are diversity aspects taken into account in different concepts and models?
A critical analysis shows whether a theoretical framework is suitable for your project or whether adjustments are necessary.
To what extent do the theoretical concepts and models that come into question for your research project take diversity aspects into account?
Implizite oder explizite "Hintergrundannahmen" innerhalb einer Forschungsgemeinschaft prägen die verwendeten Konzepte und Theorien und damit die Art und Weise, wie Forschung betrieben wird. Versuchen Sie, diese Annahmen zu analysieren und unbewusste Vorstellungen aufzudecken. Modelle vereinfachen die Sichtweise, können dadurch aber auch die Qualität und Innovationsfähigkeit der Forschung beeinträchtigen (vgl. Nieuwenhoven & Klinge 2010, S.318).
Example 3.1. Statistics and mobility: Which model takes the diversity of paths into account?
Mobility statistics are based on a model or category formation over different path purposes. In the Austrian mobility survey these are concrete: workplace, office/business, school/training, delivery and collection routes, shopping, errand, leisure, visit, other purpose (BMVIT 2016). However, care responsibilities and obligations are an important part of many of these "classical" path categories without being explicitly identified. The "mobility of care" concept takes this aspect into account, not only in statistics, and thus makes it possible to quantify the mobility behaviour of people with care responsibilities (cf. Sánchez de Madariaga 2013). At the same time, it also emphasises that people with care responsibilities face special challenges. These are often women who look after children, the elderly or people with disabilities. They often travel particularly complex routes, have to take into account the needs of the people they take care of, are dependent, for example, on the opening hours of childcare facilities, often have a lot of "luggage" (such as prams, wheelchairs, etc.). These aspects play an important role not least for the choice of transport mode and should be taken into account in transport planning as well as in vehicle development (cf. e.g. B-NK 2016, Gendered Innovations – Public Transportation 2019, Sánchez de Madariaga 2013).
Literature for example 3.1.
BMVIT (2016). Austria on the Move 2013/2014: Report on the results of the Austria-wide mobility survey "Austria on the Move 2013/2014". https://www.bmk.gv.at/dam/jcr:fbe20298-a4cf-46d9-bbee-01ad771a7fda/oeu_2013-2014_Ergebnisbericht.pdf
B-NK (2016). Mobility of persons with care responsibilities. Qualitative study (Gender Module) on the Austrian mobility survey "Austria on the Move 2013/2014". Office for Sustainable Competence (BNK). http://www.b-nk.at/wp-content/uploads/2019/12/Mobilitaet_von_Personen_mit_Betreuungsaufgaben.pdf
Gendered Innovations – Public Transportation (2019). Public Transportation: Rethinking Concepts and Theories. http://genderedinnovations.stanford.edu/case-studies/transportation.html#tabs-2
Niewenhoven, Linda & Klinge, Ineke (2010). Scientific Excellence in Applying Sex- and Gender-Sensitive Methods in Biomedical and Health Research. Journal of Women’s Health, Vol19, No. 2. DOI:10.1089/jwh.2008.1156
Sánchez de Madariaga, Inés (2013). From Women in Transport to Gender in Transport: Challenging Conceptual Frameworks for Improved Policymaking. Journal of International Affairs Vol. 67, No. 1, The Gender Issue: Beyond Exclusion (FALL/WINTER 2013), pp. 43-65. https://www.jstor.org/stable/24461671
In what contexts has the theoretical framework been developed and applied to date?
Our personal background characterises our approach to tasks and problems. The term "I-Methodology" refers to the tendency to develop products in such a way that they correspond to one's own interests and abilities, whereby other individuals (consciously or unconsciously) are disregarded (cf. European Commission 2013).
A simple differentiation between women and men can lead to stereotypical assumptions and thus ignore the actual requirements of users (cf. Buchmüller, Bath & Henze 2018).
Example 3.2. Gamification: Who plays which video games?
One example is the differentiation of "blue" (focus on combat) and "pink" (focus on fashion) video games based on the assumption that the interests and abilities of women and men are fundamentally different. However, it was shown that some of the most popular games are designed for a wide audience and are equally interesting for women and men (cf. Gendered Innovations – Video Games 2019). Nicola Marsden (2014) suggests in this context, e.g. for software development, to question exactly what can be learned from the results in order to improve the software for all users. The aim should be to make the software as inclusive as possible for different target groups and usage contexts, instead of unnecessarily reducing the circle of potential users.
Literature for example 3.2.
Buchmüller, Sandra; Bath, Corinna; Henze, Roman (2018). To whom does the driver's seat belong in the future? A case of negotiation between gender studies and automotive engineering. In Proceedings of 4th Gender&IT conference, Heilbronn, Germany (GenderIT’18). ACM, New York, NY, USA, 10 pages. Doi: 10.1145/3196839.3196866
European Commission (2013). Gendered Innovations. How Gender Analysis Contributes to Research. Report of the Expert Group “Innovation through Gender” Report: http://ec.europa.eu/research/science-society/document_library/pdf_06/gendered_innovations.pdf
Gendered Innovations – Video Games (2019). Video Games: Engineering Innovation Processes. https://genderedinnovations.stanford.edu/case-studies/games.html
Marsden, Nicola (2014). Gender UseIt. HCI, usability and UX from a gender perspective. Guidelines for practice. https://www.gender-wissen-informatik.de/content/download/199/file/Gender-UseIT_20141105_final.pdf
Is the corresponding theoretical framework suitable for integrating diversity aspects into your research, taking into account the guiding questions mentioned above?
Models make it possible to illustrate and edit complex relationships by omitting those details that are not considered relevant. Models can be used to test theories; they build on previous findings. However, they can also limit the researchers' thinking, precisely because some factors are not part of the model or the degree of abstraction or aggregation makes it difficult to include other factors or differentiate between them. By questioning the basic assumptions and critically examining which analyses are workable on the basis of the model, it is possible to both improve the results and to develop new research questions.
Example 3.3. Building technology: Is the simulation model suitable for determining energy efficiency?
Numerous simulation models for the energy efficiency of buildings have already been developed that take the behaviour of users into account. These are mostly stochastic models which, on the basis of large amounts of data from long-term studies, establish statistical relationships between environmental factors and the "strategy" of the users examined, e.g. opening windows, use of shading / lighting. In conventional simulations for buildings, complex relationships between user behaviour and energy demand are not represented (cf. Yan et al. 2015, Kim et al. 2017): For example, a simulation could show that the largest possible window area helps to make maximum use of daylight. However, when window areas are too large, users frequently find themselves dazzled by the light and therefore tend to use blinds and rely on electrical lighting. The use of blinds can also depend on whether the users would like to have a free view of the world outside, or would rather prefer to prevent others from looking in.
Literature for example 3.3.
Frigg, Roman and Hartmann, Stephan (2018). Models in Science. The Stanford Encyclopedia of Philosophy (Summer 2018 Edition), Edward N. Zalta (ed.). https://plato.stanford.edu/archives/sum2018/entries/models-science/
Kim, Jungsoo; de Dear, Richard; Parkinson, Thomas; Candido, Christhina (2017). Understanding patterns of adaptive comfort behaviour in the Sydney mixed-mode residential context. Energy and Buildings
141:274-283. https://doi.org/10.1016/j.enbuild.2017.02.061
Yan, Da; O’Brien, William; Hong, Tianzhen; Feng, Xiaohang; Gunay, H. Burak; Tahmasebi, Farhang; Mahdavi, Ardeshir (2015). Occupant behavior modeling for building performance simulation: Current state and future challenges. Energy and Buildings 107:264-278. https://doi.org/10.1016/j.enbuild.2015.08.032
4. Research design
Which research design delivers new, innovative findings?
A well-developed methodology allows you to identify all the relevant variables and to collect the necessary data.
How can you operationalise the diversity aspects that are of interest to you?
Interesting aspects cannot always be directly collected or measured – operationalisation determines which indicators are to be used to record theoretical concepts (cf. e.g. Meyer 2004, Döring 2013). This determines what exactly is observed and measured in order to subsequently test hypotheses and gain new insights.
Example 4.1. Automated driving: Research design as result of discourse
Intensive discussion processes in a research team are sometimes necessary in order to achieve a common understanding. Buchmüller, Bath & Henze (2018) describe the collaboration of engineers and gender researchers in the field of automated driving. Fundamental concepts about people, technology, science and research differ between these groups and had to be discussed. The common interest in the vehicle users was ultimately essential in order to mediate between different intentions of the research partners:
- Technical research often considers situational differences such as fatigue, distraction, stress, or mental and personal aspects (e.g. age, driving experience).
- Automated driving can be rated both positively and negatively: comfort and efficiency are rated positively; technical reliability, safety, loss of control, or reduced pleasure in driving are rated negatively. Young men in particular often consider driving as an essential part of their identity and tend to reject automated driving.
On this basis, the researchers agreed to integrate socio-cultural aspects into their research design and to concentrate on particularly demanding groups of users (e.g. highly risk-oriented drivers).
Literature for example 4.1.
Buchmüller, Sandra; Bath, Corinna; Henze, Roman (2018). To whom does the driver's seat belong in the future? A case of negotiation between gender studies and automotive engineering. In Proceedings of 4th Gender&IT conference, Heilbronn, Germany (GenderIT’18). ACM, New York, NY, USA, 10 pages. Doi: 10.1145/3196839.3196866
Döring, Nicola (2013). On the Operationalization of Gender in Questionnaires: Problems and Approaches from the Perspective of Measurement, Survey, Gender and Queer Theory. GENDER Magazine 2/2013, p.94-113. http://www.nicola-doering.de/wp-content/uploads/2014/08/D%C3%B6ring-2013-Zur-Operationalisierung-von-Geschlecht-im-Fragebogen.pdf
Meyer, Wolfgang (2004). Indicator development: A practice-oriented introduction. 2nd edition. Saarbrücken: Centre for Evaluation, 2004. CEval Working Papers 10. http://new.ceval.de/modx/fileadmin/user_upload/PDFs/workpaper10.pdf
Does the chosen methodology provide an adequate database for your research questions?
Nieuwenhoven & Klinge (2010) provide a series of questions on gender and gender-specific methods that allow gender-specific data to be collected (this also applies to other aspects of diversity):
- Is there any reason given why women or men are included or excluded?
- Is it necessary or possible to collect gender-specific data?
- Is it necessary to validate an instrument developed for different genders or different groups of people?
- Is the existing instrument validated for different genders/groups of people? If not, should it still be used for these groups?
Example 4.2. Assistance technologies: What database is required for the development of nursing robots?
Cultural differences have hardly been taken into account in the development of assistance robots. In a multidisciplinary, European-Japanese project (CARESSES), this gap is to be closed and an interculturally competent robot for elderly care is to be developed. The robot should support older people in their everyday lives and be able to adapt to their culture-specific needs and preferences in terms of behaviour, language and interaction. This is intended to increase the acceptance of the technology by those affected as well as the marketability of the robot. (cf. Bruno et al. 2017, CARESSES 2019). At the same time, it should be kept in mind that the majority of both older people and their carers are women. Differences with regard to the frequency of certain diseases, social and physical needs and interaction with technologies can play a role (cf. European Commission 2013, Gendered Innovations – Assistive Technologies 2019, Eiffe et al. 2012, Statistics Austria 2019).
By considering all aspects from the beginning, the necessary database can be created to use them for the development of assistance technologies.
Literatur for example 4.2.
Bruno, Barbara; Chong, Nak Young; Kamide, Hiroko; Kanoria, Sanjeev; Lee, Jaeryoung; Lim, Yuto; Pandey, Amit Kumar; Papadopoulos, Chris; Papadopoulos, Irena; Pecora, Federico; Saffiotti, Alessandro; Sgorbissa, Antonio (2017). The CARESSES EU-Japan Project: Making Assistive Robots Culturally Competent. In: Casiddu N., Porfirione C., Monteriù A., Cavallo F. (eds) Ambient Assisted Living. ForItAAL 2017. Lecture Notes in Electrical Engineering, vol 540. Springer, Cham. https://doi.org/10.1007/978-3-030-04672-9_10
CARESSES (2019). CARESSES - Culture-Aware Robots and Environmental Sensor Systems for Elderly Support. Projektwebsite. http://caressesrobot.org/en/
Eiffe, Franz Ferdinand; Till, Matthias; Datler, Georg; Heuberger, Richard; Glaser, Thomas; Kafka, Elisabeth; Lamei, Nadja; Skina, Magdalena; Till-Tentscher, Ursula (2012). Social situation of older people in Austria. Federal Ministry of Labour, Social Affairs and Consumer Protection (BMASK). Online: http://www.studienreihe.at/cms/Z02/Z02_2.a/1342538152381/bisherige-studien/studien-einzelband-uebersicht/band-11-soziale-lage-aelterer-menschen-in-oesterreich
European Commission (2013). Gendered Innovations. How Gender Analysis Contributes to Research. Report of the Expert Group “Innovation through Gender” Report: http://ec.europa.eu/research/science-society/document_library/pdf_06/gendered_innovations.pdf
Gendered Innovations – Assistive Technologies (2019). Exploring Markets for Assistive Technologies for the Elderly. http://genderedinnovations.stanford.edu/case-studies/robots.html#tabs-2
Niewenhoven, Linda & Klinge, Ineke (2010). Scientific Excellence in Applying Sex- and Gender-Sensitive Methods in Biomedical and Health Research. Journal of Women’s Health, Vol19, No. 2. DOI: 10.1089/jwh.2008.1156
Statistics Austria (2019). Population by age and sex. Online: https://www.statistik.at/web_de/statistiken/menschen_und_gesellschaft/bevoelkerung/bevoelkerungsstruktur/bevoelkerung_nach_alter_geschlecht/index.html
How are the perspectives of the potential user groups integrated?
By involving study participants, users or affected individuals, "blind spots" can be identified, study designs tested, or results checked. One method is the participatory design approach, in which people are involved in the development process right from the start (from setting research goals to collecting and evaluating data to usability). For example, users are encouraged to think about their needs and requirements and express them explicitly.
Example 4.3. Civil engineering: Who are the water supply experts?
When developing infrastructure for water supply in developing countries, it is important to involve women in a targeted manner: women are often responsible for fetching water in these regions by tradition and as a consequence have specialised knowledge of water sources and other challenges (cf. Schiebinger & Schraudner 2011; Gendered Innovations – Water Infrastructure). The question of who is being spoken to is therefore decisive, e.g. if the division of labour is gender-specific.
Literature for example 4.3.
Gendered Innovations – Participatory Research and Design (2019). http://genderedinnovations.stanford.edu/methods/participatory.html
Gendered Innovations – Water Infrastructure (2019). Water Infrastructure: Participatory Research and Design. http://genderedinnovations.stanford.edu/case-studies/water.html#tabs-2
Schiebinger, Londa & Schraudner, Martina (2011). Interdisciplinary Approaches to Achieving Gendered Innovations in Science, Medicine, and Engineering. Interdisciplinary Science Reviews, Vol. 3 No 2, June 2011, pp. 154-167. https://doi.org/10.1179/030801811X13013181961518
How should the study population be adapted to achieve the desired objectives?
Include different groups of people in your sample (gender, age, etc.). If you investigate specific aspects of diversity that are of particular interest to you, indicate this in your publications and explain the reasons for your choice.
Example 4.4. Software development: Whose knowledge is decisive?
Studies of the division of labour in service call centres have shown that most employees with direct customer contact are women. These people worked typically with software based on the management's assessment of the needs. The direct observation of the concrete workflows in the call centre led to the adaptation of the software used and thus to an increase in productivity. (cf. European Commission 2013, Maass et al. 2007, Russell 2008)
Literature for example 4.4.
European Commission (2013). Gendered Innovations. How Gender Analysis Contributes to Research. Report of the Expert Group “Innovation through Gender” Report: http://ec.europa.eu/research/science-society/document_library/pdf_06/gendered_innovations.pdf
Maass, Susanne & Rommes, Els (2007). Uncovering the Invisible: Gender-Sensitive Analysis of Call Center Work and Software. In Zorn, I., Maass, S., Rommes, E., Schirmer, C., & Schelhowe, H. (Eds.), Gender Designs IT: Construction and Deconstruction of Information Society Technology, pp. 97-109. Berlin: VS Verlag Für Sozialwissenschaften
Russell, Bob (2008). Call Centres: A Decade of Research. International Journal of Management Reviews, 10 (3), 195-219. https://doi.org/10.1111/j.1468-2370.2008.00241.x
5.Results
Is a different interpretation of the results conceivable?
A critical evaluation of your findings avoids distortions and explanations that can easily be questioned. It is also important for the purpose of planning the future use of your findings.
What insights does your data provide?
Analyse and report results broken down by gender and other aspects (cf. Beery & Zucker 2011, Niewenhoven & Klinge 2010, European Commission 2011):
- In many cases it will be useful for you to choose gender as the central variable and analyse other variables in relation to it (e.g. gender and age, gender and income, gender and mobility, gender and work).
- Also report if differences have turned out to be irrelevant.
- Visualise differences in tables, numbers, and conclusions.
- If possible, present individual data points as well as statistical differences of the mean value – this takes individuals with their respective variations into account.
- Communicate the influence of the examined diversity characteristics on participation, continuation, and discontinuation rates transparently.
Example 5.1. Machine learning: Which data sets were used to train algorithms?
In the area of machine learning, it is particularly important to identify transparently which data sets were used to train algorithms. The Data Nutrition Project aims to identify possible sources of bias in the data sets used before models and algorithms are developed based on this data.
To this end, a standardised overview of the contents of data sets is proposed in the style of a nutrition label. In different, flexibly combinable modules, this includes metadata, the origin of the data, variables, statistics, distributions, missing data, correlations between selected variables, etc. When it comes to people, aspects such as gender or ethnicity play an important role. (https://datanutrition.media.mit.edu/)
Literature for example 5.1.
Beery, Annaliese K. & Zucker, Irving (2011). Sex Bias in Neuroscience and Biomedical Research. In: Neurosci Biobehav Rev. 2011 January ; 35(3): 565–572. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3008499/pdf/nihms230882.pdf
European Commission ( (2011). Toolkit Gender in EU-funded research. https://publications.europa.eu/en/publication-detail/-/publication/c17a4eba-49ab-40f1-bb7b-bb6faaf8dec8/language-en
Niewenhoven, Linda & Klinge, Ineke (2010). Scientific Excellence in Applying Sex- and Gender-Sensitive Methods in Biomedical and Health Research. Journal of Women’s Health, Vol19, No. 2. DOI:10.1089/jwh.2008.1156
The Data Nutrition Project (2018). The Data Nutrition Project. Empowering data scientists and policymakers with practical tools to improve AI outcomes. Website. https://datanutrition.media.mit.edu/
What different conclusions are conceivable?
The differentiation of groups of people brings with it the danger that differences are reproduced even when other contexts are more relevant. Consider the context of your study and identify environmental influences and overlaps / intersections (e.g. culture, geography, time, experience, gender). Discuss how the new information about differences and similarities of groups of people can be used for new research projects and in practice.
Example 5.2. Medical technology: Are gender, weight or height relevant?
For example, significant differences in the knee anatomy of men and women emerged when the data was observed statistically, and this led to the development of a gender-specific knee prosthesis. However, body size (which overlaps with gender, i.e. on average women are smaller than men) is a more important factor in the selection and fitting of prostheses than gender. (cf. European Commission 2013, Gendered Innovations – De-Gendering the Knee 2019)
Literature for example 5.2.
European Commission (2013). Gendered Innovations. How Gender Analysis Contributes to Research. Report of the Expert Group “Innovation through Gender” Report: http://ec.europa.eu/research/science-society/document_library/pdf_06/gendered_innovations.pdf
Gendered Innovations – De-Gendering the Knee (2019). De-Gendering the Knee: Overemphasizing Sex Differences as a Problem. https://genderedinnovations.stanford.edu/case-studies/knee.html
6. Research team
What is the composition of the research team?
In order to achieve excellent results, it is crucial to have team members with a variety of skills and competences. Working conditions and processes can contribute significantly to cooperation and foster excellent performance.
Have you identified the expertise required to address the diversity aspects of your research project?
Depending on the importance of diversity aspects in your research, you can ...
... recruit researchers with the necessary expertise for your core team;
... assign external diversity experts to provide training and consulting so that your team can develop the necessary expertise. (Such trainings are, for example, eligible for grants under Horizon 2020 projects, cf. European Commission 2019b). Knowing the methods of gender and diversity analysis will help your research team rethink research priorities and develop new ideas.
Example 6.1. Speech recognition: Does gender swapping lead to better results?
Since the proportion of men is high among engineers and designers, this can lead to a "male standard" if there is no awareness of the consequences. For example, early speech recognition systems only recognised male voices because the technology was adapted accordingly (cf. Bath 2009).
One approach that can be used here is gender swapping, where fictitious personas are described with concrete characteristics that symbolise groups of users: If the gender of a fictitious person is exchanged, gender aspects can be specifically reflected upon. (cf. Marsden 2014, Turner & Turner 2011).
Literature for example 6.1.
Bath, Corinna (2009). Searching for methodology: Feminist technology design in computer science. 9p. http://www.informatik.uni-bremen.de/soteg/gict2009/proceedings/GICT2009_Bath-geloescht.pdf
European Commission (2019b). Gender Equality in Horizon 2020. Participant Portal H2020 Online Manual. http://ec.europa.eu/research/participants/docs/h2020-funding-guide/cross-cutting-issues/gender_en.htm
Marsden, Nicola (2014). Gender-UseIT. HCI, Usability und UX unter Gendergesichtspunkten. Leitlinien für die Praxis. https://www.gender-wissen-informatik.de/content/download/199/file/Gender-UseIT_20141105_final.pdf
Turner, Phil & Turner, Susan (2011). Is stereotyping inevitable when designing with personas? Design Studies Vol. 32, Issue 1, January 2011:30-44. https://doi.org/10.1016/j.destud.2010.06.002
Does the composition of the team in terms of men and women correspond to the potential of your research area?
A balanced team composition must not always mean a 50% share of women, since the proportions depend on the discipline. Check whether the composition of your team corresponds to the relevant scientific discipline. This can also be argued in grant applications.
Example 6.2. Reference values: What specialist potentials are there in different disciplines?
The "She-Figures 2018", for example, indicate the number of doctoral students by gender and broad field of study for 2016 for the EU-28 countries (cf. European Commission 2019a). These figures are further broken down into seven areas for the natural and engineering sciences; no details are reported for other scientific fields.
In 2017, Elsevier conducted a study based on publications listed on Scopus and ScienceDirect combined with gender data from different platforms (cf. Elsevier 2017). The study contains the proportions of researchers who published in the period 2011-2015 in 27 different subject areas (EU28). According to this study, the proportion of women in computer science is 22%, in mechanical engineering 24%, in physics and astronomy 25% and in chemical engineering 35%.
Literature for example 6.2.
Elsevier (2017). Gender in the Global Research Landscape. Analysis of research performance through gender lends across 20 years, 12 geographies, and 27 subject area. Online: https://www.elsevier.com/__data/assets/pdf_file/0008/265661/ElsevierGenderReport_final_for-web.pdf
European Commission (2019a). SHE FIGURES 2018. Directorate-General for Research and Innovation. Online: https://ec.europa.eu/info/publications/she-figures-2018_en
Which processes and structures promote individual motivation and sustainable results?
Social diversity and interdisciplinary cooperation in a research team can be enriching, but also very challenging. In order to use the potential, mutual appreciation is required. Clear roles and responsibilities, shared visions and a common understanding of key concepts and terms are helpful.
Example 6.3. Mentoring: Who gets what support?
Mentoring is a frequently used measure to promote equal opportunities in career development. Susan Colantuono has stated in her studies that the topics discussed in mentoring can be very diverse. When the focus is on building self-confidence, some essential aspects of career development are neglected. It is important for a career to take on tasks that make competencies visible and are of strategic importance. This can be, for example, the possibility to publish independently or to take on a management function. Managers should therefore pay particular attention to whether tasks are distributed fairly and how team members are supported in the development of their careers. (cf. Colantuono 2012).
Literature for example 6.3.
Colantuono, Susan L (2012). Make the Most of Mentoring – Capitalize on Mentoring and Take your Career to the Next Level; Interlude Productions, Charlestown
