Jump to content

When AI mirrors our world

Ethics, disparity and bias

When you use a language model in connection with your studies, you are not interacting with a neutral, objective search engine. You are interacting with a mirror of the internet and training data, and as such it becomes a mirror of social inequality too.

In order to use AI with academic and personal integrity, you must comprehend the ethical and material consequences of using the technology.

When AI mirrors our world

Ethics, disparity and bias

When you use a language model in connection with your studies, you are not interacting with a neutral, objective search engine. You are interacting with a mirror of the internet and training data, and as such it becomes a mirror of social inequality too.

In order to use AI with academic and personal integrity, you must comprehend the ethical and material consequences of using the technology.

Why language models are inherently biased

It is a misconception to assume that AI models are objective simply because they are data-driven. As they are statistical probability machines trained on vast quantities of data, primarily sourced from the internet, language models inevitably contain both the best and the worst aspects of human thinking.

Consequently, they will tend to:

  • Follow the majority view: AI systems generally select the statistically most dominant outcome, meaning that minority perspectives, alternative theories, and linguistic nuances are systematically underrepresented. It is worth noting that, in the context of language models, Danish cultural concepts, for example, may effectively constitute a minority perspective when compared with American ones. This is simply because the volume of American data vastly exceeds the volume of Danish data.
  • Reproduce and reinforce stereotypes: Because training data reflects historical biases and social inequalities, AI systems are likely to reproduce them. Image generators, for instance, frequently depict doctors as white men and nurses as young women because this pattern remains prevalent online. In a similar way, language models may reproduce associations between particular genders or minority groups and specific social roles or lower levels of professional competence.

What can I do myself?

Actively steer away from mainstream perspectives: Explicitly ask the model to examine an issue through the lens of specific alternative theories, minority perspectives, or non-Western contexts (for example, "Analyse this issue from a postcolonial perspective rather than within a traditional Western paradigm").

Contextualise the response within a Danish or Nordic framework: Remember that a model's default responses are often grounded in American social, legal, and cultural assumptions. Always specify if the analysis should take account of particular Danish institutional structures, legislation, or cultural characteristics.

Always verify academic claims against primary sources: Never use a language model as a primary source for empirical facts or historical information. Use it as a source of ideas and inspiration, and then verify its claims through academic library databases and scholarly literature. 

Future AI models risk being trained on an increasingly skewed and low-quality knowledge base, with a disproportionate amount of misinformation. Recent research points to a significant asymmetry in the material available online to AI chatbots, both for retrieval and as training data. While many reputable news organisations and politically neutral sources actively block AI crawlers in order to protect their content, misinformation websites and highly polarised sources often leave unrestricted access in place. This already has practical consequences today. For example, Copilot, Google, and ChatGPT cannot retrieve content directly from DR's website because access for such systems is blocked.

AI reflects your own thinking

Very often, you yourself are the greatest source of bias in an AI interaction. Our own blind spots and prejudices shape the dialogue and therefore constitute a central challenge in the relationship between humans and artificial intelligence.

We all have a biologically ingrained tendency towards confirmation bias: i.e. we unconsciously seek out and remember information that confirms our existing assumptions. This tendency can be reflected directly in the way we formulate prompts:

  • Leading prompts: If you ask, "Describe the relationship between X and Y", you are implicitly telling the model that such a relationship exists. Because language models are designed to be cooperative, they will often produce a convincing description of that relationship. This may occur even when the underlying reality is far more nuanced, or when no meaningful relationship exists at all.
  • The closed feedback loop: Interaction with AI is not a one-way input-output process but a dynamic exchange in which you influence the AI and the AI influences you. Your expectations and use of language shape how the model responds, and the model, in turn, reflects your way of thinking. If this is combined with automation bias (the tendency to place excessive trust in the system's outputs) and anchoring bias (the tendency to rely too heavily on the first information received from the AI), you may find yourself trapped in an unhelpful feedback loop. The system reinforces your assumptions, while you become increasingly confined within your own cognitive echo chamber.

These mechanisms affect more than just the quality of the responses you receive. They can also make it more difficult to maintain an active and enquiring learning process. When personal assumptions are automatically reinforced, or when AI-generated responses are accepted uncritically, there is a greater risk that the technology will gradually take over aspects of the analytical and reflective work that would otherwise contribute to your learning.

What can I do myself?

To counteract your own biases, you should adopt an adaptive and exploratory approach to your study practices and, above all, remain aware of your own influence on the interaction:

 Ask open questions: Always frame your prompts in an exploratory manner: "Is there a relationship between X and Y, and what evidence might challenge it?"

Actively seek counterarguments: Counteract your confirmation bias by explicitly prompting the model to explore arguments both for and against your own academic assumptions. 

AI creates a new learning gap

Paradoxically, research shows that AI tools tend to benefit academically strong students the most. These students possess the disciplinary knowledge and critical judgement needed to identify the model's errors and use it as a critical sparring partner. For students who are struggling academically, on the other hand, there is a much greater risk of unreflective cognitive offloading - i.e. if you value quickly AI-produced work, you learn less and risk a genuine stagnation of your learning process. 

What can I do myself?

Use AI as a Socratic sparring partner: Avoid asking the model to provide solutions or write passages for you. Instead, ask it to guide your thinking through questions. For example: "Ask me three probing questions about my choice of subject that will force me to refine and clarify my research problem."

If you would like to experiment with a chatbot specifically designed to act as a critical sparring partner that challenges your decisions and supports your academic development, you can try our Socratic PBL agent in Copilot.

Define your own academic success criteria first: Before prompting the model, take time to articulate what a strong academic response should contain, for example, which theories, concepts, or methodological requirements the task calls for. This helps you retain ownership of the process rather than allowing the model's potentially arbitrary focus to dictate your academic direction. You can also try a "brain-start"-exercise before opening your chatbot.

Turn processing into a learning activity: If you use AI to summarise a complex text, treat the summary as a reading guide rather than a substitute for the original source. Make sure you still read the key sections yourself in order to understand the nuances and underlying arguments.

AI's hidden climate costs

AI may feel immaterial, but language models place a considerable strain on the planet's resources through substantial electricity consumption and the vast quantities of water required to cool data centres. More than a third of AAU's students report that the climate and environmental consequences of generative AI are a source of concern in their everyday studies. For this reason alone, you should always consider when AI is the most appropriate tool for a task and when the same objective can be achieved using more energy-efficient alternatives, such as a conventional search engine or consulting your course literature.

What can I do myself?

Select the right tool for the task at hand: Consider whether you genuinely need an energy-intensive language model for simple information retrieval tasks that is often handled more accurately, and with a fraction of the resource consumption, by a traditional search engine or a database search.

Minimise unnecessary prompt iterations: Be precise and deliberate when formulating prompts from the outset to avoid lengthy, unstructured conversations that result in numerous unnecessary generations.

Reuse prompts and practices within your study group: Share effective prompts, working practices, and summaries within the group rather than having each member generate parallel responses to the same questions independently.

Questions for you and your project group

Built-in bias

  • Have you noticed whether the sources, theories, and examples suggested by a chatbot often seem rather one-sided, conventional, or Western-oriented? How will you actively ensure that academic nuance and critical alternative perspectives are represented in your project?

Leading prompts

  • If you look back at your three most recent conversations with AI, did you ask open and exploratory questions (e.g. "What arguments might challenge our hypothesis?"), or did you use leading prompts (e.g. "Help us explain why our solution is the best")?

Academic curiosity

  • In your study practice, how many of your prompts are driven by genuine academic curiosity, and how many are motivated by a desire for quick results (for example, asking AI to rephrase or summarise a text that you could realistically have worked through and structured yourself)?

Teaching material on generative AI when you study and use the library