Proposing Threshold Concepts in Machine Learning

ITiCSE 2026 · Madrid

Lisa Zhang

University of Toronto Mississauga

Gosia Migut

Delft University of Technology

Jesse H. Krijthe

Delft University of Technology

Who We Are

Lisa Zhang
University of Toronto Mississauga

• Associate Professor, Teaching
• MSc in ML (Toronto, Vector)
• Teaching ML since 2018: intro ML, deep learning, applied AI, AI safety
• ML/CS Education research
• EAAI co-chair 2026/2027; TOCE AE

Gosia Migut
Delft University of Technology

• Assistant Professor
• PhD in explainable ML (UvA)
• Teaching ML since 2012
• Hosts TU Delft ML Teacher Community
• ML Education Research; CS Education Research

Jesse Krijthe
Delft University of Technology

• Assistant Professor
• PhD in ML (Leiden)
• Teaching: Intro ML BSc (6×), Advanced ML MSc (6×), 50+ student projects
• Research: ML methodology & theory, causal inference, healthcare

28 years of combined ML teaching · North America + Europe
undergrad and graduate · educator and researcher

This is a position paper so let me talk a bit about our positionality. My name is Lisa Zhang, I am a teaching faculty at the University of Toronto in Canada. My two colleagues are both from the Delft University of Technology in the Netherlands, and both regret not being able to make it to ITiCSE. Gosia is more on the education-side and Jesse is an ML researcher. We are all active in CSEd education and machine learning education, with leadership roles both in our local context and more broadly.

This idea of Threshold Concept was on my mind for several years for a few reasons. - ML courses tend to be overwhelming for students with lots of models, ideas, skills to cover. - …and separately I work with writing studies faculty who had success structuring introductory academic writing courses through the TC framework. And when the three of us met last year following ITiCSE, we decided to explore ML TC more deeply

Threshold Concepts

“…akin to a portal, opening up a new and previously inaccessible way of thinking about something. It represents a transformed way of understanding, or interpreting, or viewing something without which the learner cannot progress.” — (Meyer and Land 2003)

As a quick background: threshold concepts were proposed in 2003 (Meyer and Land 2003) as a pedagogical framework and theory of learning centered on transformative experiences. They’re “like a portal” that makes accessible a way to understand the world through the lens of a particular discipline.

Threshold concepts are generally thought to have these five properties. We’ll focus on the first three today, as they’re seen as the more load-bearing ones.

A TC is transformative in that it shifts perspectives. It is troublesome, in the sense that it tends to be difficult for learners — so much that learners need to transition through what’s called the “liminal space,” where they exhibit signs of mimicry rather than true understanding. And a TC is integrative in that it unites many seemingly different ideas in a discipline.

Transformative fundamentally shifts the learner’s perspective

Troublesome difficult to grasp; conflict with prior understanding

Integrative unifies disparate concepts in a domain

Irreversible once understood, unlikely to be unlearned

Bounded limited to specific disciplinary boundaries

Examples of Threshold Concepts

Economics

⚖️

“Opportunity cost” (Meyer and Land 2003)

Computer Science

💻

Pointers; object-orientation (Boustedt et al. 2007)

Writing

✍️

“Writing is a Social and Rhetorical Activity” (Adler-Kassner and Wardle 2015)

Examples of threshold concepts include “opportunity cost” in economics, to “pointers” in CS, to broader ideas like viewing “writing as a social and rhetorical activity.” So here, what counts as a “concept” can vary in granularity.

We take a broader approach, much like what you see on the right, in hopes of identifying model-agnostic ideas that will stand the test of time. We’re also inspired by AI4K12’s “five big ideas” in AI, if you’re familiar with that.

So, unlike the CS2023 curricular guidelines, we’re not looking at “important topics,” disciplinary structure, “core ideas,” or which specific models we should teach. We’re instead centering on the transformative experiences we wish learners to undergo.

Concept granularity: ours will be broad, field-level ideas

…like the AI4K12 “five big ideas” (Touretzky et al. 2019)

Our Process

Threshold concepts are usually identified through consensus (Barradell 2013; Timmermans and Meyer 2019)

Threshold concepts are usually identified through consensus among many experts, for example, through a Delphi process.

We did not have a large group of experts. Instead, the three of us went for depth rather than breadth, usefulness rather than rigour.

We started with individual brainstorming, then held many rounds of discussion, and finally grounded our write-up of each proposed concept in the relevant literature.

We don’t claim exhaustiveness. But we see this work as both providing immediate value for structuring existing and new courses, and as a starting point for broader discussion.

I should also mention that whether TCs can be identified with empirical rigor at all is contested, which is partially why work in CS threshold concepts has slowed down, and a partially why we took this appraoch.

Our approach:

1. Brainstorm Independent brainstorming from our teaching + research experience

2. Discuss Several rounds of discussion on these candidate concepts

3. Ground Ground the arguments in prior work in ML research, ML education & ML/AI misconceptions

Whether TCs can be identified with empirical rigor at all is contested (Rowbottom 2007; Rountree and Rountree 2009)

What we brainstormed

Learning is Optimization

Bias/Variance: ML models are not
perfect

Model evaluation is empirical

“No Free Lunch”

Learning = Data + Algorithm

Different hyperparameter choices are essentially different models

Representation: everything is a vector

Data arise from probabilistic phenomena

The role of randomness

So here is an early list of ideas that we brainstormed.

…and how we grouped them

Learning is Optimization

Bias/Variance: ML models are not
perfect

Model evaluation is empirical

“No Free Lunch”

Learning = Data + Algorithm

Different hyperparameter choices are essentially different models

Representation: everything is a vector

Data arise from probabilistic phenomena

The role of randomness

In these early discussions, we realized that many ideas were And in these early discussions, we realized that many ideas were inter-related, and grouped them into these 5 major groups and named them.

…and how we grouped them

(4) ML Describes Geometric Processes

Representation: everything is a vector

(5) ML Demands a Probabilistic Lens

Data arise from probabilistic phenomena

The role of randomness

(1) Learning is Optimization

Learning is Optimization

(2) There are Tradeoffs in Sources of Error

Bias/Variance: ML models are not
perfect

(3) ML is an Empirical Science

Model evaluation is empirical

“No Free Lunch”

Learning = Data + Algorithm

Different hyperparameter choices are essentially different models

There are still inter-relations between these broad ideas, so we came up with this figure.

Five proposed threshold concepts

Seeing “learning as optimization” sits at the center.

On the right are the key transformations we want students applying ML models to undergo: understanding the tradeoffs in the sources of error, and viewing ML application as an empirical discipline — its epistemology grounded in experimentation, not derivation.

On the left are key transformations for understanding ML theory — what we want students developing new ML models to undergo: seeing machine learning not just procedurally, but as geometric processes, and also probabilistic processes.

1 · Learning is Optimization

I’ll go deep on a couple of these today to give you a sense of the kind of arguments we make in the paper, and move quickly through the rest — I hope you’ll take a look at the others yourselves. Fair warning: we’ll be talking about ML ideas in some depth here, so this next part will sound more like a class than a conference talk. Hopefully you’ll get something out of it either way.

Learning is Optimization

To describe an ML method, we describe:

• search space (model class)
• loss function
• optimization algorithm

Supervised Learning: minimize empirical risk

\[ \min_{f \in \mathcal{F}} \; \frac{1}{n}\sum_{i=1}^{n} \ell\big(f(x_i),\, y_i\big) \]

Reinforcement Learning: maximize expected discounted return

\[ \max_{\pi} \; \mathbb{E}_{\tau \sim \pi}\Big[\textstyle\sum_{t=0}^{T} \gamma^t r_t\Big] \]

So let’s start at the center: learning as optimization. What does this mean?

When we think of the word “learning,” it sounds like a complex psychological process. But when we define a learning method, we actually turn a learning problem into an optimization problem. We define: the search space (characterized by the model class), the loss function (the thing we optimize), and an optimization algorithm (like gradient descent). Through this framing, the abstract idea of “learning” gets turned into a concrete mathematical process.

For example, in supervised learning, where we minimize empirical risk across a dataset — and in reinforcement learning, where we maximize expected discounted return, using maybe Q-learning.

Learning is Optimization: Troublesome

Counters novices’ beliefs that:

🧠

ML works similarly to the human brain (Marx et al. 2024)

⚙️

ML behaviour is programmed rather than learned (Marx et al. 2024)

💾

Training data is stored inside the model (Bewersdorff et al. 2023)

This is a troublesome idea for novices because it’s counter to their preconceptions. In the literature, we see evidence that novices hold prior beliefs that ML works like the brain, that it’s programmed, or that training data is stored (Marx et al. 2024; Bewersdorff et al. 2023).

Learning is Optimization: Transformative

This shift lets researchers construct and communicate ML methods by specifying the optimization problem.

⋮

The idea is transformative in that it demystifies “learning” so it becomes a mechanical procedure rather than an intuitive notion.

Once the threshold is crossed, practitioners construct and communicate new methods by specifying the optimization problem. For example, the GAN paper introduces the approach with this objective, where “G” is the generator and “D” is the discriminator — I’d argue this is the most important line in that paper.

Learning is Optimization: Transformative

⋮

The shift also allows practitioners to influence the behaviour of a model by modifying the objective. This paper on algorithmic fairness is an example, where we can encode a preference for a certain fairness notion with this middle term “R” here.

Learning is Optimization: Integrative

Unsupervised learning: k-means clustering

Iteration 0 — click to begin

Framing learning as optimization is integrative: we saw it in supervised and reinforcement learning, and in algorithmic fairness. Here’s an unsupervised learning example. We typically introduce \(k\)-means clustering procedurally: alternating between a reassignment step, a refitting step, and repeating. But this process can be viewed as block coordinate descent on this objective!

The integrative power is being able to systematically reason about “what is this process implicitly optimizing?” to understand a new method — and to modify it.

Block coordinate descent on \(J = \sum_{i=1}^{n} \lVert x_i - \mu_{c(i)} \rVert^2\)

Systematic reasoning: “What is this process implicitly optimizing?”

Learning is Optimization: Integrative

AI alignment: does the system do what its designers intended, not just what the objective literally rewards?

Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure”

• hallucination in LLMs (optimizing for next-token prediction),
• sycophancy in LLMs (RLHF)

Issues of AI alignment become a natural extension.

3 · Machine Learning is an Empirical Science

Let’s jump to Concept (3).

Machine Learning is an Empirical Science

“Empirical”: grounded in observation and experimentation

This is an “applied ML” concept, and the term “Empirical” means grounded in observation and experimentation, rather than using theoretical or abstract reasoning. This is related to the “No Free Lunch” theorem, also in the CS2023 Curricular Guidelines.

You might have seen or used a diagram like this in your classes to show part of the model-building process, where we: train multiple models (maybe with different hyperparameters, model families, feature selection); set aside a validation set to support empirical experimentation to find the “best” choice; and use a test set for estimating generalization accuracy — again empirically.

It’s tempting to assume a certain input distribution and use theory to guide these choices, but real data is messy! Especially in high-stakes settings, nothing else can replace empirical exploration.

Machine Learning is an Empirical Science: Transformative

🔍

“Best algorithm”

→

So we want our learners to move away from thinking about the “best algorithm” under whatever assumption, and think more like a scientist performing experiments to find a good Problem-Data-Model alignment for their learning problem.

Questions related to proper evaluation — like selecting the right data set and splitting it appropriately — all become central, but traditionally underemphasized.

Machine Learning is an Empirical Science: Transformative

🎯

“Accuracy”

→

There is also a second transformative shift, where we want learners to shift from optimizing for a single metric like “accuracy” into really understanding “how, why, and for whom the model works.”

This means thinking about fairness, efficiency, robustness, interpretability, sustainability — and all of those require empirical evaluation and systematic model auditing.

Machine Learning is an Empirical Science: Troublesome

😭

Theory alone isn’t enough Accepting that theory can’t determine the “best” model is unsatisfying

🧪

Good evaluation is a skill Not always taught explicitly; reproducibility is a field-wide concern

This idea that theory is inadequate is just a hard thing for some students to accept. I also don’t think we really teach students how to conduct experiments, at least in my setting. This is probably why doing empirical evaluation well can also be hard for experts.

Machine Learning is an Empirical Science: Integrative

One lens for many failure modes:

⚖️

Overfitting & underfitting

🎛️

Lack of hyperparameter exploration

🗂️

Bias in training data

💧

Data leakage

This is an integrative concept, kind of by design. The need for empirical evaluation is a lens through which we can interpret all these failure modes. We’re taking the perspective that issues of unintentional, unanticipated harm are within the bounds of our discipline.

4 · ML Describes Geometric Processes

The last two concepts in the paper relate to geometry and probability, and might sound overly broad, so I really hope you give the arguments in the paper a chance. I’ll only have time to touch on geometry today.

ML Describes Geometric Processes: Transformative

“Natural images lie on a manifold within \(\mathbb{R}^D\)”

“The earlier layers of a Multi-Layer Perceptron learn features, and the final layer is a linear classifier on these features”

There are two transformations here that learners need to undergo to understand things like this.

The first is to think of data — any data, be it image, text, or graphs — as geometric objects. “Geometry” entails that with good representation, meaning comes from distances, symmetries, invariances, and so on. Through this lens we can accept representations like word embeddings, where the axes don’t provide meaning, but the distances between words do.

One can then think of models as acting on the data space as a whole, making these distortions and “learning features.”

A second transformation is thinking of models as geometric objects too — also with distances, symmetries, and so on — and optimizers as acting on these objects.

Data as geometric objects Points in \(\mathbb{R}^D\) with meaningful distances, symmetries, invariances

Models as geometric processes that distorts this space

Models as geometric objects …with meaningful distances, symmetries, invariances

Optimizers as geometric processes navigating this space

ML Describes Geometric Processes: Troublesome

🌀

Connecting Algebraic and Geometric Views Students struggle connecting algebraic and geometric views (Sibia et al. 2025)

📚

Counters prior CS training Earlier courses emphasize data type (chars, pixels, waveforms) and time/space complexity.

But this is troublesome. Earlier CS courses emphasize differences in data type and their representations. And empirical evidence (Sibia et al. 2025) suggests that students find it hard to connect algebraic and geometric views.

ML Describes Geometric Processes: Integrative

🔢

Data geometry Shapes what can be learned: why one-hot encode categorical variables? distributed representation?

🕸️

Model geometry Shapes what models are possible: CNNs & GNNs exploit symmetries; embeddings & transfer learning

🏔️

Loss geometry Shapes how we find good models: gradient descent + momentum, step sizes, clipping, ravines

But once understood, geometry provides a lens to answer questions about what data representations work well, shapes model design that exploits symmetries, and shapes how we tackle the optimization problem.

Implications & Discussion

How Threshold Concepts Are Used

🔄

Transformative concepts Rather than exhaustive topic coverage

⏳

Liminal time Build in time to work through liminal phases

✅

Real assessment Design assessments that reveal transformation, not mimicry

• New ML models are developed constantly
• Students report feeling overwhelmed (Sibia et al. 2025)

Threshold concepts are generally used to identify the few pivotal ideas to organize teaching around, rather than covering a discipline’s topics exhaustively.

I think this is especially useful in ML, where new models and methods arrive constantly, and students already report being overwhelmed.

Implications: New Courses & Curricula

Not every course needs every concept, with equal weight

Decide the role (user, builder, researcher?), then the transformations we want the learners to undergo

For new courses, some of these concepts might be starting points for deciding what those “pivotal ideas” might be. Not all courses might use all of these concepts, depending on what transformations we want the learners to go through. The more “applied” courses might stay to the right, for example.

We did put “learning as optimization” right in the center, and I do think that this is still important.

Implications: Existing ML Courses

Week	Lecture Topic
1	Supervised Learning; Nearest Neighbours
2	Decision Trees
3	Linear Regression
4	Feature Mapping; Classification
5	Multi-Class Classification; Multi-Layer Perceptrons
6	Neural Networks; Backpropagation
7	Bias-Variance Decomposition; Probabilistic Modeling
8	Algorithmic Fairness
9	Naive Bayes
10	Gaussian Discriminant Analysis
11	Clustering; Mixture Models; Expectation Maximization
12	Principal Component Analysis

Revisit the TCs in every unit

• consistent thread empahsizing what is foundational and cross-cutting
• ensure the transformations actually happen

Even without fully redesigning a course, you can use these threshold concepts as recurrent threads tying different models together.

My machine learning course schedule looks something like this — a lot of textbooks also look like this. And if you have a course like this, we can still revisit the core ideas in every unit, so there’s a consistent thread throughout the course and key transformations actually happen.

For example: in Decision Trees, what is the optimization problem? How is it broken down? What are the key geometric ideas? What tradeoffs are salient? — and how is that different from, say, Naive Bayes?

Because without that connective tissue, it’s harder to tell what’s foundational and cross-cutting.

How much math do students need?

Math maturity is an important pre-liminal variation

Learners can cross the threshold with minimal math

There is also this question of how much math is really necessary in ML education. Our answer: for the first three, more applied concepts, we think learners can cross with minimal calculus, linear algebra, and probability — understanding ML as empirical, and the multifaceted nature of evaluation, doesn’t take much math at all. For the last two, more theoretical concepts, math maturity becomes an important pre-liminal variation (Rountree and Rountree 2009; Meyer and Land 2003): without it, learners can still enter the liminal space, but they struggle to fully “negotiate” their way across it, and stay at the level of mimicry rather than real understanding.

That said, math self-efficacy remains a real barrier, and teaching students without a deep math background is still crucial — with careful, tailored instruction they can go far, even if the transformation doesn’t end up fully irreversible.

Math self-efficacy remains a barrier; careful instruction can help, but transformations may not stick

What would you add, change, or challenge?

Thank you!

m.a.migut@tudelft.nl

j.h.krijthe@tudelft.nl

Leave real room for discussion here. This is the big-picture recap: all five concepts together, one screen. Mentimeter prompt from the original draft: “What would you add?” — keep the on-slide question general and let follow-ups (does this match your students? what would a course built around these look like?) come out in the room rather than being spelled out on the slide.

References

Adler-Kassner, Linda, and Elizabeth Wardle. 2015. Naming What We Know: Threshold Concepts of Writing Studies. University Press of Colorado.

Barradell, Sarah. 2013. “The Identification of Threshold Concepts: A Review of Theoretical Complexities and Methodological Challenges.” Higher Education 65: 265–76.

Bewersdorff, Arne, Xiaoming Zhai, Jessica Roberts, and Claudia Nerdel. 2023. “Myths, Mis-and Preconceptions of Artificial Intelligence: A Review of the Literature.” Computers and Education: Artificial Intelligence 4: 100143.

Boustedt, Jonas, Anna Eckerdal, Robert McCartney, et al. 2007. “Threshold Concepts in Computer Science: Do They Exist and Are They Useful?” ACM Sigcse Bulletin 39 (1).

Marx, Erik, Clemens Witt, and Thiemo Leonhardt. 2024. “Identifying Secondary School Students’ Misconceptions about Machine Learning: An Interview Study.” Proceedings of the 19th WiPSCE Conference on Primary and Secondary Computing Education Research.

Meyer, Jan, and Ray Land. 2003. Threshold Concepts and Troublesome Knowledge: Linkages to Ways of Thinking and Practising Within the Disciplines.

Rountree, Janet, and Nathan Rountree. 2009. “Issues Regarding Threshold Concepts in Computer Science.” Proceedings of the Eleventh Australasian Conference on Computing Education - Volume 95 (AUS), 139–46.

Rowbottom, Darrell Patrick. 2007. “Demystifying Threshold Concepts.” Journal of Philosophy of Education 41 (2): 263–70.

Sibia, Naaz, Amber Richardson, Alice Gao, Andrew Petersen, and Lisa Zhang. 2025. “Student Perspectives on the Challenges in Machine Learning.” Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education v. 1, 9–15.

Timmermans, Julie A., and Jan H. F. Meyer. 2019. “A Framework for Working with University Teachers to Create and Embed ’Integrated Threshold Concept Knowledge’ (ITCK) in Their Practice.” International Journal for Academic Development 24 (4): 354–68.

Touretzky, David, Christina Gardner-McCune, Fred Martin, and Deborah Seehorn. 2019. “Envisioning AI for K-12: What Should Every Child Know about AI?” Proceedings of the AAAI Conference on Artificial Intelligence 33: 9795–99.