This is the title

Imagine standing at the college dining hall, faced with tens of options but holding only a small(ish) plate. How do you choose? In this course, we discuss how trial-and-error learning gives rise to the everyday behavior of humans and other animals. We take a modern, integrative approach to phenomena that grew from animal-learning paradigms such as classical conditioning and instrumental conditioning, examining them through the lens of computational models of learning and decision making and current neuroscientific knowledge. We answer questions such as: why do we have such a hard time walking away from rewards (and how is this exploited by fortune tellers?), how best can we train a dog to do new tricks (and how does this apply to bringing up children?), why do we sometimes find ourselves performing an action out of habit, even though we have no desire for its outcome (e.g., opening the fridge as we walk into the kitchen, even though we are not hungry), why do we press the “walk” button more times when we are in a hurry (surely once is enough, no?), and how do we optimally teach someone something new so that we can facilitate them changing their mind.For each topic, we discuss behavioral and psychological findings, computational algorithms that underlie this behavior, and the neural hardware (and software) that may be implementing the algorithms in the brain. The overarching goal is to develop an integrative picture of how the brain implements the computations that are necessary in order to bring about day-to-day behavior as we know it. In the second half of the course, you will work with peers on a project investigating a topic of your choice.

This is another title

This course follows an alternative grading/teaching without grades approach to learning (see more information below).

Learning objectives: Upon completion of this course, students should be able to

  1. Recognize the multiple mechanisms that can bring about a decision to behave, and understand their essential differences.
  2. Describe in computational terms how new information leads to learning in each system.
  3. Work in a group to independently research a project to pursue your own curiosity about learning and decision making.

Course topics and presentations*

*Many thanks to all my colleagues and fellow teachers who have (often unknowingly) contributed much of the material that is included in the above slides. If you are teaching a course and would like to use these materials in your own slides, feel free to email me for the keynote or powerpoint version at yael@princeton.edu
  • Prologue: Introduction (slides)
  • Act 1: Classical/Pavlovian conditioning (weeks 1-3) – basic phenomena (slides), learning from prediction errors (slides) and the Resorla-Wagner model (flipped portion + slides), second order conditioning and the Temporal Difference model (flipped portion + slides), dopamine and learning in the basal ganglia (slides), opponent processes: appetitive vs. aversive, inhibitory vs. excitatory conditioning (slides)
  • Act 2: Instrumental/operant conditioning (weeks 4-6) – Thorndike, basic procedures, modeling action selection, Actor/Critic framework, neural substrates of Actor/Critic model (slides), learning Q values, comparing models using neural data (slides), Skinner & free operant behavior , modeling free operant behavior (slides), habits (model-free learning) versus goal-directed behavior (model-based learning) (slides), everything I taught you so far is somewhat wrong (slides), so…
  • Act 3: Choose your own adventure
    • (week 7) Bayesian inference, POMDPs, latent cause inference and state representations
    • (weeks 8-10) Project presentations
    • (week 11) Changing people’s minds and behavior
    • (week 12) Computational psychiatry: mood and reinforcement learning
  • Epilogue: What have we learned?

Student projects from previous years

More about “teaching without grades”

Learning is not a spectator sport – it is a choice, and requires commitment. There is also no single metric for learning that is relevant to everyone. Like a muscle, our brain can always expand in its abilities, through practice and training. To maximize learning, research shows that goals that are just beyond your current abilities are most motivating, and promote growth. As we see in this course, research also shows that grades (for humans) and rewards (for animals) can be detrimental to learning. In particular, focusing on externally-assigned (and somewhat arbitrary) goals, and worrying about whether you are achieving them, does not promote true, deep, and consistent learning.

So we do not do that. Instead, in this course students set personal goals, track their own progress on these goals throughout the semester, amend their goals as needed (setting new goals when previous ones have been achieved; modifying goals that are not realistic), and assessing their your own learning. The course instructors help students achieve their goals for each course component by requesting students to reflect on their progress after each activity, discussing with them challenges that arise as they work toward their individual goals, how to measure and monitor their progress, etc. We will also provide feedback on all course components, to help guide students in improving in any aspect in which they are attempting growth, but without narrowing that feedback to a one-dimensional number or letter. Students’ abilities, challenges, and learning are multidimensional. We have found that providing feedback on these many dimensions is much more useful for promoting growth than are summary grades.

For more information about alternative grading I recommend this excellent episode of Jennifer Gonzalez’s fantastic Cult of Pedagogy, this practical how-to book by Starr Sackstein, and here is a twitter thread where I wrote about this method and discussed it with the community.

Recommended readings

  • Textbook: Learning and Memory: From Brain to Behavior by Gluck, Mercado & Myers (2nd ed)
    Note that this is not our textbook in the traditional sense, in that it covers more than we will be discussing (eg, memory) and does not cover everything that we will be discussing (eg, computational models)

Some books I recommend

  • Sutton, R. and Barto, A. – Reinforcement Learning (2018) – A very good and accessible book explaining the computational field of reinforcement learning (also available online)
  • Mackintosh, N.J. – The Psychology of Animal Learning (1974) – Great book, although hard to find (out of print).
  • Mackintosh, N.J. – Conditioning and Associative Learning (1983) – Shorter than the above and covers mostly theoretical aspects, not for complete beginners.
  • Dickinson, A. – Contemporary Animal Learning Theory (1980) – Although this was contemporary a long time ago, it is still very good and easy to read.

Additional resources