Matthew Rascoff on Learning, the University, and Human Agreement
Matthew Rascoff, vice provost for digital education at Stanford University, presented “Learning, the University, and Human Agreement” at the Tec de Monterrey Faculty Summit, July 5, 2022. In it, he uses the analogy of urban design to suggest a framework for thinking about digital learning spaces and the institutions and organizations that provide them. Here are the video and transcript of the event.
Transcript
MATTHEW RASCOFF: Hello, everyone. My name is Matthew Rascoff. I’m vice provost for digital education at Stanford University. And I’m so glad to be joining you and your colleagues at Tec de Monterrey today.
If you’re watching this recording, it means something went wrong with our technology setup. So I’m sorry about that. But I’m glad we have this backup plan in place. And I look forward to engaging you today and in the future as well.
I’ve also been promised a chance to visit the campus, which I’ve heard many wonderful things about over the years. And I hope we can find a way to do that sometime, maybe after this pandemic is over. God willing, one day.
So today, I want to talk about high-touch learning at scale. And the way I want to do that is through an analogy from architecture and urban design. One of the favorite classes that I took at Columbia University where I was an undergrad was an architectural history class with Professor Barry Bergdoll. And he turned me on to the idea of urban design as something that was about more than just cities, but it was really about how we choose to live together.
And the fundamental questions about communities and learning as well. And that’s kind of the starting point for my talk today, which was inspired by my own education in art history and architectural history, but brings it together with a work that I’ve done over the past two decades in digital learning and online learning. And the way that technology can support learning and our educational values.
So let me start with a bit of a history lesson. This is a picture of Denver in Colorado in the US in 1925. It was a thriving western city at that time with buildings, as you can see, of many different sizes and shapes.
Pay special attention to the clock tower in the center of this picture here. That’s going to take on some importance. That’s the Daniels & Fisher Tower, which was part of a department store of the same name built in 1910 to a height of 325 feet. It was the tallest building in that era between the Mississippi River and California. And it was modeled after the Campanile in Piazza San Marco in Venice.
Now, fast forward a few decades later, in the subsequent years after the 1920s, Denver and downtown Denver went into a steep decline that was driven by new technologies. In this case, it was the automobile and trucks that led to the suburbanization of industry and the consolidation of the small workshops that were depicted in that previous picture into larger factories.
By the 1970s, so 50 years after that first picture was taken, the Denver Urban Renewal Authority took control of many of those small lots that fell into disrepair. They demolished many of the old buildings. And they repackaged the land into larger parcels and then sold them to office developers. Many of those original buildings were just one, two, three story warehouses and light industrial. But as you can see, the new buildings that were constructed were these really gigantic office towers.
In the subsequent years, the redevelopment effort basically failed. The Urban Renewal Authority cleared far more land than was needed for these developments. And many of those offices had already moved to the suburbs. Meanwhile, these towers in downtown Denver created these huge parts of the city that were basically empty at night. And as you can see, in their wake were these gigantic parking lots that were kind of wastelands in terms of what we expect from a city, the life of a city.
In his 1971 speech, “The Room, The Street, and Human Agreement,” the architect Louis Kahn offered an alternative vision for urban planning to what you saw happened in Denver and of course in many other cities in the US and around the world in the 20th century. Kahn was an architect and an educator. He was a professor of architecture and taught at the University of Pennsylvania for many years.
He was not a professional city planner. And I think that’s important context. In his 1971 speech, his vision which he later published was a statement of his philosophy about architecture. And it was rooted in individuals. Or more precisely as you can see here, in dialogue among individuals.
That’s what’s happening. There’s two people. It’s a little bit hard to see in this picture. Two people talking.
And he designed for the human scale, not from the city scale down, but from that conversation. Those two people engaging with one another out and up from there. Khan said in his speech, the room is the beginning of architecture.
It is the place of mind. You’re in the room with its dimensions, its structure. Its light responds to its character. Its spiritual aura recognizing that whatever the human proposes and makes can become a life.
The next level up in Khan’s vision from the room was the street. The street, as he described it in his speech, was a room for the whole community. He said the street is a room of agreement.
The street is dedicated by each house owner to the city in exchange for common services. Dead end streets in cities still retain this room character. But through streets since the advent of the automobile, he was saying in 1971, have entirely lost their room quality.
I believe that city planning can start with the realization of this loss by directing the driver and to reinstate the street where people live, learn, shop, and work as the room out of commonality. So from the room to the street. And then the next level up in his vision of designing from the small to the medium to the large is the city.
Khan said in his 1971 speech, from a simple settlement, the city becomes the place of assembled institutions. The settlement was the first institution. The talents found their places.
The carpenter directed building. The thoughtful person became the teacher. The strong one, the leader. The desire to learn made the first school room. It was of human agreement.
The institution became the modus operandi. And the agreement has the immediacy of rapport, the inspiring force which recognized its commonality. And then it must be part of the human way of life supported by all people.
So what Khan is doing here, in his alternative vision to that urban renewal model, he’s building up from the smallest unit of space, the room, to the largest. And with each step comes greater organization, greater sophistication, and greater specialization. Yet there is a fundamental unity in his design which preserves the integrity of the room, even as it scales up. It scales in a cellular, modular, almost fractal format.
So I want to share a little bit about what that might mean at the city scale. And this is an image also made by Louis Kahn, not drawn from that speech, not published. It actually was produced earlier in his career. And it’s more of a theoretical statement. It’s an urban plan for the City of Philadelphia. And it’s kind of an alternative vision for how transportation might work in the city.
So in his plan for Philadelphia, which was connected to the philosophical ideas of the 1971 speech, he proposed a new traffic pattern. And that was really the key idea as represented in this image. So I want to quote just from the catalog of the Museum of Modern Art which owns this drawing just to describe what you’re seeing here and to give it a little bit of a description. Because it’s a technical drawing that’s a little bit hard to understand. And there’s no text.
The catalog entry says to untangle traffic congestion and to mitigate the proliferation of parking lots that plagued postwar cities, Khan reordered the streets according to a functional hierarchy. The drawing’s notational system corresponds to different tempos of traffic such as the stop and go movement of trucks-- those are the dotted lines, the fast flow of vehicles around the periphery-- those are the arrows at the edge, and the stasis of cars in parking garages-- those are the spirals also at the edge of the image here. To explain his movement, he invoked a historical analogy.
The girdle of expressways and parking towers circling the city center metaphorically recalled the walls and towers that protected the medieval cities of Europe. So this was how a city could play out. This is how movement could be organized in a city that was designed in that kind of bottom up fashion, starting from the room to the street, and then to the city level.
This is what it looked like in the plan for a real city. Of course, it was never enacted in Philadelphia. But this was his vision for how a city could work.
Louis Kahn was mostly ignored in his era from the perspective of urban design. And what he created for Philadelphia was dismissed, actually, by the city government there. But he wrote in 1971 that we can begin by planting trees on all existing residential streets, by redefining the order of movement, which would give these streets back to more intimate use, which would stimulate the feelings of well-being and unique street expression. The street is a community room.
And I want to posit that his ideas have become really important, actually, to architecture and to urbanism. Because 50 years after his death, a park that he designed in New York, Four Freedoms Park, was actually built, even though he, of course, was long gone. He died in the 1970s. And cities around the world are redesigning streets to look much more like his vision of a shared room.
You can see a picture at the top of a street dedicated basically to the movement of cars. And then look at the picture at the bottom of the same street, a street in Paris, which was redesigned and reopened according very much to the philosophy that Kahn described of planting trees and reclaiming the street as a community room. So why am I sharing this extended analogy from architecture and urban planning?
Because I want to argue today that Kahn’s ideas are important for how we think about spaces of learning, both digital and physical spaces of learning, and the learning communities that those spaces can either support, or like urban renewal, they can undermine. So let’s bring this back to the design of learning spaces. I believe there’s really an important lesson for us in the way we think about learning spaces and the institutions and organizations that provide them.
In the 20th century, they were responding to the advent of the automobile. That was the technological force that brought about the urban renewal movement in Colorado, in Denver, and the response from Louis Kahn. Today, of course, the technologies are different. But I would argue that some of the design considerations are similar.
In the space of learning, take the example of Coursera just to get us started, which is a sort of digital skyscraper. And you can see that almost represented in this chart here. Coursera is a giant platform that in 2022 passed 100 million learners, perhaps second only to YouTube in the size of its user base. But each of those learners effectively travels in a private vehicle with little communication or interaction with others.
They work in basically a cubicle of learning, a self-paced isolation from their fellow learners. Hardly a community room for learning. If it were a physical classroom, Coursera might be represented as a gigantic amphitheater. They have essentially recreated the technologies of the private automobile and the skyscraper and all of the isolation that’s associated with them.
Meanwhile, the traditional providers of higher education in the United States, at least-- and that’s where these data are drawn from-- are in steep decline. The growth of giant online learning platforms with the potential for winner take all economies of scale puts smaller scale institutions at risk. The process accelerated during the pandemic, but it was already well underway, as you can see from these data before it.
This is the percentage change in enrollments in different sectors of higher education in the United States. So to me, I think, the question for us is, will smaller schools be swallowed up? Will they be demolished and turned into the institutional equivalent of parking lots? Will the smaller scale model of learning, that kind of 1 to 2 story building that you saw represented in the first image of Denver in the 1925 image, will that model become obsolete? And will the walkable mixed use educational neighborhoods give way to digital skyscrapers and internet highways?
I want to offer today an alternative and more optimistic vision for the future in which diversity and pluralism and design at different scales is able to survive. We’re able to retain the community benefits of those smaller spaces for learning, even as we gain some of the cost economies of scale by serving larger numbers of students of learners. And it’s based on a model from a project at Stanford that was developed during the pandemic called Code in Place. That model is designed to be bigger than the face-to-face residential experience of Stanford, but much higher touch than a MOOC.
I’m going to let the faculty leader of Code in Place, Professor Chris Piech of the Stanford CS department describe it to you in his own words. And then I want to talk a little bit about what I think it means for how we think about design, and how we support students and learners.
[VIDEO PLAYBACK]
Hi, I’m Chris Piech. I’m an assistant professor of computer science. And I have the wonderful privilege of being part of this great team working on AI for education in the context of a Hoffman-Yee grant.
So sometimes when people hear AI in education, they might think we’re going to have fewer teachers or machines will be teaching us. And there’s two responses to that. First, we just learned that the human connection matters so much. AI just can’t replace the sort of mentorship and change in identity you get from interacting with another person.
Teaching and learning is one of the most social things we do. It’s a defining characteristic of humans. Even if AI was fantastic, it just doesn’t really hold a candle to the sorts of impacts real human beings could have.
But on the other hand, we’ve learned something else during this pandemic. Ourselves, Schoolhouse.world, we talked to a lot of people where they get volunteer teachers to come join in the process. And we all collectively learned how astounding it is, the magnitude of people who want to teach.
So to that extent, our work has really been to augment this social learning experience. How could AI make students better at learning and teachers better at communicating knowledge? And then also, how could AI augment a collaborative experience between students and teachers?
At the start of shelter in place, my fellow teachers and I were about to start a new term. A few of us came together and thought how could we make this transition an opportunity to give back. We weren’t doctors. We couldn’t help in the hospitals.
But we were teachers. Maybe we could bring that as some good for society. Because we’ve spent decades honing our teaching craft. And because we’d already started the process of developing useful AI tools for students, we were ready to offer a pretty wonderful, high-quality experience.
So we ran a course. We called it Code in Place, but it really taught the first half of our introduction to programming at Stanford. And we were able to use tools that could help students get feedback, AI techniques that could help teachers get feedback on their own teaching process. And we could do it in the context of human-led, community-centered education that we believed in.
The Interesting thing about this research is, often, we would have just done this cycle. We think about the research. And then maybe in 10 years, we would think about how we could deploy it.
But because of the pandemic, we deployed immediately. We did two things. We gave feedback to thousands of students to an exam where they would have otherwise not given feedback. We gave feedback to all these students, 16,000 pieces of feedback.
90% of them got feedback from AI. 10% got feedback from humans. And they actually rated the AI feedback as more helpful.
On the other hand, we had 2,000 teachers who we had to train. For these 2,000 teachers, we of course gave them the standard teacher training that you’d expect. But also, we had developed an AI system that could look at their process of teaching students, take their transcripts, find the moments where they’d re-voice students, find the moments where they had missed what a student had said, and used that to give them automatic feedback on their teaching.
And this led to things like asking more questions, reversing what their students said more. And the students who are the beneficiaries of this, like the class more, and were more likely to find the sessions engaging. So we had two great stories of deployment. Actually helping students get feedback they need, and helping teachers improve their own craft.
So what did the Hoffman-Yee grant allow us to do? You could’ve imagine a world in which the pandemic had hit and each of us were siloed in our own different parts of the university. I was sitting there thinking about artificial intelligence. My friends in school of education were thinking about their role in education.
We were scrambling and we would not have been able to form the team that we needed. The Hoffman-Yee grant just happened to have started a few months before the pandemic hit. So we were already in active collaboration. It was just the right combination of cross-disciplinary people in the right room at the right moment where we couldn’t have taken a moment to team form.
Our dream for 10 years from now is a symbiosis of a moonshot course and also a moonshot support that augments learning. When you’re learning online, there’s likely hundreds or even thousands of people going through the exact same experience somewhere around the world. Can we leverage this great potential for collaborations to make the future learning less lonely, more collaborative, and also to give many more people the chance to play that role of teacher?
That’s going to take a lot of AI augmentation. We have to make sure things are safe. We have to put the right pair of people together.
When people are learning and teaching together, we have to help them become better learners and teachers. But I think we have the ingredients. I think that in 10 years, we could move towards that joyful education that’s augmented by AI, but really human led.
[END PLAYBACK]
So I just want to describe what’s happening in the Code in Place course, in case it wasn’t clear entirely from the video. This is a model of scalable online learning that reached thousands of students, learners around the world. But each of those learners was placed in a section of 10 students that was led by a volunteer section leader. And that’s represented on the left of this image here.
Then each of those section leaders was themselves placed in a cohort of 20 led by a teaching leader who was responsible for training the trainer. And we were able to reach, in this kind of modular fractal format, 12,000 students taught by thousands of co-instructors, each of whom was given support, both human and AI support. So the AI is helping to improve both the performance of the learner and of the instructor at the same time in this course.
And I just want to talk a little bit about the impact that this had on various stakeholders who participated in this. Because I think, as Chris says in this video, there’s a real model for impact here. 99.6% of the section leaders completed their responsibilities, even though they weren’t being paid in this pilot. 56% of the students completed the course, much higher than what you would see in a typical MOOC that might have 5% or 6% of the students completing from beginning to end.
30% of students said they’d like to lead a section. So they wanted to step up into that responsibility. Stanford measures its teaching performance on a five-point scale. This course was given a 4.95 on that scale. And net promoter scores of 90 for the students and 70 for the section leaders.
Net promoter, for those who don’t know, is a scale that goes from negative 100 to 100. So those are astronomically high scores in terms of the satisfaction of both the students and of the teaching leaders. I want to just talk also about the educational impact that section leadership has on experiences of our students. Most of those section leaders were either Stanford students or alumni.
This is a quote from not Code in Place but a similar project that we’ve done using the same teaching and learning model, but oriented towards high school students. And you can see, this is a CS master’s student at Stanford who led a section in this course, and talks about in an essay that he wrote for the student newspaper, the impact that it had on him and, in fact, on the trajectory of his career. He said, it is going to lead him, this kind of teaching, to a different career than he would have had, into founding a nonprofit dedicated to this kind of educational innovation.
There’s also a really powerful research connection that Chris mentioned in that video. Because inside this course, the AI was providing feedback to the instructors on the quality of their facilitation of these sessions. The AI was virtually listening in to all of the sessions, developing a transcript, and then using an analysis of the transcript to give feedback to the instructors to help them improve their active learning techniques in the class. And to allow them to use these more, 14 more sophisticated strategies beyond just repeating what a student said.
We found 24% improvement in instructors’ uptake of student contributions using this AI with a randomized controlled trial. So that to me is the powerful connection between educational impact projects of this type and the research opportunities that flow from them by building in these projects, this kind of research lab behind the scenes of an educational offering like the one that Code in Place did. And what you can see here is a kind of virtual cycle of improvement that starts with an insight from cognitive science about how active learning can improve performance for students.
Then creating these kind of teaching and learning test-beds in large courses like Code in Place, supported with data driven tools to help the instructors improve their performance. And the AI feedback or the feedback for the students are both examples of that. And then that can feed into the data science research to improve those tools over time.
And what you saw in the previous slide was a paper that came out of the Code in Place project by faculty and students at Stanford who are doing this kind of data science research, educational data science research. That, of course, can then feed back into the cognitive science insights in which we can refine what we understand from small psychology experiments using the big data techniques that the data scientists are able to pioneer. To me, this model has huge implications and huge potential for how we think about high scale but also high touch learning.
In this model, what we have are really new combinations of humans and AI that I think hold the key to scaling. But scaling with quality and scaling with a human touch and scaling without demolishing that small scale building of the room that is the basis for the dialogue, that is the basis for the human connection. What would it take to actually realize this vision beyond a small pilot course that Stanford offered during the pandemic, but to really think about what the implications could be worldwide for other institutions, other faculty members to adopt?
To me, I think we need to plan for community engagement for learners, and training and feedback for the instructors. And what we need are technologies that support building out that community. Chris says in the video supporting the pairs of learners, helping them find one another and supporting them in the process of peer review and coding and processing and learning together, engaging with one another. But also giving that machine style feedback to the instructor to help them improve their active learning facilitation in a small section that would be otherwise not feasible to provide to thousands of instructors.
So Code in Place was a kind of R&D project that Stanford launched during the pandemic. But there’s actually real technologies that are now available in the marketplace that are realizing some of these ideas. And I just I want to point to just one of these as a model of what I think is possible. Because I think it demonstrates that this is not just happening in research labs. It’s actually something that is available and doable using the machine learning techniques that we have right now.
So this is a product that I love called TeachFX. And it’s both an app that you can use on your phone. And it’s a web technology that integrates into Zoom.
The idea of TeachFX. is to give feedback to instructors using AI that mines the transcript of a session in a class. And it can be either a face-to-face class using the iPhone app or an online class using the Zoom integration. And it gives feedback to the instructor that allows them to improve their teaching and learning practice, and to use more evidence-based techniques in the class. So it’s very much like that machine learning paper that you saw about Code in Place.
But this is a real product. And it’s available in the marketplace. And it’s something that schools in the United States are already using, K-12 as well as higher education. And I want to show you a little bit about what it’s like to use this app, just to kind of small demonstration that I’ll talk you through of how this works in real life with real students with a real class.
So I’m just going to start this video here. So this is me doing just a demo lesson. The names of the students are not real. And what you can see with those bars is how much participation there has been from teacher, student, how much silence there is in the class, and how much group discussion there is.
Now, we’re going to go into a demo class that’s going to mine what happened in this Zoom recording of a class. You can see the words that were spoken by the teachers and the students, how much talk there was by the teacher, me, let’s say, and the students in individual and in small group. The green represents silence. The blue is students. And the red is the teacher.
Then you have a transcript of the entire lesson. And you can go through the lesson. And you can see in that chart right below the transcript who was talking and how much and when. And you can see it in terms of minutes on the right, as well as percentage terms.
And I can go back and say at that time fast forward, go backwards, what was I saying at that time? What led me to speak so much? Now, I’ve got some examples of short student responses from the lesson. So what they actually said in response to the questions.
I’m a teacher going back to see what happened in class today. And did I use these evidence-based techniques in my class? And I’m getting feedback from this algorithm that basically mines what happened in the class or who spoke how much and what they said that allows me to improve my performance the next time I do the class.
I have a roster of all the students in the class, and how much they spoke, what time they joined, their degree of participation. How many times they talk and how long they talked. What percentage of the class that represented.
And then this longest stretches of teacher talk, this is me doing too much exposition really. You can see, OK, what was I doing at that point? Why was I speaking so much in the class? Maybe I could be allowing the students to participate more fully.
Now, we’ve got some different models of student participation. Ping pong means going back and forth, back and forth between the teacher and the students, rather than allowing them to go back and forth amongst themselves. We can see the students getting an immediate response.
Think time is a really important concept. And it’s the idea of allowing students a moment to think before you call on a student. Give them a few seconds of think time just to process the question that you just asked. And after they’ve responded, give another few seconds of think time, sometimes called wait time after they’ve said what they said to allow their classmates to think about what they said. All of these ideas are represented really powerfully in the TeachFX interface.
I think this is just such a good example of how the AI can give superpowers for educators. It can help that teacher do her job so much more effectively than she could on her own. And that shows, I mean, the potential for allowing students to participate more fully in a class like this.=
So teachers, here’s some data from TeachFX. Teachers using this app raise student talk time by 273% in the nine months subsequent to their adoption of this software. And that to me really shows the potential for taking what we know from the teaching and learning research from evidence based on the scholarship of Rachel Lotan, scholar at Stanford Graduate School of Education, turning it into algorithmic tools that help teachers realize those ideas in classrooms at scale, and then measuring the impact on teaching and learning.
And allowing us to see that these kinds of apps can help us take an idea that previously could only be represented really in professional development. We could tell a teacher, this is a good idea. Try to do this in your class. But it was not feasible to actually give this kind of feedback to have an assistant listening in on every class and giving that feedback live.
Now, we actually can do that. And we can do that in a face-to-face class and in an online class. And I see with data like this, the huge potential of AI augmentation for teachers that allows us to improve learning outcomes for students and preserve that high touch.
This is not an AI that’s teaching the students. It’s not taking the teacher out of the equation. The AI here is really, it’s a human in the loop model, which means a combination of a human plus an AI working in tandem to try to improve the overall experience for the learner.
So I’m going to bring things to a close by returning to my architectural analogy. In Louis Kahn’s life, he was really only able to complete a handful of projects and most of them were for individual buildings. But the new urbanism is a movement that revived many of Louis Kahn’s ideas 50 years later.
And now in the 21st century, some of them are coming to life with real projects. And they’re happening at the level of cities and streets, not just the individual rooms and the individual buildings, which is what he was effectively able to manage in his own lifetime. The images here are from a new development in Costa Rica called Las Catalinas. It’s still under construction, but it has several neighborhoods that have already been built.
It was designed by an architect Douglas Duany, who is a founder of the New Urbanism movement that’s taken many of Louis Kahn’s ideas into the 21st century. There are no cars in this development. The streets are all walkable and bikeable. They’re safe for children to navigate themselves.
It’s mixed use. And the residences are mostly multifamily. Though there’s diversity and some of them are single family homes as well. You can see in this picture on the right, the buildings have a warmth to them. And there’s a sense of community and of a village in the design.
This looks like it might be a medieval Italian hill town. But it’s actually been built just in the past few years. There’s no reason, in my view, why we cannot design towns to be livable, walkable, and enjoyable the way this is designed to be.
And similarly, in my view, there’s no reason why we cannot design digital spaces and communities that we ourselves would want to live in, to learn in, and to allow our children to explore in. Thank you so much for listening today. I look forward to engaging with you. And thank you for the invitation to participate in this summit. Take care.