AI for Pathways & Credentials

Can Artificial Intelligence help students navigate their educational pathway? Dr. Zachary Pardos, Associate Professor of Education at UC-Berkeley, joins us to share exciting developments in AI that shows significant promise in helping students navigate their pathway to a credential.

Transcript

Matthew Sterenberg (00:02.218)

All right, Dr. Zachary Pardos, welcome to the podcast. Zach, I’m glad you’re here because I need a AI Sherpa and I’m hoping you can be that person for me. But before we get into that conversation about how AI and machine learning can impact student pathways, tell us a little bit more about yourself and about your work.

Zach Pardos (00:25.264)

Sure. So I’m an associate professor. I don’t want to demote myself. In the Berkeley School of Education, I also teach in the data science major at Berkeley and have background in computer science, PhD. Have been studying though the application of AI to education since I was an undergrad.

2006 published the first paper with a research supervisor on estimating student knowledge using machine-learn models. And then 2015 got a couple of grants to apply AI to higher ed institutional data and have gone down a transfer and articulation and pathway rabbit hole ever since.

Matthew Sterenberg (01:18.126)

So before we get into the work you’re doing, which I think is fascinating, what is the problem statement? Why do we wanna apply AI machine learning to articulations and pathways? What are we actually trying to solve for?

Zach Pardos (01:35.632)

The statement I put on my home page is economic mobility. There’s a lot of things that inhibit economic mobility in America. One of the things that really greases the wheels of economic mobility is transfer, articulation. But there’s a lot there that has combinatoric challenges.

that make you ask the question, how do we even have any transfer pathways right now, given how difficult it is? And so at the core of that problem statement is can technology can recent technology can AI plus policy, you know, integrating AI into a socio technical system, can that chisel away at the problem of

of economic mobility achieved through credit mobility in ways that we haven’t been able to chisel at it before.

Matthew Sterenberg (02:40.354)

So when I was first learning about articulation transfer, my first response was, don’t we already have articulation agreements in place? And I think this is just kind of a funny intersection of policy, technology, and the pathways that students actually take because we have articulation agreements in place already.

I take this course at a two year, here’s what it will mean at a four year. Why is that not sufficient in our world today?

Zach Pardos (03:19.904)

Yeah. So.

If you look at the California Community College, I’m from Berkeley, which is a member of the University of California, which are nine campuses. It’s a multi-segmental system, meaning it’s, in our case, it’s three separate systems that are kind of sewn together by a combination of collaboration and legislation. There’s 116 community college strong system and 26 California state universities.

23, excuse me. And if you were to try to articulate between all the California community colleges, and let’s say a new UC that joins, given the average number of courses and departments, your average community college campus has and the average that a UC has, you would be looking at 36 million comparisons. And that’s just adding one.

And so that’s adding one, and that’s also assuming you’re only looking to articulate between conventionally equivalent departments. I’m only looking to articulate from your math department to my math department, when in fact, it’s reasonable to look outside of the same department. For example, public policy departments sometimes have very respectable statistics courses. That could very well be.

70% or greater substitutes in terms of the material for ascending schools, a statistics course in the statistics department. So that kind of combinatorics leads you then to say, well, what’s the number if we’re articulating to all UCs or what about private schools or out of state schools, out of state systems. And now you’re talking about a number where, you know, there may be more atoms and less atoms in the universe than the number of combinations of

Zach Pardos (05:22.912)

Higher education doesn’t have that kind of money to comprehensively do that sort of articulation. And as a result, articulation tends to be where we focus our attention on, right? Either there’s legislation that has some money to do this particular program articulation in this major, this area, or the focus is on common feeder schools that are local. So you kind of get a bit of a bias of articulating more thoroughly to schools in the region of a receiving institution.

So we do have program articulations, but we don’t have comprehensive program articulations.

Matthew Sterenberg (05:58.858)

And doesn’t it require like a translation? You know, you could plop down articulation agreements in front of students. What does it mean to me as a student? You know, like how does this actually color what I’m gonna take? And all the research suggests that academic advising some version of it is really beneficial, but it’s just hard to scale, right? So I think it’s always funny when institutions are like, we have these…

of policies in place or these agreements in place, but how do we actually get it to be adopted and utilized? And what is the realistic application when you think about a student, especially a student who’s maybe trying to work, right? Still figuring out at four years something that’s realistic for them. What do I even want to do? And so that’s what I think is really interesting about your work. And the analogy you used was…

We have roads, but we don’t have navigation, which I think is a really easy way to think about it, right? On Google Maps, I take a right-hand turn where I shouldn’t have. It’s gonna reroute me, right? It’s gonna tell me how to get there once I take a different path. So tell me a little bit more about that analogy, the roads versus navigation and the work that you’re doing to help students kind of in navigate their actual experience rather than just relying on these articulation agreements.

Zach Pardos (07:23.64)

Yeah, right. It’s the difference between there being a PDF that describes your degree requirements and what an advisor does to kind of tailor that policy on a PDF document to your situation. And what we’ve been looking at is, how can you help advisors do their job? And then how can you directly speak

to students to know how to navigate these pathways. Before I say that, I will say there’s been a huge, initiative in higher ed called Guided Pathways. I wanna acknowledge that program or that kind of ethos that has been very well funded. And that really this is, I see a next generation Guided Pathways where Guided Pathways certainly.

is a sensible approach to get better percentage of students finishing degree, in part because it hardcodes your path. It kind of removes or reduces elective choice. But in doing so, you’re able to plan exactly how many students should be in a particular course. You don’t get surprise-impacted courses as much. And

As long as the student knows where they’re headed to, you can say, you do this, you know, as of this effective day, you’ll get credit for all of that. But we know things don’t always go as planned, particularly in this, you know, stage of one’s life. And when it comes to community colleges that serve a wide range of learners, that stage could be, you know, deep into a career or the part of life where someone’s choosing to have children.

or already has children, but is deciding to take on a second job, right? There’s a lot of things that go on that can change that path and guided pathways doesn’t have a rerouting feature to it. It has, here’s your route. Um, and so. Jeep modern GPS do have that. And so what, what needs to be done when it comes to rerouting and higher ed? And it’s a lot.

Zach Pardos (09:40.88)

because there’s a lot of detours in higher ed. There’s a lot of closed roads. There’s a lot of roads that are really bumpy, right? And you need kind of institutional knowledge to know that’s a bumpy road. You may not wanna go down that even though it looks like it’s a shorter distance. So how do we kind of synthesize all of this? And I think an advisor is a great tool, a great resource, a great person, a connection to have to do that. The problem, as you alluded to, is that

You don’t have a one-to-one, you know, advisor, advisee ratio as the norm. Berkeley is quite lucky to have something more like a one to 31 to 40 ratio, which is about as good as you get, but the average is more like one to 1000 at, at most colleges. And so if it’s one to 1000, you have a thousand students to personalize advice to. It’s going to be hard.

you know, if not impossible to do a really deep personalization with all of those people and all of the different careers are considering and therefore different receiving schools are thinking about going to and what courses to take, given all of these objectives and constraints like I have a job that ends at 2pm. So it’s one of the most constrained choices, maybe in one’s life is what courses to take next semester.

And so part of the research has been, yeah, how do we, step A, create the pathways? So use AI to create credit pathways between institutions, which are otherwise known as articulations, but then how to then help students traverse those pathways. That’s the driving part, the roads are the paths whereby credit can travel, but then…

the student is steering that credit travel, but they need to know where to steer and which roads to go down that might not seem on the surface level to be efficient, but they historically are. So those technologies have been part of the research is these kind of rerouting technologies. What kind of data do we need? It’s often degree satisfaction data, of course, is part of the picture. So there’s a practical.

Zach Pardos (12:02.916)

problem to solve that’s not really an AI problem. It’s what format do different institutions store their degree requirements in? How do we ingest that format? How do we ingest that in a format where we’re losing as little data as possible? And so through several grants, we’ve been able to iterate on that ingestion process, get feedback from advisors, and every iteration requires less and less feedback from advisors. So we’re doing pretty good on

on our first major to try this navigation tool on was a transfer for the criminology major in the State University of New York. And now since we have that working pretty well, we’re now gonna see, well, how well did our ingestion optimization, how well will that apply to a new major? Not a newly introduced major, but new to our system. Are we gonna have to go through that same amount of work to tune for a new major or like we expect?

is there going to be diminishing amounts of fine tuning that are required? That’s the prospect for scaling, is every time you implement a kind of AI assistive approach, you need less and less manual labor to refine it on the researcher or kind of technician. And

Matthew Sterenberg (13:22.906)

So you talked about how do we roll this out? How do we scale it? What are the actual inputs that you need when you’re doing this work? How are you actually getting the data? How is it helping you navigate the history of what students have done to get a good idea of what students should do in the future? What are the inputs and the considerations that you’re having to make?

Zach Pardos (13:47.3)

Yeah. So for the path creation, we need the course catalog descriptions and any existing articulations within the system. So currently, our partnerships, research partnerships, are with at the system level. So the systems give us all the course catalog information of campuses, as well as the existing articulations between them. And then we use a variety of.

machine learning mostly borrowed from natural language processing and some of the latest work on machine translation to then learn what articulations might be missing. For the path traversal navigation piece, we need the degree data, as I mentioned. Any other kind of structured data that might be on hand, like prerequisites of courses.

Obviously, gen ed is an important degree requirement that we need a description of. And then for the personalization part, so there’s the requirement satisfaction part, which needs those data. And then there’s the history of a particular student that a campus wants a plan generated for. So what courses have they taken so far? So we don’t.

tell them to take another course that satisfies the same requirement that was unnecessary. We also want to know what institution and major they are wanting to transfer to. Some cases it’s not about transfer. And we have at Berkeley deployed an experimental system that routes a student to degree just within Berkeley with no transfer consideration.

But the input to this transfer plan generation is, is there a destination school you have in mind? What’s the major there? And then we need to have the degree requirements of that destination. And then finally, in order to personalize to what looks to be a pattern in the kind of courses a student is taking, we need the course enrollment histories at the sending and receiving school for that to maximize personalization.

Zach Pardos (16:07.072)

What we’ve recognized is those data are pretty hard to come by, and institutions need a lot more paperwork to be signed in order to get at that. So we’ve devised a way where plans can be generated without the whole student enrollment history for the last five years, which is what we’ve been using in our research papers. It could just be for the students who are wanting to transfer. And the personalization can come by them stating the kinds of courses that

they would like to be taking if they have a particular preference like oh I’m into you know computer science but I’m interested in sustainability so you know wherever there’s an option to take an elective that involves sustainability or there’s an option among degree satisfying courses but one has more to do with sustainability you know prioritize that.

Matthew Sterenberg (17:00.019)

So thinking about using the past enrollment history, is it for that specific student or students in general?

that have gone through a program.

Zach Pardos (17:14.268)

Yeah, it’s both. So in 2020, I think, we have a paper at a conference called AAAI, which is a top five AI conference, 2021, excuse me. And it uses both the history of all students to come up with a normative model of a path, which isn’t

sufficient by itself, but figures out those norms.

Matthew Sterenberg (17:42.954)

Well, I was going to say, because how do you know that was, you know, it’s normative, but does that mean it’s good? You know, like, should we be replicating history is basically the question I’m trying to get at.

Zach Pardos (17:49.838)

Night.

Yeah.

Zach Pardos (17:55.936)

Yeah, I mean, and we’ve mostly been moving away from that. So in our pilots at SUNY, we’re not using that normative model. I think you could get a benefit from that if you selected for it to only be trained on students who eventually went on to successfully transfer or graduate, right? Right. So yeah, definitely with you there.

Matthew Sterenberg (18:19.178)

Right, right, yeah.

Zach Pardos (18:24.128)

But then also the student can provide their data, their data as in what courses they want to take in the future as a personalization signal. Because it could be that the courses they’ve taken so far weren’t intended, weren’t selected as a kind of expression of the personalized interest they have. And maybe even they want to go in a different direction. So one of the problems with recommender systems is if it’s not possible to kind of.

perturb the recommendations you get, and you end up in this kind of filter bubble of all cat videos being recommended to you. At least here, there’s a feature where the student can express, no, I want to go in this direction. And so they don’t have to have changed their history, or they don’t have to take another course in order for the recommender to change paths or to be modified. They can just sort of state that they want to go in a different.

Matthew Sterenberg (19:13.57)

So as we put on our futurist hats here and think about what this could look like in the future, do you think this is a tool that academic advisors use or institutions use? Or do you think this is something that is in the hands of students? Like what do you think is maybe the closest and then what do you think is most viable?

Zach Pardos (19:39.712)

Yeah, closest is administrators. All of our pilots in building a pilot system for interaction has been with articulation administrators. Administrators, so it could be transfer staff, it could be advising, it could be the articulation officer at campus. And we’ve surveyed them.

We’ve looked at how they use the platform. We’ve looked at what kind of features in the platform lead to more adoption of recommendations, which lead to less. I won’t get into those, but if people wanna see the results of that, I have all my papers at zackpardos.com. We’ll take you to my Berkeley homepage. And then we’re gonna, faculty are next, but the problem of creating paths is so…

vast in the combinatorics, we need to leverage all stakeholders. And there’s more students than anyone else. So, uh, they’re not forgotten. I think they’re a necessary part of the equation so we can empower them to know what to petition for. Right. If the system there are, they’re transferring into doesn’t have these pathways, doesn’t have, you know, this kind of help, this technological support, or they just don’t have enough resources to, to focus on this issue.

Often it’s the students that have to say, well, but I think I should get credit for this. Here’s why. Here’s my syllabus. So I think a system like this could help them.

Matthew Sterenberg (21:06.198)

Which is a tall task, isn’t it, for a student to even have the wherewithal to advocate for themselves and to even know, like, the same thing as with healthcare, right? Your doctor says something, you’re like, sounds good. You know, you don’t think to get a second opinion or to challenge it, and the same can be true for a student pathway where they’re like, I remember registering for classes. I remember thinking, should I be the one doing this? They hand you this thick course catalog, which I’m sure.

You know, it’s all there’s better ways to consume that information now. But I remember thinking, so I’m going to try to get the classes that I want. And if I don’t, what’s next? And you had to get on a waitlist and sometimes you’d email the professor and be like, please let me into your class. Like the idea that yeah, students advocating for themselves is it’s kind of a shame that we are at that point in a, in a way, if that makes sense.

Zach Pardos (22:03.264)

Yeah, that’s the idea, I think, with the prioritization. They shouldn’t be the ones burdened with this. But in the areas where the paths haven’t been pioneered, at least give them some ammunition. That’s probably the wrong word. Some justification to back up their claim. Or if they say, this is the destination school I want, tell them what they should think about asking for if they don’t get it.

Matthew Sterenberg (22:31.586)

So one of the core challenges of this, I think is, and the guided pathways is a phenomenal initiative as you highlighted, but how, it doesn’t answer the question. How do I know what I wanna do? Which I think is always one of those funny things when we talk about all this student pathways, like it’s obvious it’s good to be, to have an idea where my destination is, right? We talked about the analogy.

earlier, which is we have roads. We don’t have great navigation. This is all kind of depending on I know where I want to go. Is there any way that I could think about the research that you’re doing that helps me know where I want to go? Right. Instead of just thinking about the pathway from here to there, can it help me navigate?

when I don’t know what my destination is.

Zach Pardos (23:30.68)

Right, where even should there be? Yeah, that’s a deep question. I’m not even, I’m not gonna suppose that technology can be the only thing, getting consulted in order to answer that very deep existential question. However, the closest thing related to it is we have been using ChatGPT in an experiment that we published to see how well it can recommend majors to students.

based on their personal interests, their any career goals that they may have at the moment, and any favorite or least favorite courses they’ve taken thus far. So we collected that data from students at Berkeley who hadn’t declared a major yet. And we didn’t show them the Chan Chi PT recommendations because we weren’t sure what the fidelity of that would be and the ethics of showing that to a real student yet.

We instead showed that to real campus advisors and asked them to rate the quality of those recommendations. For half of the advisors, we randomly selected half to first ask them what their recommendation would be to the student of major. And then we showed the chat GPT recommendations. And then for the other half, we did it the other way around to see if the chat GPT major recommendations influenced their recommendation, right? Do you see it? If they agree more, yeah, right.

Matthew Sterenberg (24:49.886)

if they would agree with it and yeah, yeah.

Zach Pardos (24:54.312)

And the result was they did agree more if they saw the chat GBT recommendation first, but it wasn’t statistically significant. We had about 40 advisors in the study. The other finding was they scored the quality fairly highly, a four out of five. Mind you, it was advisors who self-selected into this experiment. So it might be people who are interested in technology. But it could have also been people who are waiting for an opportunity to criticize technology. We just don’t know.

It’s a sample, self-selected sample. But we also found that, yeah, they expressed what’s considered to be low AI aversion. When integrating AI into the socio-technical system, particularly when it involves experts, the literature says, you’re going to run into people who dismiss the results of the AI or more quickly to do that than if the same suggestions came from a person. But here,

we’re seeing some kind of warmth to it. And I think in part, that’s because the approach is to support advisors in advising students, at least in this context. In other contexts, maybe you don’t have the resources to have that be the setup, but the advisor can certainly be supervising how technology is used and making sure that the quality is there. And maybe that’s a path forward, that integration of technology.

into education in a way that retains the humane nature of the educational.

Matthew Sterenberg (26:28.778)

I like how we just immediately went pretty much into like a Black Mirror episode with AI where we’re like, can AI tell me what I should do with my life? And we’re like, well, actually sort of, and it might actually be pretty close. And yeah, we won’t even get into like the weird feedback loop where if we just kind of perpetuate it, then it doesn’t get new inputs. And then anyways, so as we wrap up here, if

Zach Pardos (26:36.42)

Exactly.

Matthew Sterenberg (26:56.062)

I’m working at a community college or I’m advising students, like, give me the kind of the summary and why I should be excited about this moving forward. And if there’s anything on the horizon that you want to highlight as interesting research that you’re excited about, we’d love for you to highlight that as well.

Zach Pardos (27:15.)

Yeah, I think if you’re a community college, something to get excited about is the technology. You have degree audits, or you probably have a degree audit system. That degree audit system doesn’t give recommendations to students on what to take in order to satisfy the audit in kind of real time. But I think that’s knocking at the door, that ability. What will slow that is the accessibility of degree requirements. You may have to re…

scribe into another system, which is a pain. It takes a lot of resources. But the ability to articulate. So if you’re a community college and you’re excited about articulating to a receiving school’s new data science program, know that tools are being introduced. We’re working on it. I’ve parchment. A lot of other entities are working on solutions. I think that will be.

more feasible for them to do is to articulate to those destination majors and then, you know, that creates a bigger value proposition for students to start at your institution. And I would look for those tools now and increasingly in the next year. In terms of things I’m excited about, there are so many things to cover. We didn’t even get to AI for credentialing, but I think that there’s a real

a real tangible possibility and utility to use AI to help determine if a student has earned a credential, right? It can be used in this kind of system I’ve described to essentially do what a degree audit does as well as recommend. So if it can do that and it uses the kind of AI technology called neural networks that

chat, GBT and really everything autonomous cars use such that it has developed the kind of abstract representation of what it means to get a particular degree, what it means to take this calculus course at this institution. So with that kind of elemental understanding, which is still naive compared to a person but can scale very nicely and may be sophisticated enough to equate to a degree audit system, can we then take that and say, well, you know, what would it mean to

Zach Pardos (29:41.88)

to get a credential in cybersecurity? What would it mean to get a credential in this, Cisco training or what have you? And I think those credentials can both be defined more quickly with AI and more of a learner’s prior lived experiences, courses, trainings, professional experience could be more seamlessly integrated.

and articulated to those credentials. So maybe they don’t have to do, you know, the whole coursework of the credential based on what they’ve done before. There could be more fine grain analysis of the skills they’ve learned in the past. My lab does some work on skill building and with an open source system called Open Adaptive Tutor. So potentially the combination of adaptive tutoring with pathway research could lead to a kind of more dynamic streamline.

system of credentials.

Matthew Sterenberg (30:39.57)

Well, I think that’s very exciting. And you just signed up for another episode in another season. But the idea that we are really focused on micro credentials, I don’t mean parchment, I mean the community at large. So it’s all learning counts. How do we recognize what you know and how well you know it? And what you’re saying is, you know, the work that you’re doing on the articulation front is very similar, right? Let’s look at the history of the people that have gone through this program.

What do they actually know? What does it actually look like? And then if someone can prove that they know those same things, then they should be credentialed. Whereas now we kind of live in this binary, you’re a degreed or not degreed. And even if you have a diploma, I really don’t, it doesn’t really tell me a whole lot about you. Whereas you could really have a holistic view of all the courses that I’ve taken, the pathway that I was on, get a lot more credentials and just have a better idea of what the learning outcomes.

actually are, rather than just a transcript that says I gotta be in, you know, Latin American history, whatever it may be.

Zach Pardos (31:46.9)

Yeah, that’s right. That’s the vision. I will say there’s not a lot of research that has been done connecting the kind of mezzo scale of course taking with the micro scale of mastery skills and learning objectives. So there is a bit of work and I think research ahead, but because those two pieces are a lot of progress has been made on those two fronts. I think connecting of those two. Is is one of those

tasks, one of those projects and objectives that has now come into the category of it was too high risk before probably to be funded. Now it’s low enough risk to be viable with proper development.

Matthew Sterenberg (32:32.522)

Well, Dr. Zachary Pardos, thank you so much for joining me and check out his work at ZachPardos.com and pay attention to the things in this space. I really enjoyed our conversation and appreciate you joining me.

Zach Pardos (32:44.652)

Likewise, Matthew.

There’s always more to learn.