Non-degree credentials are growing rapidly, but what does non-degree credential quality look like? In this episode, we are joined by Dr. Michelle Van Noy, Director of the Education & Employment Research Center at Rutgers University. She shares a framework for non-degree credential quality and highlights the research around outcomes for non-degree credential attainers.
Matthew Sterenberg (00:00.696)
All right, I’m here with Dr. Michelle Van Noy, Director of the Employment and Education Research Center at Rutgers University. Michelle, welcome to the podcast.
Michelle (00:10.976)
Thank you, Matt. I’m so glad to be here.
Matthew Sterenberg (00:13.708)
So Michelle, let’s start off with this. You’re doing a lot of research in non-degree or non-credit credentials and quality and value. What is credential quality? What do we mean by that?
Michelle (00:28.994)
Well, that is a good question. I think it can mean many things to different people. So that’s one of the things we look at when we try and make sense of what folks mean when they’re talking about quality. It might mean something different for a student or a worker versus an employer versus an institution or a policymaker. And so we try and look at all different ways that you could define quality.
One way to think about quality is to look at the credential itself and how it was designed. So are there certain elements that would predict quality or that sort of indicate quality? Is it aligned with labor market needs? Is the instruction associated with early assessments? Are those high quality? There’s a host of different things that might be baked into the design of a credential that would make it quality. And then there’s the key question is, it lead to…
the outcomes, sort of competencies that you would expect a credential holder to have. So that’s the key thing. It doesn’t lead to skills and competencies. And then the other aspect of outcomes, of course, is outcomes of value in terms of employment and earnings or continued education or, you know, better skill in the workplace. So there’s a whole host of outcomes of value. And then there’s sort of an underlying set of processes that might translate a high quality credential in terms of
well being well designed into an outcome of value, which has to do with the credential being recognized and used. And so you take all those different elements and you think, well, gosh, you some of those are certainly more relevant to students. know, a student worker is probably concerned. Am I going to get a good job? Am I going to make good money? Am I going to be able to, you know, do the things I want to do in my life? An employer, of course, is looking at, I going to get a skilled worker? You know, policymakers are looking at ROI and, you know, other kinds of factors around,
contributions to the regional economy, for example. So it’s sort of, it’s a complicated question. I think folks want one simple answer. And I think, you we can boil it down to a few different metrics, depending on your perspective. But part of it is try to see what the goal of measuring quality is.
Matthew Sterenberg (02:30.936)
Yeah. And this has been said many, many times before, but it feels like we’re in the wild west where all of the different credentials that are coming out, you’ve got micro credentials, you got badges, certificates, you’ve got credentials issued by higher ed, you have professional certificates, you have industry certificates, you have ones that are recognized by state regulators or trade organizations. And then you have ones that are just kind of digital stickers in many ways. And so I think a lot of people are.
We’re trying to solve for, we don’t want just the degree anymore. The degree has been a proxy for what people know. So we’re trying to solve that, but we still really don’t know. There’s so much we don’t know yet about, well, is this the right solution for that problem? What do we know about the outcomes of non-degree credentials? Are they actually leading to better outcomes for students?
Michelle (03:28.258)
Well, I think you raise a really good point in how you framed your question here, which is that I think the first order of business is trying to make sense of what we mean by this whole landscape of non-credit and non-degree credentials and what these various things are intended to do and to accomplish for people.
So, and just sort of looking at the intensity of them, how long are they, what kind of skills are they intended to provide? Is it sort of a very targeted specific small set of skills that someone’s supposed to attain from a particular program or credential? Or is it preparation for entry into an occupation? Some of these credentials might actually help someone get the full range of skills needed for an occupation. And so I think that…
you know, those are very different, you’d expect very different outcomes depending on the sort of design of the credential based on what it’s intended to lead to. You know, some people might take these credentials, you know, more just to try out a different occupation. We see that sometimes in non-credit, but there’s also, you know, there’s just a lot of variation in terms of what the goals are of the programs and the credentials. And so again, part of it is trying to make sense of what they are.
There are some good frameworks out there that are beginning to do that. Chris Mullen recently did a paper with a taxonomy about credentials and kind dividing them up into different categories of macro credentials, micro credentials, and maintenance credentials. around looking at, know, a macro credential would help somebody enter into a full occupation. It’s a full body of knowledge around an occupation or a particular field of study, whereas a micro credential is more targeted.
set of skills in that taxonomy and then maintenance credentials, something that you do to kind of keep up with your field once you’re in it. But that said, mean, this is a sort of taxonomy that we’re beginning to make sense of the landscape, but I think as you started out with, people are using a lot of different terms and it’s not necessarily, they’re not always being consistently used. So when we hear someone talk about a badge versus a microcredential versus a non-degree credential versus certificate,
Michelle (05:28.0)
I think there still is a lot of confusion in the field about what each of these terms mean.
Matthew Sterenberg (05:33.348)
Yeah, and you mentioned frameworks. I wanna make a plug for a framework you put together with others. You have a conceptual framework for non-degree credential quality, and you highlighted the elements here, credential, design, competencies, market processes, and then outcomes. I think it’s fantastic, and check out all the research that they do at the EERC, Education Employment Research Center. A lot of great resources there.
One thing really stuck out to me and I want to just read from it. Market processes are basically, you know, what is the currency of this credential? You, this is word for word from your, your framework here. Market processes are an essential, but often overlooked element in understanding the quality of non-degree credentials. And I can’t help but think, you know, I have so many conversations with people about this, but are we putting the cart before the horse here? Like,
We’re working so hard to describe what these credentials are. Like how do we standardize this? And we often have neglected the exchange, the currency. Like who is willing to accept this? Because we know that number one, credentials are dynamic. They’re changing all the time. The needs of the market are changing all the time. And then they’re often hyper local, right? If you’re a community college, you have a certain set of employers that need like,
So the partnerships element, how do we engage employment is such a massive challenge that I sometimes sit back and I go, we’re doing all this work. We’re, you know, setting up for this party and we don’t even know who’s going to come yet. You know, we don’t have any guests are going to be coming to this party. And I just think it’s, it’s a significant challenge. And that’s why I think your research on outcomes is so essential because these, we have to think about how is this being exchanged?
Michelle (07:11.707)
Thank you.
Matthew Sterenberg (07:27.394)
if we’re gonna get the outcomes we want.
Michelle (07:27.394)
Thanks
Right. No, and you pointed out a really, really key element that we can see from what research has been done so far on non-credit and non-degree credentials in terms of outcomes. We did a review of the research a couple of years ago, a year ago, I guess we released it at our center, and looked at sort of the existing research. And I think one of the things you can see is that there’s a huge amount of variation in outcomes, especially for earnings and employment outcomes.
Overall, there are some modest returns, but there are some fields where there are higher returns or some fields where there’s very little in terms of return. But you do see that that varies a lot by field of study and it varies by geography and demographics. And so I think the importance of those local connections, as you mentioned, in terms of institutions reaching out to employers and having those connections are really important. And for the exact reason that you’re alluding to, which is that to some extent it’s about
making sure that the credential is recognized and valued and used. so colleges that do have those tight relationships with their employers locally, presumably are working to build those connections and to build recognition for the credentials that they’re offering. that’s a really important thing about this dynamic market, as you said, is that there’s so much that’s happening that’s new. It takes a while for folks to understand what it is we’re talking about.
There is a lot of confusion, of course. That’s what everyone keeps coming back to is the fact that there’s so many credentials. There’s a lot of confusion about what they are. And, you know, it’s for that reason that it’s difficult to know what’s valuable and what will be recognized because there needs to be that process of building trust and building recognition. It’s a dynamic, it’s a social process that has to happen in the market. Yeah.
Matthew Sterenberg (09:15.47)
Yeah, and a lot of the badging community, it’s like, how do we engage in the human resources information system, the HRIS? How do we get people to create processes to intake these so that once we build the ecosystem, it can exist on its own? There’s another element to this, which is the bridge. We’re talking about non-degree, but non-credit. How do we build a bridge to credit?
critical is that? I don’t want to say it’s a fail safe, but it gives people additional options. And we know that the two year and the four year degree are still really critical. So what is the landscape of building those bridges of non-credit or non-degree to credit? Where do we exist today?
Michelle (10:02.786)
Well, I mean, think when we look around, I think there’s definitely a lot of interest in building those bridges. People talk a lot about the importance of building pathways. It can be challenging within institutions to do that, to find the right ways to do that and to figure out how to articulate non-credit programs into degree programs. You start to get into more complicated approval processes and just making that work. But I think even assuming that institutions do that and institutions can do that and they are doing that.
You know another question and challenge is you know, how much are students going to take up those pathways? You know from the research again going back to that review the research that we did shows that you know, very few students do move along those pathways and so You know, but if you if you if you think about it also there’s there’s not very many systems in place to keep students engaged after they complete a short-term program and go on to work and so you know, we think about
within degree programs, there’s so much work being done around promoting student retention and engagement. Now we have a situation where you have students who’ve left the school and somehow we’re expecting them to come back and to reengage in Pathways without any other outreach or support or counseling or advising. So I think that’s kind of a missing piece of the puzzle that hasn’t really been tried yet to try and reengage students more. But we also know that students lives…
are sometimes messy and difficult. And so getting out into the workforce and to be able to come back can be complicated as well. But certainly more work I think needs to be done on the student side of that, why students are not coming back and how to better engage them.
Matthew Sterenberg (11:41.76)
And who are we serving with these non degree credentials? And this is another thing I saw in your research where compared to people that are degree seeking students, you know, obviously a much bigger percentage don’t have a high school degree. So when we think about building that bridge, it’s like, well, we need them to get a high school degree first, or maybe they’re, you know, the path to college is not going to be realistic. Like they’re looking for. Or short-term credential for increasing their wages or.
learning a new skill to get into a new job market or wherever it may be, like, it’s not going to be as important for them to get college credit or may not be feasible for them to get college credit. So who are we really serving with these short-term and non-degree credentials, I think is another really important question.
Michelle (12:29.442)
Yep, and that’s a question we need more data on. are doing some work. I’m interviewing students to try and better understand what their goals and intentions are in the programs. And we do see that, like you say, there are students who’ve had negative experiences in educational programs, sometimes who end up in non-credit programs. But that’s just one of many, many kinds of student experiences that you see in these programs. And so there is a lot to be done just in terms of understanding the students, who they are, and what they’re trying to do.
Matthew Sterenberg (12:55.994)
So it’s a good segue into getting more data. How do we do that? Is this gonna be institution led? Is it gonna be led by the states? Is there gonna be a federal mandate to do this? I’ve heard some rumblings of states doing some regulations on you’ve got to start reporting on your non degree credentials. What’s the landscape today? And then where do you see it? And where do you maybe hope that it will go in the next several years?
Michelle (12:58.877)
Right.
Michelle (13:23.756)
Yeah, that’s something that our center is also doing a lot of work with states on. But I will say that the interesting thing is that I’ve been part of conversations at the federal level around adding in measures of non-credit activity for quite some time. And it hasn’t quite happened in terms of adding in a measure to iPads, for example. But it was through that I came to see, well, gosh, we’re not quite ready to add this at the federal level. We don’t quite know enough about how to measure.
But I’m starting to see that there are some states around the country who are doing that. And so we have this project that we have crafted to work with states who are collecting noncredit data to learn from what they’re doing and share practices across states and look for common ways to define and measure noncredit activity. we have been working now with eight states to develop profiles of the noncredit data that they’re collecting and share those. are on our website.
And then we also have been convening a learning community of states where we bring folks together to share practices around that. And I think we’re seeing a lot of interest in states in building up these systems. We have close to 40 states that come to our meetings to just share practice and we know there are others that are doing this work as well. And we’re not just us, I there’s many other efforts right now that are sort of focused on states in terms of building up capacity to have better data.
But of course, the institutions are also the building block of that. So the states, I think, are really where there’s a lot of locus of activity, but they’re working very closely with institutions to figure out how do you build those important data collection efforts into their engagement with students, which is not always easy. the non-credit side, is sort of a desire to keep the reporting requirements minimal, to not make students complete a lot of paperwork. There’s not big, long admissions and forms. There’s a lot of…
reasons why that hasn’t been part of the process, but there’s also a need just for fundamental systems building up within institutions to collect those data better and then to work out some of the kinks in terms of trying to link those into credit bearing systems. So I think it really has to happen at all levels, but certainly there’s a good amount of activity at states and correspondingly within institutions. mean, groups like Education Design Lab, for example, has been working with.
Michelle (15:33.506)
institutions and that’s one of many elements of their work is trying to get better data on students. So there’s just a lot of efforts that have to happen and not one of them is going to be the solution but there’s a lot that has to happen to kind of build that infrastructure. Especially if more funding were to come down, if federal funding were to be passed for Workforce PAL, that’s really going to create more an impetus to really know what we’re looking at here, what kind of programs and what students’ experiences are with them.
Matthew Sterenberg (16:02.528)
In the same way that I think it’s good to be critical of non-degree credentials and to really like, what are we doing here? Let’s really understand this is new and let’s make sure we create frameworks. But after reading through your framework, I was also struck by, know, we don’t really do this with kind of the credentials that we accept every day in a certain way. you know, do we go back and we say, hey, what’s a, you know, what’s a French?
degree worth, you know, and what does it really communicate about what you know about French or history? And so I think it’s really good that we are, you know, putting a magnifying glass to this, but at the same point, I also think it’s in some ways unfair, you know, like, are we saying what is the market? What’s the currency of this degree or diploma? think so it’s more of an open thought, but has that ever occurred to you as well that I’m like, why are we the standards for this are
higher than the credentials that we accept every day and that we have no problem paying hundreds of thousands of dollars for.
Michelle (17:06.082)
I mean, think there is kind of an accepted idea of what like a bachelor’s degree, for example, means. I mean, then we do have more data. I mean, frankly, that’s true. Like if you look at like the Georgetown Center, they’ve done a lot of analysis over the years over the returns to degrees and the variation by field of study. So we do have that information and from accreditation, we have sort of learning outcomes and things like that. But I think you’re right that there is a lot of emphasis right now on trying to measure this on the non-degree side.
But I think we really do need more data. We really don’t have the same kind of data. But I think to your point, it’s valuable to be reflecting on these points and asking these questions across the board of what’s the value of this degree? What should people be learning in some ways? I think there’s a lot of focus on earnings outcomes, and that’s important. But fundamentally, if we really want to measure things, the real gold standard is did somebody learn?
Did they come out with skills? Did they come out with competencies that we think the degree or credential should reflect? So that’s a hard one to measure though.
Matthew Sterenberg (18:11.414)
And you do highlight that in your framework. this was a person from Lightcast on the podcast several episodes ago. they talked about like, we often just look at wages. And in your report, you highlight their societal benefits. We want people to keep learning. We want people to keep investing. But that’s a much harder thing to quantify. Rate your satisfaction, your happiness, your work-life balance, the fact that you
were paid the same, but you got to go into a career that allows you to be home more. It’s a lot more difficult to quantify those level of things. We just want easy answers.
Michelle (18:48.788)
Right, right. Or even for a region to say, this has improved our economic well-being overall. mean, there’s a lot of complicated things that we’d like to look at that we can’t. But fundamentally, we kind of have to start where we can. So that’s why we sort of started with what the states are collecting and non-credit. A lot of it is within community colleges. But beyond that, there’s many other systems that are doing that.
know, first step, think everyone really cares about looking at the labor market outcomes, which is, you know, a very, very important part of all of this. And so that’s kind of where a lot of the activity is happening. And then as we can try and move into some of these other ways of measuring, which are also important for understanding. So there’s a lot to be done.
Matthew Sterenberg (19:25.454)
Michelle, is there anything I didn’t ask you that I should have asked you or anything that you think is important for the audience to hear about non-degree credentials?
Michelle (19:35.898)
think that it’s interesting that everyone is sort saying, well, non-agree credentials are going to kind of replace degrees. And it may be the case that they do. think that part of it is to recognize the value of each kind of learning experience and where it can fit for different people at different points in time. So it is an interesting point of reflection and realignment about what people want to do and what’s most valuable and what’s useful for people in terms of life success, economic success.
But it is an interesting point of reflection that we’re in right now and sort of realignment. So, I’m curiously looking to see where things are gonna end up, but there’s a lot of things that were at play right now, which are interesting.
Matthew Sterenberg (20:21.54)
Quick plug for your book too, you wrote a book called Credentials, which is, it sounds like it’s about everything credentials, but I am going to give away five free copies for the people, first people that email me. If you email me at credentialsonscripted.gmail.com, I will send you a copy of Michelle’s book Credentials. So a little plug for that credentialsonscripted.gmail.com but.
Thank you so much for joining me, Michelle, yeah, I appreciate you being on the podcast.
Michelle (20:51.522)
Well, thank you. Thank you so much for having me. This has a great conversation.
