Mark McNasby on the Evolving Role of AI in Higher Ed

Full Transcript

[00:00:00] Dustin: So we are here, uh, for this episode. Going to be, I guess, finally really interfacing with and kind of dealing with head on a topic of artificial intelligence and higher education. Obviously, there's a lot of applications and a lot of implications, but you know, we'll kind of just be doing a sort of very high level perspective on the way that.

[00:00:27] AI is impacting higher education at this current moment. Uh, we are recording this for the record in August of 2023. If there's other seismic exchanges and shifts and things in the future, just let that be noted. Just really grateful for our guest. Jumping on to kind of, you know, give us sort of this, you know, exploration, kinda start our journey here on this show so we can kind of add our perspective to the mix as.

[00:00:46] You know, everything kind of unfolds here. So Mark, if you wanted to give a brief introduction, kind of summary, elevator pitch of who you are, and then we'll talk more about the work that you're doing and specifically dealing with ai. Awesome. Well, 

[00:00:58] Mark: first, Dustin, thanks [00:01:00] for having me on your show. I, I've been an entrepreneur for about the last 20 years, primarily in education technology.

[00:01:08] Started a company called Optimal Resume, where we sold resume building technology into the career centers of universities. Um, then I started another company called Brimm, which was really a, a communications platform, very similar to Slack, but really geared towards a hi, the higher education community and Brim.

[00:01:28] What we found was that students were asking, The career Center, the same types of questions over and over again, and that was our epiphany moment for why don't we create a chatbot for higher education that can help answer questions about whether it's admissions related, but all the way through the student experience to career services.

[00:01:51] I started this company in 2017 and currently we're working with about 200 higher education customers. 

[00:01:59] Dustin: Yeah. Yeah. I kinda [00:02:00] wanted to like, you know, break that down a little bit too, as we kinda explore what ivy.ai does is like that. Even just in the times, you know, since 2017 that you, uh, founded this venture and everything, like so much is been happening in this sort of space of how you sort of like integrate something like that from just, I think the, the baseline of you kind of just build sort of a knowledge based repository that sort of pulling from based on inquiries.

[00:02:24] Moment of really elevating that and sort of stacking upon that foundation to really make something like a chatbot work kind of even smarter. So I guess, yeah, just explain briefly what ivy.ai does and maybe even just kind of chart that over the, the past few years. Yeah, I 

[00:02:39] Mark: think that's actually a really good way to talk about it, is to chart it.

[00:02:42] So if you look at sort of the early days of the 2017, um, literally through 2022, Um, Ivy was, so there's basically two different types of chatbot technologies. There's retrieval, chatbot technologies, and then there's generative. Generative [00:03:00] has been in the news a lot lately. Ivy started off as a retrieval based chatbot, and essentially what that means is that Ivy has a data bank of stock answers, and then when someone asks a question to the chatbot, We try and find the best stock answer and then we fire that answer.

[00:03:19] And so that's a retrieval based chat bot. About a year ago, Ivy saw the writing on the wall with, uh, what was happening with open ai, and we decided that we needed to pivot into a generative space. And so the way that Ivy works now is rather than retrieving stock, canned answers from a knowledge base, Instead, what we do is we read in or ingest, if you will, all of the knowledge of one of our customers.

[00:03:49] So for example, like an admissions office we'll read in their website, we'll bring in all of their P D F files, and then there's several layers of ai. There's a level where, um, when [00:04:00] someone asks the question, we're finding the best. Content across all of their assets. That would be the best response for that question.

[00:04:09] But then we're using generative technology to repurpose that content in a way that's contextually relevant and sensitive to what that person asks. And so Ivy has become a generative chatbot over the last year, um, with our, our flagship product, 

[00:04:24] Dustin: IV Quantum. Yeah. I mean, and that's what's really exciting because I think we've gotten to a place where, at least in my perspective, like the value, the kind of utility of just like a chat bott on its own sort of merits of just that, that kinda retrieval model, uh, that makes a lot of sense.

[00:04:41] There's just so many use cases, obviously, like, I think anybody's kind of interacted with all of those on, you know, any company's website. But then it's just finding higher ed, you know? Different offices or resources a student might be wanting to try to find or find information about or anything like that.

[00:04:57] And it's like you could, you know, pretty build a [00:05:00] pretty robust and sort of useful knowledge base, but you know, it would be theoretically kind of limited to just sort of like, just call up response pretty simple things. So it's like how can we, you know, build upon that now? So I guess specifically you.

[00:05:14] With that more generative model that you mentioned, you know, that's obviously where everything you're kind of leading. Just to kind of, kind of, you know, summarize it, I guess in the, in the simplest sense as we start to zoom in. Like what are just some of the benefits you've seen for institutions levering this technology?

[00:05:31] Because I think just like that might be some of the resistance. Hey, the thing we have is working fine, you think, to convince me, like, sell it to me. I guess, you know, like that it, that it feels like, it's like you're clicking into a completely different sort of, you know, just perspective on what. You know, even just possible with a tool like this.

[00:05:45] Mark: Yeah. I think there's a couple of ways to think about the benefits of generative technology from the perspective of the customer itself and the perspective of the end user. So if we think about the customer, I. Itself. Think about the [00:06:00] school. So the first thing generative technologies do for a school is that it dramatically reduces the implementation time to build a chatbot.

[00:06:11] So when you're building a retrieval chatbot, you're literally trying to think of every answer that you could possibly. Um, uh, have for a student, which is a terrible use of time and energy, and instead what you're doing is you're essentially saying, Hey, crawl this website, read this document. So the building process for the chatbot becomes a lot, um, more straightforward.

[00:06:30] The second thing is that maintenance of the chatbot. Becomes much easier. So every day, the idea behind this generative approach is rather than have two sources of truth, you know, your chatbot and your website, what we want schools to do is make their website the source of truth, right? And so what we want is anytime the chatbot gets stumped, We wanna try to bring that answer back onto their website so that the next day when we crawl that content, the [00:07:00] bot has that information in the bot's brain.

[00:07:01] So from an implementation and maintenance perspective, generative technologies make schools' lives a lot easier. Now, from just the chat bot experience in general. Um, what we have found since moving to a generative, uh, model. Is that, so we ask students at the end of every conversation to rate their experience, right?

[00:07:22] Give it a one star to a five star rating, where five is the best. What we've seen after launching the generative approach is that we're getting three times as many five star ratings as before, and so there is, so what does that mean exactly? So the first thing it means is that we have better coverage for, we, we cover a lot more content than the retrieval model.

[00:07:44] So the bot has a lot more knowledge in it, which makes it. Um, more effective at answering questions, but also the fact that it's contextually relevant or I could ask a follow-up question and it will keep the context of the conversation. It's just a much more human, enjoyable [00:08:00] experience than those kind of constipated, you know, forgive the expression experiences that that retrieval based chatbots have where it's just spitting out answers at you.

[00:08:10] That don't sound human. And so it really, so at the end of the day, what that does is it improves outcomes for students, right? They can get the information that they want, which helps them either pay their tuition or, you know, break some barrier. That they were trying, which ultimately will help with retention rates and graduation rates.

[00:08:30] But we've also seen generative technology be very useful on the admissions side in generating essentially new leads as prospective applicants for the school. And so putting this engaging technology on the website, for example, at the University of Oklahoma, their freshman class increased 10% and they attribute some of that increase to the chatbot technology.

[00:08:54] Dustin: Yeah, I mean, it's just, you know, amplifying your kind of bandwidth, really. [00:09:00] Like, that's just the simplest way to put it. Where, like you say, kind of like, you know, the, the limiting, uh, you know, your colorful metaphor, but the idea that like, yeah, like if prior to this you'd be hitting a lot of walls, that would be sort of, you know, restrictive, limiting, just any of those sort of like active, you know, feelings or words there.

[00:09:18] And then, you know, almost just unleashing. You know, a chat bot and, and that utility to be more comprehensive versus like, if you just had a blind spot with whoever's sort of owning the development or, you know, building that knowledge base or something. And, you know, not to, just being able to anticipate every permutation of how somebody could answer or ask a question so you can try to answer it.

[00:09:38] And, yeah, I mean it just, it is really exciting because I think, I'd imagine maybe kind of as. I'm imagining there's some people where just that premise of the, the maintenance or sort of the limitations of like retrieval model, they're like, they didn't feel like even though it was probably better than what they were doing before, like they didn't really have as much bandwidth as they could or should have.

[00:09:58] Like the [00:10:00] idea of having to build and maintain a generative or a retrieval bot. Was not appealing. A generative one would be appealing because they're like, wow, like that's just gonna be so much easier to set up and maintain and stuff. Like, do you feel like that was kind of the case, that like, you know, retrieval wasn't quite there for some people, but then when generative is sort of proposed, it's like, oh my gosh.

[00:10:20] Yeah. Okay. That was more of what we were looking for. 

[00:10:22] Mark: Exactly. So one of the major problems with retrieval based chatbots is that information gets stale, right? So when you think about a school information's dynamic, like dates are always changing. Classes are always changing. People are always changing.

[00:10:36] It's a very dynamic, fluid situation. And, and essentially when you have a retrieval based chatbot with static answers, it can start giving answers from three semesters ago information that's old out of date. And that obviously causes another set of problems, which is, you know, obviously wrong information, which could hurt the brand and it could obviously negatively influence a student [00:11:00] outcome.

[00:11:00] And so really what the generative it's, it's, Generative technology sort of reduces that, that dependency on managing these two knowledge sources. And instead it's like, okay, let's just focus on our website or our knowledge base as that source of truth, and then have the chat bot be slave to that and almost act as an advanced search utility to find that information and then repurpose it in a more human way.

[00:11:29] And so, The retrieval sort of methodology for chatbots is going to be a dying breed for sure. There. There are always going to be certain use cases for retrieval. For example, like Ivy uses retrieval technology in certain cases. Where personal information is required. So for example, if I was a student and I asked, what is my account balance?

[00:11:53] In that case, well, we we're gonna use a retrieval response because we're ultimately gonna call an integration to [00:12:00] connect to the student information system to pull that information back in. So the best chat bots are going to be the chatbots that have. Generative technology that that are going to really improve the implementation, maintenance, and performance of the bot, but also retrieval based technologies for those edge cases where you wanna deliver that personal experience and retrieve information that is specific to that end user.

[00:12:25] Dustin: Let's play a game. What keywords does your website rank for? What doesn't it rank for that you think it should? What are a few opportunities you could be winning on if you tweaked some website copy. Okay. How'd you do? Not? Great. That's okay. 'cause our friends at DD agency want to answer all of those questions for you.

[00:12:42] And then some DD agency is a higher ed specific marketing technology agency. That has conducted countless SS e o audits for colleges and universities across the country. 

[00:12:52] Mark: In these audits, they detail where you currently rank what you could be ranking for, exactly how a copy should be tweaked on website pages 

[00:12:58] Dustin: and much more.[00:13:00] 

[00:13:00] If this sounds like something you could benefit from, give those folks a pinging and be sure to mention that enroll, if I sent you to claim a 10% discount on any of their SS e o offerings, head on 

[00:13:08] Mark: over to enroll.org/dda 

[00:13:11] Dustin: s e o, or simply follow the link in the show notes below. That will guarantee you 

[00:13:15] Mark: a 10% discount.

[00:13:16] Off of your audit again, head on 

[00:13:19] Dustin: over to enroll fify.org/dda ss e to learn more now onto the show.

[00:13:27] Yeah, that's a good clarification. Yeah. The idea of having the best of both worlds, where like there's gonna be that database that's gonna like a separate system that you can kind of plug into to get that information versus just like the publicly available website and stuff. Because I remember. Years ago when I was like playing around with the, building a knowledge base for a chat bot in a prior role.

[00:13:46] Like they were kind of working through some of that stuff of like, oh, we don't wanna like reference, like in the answer, information's gonna change because you have to keep updating it. But then it's like you would end up just like pointing to a website or something anyway. And then it's just kind of that idea of like clicking [00:14:00] now of like, well, we could just have it learn from that website that you're updating anyway.

[00:14:04] And not having to just like send somebody out. They're just getting the answer like there, whether it's. You know, you're texting it or it's in an app or the website or wherever else, you know, somebody might be interacting with something. So, um, yeah, it's really, really fascinating to, to see that. 'cause yeah, I mean, I think just the idea of like focusing in on the student impact, like that idea of just.

[00:14:27] At worse walls, like they're hitting walls, getting incorrect information or it can't get an answer, so they still have to wait. Or kind of at best like friction of having to like jump around still to like different places to get an answer because it's just like you can find the whole list of this or that here it's like, you know, hopefully they could just like get the answer and be able to.

[00:14:45] Pay their deposit or register for classes or pay their bill or do whatever else. So, and yeah, I mean, I can think it's always just pretty well proven of like a lack of awareness of resources or just like, you know, you hit friction enough, like any of us sort of bail out of like completing [00:15:00] a survey or doing whatever it might be that we're, uh, doing.

[00:15:03] So yeah, I mean like the just greater student satisfaction, the greater student retention and more, you know, completing applications and those sort of things. Yeah. I guess any other maybe like anecdotal examples I guess that, that you can see of just like AI elevating, you know, the work that you're doing with institutions or just sort of those impacts of things that kind of come to mind you?

[00:15:25] Yeah, I mean, 

[00:15:26] Mark: I think of it as really AI is kind of a force multiplier for schools and so. So I, I can see it, I can see it helping schools at a number of different levels of the institution. So, for example, I, with Ivy, our most common success story is sort of the improvement of customer service, right? So the ability to deliver accurate, unbiased answers, and to do it very quickly.

[00:15:59] Use the [00:16:00] underlying knowledge sources is a, is a huge benefit to schools because they don't have to have someone answering emails or on the phone or just handling a queue in an office. And so there's definitely a reduction we've seen for that customer service need, which allows our schools to focus on more high impact items.

[00:16:20] Right. Really helping someone. Troubleshoot. And so by sort of taking the tier one stuff off of their plate, they're able to really dial in and perform more value added services. So it's in much of the same way that a bank teller in the, you know, in the, in the eighties and nineties as we moved into a t m world started, you know, talking more about mortgages and less just doing the, the simple transactions.

[00:16:43] So they've become much more consultative. And the same thing is happening at schools where, Rather than just handling a tier one, they're performing more advisory, um, services. The other thing I've seen in addition to that is the efficiency of responses. So [00:17:00] one of the channels that Ivy has, um, built a utility for.

[00:17:05] Is the ability for the chat bot to respond to emails. And so if you think about the, so what you want to try to do is you want to try to triage the traffic from the sources where it's coming from, right? Website chats, you know, calls. Emails. And so we built a plugin for, for Outlook and for Microsoft Outlook and, and Gmail.

[00:17:28] And essentially when a student or someone emails the, the faculty member or staff member at the school, the bot can generate an automatic reply. And so what that ends up doing is it's becomes a huge time saver for them. So rather than having to refer to the underlying document where they need to, to retrieve that knowledge to answer the question, um, the bot will actually generate it and then they can edit on top of it.

[00:17:53] So what we're seeing is that, you know, from the customer service perspective, um, of handling students, there's a lot more [00:18:00] volume, but also as an individual, um, service provider. Our schools are able to get through a lot of those student requests faster because of this generative technology. And so we're also seeing in some, the AI starting to be used in the classroom too, which is going to be sort of the, the next evolution I think, of the big movement of AI in in higher education.

[00:18:25] Dustin: Yeah, I mean, and that, that's kind of definitely wanna head towards the future, I guess, you know, looking towards the horizon. But like, I mean that's exactly, I like where my brain was going was that sort of saving time. And if it's replacing anything, you know, it's replacing just the rudimentary questions you get, you know, several times.

[00:18:40] And it could be 24 7 immediate responses or just having to have things get like bottleneck with certain staff members. So it's. It's replacing, yeah, that low level stuff and augmenting, because I think it's just that dynamic that I think people are sensitive to of just, oh, is it again, you're like, you know, replacing all the staff members or something, but it's like, oh yeah, just like elevating what you're able to do, you know, [00:19:00] uh, being more consultative or whatever.

[00:19:02] And that like, yeah, like so much of higher education. Is that like customer service sort of like experience that students are having of just being, you know, made aware of what's available to them? You know, they understand how to, uh, take advantage of them and. Just getting questions answered and then like you said, like, you know, understandably so that like the academic experience and sort of faculty interactions.

[00:19:28] There's been a little bit more, I think of certainly, yeah, that kind of protectiveness or sort of skepticism and just kind of figuring out how to make it work. But even just that idea of like, you know, getting connected with the tutor or navigating an L M Ss or like certain things like that. There's definitely like a pretty clear path and it's just.

[00:19:45] For each institution or leaders, I think to be intentional about where they want to sort of plug this in and sort of utilize it and maybe just sort of roll it out over time versus just kind of forcing it on on everybody at once or something, or you know, however that might kind of, [00:20:00] uh, play out. But yeah, I mean, just the, the ability for, you know, these, these staff members to kind of, you know, use their expertise and their experience.

[00:20:11] Work with students on a more, kind of, the word I'm looking for is, but just like a deeper basis is just really powerful and just saving time and, and money and everything like. Crack this nut, I guess, you know, in a lot of different ways of like, whatever's gonna kind of resonate and if there are pain points that, you know, a particular institution is facing.

[00:20:29] So I guess, yeah, like to, to build on kinda what you're getting at, like as you're looking towards the horizon. Certainly, you know, academic experience I think is one area that we'll see a lot more. I. Development of AI and higher education, but just anything else I guess, that you're sort of seeing through, through your work with the institutions on this?

[00:20:45] On this 

[00:20:46] Mark: front? Yeah. I'm starting to see, so I think that schools are focused on the low hanging fruit right now. Um, and the low hanging fruit, I would say is more of the. The non-academic areas of the school, [00:21:00] right. The admissions, the financial aid, the registration, the bursar, those types of areas get a, they're essentially cost centers for the school and, and most of the interactions that they have can be automated.

[00:21:14] There are obviously some that are, that require human intervention because they're, they're difficult. My point is that while schools have addressed this kind of low hanging fruit, What they're seeing is that this technology can be useful in other areas of the institution related to learning. And so, for example, like right now, Ivy is working on a partnership with one of our customers where the, we're, we're building a bot based on lectures.

[00:21:43] So we're uploading 25 lectures, we're uploading the PowerPoint. Decks that were associated with the lectures, the syllabus, reading materials from the class. And then, um, we are building a bot that students can ask sort of the, the routine mundane questions about [00:22:00] what's due and, you know, do the next class schedule, but they can also ask.

[00:22:04] Questions about American literature and you know, questions that are deeply rooted in that professor's knowledge and expertise. And so I do see that there's going to be a trend in the, in the academic areas, but I also see the possible scenario of having. Companion apps, right? So imagine a situation where, um, you arrive as a freshman at a university say, here's your, your friend that's gonna be with you for the next four years.

[00:22:33] And so think about that generative, uh, sort of interaction where you could walk a person through orientation all the way through their career services experience at the school and be there as an academic assistant. Be there as if you're having a bad day and you need a little love, you know, using the A bot in that capacity.

[00:22:53] So I think there's going to be opportunities for individual sort of companion apps that will be your digital assistant, [00:23:00] that will be with you throughout your schooling experience. And so I can see a bunch of different futures. And then ultimately I, you know, I do think about, you know, this case where there is a g i.

[00:23:10] So this where there's artificial general intelligence where an AI knows everything. And so what happens when an AI can teach a class? I. Right, like that, those are really uncharted territories. What does that mean for a school? What does that mean for their brand? What does that mean for their faculty?

[00:23:28] There's a whole new world out there that's sort of bubbling up and it's unclear, um, but it's really exciting about what that could look like. Um, and, uh, and so I'm really, uh, interested to see how the future's going to unfold. 

[00:23:42] Dustin: Yeah. Yeah. I mean there's, yeah, no, no shortage of kind of like use cases in that area of like, you know, what could it either.

[00:23:49] Kind of help develop for us or facilitate for us, or you know, support with and everything. But I think, yeah, like the near future is that idea of like, you know, it's not just like, okay, jet, jet G B T, that's pulling from the [00:24:00] entire internet. It's like, no, I'd kind of want something that can kind of contour around.

[00:24:04] The academic experience that we're trying to curate here for a student where it's like, yes, I developed everything for this course as a faculty member. Like, could it just pull from, you know, the book that I wrote that we're using for this course, or all the resources that I put into the, you know, the l m s or something and it's just like pulling all those up versus being like, wait, was that module 2.13 or something that had that definition of this?

[00:24:25] Like you can just ask it and. You know what the faculty wants their student to know about that for the exam that's coming up, you know, next week or whatever, like, so I think that that's a interesting sort of clarification of just like how a AI chat bot or anything else would be sort of introduced into that sort of experience would be really tailored toward that particular, uh, You know, experience that the faculty is creating for their students.

[00:24:51] So, yeah, and I mean, I think that idea too of, you know, you're seeing just like, you know, yeah. AI art or you know, you're making a voice that sort of is saying something or whatever. [00:25:00] Like yeah, there's like all these things where it's like we're just like at the early stages and I hope that people don't get.

[00:25:06] You know, I think we've, we've seen a, a, I guess, a fair share of it, of like two alarmists of just sort of like the sky is falling, what's gonna happen or whatever. Just like, right, let's just sort of examine and scrutinize and sort of just like, you know, see what we can do with this. We're at sort of this like, sandbox stage of just sort of playing around and stuff.

[00:25:21] So, yeah, I mean there's, uh, gonna be a lot, lot more to see for sure over the coming years and everything. But we will wrap up though as we, uh, always do. Certainly, you know, if you wanna, uh, kind put some reference to anything. You know, reference, uh, for folks to check out to learn more about this, but just a final thought call to action, uh, as we wrap up the episode.

[00:25:40] Mark: Yeah, I mean, so from, from my perspective, my advice to schools on a go forward basis, starting as soon as possible, is to get your data ducks in a row to really take advantage of. The AI future that is, um, knocking on the door. You're gonna wanna have really good data [00:26:00] and then all of the AI will be powered off of that data.

[00:26:03] So it's going to be important to get your websites organized, your knowledge bases organized, um, and then I would highly recommend partnering with an AI expert, um, who can be your Sherpa through this adventure that we're all going through together. Um, but I definitely think it's gonna make sense to have industry partners that can, that can.

[00:26:23] Take the, the data management practices that you've put in place to really extract as much value from those assets, um, for the stakeholders 

[00:26:33] Dustin: of your schools. Yeah, I mean that, that is a, uh, powerful call to action. It's just like, you know, just get started, uh, get started with partners and experts and champions and all those other folks that can sort of help get started somewhere, is kinda been one of my lines that I keep saying where it's like, yeah, like start it in just one particular office that is like receptive or sort of has maybe been, you know, doing some research and building their expertise and then.

[00:26:57] You know, uh, certainly folks like yourself who are doing the hard [00:27:00] work to build all the kind of, uh, you know, software to do this. So, certainly appreciate you, the work that you and your team are doing. And you just for, for jumping on and sharing all that you did for this podcast here we Wild Ways to Connect with You and Ivy Subscription show notes for this episode.

[00:27:14] It's really great chatting and kind of solidifying some of my knowledge and I know that for be the case for other folks listening as well. So yeah, just thanks so much for, for hanging out. 

[00:27:22] Mark: Thank you Dustin. Appreciate your time.

[00:27:28] Zach: Hey all. Zach here from Enrollify. If you like this podcast, chances are you'll like other Enrollify shows too. Our podcast network is growing by the month and we've got a plethora of marketing admissions. And higher ed technology shows that are jam packed with stories, ideas, and frameworks that are all designed to empower you to become a better higher ed professional.

[00:27:47] Our shows feature a selection of the industry's best as your host. Learn from Mickey Bains, Jeremy Tier, Jaime Hunt, Corinne Meyers, Jamie Gleason, and many, many more. You can learn more about the Enrollify podcast network [00:28:00] @podcasts.enrollify.org. Our shows help higher ed marketers and admissions professionals find their next big idea.

[00:28:06] Find yours @podcasts.Enrollify.org.