Hello, welcome to this presentation on bridging program and course outcomes. My name is Dan Crawford and I'm happy to be here with you today and sharing this with you. So, upon completion of this presentation, learners will be able to describe the process for bridging program and course outcomes, be able to apply concepts of bridging outcomes through curriculum mapping, and identify strategies for evaluation and process improvement through bridging program and course outcomes. So, I think it's important that we make this differentiation and define these things. This may seem elementary, however, I know that I am recording this content in a series of presentations and I don't know which order you are watching this in, so I want to make sure that we have these definitions set in place before we advance through the rest of this content. So, program outcomes and course outcomes are different. Program level outcomes are what students can expect to attain by the time the program or degree is complete. This might include, you know, knowledge, attitudes, values, ideas, skills, could be a broad range of things. And this is, these are ideas of what our students will be able to do or look like at the end of their academic program. These focus on what the students will learn as a result of their experiences throughout the program, not what they're learning in their individual courses. Now, course level outcomes, these are knowledge or skills that a student will have developed upon completion of an individual course. You know, these, I have on the slide here, they may have relation to other course outcomes, but, you know, I think that there should always be a relation to our program outcomes. But when we think about other courses, this is within the context of one course or in the context of a series of courses, you may have relation of this course outcome to other course outcomes because these may begin to shape a broader skill set. Finally, course outcomes, when we think about the content that's introduced at the course level with course outcomes, it really follows this progression of introduction of skills or knowledge, reinforcement of that, and then mastery. And that can take place in the context of one course, and those course outcomes can reflect just that start to finish process, or this may take place across a series of courses. So now that we have defined those, you know, I think we should talk about the relationship between outcomes. So when we talk about designing outcomes, design should always begin or stem from our program outcomes. Our program outcomes help to shape what needs to be taught. Remember, our program outcomes are this picture of what our graduates need to look like. We have to know what they need to look like before we can know what we need to teach them. And so having that mindset, and I have recorded a presentation on backward course design that will provide you a lot more depth on that, but having that mindset helps you to understand this relationship between the two and how these begin to correlate and shape. Course outcomes then are designed with direct correlation to program outcomes. So program outcomes should have, like well-defined program outcomes should have subskills or a subset of concepts, ideas, knowledge, skills that build into making that program outcome. And so those start to take shape in the course level outcomes. Now design, as I mentioned, progresses from macro to micro, program to course level, and in some places it makes sense to have like even within a course like module level outcomes. And so, but it's important to remember these need to progress logically. There needs to be a clear like logical progression so that you can look backwards and forwards and see how these relate to each other and how these are linked. Now I can say that that should be a clear bi-directional relationship between the program outcomes and course outcome themes, but keep in mind that bi-directional relationship is after they've been established and being able to look bi-directionally at how they're related. That bi-directional relationship is not how they're formed. They're formed macro to micro, but once they're formed, you should be able to clearly see the bi-directional link between them. So let's start with, you know, I mentioned we have to talk about how we, you know, program outcomes can be broken down into smaller pieces of information, course level outcomes, subskills, those sorts of things. So let's talk about how we do that. So if we begin with these program outcomes, this picture of what our graduates need to look like, we can start to consider what the logical subdivisions are of that. How do we look at these big lofty goals and how do we get from where we are today of students entering our program to where they need to be? How does knowledge and learning need to be structured to get them from where they are today to where they need to be upon completion of the program? There's a lot of different things that can be considered as a part of this. Knowledge, skills, values, attitudes, behaviors are all pieces of this. You know, there's not an academic program in advanced practice nursing that is only based upon knowledge alone or is only based on values alone or only based on skills alone. These have to accompany each other. And a lot of times to get to the end goal of where we need to be, we have to realize there's complexity in this. And that complexity is layering of these different elements. So I provided a sample here just to give you an idea of how we might start to break this down. So sample program outcome that stems from the essentials is to demonstrate expertise in advanced nursing practice. Right? So this is a big goal. This is not simple. This is not like one thing that somebody can just go to class one day and learn and be done with. This is something that develops over time. And so we start to think what skill sets need to be a part of this. What does an expert advanced practice nurse look like? Well, they're going to have expertise in pathophysiology, in disease management, in pharmacology, you know, in motivational interviewing, in therapeutic communication, in population health. We can get down all sorts of pieces of this. Like these are big ideas and this is where you start to break these down. So if that is one of our program level outcomes, one, if we think about one outcome at the course level or one piece of that is that students will be able to use knowledge of pathophysiology to identify common illnesses and problems as one component. That alone does not make a great expert advanced practice nurse. But I would argue that you can't be an expert advanced practice nurse if you can't use knowledge of pathophysiology to identify common illnesses and problems. So if we think about this, we can start breaking down. And then when we have these broken down ideas, these are used to build our course level outcomes. So once we break down these ideas, like these broad program outcomes into our knowledge, skills, values, attitudes, behaviors, and so on, we need to think about how we organize or allocate these. So these, as I mentioned, become our course level outcomes. And those course level outcomes, we have to know what to do with them, right? We can't just break down all of these things into a whole bunch of course level outcomes and then just leave them in a bucket. So there are two, I would say, most common traditional methods of organizing concepts. One is course organization and the other is threading. So a course organization is if you have a common set of concepts, you take all of those and put them into a course. This course is about policy. So let's say our topic is health policy and advocacy. We're going to take everything students need to know from introduction to mastery and structure it into a course about health policy. Students will learn health policy here. And then they're going to go on to their pharmacology course and they're going to learn pharmacology and pharmacology. And they're going to go to leadership and learn leadership and leadership. They're going to go to pathophysiology and they're going to learn pathophysiology. There are certain benefits to that and drawbacks to that. The flip side is threading things and saying, okay, so we have these, I don't know, 11 different skills, knowledge, pieces, attitudes, behaviors, values, that we've identified that factor into building towards this program outcome. And we're going to take these and we're going to put these into several different courses in order to start putting the pieces of the puzzle together, if you will. So we'll use health policy as that example again. So maybe you have a course about health policy. It talks about the policy process and how policy is designed. And in a leadership course, you talk about health policy and how to lead as advocates and lead for change at the policy level. And maybe in a population health course, you talk about how policies have a direct effect on population health and how different populations maybe are marginalized because of how policies are written and how policy change can affect those individuals within population groups. So these policy concepts then become threaded across a series of courses. There are pros and cons to this as well, and you have to consider the pros and cons of each method and also what's the best fit to achieve your program outcomes. In reality, like in real curriculum design practice, most academic programs look like a mix of these. And I would say that historically and in graduate nursing education, there are some concepts that we have said, well, we're going to teach all of this right here. And that's what people need to know. Whereas there's others that we've said, oh, we're going to talk about this as we progress through. And so you think about like identification, diagnosis and management of disease conditions or pharmacology, like you have to understand like the pharmacokinetics and pharmacodynamics and those principles, but also application of those to real life caring for patients and how those fit into like disease management. I think as a profession, we thread those a lot more naturally than when we take on concepts like informatics. And I'm not saying people don't thread informatics concepts, but I will say in my experience over the years that that's an area that is more likely to see it like the majority of that content sequestered away in an informatics course. So we have to think about what's gonna be best for the program. And I think that as we're thinking about these pros and cons, there's different considerations we need to take into account. And so, one of those is complexity of the concept. So some concepts are just more complex than others. And that complexity, less complex concepts that may lend themselves better to course level, organization, whereas high complexity concepts, threading might be a better way to give students over time. Remember that progression from earlier in this presentation, introduction of concepts, reinforcement and mastery. So if you have high complexity concepts, essentially threading it over courses gives you an opportunity to move from the introduction piece to spread out more time for that reinforcement before you have to achieve mastery. So you think about resources that you have within your academic program, your institution, what resources are available. This might help to shape what the best way is to organize these ideas. And also your evaluation plan. Like, how do you plan to evaluate students' mastery of these concepts? You know, if you start to think about what that looks like, that piece can have a tremendous bearing on whether you choose to organize concepts within courses or to thread them across. Generally speaking, like from my own experience, I would tell you that when you're planning curriculum, it's a lot easier to think about putting things into a course organization and just boxing things together. But from my own experience and fully acknowledging my own biases, I think that there is a great amount of value in looking at how concepts can be threaded across the curriculum in graduate nursing education. And that's more challenging in a lot of cases, but in the end, when we're thinking about like acquisition of our program outcomes and how we're gonna get there, I think that there's a lot of value in considering how we can thread concepts across the curriculum, because you're not just teaching the concept in an abstract course, it gives you the opportunity to really put it into play and to show students how these factor into real life situations. And so, you think back to like my policy example, if you learn everything you supposedly need to know about policy in one course, and then you never address it again, I think it's hard for graduates to see how that policy information shapes their overall practice. And so, again, with my own biases acknowledged, I think it is worth consideration though, how this looks within our curriculum. So, I mentioned evaluation. And so, how do we evaluate success? Well, this differs at the course level and the program level. The course level, we can say the students achieve the course learning outcomes. Many, we have different ways of how we can measure that in a course, but at the end of the course, have they achieved those outcomes? And the evaluation of outcomes within a course, there should be a clear plan of how that's measured. And you think about the course itself, like nature of the course will help to guide that, maybe simulation is a great way to measure competency or acquisition of skills in a course, but others, maybe it's writing a paper or maybe it's an exam. And so, as you start to think about the course pieces, you wanna know if the students have been successful, and part of that rests on you making the appropriate selection of how you're going to measure and the level, right? We talked about leveling. And so, leveling is, and I think, I go into more depth on leveling in talks about both program level outcomes and also backward course design. So, I won't dive into that too much here, but I think that realizing that, making sure that the outcomes within the course are leveled in a way that they build towards program outcomes and that they build towards the level of knowledge that students need to have upon completion of that course and then further down the road of the program. And that the level of measurement that you're using should be congruent with the level of the outcome. You wouldn't have a course that focuses on performing physical examination skills and probably have just like the way you can tell that the students are successful is playing a game in the course. They're not saying that can't be a helpful learning tool or teaching tool in the classroom, but that's probably not going to be how you determine that they're fully competent in that area. You're going to want to see that they can perform their health assessment skills and that they've mastered that skillset. So, when we talk about success at a program level, I go into this more in a presentation on designing program outcomes, but I will tell you there's a variety of sources that can inform evaluation. And really you should have a robust plan on how you know that you've met the mark and how you've been successful with that. But let's talk about how we know that we've built something that works. So, curriculum mapping is a super helpful tool. And what curriculum mapping is, it's a process that ensures that all of the necessary components are accounted for. So, you started with these program outcomes, you've broken them down into all these subskills and pieces of knowledge and values and behaviors and so on. And you've decided how you're going to organize them into courses or to thread them across a series of courses. And so, how do you have quality control? Curriculum mapping is how we have quality control in this. And so, curriculum mapping gives a clear representation of how the curriculum ties to the standards and program outcomes. And so, what this allows you to do, and there's a lot of different ways that you can go about doing this, but essentially what this allows you to do is it allows you to identify your program outcomes. And if there's any other academic standards that have to be in place for your program, whether that's professionally or from an accreditation standpoint. And then to be able to map to those and to say for each of these components, like these are the courses and the course outcomes within these courses that match or that fit as a piece of this. And there's, like I said, there's a lot of different ways to do this. Probably the most traditional way is with a spreadsheet. You have the access across the top as the course numbers and the access down the side as the program outcomes and other accreditation pieces that you're mapping to. And then you can take the pieces from each of those courses and show what one they map to. But there are more and more, we're seeing curriculum mapping software that's being employed in academic programs. That can be tremendously helpful. There's a lot of setup on the front end, but once it's set up, that can be a great process for making sure that things are up to date and are clearly visualized. Mind maps can be used, any type of mind mapping software. It can be a little bit more complex of a document to read, but also it can be really nice for showing the relationships between them. Sticky notes. I put that on there sort of with a laugh, but sort of in reality, many times over the years when I've been involved in curriculum mapping and starting to look at where things fall in a curriculum, we've done it with sticky notes. And so faculty that teach in their courses, they write their course level outcomes on sticky notes and they have the things around the room on the wall, there'll be like a little heading on the wall about, this one program outcome, this one program outcome, puts their sticky notes up there. Eventually somebody has to put that into some other document. Like you can't just use sticky notes for your program accreditation. And oftentimes our accrediting bodies require that we have a curriculum map as a part of that accreditation, but sticky notes is something that you may very well come across. But what this does is when we have a well-designed curriculum map, it shows us if we have duplication of content, like maybe we're covering the same content in three different courses. It also shows where there's omission to content. Maybe we're missing a piece of this. And so this really helps to inform curriculum change decisions and understanding where we might have spots in our curriculum to improve. And so that pushes us further down the road here to curriculum revision. And really curriculum revision is kind of its own process to master. But in order to close the loop here, I felt that we should at least cover some basics. And so there's a lot of factors that can prompt revision. Like I mentioned with the curriculum mapping, looking at overlap or omissions of content in the curriculum should prompt us to say, how are we gonna remedy this? How are we gonna change what we have in place here and eliminate duplication if it's not necessary. Sometimes duplication is necessary, but if it's not eliminating that and also filling the void that maybe we've left in our planning process in areas that have been omitted. Student outcomes. So when we look back at our students and saying our students, when our students get to graduation, are they successfully achieving those program outcomes? If the answer is no, they probably need to look internally and say, what do we need to do different? You know, we put all of this together. Our best plans are not getting us where we wanna be. So how do we change this to create something that gets students to where they need to be at the time that they finish their academic program? You know, there's a number of external factors. So changes to guiding documents or expectations for our profession. Organizational change within our organization. Maybe if there's significant change, it changes some of the philosophical underpinnings for how we do this, you know? And so we have to consider how organizational change can affect our curriculum. Stakeholder input. So when our partner organizations and our community organizations tell us like, hey, your graduates really need to be able to do this and they can't right now. Makes us say, maybe we need to go back and relook at our curriculum. The scope of curriculum revision is gonna vary depending on your needs. And so I think it's important. Like when we talk about curriculum revision, it doesn't necessarily need to be like, hey, we're gonna scrap the whole thing and start over. Sometimes that might be the case, but more likely it's gonna be much more precise and surgical than that. And identifying what the problem is and addressing that problem specifically, and then making sure that it fits in from like a mapping perspective, that it fits in and is congruent with the rest of everything that's happening, and then evaluating that success as you move forward. Finally, it's important to know process for curriculum revision should be a strategic process. And that strategic process is gonna vary based on a number of factors within institutions, but that strategic process needs to be in place. This can't be something that we just approach and say, well, I think I'll do a little of this and a little of that and we'll be okay. There needs to be much more of a strategic plan in place before you approach the revision process to make sure that nothing is missed. You hate to put the time and effort into revising your curriculum only to realize that you omitted new things and you're gonna have to go right back to that process all over again. So here's some references that have helped shape this presentation and I hope this was beneficial for you. Thank you.

program outcomes

course outcomes

curriculum mapping

academic settings

evaluation methods

curriculum revision