Hello and welcome. My name is Dan Crawford and this lesson today will cover designing program outcomes. So upon completion of this presentation, you should be able to describe the process for designing learning outcomes, to apply concepts of writing high quality program outcomes, and to identify internal and external influences that affect program outcome design. So when we think about program outcomes and how they relate to curriculum design, the program outcome itself is something that is very beneficial because it ensures that the desired results are achieved. Having program outcomes, if you have a well-designed set of program outcomes, can help to avoid gaps in learning, ensure that learners really have a robust and comprehensive skillset upon completion of the program. Program outcomes provide structure to the overall design of the educational program. These give you the opportunity to really promote excellence. A well-designed set of program outcomes can ensure that graduates from the program have excellent outcomes and meet the standards that would be expected from the institution as well as the profession. And finally, program outcomes are important because these help to direct what should be taught in the academic program. So start out with defining program outcomes. Program learning outcomes can be defined as skills, competencies, and big ideas that students should be able to articulate or put into action or use after the completion of a degree or certificate. These program outcomes really focus on what students will learn as a result of their experiences throughout the program rather than what students will be taught in individual courses. And I think that's an important distinction we have to consider. The program outcomes are not just a list of what the students are going to learn across their journey, but the program outcomes are a list of the broad skill set that the graduate will have upon completion of the program. If you think about this, you can think about new learners coming into an academic program versus the end product in that academic program. The program outcomes really set the benchmark or a goal for where students should expect to be and where faculty should expect those students to be upon completion of the program. This gives clarity to the path that those students will take and sets a unified vision of expectations. So when we think about program outcomes, there are a number of basic underpinnings that we have to consider. Before you ever put the pen to the paper in designing your program outcomes, there are certain aspects that really set the foundation for the overall design of the curriculum. And these, once these elements are in place, these begin to shape the landscape for which you will be designing your program outcomes. Whether we intentionally recognize this or not, whether we talk about this or not, the reality is that all well-designed curricula are underpinned by philosophical assumptions and ideas, theoretical underpinnings, conceptual frameworks, and some element of unspoken and or unconscious assumptions. Now, from a philosophy perspective, it's a best practice when designing a program to have a common understanding of what philosophical underpinnings will guide your development of the curriculum and your program outcomes. There's a vast array of philosophies that could be considered, and part of this will be determined by the nature of your academic program, your institution, faculty values, and so on. And beyond philosophy, when it comes to philosophies, think about the other thing that would fall into this is our understanding of how individuals learn knowledge. And so that's an important piece that we also think about in the big picture is what is our understanding and our belief of how individuals learn new things? So theories, there's a number of theories also from nursing theories and grand theories and middle range theories, educational theories. So it's important to have conversations around those theoretical underpinnings. Conceptual frameworks, a well-designed curriculum should have an identified conceptual framework that guides the process of the overall design of the curriculum. And again, realizing that there's going to be unspoken or unconscious assumptions that are a part of this process and being aware of those, that those exist as you're going through designing the curriculum and outcomes. So as I alluded to previously, we have to realize that our culture influences our design. So when we think about our organization itself, each organization in each institution is going to have its own set of mission, vision, and value statements. And in those, those should be embodied in the overall curricular design. If you're coming from a large research one public institution versus a small private, maybe religious-affiliated university, there's going to be differences in what those look like. And it's important that those are reflected within the curriculum. That's one of the pieces of what makes your academic program unique to your student body. Our professional values, that's as nurses and within our own subsets of nursing knowledge, we have unique professional values. Broadly speaking, these might be, you know, including empathy and caring and the nursing process and equity. And you can think about the list of what this looks like, but there are values that guide our profession, and these should also be reflected. If you're thinking about program outcomes, there's this idea of this is what graduates from our program should look like. Our professional values need to be reflected in that. Finally, faculty values. You know, each group of faculty is a unique body of individuals. Each group of faculty come in with their own backgrounds, with their own knowledge, with their own skill sets, and with their own values. And so, you know, that's the beauty of having a diverse faculty group, is that you have a lot of representation of a lot of ideas. And so those faculty values do shape how we understand the world around us, how we understand how people learn, and how we approach teaching and learning. So, you know, we would be remiss if we didn't identify the faculty values help shape the culture in which academic outcomes are designed. So as we're thinking more now about program outcomes and the curriculum design, besides those internal cultural pieces that really shape how we move forward in designing curriculum, there's a number of external influences we also have to consider. You think about guiding documents, and to give you an example, like the American Association of Colleges of Nursing, and their essentials document for nursing education would be one example. Different professional organizations have different guiding documents for various advanced practice nursing disciplines. On those guiding documents, you have accreditation standards, each of our accrediting bodies has specific guidelines that we must meet in our academic programs, and those standards have to be considered when we're thinking about how we're getting from, you know, newly admitted students to what our graduates look like. Our state boards of nursing will have some say, different states have different levels of regulation from an educational perspective, but the state boards of nursing are an important component in understanding how to shape our programs and what standards need to be met. The evidence, right, that what do we know works from teaching, what do we know topically and skill-wise, what do we know our graduates need to embody? There's evidence that, you know, is published on an ongoing basis that gives us guidance on what that should look like. Our local healthcare organizations, really, you know, if we're being good stewards, we should understand what healthcare organizations want to see in our graduates. Our graduates are going to leave our institutions and are going to go practice in these healthcare institutions. So these local healthcare organizations are important stakeholders that we have to really consider when we're looking at our program outcomes and designing what our graduates need to look like, knowledge, skills, ideas, and so on. It's important that we know what employers want to see in our graduates. And so, you know, if we think about that, you know, there's lots of different ways you can achieve that in the design piece that probably spans beyond the scope of this presentation, but just like a few ideas. I mean, having individuals from local healthcare organizations affiliated with the College of Nursing or the college where this is taking place, the Educational Degree Program is one idea, setting up meetings with stakeholders from local organizations and seeking their input prior to finalizing your program outcomes is another potential opportunity. So as I said, we won't dive into that too much, but there are ways to go about that that I think are really valuable and help ensure that, you know, we're producing graduates that are employable and that meet the needs of the communities in which they serve. If you think about the needs of the population as a whole and what kind of demands there are for healthcare services, you know, I think that there's a lot of different ways that you can slice this up. But the reality is, is that, you know, as nurses, we have a social contract with society and within the communities in which we serve. And so it's important that our graduates from our academic programs can meet the needs of the population and can help to remedy some of the challenges we see with access to care in communities across the nation, whether that's rural areas and the challenges with access to care in rural areas, or if it's in urban areas. I mean, really, those challenges might be unique based on geographic location, but those challenges exist in some capacity, regardless of what area you're in. Beyond this, there's so many more potential external influences, and in some ways, this is going to be tied to the type of institution you're in and back into those internal factors as well. You know, if you're thinking about your mission, vision, values, that may also reflect that there's other external stakeholders that need to be included in the conversation. So when we get down to this, you know, writing meaningful program outcomes, you know, all of this background information, you know, what has to be there in place and has to be considered in designing curriculum is important, but none of that gets your outcomes on the paper, right? So let's talk about some meaningful program outcome writing strategies here. So we think about program outcomes and what they are, there's this set of skills, ideas, knowledge, and values that our students will embody upon graduation. First of all, when we're writing program outcomes, they need to be attainable and appropriately level. So our goals have to be attainable, and maybe that seems really basic, but, you know, we can have these incredibly lofty aspirations, and that's not a bad thing, but the reality is that if students can't attain our goals through their academic program, then we've missed the mark with designing our program outcomes, because our program outcomes really should be that clear picture of what the finish line looks like. So attainable goals and appropriately level, when you think about the type of academic program, we need to think about the nature of the program, the level of the degree and the level of the learning that needs to take place, and we need to make sure that our program outcomes are leveled in a way that they reflect the level of knowledge that graduates need to have upon completion of their program. And so an example, you think about the level of knowledge that a student would need graduating from like a BSN program is very different than the level of knowledge that a student needs graduating from an advanced practice nursing program, whether that's at the master's or doctoral level. And so the level that outcomes are written at, maybe both, you have to learn pathophysiology or pharmacology, but the level you need to understand and be able to apply those at is vastly different based on degree type, and so we need to make sure that our outcomes for each individual program are leveled in a way that's appropriate for that degree program. Now, these program outcomes need to be clear and measurable. They need to be clear because, again, the student, faculty, and external forces all need to understand clearly what students are accomplishing as a part of their degree program. But the other piece of this is the measurability. And so if we have goals that aren't measurable, how do we ever know that a student has acquired those or has reached that mark? So measurability is an important factor, and the way that we write our outcomes needs to include that measurable piece. We need to be describing these outcomes in a way that we can sit and look at our graduates as they near completion of their academic program and say, yes, they have achieved this, or yes, this is reflected in their practice. Well-written program outcomes should have definable subskills, and I've recorded other presentations on bridging program outcomes to course-level outcomes and backwards course design, and this is covered a little bit more in depth there, so I'll just mention it here. But really, program outcomes are what are going to shape your course-level outcomes. And so a well-written program outcome shouldn't just be a one-time teach, learn, and observe skill. Program outcomes need to be broader and be able to divide it into sub-ideas and subskills that can then be embedded in the curriculum across the academic program. Finally, meaningful program outcomes should be accompanied by an evaluation plan. You should understand how you're going to measure success for these program outcomes. And prior to moving further down the road with designing courses and content and curriculum, that evaluation plan on how you're going to measure success with acquisition of the program outcomes needs to be in place and needs to accompany the well-written program outcomes. And oftentimes, it can help to think about these simultaneously and think about these are our outcomes, and this is how we know it's successful. And when you can marry those two, then you can move forward with looking more in depth at the curricular design piece. So when we think about evaluation and evaluating whether or not our graduates are achieving program outcomes, I want you to think about that there's a broad range of how you can determine this. So I think probably the most obvious is student outcomes, right? Student grades, student performance and skills, student's academic work. That's probably the easiest piece to conceptualize here. But I think there's more than this that we can consider when it comes to evaluation. So when we're developing that evaluation plan piece, think about all of these. Think about teaching evaluations. Now, there are longstanding discussions about teaching evaluations and what the value is of teaching evaluations, whether or not that they're entirely reliable reflections of the faculty or not. Outside of those conversations, though, it's important to realize that there's some value in teaching evaluations. And so seeking those and considering those as a part of your evaluation of your academic program and evaluation of your program outcomes, whether or not your outcomes are well designed, can in some ways be reflected in teaching evaluations. Alumni survey. So when we send out surveys to people who have graduated from our program, think like one, two, three years post-graduation and say, did the skills that you had prepare you for the work that you're doing? It's important for us to know that. And this can help to shed light on some of our blind spots. And so having that as a piece of the evaluation oftentimes is very beneficial. Internal review. So reviewing your curriculum internally. And each institution has its own process for internal review. But to my knowledge, all institutions have some process for this. Every institution that I've been affiliated with over the years has had a very clear internal review process. Industry feedback. So seeking feedback, again, from those stakeholders. But the same ones about what do you need in our graduates, the follow-up conversation to that is, how are our graduates meeting your needs? So making sure that we're seeking that and having that feedback piece on understanding post-graduation how our defined outcomes and our graduates are performing. And our accreditation process. Depending on who your accrediting institution is, that process is going to look a little bit different. But we all have to go through re-accreditation process in some capacity or another. And so that re-accreditation process is an opportunity, again, to evaluate our program outcomes and the design of our curriculum. Again, institution-specific, there's a variety of other strategies that could be considered. And so I wouldn't say that you're tied specifically to this list alone. But this is a good body of ideas of ways that you can begin to conceptualize your evaluation plan. Here's references and resources for this presentation. And I thank you for your time. And I hope this has been official.

program outcomes

measurable goals

curriculum design

evaluation methods

accreditation standards

industry feedback