Assessment - Institutional Effectiveness

Planned 2023-24 Academic Year 

Reporting Timeline:

• Fall semester AoL Committee members available to coach; triad check-ins with departments
• Departments work on reports late fall or over break
• Reports due first week of Winter term – departments submit reports to AoL Committee
• Triads receive and review reports early Winter term
• AoL committee meets February
• Finalized feedback reports/meetings to respond to departments by Monday after spring break

AoL Committee activities:

• Norming Exercise - November 2023
• Triad Orientation Meeting - December 2023
• Review assigned programs with your triad

The annual update is collected through a Google Form. 

The detailed report is due from programs listed under the Cohorts tab. This pdf provides instructions.

What is due, and when is it due?

All reports due first week of Winter term 

Detailed Reports

Detailed reports are due from programs in Cohort 1. Programs will submit documentation and discussion of the work that they’ve been doing – including evidence of student learning if that has been collected, analyzed, and used to make change – based on the NMU Assessment Planning Guide which is on this webpage and has been shared with assessment liaisons. Academic department heads have received instructions. 

Departments at NMU are in various stages assessing student learning. Not all departments will have completed the work to report on all the steps of a completed assessment project in any given report. Departments should present to the AoL Committee documentation of the work that has been completed and discussion of work that is planned. Planned work should present a reasonable timeline and work plan to complete a full turn through the assessment helix, ending with the ability to report on a complete assessment project. 

If anyone needs help managing this work as an ongoing project, we are available. 

Annual Update Reports

An annual progress report is due years the program does not submit a detailed report. The Google form asks the department to briefly answer three questions:

1. Briefly, what assessment of learning work was completed in the past year?
2. Briefly, what assessment of learning work is planned for the next academic year?
3. Optional: What, if any, assistance or support would be useful as you work on assessing student learning?

Or if an accredited program is opting out of NMU AoL work, that can option can be taken by entering the department attestation in the form and skipping the above three questions and the detailed report. 

Responsibilities:

Each liaison completes the NMU Assessment Academy Training. In order for assessment of learning work on campus to be effective, faculty-led peer support with shared understanding of flexible use of the basic assessment foundations is essential. The Academy prepares faculty to engage with peers constructively, both in being the assessment resource in their department and in reviewing cross-campus assessment reports.

This can be done one of 2 ways:

• Self-Paced Academy (in EduCat): Contact Daniel Cullen (dcullen@nmu.edu) for self-registration information.
• In-Person Academy: Two 90-minute workshop sessions

Dates:

AoL Committee Work

One major responsibility as a liaison is to, as a triad member, review programs’ reports to see if they have clearly identified, communicated program learning outcomes (PLOs). AoL Committee will do this for all programs and degree types within every major; each triad will be assigned several programs to review.

The AoL Committee will meet and review the reporting schedule for programs. Roughly one-third of the programs will do a full, comprehensive report each year, while the remainder will submit updates on a yearly rotation.

Lastly, once reports are being processed and submitted, if asked, we will coach programs to help them achieve their programmatic assessment goals.

Departmental Resource

The other major responsibility as liaison is to be a resource for your department. Through the Academy and through work on the AoL Committee learning how other units are approaching assessment of student learning, you will have more expertise than others and can be a point of information and advice for assessment work. 

You will be the leader pulling together assessment efforts as faculty collaborate in understanding student learning and developing interventions to improve your students’ experience. You will have relative expertise as the person in your department working most closely with assessment – that doesn’t mean you have to feel like you know everything. The AoL Committee, the NMU Director of Institutional Accreditation and Assessment, the Center for Teaching and Learning, and other resources are at your disposal. 

In order for assessment to be effective, it can’t be something one person does in isolation. Your responsibilities within your department are to know the procedures and expectations or the Committee and communicate them to your department. You are not the person responsible for creating your department’s report(s), rather, a connection between your department and the AoL committee. It is up to your department to determine how the report is constructed and by whom.

Expectations:

Each Liaison is expected to:

1. Complete the Assessment Academy Training
2. Attend required meetings

Planned 2023-24 Academic Year 

Reporting Timeline:

• Fall semester AoL Committee members available to coach; triad check-ins with departments
• Departments work on reports late fall or over break
• Reports due first week of Winter term – departments submit reports to AoL Committee
• Triads receive and review reports early Winter term
• AoL committee meets February
• Finalized feedback reports/meetings to respond to departments by Monday after spring break

AoL Committee activities:

• Norming Exercise - November 2023
• Triad Orientation Meeting - December 2023
• Review assigned programs with your triad

Training:

For help sorting out the work, review the presentation here (clicking this link will open a pdf file, which is downloadable, in a new tab). Note that some slides have notations that provide additional explanation which might be useful. To see them, toggle comments on by clicking the dialog icon in the pdf upper right corner.

Assessment of Learning Committee Faculty Chair: Lex van Blommestein, Theatre & Dance

Departments should approach assessment of student learning as an opportunity to do work that will be useful to strengthen their programs. Approaches will vary since contexts and needs differ. A general template is available to guide the work. Click the following link to download the guide as a Word document.

Assessment Plan Template and Guide (Link will download an MS Word document.)

We have long talked about assessment of learning as a cycle. We start by articulating what we intend for our students to get out of our program. As we deliver the program, we ask research questions about student learning. We gather data that directly addresses what we wish to better understand. We use the data to develop plans to improve the program. We return to articulating our intended student learning outcomes and ask questions about our updated program. So, like a cycle, after step n we return to step 1. But we aren't running in a circle, getting nowhere. Each time we return to step 1, we have learned, changed, and moved forward. Thus, the helix, rather than the cycle, is a better metaphor for this work. (The "assessment helix" concept is based on Jeremy Penn's work.)

Program Learning Outcomes

Writing clear PLOs is an important first step. This brief handout has tips (link will download an MS Word document). We suggest spending some time working on articulating the purpose of your program, but if this work is so tedious it gets in the way of real progress, don't sweat the details. Issues like double-barreled statements will naturally be addressed as you engage in the iterative work of developing a curriculum map, moving back and forth between refining PLOs and mapping expected outcomes to program requirements. 

See the CTL Resource Library webpage for a handout on Bloom's Taxonomy and other "Learning Objectives (Bloom's Taxonomy)" resources under the "INSTRUCTIONAL DESIGN" heading. 

Curriculum Maps

Standard Templates:

Spreadsheet template. Use a spreadsheet to create a simple matrix with outcomes in columns and courses/requirements as rows to map how student learning outcomes are scaffolded across the program. Download as an MS Excel file. 

Word template. Use a table to create simple matrix with outcomes in columns and courses/requirements as rows to map how student learning outcomes are scaffolded across the program. Download as an MS Word file. 

NMU is promoting building program maps, a tool that will help students (and others) see the standard or typical pathway through a program. A curriculum map that is organized by term can be helpful in meeting multiple objectives. Download as an MS Excel file. 

Additional Options:

Spreadsheet template set up to chart a major with different tracks or concentrations. Download as an MS Excel file. 

MS Word template with courses as columns and outcomes as rows. Download as an MS Word document. 

Spreadsheet template with courses as columns and outcomes as rows and concentrations broken out. Download as an MS Excel file. 

Simple MS Word template. Download as an MS Word document. 

Example template for a program with many intended learning outcomes and few required courses. Download as an MS Word document. 

Scholarship:

Read about the value of mapping your curriculum: The Aspirational Curriculum Map: A Diagnostic Model for Action-Oriented Program Review

Assessment Plan Template

Departments should approach assessment of student learning as an opportunity to do work that will be useful to strengthen their programs. Approaches will vary since contexts and needs differ. A general template is available to guide the work. Click the following link to download the guide as a Word document.

Assessment Plan Template and Guide (Link will download an MS Word document.)

(Note: This essential item is also available on the main page under its own heading. The document is the same both there and here; it is available in multiple places for convenience.)

Degree Maps

Student Success at NMU is working with departments to develop degree maps for all undergraduate programs. Although the purposes and products are different, the related work might be done in concert with curriculum mapping. Student Success provided a templates document. Please make a copy (File -> Make a Copy) or download as an Excel document so you can make edits on your own version.

Each year all the programs in 1 of the following three cohorts submit in-depth reports to AoL Committee, and other cohorts submit brief updates to Academic Affairs.

NMU is accredited by the Higher Learning Commission (HLC) and must meet standards all member institutions commit to meeting. That institutions regularly assess student learning and act on the evidence is an obligation to stay in good accreditation standing because this is good practice. NMU should engage in healthy assessment practices to maintain strong instruction. Then, compliance with accreditation standards will follow; compliance should not drive or motivate the work, but should be a consequence of our commitment to maintaining high quality.

HLC's Assumed Practices briefly address assessment of student learning:

"Faculty participate substantially in... analysis of data and appropriate action on assessment of student learning and program completion." (B.2.d.iv.)

"Institutional data on assessment of student learning are accurate and address the full range of students who enroll." (C.6.)

HLC's expectations regarding assessment of student learning are explained in the Criteria for Accreditation, "the standards of quality by which HLC determines whether an institution merits accreditation or reaffirmation of accreditation."

4.B. The institution engages in ongoing assessment of student learning as part of its commitment to the educational outcomes of its students.

1. The institution has effective processes for assessment of student learning and for achievement of learning goals in academic and cocurricular offerings.
2. The institution uses the information gained from assessment to improve student learning.
3. The institution’s processes and methodologies to assess student learning reflect good practice, including the substantial participation of faculty, instructional and other relevant staff members. 

In order to maintain our good standing with HLC, NMU must demonstrate to the agency and the members of the HLC Peer Corps that assessment of student learning is an ongoing effective process that leads to action. That we assess student learning and that we use assessment evidence to improve student learning is required, but it is up to us to determine how we manage this work.

Another distinct, but related, process HLC expects all accredited institutions to engage in is regular review of academic programs. At NMU, the Educational Policy Committee (EPC) oversees Academic Program Review (APR). EPC expects assessment of learning and assessment precursor work (e.g., curriculum maps) to be part of APR reports.  

4.A. The institution ensures the quality of its educational offerings.

1. The institution maintains a practice of regular program reviews and acts upon the findings.

Policy regarding NMU Assessment of Learning expectations and program accreditation:

The following pertains to programs that are fully accredited by an appropriate professional organization recognized by the Council for Higher Education Accreditation. If a program has accreditation that requires assessment of student learning, the department may opt to submit to the AoL Committee accreditation documentation along with a road map to let the Committee identify the elements NMU expects. 

Alternatively, the department may submit a brief attestation that student learning is assessed and documented as part of the programmatic/specialized assessment process. In such cases, departments should be aware that:

• Support from the AoL Committee, assessment liaisons, and/or Institutional Effectiveness remains available.
• Full review by the AoL Committee remains an option. It may be that the department values input from NMU colleagues and/or it may be valued in the programmatic/specialized accreditation process. Departments that wish to use the AoL Committee feedback process to aid their program accreditation process are allowed to and encouraged to do so.
• Academic Program Review requirements should be considered carefully.

If a program is seeking accreditation, the department continues to participate in standard NMU AoL reporting until it can document external review of learning outcomes assessment.

During academic year 2020-21, due to the Covid-19 pandemic, regular programmatic assessment review was largely temporarily suspended by giving departments the option of not reporting, while the Assessment of Learning (AoL) Committee took time to review NMU’s AoL processes.  A steering group was formed including the AoL Faculty Chair (then Shaun Thunell) and IRPA (now Institutional Effectiveness) staff.  The steering group established three workgroups with AoL Committee members and other campus faculty, staff, and administrators who focused on key areas for review. Workgroups and members are listed below.

Workgroups developed recommendations around key facets of the AoL process and quality with the overarching focus being on ways to increase value and return on investment. Throughout discussions with academic departments, a common focus emerged where AoL had been primarily thought of as “report submissions.”  In order to create value and positively improve the culture, a shift needs to occur focusing on continuous improvement work, which then leads to a submission to the Committee who then provide useful cross-campus peer feedback.

Value

• Focus on departmental continuous improvement efforts rather than reporting 
• Create a database with information about department reports and the outcomes of the review process (e.g., the committee recommended X, the department implemented a change based on X, the program improved by X); utilize results in annual campus-level reporting and to identify in-service training and/or professional development opportunities 

Reduce burden for academic departments

• When possible, align with specialized accreditation expectations and timeline
• Offer flexible submission process
• Departments submit detailed reports every three years, with only simple affirmations that progress is being made due on non-reporting years

Increased process efficiency

• Emphasize faculty work already being done
• Reduce submissions to a rotating three-year cycle to allow departments time to collect evidence of student learning, reflect, and update programs  
• Streamline submission process 

Improved quality in submissions and feedback

• Create consistency and stability in the process
• Build relationships 
• Restore value and confidence for Academic Affairs to increase advocacy and accountability

AoL Workgroups Member List, Academic Year 2020-2021

AoL Faculty Chair

Shaun Thunell

Steering Group

Shaun Thunell

Dan Cullen

Jason Nicholas

Joanne Sved

Triad Review

Matt Jennings

Sarah Jones

Lex van Blommestein

Communication Workgroup

Mitchell Klett

Derek Marr

Judy Puncochar

Process Workgroup

John Centko

Dan Cullen

Ahmed Elnoshokaty

Brad Hamel

Randy Klitzke

Stephan Larson

Michael Letts

Joseph Lubig

Kathryn Newton

Jason Nicholas

Matt Smock

Shaun Thunell

Philip Yangyuoru

Professional Development Workgroup

Stacy Boyer-Davis

Dan Cullen

Jason Nicholas

Alex Stoner

Shaun Thunell

Kristen White

Liz Wuorinen

Because NMU is in the early stages of launching revised assessment of learning procedures, there haven’t been “frequently asked questions” generated yet. However, as questions come to the committee they will be posted here with answers and if at some point any questions emerge as “frequently asked,” those will be highlighted in this section. 

Can someone come to my department (or school, or college, etc.) to talk with faculty about assessment?

Yes. When? What would you like covered? We have met with faculty to talk about the current NMU approach to assessing student learning and documenting that work, and we have helped with workshops devoted to crafting and refining statements articulating program learning outcomes, curriculum mapping, and other related work. Let us know what your needs are and we can work together to develop an effective session. Email AoL@nmu.edu for more information.

May my program submit the form NMU previously required for reporting on assessment of student learning?

Yes. The guiding values of the current approach are flexibility and usefulness. Flexibility means being open to various approaches to reporting, including using the form that was previously required. If that is the tool the program finds most useful to document the work investigating student learning and acting on findings, please feel welcome to continuing to use it. If you would like to simplify the form (for example, include attachments instead of embedding documents), that would be helpful because few if any current AoL Committee members have learned to navigate that tricky Word document. Also, a program submitting that form should not expect it to be returned with notes in three different colors.  (The annual update is collected through a Google Form. The detailed report is due from programs listed under the Cohorts tab; this pdf provides instructions.)

What is due, and when is it due?

All reports are due the first week of Winter term. Detailed reports are due January 2024 from programs in Cohort 1 (see “Reporting Cohorts” on this webpage). Instructions were sent to academic department heads and are posted on this webpage under “Reporting Forms.” Annual update reports are submitted through a Google form (the link has been shared with academic department heads and is posted on this webpage under “Reporting Forms”). 

What will be done with our reports?

Detailed reports will be reviewed by the AoL Committee. Your peers will work in triads to respond to the department, offering their cross-disciplinary perspective to be helpful. Annual update reports will be filed with Academic Affairs who will respond as necessary and will make sure any requested support is offered. Status of what was due and what was submitted will be posted on this webpage. Note also that EPC has established guidelines for completing academic program review (APR), and including documentation of AoL work is an expectation for that process.