Assessment In Support Departments

Institutional effectiveness is planning, assessing, and evaluating to determine the achievement of an institution’s mission. All departments or units of the University (academic, student and academic support, and administrative) participate in the institutional effectiveness processes through annual outcomes assessment planning and reporting.

This page provides instructions and resources for those who have been assigned responsibility for coordinating assessment planning and reporting for their units. If you have questions or can’t find what you are looking for, please contact Dr. Steve Atkins, Director of Institutional Effectiveness, at atkins_s@calu.edu.

Getting Started with Administrative and Educational Support Unit Assessment

Getting Started with Assessment

Outcomes assessment involves systematically collecting and analyzing evidence to determine how well the unit is accomplishing its intended purposes and – most importantly – using the findings to improve performance. Outcomes assessment also measures the quality and impact of the unit’s services to its stakeholders or in fulfilling the University’s mission.

How it Works – The Process

The assessment process involves

Establishing expected outcomes (Expected outcomes are specific statements about what should occur as a result of the core services or functions your unit performs).

Measuring, observing and documenting the degree to which outcomes are achieved
Findings ways to improve based on your measures and observations
Implementing improvement plans and assessing the effectiveness of changes

Useful Categories for Administrative and Service Outcomes:

Behavioral outcomes: Gains you want those you serve to make. What can someone do after interacting with your unit?
Process statements: Accomplishments of your unit’s functions

Level or volume of activity
Efficiency with which you conduct the processes
Compliance with external standards or regulations

Satisfaction statements: Client satisfaction with your unit’s processes or services

A detailed guide to outcomes assessment planning and reporting for administrative and educational support units can be found here.

Assessment loop.

The Annual Assessment Plan:

At the beginning of the fiscal year, the unit members decide and document what outcomes and assessment methods will be used. Each plan contains

the Unit’s Mission Statement
specific outcomes to be assessed during the next academic year (no more than 2-5 per unit/department). Include outcomes of any innovations, new projects and initiatives, and any local, state or national efforts in which they participate
criterion for Success (e.g. 80% of students will express satisfaction…) for each outcome
assessment measure for each outcome (e.g. survey, rubric, written records, archival data). Results will be used to make suggested improvements in programs and services

The Annual Results Report:

After reviewing the assessment data, unit members meet to decide on plans for the next year based on the outcomes of the previous year and the Strategic Plan. The Results report contains

brief results of each method and the extent to which the outcome or goal was achieved
a summary or conclusions regarding strengths or opportunities for improvement based on the results
actions that will be taken as a result of the data

The Assessment Process

Step 1: Developing a unit level mission statement that ensures alignment with the University’s mission and strategic goals.

The unit’s mission statement is a broad statement about its overarching purpose. The mission should: 1) align with the University’s mission statement and strategic goals; 2) reflect the administrative or support unit’s contribution to the institution and its stakeholders (i.e. students, faculty, staff, alumni, etc.); 3) describe the unit’s distinct purpose, primary functions, and activities; 4) identify the stakeholders; and 5) be clear yet briefly defined.

The Mission Statement should . . .

Be clear and concise
Be distinctive and specific to the unit
State the purpose of the unit
Indicate the unit’s primary activities
Identify/acknowledge the stakeholders
Identify any clarifying statements that are specific to the unit
Ensure congruence with the college mission (See PennWest California mission statement below)

PennWest California mission statement: “The mission of PennWest California is to provide a high-quality, student-centered education that prepares an increasingly diverse community of lifelong learners to contribute responsibly and creatively to the regional, national and global society, while serving as a resource to advance the region's cultural, social and economic

Step 2: Setting intended outcomes (goals/objectives)

Administrative and student support outcomes align with unit mission statements and answer how the University - including students, faculty and staff - benefits from utilizing the services or functions provided by these units. Exhibit 1 contains examples of unit outcomes aligned with the five goals in the PennWest California Strategic Plan. Outcomes are not only indicative of efficiency and quality of service but also focus on what your target audience knows, can do, or values because of interacting with your staff. These outcomes should:

Focus attention on what matters the most: key functions, services, processes
Address improvements in: performance, efficiency, learning, skill-level of unit members and users, effectiveness, quality, customer service, cost, opportunities for the University
Include at least one outcome that refers to customer satisfaction where applicable
Be distinctive to your unit
Be actionable: you can take action as a result of the outcome
Be aligned to a goal in the 2015-2020 PennWest California Strategic Plan (the established Institutional Effectiveness-Outcomes Assessment planning and reporting forms will help ensure that outcomes are related to the PennWest California Mission, Goals and Strategic Plan)
Be S.M.A.R.T. --
- Specific – highlights unit’s unique contribution to the University
- Measurable – quantifiable to be able to determine extent that outcome was met
- Attainable – outcome can be achieved with resources available
- Relevant – shows activity directly linked to the key processes and services of unit
- Timely – can be done within a set time frame

Useful Categories for Administrative and Service Outcomes:

Behavioral outcomes: Gains you want those you serve to make. What can someone do after interacting with your unit?
Process statements: Accomplishments of your unit’s functions
- Level or volume of activity
- Efficiency with which you conduct the processes
- Compliance with external standards or regulations
Satisfaction statements: Client satisfaction with your unit’s processes or services

Use Action Verbs for Writing Outcomes: Use action verbs such as increase, decrease, reduce, minimize, maximize, complete, promote, design, develop, offer, conduct, deploy, implement, expand, etc.

…will increase or reduce…
…will be satisfied with…
…will design
…will comply with…
…will apply
…will complete a process effectively…

…will enhance or improve
…will complete a project / task by end of…
…will assist students with…
…students will be able to…

EXHIBIT 1: Samples of Unit Outcomes Aligned to PennWest California Strategic Goals

Strategic Goal 1: Enhance the academic excellence and experience of our students

Teaching and Learning Center: Increase faculty utilizing the services of the TLC by 15%
Teaching and Learning Center: 95% of faculty, administration and staff will be satisfied with the quality of service and information supplied by the TLC
Faculty Professional Development Center: First-year and senior participation rates in High-Impact Educational Practice (HIP) area (learning communities, service learning, internships, undergraduate research, and study abroad) will meet or exceed national and regional NSSE Engagement Indicators scores
Career and Professional Development Center: Increase percent of students completing internships and co-ops by 10% annually
The Office of Grants and Development: Increase faculty awareness of funding opportunities by knowing and disseminating sponsor funding sources and

Strategic Goal 2: Operate with sound and efficient fiscal and governance practices

Finance and Accounting: Increase level of employee satisfaction with processing of travel reimbursement claims by 10%
Alumni Relations: Increase fundraising total by 2% annually
Finance and Accounting: A minimum of ten major solicitations will be requested for the major campaign (log/count of major solicitations)
PennWest California Foundation: Increase auxiliary income as a % of revenue
Admissions: Increase the yield rate by 15%
Financial Aid: Decrease time from application receipt to disbursement by X time
Food Services: Incorporate sustainable economically and operationally feasible practices
Alumni Office: Increase total private gift support

Strategic Goal 3: Create a transformative learning and working

Career and Professional Development Center: Students will demonstrate knowledge of workforce and professional careers including possibilities for employment, how to search for jobs, how to prepare resumes, and professional expectations
Office of Diversity, Equity and Inclusion: 50% increase programming for student support and engagement through new Jennie Adams Carter Center for Diversity, Equity and Inclusion

Counseling Center: An increase in support for students and others experiencing bias
Office of Academic Success: Implementation of the University-wide climate survey

Strategic Goal 4: Continue to promote/provide public service to the Commonwealth, the region, the nation, and the world through outreach initiatives

Alumni Relations: 15% increase in alumni participation in all University activities interactions

Faculty Professional Development Center: Increase faculty and staff participation in regional boards and advisory groups by 25% by 2020

Marketing: Increase the number of unique views of the University’s news (on all campus websites and social media platforms) by at least 5%

Admissions: Respond to emails submitted to admissions in 1 business day or less as a means to improving service, recruitment, and ultimately enrollment outcomes

Alumni Office: Increase satisfaction scores on the annual Alumni Opinion Survey

Strategic Goal 5: To continue to enhance the quality of student life

Office of Diversity, Equity and Inclusion: All student-athletes will receive information on the programming provided within the Office of Multicultural Affairs

Athletics: Offer five new intramural sport seasons by utilizing outdoor facilities created by campus master plan
Career and Professional Development Center: 20% expansion in opportunities for civic engagement and service learning as measured in OrgSync
Office of Student Affairs: The institution will meet or exceed NSSE Engagement Indicators scores for student perception of safety and belonging
Office of Student Affairs: Meet or exceed NSSE Engagement Indicators in High- Impact Advising Practice as evidenced by student responses on survey items
Office of Student Affairs: 70% of first-year students will develop an educational plan during their first semester
Athletics: Students will demonstrate an ability to successfully balance priorities through engagement in athletics, community service and personal development
Health Services: 25% increase in student participation in wellness and prevention informational activities planned and implemented each semester

Step 3: Selecting and developing assessment measures

Once you have defined 2-3 outcomes, the next step is to choose an assessment measure. That is, identify the sources of evidence you will use to determine whether you are achieving the expected outcome(s). Have an assessment method for each outcome. The assessment method must be data-driven. (Ex: frequency of major course offerings, students enrolled, student surveys, exit surveys, survey students who dropped from program, efficiency measures, demand, satisfaction surveys, etc.). When possible, you should use multiple measures. A composite of results can yield a more realistic picture of your unit’s performance. You can create “home grown” surveys (referred to as a “local survey”) internally by using local expertise. Exhibit 2 presents an example of a locally constructed user survey for internships. You can also use published instruments (NSSE, First Destination, Student Satisfaction Inventory (SSI), etc.) Direct/objective and indirect/subjective (interviews, focus groups) measures are options. You should use at least one direct/objective measurement method – frequency counts, for example, as a measurement method. Appendix D (Inventory of Measurement Methods) provides a wealth of sample assessment measurement across typical non-academic departments.

EXHIBIT 2 Example of a Local Survey

This survey could be used by the Career and Professional Development Center. It would measure the following intended outcomes for the unit (from Exhibit 1):

Students will demonstrate knowledge of workforce and professional careers
Increase percent of students completing internships and co-ops by 10% annually
20% expansion in opportunities for civic engagement and service learning

Have you completed an internship? Have you completed a co-op?
How well did you achieve each of the following learning outcomes: (subjective measure) (Use scale such as “strongly agree, agree, neither agree or disagree, disagree, strongly disagree”)
1. I am more knowledgeable of workforce careers
2. I am more knowledgeable of professional careers
3. I am more knowledgeable about how to search for jobs
4. I am more knowledgeable about how to a prepare resume

Step 4: Identifying at what level (criteria) the goal is to be achieved (expected results)

Sample Criteria or Targets

Criterion for a comprehensive exam: At least 90% of students earn a 7 or better (on a 10-point scale) on each of the following dimensions of the comprehensive exam: basic knowledge of _____; application of knowledge to a given problem _____.

Criterion for a national exam: The average ____ score of all students who take the national _____ examination, administered twice each year, will equal or exceed the national average for the _____ section.

Criterion for an employer survey: Based on an employer survey, which is administered once every two years, at least 80% of the employers will be satisfied with the knowledge of ethics and conduct of our student interns.

Criterion for a course evaluation: At least 90% of our students who complete the ____ course evaluation will report that the course was “beneficial” or “very beneficial” in enhancing understanding of self and others and the ability to work with others.

Sample Criteria and Targets adapted from: College of Charleston Assessment Guide:

http://oiep.cofc.edu/documents/assessment-documents/assessment-guide-5-8-14.pdf

Step 5: Collecting and analyzing the data

After collecting the data, the assessment team reconvenes to analyze and discuss the data. The team should determine by the raw data to what extent the outcome was met and what further action is needed. A thorough analysis of the data should lead to a candid evaluation of program strengths and opportunities for improvement. Questions to consider in conducting the analysis, include:

What did you learn from your results (strengths and opportunities for improvement)?
What part of the objective was met/not met?
What does the data indicate about the quality of services provided?
Why was the target, or why wasn’t the target, achieved?
What will require continued attention?
Why did students achieve X but not Y?
What does the data indicate about the satisfaction of the client?
What relationships exist among the results?
What kinds of students/customers are least likely to complete the program?
Who/what was positively impacted and how?
What was the most valuable thing learned from the assessment results?
Was the assessment tool sufficient or does it need revision?

*Note: Analyzing the data generally requires descriptive statistics. However, when comparisons are made between groups or within groups (between two time points, for example) different statistics may be needed. If you need direction with this task, ask the Office of Institutional Effectiveness or faculty with backgrounds in education or social science research methods for referrals to people with expertise in inferential statistics.

Step 6: Using the result for improvement (closing the loop)

Assessment is about evidence-based planning and decision making rather than anecdotes, gut instinct, or an appeal to experience. Thus, after completing an analysis of the data and identifying strengths and opportunities for improvement, an action plan is developed for outcomes. Areas where outcome assessment was below the expected levels of achievement should have an action plan that identifies improvement strategies and needed change. The action plan is focused on proposing ways to improve services and should be feasible relative to available resources and time.

Assessment results should be used to inform decision making concerning unit operations, resource allocation, and budget planning. The previous assessment steps, from defining outcomes to collecting data, are of minimal value unless the results of analyzing the data are used for taking action to improve. Defining an action plan is often referred to as “closing the loop” and should involve staff members who have discussed the extent to which assessment results met unit expectations and criteria.

Summarize findings, highlight surprising or important findings, and cover anything that happened within the unit that may not have fit nicely under an objective but is indicative of unit performance. The summary does not have to be long. Brag about the things going well, and don’t forget to highlight budget implications based on findings for the year. If you were successful in achieving your criterion, hypothesize as to why that happened (for example, was it adequate staff? Good staff training?) and propose a plan to test that hypothesis so you can ensure you’ll continue to meet criterion in future measures of it. Consider increasing the criterion to pursue continuous quality improvement. The template to report your findings and use of results is in Appendix B.

Step 7: Follow-up process on the implementation and impact of prior-year action plans

This final step is often the most neglected aspect of assessment planning. As part of the follow- up process in effect, action plans are to be reviewed annually until they are fully implemented, and the results of implementation is to be assessed and reported to the University Strategic Assessment Committee (USAC). The method for determining whether the change has led to improved services or learning, may be achieved by simply repeating the previous assessment plan. Thus, the assessment process is cyclical and ongoing in nature as it moves through the seven steps of the outcome assessment model.