Accreditation
Glossary of Assessment Terms
A
The connection between learning objectives, learning activities, and assessment. An aligned course means that the learning objectives, activities, and assessments match up so students learn what is intended and that what they're learning is accurately assessed (Maki, 2004; O'Reilly; Suskie, 2010)
The systematic, reported evaluation of student learning outcomes for demonstrating effectiveness and improving offerings (Harvey, 2004; McTighe & Ferrera, 1998; Morante, 2003; Suskie, 2010).
Academic program assessment consists of a systematic, collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development. It is a continuous process focused on understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance (Angelo, 1995; Palomba & Banta, 1999).
Also see:
The assessment of some units, such as a department, program or entire institution, which is used to satisfy some group of external stakeholders. Stakeholders may include accreditation agencies, state government, or trustees. Results are often compared across similar units, such as other similar programs and are always summative (Leskes, 2002; Maki, 2004; Suskie, 2010).
An example of assessment for accountability would be ABET accreditation in engineering schools, whereby ABET creates a set of standards that must be met in order for an engineering school to receive ABET accreditation status.
Assessment activities designed to feed the results directly, and ideally, immediately, back into revising the course, program, or institution with the goal of improving student learning. Both formative and summative assessment may be used to guide improvements (Leskes, 2002; Suskie, 2010).
Specific tool used to collect information about students' knowledge or performance. For example standardized exam, performance rubric, portfolio of papers or projects, etc. (Rice University).
Also see:
Comprehensive description of how a learning outcome is going to be assessed. It includes descriptions of the assessment instrument, the population to be assessed, and at what point in time, as well as how reliability and validity will be addressed (Rice University).
Also see:
Document that includes the following sections: Mission Statement, Learning Outcome for academic program (or unit outcome for support units), Assessment Method, and Performance Targets (Morante, 2003; Rice University).
Also see:
A document that includes the assessment plan with the addition of the assessment results and the use of assessment results. The results are the actual data stated in terms of the performance criteria. Example: 80% of students scored a 4.5 or higher (N=12); 13% of students scored a 3.4 to 4.4 (N=2); and 7% of students scored 2.5 to 3.4 (N=1) (USF, 2018).
The use of assessment results section is where the interpretation and analysis of the results is stated. It should include specific, actionable "next steps" the program will take to develop or improve on a programmatic level (e.g., curriculum mapping, revisit or revise the assessment method/rubric, revisions to plan of study/curricular offerings, development of new modules/courses, faculty development, etc.) (Maki, 2004; Morante, 2003; Suskie, 2010; USF, 2018).
Also see:
B
An actual measurement of group performance against an established standard at defined points along the path toward the standard. Subsequent measurements of group performance use the benchmarks to measure progress toward achievement (Morante, 2003; New Horizons for Learning; Suskie, 2010)
C
The process of establishing inter-rater reliability. Developed over successive applications of a scoring rubric to student work over an undefined period (i.e. semesters, years) (Maki, 2004).
Capstone Experience/Student Teaching
An activity for graduating seniors that is designed to demonstrate comprehensive learning in the major through some type of product or performance (Morante, 2003; Palomba & Banta, 1999; Suskie, 2010).
Detailed analyses of projects or problems that result in exemplary models. Counseling and School Psychology programs ask students to generate a course of action for dealing with a client based upon a comprehensive profile of that client (Maki, 2004; Morante, 2003; Rice University).
The systematic and on-going study of what and how students are learning in a particular classroom; often designed for individual faculty who wish to improve their teaching of a specific course. Classroom assessment differs from tests and other forms of student assessment in that it is targeted at course improvement, rather than at assigning grades (Morante, 2003; National Teaching and Learning Forum).
The individual's demonstrated capacity to perform, i.e., the possession of knowledge, skills, and personal characteristics needed to satisfy the special demands or requirements of a particular situation (Clark College; DeMars et al., 2002).
A combination of skills, ability and knowledge needed to perform a specific task at a specified criterion. Usually a near-term target of a larger expected outcome. Students must normally learn and/or demonstrate several competencies for each student learning outcome (Clark College; Morante, 2003; Suskie, 2010).
Graphical representation used to reveal how students organize their knowledge about a concept or process. May include concepts, usually represented in enclosed circles or boxes, and relationships between concepts, indicated by a line connecting two concepts (Novak & Cañas, 2006).
Process of evaluating curriculum in relation to intended outcomes to ensure that students are receiving appropriate instruction and to enable the program/department to identify gaps in the curriculum and provide an overview of program accomplishments (Rice University; Suskie, 2010).
D
Performances that show skills students have mastered. Theater students may be assessed on their acting skills demonstrated in theatrical productions (Palomba & Banta, 1999).
Direct Measures of Assessment: (see also Indirect Measures of Assessment): Direct measures of assessment gather evidence about student learning based on student performance that demonstrates the learning itself. They can be value added, related to standards, qualitative or quantitative, embedded or not, using local or external criteria (Eder, 2004; Leskes, 2002; Maki, 2004; Morante, 2003; Suskie, 2010).
Examples of direct measures of student learning include:
- Written assignments
- Presentations
- Portfolios
- Comparisons between pre and post-test scores (this does not have to be the exact same test as long as one is testing the same area of knowledge or skills)
- Class assignments or tests
- Capstone projects, including theses, field studies and papers
- Case studies
- Team/group projects and presentations
- Field supervisor evaluations from internships and field experience
- Service-learning projects
- Ratings by employers of the level of skills and knowledge possessed by our graduates
- Student performance on standardized tests, such as licensure exams or the ETS content tests
E
Embedded Assessment/Course-embedded Assessment
A means of gathering information about student learning that is built into and a natural part of the teaching-learning process. Classroom assignments that are evaluated to assign students a grade are often used for assessment purposes. Embedded assessment can assess individual student performance or aggregate the information to provide information about the course or program; can be formative or summative, quantitative or qualitative (Maki, 2004; Morante, 2003; Suskie, 2010).
Example: As part of a course, expecting each senior to complete a research paper that is graded for content and style, but is also assessed for advanced ability to locate and evaluate web-based information (as part of a college-wide outcome to demonstrate information literacy) (Leskes, 2002).
The use of assessment findings (evidence/data) to judge program effectiveness; used as a basis for making decisions about program changes or improvement (Allen, Noel, Rienzi, & McMillin, 2002).
Standardized tests that may include multiple-choice, true-false, response selection, essays and/or problems (University of Nebraska at Kearney).
- National Exams are usually produced by a professional organization, such as the American Chemical Society, or by a commercial company, such as ETS (Educational Testing Service). National exams are usually nationally normed so there is feedback about the relative accomplishment of participants.
- Local exams are produced by the department or institution using them. Usually careful attention is given to reliability and validity and data are kept to compare current participants with past groups of participants.
F
A planned classroom practice to elicit evidence of learning minute to minute, day by day in the classroom. Purpose: inform teachers of what students know or do not know, help students understand what it is they are ready to learn next, so teachers can adjust their instruction accordingly for each of their students (Goodrich, 2012; Harvey, 2004; Maki, 2004; Suskie, 2010).
Formative assessment refers to the gathering of information or data about student learning during a course or program that is used to guide improvements in teaching and learning. Formative assessment activities are usually low stakes or no-stakes; they do not contribute substantially to the final evaluation or grade of the student or may not even be assessed at the individual student level (Leskes, 2002; Maki, 2004; McTighe & Ferrera, 1998).
For example, posing a question in class and asking for a show of hands in support of different response options would be a formative assessment at the class level. Observing how many students responded incorrectly would be used to guide further teaching (Leskes, 2002).
Also see:
I
Indirect Measures of Assessment
Indirect Measures of Assessment: (see also Direct Measures of Assessment): Uses perceptions, reflections, or secondary evidence to make inferences about student learning (Eder, 2004; Leskes, 2002; Maki, 2004; Morante, 2003; Suskie, 2010).
Examples of indirect measures of student learning include:
Course grades
Surveys of employers
Student self-assessments
Number of alumni admitted to graduate school
Number of students presenting or publishing research
Honors, awards, scholarships, and other forms of public recognition earned by students
and alumni
The extent to which an institution achieves its mission and goals. Institutional Effectiveness is monitored through ongoing, integrated, institution-wide research-based planning and evaluation processes that 1) incorporate a systematic review of institutional mission, goals, and outcomes; 2) result in continuing improvement of institutional quality; and 3) demonstrate the institution is effectively accomplishing its mission (Morante, 2003; SACSCOC, 2006).
Inter-rater reliability: "the degree to which different individual observers or graders agree in their scoring" (Maki, 2004, p. 93). It serves to reach common calibrations among scorers. The process ensures that scorers' ratings are reliable across different samples and samples from representative student populations (Maki, 2004).
L
The specific design and implementation of learning opportunities (inquiry, exploration, discovery, listening, observation, reading, writing, planning, discussion, practice, experimentation) that lead to the desired learning outcomes (Clark College).
The measurement of learning outcomes. Learning Outcomes Assessment examines student demonstrations of the results of learning. The process includes four cyclical steps: 1) teaching and learning, 2) developing questions and gathering information about student learning, 3) analyzing the information and drawing conclusions, and 4) reflecting and planning.
It documents the alignment (or dissonance) between the intended learning (as stated in the outcomes) and the actual learning (as demonstrated by the student). The practice of Learning Outcomes Assessment is collaborative and is intended to inform. Its goal is to continually improve student learning (Clark College).
Also see:
Learning Outcomes/Student Learning Outcomes/Objectives
Learning outcomes are operational statements describing specific student behaviors that evidence the acquisition of desired knowledge, skills, abilities, capacities, attitudes, or dispositions. Learning outcomes can be usefully thought of as behavioral criteria for determining whether students are achieving the educational objectives of a program, and, ultimately, whether overall program goals are being successfully met Harvey, 2004; Maki, 2004; Morante, 2003, Suskie, 2010).
Outcomes are sometimes treated as synonymous with objectives, though objectives are usually more general statements of what students are expected to achieve in an academic program (Allen, Noel, Rienzi, & McMillin, 2002).
A learning outcome is a specific, observable, and measurable knowledge or skill that the student gains/develops as a result of a specific course. Thus, learning outcomes are clearly stated in the course syllabus.
There are three categories of student learning outcomes.
- Cognitive outcome/declarative knowledge: What students KNOW; knowledge, comprehension, application, analysis, synthesis, and evaluation.
- Affective outcome/attitudes, values, or habits of mind: What students CARE about; students' feelings, attitudes, interests, and preferences.
- Performance outcome/procedural knowledge: What students CAN DO; skilled performance, production of something new (e.g., a paper, project, piece of artwork), critical thinking skills (e.g., analysis and evaluation). (Allen, Noel, Rienzi, & McMillin, 2002; McTighe & Ferrera, 1998).
M
The assertion of an institution's or unit's fundamental aspirations. Mission statements generally describe an organization's purpose, role, and scope (Gardiner, Anderson, & Cambridge, 1997).
N
An assessment where student performance or performances are compared to a larger group. Usually the larger group or "norm group" is a national sample representing a wide and diverse cross-section of students. Students, schools, districts, and even states are compared or rank-ordered in relation to the norm group. The purpose of a norm-referenced assessment is usually to sort students and not to measure achievement towards some criterion of performance (Harvey, 2004; Maki, 2004; McTighe & Ferrera, 1998; American Public University System).
O
Defining an outcome or objective in a way that facilitates measurements (Rice University).
P
Performance Criteria/Performance Targets
Quantitative measures by which student performance is evaluated. Performance criteria help assessors maintain objectivity and provide students with important information about expectations, giving them a target or goal to which they may strive (New Horizons for Learning; Suskie, 2010).
A systematic and organized collection of a student's work that exhibits to others the direct evidence of a student's efforts, achievements, and progress over a period of time. The collection should involve the student in selection of its contents, and should include information about the performance criteria, the rubric or criteria for judging merit, and evidence of student self-reflection or evaluation (Maki, 2004; McTighe & Ferrera, 1998; Morante, 2003; Suskie, 2010).
It should include representative work, providing documentation of the learner's performance and a basis for evaluation of the student's progress. Portfolios may include a variety of demonstrations of learning and have been gathered in the form of a physical collection of materials, videos, CD-ROMs, reflective journals, etc. (New Horizons for Learning; Smith & Tillema, 2003).
Uses the department or program as the level of analysis. Can be quantitative or qualitative, formative or summative, standards-based or value added, and used for improvement or for accountability. Ideally, program goals and objectives would serve as a basis for the assessment. Example: How well can senior engineering students apply engineering concepts and skills to solve an engineering problem? This may be assessed through a capstone project, by combining performance data from multiple senior level courses, collecting ratings from internship employers, etc. If a goal is to assess value added, some comparison of the performance to newly declared majors would be included (Leskes, 2002).
Also see:
Program outcome/program goal/program objective
The knowledge, skills, and abilities students should possess when they complete a program. Educational or degree programs are more than a collection of random courses. Educational programs prepare students for a range of particular outcomes that can be stated in measurable terms. Program assessment seeks to determine the extent to which students in the program can demonstrate these outcomes (George Mason University; Suskie, 2010).
Also see:
Q
Collects data that does not lend itself to quantitative methods, but rather to interpretive criteria (Leskes, 2002; Maki, 2004; Suskie, 2010).
Collects data that can be analyzed using quantitative methods (see "assessment for accountability" for an example) (Leskes, 2002; Maki, 2004; Suskie, 2010).
R
Capacity of an assessment method to produce consistent and repeatable results. A reliable assessment instrument or technique performs in a stable way over many uses at different times (Gardiner, Anderson, & Cambridge, 1997; Maki, 2004; McTighe & Ferrera, 1998; Morante, 2003).
A rubric is a scoring tool that explicitly represents the performance expectations for an assignment or piece of work. A rubric divides the assigned work into component parts and provides clear descriptions of the characteristics of the work associated with each component, at varying levels of mastery. Rubrics can be used for a wide array of assignments: papers, projects, oral presentations, artistic performances, group projects, etc. Rubrics can be used as scoring or grading guides, to provide formative feedback to support and guide ongoing learning efforts, or both (Carnegie Mellon University; McTighe & Ferrera, 1998; Morante, 2003; Suskie, 2010).
S
Established level of accomplishment that all students are expected to meet or exceed. Standards do not imply standardization of a program or of testing. Performance or learning standards may be met through multiple pathways and demonstrated in various ways. For example, instruction designed to meet a standard for verbal foreign language competence may include classroom conversations, one-on-one interactions with a TA, of the use of computer software. Assessing competence may be done by carrying on a conversation about daily activities or a common scenario, such as eating in a restaurant, or using a standardized test, using a rubric or a grading key to score correct grammar and comprehensible pronunciation (Leskes, 2002; McTighe & Ferrera, 1998; Suskie, 2010).
Also see:
Long-term (i.e., often 3-5 year periods) planning at the level of the whole institution or unit that focuses on adaptation to the organization's external environment and the future. Guided by a vision of the organization in the future, strategic planning attempts to position the organization favorably with respect to needed resources (Gardiner, Anderson, & Cambridge, 1997)
The gathering of information at the conclusion of a course, program, or undergraduate career to improve learning or to meet accountability demands. When used for improvement, it impacts the next cohort of students taking the course or program. Example: examining student final exams in a course to see if certain specific areas of the curriculum were understood less than others (Leskes, 2002; Maki, 2004; McTighe & Ferrera, 1998; Suskie, 2010).
T
The use of a combination of assessment methods, such as using surveys, interviews, and observations to measure a unit outcome (Rice University).
U
Intended outcomes that reflect the area or service that can be improved using current resources and personnel and are assessable within one assessment cycle. Unit outcomes should be under the direct control of the unit and align with a university strategic plan goal, objective, and strategy. For administrative units, unit outcomes are primarily process-oriented, describing the support process/service the unit intends to address. For academic student support units, unit outcomes may include both process and student learning outcomes (Morante, 2003; Rice University).
Also see:
V
The extent to which an assessment method measures or assesses what it claims to measure or assess. A valid assessment instrument or technique produces results that can lead to valid inferences (Maki, 2004; McTighe & Ferrera, 1998; Morante, 2003).
The increase in learning that occurs during a course, program, or undergraduate education. May either focus on the individual student (how much better a student can write, for example, at the end than at the beginning) or on a cohort of students (whether senior papers demonstrate more sophisticated writing skills-in the aggregate-than freshman papers). Requires a baseline measurement for comparison (Leskes, 2002; Morante, 2003; Suskie, 2010).
References
References:
Allen, M., Noel, R.C., Rienzi, B.M., & McMillin, D.J. (2002). Outcomes Assessment Handbook. Long Beach, CA: California State University, Institute for Teaching and Learning.
American Public University System. (n.d.). Learning Outcomes Glossary. Retrieved from http://www.apus.edu/academic-community/learning-outcomes-assessment/glossary
Angelo, T. (1995). Reassessing (and defining) assessment. The AAHE Bulletin, 48(2), 7-9.
Bloom, B.S. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive Domain. White Plains. NY: Longman.
Carnegie Mellon University. (n.d.). Rubrics - Eberly Center - Carnegie Mellon University. Retrieved from https://www.cmu.edu/teaching/designteach/teach/rubrics.html
Clark College. (n.d.). Glossary of Assessment Terms. Retrieved from http://www.clark.edu/tlc/outcome_assessment/resources.php#glossary
DeMars, C.E., Cameron, L., & Erwin, T.D. (2002). Information literacy as foundational: determining competence. JGE: The Journal of General Education, 52(4), 253.
Eder, D.J. (2004). General education assessment within the disciplines. JGE: The Journal of General Education, 53(4), 135.
Gardiner, L. F.; Anderson, C.; & Cambridge, B. L. (1997). Learning through Assessment: A Resource Guide for Higher Education. Washington, DC: American Association for Higher Education.
George Mason University. (n.d.). Glossary of Assessment Terms. Retrieved from https://ira.gmu.edu/resources/glossary-of-assessment-terms/
Goodrich, K. (2012, July 23). What is formative assessment? Retrieved from https://www.nwea.org/blog/2012/what-is-formative-assessment/
Harvey, L. (2004). Analytic Quality Glossary. Quality Research International. Retrieved from http://www.qualityresearchinternational.com/glossary/
Leskes, A. (2002). Beyond confusion: an assessment glossary. (Realty Check). Peer Review, 42.
Maki, P.L. (2004). Assessing for learning: building a sustainable commitment across the institution. Sterling, VA: AAHE.
McTighe, J., & Ferrara, S. (1998). Assessing learning in the classroom. Washington D.C.: National Education Association.
Morante, E. A. (2003). A Handbook on Outcomes Assessment for Two Year Colleges. Palm Springs, CA: College of the Desert.
New Horizons for Learning. (2002). Glossary of Assessment Terms.
Novak, J. D., & Cañas, A. J. (2006). The Theory of Underlying Concept Maps and How to Construct and Use Them. Pensacola,FL: Institute for Human and Machine Cognition.
O'Reilly, L. (n.d.). Assessment & Instructional Alignment: An Online Tutorial for Faculty. Retrieved July 5, 2018, from:
Palomba, C., & Banta, T. (1999). Assessment essentials: planning, implementing, and improving assessment in higher education. San Francisco, CA: Jossey Bass.
Rice University. (n.d.). Glossary of Assessment Terms. Retrieved from http://oie.rice.edu/strengthening-our-programs/glossary/
SACSCOC (2006). Principles of Accreditation: Foundations for Quality Enchantment (2nd ed.). Decatur, GA.
Smith, K., & Tillema, H. (2003). Clarifying different types of portfolios. Assessment & Evaluation in Higher Education, 28(6), 625.
Suskie, L. (2010). Assessing student learning: A common sense guide. San Francisco, CA: Anker Publishing Company, Inc.
University of Nebraska at Kearney (n.d.) Glossary of Assessment Terms. Retrieved from http://www.unk.edu/academic_affairs/assessment/glossary.php
University of South Florida (USF) (2018). Institutional Effectiveness System for Assessment Management User's Guide. Unpublished handbook, Office of Decision Support.
In addition to the primary sources above, this glossary draws from the collective expertise of the following secondary sources:
1. Assessment Glossary compiled by the American Public University System: http://www.apus.edu/academic-community/learning-outcomes-assessment/glossary
2. Glossary of Assessment Terms compiled by the Office of Institutional Effectiveness at Rice University: http://oie.rice.edu/strengthening-our-programs/glossary/
3. Common Assessment Terms compiled by the Eberly Center at Carnegie Mellon University: http://www.cmu.edu/teaching/assessment/basics/glossary.html
4. Glossary of Assessment Terms compiled by the University of Nebraska at Kearney: http://www.unk.edu/academic_affairs/assessment/glossary.php
5. Glossary of Assessment terms from Clark College: http://www.clark.edu/tlc/outcome_assessment/documents/GlossaryofAssessmentTerms.pdf
6. Glossary of Assessment Terms from George Mason University: https://ira.gmu.edu/resources/glossary-of-assessment-terms/