Menu Menu

Minimize

Unit Assessment Process Unit Assessment Process

Minimize

What is program assessment

Difference between Objectives and Outcomes

Linking Institutional Learning Outcomes to Course/Program Learning Outcomes and Strategic Plan Goals


Unit Assessment Process


Selected Basic Terms Associated with Doing Outcome Assessment

By: Sesime Adanu

What is assessment

Assessment is an ongoing process, which involves the use of varied methods and data to evaluate student learning so as to determine whether or not learning outcomes are being met; use the findings to identify weaknesses or gaps and address them with the view to improve teaching and learning.

  • The purpose is to ensure continuous improvement by developing measurable outcomes, assessing them, and making changes where needed to improve the learning experience of students.

Why the need for assessment:

It helps address questions such as whether or not students are meeting the learning goals set by faculty in the respective courses and programs of study.

  • Assessment reveals the strengths and weaknesses associated with learning outcomes and suggests ways to address the weaknesses to improve student learning. Thus, determine if what is being done is working and if not what can be done to make it better.
  • Assessment is also done to help the college meet its accreditation requirements from accrediting bodies. Without accreditation, the Colleges loses its ability to award federal financial aid, and may have to close down.
  • To effectively and fairly assess students, faculty need to clearly communicate the learning outcomes (goals) of the course/program in the course syllabus so students know exactly what they need to do to become successful in the course/program based on established rubrics by faculty.
  • The College's accrediting body, the Middle States Commission on Higher Education (MSCHE) as part of its assessment stipulations expects the college to "systematically evaluate its educational and other programs and make public how well and in what ways it is accomplishing its purpose". "Ensure that students learning programs and opportunities are characterized by rigor, coherence, and appropriate assessment of student achievement through all educational offerings, regardless of certificate status, degree level or delivery modality".
  • Use assessment results to review and revise academic programs and support services and provide professional development activities, as well as improve student success such as retention, graduation, transfer and placement rates.
  • SUNY Mandate- all SUNY "campuses will enhance quality by developing and implementing plans for the regular assessment of institutional effectiveness, academic programs and General Education such that the campus meets or exceeds the assessment standards set by the Commissioner of Education, the Middle States Commission on Higher Education, and as appropriate, programmatic accreditation bodies".

The two main perspectives on outcome assessment:

  1. Direct assessment with examples: This form of assessment provides direct evidence of student knowledge, skills and capabilities and are measured through a wide range of means such as term papers, exams, tests, portfolios, standardized test scores among other considerations. Assessment of this kind provides direct evidence with regard to the types of skills or knowledge students are expected and able to acquire as evident through their work per agreed upon rubrics.
  2. Indirect assessment with examples: Unlike direct assessment, indirect assessment assesses the perceptions, opinions or thoughts of students in a course, activity or program. These perspectives could be gathered through, surveys (examples course evaluations, alumni surveys, student opinion surveys, campus climate survey etc), interviews, focus group discussions and others. Indirect assessments do not provide a strong evidence of student learning like direct assessment. They supplement direct assessment in providing a robust assessment outcome. Example, asking students through a survey as to what they have learned from a course or program (indirect assessment) compared to their actual performance in the course/program based on grades (direct assessment). As a be best practice, it is recommend that both direct and indirect assessments are used for student learning outcome assessment.

What is program assessment:

Program assessment refers to the process where a program is self-assessed by faculty teaching in the program aimed at ensuring continuous improvement on identified indicators of success such as program learning outcomes, retention, persistence, among others. The goal is to ensure that students are learning what the intended program outcomes indicate they will learn when they complete the program. Program assessment is a requirement from SUNY Central Administration and MSCHE.

  • Per SUNY Policy #1151; "Academic program evaluations shall generally occur on a five-to-seven-year cycle, or a cycle of programmatic accreditation of ten years or less."
  • Per SUNY Policy #1151: "A campus shall develop and implement a plan for the periodic evaluation of each of its registered academic programs" and the assessment plan must meet or exceed MSCHE requirements.

Program level learning outcome:

Program level learning outcomes are those learning-related outcomes that a student should be able to demonstrate upon successful completion of a program.

Program level outcomes:

Program level outcomes demonstrate a program's performance but are not specific to areas or domains of student learning (e.g., employment rates could be a program outcome but they are not specific to an area of academic content that the student may be expected to know/demonstrate/perform on upon the completion of a program).

Middle States under standard 4 requires periodic assessment of the effectiveness of programs supporting the student experience.

What are program learning goals:

They are the basis or standards upon which a program is assessed to determine whether or not it is meeting its intended outcomes. They provide context on the purpose of a program and the expected outcome/result at the end of the program.

  • Program learning goals are written in the form of broad statements and are usually long term in nature. They are followed by objectives, which provide the specifics on how and when assessments are done.

Example: Upon the completion of this program:

  • students will be able to know and apply the theory of cognitive behavior; will be able to interpret and apply data to solving community problems; will be able to learn and apply the mechanics of developing programming codes.

What are Program/Course Learning Objectives:

Unlike goals, which are broad statements, objectives are very specific and specify expected results at the end of a course/program as well as show how the results will be achieved. Objectives are usually aligned with benchmarks, which state the yardstick against which they will be measured.

Example: upon the completion of this course/program, students will be able to:

  • learn and apply the theory of cognitive behavior in diagnosing and counseling learning disability.
  • learn how to write an organized term paper to include introduction, problem statement, critical thinking, conclusion, researching the literature and citing of references using the APA style.
  • As stated earlier, objectives need to be specific (who, what, where, how), measurable, achievable, relevant, and with a specified timeframe (when will the objective be assessed and attained?)

Difference between Objectives and Outcomes:

Objectives state the specifics on what is expected to be learned or attained at the end of a program while outcomes provide the evidence or results that indicate whether or not the objective was attained.

What are course/program learning outcomes:

These are specific skills or knowledge that students are expected to acquire and are able to demonstrate upon the completion of a course or program.

Learning outcomes (objectives) could be viewed from three perspectives.

Cognitive, affective and behavioral.

  • Cognitive outcomes focus on what students are expected to know
  • Affective outcomes focus on what students are expected to think or care about
  • Behavioral outcomes focus on what students are expected to be able to do upon graduation from a program.

Examples- upon completion of this course/program students will be able to:

  • Develop and organize an electronic portfolio (cognitive).
  • Be able to acquire critical thinking skills that make them able to write terms papers, analyze varied perspectives to an issue, and draw sound conclusions (affective).
  • Acquire skills and self-regulation that help them with time management and completion of respective course assignments (behavioral).

Difference between output and outcome:

Output refers to the count or evidence at the end of a process, activity, event or program. Such results are usually attained in the immediate short term. Outputs do not provide specific evidence as to the quality or impact an event or activity has had on trainees or beneficiaries.

  • Example, a resume writing workshop on campus attracted 40 students in attendance. The 40 students who attended will be the output.
  • Second example- 20 students out of 60 had ‘A's in a course/program, while 10 had failed grades. This does not tell us anything about the subject matter on which the students were graded based on rubrics.

Outcomes

Outcomes, on the other hand provide evidence that demonstrates what has been attained or achieved as a result of an event, activity, or taking a particular course. Outcomes also address evidence on quality, impact or value.

  • Outcomes focus on the assessment and measurement of the success of the process and for that matter program or course effectiveness. An outcome based on the example given with the 20 students will be- 20 students had ‘A's based on the program/course rubrics used in the assessed of students' term paper with examination of their introduction, organization, critical thinking etc), students ability to identify key concepts, ability to apply concepts to solving problems. An outcome related to the resume writing workshop will be measuring things like how well the attendees were able to design their resume, use the right phraseology on work experiences etc.
  • Bloom's taxonomy with regard to active verbs to aid in the development of measurable outcomes.
Action Words from Bloom's Taxonomy
KnowledgeUnderstandApplyAnalyzeEvaluateCreate
define
identify
describe
label
list
name
state
match
recognize
select
examine
locate
memorize
quote
recall
reproduce
tabulate
tell
copy
discover
duplicate
enumerate
listen
observe
omit
read
recite
record
repeat
retell
visualize
explain
describe
interpret
paraphrase
summarize
classify
compare
differentiate
discuss
distinguish
extend
predict
associate
contrast
convert
demonstrate
estimate
express
identify
indicate
infer
relate
select
translate
ask
cite
discover
generalize
give examples
group
illustrate
judge
observe
order
report
represent
research
review
show
trace
solve
apply
illustrate
modify
use
calculate
change
choose
demonstrate
discover
experiment
relate
show
sketch
complete
construct
dramatize
interpret
manipulate
paint
prepare
produce
Report
Teach
Act
Administer
Articulate
Chart
Collect
Compute
Determine
Develop
Employ
Establish
Examine
Explain
Interview
Judge
List
Operate
Practice
Predict
analyze
compare
classify
contrast
distinguish
infer
separate
explain
select
categorize
connect
differentiate
discriminate
divide
order
point out
prioritize
subdivide
survey
advertise
appraise
break down
calculate
conclude
correlate
criticize
deduce
devise
diagram
dissect
estimate
evaluate
experiment
focus
illustrate
organize
outline
plan
question
test
reframe
criticize
evaluate
order
appraise
judge
support
compare
decide
discriminate
recommend
summarize
assess
choose
convince
defend
estimate
find errors
grade
measure
predict
rank
score
select
test
argue
conclude
consider
critique
debate
distinguish
editorialize
justify
persuade
rate
weigh
design
compose
create
plan
combine
formulate
invent
hypothesize
substitute
write
compile
construct
develop
generalize
integrate
modify
organize
prepare
produce
rearrange
rewrite
role-play
adapt
anticipate
arrange
assemble
choose
collaborate
collect
devise
express
facilitate
imagine
infer
intervene
justify
make
manage
negotiate
originate
propose
reorganize

What are benchmarks:

Benchmarks are specific measures, which help determine the effectiveness of a program, activity or event. To determine effectiveness, a benchmark has to be established against which comparisons are made. Thus, benchmarks provide the yardstick against which to determine success or failure. Benchmarks can be internal or external. Internal benchmarks refer to the situation where a program/course uses its own data as baseline to determine whether or not any improvement was made. With external benchmarking, data from a similar course or program in another institution or department is used as the yardstick to determine how ones program compares with that of another institution or department.

What are rubrics:

Specific sets of criteria that are clearly defined and used by faculty to assess student performance in a course or program. It is thus a form of grading scheme used by faculty. The criterion used assesses a student's ability at each level of performance and assign values to each level. It provides levels of proficiency that describe a continuum from excellent to poor performance. To help students have a good sense on how they are being assessed, it is advisable that faculty share with students through the course syllabus information on grading rubric.

What is a measurable outcome:

Outcomes that are crafted with the ability to quantify or measure specific tangible results. Outcomes could also be assessed qualitatively. A measurable outcome needs to focus on a specific issue or characteristics, have a clearly identified means of determining results as well as a clear criteria on how success will be measured.

  • Bad example: Upon the completion of this program, students will become successful at whatever they do.
  • Good example: Upon the completion of this program, students will be able to develop a project plan, apply theory to solving problems, and do market analysis.

Example of Direct Outcome Assessment Using Bloom's Taxonomy, Course Learning Outcomes, Benchmarks, and Assessment Findings

Course learning outcomes for COM 210- Upon the completion of this course, students will be able to:

  1. Enumerate (Bloom's active verb) practical and theoretical knowledge in advanced lighting practices, and repeat (Bloom's active verb) the process in both the environments of field production and studio production.
Name of Office/Department: Communication and Media Arts
Name of Person Responsible: John Zoningo
Email Address: example@sunybroome.edu
SUNY Broome Strategic Plan Goal and ObjectiveUnit/Dept. Goal to AssessMeans of Assessment (How will goal be assessed?)Benchmark-Expected Outcome (Target expected to be met)Summary of Assessment FindingsUse of ResultsBudget/Resource Allocation Implications
Goal 2:Teaching and LearningStudents expected to enumerate practical and theoretical knowledge in advanced lighting practices, and repeat the process in both the environments of field production and studio production.Theoretical knowledge to be tested through mid-term paper and practical knowledge tested via field work tapping events at the local farmers market.Students are expected to score at least 80% in each of the 3 lighting theories; 80 % of the students are expected to have the right light balance filming indoors and outdoors using created rubric.Of the 20 students in the class, 17 scored at least 85% in theories. The goal was, therefore, exceeded by 5%. On advanced lightening, 9 of the students accurately met the threshold using established rubric. 11 did not. Goal was, therefore not met.Feedback from students revealed that 3 did not pass theories because of lack of understanding of key concepts. Subsequent courses will be taught with supplementary materials explaining the key concepts. To help students master the right light balance, two extra class time will be devoted to practicing these skills by taking students into the community for practice.The program will need $20,000 to replace 4 broken lighting systems as well as buy 6 more to make the practical side of the class more accessible to the students.

Closing the assessment loop on the advanced lighting goal as indicated above; a goal that was not met. This will be done in the following academic year to close the loop.

Faculty followed up on student feedback obtained from prior course evaluation and used it for the next class taught by developing a supplementary material that specified the key steps to be followed in understanding the theory of lighting. Two extra class times were spent sending the class into the community to practice lighting skills under the guidance of faculty. The college provided the program with $18,000 out of the $20,000 requested to replace equipment. Students did a fundraiser on campus to make up for the $2,000 shortfall. The implementation of these measures led to 20 of the 21 students enrolled in the next course passing the lighting theory and practice aspects of the course with a grade of C or higher based on the same rubric used for the prior course. One student had a D grade. The goals were met this time around and the assessment loop (gap) is now closed.

Example 2.

Course learning outcomes for BUS 150- Upon the completion of this course, students will be able to:

  1. Set-up (Bloom's active verb) and produce (Bloom's active verb) a program that runs a hypothetical non-profit organization's online application form.
Name of Office/Department/Program: Business Information Technology
Name of Person Responsible: Alexo Batarra
Email Address: example@sunybroome.edu
SUNY Broome Strategic Plan Goal and ObjectiveUnit/Dept. Goal to AssessMeans of Assessment (How will goal be assessed?)Benchmark-Expected Outcome (Target expected to be met)Summary of Assessment FindingsUse of ResultsBudget/Resource Allocation Implications
Goal 2: Teaching and LearningStudents are expected to set-up and produce a program that runs a hypothetical non profit organization's online application form.Students set up and formatting of the application page, layout of the questions, and writing programming codes that mandate key questions to be answered before applicants move to the next question.All students will be expected to complete the assignment to specifications under "means of assessment"Of the 15 students who completed the course, 13 were successful in setting up and developing programs that run the non profit online application form. Goal was not met. Two students had difficulty completing the programming component of the assignment.Faculty to work with the Learning Assistance Department to hire a peer tutor to provide support for students needing help in coding and programming.An amount of $12,000 will be need to pay hourly wages for peer tutor for the academic year.

 

  • Closing the assessment loop on students who had difficulty with coding and programing as indicated above.

Faculty worked with the LAD department to recruit a peer tutor who now provides a one-on-one tutorial support for students who need help understanding the coding process. This effort led to all students who took the course successfully completing it with a passing grade. The goal is, therefore, met. The assessment loop is now closed.

  • Examples of Indirect Assessment:

Faculty teaching in courses or programs could assess students' perception of courses through indirect means such as students' feedback through end of term/semester course evaluations, surveys administered to students to seek their views of how the course was taught as well as its content. Also, use of results from the SUNY Student Opinion Survey, program job placement data, program review data, alumni survey, graduation and retentions data, and student demographic statistics. 

For example, 45% of the students who took a course indicated in the course evaluation that they had difficulty understanding most of the content in the textbook used for the course and will appreciate the formation of discussion groups to help them better digest the course material. Faculty took the suggestions into consideration and reviewed other books to select a different textbook that was more digestible for students. Discussion groups were also formed for the next class. Course evaluations for the next class revealed that 80% of the students were satisfied with the new textbook, and 85% liked the formation of the discussion groups. The two identified gaps are now addressed and closed.


Linking Institutional Learning Outcomes to Course/Program Learning Outcomes and Strategic Plan Goals

After measurable outcome goals are developed, they need to be linked to one of the six Broome Institutional Learning Outcomes (ILOs) as well as applicable goal(s) of the strategic plan. This is attainable by identifying the outcome goals that are more in alignment with any of the ILOs or strategic plan goals.

What are outcome assessment findings/results

These are the end results arrived at after the goals and objectives developed are assessed at the end of the assessment cycle to determine whether or not established benchmark was met.

Linking outcome assessment results to the allocation of resources (use of results)

After outcome assessment findings are made, the results have to be used in requesting budgetary resources to support continuous improvement. This helps focus assessment on issues or areas that contribute to improved student learning.

Curriculum mapping from courses to program learning outcomes

This refers to the process where course-learning outcomes are mapped to program learning outcomes to help assess the effectiveness of program learning outcomes. An example of a Curriculum map is presented below:

Curriculum Map of Program Learning Outcomes

  • Name of Program: Submitted by: Program Coordinator: Date submitted:
  • KEY: I-Introductory; S-Secondary; A-Advanced
Business Administration Program Learning Outcomes
Courses Mapped to Program Learning OutcomesIntroductory Knowledge of marketingGlobal Context of Business ManagementU.S. Business ModelsPrinciplesPrinciples and Processes of the Stock MarketBusiness Consumer Behavior and TheoriesApplied Marketing Entre- preneur- shipProfessional Industry Prepa- ration
Bus 101IIII  II
Bus 102 S I, SI II
Bus 201 S I, S III, S
Bus 202    I, SIII, S
Bus 203SSIS II, S 
Bus 204SSISSII, S 
Bus 301 ASS, A SSS
Bus 302    S, ASSS
Bus 305 AAS, A SS, AS,A
Bus 306   S, AS, A AA
Bus 309 S, A S, A S, AS, AA
Bus 310  AI, S, ASS, AS, AS, A
Bus 401  AAAI, S, AAS, A
Bus 402 AAAAAS, AA
Bus 403  AAAAAA
Bus 405  AAAAAA

For Gen Ed courses and other courses without a program, map to Gen Ed and Institutional Learning Outcomes (ILOs). The reason being that there are no program learning outcomes to map to.


Staging Enabled