What Constitutes "Rigor" in Graduate Coursework?

In several posts, as contributors compare distance learning graduate programs, many speak of the "rigor" of the program as a measure of its quality. I have the sense though that the term is highly subjective. I'm curious as to what different posters consider rigor. Is it volume of coursework? Is it difficulty of the exams? Is it the quantity/quality of the bibliographic sources for research?

If it is volume of work, then I would question how quantity automatically translates into quality. If it is the difficulty level of the exams, then one person's rigor is another's ease. Well, you get the idea. As I navigate the waters towards selection of a doctoral program, I've been considering the advice of those who tout their alma mater as bastions of "rigor."

Perhaps some of you can tell me just how you measure that. Thanks.

 

This is a great question. The term "rigor" is often used with respect to accreditation and approval purposes with no precise definition.

Rigor is often in the eyes of the beholder. A French course for someone like myself (with no aptitude for languages) would be rigorous for me but simple for others.

We once had a course challenged as lacking rigor. When we were able to document that similar courses in RA accredited institutions were comparable, the issue was quickly dropped.

A good rule of thumb is if a course or program is similar to previously accredited courses or programs, it is rigorous.

We all like to think we graduated from rigorous programs.

David L. Boyd

 

Rigor sets in shortly after decease. It usually begins in the neck and jaw, gradually spreading down into the trunk and into the extremities.
Depending on age, physical condition, and ambiant temperature, rigor begins to pass off anywhere from a few hours after onset to as much as a day later.
Now ask me about "livor"!

Nosborne JD
(Who's just being obtuse.)

 

I'm almost afraid to, Nosborne. But I appreciate the humor.

 

This is a great, and very necessary question. I will not even try to attempt a definition of rigor. My take right now is the question of how can we characterize a PROGRAM or SCHOOL, in its totality, as rigorous? I have seen many, many instances where the same COURSE at the same university has very different standards of "rigor," simply because of the professor--which is not to say that the "easier" class offers less of a learning potential. Hardness, or difficulty, which I take here to be something of the same thing as what most people mean by rigor, does not equate in any direct way to learning. One could work very hard for a "B" in one section of a course, while another section of the same course might yield, with a much lighter work load, the same or better grade. And who can say in which class there was more actual learning? I remember a class I had to drop in the fall quarter one year, it was a class on primatology, and I took it that summer with the same professor. The summer class was wildly less demanding than the one in the fall. I don't know if the professor just wanted to have a lighter work load during vacation or if something happened that drastically altered his outlook on life. (He seemed like a happier man.) Anyway just a story, but I take people's claims of rigor with a very generous grain of salt. Personally, I have never been in a program that proved to be as rigorous as I thought it might be. My last obsrvation is that there are at least three kinds of students: those who out of pride or the need to feel that their program is hard actually make it so, those--the majority--who take it as they find it, and those few who know how to make any program easy. Great question! Next week let's talk about GRADES.

Craig

 

I've been involved in a number of discussion of "rigor" in graduate work. Some areas that I've heard focused on include:

1. Quality of literature - If a graduate course is based on a single textbook with no inclusion of original source material, this is a red flag. While textbooks perform a valuable role in summarizing a topic, graduate students should be able to read original literature. Rigorous graduate courses employ multiple literature sources - grads students should be doing extensive reading of the literature. Further, graduate students should know how to find relevant literature and how to evaluate the quality of what they've read.

2. Quality of assignments - Graduate work should require the student to move up Bloom's taxonomy (see below). Graduate work that sticks to the "knowledge" level isn't graduate work - it is undergraduate work - or less.

3. Examination - The use of multiple choice exams is often frowned upon at a graduate level. My take is that the bulk of graduate level exams should require the student to express himself in words or math - again, to move down Bloom's taxonomy. While I occasionally use some multiple choice to test terms and concepts at the "knowledge" level - I use short answer and problems to go deeper.

4. Quantity of work - beyond points 1-3 above, graduate work should require significant effort. If undergraduates are expected to do two hours of prep per hour of class, graduate students should be working 3 hours or more. While I know this is an input measure - I believe that "no pain" leads to "no gain". I haven't found a shortcut to learning - just as there are no shortcuts to physical fitness.

It strikes me that the current wave of graduate programs (especially the for profits) that try to compress learning into six week modules and completion of doctorates in 2-3 years fail on many of these counts. The fact that high quality, full-time on-ground graduate programs do better on these points speaks volume to the difference. While I wouldn't say that DL programs aren't creditable - I see a marked difference between a DL PhD and a high quality, traditional PhD. The letters are the same - the rigor is far less in the case of most DL programs.

Regards - Andy


From: http://www.coun.uvic.ca/learn/program/hndouts/bloom.html

Knowledge
observation and recall of information
knowledge of dates, events, places
knowledge of major ideas
mastery of subject matter
Question Cues:
list, define, tell, describe, identify, show, label, collect, examine, tabulate, quote, name, who, when, where, etc.

Comprehension
understanding information
grasp meaning
translate knowledge into new context
interpret facts, compare, contrast
order, group, infer causes
predict consequences
Question Cues:
summarize, describe, interpret, contrast, predict, associate, distinguish, estimate, differentiate, discuss, extend

Application
use information
use methods, concepts, theories in new situations
solve problems using required skills or knowledge
Questions Cues:
apply, demonstrate, calculate, complete, illustrate, show, solve, examine, modify, relate, change, classify, experiment, discover

Analysis
seeing patterns
organization of parts
recognition of hidden meanings
identification of components
Question Cues:
analyze, separate, order, explain, connect, classify, arrange, divide, compare, select, explain, infer

Synthesis
use old ideas to create new ones
generalize from given facts
relate knowledge from several areas
predict, draw conclusions
Question Cues:
combine, integrate, modify, rearrange, substitute, plan, create, design, invent, what if?, compose, formulate, prepare, generalize, rewrite

Evaluation
compare and discriminate between ideas
assess value of theories, presentations
make choices based on reasoned argument
verify value of evidence
recognize subjectivity
Question Cues
assess, decide, rank, grade, test, measure, recommend, convince, select, judge, explain, discriminate, support, conclude, compare, summarize

 

From: Letter from Dr. James Perley, Chair Committee on Accrediting of Colleges and Universities American Association of University Professors
http://www.aaup.org/lduc/319let.htm

"A bachelor's degree from an accredited college or university in this country should indicate something to the world; we believe that it should certify the completion of a varied educational program, characterized by rigor and high standards, which has required the student to reason and analyze and has emphasized the ability to think critically. "



Charles Graham Indiana University , Chris Essex Indiana University
http://www.aect.org/Events/Atlanta/Presentations/detail.asp?ProposalID=27

"What it IS * critical thinking * high standards and expectations * process more than product * cognitive development What it IS NOT * not grades * not memorization * not regurgitation Interesting but uncommon responses to the academic rigor definition question.... <more>"

Classical Christian School rigor
http://www.johnknoxpca.org/comm/page5.html
"Therefore, in the place of academic rigor, let us use 'essential academic work.' In addition to teaching some specific subjects, classical education should be characterized by schools that require essential academic work from the students.
Essential academic work is comprised of instruction that furthers the development of skills in our students, appropriately challenges the mental capabilities of the students, and requires the students to work diligently throughout the school year."

James Mbuva, National University
http://nssa.apsu.edu/NSSAJ/NSSAJ171/NSSAJ171PDF/NSSAJ17_1_8.pdf

The No Child Left Behind Act of 2001
http://www.ed.gov/offices/OESE/esea/progsum/title1a.html

Mathematical rigor
http://www2.msstate.edu/~jmyers/Philosophy.htm

 

Does a portfolio credit/CLEP degree adequately meet this definition (eg. requiring & developing analytical ability)? In the sense that it is documenting learning does it really document learning which is characterized "..by rigor, high standards, requiring.........."?

North

 

Well, here's what I remember about my own graduate school experience: Having to do a HUGE amount of reading (and actually having to do it, not just pretend to do it) from a variety of sources (mainly journal articles and classic texts), being expected, not only to regurgitate it on demand, but to integrate it with the other various materials of the course, and also those of other courses as well as your own thoughts, and being able to apply all this to novel situations and problems. I also remember plenty of nights where the number of cups of coffee greatly exceeded the number of hours of sleep. Is this rigor?
Jack

 

Jack! Did we go to the same university. You are not from Saskatchewan are you?

North

 

Rigor means a level of difficulty that is appropriate for the grade level and that meets state and national standards, coupled with contextual methodology.

(Contextual methodology means academic content and skills taught by utilizing real-world problems and projects in such a way that helps students understand the application of knowledge.)

Whew!

Having said that, rigor is that cramp you get between the thumb and index finger, when typing with two fingers.

 

Rigor is not in the eye of the beholder; it never stoops so low as to acquiesce to the lowest common denominator. Moreover, it is, to a great extent, self-determined.

As a student, the only time I feel I have crossed the threshold of rigor, is if I demolish expectations, thoroughly amaze my teacher, and make him or her confront something new.

 

Craig,

I think your perspective on this subject is excellent. As I've communicated with some posters regarding DL schools, most of them invariably seek to make their school or program "credible" by suggesting how difficult it is; that is to say, they claim that the program is rigorous. I appreciate the respone.

 

Great thoughts, Andy. I'm especially intrigued by the comparison between DL and traditional schools.

 

Gus,

I find two of your thoughts somewhat antithetical. "Rigor," you say, "is not in the eye of the beholder." Yet you say, "it is, to a great extent, self-determined." Isn't anything that is self-determined relatively subjective?

 

Not at all; allow me to explain. When I stated that rigor was not in the eye of the beholder, I meant that the mere fact that an individual has difficulty or considers a course challenging does not, in and of itself, make that course rigorous. (Yet this is precisely the justification many students and graduates of less than wonderful schools give for why the school or degree is legitimate.)

On the other hand (and this is true of practically every course I’ve ever taken), I have witnessed individuals who have done considerably less work than me, and yet earned the same grade. In other words, by choosing (self-determination) to make an extra effort, I made assignments and the coursework much more rigorous.

And, no, not everything that is self-determined is subjective. If I choose to study 3 hours per day instead of 2, explore a subject much more in depth than that which is required and write a 15-page essay instead of 5, this isn’t subjective; on the contrary, it is readily quantifiable (as is the knowledge gained). In fact, for the most part, the whole concept of grades is a system of quantifying the self–determined rigor a student has chosen to undertake.

 

Point well taken, Gus. Thanks for the thoughts.

David