Post-CBT CLEPs are changing--are you keeping abreast?

Discussion in 'General Distance Learning Discussions' started by Orson, Nov 25, 2002.

Loading...
  1. Orson

    Orson New Member

    Anyone who has perused CLEP sample exams probably noticed an interesting evolution since computer based testing was adopted in the summer of 2001.

    Ths old CLEP practice books are no longer accurate (like those the local library has); Excelsior (and others?) adopted new grade equivalencies; many no longer are offered for grades; the new "pass" cutoffs are set to 50.

    But time marches on and so do the new sample exams. You can purchase them individually for $6--http://www.digitalriver.com/dr/v2/ec_Main.Entry?SP=10007&SID=32595&CID=0&DSP=0&CUR=840&PGRP=0&CACHE_ID=0
    --but do the sample exam books The College Board sells keep up? Anyone know for certain?

    For example, CLEP Intro to Ed Psych was revised as "a new exam" September 2001. This lists new ACE (American Council on Education) recommended "C" grade is listed (corresponding to "50"); a non-ACE "B" grade is listed, which is explained with: ""The mean score of students in the reference group who earned a final course grade of B is also noted in Table 1." Presumably, this comforts or otherwise better informs the test candidate. It lacks percentile distributions, although a footnote states that "candidate response data is needed to provide a percentile distribution. when this data is collected, percentile information will be provided." Accordingly, Excelsior grants no letter grade, merely "Pass" credit.

    But as of September 2002, more exams were revised and some subtle but possibly importamt changes may be noted. For instance, Human Growth and Dev., Biology, and Western Civ. I (and perhaps others) were "revised." New ACE recommended grade cuts for "C." were made. Somehow a "B" cut also appears--not ACE recommended, however. But unlike older pre-CBT exams, no percentiles are reported--instead they are "available upon request." And under "The Crdit Granting Score," it reads:

    "In the past, some colleges set their credit-granting scores at a higher level than the ACE rcommended socres because they felt the norming group did not reflect the ability of their students." True enough. But it continues in bold print: "Note that the new ACE recommended scores are determined by faculty currently teaching the comparable course and are higher than those produced by a norming study. Therefore these faculty judgements should be considered when local credit granting decisions are made."

    This means that these tests are now intentionally made to be more difficult than earlier ones, reviving a more positive reputation that may have been tarnished in the past. This is analogous to the renorming of SATs in the early 1990s, although with an opposite effect.

    But does this mean that ACE norming data will soon be forthcoming?

    --Orson
     
  2. Christopher Green

    Christopher Green New Member

    Orson~~~

    Thanks Orson~~~

    I realize some colleges (I'm unaware which ones) award credit for CLEPS based on their own assessment of the scores rather than the ACE recommendation. Do you think this change means that the ACE "recommended" grades will be considered with more respect than before? Why do you think this change has taken place?

    Chris
     
  3. Lawrie Miller

    Lawrie Miller New Member

    Orson:
    Ths old CLEP practice books are no longer accurate (like those the local library has);

    Lawrie:
    In terms of subject material and level at which it is examined, I know of no difference in pre and post CBT exam sets. Where do the exam questions materially differ pre and post CBT? I see no evidence that they do. Do real pass marks differ in terms of actual relative performance pre and post CBT? Again, I see no meaningful difference (the same performance is required to score (say) an old "43" pass threshold as the new "50" pass threshold). The sample questions are the very same in the old "Official CLEP Study Guide", 1998 edition, and the latest CBT test information guides dated September 2002 - same questions, same order, same number.

    Orson:
    Excelsior (and others?) adopted new grade equivalencies; many no longer are offered for grades; the new "pass" cutoffs are set to 50.

    Lawrie:
    Yes. While the test medium of delivery has changed, and scoring rubric altered, the performance scoring is graded to the curve, such that the net effect is a CBT exam that measures candidates relative performances using the same material at the same level, as the paper and pencil exams. Whether we designate a passing standard score using a pass indicator called "43" or "50" or "3 apples and a pear", makes no difference. What is important is the performance in the test relative to that of the set of traditional college students. The required relative performance has not changed (for the most part), the questions have not changed, nor have the number of questions.

    Clearly, exams evolve to incorporate new important changes in the field tested, they are not static, but they are still graded to the curve, and standard scores pass thresholds still represent the same relative performance. There may be some tweaking to better approximate the distribution of scores of the set of traditional college students, but the effects are going to be marginal.

    Orson:
    For example, CLEP Intro to Ed Psych was revised as "a new exam" September 2001. This lists new ACE (American Council on Education) recommended "C" grade is listed (corresponding to "50");

    Lawrie:
    Looking at this particular test as an example, the sample questions are identical pre and post CBT. The order of the sample questions is the same pre and post CBT, and the number of questions is the same pre and post CBT. The required relative performance in the exam to earn a pass is the very same for the paper and pencil or CBT exams ((regardless of standard scoring nomenclature).

    The scoring system has been rationalized in the CBT. Standard scores representing a pass at grade C have been reset to a common datum, namely 50. However, they represent essentially the same level of performance as as the paper and pencil test standard score pass thresholds.

    Orson:
    many no longer are offered for grades; . . . Accordingly, Excelsior grants no letter grade, merely "Pass" credit.

    Lawrie:
    I addressed this in April/June 2001, before the switch to CBT. See also BA in 4 Weeks CLEP CBT Update. Grades are not offered yet for some exams because there are insufficient data to grade the curve. When sufficient data become available, they will be graded. This is clearly stated in the Excelsior literature.


    Lawrie Miller
    http://geocities.com/ba_in_4_weeks/
     
  4. Orson

    Orson New Member

    THANKS Lawrie...

    Exact same questions...WOW.

    So we suckers aren't even getting anything new for our money!
    Any other reactions out there?

    --Orson
     
  5. Lawrie Miller

    Lawrie Miller New Member

    Re: Re: Post-CBT CLEPs are changing--are you keeping abreast?

    For example, CLEP Intro to Ed Psych was revised as "a new exam" September 2001.
    In fact, the old practice exam for this subject only provides 25 questions and the CBT, "Test Information Guide" (1), provides 50 questions. It may be however, that the retail version of the CBT practice exam only provides 25 questions - don't know.

    In any case, the 25 questions detailed in the old practice exam (2), and the first 25 question in the new CBT practice exam are identical in all respects (exact same Q and A, in the exact same order).

    The value of the indicator in the old exam of the requirement for a pass ("C") was that around half the questions be answered correctly, out of 100 (note that this is not the same as "raw score", since the old exam had an incorrect answer penalty which would reduce total score, but regardless, if you got around half correct, you'd pass). This translated to a scaled score of 47.

    The requirement in the new CBT exam is a raw score in the band 44 to 48 correct out of 100 (which is the same as total correct score since no penalty in the CBT exam), for a pass ("C" grade). This translates to a scaled score of 50.

    These two scales are not the same, of course.

    The above refer exclusively to comparisons between the old and new CLEP "Introduction to Educational Psychology", exam. Other old and new exam comparisons would yield different numbers.

    In general, the new CLEP CBT exam, raw to scaled conversions, "were determined by equating their raw scores to scores on one of the paper based forms. The equating process incorporated" (1), changes in formula scoring made necessary by the move from penalty to no-penalty scoring.

    In the final analysis, it seems the pass threshold of performance relative to the set of the performances of traditional students, remained the about the same, in the old and new exams (as noted in my previous post).

    See:
    1. CLEP Intro to Ed Psych, Test Information Guide Sept 2001
    2. CLEP Official study Guide 1998, Intro to Ed Psych, and Appendix D, tables 1, 2, and 3.
     

Share This Page