BA in 4 Weeks - Journal Review of quality

This decanted from an over full thread. It is also rather long.

This is in reply to Andy's question and suggestion. Below, I define what kind of graduate BA in 4 Weeks methods produce, relative to the traditional norm. The relative worth of the degree earned is defined.

Methods are explored detailing how we might prove or disprove the claims made for BA in 4 Weeks techniques. sketches of data gathering strategies, hypotheses formation, and hypotheses testing, are explored.

Validation can only come from conclusions based on empirical data. It is the effects of a process that have relevance, not the form of the process.

Further, validation requires that the claims made for the processes and methods, are in fact realized. The claim made by BA in 4 Weeks AKA me, is that there is no significant difference in the academic performance of graduates with degrees earned using the methods delineated in BA in 4 Weeks and those earned by more traditional means. Should any difference exist, that difference will be positive in outcomes relative to the traditional norm.

There is no significant difference in average performance in graduate school between the two groups.

There is no significant difference in the average performance in employment between the two groups.

It is a measure of my confidence in BA in 4 Weeks that I have left no room for wiggling.

.............................................

MEANS AND METHODS
I think what we need is accumulation of readily available comparative objective data, and some kind of longitudinal assessment.

The performance of those going through the BA in 4 Weeks process and earning a degree can and is measured against the performance of traditional college students every single time they sit a standardized examination. Every single time. The objective, data measuring relative performance is there, every single time.

Yet, I have no objection to BA in 4 Weeks processes and methodologies being scrutinized and critiqued by anyone. However, my view is, as noted, that there are two components necessary to measure its effectiveness: near term outcomes, and long term outcomes.

COMPARIVE ACADEMIC PERFORMACE
Near term outcomes are easy to assess: a standardized exam is passed or it is not. The GRE subject Exam measures relative performance between graduates (usually graduates) in some discipline. It is an excellent objective method of comparing learning in some specific field.

In fact, I'd love to see a study of the comparative performances of those earning a degree primarily by examination against a pool of more traditionally educated graduates, using that ready made parametric tool, since there is already a wealth of data establishing the range of performances of traditionally educated graduates.

We may also look at performances in graduate school and compare these with the norm in that same environment.

COMPARATIVE MEASURES OF UTILITY
We must also try to assess utility. What is the value of the degree in getting the holder to where he wants to be? What is important really is the measured difference in the value of the degree with all other influencing factors removed, against the value of the average traditional degree with all influencing factors removed. The difference between these two values is a measure of the MARGINAL UTILITY of the degree under test.

MARGINAL UTILITY AND HYPOTHESES
Clearly, this value can be positive or negative or zero. Zero indicates no difference between the traditional degree (control) and the degree earned by examination (the variable under test).

It will be immediately obvious that this approach lends itself to formation of a distribution which is a measure of the deviation from the norm. Hypotheses may be developed and null hypotheses formed and tested e.g., "There is no significant difference in utility". Which is to say, there is no difference in the sample means (control and test) due to or caused by the variable under test (marginal utility of the exam-based degree).

RESULTS
Thus we can determine whether or not a degree earned by the BA in 4 Weeks processes and methods, is as useful as a traditionally degree. Further, we can determine if any bias that does exist is negative or positive (which horizontal direction the test sample mean is skewed).


MEASUREMENT AND DATA GATHERING TECHNIQUES
Now, backing up a bit, how can we actually measure utility (I've already given detailed definitions of utility both here and in AED)? What is "average utility" from which the benchmark control will be derived?

"Average utility", is the mean of the usefulness of the average degree. How can we measure this? One approach is to look how far, on average, a non prestige degree will take the holder (to remove the outliers that skew results). There will be a distribution formed with (say) the bulk of those with non prestige degrees reaching a certain level, defined by salary (and maybe job title).

Fortunately, these data are already available for many degree programs, and for whole institutions. They are often a feature of program and institution catalogs.

We compare the average means of performance (salary level - and perhaps job level) of the control group (the traditionally educated graduates) with the performance of the variable under tests (the exam-based graduates). With sufficient sample size for the test group, the comparison of the means is a comparison of utility (since our samples are in other respects the same - this dependent on using the correct data gathering techniques) and the difference between the means is a measure of marginal utility

Comments and suggestions to [email protected] are welcome


Lawrie Miller
author: BA in 4 Weeks and Accelerated Master's Degrees by Distance Learning*
http://geocities.com/ba_in_4_weeks/

 

Lawrie - I give, can you tell me:

"has any of your BA in 4 weeks worked been submitted to a refereed journal? If not, have you considered this as a way to validate your work? "

I appreciate your lengthy response and will comment on it in a minute. I was hoping you'd say either "Yes" or "No" at some point, however, to your original question. Perhaps I missed it.

One point on measuring whether there is a significant difference - to do so in a rigorous way you need to randomly select subjects. If you look at self selected pools of people that pursue outcome based versus traditional results, you really can't answer the research questions you've put forth.

One thing I am going to look at, as I have time, is the following hypothesis:

The acceptance of CLEP and GRE exams for academic credit is negatively related to the academic reputation of the institution.

I suspect (but could be wrong) that as one ascends the academic pecking order, CLEP and GRE examswill become less acceptable as a way to earn academic credit.

I'll let you know what I find.

Regards - Andy

 

Re: Re: BA in 4 Weeks - Journal Review of quality

I was presenting a sketch to those already familiar with the fundamentals of statistical methods. It is a shorthand proposal. Clearly that is the case Andy, the sample must be randomly selected and I did not indicate otherwise. Is hoped it might be clear to you from the rest of my exposition that I would know that?

I was thinking you might have something germane to say about the meat of the content? What do you think of the design? Do you agree, in principle with the proposed data gathering techniques? Are they all practical? Do you agree with the way I arrived at the hypothesis? Do you agree with the precepts of the hypothesis?

I feel I've asked for comment on my sleek new auto body design, and your response is, "it will have to have wheels or it wont go".


Andy:
One thing I am going to look at, as I have time, is the following hypothesis:

The acceptance of CLEP and GRE exams for academic credit is negatively related to the academic reputation of the institution.

Lawrie:
Why?

Andy:
I suspect (but could be wrong) that as one ascends the academic pecking order, CLEP and GRE exams will become less acceptable as a way to earn academic credit.


Lawrie:
You could be right but what does it prove that is relevant? All that will do is indicate a spread of attitudes and other circumstances. It would be less than wise to interpret the data solely in terms of attitudes, and it's not clear to me how you will credibly suppress extraneous variables, but what do I know.

It isn't germane to this issue, clearly defined right at the start of the thread. Thought the core of this had to do with determining if there is any difference in quality between degrees earned by exam and those earned in the tradition way, in terms of relative outcomes and in terms of relative utility. How does your analysis impact that end?

You may just be mentioning in in passing and I have misinterpreted your intention in which case, OK, but what about my sleek new auto body?




Lawrie Miller
author: BA in 4 Weeks and Accelerated Master's Degrees by Distance Learning*
http://geocities.com/ba_in_4_weeks/

 

Re: Re: BA in 4 Weeks - Journal Review of quality

Validation can only come from conclusions based on empirical data. It is the effects of a process that have relevance, not the form of the process.

This is precisely why I laid out my proposal. Will any review adhere to these basic tenets of good scholarship? We have debated this many time and I have reiterated it here.


THE TENETS OF BA in 4 Weeks or Words to test by:-

The veracity and effectiveness of an education or learning methodology, or process, can only be determined in terms of outcomes.

"Method", "process", have no meaning, life, or importance, other than when expressed through outcomes.

Outcomes are measurements of differences.

All testing is a measurement of a delta, the difference between a mark and what is, or the difference between what is and what was.

Education and learning cannot be credibly described other than in terms of outcomes.

Outcomes are measurable parametric entities


Lawrie Miller
author: BA in 4 Weeks and Accelerated Master's Degrees by Distance Learning*
http://geocities.com/ba_in_4_weeks/

 

.
BA IN 4 WEEKS HALL OF QUACKERY ;- or Chants of the Academy

"Education is about more than testing" - QUACK

"Seek an education not certification" - - QUACK QUACK

"Education is not certification" - - QUACK QUACK QUACK

"Some learning/education can't be assessed by examinations" - QUACK! QUACK! QUACK! QUACK! AFLACK!!!!


Lawrie Miller
author: BA in 4 Weeks and Accelerated Master's Degrees by Distance Learning*
http://geocities.com/ba_in_4_weeks/

 

Re: Re: BA in 4 Weeks - Journal Review of quality

IIRC, even Harvard accepts CLEP exams for credit. How many and which ones, I never looked into.


Bruce

 

Re: Re: BA in 4 Weeks - Journal Review of quality

I may have missed it also, but I believe the answer is "no".


Bruce (who is used to looking for hidden answers after watching 5 gubernatorial debates)

 

Re: Re: Re: BA in 4 Weeks - Journal Review of quality

And how is one to tell if the lack of acceptance of CLEP or GRE exams is not just an emotional (rather than rational) decision on the part of administrators at any particular institution? Let's not forget how tightly some in the schooling industry cling to the status quo, in which students are clearly subjects of the institution. It is too bad that the premise under which the students volunteer for this relationship is all wrong. They have been promised a well-rounded education, what they really get--in any case--is a credential based upon a course of study that is pretty much locked in stone.

Accepting too many CLEP or GRE exams for credit would expose that faulty premise, wouldn't it? Hence, fear displaces reason.

 

Re: Re: Re: BA in 4 Weeks - Journal Review of quality

I'm sorry, I thought I was clear. I will try to be succinct.

1. Validation can only come from conclusions based on empirical data. It is the effects of a process that have relevance, not the form of the process.

There is no other validation. How could there be? That's not a rhetorical question. If there's another way, it is beyond my ken. With respect to this issue I have another statement I'd like to add to (1) above and in the context of this question as it relates to determining the effectiveness of BA in 4 Weeks methodologies and processes.

2. Validation based on opinion is bunk.

Now, if a scientifically conducted analysis of the kind I have outlined in post one this thread were to be conducted, the results and methods used in that analysis could certainly be reviewed for validity. Absolutely. In fact, to ensure credibility, I would not have it any other way.

Of course, there is a way in the interim that I find convincing, to date. The majority who embark on the process are successful in reaching their goal, be it earning 3 credit hours or a whole degree.

I would ask Andy to consider that there are some today with bachelor's degrees who most likely (according to their testimony) would never even have attempted earning a degree, had BA in 4 Weeks not been their for them. There are countless more who have earned their degree with the help of exam based credit who most likely would not have earned a degree otherwise.

If even a tenth of the negative opinion served up here by some on this issue had validity, we would know it, because that cohort of degree holders would be identifiable in the data by its rate of failures.

Truth is of course, all experience and data indicate there is no significant difference between cohorts. Yet the nattering nabobs of negativism*, continue with there defamation and their alchemy, while just over the rise, people are using BA in 4 Weeks to assist them in earning their degree.

I hope this brief appendix to my earlier comments, clears up any misunderstanding.



*I know Spiro Agnew will not mind me stealing this (he's dead).

Lawrie Miller
author: BA in 4 Weeks and Accelerated Master's Degrees by Distance Learning
http://geocities.com/ba_in_4_weeks/

 

Lawrie - I guess we'll have to agree to disagree.

Regards - Andy

Hardly - this statement is certainly true. Look at a dictionary definition of "education". It certainly isn't "testing".

Another true statement. Again, check a dictionary. The two concepts are different.

Another absolutely true statement. As Deming the great QC guru pointed out leadership should not "Think that measures will contain all the information or knowledge required for managing a system." There are subjective aspects of education that can't be easily assessed by simple examination.

An example - the skills required to work as a medical doctor aren't measured by a single test - MDs go through several steps before they practice medicine.

 

Here are some comments on your proposal. Overall, the idea is good - but it is practically impossible to measure.

Regards - Andy

Researchers are a cautious lot - they don't "prove" or "disprove" their hypotheses - they "support" or "fail to support" them.

This is a laudable hypothesis to test. However, it will be very difficult to test. First, you'll have to identify some subjects to participate in this test - and then randomly assign them to treatment groups (e.g. BA in 4 weeks or tradtional). Then you'll have to follow the two groups through undergraduate and graduate study. Throughout you'll have to be on the lookout for confounding factors.

Another concern - the participants needs to be from a homogenous group - say mid-career professionals who lack undergraduate and graduate degrees. If you have too much variation in the group - say 18 year olds and 44 year olds, you won't have any hope of testing what you're after.

Again, these are good hypotheses.

Performance in graduate school is easy enough to measure - simply obtain their GPAs. However, what about the difference in graduate schools attended by participants? What if a disportionate number of BA in 4 weeks folks go to institutions with rampant grade inflation and traditional students tend to go to more traditional grad schools? Or vice versa. Again - you won't be able to fairly compare.

As for performance in employment - this isn't easily measured. Each employer has a different way to measure this.

But Lawrie - to make this comparison meaningful, you'll have to compare for a host of other factors. Age is one - typically older workers make more than younger workers. A group of 44 year old BA in 4 year grads may make more than a group of 25 year old traditional grads - but that may have nothing to do with how they obtained their degrees.

Salary level sounds good - but there are regional salary differences. $100,000 in salary in NYC isn't the same thing as $100,000 in Montgomery, Alabama. Job level? How do you hope to measure that?

 

Re: Re: Re: Re: BA in 4 Weeks - Journal Review of quality

I certainly don't want to answer for Andy, but I believe all he was looking for was "yes" (as in "yes, I've submitted it to a refereed journal") or "no" (as in "no, I haven't submitted it to a refereed journal").


Bruce

 

Statistical tesitng

Without provoking the emotional wrath of anybody - I am too near retirement to be really concerned with the 'firebrands' who have flared up in my academic life over 31 years - most of whom disappeared into a well deserved obscurity, I think I should re-state a sound statistical "rule":

"if you cannot distinguish/identify two sub-sets within a sample population by the tests devisable by an accepted test, then you cannot say with any credible degree of confidence that there is a difference between them."

At least that is what I used teach many years ago when I taught statistics (or "sadistics", as I called it then) to undergraduate and, later, potential post-graduate students about to start their Masters.

The assertions of someone that there is no difference and the questions raised by another that there may be identifiable differences if the tests are refined to take account of attributes not captured in the current tests, are both compatible. The question is empirical, not opinionated.

 

So what? Few, if any complex skills can be assessed in a single examination, that is not now, nor has it ever been at issue or an issue.

The point is that the skills are tested, all of them. If competencies cannot be demonstrated in some manner they are unknowable.

I think you should re-read Demming's statement, Andy. He is saying that some aspects of education are not easily tested, not that they can't be tested. No one has argued that all learning is easily tested. Given that, the inclusion of the quote is meaningless.

As you well know, within the context of the debate and the narrowed meaning of the terms when used in the debate, this is bunk. You have broadened the definitions to include meanings never intended by the purveyors of this quackery. The issue is how learning is transferred and how we know that it is transferred. Education has no purpose, no meaning, other than in the context of its effect on outcomes.

You know, this sort of device does not further understanding of the issues one bit, Andy. Come now, do try to offer reasonable arguments that address the issues, which are to do with efficacy and outcomes. I see no point in playing point games. There are real issues here, why not address them.

Tell us your position. I have declared mine. Time for you to step up to the plate. Are you saying that some learning cannot be tested, Are you saying that or not?


Lawrie Miller
author: BA in 4 Weeks and Accelerated Master's Degrees by Distance Learning
http://geocities.com/ba_in_4_weeks/

 

You're thinking too much here folks.

Degrees by testing exist. Degrees by testing will continue.

Degrees by testing have worked for many people. They may not be suitable for many others.

I would assume that many of the people doing degrees by testing already have an education, they just lack a degree.

Lawrie's website has probably encouraged thousands of people to seek a degree. This, by itself, is amazing.

 

Re: Re: Re: Re: Re: BA in 4 Weeks - Journal Review of quality

Oh, I was answering his follow up question: will you or would you submit it for review? The answer to that question is delineated in my replies.

 

Re: Statistical tesitng

If indeed you did not wish to provoke an emotional response, why did you make these derogatory remarks? While declaring you are above the fray, you simultaneously deliver a gratuitous backhand.

I thought the rest of your post was interesting - what was the exact point - could you clarify?

 

Education is about acquiring and assimilating knowledge and understanding.

In a previous thread, Lawrie tried to argue that one's own internal process of thinking about, questioning and comparing information constitutes a sort of testing. I suppose that's true. But that broader definition of 'testing' isn't what is being referred to when we speak of 'certification' or receiving university credit through 'testing', is it? The latter are external social measures that arguably make the results of the inner process objective and reliable for others to trust.

Obviously both are important and valuable. But education is prior to certification. That's true because many people have no need to have their growth in knowledge and understanding certified. It's also true because certifying those who lack knowledge and understanding is the definition of "degree mill".

A large part of people's educations is never certified.

I think that I agree with that.

I can learn Zen meditation or to enjoy Balinese gamelan music.

But even in more conventional academic subjects, education involves the ability to integrate our newly acquired knowledge with our existing understanding, and to deploy it in real world situations. It's hard to see how GRE or CLEP exams could capture that. The real test would be one's subsequent life after receiving the education, I guess.

 

Re: Re: Re: Re: Re: BA in 4 Weeks - Journal Review of quality

Bruce - you are correct. I was just hoping to hear "yes" or "no".

Thanks - Andy

 

Re: Re: BA in 4 Weeks - Journal Review of quality

Thank you Andy. Two points,

1. as noted a number of times this is only a rough sketch it is not nor was it ever meant to be a detailed proposal.

2. No model used is going to be perfect, There is not a design around that is flawless. All have drawbacks. Some cannot.

What is this:-

Andy:
"However, it will be very difficult to test. First, you'll have to identify some subjects to participate in this test - and then randomly assign them to treatment groups (e.g. BA in 4 weeks or traditional).

Lawrie:
Yup. in what model can you dispense with that? Isn't that par for the course?

Andy:
Then you'll have to follow the two groups through undergraduate and graduate study. Throughout you'll have to be on the lookout for confounding factors."

Lawrie:
What? Why must this be longitudinal study? I can do a cross-sectional? I don't understand. Explain why I need do a longitudinal study.

And confounding factors are always an issue, why would it make more difficult here?


Andy:
Another concern - the participants needs to be from a homogenous group - say mid-career professionals who lack undergraduate and graduate degrees. If you have too much variation in the group - say 18 year olds and 44 year olds, you won't have any hope of testing what you're after.

Lawrie:
Excuse me? I don't see why that should be at all. We are going to test whether there is a difference between the sample means of two randomly selected representative populations. Representative of the whole graduate population. Why on earth do I need to pick the cohort you suggest? It would not be representative of the population. The test would not, could not validate or even address the question under study.

It is true that it has been well over two decades since I did any of this but as I remember it, your take on this is wildly off the mark.


Andy:
Performance in graduate school is easy enough to measure - simply obtain their GPAs. However, what about the difference in graduate schools attended by participants? What if a disportionate number of BA in 4 weeks folks go to institutions with rampant grade inflation and traditional students tend to go to more traditional grad schools? Or vice versa. Again - you won't be able to fairly compare.

Lawrie:
We select from a random sample to mirror the general population of graduates. That would or could include relative ranking of schools attended. The measurement of the the difference of the two means of two randomly selected samples (control population and population under test) assumes any difference is due to the factor under test since we will have factored out confounding variables by means an example of which is detailed above.

I don't see that this is different from problems encountered in most studies. The more I look at it the more "normal" it seems.

Andy:
As for performance in employment - this isn't easily measured. Each employer has a different way to measure this.

Lawrie:
We have previously defined performance in terms of salary and maybe job title. Once again, Andy, if the two sample means are representative of the general population, they will most likely reflect the normal distribution of salaries and job titles, and the normal distribution of variation with in them. There will be no difference in the sample means due to these factors.

That is the whole purpose of a randomly selected sample population.


Andy:
But Lawrie - to make this comparison meaningful, you'll have to compare for a host of other factors. Age is one - typically older workers make more than younger workers. A group of 44 year old BA in 4 year grads may make more than a group of 25 year old traditional grads - but that may have nothing to do with how they obtained their degrees.

Lawrie:
No you do not. you test the difference of performance in populations. That is why we are testing the difference between the means of the two representative samples, We are testing the whole that the two population means are the same. We have removed confounding factors, any deviation from the population mean in then due to the factor under test,

So that it is clear, we do not test for the factors you list. We generate two randomly selected representative samples populations. That includes demographics. That's what representative means, but you do not have to compare]/b] these factors if the sample means are representative of the population. What we compare is the means of the two sample populations. Differences present are assumed to be due to the factor under test.


CONCLUSION
This rough proposal is certainly within the normal range of problem types and solution methods. There is nothing especially difficult inherent in gathering data and testing it. There are problems with every study, but there are no more problems inherent in this proposal. It is simply wrong to say it cannot be done, If this cannot be done, few other studies would see the light of day.