Deejay Systems began out of a moment of anxiousness. While in graduate school working on the development of a biology course for non-science majors, the need for accountability became apparent.

The course was based upon nearly three hundred student interviews conducted over an academic year. Essentially, students who were enrolled in earlier versions of the course were asked for input. Upon that, objectives were drafted and instructional methods were devised that were intended to measure up to the student expectations. In addition, test questions intended to measure student achievement were written. All in all, the development of the new course followed the traditional educational model. Indeed, it was praiseworthy. However, as put into place, the feedback mechanism of tight scrutiny, specific analysis, and strict accountability that was needed to provide credibility was missing.

By chance, I came upon a publication by Russell Ackoff and others,1 that included a reference to the resource maximization tools that were developed by the British during World War II. Later, these tools were combined with the scientific method to organize and optimize the effort of an interdisciplinary instructional research team. The adapted method became known as “Operations Research.”

In another instance, the British resource utilization tools were adapted by the United States Department of Defense in the development of the Polaris project. It was implemented as an economic instrument known as PERT, Program Evaluation and Review Technique. One element focused on sequential relationships and the interdependencies of each process step. It was a method for planning programs in relation to objectives and re-planning research events and activities.2

The Critical Path Method, CPM, also evolved from the British maximization tools. With this adaptation, management became effective in the control of defined and organized process implementation.

The application of an eclectic version of these management techniques and methods enabled a continuous improvement cycle. That which emerged was as follows:








According to this defined construct, every element in the instructional sequence could then be teased out and thoroughly analyzed for its contribution in the teaching – learning enterprise. Every element became noticeable and every component could be evaluated for its effectiveness in teaching and for a proportional learning efficiency. The entire package evolved into a systems analysis approach applied to the design of instruction.

It worked. Using this systems analysis tool, absolute accountability in instruction became possible and impersonalized. Tinkering with elements in the instructional process was correctly the responsibility of the teacher instructional designer. The underlying question would always be, “What components in what series would most effectively and efficiently bring about the attainment of the prescribed goals?”

The answer was at least in part dependent upon the worthiness of the evaluation instrument. Solid test and measurements statistical tools were needed for the assurance that designed systems were in fact leading to desired outcomes.

It had to be rightly established that the test instruments and evaluation tools were reliable and valid as well as provided reasonable difficulty and adequate discrimination among the test takers.

Foremost, the evaluation instrument had to be reliable, that is trustworthy. The score obtained by individuals taking a test had to provide confidence that the results reflected real performer ability free of chance error. The Kuder-Richardson statistical method was utilized. Also, it was determined that a test with a (0.7 or higher) reliability coefficient would be considered valid. Furthermore, it was established that the overall validity of an evaluation instrument depended up the validity of each of its separate items.

Now, the discrimination power of an item was recognized as a correlation between the percentage of individuals in the upper half of the population correctly responding to any item compared with the percentage of individuals in the lower half correctly responding to the same item. If the percentage of both groups were found to be equal, then the discrimination ability among the individuals would be nil. Obviously, there would be no discrimination if the item did not differentiate between upper half and bottom half within the group. If the percentage of the upper half were greater, the item would be positively valid in its discrimination. Whereas, if those individuals in the lower half more frequently responded correctly to an item, its discrimination would be negative. The item would be of no value. The item would offer no insight into the capability of the instructional sequence.

Difficulty of an item determined by the number of individuals responding correctly was viewed as useful in providing some insight into the structure of any item. Within its context, it was seen as contributing a measure to the average score of the individuals in a group. Total mastery or absolute perfection might be welcomed but then would that be because of the instruction or because of quirk in the evaluation item that enabled every individual in the population to respond correctly? Furthermore, the item would have no discrimination power. And, on the other hand if too few responded correctly would that indicate failure? Again, was some quirk operating that obscured the results? Much had to be considered and much was considered in the design, implementation, evaluation, adjustment, and recycling of instruction sequences and the evaluation instruments.

The bottom line is, developing a system of accountability enables the right disposition in the design of instruction and its ongoing improvement through systems analysis.

These systems analysis applications have been successfully utilized in maximizing instructional effectiveness and learning efficiency.

1 & 2 refer to dissertation p. 15.
Another version refer to Priority p. 13. Recalling a history class.

Creation date : 14/11/2005 @ 18:06
Last update : 15/11/2005 @ 15:31
Category : Deejay Systems
Page read 500 times

 Print preview      Print the page