Because I have been so vocal in my criticism of the CSSGB test that we took on Saturday, I’m making an equally public statement admitting that apparently, I was wrong about a few important things.
I love learning new things, and I generally have little trouble assimilating and applying new information. With adequate preparation, I usually test well. So when I leave a test that seems to be testing for something other than the topic I’ve just spent 12 weeks immersed in, it freaks me out. I was convinced the test had validity problems.
Maybe not.
ASQ certification exams use a “cut-score” process. It takes 550 points out of a possible 750 (approximately 73%) to pass. The Cut Score Process determines the passing score, and Scaled Scoring allows adjustments for exam difficulty on subsequent forms of the test. To oversimplify, the cut point for a passing grade is determined by a panel of twelve to fifteen subject matter experts, who select test questions by the criteria that eighty to ninety percent of qualified SSGBs should know the correct answer. Scaling is accomplished by “equating,” in which older questions and new questions are included in the exam. The mean student scores for the older questions are compared to the current test scores as an indicator of the current test difficulty, and the test is “scaled” accordingly. In other words, “If [candidates] get a hard test, they won’t have to get as many questions right to meet the standard. If they get an easy test, they will have to get more of those easy questions right in order to meet the standard.” The complete explanation for the test development and scoring processes is here.
I can attest that there were a few questions included in our exam that were very close to questions in the Primer, but what about those infernal nonsense questions that have multiple correct answers? Questions that were not only unfamiliar, but defied my intuition and logic? From Govind Ramu’s article, “Test Run: Improve Your Chances of Passing an ASQ Certification Exam,” I draw a clue: “The questions require application of knowledge and experience.” Added to the cut process, where questions are selected by SMEs to reflect the body of knowledge possessed by active practitioners, it is clear that students like Kay and Mark, who have real world experiences with which to anchor these principles, are at an advantage. Those are the students ASQ is seeking for certification.
Students like me, with far less concrete experience in QC and faded memories of statistical and algebraic principles, are at a disadvantage. Beyond my personal limitations, I think that the classes we took together are more accurately described as review. In my degree studies, I spent an entire semester on SPC (Statistical Process Control) – which we did not even get to in LSS Advanced Topics because we ran out of time. It also seems to me that we did not have nearly enough practical application of concepts.
For instance, it would have been useful to have more problems that illustrated various ways data can be presented to a CSSGB, and how to properly select the correct distribution or chart to analyze those data. Of course, I understand the resistance to spending an additional $400 and another six weeks on yet another course. But I think the presentation of additional practical applications would significantly improve student comprehension. After twelve weeks of focused study, my single biggest problem is this: when I am presented with data, as in the test questions, I often don’t know what I am looking at. What formula or distribution am I expected to use? Only experience and/or additional instruction will cure that problem.
The range and amount of typographical and content errors in the handbook did not help, nor did the differences between the handbook and the Primer. There were two different editions of the handbook among our class bearing identical ISBN, publication and edition numbers, but containing different content. It is beyond ironic for a textbook with quality control as a topic to have so much errata – it is outrageous. But I also confess that my reaction to this might be exaggerated because of my frustration with the obstacle to learning those discrepancies presented. After a few weeks have passed, I’ll get the book out again and see if I still feel the same way.
And of course, you gotta love test questions that test your ability to see through legerdemain, not your grasp of statistics. Or questions that are based on the subjective judgment of the author. And there is something patently ridiculous about testing people about their ability to calculate variances on a calculator. Yes. A freaking calculator. As in grammar school. Using a calculator in this type of exam tests my ability to avoid fat-fingering the entries, not so much my ability to derive descriptive statistics. The only time you need a calculator in Quality World is when you sit for one of these tests, because everyone in the real world loads their data into excel or Minitab and presses “enter.” Strangely, my TI-68, purchased in 1989, was disallowed by a proctor. Fortunately, I expected the worst and brought two calculators. One by TI, the other by Fisher Price.
I wish that I had known, before I spent two grand that I did not have, that certification requires more practical experience, and that I had really taken the time to figure out that this set of courses is inadequate preparation for certification. Mind you, there was no deceit here: Katharina said more than once that the courses at EvCC are not intended to be preparation for the exam. I saw the course listings for something that keeps appearing in listings for jobs I want to apply for, and I got excited about the prospect of adding that credential to my resume. It’s that simple.
Having said all of that, I have come to genuinely care for my fellow students and instructors. I enjoyed their company and camaraderie, and I dearly hope that I left everyone with something of value as well. It was all time and money well spent in a broader way – I made valuable acquaintances and learned a number of things that have application to my life in unexpected ways. For example, the 5S principles have enabled me to focus on some elements of personal organization that have eluded me since I moved to the PNW. And being reacquainted with algebra and stats has given me a wakeup call about how much facility I have lost in those areas.
To think that once upon a time, I actually passed calculus. After I completed the exam, I went back over all the questions, and changed twelve answers. So if my previous history with exams holds true, my best possible score is less than 85%. Right now, you should have a mental image of me, head bowed in a face palm, groaning softly.