Sex and Cash

I'm from Walkerton

...and if you're Canadian you probably know what that means. Don't drink the water eh?

Well, yeah. Except it's been nearly 5 years since the outbreak (of E coli from an improperly fixed well, a massive rain storm and a tangle of factors including insufficient chlorination of the water that killed 7 people and sickened some 2,300 - for those just tuning in)

And I haven't found a cheerful way to spin the subject yet. It was awful and many people still suffer. But I will relate my words in hopes that it will make us all think about it. For though the book is closed on the inquiry, jail sentences have been set, and most people think of it as over and irrelevant, from my perspective that is far from the case.

Even the most embittered and devastated E. Coli survivor would agree that if the citizens of Walkerton had one voice, it would wish that no other town ever go through what they did. A thorough root cause analysis is necessary to understand the origins of the disaster, which we clearly have to do if we want to make lasting changes that will truly prevent something like this from recurring.

The correct verdict for the implicated peons (in this case the Koebel brothers, though it could be argued that it could have been anybody) is certainly important, but equally necessary though less obvious, is an analysis of the system as a whole. This is the only way to serve and protect the people whose interests require tending well into the future.

To explain why my brain goes that way, let me switch into a work analogy. Our mandate, as a company, is to send a prescribed number of geronimators (a hypothetical widget invented by a friend of mine) with no impermissible defects to our customer on a daily basis. We have a set companies who set us the components (collectively known as a supply chain) and set of inspection procedures, errorproofing measures, machinery and operators (collectively known as a process) designed to help us achieve this mandate.

Let's say a screw on one of the clamps on one of our inspection machines comes loose and wedged on the face of the clamping pad. Now the screw is positioned so that it imprints each geronimator housing with a little dint. (This is an impermissible defect according to one customer, but another would likely let it pass. So here we have the concept of the relative nature of a given standard - not exactly my point, but definitely worth considering in the overall scheme of defective product identification). The dinted product continue running through the assembly line. The dint compromises the seal on the leak testing device on our assembly line, but the volume of air escaping is sufficiently small that each unit still receives a 'pass' signal from the machine. The dint is undetected by the unloading person's visual check, who is busy looking for a different defect on a different part of the geronimator.

Now the dinted product is being loaded into the carry totes by the thousand, and being shipped to Illinois to be assembled onto engines at the customer's plant. The first 500 dinted units are unpacked and assembled there without notice, but when the first dinted geronimator hits the equipment that tests the leak rate of the whole engine, each begins to fail. Customer workers stop their assembly process and discover the dint on the product that we shipped to them, and pronounce it the root cause of the failure. Several things now happen:

1) Sorting: a) Customer workers begin a manual inspection of all product in stock at their facility to determine whether they have the same defect.
b) Our workers begin a manual inspection of all product in stock at our facility to determine whether they have the same defect.

2) Containment: Appropriate measures are taken to separate good pumps from bad pumps.

3) Charging: We are charged for the labour involved in 1a) and b), 2 and sometimes an additional fee. (For GM it is $10K automatically for a new Quality issue).

4) Root Cause Analysis and Elimination: We analyze the defect and brainstorm on where it could have been created. We perform a check on the assembly line, find the loose screw and tighten it.

5) Systems Review: We assemble a team of process experts to investigate the root of the problem to ensure it never happens again. This team would examine such questions as: Why did the screw come loose in the first place? Why did we select that type of screw? Where was the vibration coming from that made it loosen? Why wasn't the loose screw detected? Why wasn't the dint detected by the leak tester on our assembly line? Why wasn't the dint visually detected by the operator? Was there not sufficient lighting? Not sufficient time? then follow up to correct them. Through this we might learn that the operator should have caught the defect, that the leak tester's parameters were mis-set, that the maintenance mechanic's failure to do preventive maintenance on the assembly line that day resulted in the screw coming loose. Most often, a combination of factors is found to be the root cause of the problematic outcome.

6) Documentation and Analysis: The findings of 4 and 5 are written down and communicated to the customer. They are interested in 5 more than 4. The findings are also reviewed periodically within our company to allow designers of new processes to avoid similar issues in the future.

If we had a nurse or development worker in this conversation, he/she might automatically gravitate toward #1,2: emergency relief, getting the problem under control. My lawyer friend Sean's concern rested nearly solely on finding the right sentence (punishment fit for the crime) for the responsible party - which makes sense, since he comes in on #3. As a Continual Improvement/Manufacturing Engineer, I am chiefly involved in #4, 5 and 6, so I switch into that gear automatically when considering any problem.

There are always new product flowing out the door; regardless of what the problem was, it needs to go away. After the old tap fixtures are being disinfected and the E coli-stricken are released from being cared for in hospital (#1, 2), the wrong-doers are sentenced (#3) and the issues are probed in a public inquiry (#4). Unavoidably, new water is soon flowing through the taps. The only way to guarantee (or maximize the chances) that the water is safe is through rigourous and extensive and blameless and tireless pursuit of #5. Then make sure everybody knows about it, all the angles, in #6.

To these last steps, determining who's responsible is only useful to the point that now we now how to fix it - and determining which consequences are appropriate is only useful to the point that their administration truly prevents the problem from recurring. This is the frame of reference from which I approach the situation, and the basis on which I base my fear: focusing on Stan and Frank Koebel is the equivalent of yelling at the operator who missed the dint and closing the investigation right there. It provides us no safety, no comfort, no real solace because there no assurance that it will not occur again. The screw could vibrate right out of the machine again and the ride begins anew - with a different schmuck taking the fall, but no real progress made. And to me, both as a citizen of Walkerton and a citizen of the world, that outcome is simply unacceptable.

posted by ejdl @ 2:35 PM