Purpose:
To explore the differences between a measurement instrument that has a linear relationship between the thing measured and the result v. a measuring instrument that has a calibration curve.
To search for an admission that the calibration curve can shift up or down, left or right, or rotate around a point over time.
To cross-examine the CFS expert on the lack of empirical study supporting the ATC/CFS hypothesis that a single point control test can always detect drift.
To obtain admissions about the differences between new instruments and aging instruments.
[Notwithstanding the parallel universe comment by the Court below, it is respectfully submitted that an understanding of these concepts is essential to a proper understanding of scientific reliability of approved instruments. What are the limitations of control tests in establishing reliability? Is one parallel universe that of junk forensic science and the other parallel universe that of empirically-based measurement science?]
MR. BISS: Q. So, sir, we seem to have a difference in principle between your perception, or your perspective I should say. A. My opinion. Q. Your – your opinion and that of Mr. Kupferschmidt, specifically as to what is a calibration check or a control check, and my understanding is that the control checks that are being run in Ontario these days by police officers at time of use, are control tests at 100 milligrams per 100 mils. A. That's correct. And it was like that before with the Intoxilyzer 5000C as well. Q. That’s not a control test at 120 milligrams per 100 mils or 50 milligrams per 100 mils. A. Correct. Q. And so, the question is: how is it that a control test, and I think this is the big difference between your opinion and Mr. Kupferschmidt’s, how is it that a control test at 100 milligrams per 100 mils can check the reliability of the calibration curve on an instrument that has a measuring range of 0 to 600 milligrams per 100 mils? A. That has to do with the curve that you were trying to describe earlier, with respect to when the instrument is calibrated, and so you’re using a single point in this case, a calibration check at 100 milligrams of alcohol
in 100 millilitres of blood to see whether or not the calibration of the instrument has drifted from its setting or at time of manufacture. All right? And the Intoxilyzer 8000C and it’s predecessor, the 5000C has never been shown, scientifically in publications or in presentations to show that the calibration of the device drifts. There’s the potential for that and that is a check to make sure that that calibration hasn’t changed, again, since the time of manufacture, or since last service. Q. Well, here’s my problem. If we were doing – dealing with a linear – straight line calibration curve, I understand your opinion. A. Yes. Q. I mean, maybe the straight line has shifted up or down on the graph, but I want to respectfully suggest to you that it is impossible to check changes to a curve across a 0 to 600 milligram per 100 mil range by only looking at one point on the curve. A. Well, that point – the entire curve would shift if the calibration of the instrument changed. And so that’s how you can check the calibration and see whether or not it has drifted or changed from that single point. Q. So the whole.... A. And that’s been the basis for checking calibration of a device, whether that’s the Breathalyser 900A, or an Intoxilyzer 5000C, and now the Intoxilyzer 8000C.
[Defence lawyers need to ask more questions about the lack of empirical studies that substantiate that "basis". Why is single point control test the basis? Is that basis police apologetic "science" or is it based in real science? Is it junk forensic science that should be challenged by ]
[The following part of the cross-examination contemplates a hypothetical, "what if" calibration of the instrument has changed since the last full calibration by the manufacturer or Canadian authorized service centre]
Q. So the whole curve would shift up or the whole curve shift down? A. Correct. Q. So that the new curve would always be parallel, if I guess, I – I’m not sure that’s... A. Yes.
Q. ...the right wording, ‘cause we’re talking about a curve, or two curves. A. Correct. Q. You’re saying that the new curve would always be parallel to the old curve? A. That's correct. Q. Now, where is the.... A. Or it could shift in some other kind of manner, but that would be obvious by the standard when it was run, that you would see a gross discrepancy in the result being outside the acceptable range. And that’s why there is that range associated with the calibration check, because there are variables associated with why that result may not be exactly 100.
[To the best of my knowledge there is no empirical study that supports this hypothesis of "you would see a gross discrepancy"]
Q. But the whole curve could shift around the 100 level. In other words, the curve could rotate clockwise or counter clockwise? A. Hypothetically, yes. I’ve never heard of that phenomenon. It’s never been presented. Q. So, where’s the scientific literature to suggest that in these circumstances the whole calibration curve would fit up or down in a parallel fashion? A. I’d have to think about that. My apologies, Your Honour, it’s just not something I’ve had to think about for quite some time. MR. O: Should we take the lunch break? THE COURT: Yeah, we can give you time to mull it over. Can I just – maybe just so my own thought processes can ruminate on this interesting topic over lunch, a speedometer on a motor vehicle, if you were to calibrate speed say, at 50 kilometres an hour and its –
depending on the measuring device – confirmed that that speedometer is accurate, that it’s measuring accurately the 50 – at 50 kilometers an hour, then presumably you could infer from that, that it would also accurately measure at a higher or lower speed. Now is that a linear – an example of linear calibration? A. Yes. THE COURT: Okay. A. ‘Cause it’s based on the tire rotation speed. THE COURT: Right. A. And how fast the.... THE COURT: There’s no curve – there’s no calibration curve involved here, ‘cause we’re dealing – is that correct? A. In that example? Yes. THE COURT: Right. So, I gather there’s some distinction to be drawn here, which is, I’m coming to understand in my crash course on scientific theory that I’m receiving here as a result of this discourse, that the calibration curve in an instrument like the Intoxilyzer 8000C can move? The curve that originally existed when the machine came into operation that was confirmed against a set standard, that may vary over time? Is that – is that an accurate understanding of – I appreciate there’s a calibration every time you use the machine, there’s a calibration test. A. Check, yes. THE COURT: Or, check. And that’s – that’s
confirming, as I understand it, that the device is capable of measuring a set standard within an acceptable range? A. Yes. THE COURT: But that the curve that I’m hearing about, may not remain constant. This calibration curve from what existed at the time the machine first came into use. Is that correct? It can vary? A. That’s correct. It’s called drift. THE COURT: Right A. But there’s never been anything shown by the manufacturer or by any expert anywhere, or presented or published that shows that the calibration of the device shifts. And that also applied to the Intoxilyzer 5000C. THE COURT: Even though there’s this concept of drift, the – the measurements remain reliable and accurate... A. Correct, and that’s.... Q. ...over time. A. That’s the reason for doing the calibration check is to ensure that there hasn’t been any drift over time. THE COURT: Right. A. And that’s the difference between a new instrument and an old instrument. THE COURT: All right, well, maybe you can consider the parallel universe, that was referenced in the question over lunch and let’s carry on at 2:15.
[Below please see an example of an old 5000EN instrument where the calibration curve has shifted over time. It needs to be re-calibrated. However, it will eventually (after many tries) pass its start up diagnostics and produce a good single point control test at 100.]
Note for the highly-motivated: Although the cross-examination above discussed a shift in the calibration curve, strictly speaking that's not correct terminology. What we're talk about here is really shift in instrument response over time - the new (over time) correct calibration curve for the response doesn't match the programmed (at the factory) old calibration curve. The old calibration curve may have been correct all those years ago, but over time there needs to be a re-calibration to establish a new and correct calibration curve. The cross-examination above makes sense if you understand this difference in terminology.
See blog entry "Why Response Drifts Over Time", where this cross-examination continues. See blog entry "Reliability - Calibration is a Curve" for more explanation.