Change at One Point but Not at Another Point that is Equivalent
Updated: May 1, 2022
This is a sample excerpt from a cross-examination of a Crown expert using data from periodic inspection of an approved instrument. The multiple-point control test data from an annual periodic inspection in Ontario or from a re-calibration as in Vallentgoed can be very useful to the defence in cross-examing a breath tech or a Crown expert to identify drift in accuracy and precision over time.
To suggest that there is empirical evidence in the matter before the Court of change in response by the approved instrument at the lower end of the measuring interval (reading much lower) and at the upper end of the measuring interval (reading slightly higher) without corresponding change in response at the single data point control test level.
To suggest that linearity has therefore changed, and in other words, the calibration curve programmed into the instrument, at the factory or on the last re-calibration, is no longer reliable.
To lay the foundation for the O'Connor order.
To connect the lack of multi-point calibration of an approved breath instrument in evidentiary breath testing with the lack of adequate multi-point calibration of the Elisa screening system (lack of multiple (5) calibrators) used in drug/hair testing - the confusion of quantitative and qualitative analysis - criticized in the Motherisk Inquiry report.
To challenge the evidence of the government expert as to the adequacy of single point control tests to establish linearity in an instrument, that has not recently been re-calibrated across the measuring interval.
To suggest that the empirical evidence in the facts of this case - the annual inspections - supports the inference that the calibration curve has shifted, rotated, or stretched (or more correctly that although the learned calibration curve was fixed at time of last calibration, it is now different -shifted, rotated, or stretched, over time - from what the correct calibration curve would be if the instrument was properly re-calibrated) i.e. accuracy and precision have drifted over time.
[sample excerpt from cross-examination]
Q. I want to suggest to you, that inevitably, what you’re saying is that an Intoxilyzer 8000C that has been checked in this way is conducting a qualitative analysis, not a quantitative analysis. A. That's correct.
[in fairness the witness may have misspoken, or misunderstood the question]
Q. And that’s because you’re checking it, whether or not it is above or below a different level, a particular level of 100.
A. Yes, because there’s no scientific reason why the calibration curve should change at the high end or the low end and not in the middle.
[but see Beer-Lambert law over-simplification, bandwidth of filters may change over time, instrumental deviations, linearity studies are done on new instruments]
Q. So, if an Intoxilyzer 8000C – if we aren’t confident of the linearity of the Intoxilyzer 8000C and all we have is our calibration checks at 100 milligrams per 100 mils we – and we conduct an analysis on an 8000C we are conducting a qualitative analysis. A. Sorry. Can you repeat that? Q. Well, as I understand it, in order to have a quantitative analysis, you need to have an instrument which is able to reliable – reliably reach – provide an indication of a concentration at 50, 70, 90, 120, 130, 150 right across the measuring interval of the instrument. If it’s going to be an instrument that is capable of a quantitative analysis, right? A. It is a qualitative – sorry, it is a quantitative analysis because you’re checking the instrument at one point. The rest of the curve is not going to change. In addition to that, you also have the I-T-P which is an
internal electronic check of the instrument, to make sure that, again, the calibration of the device has not changed. Q. But if the....
A. And the response has not changed.
Q. But if the instrument’s response to real values has changed, although the calibration curve is saved in the memory of the instrument, then we still have a problem with respect to linearity of the measurement. So, we may get, for example, results in the 50 or 40 range reading too low, and results in the 150, 200, 300 range reading high. A. That’s not my experience, so that’s not something I can scientifically be able to say that that would happen. I’m not aware of how that could happen. How you could have a change in response at one point but not at another point that would be equivalent. Q. But here, we have an empirical indication, right in the case before us, of an instrument that seems to be reading low at 40 or 50 milligrams per 100 mils, is reading close to but maybe a bit lower at 100 milligrams per 100 mils and is reading maybe slightly high at 300 milligrams per 100 mils.
A. That’s one interpretation, yes. Q. So, an inference might be drawn from that information that we need more information to try and check out the whole question of the linearity of this instrument because reliability of its measurement properties across the whole measuring intervals, that reliability may be compromised. A. Sorry, again, one more time? Q. Well, my suggestion to you sir, is that if we have some indications in the annual periodic inspections... A. Yes.
[graph and spreadsheet created by cross-examiner from data obtained from periodic inspection made an exhibit]
Q. ...that the instrument is reading low in
the 40 or 50 milligram per 100 mil range; it is reading slightly low at 100 milligrams per 100 mils, and it’s reading at or slightly higher than 300 milligrams per 100 mils using 300 test solution, then the question – the whole question of linearity of the instrument is of concern to us such that we need to dig deeper to try and find out whether or not this instrument has incurred drift in accuracy and precision right across the whole measurement interval. A. And I’m not sure what kind of information you would use to determine that. Q. Well, I suppose if thorough periodic inspections were being done, with numerous values at each of 50, 100, 300, maybe other numbers, every year, then we would have some more information about just that. Right?
A. You could, yes. Q. But here we have.... A. But other historical information such as the information from other breath tests that were done at other times has no implication on the linearity of the instrument.
[perhaps he's right on the latter point with respect to linearity, but data of control tests over time is useful in checking for drift in accuracy and precision over time, it is the annual maintenance reports and contemporaneous documentation that tell us about changes in linearity ]
#calibrationcurve #autocalibration #linearity #accuracy #precision #reliability #learned