Purpose:
To obtain an acknowledgement from a CFS scientist that even though he would exclude data outside (the Maximum Permissible Error in Ontario) 90 to 110, in calculating precision, at least one police service in Ontario is using data outside 90 to 110 in calculations of precision during periodic inspection.
The point is that even though the Crown's expert has opined that data (at 34.0 +/- .2 C) outside the +/- 10 mg/100mls acceptable values should be excluded from any calculation of precision / standard deviation for the instrument before the Court, another police service uses a methodology where values below 90 are included in calculation of standard deviation during periodic inspection. Quaere: Is there no metrological supervision of police services in Ontario? Are police in Ontario free to conduct their own periodic inspections of instruments without supervision of their methodology? Is the police service, in the matter before the Court, free to keep an instrument in service, for a forensic purpose, without regard to whether or not its precision has drifted unacceptably?
Q. So, it would be completely improper if during a periodic inspection someone was using a data point of 88 during that periodic inspection? I’m showing you sir, a periodic inspection March 29th, 2012. This is South Simcoe Police Service. MR. O: And what is that? I don’t understand how this is relevant. A document from the South Simcoe Police. MR. BISS: It’s important Your Honour, because
I’m going to ask the witness about the methodology, again, that’s used by the Intoxilyzer 8000C, the software, when calculating standard deviation. And in connection with that question, I mean, I thought a long time ago we heard from Mr. P, his perspective that readings that were outside of the acceptable range would be excluded and just couldn’t be used. THE COURT: Yes. MR. BISS: Right? My point is that maybe that’s not exactly correct. THE COURT: Well, it’s.... MR. BISS: And I have an example to put to him. THE COURT: Sure. Well, let’s hear it Mr. O then you – you have the right to reexamine. I’m not sure how relevant this is because I’m not sure I’m completely comprehending exactly the import of what we’re talking about, but I’m sure by the end of time here I will. So.... MR. O: All right, thank you Your Honour. THE COURT: Okay. MR. BISS: Right. Q. So, I’m showing you a periodic inspection report from South Simcoe Police Service. March the 29th, 2012. A. Yes. Q. And the page – one, two, the fifth page. The last page there’s a calculation of standard deviation during that periodic inspection report. And what were the test values that were considered by the instrument? A. All right, so the values that are listed as
the calibration check using the wet bath simulator are 88, 91, 92, 92, 93, 93, 94, 94, 94 and 95. And so the officer who conducted this – I can tell you their badge number but I can’t read their signature. Yes, so in my estimation that 88 should not be included and unfortunately – and I don’t mean to be you know, demeaning to police officers, they don’t often have a lot of science background or mathematical background for doing this kind of calculation. THE COURT: So, they’re like judges, really. I think. A. I’ll stop there. And so, unfortunately, the instrument cannot remove a point from the calculation. It just does whatever values are in there. And what should have happened here is, the officer should have repeated it again such that all 10 values that were obtained were within the acceptable range. Now here, the average is 92.5999 with a standard deviation of 2.0110. So, within the acceptable range and again it appears here as in this case, that a lot of the calibration checks are the low end of the acceptable range. MR. BISS: Q. So, the question is, why is it that for purposes of calculating precision of the instrument, because that’s after all, what we’re doing by assessing standard deviation, why is it that when you’re assessing precision of the instrument that it’s appropriate to use a 91 or a 92 but not appropriate to use an 88? A. For the purposes – the fact that again, you use only the – the instrument will only allow testing to proceed if the calibration check result is above 90 or less than 110. Q. But what’s that got to do with precision testing on the instrument? A. The expected result in this case is
supposed to be 100. Q. Yes. A. And the acceptable range is 90 to 110. Q. Yes. A. All right? These values are low but acceptable, but that 88 is not acceptable. Q. But if we are.... A. And so therefore it’s an outlier and therefore should not be in the calculation. And the officer would either have to know how to do the calculation manually, or should have – and I’ve had another case like this – where he should have repeated the whole thing. Q. Now, I thought.... A. And he must have missed that – or she, I don’t know. Q. I thought outliers were data points that were 4.5 or more standard deviations away from the mean. A. It’s anything that’s outside the minimum or the maximum in most cases. But in this case, for the purposes of this, anything that is outside the acceptable range is unacceptable. Q. Well, it’s unacceptable in terms of testing. I mean, I suppose that you or I might be critical of whoever it was who did this stability but the point is, that until we know what the cause of that unusual result, what that low result is, we can’t just assume, we can’t just speculate as to what the cause was of the low cal-check or the low cal checks that affect the precision, affect the calculation of standard deviation. A. In my opinion, that result should be excluded from the calculation so you would use the 9 points. I had a case involving an O.P.P. instrument where the exact
same thing happened and there was a comment from another expert, not the one in this case, who commented on that and said um, as a result of the low calibration check, that it caused the standard deviation to be in excess of what it should be for the approved instrument status. Right, just like what we’re talking about now. But when you remove that point, then all of a sudden, the instrument, the standard deviation became acceptable because that point was removed. Q. I mean, it seems to me that we need to be looking at this, at this point, not from the perspective of the police officer but we need to be looking at it from the perspective of has there been a change. Leave – there’ll be an argument about whether it’s significant or not. Has there been a change in the precision of the instrument over time? If that’s the issue that we’re trying to determine, the fact that there may be a guideline or a practice from the Centre that says the cal-checks need to be 90 to 110, I’d suggest to you that’s irrelevant in the determination of precision. A. There’s a well-known phenomenon in the scientific community, and in breath testing, that when – often when the first calibration check is done, it’s often outside the acceptable range. Q. Yes. A. And there’s been – I’m trying to remember the name of the publication – that if that were to occur, that point should not be included in the calculation. Q. What calculation? A. Right? So, basically the calculation of the standard deviation and the mean. Right? So, basically – sorry, I’m trying to think. I don’t know if it was Kurt Dubowski (ph) or someone else, but essentially that – if that first calibration check is outside the acceptable range
because of a known phenomenon that we know about, with respect to the first calibration check that’s done, because possibly the head space is not up to proper operating temperature, that that data point would be excluded from the calculation.