Purpose:
To separate conceptually control tests from calibration
To obtain an admission that there are no "assessors" in the province of Ontario
To obtain an admission that the historical maintenance records show a deterioration of performance - deterioration of linearity in this particular instrument
The strategy of using control tests as indication of reliability depends upon completeness of records and transparency
To obtain an admission that at the bottom of the valley in the infrared spectrum at approximately 9.4 microns, your detection limit will certainly be degraded and precision will also suffer in most instances, if you are no longer taking the reading at the absorbant’s maximum.
To obtain an admission as to what may happen if the detector becomes dirty.
Cross-examination of CFS scientist on United Nations document reviewing the basics of ISO 17025:
MR. BISS: Q. The guidebook called “Complying with ISO 17025”, a practical guidebook.
A. Sorry. Sorry, just give me one second here. I'm trying to reorganize everything. Okay.
Q. And I wonder if you could turn, please, to section 8.4. and – that has to do with service and calibration schedule.
A. Yes?
Q. And I, I want to, I want to expand on the whole question of in-house checks as opposed to more detailed kinds of checks. The first paragraph there says that:
Before introducing any piece of equipment into service, the management should decide upon a service maintenance calibration and performance-checking schedule. This will normally be a combination of service from the supplier of in-house checks and calibrations.
Do you think that’s consistent with the evidence you’ve just given?
A. Yes.
Q. And then on – in the third paragraph, it says “Servicing and preventative maintenance should be as recommended by the manufacturer who may also be able to carry out calibration checks and adjustments.” Do you think that’s consistent with what you’ve said so far?
A. Yes. And I think if we go back to the first paragraph...
Q. Yes.
A. ...it says:
There’s no need to have outside service or service contracts but there would be a general onus on that laboratory to satisfy any assessors that the arrangements are adequate to ensure proper and reliable functioning of the equipment.
Q. So there’s an onus on the laboratory to satisfy any assessors?
A. Yes.
Q. But that assumes that there are some assessors?
A. Correct.
Q. In the...
A. Which there aren't in the province of Ontario or Canada?
Q. I'm sorry, there are not.
A. There are not.
Q. There are not any assessors in the province of Ontario or in Canada. In the particular field...
A. That’s a quality assurance issue.
Q. In the particular field that we’re dealing with, there are no such assessors?
A. Correct.
Q. On the fourth paragraph...
A. Sorry. Can you go back to the, the third paragraph, servicing and preventative maintenance?
Q. “Servicing and preventative maintenance should be as recommended by the manufacture who may also be able to carry out calibration checks and adjustments.”
A. Correct. And we know that CMI does not make any recommendations about how often the instrument needs to be calibrated.
Q. Yeah, on the Intoxilyzer 8000c?
A. Correct. And they never did with the 5000c either.
Q. They – but they have with the – with respect to the Intoxilyzer 9000?
A. I have no idea about the 9000.
Q. Okay. Before then -
A. But with respect to this case here, right, that the instrument was calibrated, originally, when it was manufactured and then again on – in 2011 by Thomas Electric.
Q. There was an inspection?
A. Inspection.
Q. Yes.
A. Okay. And then subsequent inspection. And, again, there’s no schedule for calibration of the device, unlike there is for the 6810 – the Alcotest 6810, which the manufacturer says it should be recalibrated every six months.
Q. Yes.
A. And of course it has a lock-out specification that if it reaches six months that the instrument can't – the device can't be used.
Q. Right.
A. But the Centre of Forensic Sciences and the Alcohol Test Committee have recommendations about verifying...
Q. Yes.
A. ...right, the calibration of that device and doing an accuracy check.
Q. Yes.
A. Right? Either every 31 days or at least every 14 days or 15 days, right?
Q. Yes.
A. And if that verification passes, then the instrument is deemed to have – sorry I keep saying instrument, the device has maintained its calibration and therefore can continue to be used for that period of time, whichever police service is using whatever date, whether it’s the Alcohol Test Committee or the Centre of Forensic Sciences, up until the time of six months.
Q. Yes.
A. At which point it has to be recalibrated, but there is no such requirement for the 8000c.
Q. We just don’t have that system in Canada or in Ontario?
A. Correct. It’s done as needed or as required based on the performance of the instrument in the field.
Q. As a matter of policy, not as a matter of business?
A. As a recommendation.
Q. So it’s only a recommendation and as you just pointed out a moment ago, we have no assessors that the users in-house of this equipment have to be responsibility to?
A. That is correct.
Q. Fourth paragraph:
In-house checks on equipment should be scheduled to cover the gaps between any service visits and calibrations. Some equipment is effectively checked each time it is used by means of reference samples which are run as part of quality control.
Now I think that’s consistent with the evidence you’ve given?
A. Yes.
Q. But the point that I would make with you and ask you about is there’s a comment there about reference samples, it’s plural. It’s not just one reference sample, it’s reference samples which are run as part of quality control. Right?
A. That’s what that reads. Your interpretation is that it involves one or more concentrations or it could be interpretation that more than one of the same concentration are used to assess that. Again, ‘cause in the United States, many police services, who are under various bodies of jurisdiction, don’t do calibration checks as part of each and every breath test. They do monthly proficiency tests, so unknowns that they test to see whether or not the calibration of the instrument has changed at that point...
Q. Yes.
A. ...and the accuracy and precision of the instrument. But the calibration of the instrument is checked each and every time a breath test is conducted and twice during each breath – or sorry, during each subject test, if two tests are obtained, and an additional one prior to that to see whether or not that calibration has shifted again from when it was last set.
Q. But our quality control system that the police are using only uses a sample, a reference sample, at 100 milligrams per 100 mls?
A. That is correct.
Q. Not across the measuring interval?
A. That is correct.
Q. All right. The next paragraph, next sentence says: “Maintenance and calibration are then carried out on an as needed basis when these checks show performance deterioration.” Now in the particular matter before the Court, we have indication, in 2012, in 2013 and in 2014 of performance deterioration during the periodic inspections.
A. That’s what appears to be occurring, yes. Could there be another explanation for that? Possibly, yes.
Q. All right. So we have an indication of performance deterioration, but the point is that nobody, notwithstanding those indications of performance deterioration, it does not appear that the instrument was recalibrated until after May of 2015; after May 20th, 2015.
A. That is correct.
Q. So in this document it says, this is the next sentence, this is a perfectly legitimate strategy provided that the results from the reference samples are recorded either in the equipment blog or along with the analytical data. Now that assumes that the results are being completely recorded and are transparent.
A. Yes.
Q. Under the last paragraph, though, it says, do not, however, lose sight of the fact that such checks can conceal underlying deterioration in performance. And then it goes on to talk about a calorimeter. What’s a calorimeter?
A. It’s a device that’s used to measure a particular quantity of something using a, a device that looks at light, or that uses light to measure that...
Q. All right.
A. ...change.
Q. So, it’s...
A. So the breathalyzer.
Q. Yeah, but...
A. Remember the breathalyzer had the two vials and it measured the change in colour between the two vials as a way of figuring out what the concentration was.
Q. So the example they’re giving there is that a calorimeter may give a perfectly acceptable result, even if its wavelength calibration shifted, provided it is calibrated with the standards, that’s plural, at each use.
A. That is the language that’s stated there, yes.
Q. And they’re using the word calibrated as opposed to calibration checks.
A. That’s what it says.
Q. And then it says, however, if you are no longer taking the reading at the absorbant’s maximum, in other words I would suggest to you at the bottom of the valley in the infrared spectrum at approximately 9.4 microns, your detection limit will certainly be degraded and precision will also suffer in most instances.
A. Yes, that’s a possibility, yes.
Q. So my suggestion is that if you start taking the reading — if the detector is taking the reading at a bandwidth that is different than the design of the instrument because the filter had deteriorated, then you may be in a situation where your precision has suffered.
A. That is what that says, yes.
Q. And at the very, very bottom of the page, at page 60, it talks about what may happen if the detector becomes dirty and the signal to noise and dynamic range performance are degraded.
A. Yes, that can also affect the values.
Q. So these are all concerns of what may happen over time.
A. Yes, they’re all possibilities, yes.
Q. And that’s why we always need to be concerned about the question of drift in accuracy and precision over time.
A. Correct.
Q. Top of page 61, for this kind of reason, most equipment will require some formal checks even if it is effectively checked with the standards at each use. Do you agree with that statement?
A. Yes.