Durability Over Time
Updated: Oct 10, 2022
Purpose:
To obtain admissions respecting the inadequacy of ATC / CFS recommendations in requiring "stability of metrological characteristics" over time.
To obtain admissions that the Diagnostics test function only checks stability over a few seconds at best, not over weeks, months, or years following calibration by the manufacturer.
To underline the concept of "over time" in Hodgson's definition of "reliability".
Q. Thanks. So, I want to move on to question in R-1-2-6 of something that they call durability. On page 15, Tab Number 7, under paragraph 5.11. A. Yes.
Q. Something called durability. Now, my understanding is that durability relates to – is a concept that has to do with over time. Is that your – in reading that paragraph, does that seem to make sense to you? A. Yes. Q. So, under durability, they say that the provisions in 5.2, 5.4 or 5.5, 5.6, 5.8 and 5.10 shall be that durably, and then if just turn back to that; 5.2 has to do with the maximum permissible error. A. Yes. Q. Five point four has to do with repeatability.
A. Yes. Q. That’s the calculation of a standard deviation in 5.4? A. Yeah. Q. That’s a measure of precision? A. It is. Q. Five point five, having to do with drift. There’s specific specifications for drift? A. Yes. Q. Five point six has to do with memory? A. Effects. Q. Yes. And 5.8 has to do with minimum requirements for rated operating conditions. A. Yes. Q. Those are to be met durably, which I gather means those requirements need to be met over time. A. Yes. Q. Right? A. Sustained over time, yes. Q. Now, do we have any equivalent to these provisions in the Alcohol Test Committee either evaluation procedures, best practices, or operating procedures with respect to the requirements of the A-T-C having to be met durably over time? Is there any specification of that, that you know of? A. There’s no direct mention or link to durability but the breath process as it stands, every day, evaluates the instrument. So, when it’s looking at air blanks, it’s checking the environment to see whether or not there’s anything there that could interfere with the tests. It’s doing diagnostic tests before it measures the calibration check and before it measures the subject test. And then the
calibration check is required to be within a certain range, 90 to 110 milligrams of alcohol in 100 millilitres of blood. And that is monitored during every breath test. Then you have the actual breath test itself, and then you have this repeated twice until you have – or at least two times – until you have two tests in good agreement. And all of those, in my opinion, would be issues associated with durability because you’re looking, over time, to see whether or not that’s changing and whether or not the instrument would require service, and so by having the calibration checks monitored every single time a breath test is conducted, that checks for drift and durability, in my opinion.
Q. I see. It checks for drift over a matter of a couple of seconds. A. Well, it’s also – you’re looking at it from the point of is there some kind of gross change in the calibration. Q. All right, can I just ask you this? A. Sure. Q. The diagnostic test process, that sequence that is run by an Intoxilyzer 8000C, that checks for positive or negative drift but it checks it over a matter of a couple of seconds. It does not check for drift relative to what the – to the signal coming off the instrument at other times; a week before, two weeks before, a month before, five months before. A. Correct. It’s assessing the instrument operation at that time.
Q. So, it’s – but it’s not in relationship, from a durability perspective, that check has nothing to do with over time longer than a couple of seconds. A. That's correct. But it’s making sure that
there hasn’t been any exaggerated drift. Because if it’s – the results of any of those are outside the acceptable range, then obviously, something has to be done to get it back to that point where it is within the acceptable range; some kind of trouble shooting or maintenance. Q. But my point is, it’s not a check of drift. There’s no – nothing in the diagnostic sequence of an Intoxilyzer 8000C that checks for drift, in the diagnostic sequence, that checks for drift over four hours, over two months, over five years? A. Correct. Q. Yet in O-I-M-L R-1-26 at page 11 it seems that their expectation is that the instrument is going to not drift under 5.5.1 by the specified tolerance over four hours under 5.5.2 over the four hours in 5.5.2.2. over two months. Right? A. Correct. That is their recommendation. Q. And they say that those requirements have to be met durably, which means over time, which means long term. A. Correct. Q. And with respect to repeatability, standard deviation, their expectation is that there – that instruments that meet these standards are going to have conformed to a requirement for specific precision and that that’s going to be met durably over time. A. Correct. Yes. Q. Now, I – I realise that when an Intoxilyzer 8000C is submitted for type approval, initial evaluation or – perhaps two of them that are submitted; maybe one to the C.F.S. and one to the R.C.M.P. for type approval, that someone does a group of 30 or 40 or 50 tests at various values, not
just to check linearity but also to check precision of the instrument. A. Yes. MR. O'NEILL: I’m just wondering about the relevance of that, Your Honour. The approval process and the tests that occur during the approval process. MR. BISS: Well, that was the preamble to my question Your Honour. THE COURT: That’s sort of what I surmised. MR. BISS: Q. It’s done at the time of type approval on a different instrument than what’s going to actually be used out in the field, right? A. Correct. It’s assumed that the instruments that are sent for evaluation are a random lot. Q. Yes. A. Right? That are then sent out and that everything that those instruments do would be the same and be applied to the other instruments. And the same analogy is that the alcohol standard solution that’s manufactured by a particular manufacturer is checked and then it is a random lot that the Centre of Forensic Sciences receives an analyses, and if it’s considered to be acceptable within – like, the concentration is deemed to be within the acceptable range, then a certificate is prepared for that, and that applies to the entire lot because it’s assumed that – in science – that the random sample applies to the population in the same way that it would with an instrument that’s being tested. Q. Even if the manufactured date of the instrument that goes out in the field is a year, two years, three years, post ministerial approval? A. Yes.
Q. It’s assumed it’s going to be a random sample just like the others? A. Absolutely, yes. Q. There’s – there’s – there’s no defined process in our best practices or in our operating procedures or in our equipment evaluation procedures to ensure that an instrument – a new instrument that’s first placed into service will be performing with exactly the same or within a similar range of accuracy and precision as the instrument that received type approval. A. So, the instruments, before they’re – after they’re manufactured and sent to the police service, they’re – they go through an assessment process to make sure that they are within the manufacturer’s specifications and tested, and then in the field they are tested each and every time the instrument’s used. And that’s why you say that the – the proper working order of the instrument is assessed at the time of use. Q. But there’s no requirement – there’s nothing in the Alcohol Test Committee recommendations. There’s nothing in any Centre of Forensic Science recommendation that specifies how the accuracy and the precision is going to be inspected on a new instrument going into service so that it matches up; so, that it is identical or close to the instrument that was received for type approval. A. Well, again, the parameters that are used for evaluation are completely separate from those for operation in the field. And... Q. So.... A. ...the assessment at the time of operation is used to determine whether the instrument’s in proper
working order. If it’s deemed to be outside that then it would be sent for service and then you would return the instrument to the manufacturer’s specifications or as close to as you can, and then – then it would go back into the field and it would be assessed again on – every time it’s used. Q. But there’s no guidance for this deeming. The deeming is random. It’s at the whim of the police officers who are using the equipment and make the decision to send the instrument back for recalibration.
...
THE COURT: Yeah, it’s probably easier just to let you question away here. MR. BISS: I can – I can move along a bit, Your Honour. Q. I – I – can I just ask you this, Mr. P.?
A. Yes. Q. Well, just be really, really practical. If
I buy gas at the gas pump then there is going to be a sticker on the gas pump that looks kind of like this...
A. Thank you. Q. ...from an entity called Measurement Canada, and it’s bigger in the picture that you’ve got than it actually is on the sticker on the gas pump. A. Yes, I’ve seen it on the pump when I get my gas. Q. And you can see that there’s one sticker stuck overtop of another sticker. That’s pretty easy to see, and it has – the one that you’re looking at has an inspection date in 2015 and then there’s a – and a particular month, the 9th month, and it expires, it looks like, in 2017, maybe the 9th month of 2017. Have I got that right? A. That appears to be correct, yes. Q. Now, so first of all, there is a sticker system. Number two, the sticker notes the date of the last – of the last inspection or calibration. Number three, the sticker has an expiry date on it. A. I’m not sure if they do a calibration at the time of inspection. It just says inspection and I don’t know what their definition of inspection is. Q. All right. But whether it’s an inspection or whether it’s a calibration the point is, that there is a system in place, in existence, with respect to Measurement Canada setting up a system for gas pumps whereby somebody does an inspection, they put on a sticker and it’s been inspected, so everybody knows that it’s been inspected. Right? A. Yes. Q. And your experience would be, when you go to a retail store and you buy food, their scales have similar kinds of stickers.
A. Um, I can’t think of any offhand, but I would imagine there are. But this would be for commercial use, as opposed to legal – legal use. Q. Right, well, can I ask you.... A. Which an Intoxilyzer is used for. Q. Can I ask you this? There’s no similar sticker that’s been put onto Intoxilyzer 8000Cs in Ontario. A. Not that I’ve seen, no. Q. I mean, the reason why I ask you that is, if one looks at our O-I-M-L R-1-2-6 from 1998 and I think when we look at the certificate of calibration from C.M.I. earlier, that was contained in – it was in Exhibit Number 12, ah good, you have it. A. Yes. Q. It has a reference to the instrument complies with O-I-M-L R-1-2-6 1998. A. Yes. Q. And we’re looking at Tab 7 in the O-I-M-L volume which is Exhibit 31, at Tab 7, which apparently, these Intoxilyzer 8000Cs complied with, according to that certificate. Page number 10 at Tab 6 under 6.15. A. Just one moment, please. Page 10, yes. Q. Under 6.15 – 6.15.1. MR. BISS: Do you have it, Your Honour? THE COURT: Yes. MR. BISS: Q. It says an E-D-A shall be marked indelibly and irremovably with the following information. And then the last bullet reads, “length of time or the number of analyses permitted between maintenance operations, including adjustment, in accordance with 3.9 described in the user manual supplied by the manufacturer. All this information shall be legible and grouped together on a visible part of the
E-D-A.” Now, my question is, there’s no such marking on any of the Intoxilyzer 8000Cs that we’re using in Ontario. Right? A. That's correct. This is an international recommendation that again, is not binding to Ontario or Canada. Q. I’m just trying to figure out how it can be that the instrument could conform to O-I-M-L R-1-2-6, according to the manufacturer’s certificate of calibration and yet, there’s non-compliance with that requirement. A. I can’t... Q. Makes no... A. ...answer that question. Q. ...sense to you? A. Yeah. Q. And with respect to the current standard from O-I-M-L at Tab 7, the provisions that I read to you with respect to durability.... A. If I might. Q. I’m sorry?
A. There is a sticker on the Intoxilyzer 8000C that the manufacturer places on the seam, so basically the Intoxilyzer consists of – it’s like a little plastic tacklebox if you will, right? With a bottom half and a top half. And so, the – so, if you need to do any instrument maintenance, you can take the top part off and have access to the internal components. That there are stickers on the side of the Intoxilyzer 8000C on those seams that if they’re broken then any warranty or certification is void. I don’t know if that’s the same thing.... Q. All right. A. ...as what’s being described here. Q. On page 15 of Tab 7 again, under durability
that we were looking at before. A. Sorry, just one second. Yes. Q. The second paragraph, “shall be designed to maintain stability of its metrological characteristics over a period of time to be specified by the manufacturer, which shall be at least as long as the verification period.” To the best of your knowledge, has C.M.I. specified for the 8000C a particular period of time over which it’s designed to maintain it’s stability of metrological characteristics? A. I’m not aware of any, no.
See also VIML at oiml.org
Canada is a member state of VIML
The Foreword to VIML is reproduced below.
"Durability" is also a concept in engineering. There are numerous videos on Durability on YouTube.
#Hodgson #reliability #durability #metrologicalcharacteristics #overtime #calibration