Is an Approved Instrument Really a Screening Device?
Updated: Oct 10, 2022
To suggest that an AI is a quantitative analysis instrument and an ASD is a qualitative analysis instrument and suggest that the government scientists are confusing this distinction as happened at the Motherisk Lab in Toronto.
To suggest that such confusion, if it persists, will lead to unconstitutionality of various Criminal Code sections or section 7 and 8 Charter violations.
Discussion and Argument During Cross-Examination of Crown Expert on Relevance of International Literature on Scientific Reliability
Commentary: The issue is reliability per St-Onge, not accuracy. What did St-Onge say say about reliability? Can the Alcohol Test Committee define science in a manner that contradicts international scientific literature?
What does the international scientific literature e.g. OIML R126 have to say about durability of metrological characteristics "over time"? So why aren't questions about durability over time relevant to a 258(1)(c) issue?
Notice the specific terminology in the argument below: "type approval", "verification", "re-verification", "metrological characteristics". See OIML.org.
MR. O: I don’t – I don’t understand how that’s relevant and it seems like argument as well Your Honour. THE COURT: How is it relevant? MR. BISS: Well, it’s relevant, Your Honour, in the sense that in Canada we don’t seem to have any kind of formal process with respect to evidentiary breath testing equipment to ensure that an instrument that – a new instrument, when it’s purchased, matches up with the characteristics of the instrument that was submitted for type approval. And then my further set of questions would be, we don’t seem to have any kind of reverification process to try and confirm that the instrument that’s out in the field still, a year later, two years later, three years later, still conforms to those same manufacturer’s specifications. THE COURT: Well, and I think the witness is suggesting in the answers given that that’s accurate. But that concern not particularly relevant to the issue of the accuracy of the individual test results in any event. I mean,
that’s the issue that’s joined here. I gather, based on reference to international standards and all of the research that you’ve conducted in this area, you may have a general view that there’s much to be desired about the accuracy and the precision of the breath test results generally, that are obtained, let’s just address the province we’re interested in, in Ontario. I gather that might be the position that’s being advanced. And I referenced on Friday, we’re not here to revisit the breath test procedure and the structure of the provisions in the Criminal Code. Those we have to accept or else you should be making this presentation to Parliament. But I understand that there are some concerns based on application of international standards; I’m not even sure whether these international standards apply. I appreciate that they’re international standards. But I haven’t heard anything in terms of reference to any particular regulations or legislation governing the laws relating to the Intoxilyzer 8000C as it’s employed in this province that suggests that there’s any reference to the international standards. We have a dispute here between the expert that you have called and the Centre of Forensic Sciences, represented today by our witness with respect to how you measure such things as reliability and accuracy and tolerance for error. You have a different view that you’re advancing on behalf of your client.
I understand that, with reference to scientific principle, that you’re going to urge, in due course, ought be the standard, I gather. But I – I’m not sure how it’s assisting to address how the qualified – or how the approved instrument, in this case is determined to be in sound working order when it comes into service. We’re dealing, in this case, with an older machine, correct? A. Yes. THE COURT: And I think the dispute, as I understood it, as identified in your application is, that over time there’s reason to believe that this particular device, the one used to test your client’s - the blood alcohol concentration in your client’s breath sample, is of dubious reliability. That’s what I’m interested in and it’s – you know, you may want to challenge the entire regime but I’m only interested in one test subject and accordingly, I’m only really interested in this particular Intoxilyzer. So.... MR. BISS: I – I – I understand that, Your Honour and that’s where I’m headed, specifically, but you see, from my perspective, I read St‑Onge Lamoureux. I see them talking about reliability, scientific reliability. I read Lam and I see it talking about the reliability – the scientific reliability of the measurement result. And then I look at the question of what is a measurement result, and I
recognize that every measurement result has an aspect to it, of both accuracy and precision. And then I read Brian Hodgson’s article that we started out with, which both experts accept as a good – having a good definition of reliability that talks about significant drift in accuracy and precision over time. Over time, just like durability. Accuracy and precision. So, the question ultimately becomes one, and it’s a really difficult question for the Court, for every Court to consider when a lawyer starts arguing reliability and I think this is the whole point of Lam, the lawyer needs to address the question of a change over time because that’s what reliability is, according to the definition in Hodgson. The definition that Hodgson provided to the Supreme Court of Canada in St-Onge. Now, if we’re going to look at a change over time we need to compare something at one time with something at another time. The question is, what is it that we compare? The perspective of the Centre of Forensic Sciences seems to be – and from Mr. P – and I understand where he’s coming from and I respect him on it, his point is that one measures accuracy and precision at the time of the subject test and you get it from the very, very limited information that comes from the test record. I’m suggesting that accuracy and precision – an evaluation of accuracy and precision if you’re looking at over time, accuracy means something, precision means
something, you need to have information on which to base that. And so, the question then becomes one of what information is available to us to assess what was the accuracy and precision of this particular instrument when it was first put into service? What is the accuracy and precision of this instrument at the time of my client’s subject tests? Those two periods of time have to be compared with each other, and it may indeed be a mathematical calculation, particularly with respect to accuracy because we heard that with accuracy you calculate a number of data points and you take an average, and precision is something else, which I can explore later on with Mr. P, but my general understanding of standard – of precision – has to do with something called standard deviation. That’s why asked Mr. P the questions on Friday about manufacturer’s specification for standard deviation. THE COURT: Right, well, look. MR. BISS: Standard deviation has to do with the variation of the data. THE COURT: I’m just – I’m going to let you save your thunder for final submissions on this. I think the Crown’s objection was you know, Canada’s policy, or Ontario’s policy, with respect to how approved instruments come into service, I appreciate how they’re periodically checked and the maintenance schedule may have relevance here, I’m not sure
how relevant any of this is, quite frankly, with respect to your particular concern, and that’s your client’s interests in this matter, but I’m assuming that you must have concluded that if this submission is accepted in whole or in part it will have a positive impact on the outcome of the case as far as your client is concerned. But the – Canada’s policy in terms of how these instruments are approved and what the Alcohol Test Committee has to say about it, whether that compares favourably with international standards, I think, really, is a matter beyond the purview of this application, quite frankly. I appreciate it’s interesting from a scientific perspective. But the Alcohol Test Committee has dictated what the criteria is going to be. That’s a fact. You can critique it, but you’re going to have to lead some persuasive evidence here, because the onus is on you in this application, on behalf of your client, that would suggest that this particular Intoxilyzer was rendered unreliable; the results cannot be relied upon and as a consequence, I can’t be certain that the blood alcohol concentration level is as stated and more particularly, I can’t be certain that it’s beyond the legal limit. That’s the issue. Not the exact reading. Not your client’s exact blood alcohol concentration reading, because in effect, the offence calls for the Intoxilyzer results to, in effect, act as a screening aid. I appreciate we get a precise result.
MR. BISS: I – with respect, Your Honour, I respectfully disagree. THE COURT: Well, I know you do. MR. BISS: And – and – and.... THE COURT: But in effect, well, just listen to me for a second. MR. BISS: Yes. THE COURT: Because I think it’s important you understand at least my perspective on this, it’s not a screening device. I get that. It’s not whether you’re over the legal limit or not, but apart from readings in the aggravated range, the actual blood alcohol concentration reading, whether it’s 130 or 140 or 150, is in large measure irrelevant. Is it not? MR. BISS: I respectfully disagree, Your Honour, and the reason why I say that is, that if we think in those terms we render a number of sections unconstitutional. And the reason why is, the way that the sections have been drafted is – and that’s why I asked Mr. P the questions about quantitative analysis and I showed him the sections of the Criminal Code. They read in terms of a quantitative analysis. THE COURT: Yes.
MR. BISS: Not a yes/no screening device analysis. They read in terms of a quantitative analysis. So, if – just in terms of the demand section, if a police officer on the road, who has otherwise reasonable and probable grounds to make a breath demand to blow into an
approved instrument, if the demand is in relationship to an instrument that’s not capable of a quantitative analysis, of a reliable quantitative analysis, then we’ve got a major section 8 problem, I think. That’s my submission. Parliament’s whole scheme is to expect that it’s a quantitative analysis. That’s there in the definition. The whole concept of what an approved instrument is. We’re not testing on screening devices. We’re testing on a quantitative analysis kind of instrument. That’s what an approved instrument is. That’s why I asked Mr. P those questions. To do with section 8, the demand is in relationship to an instrument that can conduct an analysis of a concentration. That’s a – in relationship to a quantitative analysis. The wording of 258(1)(c), when it talks about the concentration is, Parliament’s scheme is that these instruments are conducting quantitative analysis. The section to do with greater penalties talks about a specific concentration. THE COURT: Right, well I – I grant you that, but apart from that section the offence is having more than the lawfully or legally permissible amount of alcohol in one’s blood. So, to that extent, is all I’m suggesting to you, and I’m following your argument, is that the actual blood alcohol concentration reading for legal purposes, you’re saying it raises all sorts of Charter implications. I’m not sure
I’m following that, but the offence is made out if it’s established that the blood alcohol concentration exceeds the legal limit. That’s the offence. MR. BISS: Well, there are various ways that the Crown can establish that. But if the Crown’s going to rely on 258(1)(c). THE COURT: Yes. MR. BISS: 258(1)(c) contemplates a quantitative analysis. THE COURT: Agreed. MR. BISS: Using an approved instrument. That’s what it says. THE COURT: Yes. MR. BISS: And that – a quantitative analysis is a analysis of an instrument which has specific accuracy and precision. It has nothing to do with whether it’s screened above or below a certain level. It’s a concentration that is an analytical result. THE COURT: Well, let’s assume at the end of the day you – you establish that the variation here may be plus or minus 10 percent or more, you’re saying that would render the reading unreliable. MR. BISS: I’m suggesting the question is not – and the difficulty, Your Honour, is I don’t – I’m trying to stay out of all of the Singh litigation which deals with issues of variability and measurement uncertainty. It’s complicated and we might touch on it a little bit but I’m – I’m trying to, to some extent,
leave that for Singh because I – I don’t want to confuse the issues. The point in this case is: Can the Crown rely on 258(1)(c) which is, after all, what St‑Onge Lamoureux is all about. Can the Crown rely on 258(1)(c)? And we know now, that 258(1)(c) does not require the accused to show a link – a link between a specific malfunction or operator error and a particular number. That’s gone, constitutionally. We also know that the accused does not have to establish or at least raise a reasonable doubt as a result of St‑Onge Lamoureux with respect to the section that says that the accused blood alcohol concentration was below 80. We don’t have to do that. Thing 2 and Thing 3, sometimes people call them, in 258(1)(c). 258(1)(c) Thing 1 is still there, and the Supreme Court of Canada said in St‑Onge Lamoureux that it’s not unconstitutional, on the surface it is unconstitutional, but saved by section 1 and it’s saved by section 1 because Parliament has assessed and is looking at these issues from the perspective of scientific reliability. We no longer introduce evidence to the contrary to contradict the machine. We look at the scientific reliability of the machine. And if 258(1)(c) is dealing with a quantitative analysis on something called an approved instrument, as a result of a demand by a police officer with reasonable and probable grounds, to have an analysis of a
concentration done, then the instrument needs to be scientifically reliable or else 258(1)(c) isn’t going to apply. Now it may be that the Crown can establish a blood alcohol concentration above a particular limit in another way. But if the Crown wants to rely on section 258(1)(c) I think we’re stuck with this practical issue that the Supreme Court of Canada was so clear about in St‑Onge Lamoureux, that we’re dealing with scientific reliability. And – and it’s a difficult task for the defence because we don’t have to – and – and I think this is the whole point of Lam, we don’t just say oh, it missed one maintenance interval. It missed it by a week, or it missed it by three weeks. We don’t say oh, there was a cal-check that was outside of 90 to 110. The point is, what is the effect of that malfunction or that operator on scientific reliability? Scientific reliability as best I can tell, in reading St‑ Onge Lamoureux the Supreme Court of Canada is using Hodgson’s article, as the foundation of their decision and with respect to specifically, instrumental reliability, because that’s a subsection within Hodgson’s document.
Scientific reliability has to do with no significant drift in accuracy and precision over time. And if the defence is going to raise a reasonable doubt, that’s where the reasonable doubt needs to be, or at least a significant part of the defence reasonable
doubt needs to be a significant change, and I know the experts will always argue about what’s significant and what’s not significant, and Mr. P may have a different perspective. Mr. Kupferschmidt, other experts may have a different perspective and ultimately the Court’s got to assess it. And that’s why, whatever the particular rule is, whether in an international standard or in a – in an Ontario standard isn’t the point. The point is, that a Court, at the end of the day, needs to assess that scientific reliability. Its - over time, its accuracy, and its precision. Now the perspective of the Centre of Forensic Sciences is that you define accuracy. Accuracy defined a way as something [indiscernible] applies at the time of the subject tests with two numbers that are coming out. That’s accuracy, whether it’s the two numbers of the control checks or whether it’s the two numbers of the two subject tests. That’s the perspective of the Centre of Forensic Sciences. The perspective of other experts, including Mr. Kupferschmidt, is that you assess accuracy and precision, yes, and there are ways you – there are scientific ways to assess accuracy and precision, but you have to look at them from the perspective of over time, and that’s why one needs to look at, and compare one thing with another thing, using the Hodgson definition. So, that’s the practical difficulty that the defence has in every case. How – how does one deal with the issue? If
it’s – if it’s an – if the issue is instrumental reliability, which it is here, you know, may – there may be an issue with respect to instrumental in the sense that it’s the whole system, there may be a simulator issue too, which we haven’t explored yet, but with respect to the Intoxilyzer itself, the question becomes one of over time, what’s the data at the time of the test or around the time of the test, and what’s the data six months before, a year before, two years before? We have not all that much data to work with because of what the police do. The police don’t seem to have any clear guidance as to how they deal with all of this. But that’s why it’s so important on an O’Connor application like this, particularly, and – and again, I respect the Jackson decision. I understand Justice Watt’s perspective and why he made his decision. And my simple explanation for that is, he’s dealing with a new instrument that recently had been calibrated. If you look at – in the decision, there’s a clear indication it wasn’t even due for its first maintenance. There weren’t any maintenance records yet. It had its original certification from the manufacturer. Here we’ve got a completely and totally different situation. There’s a lot of water under the bridge. We’ve got some data to work with, fortunately, and my – at the end of the day my submission is going to be that that data at least has indications for now, which is – which
helps us for now. Whether it will help us ultimately at trial or not, is another story. But for now, we have indications of drift in both accuracy and precision based on the information that we’ve got now. We have some information. The problem, ultimately, is – and I’m going to ask Mr. P some questions in a few minutes, about the whole question of what outliers are, so he knows what I’m going to be telling him, I’m going to ask him what outliers are and the problem is that when you’re assessing precision you’ve got to be concerned about outliers, otherwise, as a scientist, you may be being speculative. And that’s an important issue that’s got to be nailed down because Mr. P and Mr. Kupferschmidt, they will probably do their math the same way. But one may think that some data is more relevant than other data. Some may consider something to be an outlier and others consider it not to be an outlier. So, that’s why the surrounding context is so important. So, I made a lot of submissions already, Your Honour.
#Motherisk #quantitativeanalysis #screeningdevice #qualitativeanalysis