84
SOUTHERN DISTRICT REPORTERS, P.C.(212) 805-0300
HB6HJON3 Mitchell - Direct
short one so we ensure that the doctor gets --
THE COURT: Absolutely. Whenever you reach a breaking
point, just let me know, and we can take a lunch break at that
point.
MR. McKAY: The government calls Dr. Adele Mitchell.
THE COURT: Dr. Mitchell, if you could step up,
please.
ADELE MITCHELL,
called as a witness by the Government,
having been duly sworn, testified as follows:
DIRECT EXAMINATION
MR. McKAY: Thank you, your Honor.
Q. Dr. Mitchell, could you please describe your educational
background.
A. Yes. I have a bachelor's degree in math. Earned that in
1991. And then I earned a master's degree in genetic
epidemiology and human genetics. Genetic epidemiology is a
statistical genetics field. That was earned in the year 2000
from the Johns Hopkins University School of Public Health. And
then I earned a Ph.D. in human genetics and molecular biology
in 2004 from the Johns Hopkins University School of Medicine.
That was followed by a two-year postdoctoral period at the
University of Washington in the departments of statistics and
genome sciences.
Q. Can you please describe your coursework and the subject of
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
85
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
your dissertation.
A. My Ph.D. coursework was within the school of medicine, and
it included about two thirds of the first two years of the M.D.
coursework, so a lot of human biology. And in addition to
that, there were courses specific to genetics, including human
genetics, model animal genetics, population genetics, and
statistics.
Q. What was the subject of your Ph.D. dissertation?
A. I was looking at uncertainty in genotype data and how that
affects downstream analyses. So, for example, if genotypes are
incorrectly called and then a subsequent association testing is
performed to try to look for variance that are associated with
a disease, the errors in the initial part can -- unbiased
errors in the initial part can in fact introduce a bias into
the association results. So I was modeling that and showing
how that can happen.
Q. What was the subject of your focus of your postdoctoral
work?
A. As a postdoctoral fellow, I developed a method that would
simultaneously estimate the genotyping error rate and the
association. So it was modeling the error so that we could
look for the association with disease without the bias that
could be introduced by the error.
Q. Have you received grant funding for your research in the
past?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
86
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
A. I have.
Q. Where do you work now?
A. I work for a small pharmaceutical company called Eisai,
spelled E-i-s-a-i. The company is based in Japan, but I'm in
Boston at a research site there.
Q. How long have you worked there?
A. I've worked there since February 2017.
Q. What did you do immediately before that?
A. Before that I was at Merck research labs also in Boston
from March 2014 until I started at Eisai.
Q. Did you ever work for the New York City Office of the Chief
Medical Examiner?
A. I did.
Q. When did you work there?
A. I worked there from fall of 2008 through March 2014.
Q. What were your duties and responsibilities at the OCME?
A. I led a research group there. We were focused on improving
forensic methods, both in the laboratory and analytical
methods.
Q. Is one of the things that you worked on the development of
something called the Forensic Statistical Tool?
A. Yes.
Q. Did you develop other techniques as well?
A. Yes.
Q. Just generally speaking, what were some of the other things
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
87
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
you worked on?
A. We worked on improving yield of DNA from a forensic sample,
improving storage, improving the profiles that we could get
from mixtures. We worked on single cell analysis, a variety of
techniques to improve forensic -- the information obtained from
forensic evidence.
Q. Did you lead any trainings while you were at OCME?
A. I did. I trained analysts in population genetics and
forensic statistics, and then once the FST was developed, I did
the training of all of the analysts in the methods behind the
technique and how to apply them to casework.
Q. Have you held any teaching positions in your life?
A. I have. I started out my career as a high school math and
physics teacher. That was from 1992 to 1999. For two years
before I worked at OCME, I was faculty at Mount Sinai School of
Medicine, and I continued on a part-time basis for another two
years after I started at OCME. I've also taught in workshops
and seminars nationally and internationally on forensic
statistics and likelihood ratios.
Q. During your time at OCME, did you win any awards?
A. I did.
Q. Without embarrassing you too much, could you tell us about
that award.
A. I won an award called the Fred Hayes Prize. It's awarded
annually to city employees who have performed in an outstanding
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
88
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
way in some area, and it's for all city departments. So this
was awarded to me for my work on FST.
Q. Have you testified previously about DNA?
A. I have.
Q. About how many times?
A. Three times.
Q. Were you qualified as an expert witness when you testified
previously?
A. I was.
Q. Have you ever consulted -- three times you've testified,
were you testifying for the prosecution or the defense?
A. So there was one hearing in Brooklyn where I was called by
prosecution and then I was recalled by defense in that case,
and then the other two cases, I was prosecution witness.
Q. Have you ever consulted on DNA as a defense expert witness?
A. I have.
Q. Just generally speaking, what was the nature of your work
as a defense expert?
A. I'm occasionally contacted to review a DNA case for a
defense attorney, and so it might involve examining the DNA
results and giving my opinion on the methods that were used and
the conclusions that were drawn from those results.
Q. Was there one particular case in Washington State of
interest to you?
A. There was. I worked for defense on a case where the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
89
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
Washington State DNA lab had produced a report that on its face
looked like it was informative in the case, and so I got the
case file and went through it and determined that the sample
was actually not -- not of high enough quality to draw any
conclusions from. So I worked with the defense attorney on
that, and we then consulted with the prosecutor on that who
ended up not using the DNA evidence once he learned of its
condition.
Q. In the DNA sample analysis in that case, does OCME generate
similar analysis of similar samples?
A. No. This sample had results at only two of the STR loci.
So OCME would never draw a conclusion from a sample like that.
Q. If you turn in that very large binder in front of you to
the beginning to the tab Government Exhibit 1, would you look
at that and see if you recognize it.
Mr. DeLuca, could you put it on the screen for her as
well.
THE COURT: Dr. Mitchell, was the work you just
described for the defendant in Washington State, was that the
work that you've done as a consultant?
THE WITNESS: Yes.
THE COURT: All right. I'm sorry. Go ahead.
Q. Do you see Exhibit 1 there?
A. I do.
Q. What is that?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
90
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
A. That is my CV.
Q. Does it accurately describe your education and
qualifications?
A. Yes.
MR. McKAY: Government offers Exhibit 1.
MR. ROSANIA: No objection.
THE COURT: Government Exhibit 1 is admitted in
evidence.
(Government's Exhibit 1 received in evidence)
BY MR. McKAY:
Q. Looking at pages 2 and 3, there's a long list there. What
is that list?
A. These are peer-reviewed publications on which I am an
author.
Q. Without going through every one, can you just summarize
some of the general topics about which you published.
A. Yes. So the earlier publications are -- is work that I did
as a student, the first four there. The numbers two, three,
and four are the work that I described earlier, looking at what
are the effects of undetected genotyping error on statistical
analyses. Then there are some disease-specific analyses that I
contributed to. And there's one on the top of the second page
where this was a collaboration with the San Diego Zoo. They
had a small set of endangered iguanas, and they wanted to set
up a breeding program. So I advised them on how best to pair
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
91
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
up the animals based on their genetics. Then following that
are some forensic publications and a few more disease-specific
publications in there.
Q. If you turn in your binder to Government Exhibit 2, can you
tell me if you recognize that document.
A. Yes. This is a paper describing application of the
Forensic Statistical Tool to casework samples.
Q. Who wrote that paper?
A. I'm the first author on that, which indicates that I did
the bulk of the work and I wrote up the manuscript.
MR. McKAY: Government offers Exhibit 2.
MR. ROSANIA: No objection.
THE COURT: Government Exhibit 2 is admitted in
evidence.
(Government's Exhibit 2 received in evidence)
BY MR. McKAY:
Q. Where is this article published? What publication?
A. A journal called Forensic Science International: Genetics.
Q. Is that a peer-reviewed publication?
A. It is.
Q. What does it mean to be peer reviewed?
A. When a scientist is interested in publishing some results,
they write a manuscript and send it to the editor of a journal
of interest. That editor will then do a preliminary evaluation
to determine if that may be appropriate for their journal. If
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
92
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
it may be appropriate, they send it to usually two experts in
the field who do quite a thorough evaluation of the methods and
the conclusions. Those experts can recommend that the
publication be admitted or accepted or they can ask for some
additional work or they can reject a manuscript. Generally, if
the two reviewers agree, the editor goes with their
recommendation. Sometimes if they disagree, the editor might
send it to a third expert in order to get a third opinion.
Q. Have you ever served as a peer reviewer for any
publications?
A. I have.
Q. What publications?
A. I've reviewed articles for several journals. One of the
main ones is this one Forensic Science International: Genetics.
But outside of forensics, I've also reviewed articles for
journals called Genetic Epidemiology, Heredity, Genetics, and
then one on statistical applications in molecular biology.
Q. This article, Exhibit 2, this was published after being
researched by individuals with expert advertise in the subject
matter?
A. Yes.
Q. If you'd turn in your binder to Exhibit 3, do you recognize
that article?
A. Yes.
Q. What is that?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
93
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
A. This is an article describing the validation of FST.
MR. McKAY: Government offers Exhibit 3.
MR. STRAZZA: No objection.
THE COURT: Exhibit 3's admitted in evidence.
(Government's Exhibit 3 received in evidence)
BY MR. McKAY:
Q. What this publication is this?
A. This is also in Forensic Science International: Genetics.
Q. Are you familiar with the Organization of the Scientific
Area Committees?
A. Yes.
Q. Are you a member?
A. I'm not a member of the organization, but I am a member of
one of the subcommittees of that group.
Q. What subcommittee are you on?
A. I'm on a subcommittee that was tasked with developing
validation guidelines for probabilistic genotyping methods.
Q. And to whom do those guidelines apply?
A. The guidelines apply to forensics laboratories in the U.S.
This group is organized by NIST, the National Institute of
Standards and Technology, and they have taken on the task to
develop guidelines in several different forensics areas, and
one of them is probabilistic genotyping. So several people in
the field were recruited to provide guidance and to develop
these methods for probabilistic genotyping.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
94
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
Q. Were you one of those persons recruited to provide
guidance?
A. Yes.
Q. Was this after you had done and published about the FST
validation?
A. This was after. I believe it started in 2014.
MR. McKAY: Your Honor, at this time I'd offer
Dr. Mitchell as an expert in human genetics, molecular biology,
forensic science, and the statistical analysis thereof.
THE COURT: OK. Any objection?
MR. STRAZZA: I would like a brief voir dire, but I
just want to write down what the government just said.
THE COURT: All right.
MR. STRAZZA: Can you say that again.
MR. McKAY: Human genetics, molecular biology,
forensic science, and the statistical analysis thereof.
THE COURT: OK.
MR. STRAZZA: May I?
THE COURT: Yes.
VOIR DIRE EXAMINATION
Q. Good afternoon, Dr. Mitchell.
A. Good afternoon.
Q. You are a forensic research scientist; is that correct?
A. I have been in the past. At this point my work is in a
pharmaceutical setting.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
95
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
Q. When you worked for OCME, you were a forensic research
scientist; right?
A. Yes, yes.
Q. And that's to be distinguished from a casework analyst;
right?
A. Yes, that's right.
Q. And, in fact, you never worked on actual forensic casework
in your career; right?
A. That's true.
Q. And you were specifically hired by OCME to develop the FST
program which is the subject of this hearing; right?
A. Yes, yes.
Q. And you were hired to perform the validation in that
instance; correct?
A. Yes.
Q. You had no experience with forensics prior to your work at
OCME; right?
A. Right.
Q. And I think you said this, but you don't work on forensics
now; right?
A. Correct, although I occasionally consult on forensic cases.
Q. You mentioned that you had testified in Brooklyn as a
defense witness; right?
A. I was originally called by the prosecution and then defense
recalled me.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
96
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
Q. Right. So I just want to be clear. I want to make sure
there's no misunderstandings.
A. Yes.
Q. The defense's position in that case was that FST was
unreliable; right?
A. I'm not sure if I would exactly characterize it that way,
but yes.
Q. How would you characterize the defense position in that
case?
MR. McKAY: Objection.
THE COURT: How does this go again --
MR. STRAZZA: Because one may think that she testified
on behalf of the defense in that case as a defense witness to
support -- it's misleading, Judge, and I just wanted to get out
what really happened in that case.
THE COURT: Why is that relevant to voir dire? In
other words, I'm just trying to figure out -- I understand the
nature of the question, but I don't understand the nature of
the question in the context of voir dire.
MR. STRAZZA: OK. I'll withdraw and revisit on cross.
THE COURT: OK.
MR. STRAZZA: We have no objection to Dr. Mitchell
being admitted as an expert in human genetics, molecular
biology, and forensic science with the understanding that's as
a forensic science researcher, not casework analyst.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
97
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
THE COURT: That's fine.
MR. STRAZZA: Thanks.
THE COURT: I understand.
All right. Mr. McKay, you may continue.
DIRECT EXAMINATION (Cont'd)
BY MR. McKAY:
Q. Dr. Mitchell, you mentioned this, but during the course of
your time at OCME, did you develop something called the
Forensic Statistical Tool, or FST?
A. I did.
Q. We'll talk in great detail about FST in a minute, but can
you give us to start just an overview of what FST does.
A. Yes. It is a method for putting a numerical weight on a
comparison between an individual's DNA profile and a forensic
mixture, a mixture of DNA from two or more people.
Q. Does FST tell you whether the DNA mixture contains the
defendant's DNA?
A. No.
Q. What does it express? What is the nature of the report?
A. It's a way of determining whether a mixture can be better
explained with a defendant included in the mixture or could it
be explained better if that defendant is not a contributor but
instead a randomly selected person from the population as a
contributor. So it compares two scenarios, one in which a
defendant is part of the mixture and one in which they're not,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
98
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
and it asks which of these is better supported by the
evidence --
Q. What is?
A. -- and it quantifies the degree of support.
Q. When you say "quantifies the degree of support," is that a
measure of the strength of the evidence?
A. Yes. So, specifically, what it's calculating is the
probability of getting the evidence if the defendant is a
contributor compared to the probability of getting the evidence
if that defendant is not a contributor.
Q. Why did you want to develop FST?
A. It's very important to me that DNA evidence be
appropriately weighted. DNA plays a special role in forensics,
and I want to be sure that it's not given too much weight or
too little weight.
Q. Before you developed FST, do you have a sense of how OCME
analysts would have reported the findings when they analyzed a
DNA mixture?
A. Yes. In certain circumstances, they could calculate a
statistic called the combined probability of inclusion, or the
CPI. And that would be applied in a situation where drop-out
and drop-in were not suspected. Meaning, looking at a mixture
without looking at a defendant's profile, looking at a mixture
and asking, are there characteristics of this mixture that
might indicate that some alleles could have dropped out? So
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
99
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
things that can indicate that would be a small amount of DNA to
begin with, a lot of peaks below the threshold where alleles
are called, things like that. So if those things are not
happening and a defendant's profile could be completely
included in the mixture, so at each locus tested there are
alleles that could have come from the defendant, then this
calculation can be done. And so the result of that would be to
say this person cannot be excluded as a contributor and
X percent of the population also could not be excluded as a
contributor.
But that can only be used in those limited
circumstances. And if there's a situation where alleles that
could have come from the defendant show up at every locus
except for one, there was no way to put a number on that
result. So prior to using FST, OCME analysts and other
forensics labs as well -- it's not limited to OCME -- would
have used kind of a descriptive conclusion like this person
cannot be excluded, but there would be no sense of the weight,
who else could not be excluded. Or if a couple of alleles were
missing, they might say no conclusions can be drawn about this
comparison. Finally, if several alleles were missing, the
analyst would say this person is excluded as a contributor. So
it was this kind of levels of description about the comparison
but no numbers.
Q. How would you characterize the range within each of those
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
100
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
levels?
A. Those are quite a big range -- and I'll show some plots
when we look at our validation -- if comparison profiles are
categorized as either fully included in the evidence or cannot
be excluded or no conclusions or excluded. So each of those is
a pretty big range of likelihood ratios.
Q. Now, you said that FST allows you to put a number on what
previously was qualitatively described; is that fair to say?
A. Yes, that's true.
Q. Does that number that FST reports, does that purport to be
exact?
A. No, it's an estimate. And we have tried our hardest to
make sure it's not an overestimate.
Q. We'll come back to the concept of under or overestimate
later on, but first I'd like you to take a look at your screen.
Mr. DeLuca, would you pull up Government Exhibit 0.
Do you recognize that, Dr. Mitchell?
A. I do.
Q. What is it?
A. This is a presentation that I prepared describing what
underlies FST and how we validated it, how we developed and
validated it at OCME.
Q. Will this help you illustrate some of the concepts you'll
talk about today?
A. Yes, it will.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
101
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
MR. McKAY: Government offers Exhibit 0.
MR. STRAZZA: No objection.
THE COURT: Government Exhibit 0 is admitted in
evidence.
(Government's Exhibit 0 received in evidence)
MR. McKAY: Turning to the next slide, Mr. DeLuca.
Q. What is a likelihood ratio, generally speaking?
A. A likelihood ratio is a ratio of two probabilities. The
first one is the probability of getting the data that you're
interested in, whether forensics or something else, if a
particular scenario is true. And the second probability is the
probability of getting those same results if some different
mutually exclusive scenario is true. If the likelihood ratio
turns out to be greater than 1, that's indicating that the data
are supporting the first scenario over the second, and if it's
less than 1, that would indicate that the data support the
second scenario over the first.
Q. Now, are likelihood ratios used just in forensic DNA
analysis or in other fields as well?
A. They are used in other fields, numerous other fields.
Q. Can you give an example.
A. Yes. On the next slide is an example of two competing
scenarios in a medical situation. Maybe the patient has
cancer; maybe the patient doesn't have cancer. So a test is
performed, and the question that is often analyzed in this
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
102
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
situation is what's the probability of obtaining these test
results if the patient has cancer versus obtaining the test
results if the patient does not have cancer?
Q. How is the comparison commonly described in the forensic
setting?
A. In the forensic setting, it's usually a comparison of
looking at a mixture and asking, what's the probability of
getting this mixture if the suspect or someone else of interest
is a contributor to the mixture versus what's the probability
of getting this mixture if this person is not a contributor and
instead an unknown, unrelated person is a contributor?
Q. How would that likelihood ratio be expressed in that
setting?
A. So on the next slide, scenario one is often called the
prosecution scenario. So it often includes a defendant.
Scenario two is often called the defense scenario where that
defendant is replaced by an unknown, unrelated person. If the
likelihood ratio is greater than 1, then the genetic data are
supporting the prosecution scenario over the defense. And if
it's less than 1, the data are supporting the defense scenario
over that of the prosecution.
Q. What are Bayesian statistics?
A. So the likelihood ratio is a component of Bayesian
statistics. And these were first developed, this way of
looking at data was first developed in the 1700s by Thomas
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
103
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
Bayes, Reverend Thomas Bayes. And between then and now
especially with the development of computing capabilities,
Bayesian statistics have been applied to pretty much every area
of science and beyond.
Q. And are likelihood ratios in the forensic setting, is that
a new concept?
A. No, it's not. Likelihood ratios have been used for
forensic DNA analysis since DNA analysis was introduced in a
forensic setting. The random match probability is a commonly
reported statistic for a single-source sample, and that is a
likelihood ratio. Likelihood ratios are used for kinship
calculations in a kind of a setting where we would say, what's
the probability of these two people sharing this many alleles
if they are related versus if they are not related? So that
would be a kinship setting. It was extensively used during the
World Trade Center human remains identification process because
it's a standard calculation for that. There is a software
package on the FBI website, or at least it was at the time we
were developing FST. I haven't revisited it in a couple of
years. I don't know if it's still there. But there were
methods for performing a likelihood ratio mixture analysis on
the FBI website.
THE COURT: Mr. McKay, let me ask this: The random
patch probability, was the other DNA test results that we have
in this case, do you recall, was that a random match
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
104
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
probability?
MR. McKAY: I do, but I don't think Dr. Mitchell has
specific --
THE COURT: I'm just asking you, actually.
MR. McKAY: So my understanding is that the other
sample, the hat, is a two-person deducible mixture --
THE COURT: OK.
MR. McKAY: -- such that you could identify the major
contributor and then apply random match probability.
THE COURT: Got it. Thank you.
BY MR. McKAY:
Q. I'll ask you, Dr. Mitchell, if you had a two-person
deducible mixture hypothetically in which you could identify
the major contributor, could you apply random match probability
to that scenario?
A. You could.
Q. You would not run FST as to the major contributor in that
scenario?
A. That's correct.
Q. Now, you mentioned earlier the CPI. Over the course of the
last -- as an alternative to the likelihood ratio method of
analyzing DNA mixtures; is that right?
A. I'm sorry. Could you repeat that.
Q. I botched that one.
Is the CPI an alternative to the likelihood ratio
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
105
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
method of analyzing complex mixtures?
A. In limited circumstances, yes.
Q. Over the course of the last decade, has there been any
trend regarding the use of CPI versus the use of likelihood
ratios?
A. Yes. CPI used to be a very commonly applied statistic, and
the trend is changing. In the last several years, more labs
are moving to a likelihood ratio framework.
Q. Is FST the only forensic analysis tool that uses likelihood
ratios?
A. No, there are many.
Q. Do you have some examples of other programs?
A. So there are a few others that are very similar to FST.
Examples of those would be Lab Retriever, LRmix, likeLTD, there
are a couple of others that I can't recall offhand. So those
work just like FST where they allow for alleles to drop out or
to drop in, and I can define that a little bit better in a
minute. And they have different ways of estimating the
probability that an allele might drop out or drop in. Then
there are some more complex programs that would be called
probabilitistic genotyping methods where the characteristics of
the electropherograms are used by a computer program to try to
sort out the individual profiles of the contributors. So FST
isn't doing that. It's taking the alleles that are called by
the standard genotyping software and doing its calculation just
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
106
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
using those, but it's not doing the separation of the
individual profiles like these other programs, these two other
programs can do.
Q. Is it fair to say that FST or programs sharing a common
approach to FST are in use by forensic laboratories worldwide?
A. Yes.
Q. I'd like to talk about the development of FST. Was it all
done in one phase or was it more than one phase?
A. Well, there were two phases to the project. The first
phase was the development of the program and the estimation of
the parameters that we were interested in, and then the second
phase was a validation or testing of the program.
Q. Let's start with the first phase. But first can you
identify the term "drop-out"?
A. Drop-out is when is somebody really is a contributor to a
sample, but yet one or more of their alleles are missing from
that sample.
Q. What is drop-in?
A. Drop-in is when there's an allele that is labeled in a
mixture that does not come from any of the true contributors to
a mixture. So that can happen -- there seems to be bits of DNA
everywhere. So an allele can be labeled even if it's not part
of a mixture made up of known individuals, and we've seen that
in mixtures that we've made using cheek swabs and blood. We
would see, if it's a high template sample, the entire profiles
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
107
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
of the contributors, but sometimes there will be an extraneous
allele that we don't know where it came from.
Q. That's drop-in?
A. Yes.
Q. What are some of the reasons you might see drop-out?
A. If there is not a lot of DNA in a sample, sometimes some
parts don't amplify very well, and they don't show up. Some of
the forensic loci amplify better than others. So there are
somewhere drop-out is less likely and others where it's more
likely. And it's -- at a certain level it's a stochastic
process. So sometimes one just doesn't show up by chance.
Q. Does FST attempt to account for the phenomena of drop-in
and drop-out?
A. What we did was estimate the rate at which it happens,
and -- is that what you mean by account for it?
Q. Well, you mentioned CPI.
A. Yes.
Q. Does CPI account for the possibility of drop-in and
drop-out?
A. No, it cannot be modeled in that analysis.
Q. Does FST have as a component a consideration of drop-in and
drop-out?
A. Yes, it does.
Q. Why does it incorporate that function?
A. Because that's the nature of the samples that we see in
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
108
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
forensics. If we have a single-source blood sample, and it's
amplified. It's very easy to analyze. But evidence comes in
all forms, from touched items to mixed saliva and blood, all
kinds of combinations. And we don't always see the entire
profile of people who contributed to the sample, and we've seen
that in mock casework samples that we've made in the lab.
Q. So drop-out actually happens?
A. Yes.
Q. When you're testing DNA from a piece of evidence, is it
possible to know what the true drop-out rate is?
A. No.
Q. So what does FST do, then, to try to estimate drop-out
rates? How do you determine estimated drop-out rates?
A. We made a large number of purposeful mixtures of DNA from
two contributors and three contributors. We used about ten
different people in various combinations to set up the
two-person mixtures and the three-person mixtures. And then we
started with different amounts of template DNA, from very small
amounts to very large am amounts, and we looked and we counted
how often did those contributors' alleles not show up. And we
looked at the different loci that are amplified for a forensic
comparison and compared those -- those loci. So some loci had
more drop-out than others and some had less.
Q. So when you make a sample of known contributors, you know
whose DNA is going into the mixture; right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
109
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
A. Yes.
Q. So do you know what alleles you would expect to see in the
mixture?
A. Yes.
Q. In your test results, did you always see exactly the same
alleles that you would have expected to see?
A. No.
Q. Now --
A. Sometimes a true contributor's allele or alleles would not
show up, and sometimes there would be extraneous alleles that
did not come from the people whose DNA we had put into the
sample.
Q. So that is an illustration of drop-in and drop-out; is that
right?
A. Yes.
Q. How long did that process take, estimating the drop-out
rates?
A. It was about a year to do the drop-out rate estimation and
to write the program.
Q. During that study, what factors did you see affecting
drop-out rates?
A. If you go to the next slide, we can look. The biggest
predictor of the probability of drop-out is the amount of DNA
in the sample, the starting amount of DNA. So we looked at
samples in OCME's low template range, which is 25 to 100
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
110
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
picograms, and then we looked in the high template range, which
is 100 to 500 picograms. And so the amount of DNA is a very
important predictor of the probability of an allele dropping
out.
The next slide, other predictors are the number of
contributors to the mixture. We looked at two-person,
three-person, and four-person. OCME eventually decided not to
implement the four-person mixtures in casework. So the number
of contributors was important and the relative contribution
from each contributor. If there are major and minor
contributors, the minor contributors are intuitively more
likely to drop out than the major contributors' alleles. So
here are some examples. The first one on the top would be one
locus at -- in a mixture that cannot be separated. So these
peaks are all about the same height. It's not possible to pair
them up based on their height.
In the second picture, the 9 and the 12 alleles are
much taller than the 11 alleles. So this is an example where
those two alleles could probably be paired together. Those
probably came from the major contributor to that mixture, and
the 11 came from the minor contributor, and we don't know what
that person's second allele is. It could be another 11 or it
could be a 9 or a 12 or it could have dropped out.
Q. Now, a moment ago you talked about how the amount of DNA
was one of the most important factors.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
111
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
A. Yes.
Q. Is that sometimes called quantitation or quant?
A. It is.
Q. Generally speaking, did you see that drop rates were higher
or lower as you increased the quant?
A. As the quant increases, the probability of drop-out
decreases.
Q. So in the rates that you ultimately chose to use in FST,
was quant one of the factors that went into setting the rates?
A. Yes.
Q. Did you consider any alternatives for quant?
A. We did. Early on we looked at the peak heights in the
electropherograms. So using on this slide other -- some other
groups have estimated the probability of drop-out based on the
height of the peaks that are labeled in the mixture.
Q. Why did you choose to go with quant over peak heights
ultimately?
A. OCME has a large number of electrophoresis instruments, and
we found that the peak height was less useful in predicting the
probability of drop-out across all of the instruments than the
starting amount of DNA. So we tried looking at peak height to
see if we could get a good handle on the probability of
drop-out, and it was not as predictive of the drop-out
probability as the starting amount of DNA.
THE COURT: Just to be clear, is that because of the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
112
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON3 Mitchell - Direct
different machines that were used?
THE WITNESS: The machines were different, and we just
found that, in our hands, the quant was the better method. And
other labs may make a different decision and they might have a
different experience with peak heights versus quant in their
lab, but at OCME we found that the quant was a better
predictor.
THE COURT: OK. Go ahead.
Q. In the decision to use quant as one of the applicable
factors, was that described in the peer-reviewed publication
you wrote about the FST program?
A. Yes.
Q. And was that decision presented to the DNA subcommittee of
New York that ultimately had the sign-off to approve FST?
A. Yes.
Q. Now, based on what you described, did you determine some
empirical drop-out rates?
A. We did.
Q. Were any adjustments then made to those rates?
A. Yes. What we ended up using in the program was not the
estimates themselves but one standard deviation below the
estimate.
(Continued on next page)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
113
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
BY MR. MCKAY:
Q. And why did you choose to reduce or adjust it by one
standard condition?
A. So, the next slide shows a picture of the main motivation
for doing this.
So, I have two plots here, and these are both showing
likelihood ratios that were obtained from noncontributors to
mixtures, so people who were not part of the mock casework
samples that we made. We tested FST using people who really
did contribute to the mixtures and a set of people who really
didn't. So, this is -- there is a database of about 1250
noncontributors, and we ran FST using each of those people as
the suspect, if you will.
So the plot on the left shows the distribution of
likelihood ratios that we got when we were using the lower of
two possible drop-out rates. And the average there you can see
the peak there is about minus 18. That represents a likelihood
ratio of ten to the minus 18, so very, very small. So that's
good, we want noncontributors to generate very low likelihood
ratios so that they are excluded from the mixture.
On the right is a plot using a slightly higher
drop-out rate or slightly higher drop-out rates, and that
shifts the entire distribution up. So in that second plot the
average is now about minus 12. So using those higher drop-out
rates has shifted the distribution of likelihood ratios for
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
114
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
noncontributors a bit higher. So this is showing that if lower
drop-out probabilities are used, on average LRs will be
obtained for noncontributors. And that was really important
that we keep the likelihood ratios low for noncontributors so
we're not falsely including people. So, our main motivation in
underestimating the drop-out probabilities was to keep LRs for
noncontributors low.
Q. If I may just ask a few follow-up questions on that.
A. OK.
Q. First of all you used the term underestimating the drop-out
rate. When you say underestimating, what do you mean?
A. So that phrase is really only meaningful in the context of
a true contributor. So if we have a true contributor to a
mixture, we can look at how often that person's alleles drop
out, so that would be their true drop-out rate. And on the
next slide I discuss a side effect of our decision to use lower
drop-out rates in FST.
So, we were aiming to reduce the LR for
noncontributors, but what can also happen is it may also reduce
the LRs for true contributors, and that's because if we were to
use a true contributor's exact drop-out probability in the
calculation, we would be modeling that situation as well as we
could, so we would maximize the likelihood ratio for that true
contributor.
If we use a lower probability of drop-out for that
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
115
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
true contributor, the model doesn't fit as well, so it tends to
reduce the likelihood ratio for that true contributor. I will
show you some plots on the next slide where we tested that.
Q. Before we get to that, may I interrupt you for one second.
So underestimate means to have a drop-out rate lower than the
true drop-out rate; is that correct?
A. Correct, for a true contributor.
Q. So that is different from lowering the drop-out rate?
A. Yes, or it's a subset of lowering the drop-out rate.
To test this statement that I'm making saying that
underestimating the drop-out rate tends to lower a likelihood
ratio one must know what is the actual drop-out rate of that
real contributor, and then to compare what do you get if you
use that versus what do you get if you use something lower.
MR. MCKAY: If we could go back to the slide we were
just looking at, Mr. Deluca. I just want to clarify a few
things, if possible.
Q. So the X axis, the horizontal axis, what is the value on
that axis?
A. Those are the exponents on the likelihood ratio. So it's
sometimes also called a log likelihood ratio. So, for example,
the minus 30 means ten to the minus 30 was the likelihood ratio
obtained for those noncontributors.
Q. So ten to the minus 30 would be a very strong exclusion; is
that right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
116
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A. Absolutely, yes.
Q. And what is the zero on that X axis?
A. The zero represents a likelihood ratio of one. So that's
the point where the DNA evidence is not supporting one scenario
over another.
So, we were happy to see that this turned out to be at
the far right of the distribution. There are very few
observations to the right of zero. So, that is saying that
very few noncontributors in our validation generated a
likelihood ratio of greater than one. The vast, vast majority
are far less than one and in fact down to ten to the minus 40.
Q. So results that are left of the zero are noncontributors
for whom the likelihood ratio was less than one.
A. Yes. So those would be exclusionary likelihood ratios.
Q. And those small number of results to the right of the zero
are what you might call false positives?
A. Exactly.
Q. And when you compare the graph on the left to the graph on
the right, what does that tell you?
A. So, the graph on the left was the plot generated using the
drop-out probabilities that we use for mixtures that are
approximately one to one. So, these are lower drop-out
probabilities than the ones on the right. The ones on the
right are the drop-out probabilities used for the minor
contributor to an unequal mixture.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
117
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
So the ones on the left are lower, the ones on the
right are higher, the drop-out rates; and using higher drop-out
rates is shifting the noncontributor curve to the right. But
still at this point there are still very few observations above
zero, or very few likelihood ratios above one.
Q. And I think you touched on this, but the goal of
underestimating is what with respect to noncontributing
findings?
A. The goal of underestimating is to keep LRs for
noncontributors as low as possible while still being able to
generate higher likelihood ratios for true contributors. So
it's kind of a balance. If we're exclusively focusing on the
noncontributors -- we have to focus on both noncontributors and
contributors. We're trying to separate the two.
Q. To be fair, a noncontributor is someone whose DNA evidence
is not in the mixture.
A. Right, it's somebody who did not contribute any DNA to the
forensic mixture.
Q. So why is your primary concern lowering LRs for
noncontributing defendants?
A. Because these people did not leave any DNA on this sample,
whatever it is. They are not a contributor to the sample, so
they're not -- the evidence is not supporting their inclusion,
and we want to make sure that the likelihood ratio reflects
that as much as possible.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
118
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
Q. Does that underestimation come with a cost or side effect
with respect to true contributors?
A. It does. It can reduce the LR for a true contributor. So
maybe somebody who really did contribute to the mixture may not
generate as high a likelihood ratio as they would if we used a
higher drop-out probability, but it's a price we're not willing
to pay.
Q. And why are you not willing to --
A. I guess I didn't phrase that very well. We're not willing
to use the higher drop-out estimates because although that
would raise the LR of true contributors, it would also raise
the LR of noncontributors. So we're willing to take a little
bit of a hit on the true contributors, reducing their LR a
little bit, to try to guard against falsely including someone.
Q. Now, have you ever tried to test the argument that
underestimating drop-out lowers or reduces LRs for true
contributors?
A. Yes, that's on the next slide. We took 19 of our
validation samples, some two person mixtures, some three person
mixtures, and for each true contributor to those samples we
calculated their true drop-out rates, and we did the FST
calculation using those true drop-out rates. We then cut those
true drop-out rates in half and repeated the analysis so that
we could compare what do we get if we exactly get the drop-out
probability right, and what do we get if we underestimated the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
119
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
drop-out probability. So, these two graphs show the results
from that.
Q. Let me ask you first when did you do that research?
A. This was done -- this is a master's student's work that she
was using the validation samples, so they had already been
generated. I don't know exactly when she was doing this with
regard to the completed validation.
Q. Now, focusing first on the plot on the left, what do we
have on the X axis?
A. So, on the X axis or the horizontal axis, these are log
likelihood ratios, so this is the exponent on the likelihood
ratio that we obtained when we used the real contributor's real
drop-out rates.
So, for example, those two red dots on the left, those
are near minus three. That means for those comparisons, people
that really did contribute to the mixture, when we did this
analysis using their real drop-out rates we got to about ten to
the minus three LR. So there was probably quite a bit of
drop-out for those samples. Even though they were real
contributors, they actually didn't look like it because several
of their alleles had dropped out, so that's why they've
generated a low likelihood ratio.
Over on the right, so if you go up from six, there is
a red dot pretty high up. That person using their true
drop-out probability, a likelihood ratio of ten to the sixth or
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
120
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
one million was generated.
Q. What is on the Y axis?
A. The Y axis is a measurement of the log likelihood ratio
that we obtained when we used not their true drop-out rate but
half of that, so when we underestimated the drop-out rate what
we got.
So, again to use that top right red dot, the one that
generated a likelihood ratio of about ten to the fifth or
between ten to the fifth and ten to the sixth on the X axis,
they also generated a likelihood ratio of about ten to the
fifth or ten to the sixth using the underestimated drop-out
rates.
Q. And what are the dots?
A. So, each dot represents one true contributor, and it's
calculating the likelihood ratio using their true drop-out rate
and their underestimated drop-out rate.
Q. Why are some blue and some red?
A. The blue ones were samples amplifying OCME's standard
amplification protocol, so that's samples that are at least a
hundred picograms. The red dots that were generated from --
I'm backwards. The red dots are the standard protocol. ID28
means the Identifiler kit with 28 cycles. So the red dots are
the standard protocol. Then the blue dots are the high
sensitivity protocol. So those are the smaller amounts of DNA.
So I included that color coding to look for patterns
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
121
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
in one protocol versus another. But they both show the same
pattern, which is using these two drop-out rates -- either the
true rate or the underestimated rate -- the likelihood ratio is
either similar or lower when using the underestimated rates.
So, the points falling below the diagonal line are points where
the LR was lower using underestimated drop-out rates instead of
the true drop-out rates.
Q. So what is the significance of a point that is exactly on
the line?
A. A point exactly on the line would have generated a
likelihood ratio exactly the same using both of those drop-out
probabilities.
Q. And a point below the line is what again?
A. A point below the line is a point where underestimating the
drop-out rate gave a lower likelihood ratio than getting the
drop-out rate exactly right.
Q. And a point above the line is what?
A. A point above the line is where underestimating the
drop-out rate gave a higher likelihood ratio than using the
true drop-out probability.
Q. And so generally speaking what did you conclude about
underestimating drop-out rates for true contributors based on
this?
A. Generally we saw that underestimating the drop-out rate led
to a lower likelihood ratio for a true contributor than getting
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
122
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
it exactly right.
Q. Now, there are some examples where that was not true,
right?
A. Yes.
Q. And those are the dots that are above the line?
A. Yes.
Q. But generally speaking that conclusion held true?
A. Yes.
Q. OK. And what is the plot on the right?
A. The plot on the right is the same analysis but using the
three person samples. The one on the left was using two person
mixtures, and these are three person.
Q. And did you draw the same conclusions with respect to three
person samples?
A. We did. The results were either comparable using the
underestimated rates, or they were lower using the
underestimated rates, again because the points fall either very
close to the line or below the line.
MR. MCKAY: Your Honor, at this point I'm about to
move to a new topic which will take about 20 to 30 minutes.
Would this be a good time for the lunch break?
THE COURT: Sure. How long would you like for the
lunch break? It's now about 1:20. Do you want to come back at
about two?
MR. MCKAY: That works for us, your Honor.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
123
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
THE COURT: 2 o'clock? OK.
MR. MCKAY: Just timing wise, I'm about halfway
through, so if we came back at two, I would be done at let's
say 3:15 or 3:30.
Does that work on your timing?
MR. STRAZZA: Yes.
THE COURT: In other words, I think what he was trying
to ask is how long you anticipate your cross being.
MR. ROSANIA: I know. We had discussed this earlier,
and obviously that depends upon the answers to the questions,
but I think that generally works.
THE COURT: OK. All right. We'll take our lunch
break. We will stand adjourned until 2 o'clock. Thank you.
You may step down.
(Luncheon recess)
(Continued on next page)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
124
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A F T E R N O O N S E S S I O N
2:00 p.m.
THE COURT: You may continue the direct examination of
Dr. Mitchell.
MR. MCKAY: Thank you, your Honor.
BY MR. MCKAY:
Q. Dr. Mitchell, so are the calculations performed by FST, are
those calculations done by hand or using the computer program?
A. The computer program does it.
Q. And how long would it take to calculate a likelihood ratio
for a complex mixture by hand?
A. It would not be reasonably possible. How long would it
take? I've done it for one locus for a few alleles, and that
can take an hour maybe, but as the number of alleles increases
and the number of loci increase it would not be reasonable to
do it by hand. It's not a difficult calculation; it's just
multiplying and adding and dividing, but just very tedious.
Q. The volume of the number of calculations to be done
requires a computer program.
A. Yes.
Q. So I won't ask you to calculate a full LR, but can we walk
through an example or simplify an example of how the math
works?
A. Yes.
Q. Go ahead.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
125
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A. OK. So this is an example. Let's say we have a two person
mixture and this is an intimate sample from the victim, and
there is a suspect profile, and the victim profile is available
as well. The question of interest here would be is the suspect
a contributor to this mixture. So in this first situation that
I will show, the suspect and the victim together could
completely explain the evidence sample. So on the next slide
is a hypothetical four locus profile.
Q. Just one second. In this first example, are you going to
factor in drop-in and drop-out or no?
A. I am not. Firstly show how it's calculated without
modeling drop-out and drop-in.
So, in this evidence sample if you were to put
together the suspect and the victim profiles, you would obtain
the evidence results. And if one is not modeling drop-out and
drop-in, this must be the case in order to compute a likelihood
ratio.
The next slide, please. The prosecution hypothesis in
this example will be that the sample originated from the victim
and the suspect. And the defense hypothesis or defense
scenario will be that the sample originated from the victim and
an unknown unrelated person. So, the likelihood ratio
calculation is shown below. That PR parentheses means
probability. And the vertical bar means given. So this first
fraction here we're going to calculate the probability of the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
126
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
evidence if the prosecution scenario is true, or given that the
prosecution scenario is true.
We will then calculate the probability of the evidence
given that the defense scenario is true.
So in the numerator we are calculating the probability
of the evidence if the contributors are the victim and the
suspect. And in the denominator we will calculate the
probability of the evidence if the victim and an unknown
unrelated person are the contributors.
Next slide. When we're not accounting or allowing for
drop-out and drop-in, in the numerator the probability of
getting that evidence sample if the true contributors are the
victim and the suspect is one, because it's made up of exactly
what the victim and the suspect have at those loci. So if you
put them together, there is a 100 percent chance that you will
get that evidence mixture if no drop-out and no drop-in are
considered. So the numerator is easy.
The denominator we are going to replace the suspect
with a randomly suspected person and examine the situation that
the sample comes from the victim and an unknown unrelated
person.
Next slide. So, in the denominator, this unknown
contributor would have to account for any alleles in the
mixture that are not explained by the victim. So, the evidence
at the first locus is 14, 15, 16, and the victim has a 15 and a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
127
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
16. Therefore, the second contributor, whoever it may be, must
provide a 14 to the mixture. And at the second locus they
would need to provide a 29, as the victim explains the 31, but
the 29 would be unexplained by the victim.
If we go to the next slide, I have listed out all the
possible genotypes of the unknown person. At the first locus,
the evidence contains a 14, 15, 16. The victim is 15, 16. So
the second contributor must have a 14. But their second allele
could be a 14 or a 15 or a 16. So, there are three possible
genotypes for a second contributor at that locus.
At the second locus the victim has a 31 allele and the
evidence has a 29 and a 31. So the unknown contributor could
be homozygous for the 29 allele, or they could have a 29 and a
31.
At the third locus, because there are four distinct
alleles and two of them could be attributed to the victim, the
other two must come from the second contributor. So at that
locus there is only one possibility for the second contributor,
and that's 8 and 11.
On the next slide I have taken each of these genotypes
and estimated population frequencies of those genotypes, so
that's the denominator of the likelihood ratio there.
In that first factor P14 squared is the estimated
population frequency of people with a 14, 14 genotype in the
population. The two times P14, P15 represents the estimated
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
128
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
frequency of people with a 14, 15 genotype, and the same with
the 14, 16.
Q. If I can stop you for just one second. Where do the
estimated population frequencies of people with any particular
genotype, where do those figures come from?
A. There are several options, more than several options for
estimating these frequencies. OCME developed its own
population database based on post-mortem samples from the City
of New York, and they've estimated allele frequencies for four
separate populations within New York: Caucasian, African
American, Hispanic and Asian individuals.
Q. You can continue. Thank you.
A. OK. So, at this point all that would be necessary would be
to plug in the frequencies of those alleles or those expected
genotype frequencies in each of those factors. The denominator
here -- sorry. I lost my train of thought.
Next slide. So the value of this likelihood ratio is
going to depend on the frequency of the alleles in the mixture
that match the suspect and can't be explained by the victim,
because those are the ones that we would need to draw from the
population. So, we want to know what would be the chance that
we would draw this from the population if in the denominator
when our suspect is not in the model. So if there are alleles
that are rare, that the suspect has that are in the mixture and
not explained by the victim, it would be unlikely that a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
129
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
randomly chosen person would carry them, so that ends up
increasing the LR, because it decreases the probability that
this mixture would happen by chance from the random population.
A high likelihood ratio means that a mixture is
explained better with the suspect than without them, and
conversely a low likelihood ratio would mean the mixture is
explained better by a randomly chosen person than by the
suspect.
So in the next slide I have assigned hypothetical
allele frequencies here and just plugged those numbers in down
below, and in this circumstance I circled the alleles that
matched the suspect in this example, and so that using these
fairly common alleles like a 33 percent allele frequency, for
example, for allele 14 at locus 1, we end up with a likelihood
ratio of 87, just using these values.
On the next slide I show the interpretation of that
number. So the 87 means the DNA mixture found on this item is
87 times more probable if it came from the suspect and the
victim than if it came from the victim and an unknown unrelated
person. And then OCME applies a descriptive scale to these
values, and the interpretation there would be that there is
moderate support for this being a mixture of DNA from the
suspect and the victim rather than the victim and an unknown
unrelated person.
The scale is shown on the next slide, so these are the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
130
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
descriptors that OCME uses for likelihood ratios in the range
of one to ten. That would be limited support for one
hypothesis over the other. Ten to 100 would be moderate
support. 100 to 1,000 is strong support, and greater than
1,000 is very strong support. This scale was suggested by a
couple of publications by solid forensic scientists Bruce Weir
and John -- not John Buckleton. I'm blanking.
Q. Is it John Butler?
A. No, it's not John Butler. It's David Balding is the 2005
publication. The 1998 publication is Bruce Weir, W-e-i-r.
So if a likelihood ratio turns out to be less than
one, the reciprocal is taken, and it's reported in this same
fashion saying the evidence is providing limited support for,
for example, the victim and an unknown unrelated person rather
than the victim and the suspect. So the scale is the same
whichever way the LR ended up going.
The next slide shows what might happen if instead of
having the suspect's alleles be quite common, I have assigned
lower frequencies to those alleles just to show that alleles
that are very rare can change the likelihood ratio.
So, in this example that allele 14 which was carried
by the suspect, it would be unusual to draw that allele from a
random population, so that will increase the support for the
suspect being a contributor over an unknown person being a
contributor.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
131
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
Q. So using those less common allele frequencies, what was the
result that you got?
A. So this result turned out to be a little over 21 million,
and so on the next slide the interpretation would be the DNA
mixture found on this item is 21.6 million times more probable
if it is a mixture of DNA from the suspect and the victim than
if it is a mixture from the victim and an unknown, unrelated
person.
So in this example we would say that this is very
strong support for this being a mixture of DNA from the suspect
and the victim compared to the victim and an unknown, unrelated
person.
Q. Can you now do a simplified example where we factor in
drop-in and drop-out?
A. Yes. So we did not allow drop-in and drop-out in that
previous example, but now we will, but I will do just one
single allele rather than the four loci with multiple alleles.
So when we start to think about drop-out and drop-in,
we need to allow each contributor's alleles to be missing --
whether these are the known individuals or the unknown
person -- and we need to allow for extra alleles that are not
explained by the contributors in each particular scenario.
So then on the next slide I will show what I will be
modeling. So this will be a single source sample, which is not
something that OCME uses FST to analyze but just for the sake
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
132
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
of a simple example.
So, in this example the scenario in the numerator will
be that this DNA -- the suspect is the source of this DNA --
and in the denominator an unknown, unrelated person will be the
source of the DNA.
So we will set up the likelihood ratio in the same
way, but for each contributor or potential contributor we will
also need to ask if this person is a contributor to the sample,
did any of their alleles drop out; and also we will need to ask
if this person is the source of the DNA, are there any extra
alleles that we would call drop-in, if that person is the true
source.
So the next slide shows which locus I used. So I'm
going to use locus 3 where there was an 8 allele in the
evidence, and the suspect has an 8 and a 12. So on the next
slide I will show how I set it up.
In the previous example when we did not include
drop-out and drop-in, the numerator was 1, because if you put
those two people's profiles together, you would get the
evidence with the probability of 1.
Here the numerator will be less than one, because it's
not certain that you would get that result in the evidence if
drop-out is allowed. And we're also going to allow unknown
contributors' alleles to drop out.
So, on the next slide in the numerator the suspect is
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
133
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
an 8, 12 and the evidence is an 8, so the probability of
getting that evidence if the suspect is the source of the DNA
is going to be the probability that one allele from a
heterozygote drops out and the probability that no additional
alleles drop in, because that's what would need to happen if
the true contributor is an 8, 12 and the evidence is an 8.
In the denominator we let an unknown contributor have
a genotype that's made up of any combination of the alleles
seen in the mixture and any other alleles that didn't show up.
So, we will let this person have an 8 and an 8 or an 8 and
something else that we call W, or two something elses, W, W.
So, we will need to find the population frequency for each of
those and then multiply each of those by the probability of
drop-out and/or drop-in that would be required to attain the
evidence sample.
Next. So the numerator is going to be the probability
of one heterozygous drop-out times the probability of no
drop-in. In the denominator we let the unknown person have
genotype 8, 8; 8 something else; or two copies of something
else. So if that person is an 8, 8, we need to know what's the
probability that there is no drop-out from a homozygote, and
multiply that by the probability of no drop-in. Because if the
true source is 8, 8, and we see an 8, there was no drop-out and
there was no drop-in. If the source is 8 something else, one
of those alleles dropped out, so we need to know the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
134
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
probability of one drop-out from a heterozygote; and again no
drop-in because there are no extra alleles there.
In the final situation, if the true contributor has
two alleles that don't show up in the mixture, we need to know
what's the probability that those alleles don't drop out, but
then we need one drop-in in that example because an 8 shows up
in the mixture.
So here is what it is if we put these in as variables.
So in the numerator we have the probability of one drop-out
from a heterozygote -- that's D1 -- times no drop-in -- that's
the C zero. Then in the bottom those three terms represent an
unknown contributor with genotype 8, 8, where that homozygous
allele did not drop out and no alleles dropped in. And the
second term represents an unknown contributor with genotype 8W,
where one allele dropped out and no alleles dropped in. And
finally they can have two copies of something else, both of
which dropped out, and then the 8 would have to be a drop-in.
So it's not difficult mathematically. It can just
become very tedious to write out all of these possibilities,
especially if there are several unknown contributors in this
scenario.
Q. Now, have you written up a third example on the slides here
as well?
A. I have.
Q. And what does that example show?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
135
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A. That would be example 3, the next slide, which I can go
into or I don't need to go into.
Q. Well, let me ask this. What does example 3 show compared
to what we've just seen?
A. So example 3 shows a mixture, a mixture where we're
allowing drop-out and drop-in, and we have in the numerator we
have the suspect and one unknown unrelated person, and in the
denominator we have two unknown unrelated people. So, for each
of those unknown people we would have to list out all of their
possible genotypes, estimate those frequencies and then
consider whether drop-out and/or drop-in would have had to
occur to give us the mixture that we got.
Q. In terms of the calculations that you performed in your
next example, are they the same or different from the
calculations you just did?
A. They are the same; there are just more of them.
MR. MCKAY: Unless the court feels otherwise, I think
in the interest of time we may move on from there.
THE COURT: No, we can move on as long as you don't
ask me to try and do it.
Q. So, let's move on and talk about the second -- you said
earlier there were two phases in the development of FST; is
that right?
A. Yes.
Q. And what was that second phase?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
136
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A. The second phase was the validation.
Q. What does it mean -- what does validation mean in this
context?
A. Validation is a procedure performed by any forensics lab
when they are about to start using either a lab protocol or an
analytical protocol that they have not used before. So they
need to demonstrate that it functions as expected in their
hands.
Q. And do labs validate the protocols and procedures of other
labs?
A. No, a lab will only validate something that they themselves
are planning to use.
Q. The committee that you mentioned earlier, the subcommittee
that you were on, remind us what sorts of standards were you
setting on that subcommittee?
A. We were setting up guidelines for how a lab should go about
validating a probablistic genotyping software tool.
Q. So when did the validation for FST begin approximately?
A. It began midyear 2009.
Q. Approximately how long did it take?
A. About a year and a half.
MR. MCKAY: And, Mr. Deluca, if you will bring up the
slide show again.
Q. Would you explain to the court how the validation for FST
was designed and carried out.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
137
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A. Yes. So the next slide shows the design. So we generated
mock case work samples of a variety of types. We created
mixtures of blood or saliva -- or DNA isolated from DNA or
saliva from two or three people. We had samples that were
handled by two, three or four people. And we had some samples
that we purposely degraded using UV light. And these samples
were processed according to the OCME case work protocol. That
protocol includes estimating the number of contributors to the
mixture, determining if this is a mixture ratio that is
approximately one-to-one, or if it's not one-to-one and a major
contributor's profile could be determined.
We then compared -- we then calculated a likelihood
ratio using FST for each of the true donors to those samples,
and we also computed FST for each noncontributor in a database
of approximately 1250 noncontributors.
So, for each sample we could tell what do the
likelihood ratios look like for the people who really did
contribute to the sample, and what do they look like for people
who did not contribute to the sample.
Q. I will ask you about the results of those tests in a
second. But first can you talk more about the manner in which
you created the samples?
A. Yes. So, on the next slide there is a description of the
different types of samples that we used.
So, we looked at both two person and three person
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
138
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
mixtures, and we looked at low template and high template
samples to make sure that we were covering all of those
options. And we ended up with 439 samples that fit the case
work criteria for going forward with analysis. Not all of them
did. Some of them, for example, didn't have enough loci that
amplified, or it turned out to be a single source sample, or
had too many alleles, and it didn't fit our three person
mixture criteria. So there were 439 in the end.
We ran these in FST using the assessments of the
mixtures that were made without knowing who the true
contributors were, or how many there were, or knowing anything
about the sample going into it.
Q. If I can interrupt for a second. How did you decide how
many mixtures to make?
A. Well, we just kept making more and more mixtures until it
was approaching the time that we were going to meet with the
DNA subcommittee and represent the results to them. So it was
essentially as many as we could in order to get into this
meeting where we presented the final validation.
Q. Did the scientific community have any guidelines or
guidance at this time about how many mixtures should be made
for validation?
A. There were some guidelines that were put out by SWGDAM,
which is the scientific working group on DNA analysis methods
that develops guidelines for all kinds of forensic lab
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
139
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
protocols, and they had recommended using at least 50 mixtures
in a validation study.
Q. If you turn in that big binder in front of you to
Government Exhibit 15, can you tell me whether you recognize
that document.
A. Yes.
Q. What is it?
A. These are the SWGDAM guidelines for validations in forensic
labs that were put out in July 2004.
MR. MCKAY: The government offers Exhibit 15.
MR. STRAZZA: No objection.
THE COURT: OK. Government Exhibit 15 is admitted in
evidence.
(Government Exhibit 15 received in evidence)
Q. And if you turn to the second page and look down to section
3, is that the recommendation that you were referring to?
A. Yes, it is.
Q. And how many samples does SWGDAM recommend using?
A. A total of at least 50 samples.
Q. How many did OCME use?
A. In this validation we used 439 samples.
Q. Now, did many of those samples use some of the same
contributors as other samples?
A. Yes.
Q. Does SWGDAM or any other -- or did SWGDAM or any other
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
140
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
forensic body provide any guidance on how many different
contributors you should use?
A. No, there is no guidance.
Q. And was the fact that some of the mixtures contained the
same contributors as other mixtures disclose to the DNA
subcommittee that ultimately approved FST?
A. Yes.
Q. And what did the subcommittee say about the number of
samples tested?
A. One of the subcommittee members, John Carmedi, who is a
population geneticist said it was the largest validations he
had ever seen.
Q. Are you familiar with the program Lab Retriever?
A. Yes.
Q. And are you familiar with how Lab Retriever is validated?
A. I saw a presentation at a meeting of the American Academy
of Forensic Sciences in which one of the developers of Lab
Retriever presented a validation that they had performed using
two -- DNA from two donors that they had amplified each 20
times.
Q. So before I interrupted, you were working through the
PowerPoint.
Mr. Deluca, can you bring that up, please.
And I think you might have been on slide 52.
A. OK.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
141
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
Q. And continue explaining the validation.
A. OK. So the validation mixtures included touched samples --
which often are encountered in case work -- and some
nondegraded mixtures of saliva and blood, and then it also
included some mixtures where one or both of the components were
degraded by UV light.
Some of the mixtures that we used for testing were
mixtures that we had initially developed with the intention of
using them for drop-out rate estimation, but then we did not
use them for drop-out rate estimation, but we used them for
validation, and they contained mixture ratios that were
different than those that were raised to estimate the drop-out
probabilities.
The samples were processed as if they were case work
samples. So DNA was extracted, quantified, amplified,
separated and analyzed according to OCME case work protocols.
Once the mock case work sample was processed, they
were classified as suitable or nonsuitable for comparison based
on OCME case work protocols. The number of contributors to the
samples were estimated and, if possible, major and/or minor
contributor profiles were determined.
And all of this was done without knowing the true
composition of the mixture or anything about the mixture.
Q. You talked about classifying samples as suitable for
comparison. Can you elaborate on that?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
142
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
A. There are some things that can happen with an evidence
sample that would deem it not suitable for comparison. For
example, I mentioned earlier that I had consulted on a case
where another lab had drawn conclusions from a sample where
only two loci had results, so that would be far below OCME's
minimum number of loci to go forward with a sample, for
example. And there are other indications that could deem a
sample not suitable for comparison.
Q. These samples, was a visual analysis of them done?
A. Yes. So these are the categories that OCME previously used
to qualitatively describe the relationship between a suspect
profile and an evidence mixture. So if all of the comparison
samples' alleles were labeled in the mixture, they would say
that this person was included as a possible contributor to the
mixture. If most but not all were labeled, this person could
not be excluded. And it's a little bit fuzzy here, most if not
all, because some loci are more robust than others, so if a
comparison sample or a suspect profile, if alleles were missing
at a couple of the loci that are very robust and we expect them
to amplify, we may not be as forgiving of them missing, so that
might move it down to a no conclusions category. But if a
couple of alleles were missing at one of the loci where
drop-out is a common occurrence, then that person would not be
excluded from the mixture.
If a couple more are missing, no conclusions can be
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
143
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
drawn. And if several are missing, that person would be
excluded as a contributor to the mixture in a case work
protocol.
Q. Now, during the validation when a manual evaluation yielded
a conclusion of excluded, was the sample nevertheless run
through FST?
A. We did run it through FST just to see what it would look
like. In case work if a comparison person is manually excluded
from a mixture, FST is not run.
Q. And in case work, is FST run when the manual call is no
conclusions?
A. It is not.
Q. Did it used to be for no conclusions?
A. It.
Q. And at some point that change was made?
A. At some point that changed, yes. And I was not part of
implementing that change; I don't know what the rationale was
behind it.
Q. But today in practice FST would only be used if "included"
or "cannot be excluded" was the manual call?
A. Yes, that's true.
Q. But in the validation study all samples were run through
FST regardless?
A. Yes, we wanted to see how it performed on all types of
comparisons.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
144
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
Q. And what were the results of the performance for all types
of comparisons?
A. If you go to the next slide, so looking at the mixtures
that were categorized as two person mixtures by the case work
protocols, some of these may have been three person mixtures
where one of the contributors dropped out enough that it
reached the two person criteria. These were the four
categories. Each donor -- each true donor was categorized as
"included" or the major contributor, "could not be excluded,"
or "no conclusions," or "excluded." And then FST was run using
each of these true contributors as a suspect, and the model
that was used was the suspect plus one unknown person in the
numerator and two unknown people in the denominator. So it's a
two person mixture where we don't have a known contributor to
the mixture.
Q. So in this slide, is it right that 119 true contributors
were manually excluded?
A. Yes.
Q. So how can it be the true contributor would be excluded
after a manual evaluation?
A. If several alleles were not labeled in the mixture, that
person would be manually excluded, even though in these samples
those people really were contributors, but there was enough
drop-out that they didn't look like contributors.
Q. And if we look at the next slide, can you talk to us about
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
145
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
the results.
A. Yes. So, this shows the likelihood ratios or actually a
log of the likelihood ratios, so that the exponent of the
likelihood ratio on the vertical axis. So at the top is ten,
which means that's a likelihood ratio of ten to the ten. The
zero represents a likelihood ratio of one. So that's the level
where we would say this mixture is not supporting one scenario
over the other. And then results that are below zero are
likelihood ratios that are less than one, and those are the
ones that support the scenario that includes the unknown
contributor in place of the suspect, or in this case these are
all actually true contributors.
Q. So if we're looking at the plot on the left, the first of
the four boxes, what is the manual call that that corresponds
to?
A. These were contributors that were visually included in the
mixture, so their alleles showed up at all loci that were
tested.
Q. What is the heavy bar in the middle of that?
A. The heavy bar is the median result. So the median here was
about ten to the ten. But there is a spread. You can see that
one of them, one of the observations is down close to one.
This could happen in a situation where all of a person's
alleles do show up in the mixture but they are all very common
alleles, so it's not lending much support, if any, to that
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
146
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
individual being a contributor to the mixture based on the
likelihood ratio.
Q. And if the heavy bar is the median, what are the thinner
bars on either side of the heavy bar?
A. The bar just below the heavy bar is the 25th percentile,
and the bar just above the heavy bar is the 75th percentile.
Q. And the dotted line and the thin line that goes below it,
what do those represent?
A. So, in some cases, for example, in the graph on the right,
the second box plot, there are no dots beyond the ends of those
whiskers, so those represent the lowest value obtained and the
highest value obtained. And with this type of plot if there
are observations that are beyond one and a half times the
height of the box, above the box, those are represented by
dots, above or below the box, with a distance of one and a half
times the box height away. So those dots are still
observations; they are just kind of extreme observations.
Q. So as you move to the second column here that's labeled
CBE, what are you looking at with that box?
A. Those were comparisons for which the donor was deemed
cannot be excluded. So in those comparisons generally one or
two of the donor's alleles were missing from the mixture. So
this now the median is down from about ten to the ten to ten to
the fifth.
Q. However, in "cannot be excluded" do you have some results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
147
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON4 Mitchell, direct
that are below a likelihood ratio of one?
A. Yes. So, some of these individuals who are missing one or
two alleles, the likelihood ratio actually turns out to be
quite low, less than one.
Q. And without walking through each and every one of these
boxes, generally speaking what is the conclusion that you draw
from these charts?
A. So as we go left to right across these categories we're
going from a situation where all of the donor's alleles show up
in the mixture to more and more of the alleles are missing from
the mixture. And as would be expected, the likelihood ratios
go down as more and more of the contributor's alleles are
missing.
Q. So moving to the next slide, what does that tell you about
FST's performance relative to a quantitative assessment?
A. The likelihood ratios were consistent with the qualitative
assessments but they provided more information.
Q. And did you generate similar results for three person
mixtures?
A. We did. The next slide shows the break-down of the three
person mixtures, or these again are the mixtures that were
estimated to include three persons by case work protocols.
Q. And moving to the next slide, what were the results for the
three person mixtures?
A. And the results were similar, although for three person
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
148
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
mixtures on average they were a little bit lower. So, if you
recall, the two person mixtures the included category was
around ten to the tenth, and now it's down to maybe ten to the
sixth.
THE COURT: By they, are you referring to the
likelihood ratio?
THE WITNESS: Yes. Yes.
THE COURT: Go ahead.
Q. That's the median likelihood ratio?
A. The median likelihood ratio was a bit lower for the three
person mixtures than the two persons.
Q. So, overall what was your conclusion about the performance
of the likelihood ratios as opposed to qualitative assessments?
A. The likelihood ratios are consistent with the quantitative
assessments, but they provide more information and sometimes
very valuable information, such as a "cannot be excluded"
categorization sometimes generates a likelihood ratio that is
above one and sometimes it's below one. And this would be
really important information in a case. If the manual
assessment was this person cannot be excluded, sometimes the
more granular result provided by a likelihood ratio could
actually be the other way.
(Continued on next page)
BY MR. McKAY:
Q. Now, was part of the validation study what you might call a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
149
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
false positive study or noncontributor testing?
A. Yes.
Q. Could you describe what that entailed.
A. Yes. We wanted to know if someone who did not contribute
to the sample was tested, what did the results look like. How
often would somebody who didn't contribute to the sample
generate a likelihood ratio of greater than one or greater than
ten or greater than a thousand?
Q. So what did you do to perform that test?
A. So we used a database that we had. It's a population
database, the one we used to estimate allele frequency in New
York City. That included just over 1,200 individuals, and we
treated each of those people, one at a time, as the suspect in
all of these validation mixtures. And we computed the
likelihood ratio under the scenario that they were a
contributor compared to the scenario that they were not a
contributor.
Q. What were the results of those comparisons?
A. So overall, on the next slide we did about a half a million
comparisons with these noncontributors, and about .03 percent
of the time a noncontributor gave a likelihood ratio greater
than one.
Q. When you say .03 percent --
A. .03 percent.
Q. That's not three -- it's not 300 percent?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
150
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. It's three per 10,000.
Q. Moving to the next slide, what does this show you?
A. This shows the distribution of the likelihood ratios for
noncontributors. So these are similar to the plots that I
showed earlier where the zero on the X axis represents a
likelihood ratio of one. So there are very few observations to
the right of that. Very few likelihood ratios greater than one
were obtained. These were the two-person samples or the
samples that were deemed two-person by casework protocols. And
I'm showing you the results for the mixtures that were one to
one on the right, and the ones that were not one to one on the
left.
Q. Everything to the left of the zero is a likelihood ratio of
less than one, which is a result that favors the defense
hypothesis; is that right?
A. Yes. And on average, these were very, very low. So ten to
the minus 20 is there in the middle.
Q. It's only the data shown to the right of the zero that is
what you would call a false positive?
A. Yes.
Q. Moving on to the next, what does this one show you?
A. This is the same set of plots for the mixtures that were
deemed three-person samples by the casework protocols. So,
again, the average result is somewhere in the 10 to the minus
10 to 10 to the minus 20 range, and there are very few
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
151
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
observations where the likelihood ratio is greater than one.
Q. Moving to the next slide, what does this show you?
A. So the number that I mentioned before, .03 percent, the
three per 10,000, that is the frequency with which likelihood
ratios greater than one were generated for noncontributors.
That happened 163 times in the half a million tests that we
did. Fifty-six times a likelihood ratio greater than 10 was
generated. So that's a .01 percent rate. Fourteen times there
was likelihood ratio greater than 100. Five times there was a
rate greater than 1,000. And only one time there was one
greater than 10,000. And that was a situation where it was a
three-person mixture and one of the noncontributors in the
database, I believe all of that person's alleles showed up in
the mixture by chance. There may have been one missing. I
can't remember for sure.
Q. The first number, 163 --
A. Yes.
Q. -- does that encompass the 56 and the 14 and the 5?
A. Yes.
Q. The 1?
A. Yes.
Q. What does the distribution of this chart tell you about
where you'd expect to see false positives?
A. Most of our false positives were between 1 and 10,
likelihood ratios between 1 and 10.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
152
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. So if you get a likelihood ratio that favors the
prosecution in the very strong support category, so greater
than 1,000, what was the rate of false positives?
A. That was .0009 percent. So five out of half a million.
Q. What does that tell you about the relationship between the
strength of the likelihood ratio and the possibility of a false
positive?
A. The higher the likelihood ratio, the less likely it is that
a noncontributor would generate a likelihood ratio that high by
chance.
Q. We talked earlier about the concept of a manual exclusion?
A. Yes.
Q. Remind us again what that is.
A. So the casework protocol, if an analyst looks at a mixture
and looks at a comparison profile and manually excludes that
person from the mixture, for example, if several alleles are
missing, they do not go on to generate a likelihood ratio.
That's where the analysis stops. It's an exclusion.
Q. And that's in real world, in casework?
A. That's in real world casework. We went on to compute the
likelihood ratio for all of those just to see what they looked
like, but in casework, if a sample is deemed no conclusions can
be drawn or this person is excluded, the LR is not computed.
Q. Did you go and count in this chart how many of these would
have been manual exclusions in casework?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
153
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. We did not.
Q. In theory, though, is it possible this chart encompasses a
number of samples that would have been a manual exclusion?
A. Sure, yes.
Q. In fact, would you expect that it was likely that some or
even many of these were manual exclusions?
A. I would and expect that. And some, if not many, would be
in the no conclusions category.
Q. Which under current protocols would not be run through FST?
A. Correct.
Q. In fact, a number of these samples in real life, in
casework, may not actually generate a statistic under FST?
A. That's true.
Q. If you'll turn in your binder to the tab of Government
Exhibit 7, which is the very hefty one. Do you recognize is
that?
A. Yes.
Q. What is that document or series of documents?
A. These are the executive summaries from the 22 volumes that
comprise the validation.
MR. McKAY: Government offers Exhibit 7.
MR. STRAZZA: No objection.
THE COURT: Government Exhibit 7 is admitted in
evidence.
(Government's Exhibit 7 received in evidence)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
154
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
BY MR. McKAY:
Q. You said this is the executive summary. This is not the
whole validation?
A. That's true. The whole validation included one or two
three-inch, three-ring binders per volume, and there are 22
volumes. These are just the summaries.
Q. When you were describing the validation process earlier,
you mentioned degradation. Can you tell us what degradation
is.
A. Degradation is when DNA is broken down by environmental
factors such as UV light or heat.
Q. Did you attempt to account for degradation in the
validation study?
A. We did. At one point we estimated drop-out rates for
degraded samples which were higher than the drop-out rates for
non-degraded samples, and then we looked to see whether using
those higher rates improved the LRs for true contributors, so
did they generate higher LRs for true contributors and did they
also maintain the lower LRs for noncontributors? And we found
that we did not improve the separation between the two groups
using this degraded module to the program.
Q. So what did you do?
A. So degraded samples are now analyzed through the standard
protocol, and we did include a set of degraded samples in the
validation as well.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
155
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. Was what you just described about how you addressed the
issue of degradation, was that presented to the DNA
subcommittee that ultimately approved FST?
A. Yes.
Q. Was it published in your peer-reviewed article about the
FST validation process?
A. Yes.
Q. Would you say that FST is validated for use with degraded
samples?
A. Yes.
Q. How so?
A. We included degraded samples in the validation. They are
sometimes less informative than non-degraded samples; meaning,
likelihood ratios are closer to 1 than they would be if we had
a full -- full, non-degraded sample.
Q. What is the significance or what role does race play in
determining a likelihood ratio?
A. So at the forensic loci, the alleles occur at somewhat
different frequencies in different populations globally. So an
allele that might be quite common in one population could be
fairly rare in another.
Q. So the math that we were looking at earlier, when you were
plugging in allele frequencies, that may differ by race?
A. Yes. So OCME calculates the likelihood ratio four times
using its four different New York City population allele
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
156
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
frequency estimates, and then the value that's reported in the
casework report is the lowest of those four values.
Q. So FST generates not actually one likelihood ratio but
four?
A. Yes.
Q. But the one that is reported is which?
A. The lowest of the four. We don't know the ancestry of the
contributors to any forensic sample, so we compute it four
times and take the lowest.
Q. Did you conduct separate validation studies for the races
of each given contributor?
A. All four calculations were done for each of the validation
samples, and the values that are reported are the ones that
would be reported in casework, the lowest of the four.
Q. Were there any population geneticists on the DNA
subcommittee that approved FST?
A. Yes, there were to: George Carmody and Ranajit
Chakraborty.
Q. Did either of them express any concern to you about the
racial composition of your validation mixtures?
A. They did not.
THE COURT: At the time?
MR. McKAY: At the time.
Q. When you've been describing the manner in which FST reports
its conclusions, I think you've used the term "unknown,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
157
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
unrelated." What is the significance of unrelated here?
A. In the likelihood ratio, we model replacing the suspect
with an unknown, unrelated person. In reality, the contributor
to a mixture may in fact be someone who is related to the
suspect, and so in that circumstance, the calculation -- the
population frequencies are different than they would be for a
relative of the suspect.
Q. Would you expect that two brothers would have alleles that
are more similar or less similar than a randomly selected
member of the community?
A. They would have alleles that are more similar.
Q. Did the validation study discuss at all the question of
relatedness?
A. No.
Q. Did you consider whether to incorporate relatedness in FST?
A. That was something that was on the list of things that I
wanted to do.
Q. But to be clear, FST as currently formulated does not
consider the question of relatedness?
A. That's true.
Q. You mentioned that FST issues its report. Does it specify
that it's not purporting to address relatedness?
A. It reports it as unknown, unrelated person.
Q. In a case in which the defendant made a claim or there's a
defense hypothesis that a relative was a possible contributor
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
158
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
or possible suspect, could defense counsel present that to OCME
and ask them to do any testing on the relative?
A. Yes. If they could obtain a DNA sample from the relative,
that could be done.
Q. Would OCME then test that DNA sample?
A. I believe that they would, but I was not involved in that
work at OCME.
Q. Has FST been peer reviewed?
A. Yes.
Q. In what ways?
A. The publications have been peer reviewed as described
earlier, and the DNA subcommittee reviewed the validation, the
development and the validation in great detail. In addition, I
and others have presented the work in progress and the final
validation and discussion of other areas of interest
surrounding OCME at -- I'm sorry, surrounding FST at national
and international forensics meetings.
Q. You mentioned the DNA subcommittee and we made reference to
it, but what is the DNA subcommittee?
A. Any new lab or analytical procedure -- any procedure that
is new to a forensics lab in the state of New York must be
approved by the New York State Commission on Forensic Science.
The DNA subcommittee handles the details of any DNA-related
procedure that's new, and the DNA subcommittee makes a binding
recommendation to the larger committee about whether or not to
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
159
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
approve a new DNA protocol.
Q. Who are some of the members of the DNA subcommittee at the
time that FST was considered and approved?
A. I've already mentioned George Carmody and Ranajit
Chakraborty, population geneticists. Jack Ballantyne was the
chair of the committee. He is or was faculty of University of
Central Florida and is a very well-known, well-respected
forensic geneticist. In addition, Eric Buel was on the
committee. He is the director of the Vermont Public Lab,
public DNA lab. In addition, there was a molecular biologist
by the name of Mark Batzer, and a public health epidemiologist,
an M.D., named Ann Walsh. And Charles Hirsch, the chief
medical examiner at the time, was also on the DNA subcommittee,
although he did not vote on OCME-related protocols.
Q. Is it fair to say that the subcommittee is a collection of
experts in the fields that are relevant to forensic DNA work?
A. Yes.
Q. What is your understanding of what the subcommittee's role
was or function was in reviewing a program like FST?
A. We met with the subcommittee four times over the course of
a year and a half, and the initial meeting was one where we
presented what we were interested in doing and some very
preliminary work that we had done. Then we met with them for
two progress reports where we presented our results thus far
and got feedback from them, and they at various times asked us
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
160
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
to do some additional work or to answer some additional
questions. Then finally, in October of 2010, we made a final
presentation to them, and they approved the method for
casework.
Q. Before these meetings with the subcommittee, did you
provide any materials to them?
A. Yes. At least two weeks in advance we sent them a summary,
including -- either exactly an executive summary or something
close to an executive summary that's presented here for them to
review prior to the meeting.
Q. What kind of materials did you present to them at the
actual meetings?
A. At the actual meetings, we had summary slides, but we also
had the material they sent them ahead of time that we could
refer to and they could refer to, to get into more detail of
what we had done.
Q. What happened at those actual meetings? What did you talk
about?
A. We talked about questions that we were working on. For
example, if we were going to implement the degraded protocol,
we talked about the numbers of samples that we were generating.
We showed the validation results as they developed. They asked
us questions about our methods as well as about our samples.
Q. You said they asked you questions. Were they substantive
questions?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
161
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. Yes.
Q. Did they have the opportunity to ask you to do additional
research?
A. They did.
Q. Did they, in fact, ask you to do additional research at
times?
A. They did.
Q. Did you also have informal interactions with members of the
subcommittee outside the context of the meetings?
A. Yes.
Q. If you turn in your binder to Government Exhibits 8A, 8B,
8C, and 8D, tell me if you recognize those documents.
A. These are presentations made by OCME to the DNA
subcommittee at the four meetings that I mentioned.
Q. Is there one presentation for each of the four meetings?
A. Yes.
MR. McKAY: Government offers 8A through 8D.
MR. STRAZZA: No objection.
THE COURT: Government Exhibits 8A, 8B, 8C, and 8D are
admitted in evidence.
(Government's Exhibits 8A, 8B, 8C, and 8D received in
evidence)
BY MR. McKAY:
Q. If you turn to Government Exhibit 9, can you tell me if you
recognize that?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
162
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. Yes.
Q. What is that?
A. This is a letter from Jack Ballantyne, who was the chair of
the DNA subcommittee to Sean Byrne, who was the acting
commissioner and chair of the Commission on Forensic Science,
stating that they had approved FST for use with forensic
casework by OCME.
MR. McKAY: Government offers Exhibit 9.
MR. STRAZZA: No objection.
THE COURT: Government Exhibit 9 is admitted in
evidence.
(Government's Exhibit 9 received in evidence)
BY MR. McKAY:
Q. Do you see that letter references an October 8, 2010,
meeting at which FST was approved?
A. Yes.
Q. Were you present for that meeting?
A. I was.
Q. Would you turn to Government Exhibit 10. Can you tell me
if you recognize that document.
A. These are the minutes from that October 10 meeting of the
DNA subcommittee.
Q. Do you see down at the bottom there does it list you as an
attendee?
A. It does.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
163
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
MR. McKAY: Government offers Exhibit 10.
MR. STRAZZA: No objection.
THE COURT: Government Exhibit 10 is admitted in
evidence.
(Government's Exhibit 10 received in evidence)
BY MR. McKAY:
Q. If you turn to the third page, you see the bolded
paragraph?
A. Yes.
Q. Without reading the whole thing, generally speaking, what
does that paragraph report?
A. This notes that the presentation of the validation was
given by Theresa Caragine to the DNA subcommittee, and the DNA
subcommittee had reviewed and evaluated the FST and approved it
for forensic casework.
Q. Who seconded the motion to approve FST?
A. Dr. Chakraborty.
Q. After the subcommittee approved FST, was there any other
approvals required?
A. The scientific forensic commission approved FST.
Q. What is the relationship between the scientific forensic
commission and the subcommittee?
A. The subcommittee deals with the details of DNA analysis.
The commission itself does not include scientists, but rather
attorneys and judges and others -- other legal professionals,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
164
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
although there is one scientist on there, Marina Stajic, who is
an OCME scientist, or was at the time. So the DNA subcommittee
deals with the details of the DNA methods and makes a binding
recommendation to the commission, although the commission has
the opportunity to ask questions or even to require additional
work if they so chose.
Q. Did members of the full commission in fact ask questions?
A. They did.
Q. Did they ultimately accept the binding recommendation of
the subcommittee?
A. They did.
Q. Were you present at the meeting at which it was accepted by
the full commission?
A. I was.
Q. If you look at Government Exhibits 11, will you tell me if
you recognize that.
A. These are the minutes from the New York State Commission on
Forensic Science meeting on December 7, 2010, at which FST was
approved.
MR. McKAY: Government offers Exhibit 11.
MR. STRAZZA: No objection.
THE COURT: Government Exhibit 11 is admitted in
evidence.
(Government's Exhibit 11 received in evidence)
BY MR. McKAY:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
165
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. If you turn to Exhibit 12, can you tell me if you recognize
that.
A. Yes.
Q. What is that?
A. This is a letter from Sean Byrne, the chair of the forensic
science commission, to Dr. Mechthild Prinz, who was at the time
the director of the Department of Forensic Biology at OCME.
MR. McKAY: Government offers Exhibit 12.
MR. STRAZZA: No objection.
THE COURT: All right. Exhibit 12's admitted in
evidence.
(Government's Exhibit 12 received in evidence)
BY MR. McKAY:
Q. What does that letter say?
A. The letter states that FST has been approved by the
Commission on Forensic Science for use with forensic casework
by OCME.
Q. Now, did there come a time before FST was approved that
Dr. Chakraborty raised a question he had about the validation
study?
A. Yes.
Q. What was the nature of his question or concern, as best you
recall?
A. He wanted to know if having drop-out at one locus was
associated with drop-out at another locus. So if we knew there
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
166
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
was drop-out at locus one, did that give us any information
about the probability of drop-out at locus two, for example.
Q. How did Dr. Chakraborty raise that concerns to you?
A. He raised it during a DNA subcommittee meeting.
Q. What did you do in response to his concern?
A. I did a conditional analysis, which is a standard
statistical procedure where I included information about
whether or not there was drop-out at one locus and then looked
to see if that affected the probability of drop-out at any
other locus.
Q. Did you ultimately present that work to the subcommittee?
A. I did. And I found the drop-out probabilities to be
independent from locus to locus.
MR. McKAY: Mr. DeLuca, will you pull up Government
Exhibit 8C and turn to slide 25.
Q. What is this slide?
A. This is a summary of the testing of the independence of
drop-out rates that I presented at a DNA subcommittee meeting.
Q. If you could look at the next slide and tell us what that
is -- actually, go back one.
A. Here, I describe the question that he had asked and the
approach that I took.
Q. Now, after you presented this additional research on
Dr. Chakraborty's question, what happened next with respect to
this issue?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
167
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. He said that he had some other ideas about additional
analyses that he would like us to do.
Q. Did you subsequently speak with him about those ideas?
A. Yes. He said: Contact me after the meeting, and we can
talk about what to do. So I repeatedly attempted to contact
him. He did reply to one email saying he was about to go out
of town and for me to contact him again the next week, which I
did. He did not reply, and I repeatedly tried to call him and
was unable to get ahold of him.
Q. When you ultimately presented the validation study to the
DNA subcommittee for approval, did you describe the steps you'd
taken with respect to this issue that Dr. Chakraborty had
raised?
A. I don't remember if we -- if I specifically described that,
but he did not bring up this issue at the meeting again.
MR. McKAY: Mr. DeLuca, if you'll pull up Government
Exhibits --
THE COURT: Just to be clear, though, did he ever with
any kind of specificity indicate what he was thinking?
THE WITNESS: No.
THE COURT: OK.
THE WITNESS: No.
THE COURT: So during the meeting, he just said: Oh,
you know, I have some thoughts on this. Just give me a ring
after the meeting?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
168
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
THE WITNESS: Yes.
THE COURT: All right. Go ahead.
MR. McKAY: Thank you.
Mr. DeLuca, would you pull up Government Exhibit 7.
Q. Just looking at the first page, what was this exhibit
again?
A. These are the executive summaries of the volumes included
in the FST validation.
MR. McKAY: Mr. DeLuca, would you turn to page 713 of
the PDF.
Q. Dr. Mitchell, what are we looking at here?
A. This is a summary of the analysis that I did to determine
whether drop-out was independent from locus to locus.
Q. And was this summary available to the DNA subcommittee
before they voted to approve FST?
A. Yes.
Q. Was it available to Dr. Chakraborty before he seconded the
motion to approve FST?
A. Yes.
MR. McKAY: You can take that down, Mr. DeLuca.
Q. Now, you testified earlier that the calculations underlying
FST take a long time to do by hand; correct?
A. Yes.
Q. So FST has a program, a software program, to perform those
calculations?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
169
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. Yes.
Q. Are you a computer programmer?
A. I do some computer programming in my work, but I am not a
formally trained programmer.
Q. Who did the computer programming of FST?
A. Programmers that were contracted by OCME.
Q. What was your role? How did you interact with those
programmers?
A. I described what needed to be done, and as they developed
functions within the program, I would test those functions
individually to determine whether they were working correctly.
Q. How would you test those functions to determine if they
were working correctly?
A. For a small number of alleles in a sample, I could perform
manual calculations, which I did; or if it was not possible to
evaluate the entire function, they would create intermediate
outputs for me to check to confirm that they were working
correctly.
Q. When you performed those checks, did you find that you were
getting the expected outputs?
A. Yes.
Q. From your perspective, what was the goal of the computer
program or the source code underlying the computer program?
A. To compute a likelihood ratio that was not possible to
compute by hand in real time.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
170
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. Did you want it to do the math correctly?
A. Yes.
Q. Did you care about the style of the coding?
A. No.
Q. Did the program --
THE COURT: Well, what do you mean by "style"?
THE WITNESS: I did not look at the source code and
weigh in on the way that they were naming variables or
commenting in the code or, you know, sometimes there are a
choice of several functions that one could use to do the same
procedure. So I didn't -- I didn't weigh in on what the
specific code said.
THE COURT: OK. Go ahead.
Q. Are there different code languages that one could use for
computer programming?
A. Yes.
Q. But as between any particular code language, did you have a
preference?
A. I did not.
Q. You wanted it to do the math right?
A. Yes.
Q. And did it do the math right?
A. Yes.
Q. How do you know that?
A. Because I could manually confirm that.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
171
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. When you did the validation study, when you generated these
results that you've been walking us through, were you using
that computer program?
A. Yes.
Q. Did it generate results broadly consistent with what you
would have expected to see?
A. Yes.
Q. After the time that FST was approved for use in casework,
did there come a time that OCME detected an issue with the
source code?
A. Yes.
Q. What happened?
A. FST was put online in April 2011, and one of the first few
samples that was run generated a negative likelihood ratio,
which does not make sense mathematically.
Q. When you say "negative likelihood ratio," you mean actually
a negative value?
A. Yes. Not the negative exponent, but a negative value.
Q. Because we've been looking today at likelihood ratios
greater than 1 or less than 1?
A. Right. On the log scale the exponents are greater than
zero or less than zero, but the likelihood ratio itself is
always greater than zero because it's a ratio of two
probabilities, both of which have to be between zero and one.
Q. Were you able to figure out how it came to be that you
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
172
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
generated a negative likelihood ratio?
A. Yes.
Q. How did that happen?
A. There are two conservative measures that OCME takes that
are standard measures that labs, many labs -- many if not all
labs use with random match probability. And those are if an
allele is extremely rare, say, one in a thousand or if it's not
observed in the frequency database that's used by the lab,
there is a standard protocol that is to assume that it was seen
five times in the sample. So to artificially inflate its
frequency to about -- in our sample would have been 1 or
2 percent. And the effect of that is a very rare allele that
is carried by the suspect and shows up in the mixture can be
quite influential in the LR because it will be unlikely to
randomly sample that from the population. So increasing the
frequency of alleles used is a way of making sure that does not
receive too much weight. It's a way of down-weighting super
rare alleles.
The second adjustment we do is if a population is
actually not one freely mixing population but instead there are
pockets of the population that kind of stick together, the
frequency with which homozygotes are found can be a little
higher than expected. So we and other forensic labs make a
slight adjustment to the expected frequency of homozygous
genotypes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
173
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
So those two things together, if there was a locus
that had alleles that added up almost to a 100 percent, when
those conservative adjustments are made, it could bump the
frequency above 100 percent, which would be nonsensical in the
calculation. So what we did was if there is a locus at which
the alleles add up to 97 percent or higher, that locus is not
used in the calculation. So what had happened in that one
sample was there were several common alleles at a locus and a
couple of rare alleles that bumped the entire frequency up to
1, and then when we did the second -- the homozygous frequency
adjustment, it gave the total genotype frequencies go higher
than 1. So we needed to avoid getting into that situation in
the future.
Q. Let me see if I can -- what did you do to avoid getting
into that situation?
A. We then made it impossible for the total genotype frequency
to go higher than 1.
Q. How did you do that?
A. By not using a locus at which the alleles added up to
97 percent or higher.
Q. So in a hypothetical situation where the alleles present at
the locus added up to 98 percent of the population, if you
applied the 3 percent theta correction factor, you'd be at
101 percent; is that right?
A. It doesn't exactly translate that way, but it's very close.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
174
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
So the adjustment that I'm talking about with the homozygous
frequency is called a theta adjustment, and it's fairly
universally applied to guard against this structured population
problem. So at a locus in the sample, if the alleles that are
labeled at a locus in the sample add up in frequency to .97 or
higher, that locus is treated as a 1 or treated as an
uninformative locus in the calculation.
Q. So when that allele frequency cap comes into play, what
kind of -- well, in situations in which it comes into play,
what can you say about the alleles that are in that mixture?
A. That they include close to -- they include, generally, all
of the common alleles at that locus. There are some loci that
have two or three very common alleles that together might add
up to close to 95 percent. So those loci often don't provide
much information anyway, because almost everyone could be a
contributor to the mixture at that locus.
THE COURT: I guess just a question I have is,
obviously, there were a lot of things in going through the
validation process that were utilized. Why wasn't this, the
procedure, the theta, whatever it is, something that was
basically looked at beforehand? In other words, there are a
lot of things that were done. It sounds like the use of the
theta was something that was done, it is commonly done, but it
wasn't factored in when going through the validation process.
THE WITNESS: We didn't see any likelihood ratios that
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
175
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
were negative. That was the trigger that we'd identified a
problem during the validation.
MR. McKAY: If I may follow up on that question, your
Honor?
THE COURT: Yes.
MR. McKAY: Mr. DeLuca, could you pull up Government
Exhibit 8A and turn to slide 46.
Q. So government 8A is one of the PowerPoint presentations
before the DNA subcommittee.
A. Yes.
Q. What are you describing in this PowerPoint presentation?
A. Here's a note saying in the simple examples that I showed
to the subcommittee about how the calculations were done, we
did not include this theta correction, this adjustment for the
homozygous genotype frequency. However, as we say here, we do
include this adjustment in FST.
Q. So it was explained to the DNA subcommittee that you'd be
using the 3 percent theta correction adjustment; is that right?
A. Yes.
Q. And was that also in your peer-reviewed article about FST?
A. Yes.
Q. And was it also disclosed to the subcommittee that you'd be
using this 2 percent, I believe you said, minimum allele
frequency?
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
176
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. So the two conservative adjustments that you were going to
make were disclosed to the subcommittee; is that right?
A. Yes.
Q. But this allele cap, which had to be put in place to stop
the byproduct of these adjustments, that was not contemplated
at the time; is that fair to say?
A. That's fair.
Q. So you said in one of your earlier answers, I think, that
the loci at which this function would come into play are ones
in which there are very common alleles; is that right?
A. Yes, that's generally true.
Q. And you said that that did not provide that much
information. What do you mean by that?
A. Often those loci are not very informative because, for
example, at one locus the alleles 8, 9, and 10 together might
make up 95 percent of the allele frequencies in a population.
So often then when one is comparing, say, a suspect profile to
the mixture, the suspect has some combination of 8, 9, and 10.
So it's very common at those loci to not really learn much from
that locus because many people that are compared to that
mixture have alleles that show up in the mixture.
Q. So the discriminatory power of that particular locus is not
as high as one in which there were more rare alleles; is that
right?
A. That's true.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
177
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. So would you expect the effect of that function of capping
the allele frequency -- what would you expect that to have in
any given case in terms of the likelihood ratio?
A. So we're treating that locus as a likelihood ratio of 1.
We're treating it as an uninformative locus in the calculation.
So, in general, I would expect that doing that would pull the
likelihood ratio for the entire profile closer to 1, so making
it less strong in either direction.
Q. Would you necessarily see that in every case, or that's
what you would suspect?
A. No, not necessarily. That would be the average result.
Q. So after that change was made to the source code, what, if
anything, did FST do to test to see what the results were?
A. So we did a performance check. We have a set of
comparisons that were run every time anything changed, and
usually the change would be something with the interface or
adding a function for research purposes or moving it to a
different server. So we would run these -- there are 12
mixtures. We would run and confirm that the output was the
same before and after the change.
Q. When you say you'd run these 12 mixtures, what would you
compare these mixtures to?
A. We would compare the true contributors to the mixture, at
least one of them. I don't know if we did all the contributors
per mixture. And then we would compare our database of
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
178
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
noncontributors. So we would do about 1,250 noncontributor
comparisons and at least one, possibly more, true contributor
comparisons to -- with these performance check samples.
Q. So if you have 12 samples, did you do 1,250 estimated
for -- 1,250 comparisons for each of those 12 samples?
A. Yes.
Q. Turn your binder to Government Exhibit 16 which doesn't
have its own tab. I think it's just behind 15.
A. Yes.
Q. Do you recognize that document?
A. I do.
Q. What is it?
A. This is a document that was generated summarizing the
results when we did the performance check after we implemented
the .97 cap.
MR. McKAY: Government offers Exhibit 16.
MR. STRAZZA: No objection.
THE COURT: All right. Government Exhibit 16 is
admitted in evidence.
(Government's Exhibit 16 received in evidence)
BY MR. McKAY:
Q. What were the results of that performance check?
A. So 10 of the 12 performance check samples were unaffected,
and two of them then triggered the .97 cap. So for those two,
we summarized the likelihood ratios for the true contributors.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
179
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
One that had gone from .34 to .42 before and after the cap; the
other that went from 3.82 times 10 to the fourth -- excuse me,
it started out at 3.38 times 10 to the fourth before the cap
and it went to 3.82 times 10 to the fourth after the cap.
Q. So starting with the first result that changed, .34 to .42,
how would those two numbers be expressed in terms of the
likelihood ratio?
A. Those are both less than 1, and they are not qualitatively
different. They're not an order of magnitude different.
Q. What about the second, the 3.38 to the fourth and 3.82 to
the fourth?
A. The same thing, these are both times 10 to the fourth. So
they're in the same order of magnitude and a slight change in
the actual value, but not a substantial change.
Q. So in both instances, the specific number changed; is that
right?
A. Yes.
Q. But the bottom line conclusion in terms of the expression
of the strength of the evidence, did that change?
A. No.
Q. Now, when you made these 1,250 or so comparisons for each
of the 12 samples, were each -- noncontributor comparisons for
each of the 12 samples, were each of those noncontributor
samples manually evaluated to see if they were suitable for
FST?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
180
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. Do you mean following the casework protocol, whether a
person is excluded --
Q. Yes.
A. -- when they would not?
Q. That's what I'm asking.
A. So in the validation, we did something that was a little
bit different than casework protocol in this situation because
in casework, if someone is manually excluded, FST is not
computed. But we did compute FST on all of these
noncontributors because we wanted to see how it performed even
though, in a casework situation, many of those would not have
an LR computed. They would simply be excluded.
Q. So that's in the performance check. Is that also true of
the validation that you had done that you didn't do the manual
exclusions in the validation study?
A. Yes.
Q. Now, in reviewing the results of the performance check or
having reviewed them -- have you reviewed the results of the
performance check?
A. Just right here in front of me?
Q. Well, besides standing on the witness stand today, have you
reviewed the results of the performance check?
A. Yes.
Q. Are you aware of an instance in which a noncontributor
sample went from having an LR of less than 1 before the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
181
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
modification to the source code to having an LR of greater than
1 after the modification of source code?
A. Yes.
Q. Do you recall what the specific results were before and
after?
A. I don't know exactly what it was before, but it was
something like .5, to my estimate. And then after the cap, it
went up to about 29.
Q. Have you reviewed that specific profile?
A. I have.
Q. Based on your review of that profile, what can you say
about whether it would have been a manual -- what the manual
call would have been had that been found in casework?
A. In casework, the manual call for that sample would have
been no conclusions can be drawn.
Q. So in casework would that sample have generated a
likelihood ratio?
A. It would not.
Q. Are you aware of any other programs or methods that perform
a similar function to what we've just been talking about?
A. Yes.
Q. What example are you thinking of?
A. It's fairly common when computing a CPI, or a combined
probability of inclusion, to first examine the evidence sample
and make decisions about which loci will be included in the CPI
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
182
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
and which will not. This is before looking at any suspect
profile.
Q. If you'd turn in your binder to Government Exhibit 14.
Mr. DeLuca, if you'd put that on the screen as well.
Might be easier.
Do you recognize that article?
A. I do.
Q. What is that?
A. This is an article describing this very process, to look at
a forensic mixture and to make some decisions about which loci
will be included in the statistical calculation and which ones
won't.
MR. McKAY: Government offers Exhibit 14.
MR. STRAZZA: No objection.
THE COURT: All right. Exhibit 14 is admitted into
evidence.
(Government's Exhibit 14 received in evidence)
BY MR. McKAY:
Q. Who are some of the authors of that article?
A. The first article is Fred Bieber. He is currently on the
DNA subcommittee of the forensic science commission. The
second is John Buckleton. He is one of the developers of the
STRmix, or STRmix program. Bruce Budowle is a former FBI
scientist, now an academic researcher, in Texas who used to be
a big proponent of the CPI but is now a likelihood ratio user.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
183
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
John Butler, who is at NIST, has done extensive research in
forensic DNA testing. And Mike Coble, who is also now at NIST
with John Butler.
Q. Is it fair to say this is a well-respected group of
forensic scientists?
A. Yes, it is.
Q. Is this a peer-reviewed publication?
A. Yes.
Q. If you look at the sixth page, I believe it is, and look at
Rule No. 1, in the bottom left corner, what are they describing
there?
A. They are describing a lab creating a locus qualifying rule,
they call it. So they say if a locus is going to be used -- a
locus is included for use in a CPI calculation if drop-out is
considered to be highly unlikely. So this is saying a lab can
use their internal protocols to identify loci in a mixture that
they do not want to include in a statistical calculation as
long as they have stated standards for making that decision and
as long as it is not made in comparison with a suspect profile.
So loci to be included in this calculation are identified. And
then if a suspect profile comes in and is compared to that
mixture, in this setting only the loci that were predetermined
to be part of that calculation will be used for comparison with
the suspect.
Q. Is that -- go ahead.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
184
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
A. In other words, if the suspect has an allele that does not
show up at one of the loci that they had excluded by whatever
their criteria were, it's kind of too bad for that suspect.
That potential exclusion will not be considered. The
statistics will be calculated only on the predetermined loci.
Q. Is this the function that you were describing a minute ago?
A. Well, it's a little bit different because we are -- we
would still --
Q. I think my question was unclear.
A. OK.
Q. I don't mean is this the function in FST.
A. Oh, yes.
Q. When I asked you are there other programs that perform
similar functions, is this what you were describing?
A. Yes.
Q. Now, to be clear, the allele cap, allele frequency cap
function in FST, does that come into play before consideration
of a suspect pedigree or after?
A. That -- the determination not to include a locus in the
calculation is made without considering the suspect profile,
but if a suspect profile comes in and is being manually
compared to the mixture, that locus can still be used to
exclude the suspect.
Q. So at the visual examination step --
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
185
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
Q. -- you would consider that locus?
A. Yes.
Q. But once the FST program runs, it mechanically excludes
that locus?
A. Treats that locus as uninformative, yes.
Q. Does it do that without consideration of whether or not the
suspect's alleles are present?
A. Yes.
Q. Are you aware that there has been litigation over whether
or not OCME should make the source code public?
A. Yes.
Q. Is it now public?
A. Yes.
Q. What was your view about whether to make it publicly
available?
A. I wanted it to be open source from the beginning. I
advocated for that.
Q. What is your understanding of why OCME resisted efforts to
make it publicly available?
A. It was a decision in the legal department, and my
understanding is that OCME wanted to reserve the right to
market it at some point if they wanted to and also did not want
to get caught up in how other labs were applying the method.
Didn't want to be responsible for ways in which other labs
might apply the method.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
186
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON5 Mitchell - Direct
MR. McKAY: One moment, your Honor.
THE COURT: Yes.
Q. Dr. Mitchell, in your view, are programs that analyze DNA
mixtures and generate likelihood ratios generally accepted in
the scientific community?
A. Yes.
Q. Is the method used by FST generally accepted in the
scientific community?
A. Yes.
MR. McKAY: No further questions.
THE COURT: Cross-examination.
MR. STRAZZA: After discussing it with the government,
may we have five minutes?
THE COURT: Oh, yes. Sure.
MR. STRAZZA: Thanks.
THE COURT: Doctor, you can step down. Since you've
been turned over and you're now own cross-examination, just
don't have any substantive conversations with the government
folks, OK?
THE WITNESS: OK.
THE COURT: Thank you.
(Recess)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
187
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
THE COURT: OK. Cross-examination.
CROSS EXAMINATION
BY MR. STRAZZA:
Q. Good afternoon, Dr. Mitchell.
A. Good afternoon.
Q. Before I start, I heard you say something towards the end
of your direct examination about -- I don't know if I got this
right -- by kind of too bad for that suspect. Did you use that
term?
A. I did.
Q. What did you mean by that?
A. In the CPI calculation when certain loci are not going to
be used in the calculation -- OCME doesn't do this. This is a
common practice in other labs -- and then a suspect profile
comes in, a suspect's alleles may not show up at the loci that
have been determined not to be part of the calculation, but
that will be ignored, and instead the calculation will use only
the predetermined loci in the probability of inclusion. And
that would be unfortunate for that defendant because their
alleles were missing at a different locus.
Q. Are you saying that those esteemed authors of that article
think that "kind of too bad for a subject" should be generally
accepted within the scientific community?
A. That was poor phrasing on my part.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
188
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. Are you saying that a situation that could actually work to
the detriment of a suspect should be generally accepted within
the scientific community if it's not accurate?
A. No, I'm not saying that.
Q. Interpretation of DNA mixtures derived from crime scene
evidence is a major issue in forensic DNA analysis, right?
A. Excuse me, can you say it again.
Q. Sure. Interpretation of DNA mixtures from crime scene
evidence is a major challenge or issue within the forensic DNA
analysis community, right?
A. I would say it's an important issue, yes.
Q. It's what we've been talking about all day, right?
A. Yes.
Q. And mixtures arise when two or more individuals contribute
to an evidence sample, right?
A. Yes.
Q. And the contributors to an evidence sample can include
victims, suspects, etc., right?
A. Yes.
Q. And for some samples you can separate the contributors,
right?
A. Yes.
Q. And for others you cannot.
A. Right.
Q. In the situations where you can separate, the random match
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
189
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
possibility is conducted, right?
A. Yes.
Q. By OCME. And in the other situations where you cannot
separate the contributors to the sample, that's where a
likelihood ratio can come into play to help provide a
statistic, right?
A. Yes.
Q. And in those situations it evaluates the strength of one
scenario versus another scenario, right?
A. It evaluates support for one scenario versus another
scenario.
Q. OK. And that's exactly why you designed FST, right?
A. Right.
Q. In this case we are talking about one of the scenarios
being that the case sample that was analyzed was made up of
Dean Jones and two unknown contributors, right?
A. I am not familiar with the specifics of the case.
Q. You haven't looked at the case file in this case?
A. No.
Q. Ever.
A. No.
THE COURT: What do you mean by case file?
MR. STRAZZA: The materials that were produced by OCME
with respect to Dean Jones.
THE COURT: OK.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
190
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. Is it your testimony that in preparation for this hearing
you never looked at the case file?
A. Yes.
Q. The FST model was designed for contributors that were
unrelated to one another, right?
A. Yes.
Q. So the likelihood ratio produced by FST would not be
suitable if the contributors were father and son, for example.
A. It does not address that scenario, no.
Q. Or brother and brother, right?
A. That's correct, it does not address that scenario.
Q. And why is that?
A. Because when the genotype frequencies are estimated, it is
using population frequencies not relative frequencies.
Q. And because relatives will have a higher frequency of the
same type of alleles, right?
A. Relatives tend to share more alleles than two randomly
selected persons from a population.
Q. And I believe you said that what you could do in that
situation is if a defendant wants to present somebody else's --
you know, a sibling, or a son or a father, they could just
present that to OCME, and OCME would test that for them, right?
A. I said I'm not actually familiar with the procedure, what
would actually happen, because I don't --
Q. So the answer is you don't know.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
191
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. I don't know what OCME's procedure is for dealing with a
situation like that.
Q. OK, I misunderstood you. I thought you said that could be
done.
A. It could be done programatically. FST could do that is
what I meant. I don't know what the protocol would be if that
came up in a case.
Q. Have you ever heard of it being done?
A. Heard of what being done?
Q. What you just described, where a suspect or through their
attorney provided somebody else's, a relative's, sample to be
tested with FST, to be compared to the case sample.
A. I don't know of a specific case, no.
Q. As you sit there, do you understand the problems with that?
A. Can you be more specific?
Q. Sure. Do you understand why a suspect might not want to
implicate one of their relatives?
A. Oh, certainly, yes.
THE COURT: Well, I mean, look, as a basic legal
matter the suspect doesn't have to put on any proof of
anything, so the issue may not necessarily be one of -- right
here we're deciding a Daubert issue which is a threshold issue
about whether or not this evidence could possibly be
considered. I think it's a separate issue. And again I'm just
thinking off the top of my head, because I didn't realize that
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
192
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
there was a potential argument related to this. The separate
issue then I think may relate to how relevant the actual
information would be at the end of the day for the jury. And
I'm not sure whether that's the balancing of the weight that
gets put to it or not, but I haven't sort of thought it
through.
MR. STRAZZA: I guess my position is, Judge, if we're
talking about whether something is generally accepted for an
analysis of crime scene evidence, if this issue is prevalent in
that type of situation, wouldn't that go towards whether or not
something should be will generally accepted?
THE COURT: Let me ask this, doctor. The other tools
that are used, do they somehow make or compensate or take into
account relatedness?
THE WITNESS: The CPI does not. The random match
probability can.
THE COURT: Random match probability. I understand
that, but actually what I am talking about are other programs
like FST. In other words, do you know whether they take into
account the relatedness?
THE WITNESS: I don't know whether or not they do.
THE COURT: OK, go ahead.
MR. STRAZZA: And I am sorry, I just want to make sure
I heard your Honor correctly. Was the question whether or not
some of those other programs that came up during the testimony,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
193
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
whether they account for that? Was that your question?
THE COURT: Correct.
Q. And the answer is I don't know?
A. Years ago they did not. I don't know if they do now.
Q. Some of the programs that the judge is referring to didn't
exist years ago, right?
A. Right, but some did.
THE COURT: Well, when you say years ago, what
timeframe?
THE WITNESS: When I was developing FST, the LR mix
program, for example, did not account for relatives. There may
have been further work done on that program, it may do that
now, but it did not when we developed FST.
Q. How about Lab Retriever?
A. No.
Q. How about TrueAllele?
A. I believe TrueAllele can account for relatives, although it
is in a different category than FST because it does
probablistic genotyping and attempting to separate the
contributors in the mixture probabilistically, which is beyond
what FST does.
Q. And that was around when you were developing FST, right?
A. That was. I don't know if the relatedness component was
available. And at the time we were developing FST, TrueAllele
was not appropriate for low template samples.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
194
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. Well, but wasn't relatedness one of the limitations that
was discussed and actually written about when you were
developing FST?
A. Yes.
Q. And so you mean to tell me you didn't discuss that with the
people who were using TrueAllele, and that didn't come up at
the workshops and stuff like that?
A. I believe it was mentioned at some point that TrueAllele
can do that, but I don't remember when that happened.
Q. In order for FST to be used on a sample, the sample must be
considered informative, right?
A. What do you mean by informative?
Q. Well, according to the protocols for FST there is a
requirement that the sample must be informative first, so that
was going to be my question for you.
A. I don't know what you mean by informative.
Q. OK. Did you help design the protocols for FST use?
A. Yes.
Q. Did you include the language that before a sample could be
used with the FST program that it must be informative?
A. I don't remember writing that.
Q. OK. Well, it has to be suitable for comparison, right?
A. Yes.
Q. And prior to the use of FST, OCME classified mixture
samples as included or major, right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
195
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. Um-hum.
Q. Cannot be excluded, right?
A. Yes.
Q. And that was defined as most of the known contributor's
alleles are labeled, right?
A. Yes.
Q. And then no conclusions can be drawn, right? That was
another category?
A. Right.
Q. And that was defined as several of known contributor's
alleles are not labeled but cannot be ruled out, right?
A. I guess I would make a correction. I wouldn't call it the
known contributor's alleles. I would call it a comparison
person's alleles.
Q. Thank you.
A. OK.
Q. So let's apply that to the definition and the one before
that as well, OK?
A. Yes.
Q. What is the difference between most of known contributor's
alleles labeled -- or comparison contributor's labeled -- and
several of comparison contributor alleles not labeled? Where
is the distinction there?
A. Some loci amplify better than others, so if a comparison
profiles alleles are missing at one of the robust loci, that
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
196
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
may not rise to the level of -- that may rise to the level of
"no conclusions can be drawn," whereas if two alleles are
missing at a less robust locus, that might be "cannot be
excluded". So, this part -- I am not an expert on that part; I
was not a case work analyst. So, the case work analysts -- who
have a lot of experience working with loci that may or may not
be more robust than another -- they are in a better position to
make these calls.
I was only making -- I was only using the calls that
case work analysts made in the validation. I did not make
those calls. The case work analysts made the calls, and then I
ran the FST and divided them up by the categories that were
given by the analysts. I was not making those calls.
Q. And in your expert opinion, are those calls subjective?
A. They are, yes.
Q. And are you aware of whether or not there were protocols in
place at the OCME for making those calls?
A. I don't know the specifics.
Q. I mean in fact what we just discussed, right, the
definitions I just read to you, are the protocols themselves,
right?
A. I don't know if those were in the case work protocols.
Because I was not a case work analyst.
Q. Would you agree that if the wrong call was made with
respect to that call, that specific issue, that a mixture might
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
197
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
be analyzed by -- or run through FST, that shouldn't otherwise
be? Do you understand the question? I can reword that if you
want.
A. Can you reword that.
Q. Sure. If that is not done correctly -- right? If the
analyst who is looking at more robust loci than others, if that
distinction is not made properly, that could affect whether or
not a sample ultimately gets run by FST, right?
A. Yes, yes.
Q. Do you know whether or not those guidelines or protocols
were created based upon validated studies?
A. I was not part of -- you mean to categorize cannot be
excluded, etc.?
Q. Yes.
A. I don't know.
Q. Since you weren't a part of them though, you are aware that
those guidelines or protocols were created prior to the
implementation of FST, right?
A. Yes.
Q. And so they were not created for FST, right?
A. That's true.
THE COURT: Mr. Strazza, let me ask this just in terms
of the nature of the motion. Is there -- because I know there
wasn't any briefing on this. But is there a Daubert challenge
to that determination? As I understand it, that determination
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
198
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
still gets made, but is there a challenge to that?
MR. STRAZZA: The issue is that when FST was validated
should it have included a new or a more appropriate protocol to
whether or not a mixture should be run in FST.
THE COURT: What you are raising is an issue relative
to FST, not an issue relative to what was done prior to that,
which I assume -- although I don't know -- Daubert challenges
had been made, and it was determined that most of the time that
evidence came in.
MR. STRAZZA: Except that FST relies upon that, and
what the witness just said is if that's done incorrectly, the
FST either is or isn't going to come into play. So the
ultimate issue that I will be arguing to the court is whether
or not that process or that issue should have been part of
FST's validation.
THE COURT: OK.
MR. MCKAY: I just note that that issue was never
raised in any prior briefing, your Honor.
THE COURT: Also, I mean, look, I don't know to what
extent I am --
The issue is --
I mean what I think the question is are those things
that experts should have looked at at the time, in other words,
those people doing the peer review, those people doing other
things. And I will have to give that some more thought.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
199
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Because, as I --
Actually, doctor, do you know, was that issue raised
in the back and forth? In other words, the fact that the FST
was taking the subjective results of the -- that were generated
and then using that with the tool?
THE WITNESS: There is no inappropriate time to use
FST. It's valid for all of those categories. It's a lab
decision which samples they're going to run based on if alleles
are missing and a person is excluded, then they have chosen not
to run it on those samples. FST was validated across the
board.
MR. STRAZZA: If I may just clarify, because this is
probably going to come up in my other questions.
THE COURT: Go ahead.
MR. STRAZZA: The thing is this, Judge. FST puts a
statistical weight that we're arguing should not be presented
to a jury. If there is a situation that would have -- meaning
if FST is relying on a protocol that maybe was validated for a
different purpose but not for the purpose of putting a
statistical value in front of a jury, then maybe it's something
that need to be reconsidered.
THE COURT: Well, as I understand what the doctor just
said, scientifically it was validated for each of the
categories.
What I think you're saying is that, well, did it take
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
200
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
into account the subjective nature of that determination in
some way.
MR. STRAZZA: Can I just give you an example? Let's
say when they were validated for different reasons, let's say
the margin of error rate was plus or minus 30 percent. Right?
And let's say that when FST was designed maybe it should
have -- and there was the 30 percent margin of error of whether
or not this evidence that's analyzed by FST is going to be
presented to the jury. Maybe the 30 percent margin of error
wouldn't have mattered presenting to a jury "could be excluded,
could not be excluded," whatever, but it sure matters when you
are putting a statistical value on that and coming up with a
number. That's my argument.
THE COURT: I understand the nature of the argument.
MR. MCKAY: Your Honor, could we have our legal
arguments after the witness is done with her testimony?
THE COURT: I'm sorry, I started this. Because I
wasn't sure again whether there was a challenge to the earlier
stuff. Because one way to -- well, never mind. Go ahead.
Continue your cross-examination.
MR. STRAZZA: I'm challenging everything, Judge.
THE COURT: No, it's just that the other stuff --
well, just go ahead.
MR. STRAZZA: I'm just finding my place.
THE COURT: Sure.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
201
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. In order for FST to generate an accurate likelihood ratio,
there must be a number of contributors that has to be
determined, correct?
A. Yes, this is true for all likelihood ratios.
Q. For the purposes of these questions, I'm just talking about
FST.
A. OK.
Q. So in determining the number of contributors, the analysts
at OCME look at the number of labeled alleles, correct?
A. Yes.
Q. And the analyst doesn't label the alleles him or herself,
right?
A. Right, the software does.
Q. And that software is call Gene Mapper, right?
A. That's what they were using.
Q. Or was.
A. Yes.
Q. For the purposes of these questions I'm referring to when
you created FST.
A. OK.
Q. And Gene Mapper creates an electropherogram, right? That's
one of the charts that is produced after it is run through Gene
Mapper, right?
A. There is an electropherogram and allele calls, yes.
Q. And the electropherogram produces peak heights across
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
202
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
different loci, right?
A. Yes.
Q. And not all peak heights are labeled as alleles on that
electropherogram, right?
A. Not all peaks are labeled, that's true.
Q. And so it's only those that are above a predetermined
threshold that was set by OCME, right?
A. Yes.
Q. And that's 75 RFU?
A. Yes.
Q. And who chose that threshold?
A. That was chosen based on the validation of the software and
the equipment. There are methods published for looking at the
noise in the signal and then setting the threshold some
distance above the noise to ensure that noise is not labeled,
that only true peaks by this definition are labeled.
Q. When you say software and equipment, you are not talking
about FST.
A. That's correct.
Q. That threshold was put in place before FST, right?
A. Yes, yes.
Q. And I am assuming that -- do you know whether or not that
threshold was set based upon empirical studies?
A. I would say that it is. I was not part of those studies.
Q. Do you know though as you sit here whether or not it was?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
203
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. I have not seen it, but it is supposed to be.
Q. And were those empirical studies supposed to be validated?
A. I don't -- this is not my area of expertise.
Q. I understand. But you worked out of OCME for a period of
time, and if you can answer the question, please do; if you
can't, you can't.
A. That's outside my area.
Q. That's not what I'm asking you. Do you know whether or not
the methodology for selecting 75 RFU as the threshold was
validated? Yes or no?
A. Well, I haven't -- I have not seen the validations, so I
can't say that I know what happened in the --
Anything that's happening in the lab had to be
validated, but I have not seen the validation of this, the
selection of this threshold.
Q. And again that was created before FST, right?
A. Right.
Q. So if it were validated, it was validated for other uses
besides use with FST, right?
A. Right, for any down-stream analyses, yes.
Q. And you are aware that other labs have different thresholds
than 75 RFU, right?
A. Yes.
Q. Some are much lower, right?
A. I've heard of 50. I don't have --
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
204
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. Have you heard of 35?
A. No.
Q. Are you aware of whether or not the software that OCME uses
now uses lower thresholds?
A. I don't know.
Q. You would agree that since the thresholds affect which
alleles are ultimately called, then the thresholds affect the
ultimate determination -- the determination of the ultimate
number of contributors that gets input into FST? You agree
with that statement?
A. Are you saying if they used a different threshold they
might come up with a different number of contributors?
Q. That's not what I was saying, but that's another -- that's
saying -- I would like to know the answer to that question.
A. It's possible. But 75 is what they use and what we used
for FST.
Q. But the threshold that is used affects -- or could
affect -- the number of contributors determined by the
analysts, correct?
A. It could.
Q. And when FST was in place, when you created it, the
protocols that the analysts were using to determine the number
of contributors that they would input into FST said that if
three alleles or more were labeled -- or if three alleles were
labeled at at least two loci that would be considered a two
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
205
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
person mixture, right?
A. Yes, yes.
Q. And if five alleles were labeled at two or more loci, that
would be a three person, right?
A. Yes.
Q. And if seven alleles at two or more loci, that would be a
four person?
A. I don't remember that specific one.
Q. And that's because it doesn't matter, because FST wasn't
validated to use more than three person samples, right?
A. Right, we only did three person -- two person and three
person.
Q. Was that protocol that I just described to you, was that
based upon empirical studies?
A. That that you just described is standard across the field.
Q. Do you know whether or not that was based upon any
empirical studies at OCME?
A. We did empirical studies on mixtures and looked at many
characteristics of the mixtures, and those were two of them,
yes.
Q. Before you started using that for FST, were those protocols
validated?
A. Again, I wasn't there; I don't know.
Q. You don't know? You don't know whether or not it was a
validated protocol in effect when you started creating FST?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
206
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. I was not there for that time period --
Q. I understand that.
A. -- so my answer is the same as with the analytical
threshold. I did not see that, so I can't testify to that.
Q. Dr. Mitchell, I know you don't want to testify to anything
that wasn't part of your involvement with FST, but my
questions -- I'm going to ask you respectfully to listen to the
question of whether or not you know something or not.
MR. MCKAY: Objection, your Honor.
MR. STRAZZA: Can you do that, please?
THE COURT: Well, here is -- I mean let me see if I
can get some clarity on this.
So, with regard to the validation, you hadn't actually
seen the validation papers relating to that?
THE WITNESS: Correct.
THE COURT: But based upon your experience at OCME,
and as in this field -- and by this field -- is it your
understanding that validation would have had to have -- it
would have had to have been validated?
THE WITNESS: Yes. Although I'm not sure everything
like this has to be validated. At least two loci with three or
more alleles, that's a standard definition of a two person
mixture across the field. And I don't know whether every lab
is required to empirically validate that.
Q. Would you use an unvalidated measurement to start the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
207
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
process of your FST program?
A. We did empirical studies on the number of contributors to
the mixtures. I don't know what they did before I was there.
Q. Are those studies included in your validation?
A. It's a published paper that I think that I saw in this
binder.
Q. What is the title of that?
A. I don't remember. Can I --
THE COURT: Go ahead.
THE WITNESS: Take a look?
THE COURT: You can take a look, sure.
MR. MCKAY: If it speeds things up, she may be
referring to Exhibit 4.
THE COURT: Take a look at 4 and see if that's what
you were talking about.
THE WITNESS: Yes.
Q. Is that commonly referred to as the Perez paper?
A. Yes.
Q. Those protocols were not part of the FST validation though,
right?
A. These are characteristics of mixtures that can be used to
help determine the number of contributors to the mixture.
Q. I understand that. The question is: Was determining the
number of contributors or what's contained in that paper, was
that part of the FST validation study?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
208
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. This was part of determining the number of contributors to
the mixtures.
Q. Maybe I'm missing something. I understand that, and I
understand the results of the studies were published in that
paper.
A. Yes.
Q. But is that information contained in the validation studies
for FST?
A. Oh, no, it's not described in that same way.
Q. Because that article isn't about FST, right?
A. Right.
Q. That article was about determining the number of
contributors in general, right?
A. Yes, yes.
Q. So my question is: When you created FST -- well,
withdrawn. FST relies upon a determination of the number of
contributors, correct?
A. Yes.
Q. And in fact FST will create a likelihood ratio that is not
accurate unless a proper determination is made with respect to
the number of contributors, correct?
A. It is still an accurate comparison. The probability of
getting those alleles in the mixture if scenario one is true to
the probability of getting the alleles in the mixture if
scenario two is true, that remains valid regardless of how many
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
209
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
people are actually in the mixture.
Q. I know, but that's not what I asked you, Dr. Mitchell.
What I'm saying is you need to accurately determine the number
of contributors to the mixture in order to run FST, right?
A. I am saying that the calculation is still a valid
comparison even if you are wrong about the number of
contributors. It doesn't throw everything out.
Q. Let me ask, are you saying the number of contributors input
in FST is irrelevant?
A. No.
Q. But you just said that the likelihood ratio produced is
still valid if the number of contributors is wrong. Did you
say that?
A. It is still a valid comparison of the two scenarios that
you use even if the scenarios are not correct.
Q. So then why does it matter?
A. Because we do our best to come up with the best estimate
for the number of contributors that we can.
Q. Because that will affect the accuracy of the likelihood
ratio, right?
A. It may affect the relevance in a particular theory of the
case, but a likelihood ratio is still a valid calculation
comparing those two scenarios.
MR. STRAZZA: Judge, I honestly don't know if I got my
answer.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
210
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
THE COURT: No, I think the issue may be that this
ratio is not -- it's a probability, right? And so I think --
so while -- you know, accuracy one way or the other -- I'm not
sure if that's the way to necessarily best think about it.
In other words, so there is a range of possibility.
So, for example, if this was something that you could
empirically prove, then using the different tools should get
you just about the same results. But what is happening here,
as I understand it, is it is -- so whether it's two
contributors or three contributors, that's a probability. Am I
correct about that?
THE WITNESS: We do our best to determine whether it's
two or three contributors.
Q. But it's my understanding most respectfully that a
likelihood ratio is not a probability.
A. It is a ratio of two probabilities.
Q. But here is the thing. What if you determined that there
is four contributors to the mixture? Are you going to come up
with a likelihood ratio?
A. We didn't validate the four person mixtures in FST.
Q. So the answer is no, right?
A. Right, not with FST right now, no.
Q. So then it would seem like properly determining the number
of contributors would be relevant, to say the least, in getting
an accurate likelihood ratio.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
211
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
MR. MCKAY: Your Honor, I think this has been asked
and answered several times.
THE COURT: No, I will allow it.
You can answer, doctor.
A. As I said, the comparison remains valid whatever scenarios
are put in there. It may not be relevant to the case, or OCME
had decided not to go forward with the four person mixtures.
So, obviously if something was a four person mixture, that
would make a difference if it's a three person or a four person
mixture because it would be run if it was a three person and
not run if it was a four-person.
Q. So being that it's relevant in that scenario, my question
is: Was the process for determining the number of contributors
part of the FTS validation? Yes or no.
A. No.
Q. So when the process of determining the number of
contributors was validated, it was validated for a different
purpose other than to input that figure into FST.
A. Its intended purpose was for a variety of --
Q. Other than to input into FST?
A. No, including FST.
Q. That's not true, doctor, based upon what you just said,
because you said it was created -- it was validated before you
created FST, right?
A. The guidelines that you mentioned, that three alleles at
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
212
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
two or more loci would be called a two person mixture, that is
not something I validated on its own. I looked at all kinds of
characteristics of two, three and four person mixtures.
Q. OK, I'm going to move on.
THE COURT: OK.
Q. Would you agree that determining the exact number of
contributors to a mixture is impossible?
I will word it differently. I see your face. I will
word it differently. My apologies.
Would you agree that it's impossible to know whether
or not the analysts determined the precise number of
contributors to a mixture?
A. In a case work sample, yes.
Q. And so the analyst's conclusion at that point is just an
estimate -- is just an estimation, right?
A. It's an estimate using all of these tools that we have.
Q. And using all of those tools that you have, it's often an
underestimate, right?
A. I don't know, because I don't know the true number of
contributors.
Q. I'm going to read you a quote and tell me if you agree with
this quote or not.
"Samples must be categorized as a single source for
mixtures, and for mixtures the number of contributors must be
estimated. Characteristics of a mixture can be used to
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
213
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
determine whether to treat the mixture as two, three or four
people, yet due to allele sharing amongst related and unrelated
individuals, there will always be a level of uncertainty to
this determination. Using only the maximum number of alleles
observed at any locus to estimate the number of contributors to
a mixture will often lead to an underestimate."
Do you agree with that?
A. OK, that is one -- yes, that is one particular tool for
estimating the number of contributors, yes.
Q. OK. Isn't that exactly how OCME estimates the number of
contributors?
A. They use several characteristics of the mixture to estimate
the number of contributors.
Q. In addition to doing that, what else do they do?
A. There are other characteristics; they are listed in this
paper.
Q. Well, you wrote the paper, right? You were one of the
authors on the paper?
A. Yes, I'm one of the authors.
Q. So, I'm just asking you what are some of the other things
that OCME does.
A. So, they can look at the total number of alleles labeled in
a mixture. They can look at the number of loci where there are
four or more labeled alleles. For example, sometimes a three
person mixture doesn't have any loci with five alleles but it
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
214
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
has many loci with four alleles. So, there are other
guidelines in here that are not just about the maximum number
of alleles that appear at a locus.
Q. And are those guidelines required for determining a mixture
that gets used -- the number of contributors to a mixture that
gets used by FST?
A. Yes.
Q. Is that required?
A. That's what's used, yes.
Q. Those are the guidelines that the analysts must follow at
OCME before determining the number of contributors for FST?
MR. MCKAY: Your Honor, if I could object.
Mr. Strazza specifically objected to Dr. Mitchell's expertise
to qualify about case work at FST. We have brought in another
witness who has extensive case work experience. Some of these
questions may be better posed to that witness.
THE COURT: I guess, doctor, if you know.
In other words, are you asking what the protocols are
that they would use?
MR. STRAZZA: I mean Dr. Mitchell wrote a paper on
what should be done. Dr. Mitchell created FST. So the
question is -- and we will get to this -- but she also trained
the analysts to use FST. So, the question is: Are they
required to use that protocol that she just described to
determine the number of contributors that they use with FST?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
215
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. Yes, there are guidelines in the OCME case work protocols
describing the findings from this paper to help the analyst
estimate the number of contributors.
Q. Was that information contained in the validation studies
for FST?
A. It's a separate validation where we looked at the number of
contributors. It is not part of the FST validation, this
examining the characteristics of two, three and four person
mixtures.
Q. So the answer is no.
A. I guess, the way that you're asking the question. It's not
part of the FST validation; it's a separate validation.
Q. When you created FST, you were aware of the issue of
underestimating the number of contributors, right?
A. That can happen, yes.
Q. And you're OK with it because in your opinion that's
conservative, right?
A. That's anecdotally what we saw, yes.
Q. And in your opinion that's conservative, right?
A. What we saw is knowing the true number of contributors to
the mixture, if we ran FST using that number of contributors,
generally for a true contributor we got the highest LR. If we
ran it with fewer contributors or more contributors, it didn't
fit as well and it didn't generate as high an LR for a true
contributor as it did when we got the number right.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
216
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. And are those studies contained in the validation
materials?
A. No.
Q. Where are the results of those studies?
A. Those were anecdotal analyses that we did that are not
contained in the validation.
Q. Are we able to view those anecdotal analyses?
A. No, but you could run them now because FST is open source.
Q. Let's talk about that very briefly.
A. OK.
Q. FST -- how long has FST been open source?
A. About two weeks I think.
Q. And that all came about after OCME tried to prevent FST
from trying to become open source, right?
A. The legal department did not want it to be open source,
that's correct.
Q. And when you say open source, did OCME put it online for it
to be open source?
A. I don't know.
MR. MCKAY: Objection, your Honor.
THE COURT: I will allow it. It doesn't matter.
Q. Do you know whether or not OCME made it public?
A. I don't know the steps of what happened.
Q. Underestimating the number of contributors as applied to
FST is not always conservative to the suspect, is it?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
217
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. I have not done rigorous studies to show that, no.
Q. And without doing a study, if a four person mixture is
determined to be a three person mixture, and the likelihood
ratio -- withdrawn.
Without doing studies, if a four person mixture is
determined to be a three person mixture, it would get run in
FST when it otherwise shouldn't be, right?
A. If it were determined to be a four person mixture it would
not be run in FST; if it were determined to be a three person
mixture, it would.
Q. And if that likelihood ratio was used against a defendant
at a criminal trial, that wouldn't be conservative, right?
A. Again, the models are still valid. The three person
comparison in the numerator and the three person comparison in
the denominator are still valid comparisons.
If it truly were a four person mixture but it met our
criteria for a three person mixture, we have no way of knowing
that it was truly a four person mixture.
Q. OK. But that could create a situation that's not
conservative for the suspect, right?
A. It could create a situation where there is a likelihood
ratio presented where in other circumstances it would not be.
Q. And that would not always be conservative to a suspect,
right?
A. What do you mean by conservative?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
218
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. I will move on.
A. OK.
Q. The number of contributors to a mixture becomes more
uncertain as the number of contributors increases. Is that
correct?
A. Yes.
Q. And that's due to smaller amounts of each contributor's DNA
being part of the mixture, right?
A. I would say it's due to a higher chance of sharing alleles
when you have more people in the mixture.
Q. And allele sharing is when the different contributors to
the same mixture have the same alleles, so when the results are
analyzed certain contributors' alleles will be masked or
blocked out or won't come up separately, right?
A. Yeah, there would be some overlap.
Q. So they wouldn't show -- right. So you wouldn't --
A. There might be one peak that includes DNA from two
different people.
Q. So that would include a smaller amount of the number of
labeled alleles, right?
A. Smaller than what?
Q. Smaller than if all the true alleles were called.
A. Right, if all the contributors were heterozygous and they
all had different alleles, yes, that would be the maximum
possible number of alleles, yeah.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
219
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. And obviously that could affect the determination of the
number of contributors then, because there will be less alleles
at a given locus, right?
A. Yes, but we looked at those patterns, so we know that that
happens.
Q. That wasn't my question.
THE COURT: Counsel, let the doctor finish answering,
and then you can ask your question.
MR. STRAZZA: OK. We might have a time problem,
Judge, I'm just saying.
THE COURT: Well, whatever it will be, it will be.
Q. Are you familiar with John Butler?
A. Yes.
Q. Who is John Butler?
A. He is a scientist at NIST.
Q. Well respected in the community?
A. Yes.
Q. Are you familiar with his cautionary warning regarding the
uncertainty of determining the number of contributors?
A. Can you be more specific?
Q. Yes. Are you familiar with John Butler's statement that
less than 50 percent of all four person mixtures would leave
seven alleles at two loci?
A. I don't remember the specific numbers.
Q. Are you familiar with his position generally about the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
220
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
uncertainty in determining the number of contributors?
A. I'm not well versed in it. I know that it's been
mentioned.
Q. You know that he's written about it, right?
A. Yes.
Q. Would you agree with the fact that certain analysts may
disagree when determining the number of contributors to a
mixture?
A. Yes.
Q. And are you ever aware of that happening before?
A. Yes.
Q. Does it happen quite often?
A. I don't know.
Q. Are you aware -- are you aware of it happening often?
A. I am aware of at least two -- I'm aware of two
circumstances. I don't know what often means.
Q. Are you aware that there is a protocol in effect for the
situation when an analyst disagrees with another analyst in
regards to determining the number of contributors to a mixture?
A. I am aware that a protocol exists, yes.
Q. And so if a protocol exists, that's because it comes up,
right?
A. I'm sorry? I said --
Q. Why are protocols created?
A. I said I know of two specific instances where two people in
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
221
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
the lab disagreed about the number of contributors.
Q. Do you know why they disagreed about the number of
contributors?
A. I don't know the details.
Q. Could drop-out affect -- could drop-out -- could the
presence of drop-out cause a disagreement between analysts when
determining the number of contributors?
A. Can you --
Q. Could the phenomena of drop-out affect whether or not one
analyst thinks it's one number and another analyst thinks it's
another number of contributors?
A. One analyst could think it was more contributors with more
drop-out. Is that what you mean?
Q. Yes.
A. Yes.
Q. How about drop-in, could that affect a determination of the
number of contributors?
A. Yes.
Q. And how about allele stacking, could that affect the
determination?
A. Sharing of alleles?
Q. Yes.
A. Yes.
Q. How about stutter, could that affect?
A. It could, although it's generally not as much of an issue,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
222
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
I think.
Q. And how about a degraded sample, could a degraded sample
affect the determination of the number of contributors?
A. Possibly. With a degraded sample you have less
information.
Q. Why is that? Can you explain why that is, please?
A. So the way that the forensic loci are set up, they go from
shorter to longer amplified pieces, and when a sample is
degraded the DNA is broken apart, so the longer amplicons might
be too degraded to show up. So what the profile looks like is
there may be results in any row in the electropherogram,
results at the first locus or loci, and then fewer results as
you go left to right in the electropherogram.
Q. Does that create what is called a slope effect?
A. It can, yes.
Q. And so there is more drop-out at the --
A. At the larger loci, yes.
Q. Does FST account for that?
A. FST does not explicitly model degradation, but degraded
samples can be used with FST.
Q. Well, first off, would you agree with me that degradation
is an issue with respect to crime scene samples?
A. Sure.
Q. And why would that be?
A. Degradation is often caused by environmental factors like
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
223
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
exposure to sun or high temperatures.
Q. And how about with touch samples, would degradation be more
prevalent in touched samples?
A. It can be.
Q. What's a touched sample?
A. A touched sample is an evidence item that does not contain
blood or saliva but is obtained -- DNA is obtained from
something that has been touched.
Q. And so --
A. I guess it could contain blood or saliva, but it's
something, an evidence item that has been touched.
Q. So with touched samples and degradation, you would agree
that the drop-out rates are different for those samples, for
those true samples, than other samples, for example, buccal
swabs or things like that?
A. Yes, they can be.
Q. Can be? Is it quite common?
A. Yes.
Q. But FST doesn't account for that?
A. It does not explicitly model it, but key degraded samples
can be used with FST.
Q. In the same fashion that nondegraded samples can be used.
There is no determination made, right?
A. Right.
Q. So, the FST worksheet has changed over time, right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
224
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. What do you mean by the worksheet?
Q. The sheet that's created that shows the different variables
that were input into the running of the program for that
specific sample.
A. The result sheet? Yes.
Q. Yes.
A. OK.
Q. That's changed over time, right?
A. Yes.
Q. When FST first started there was a section for whether or
not a sample was degraded or not degraded, right?
A. That did get printed out on the results sheet, yes.
Q. But in fact there was no way for the analyst running FST to
know whether or not the sample was actually degraded or not,
right?
A. No, that's not true. But it wouldn't be run any
differently if it were degraded or not.
Q. It was meaningless, right?
A. The reason it was there was that we had added a model for
degradation, and we were attempting to create better separation
between true contributors and noncontributors for degraded
samples, but we were not able to do that, so it was not
enhancing the program; it was not increasing the separation
between true contributors and noncontributors to model
degradation.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
225
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. And you conducted studies when you were trying to
incorporate that?
A. We did. We did.
Q. Where are those studies?
A. In the validation.
Q. And specifically where? Which studies are those?
A. There is a section on degraded samples.
Q. And it shows what the likelihood ratios for the degraded
samples were versus likelihood ratios for nondegraded samples?
A. Yes, there is a section of degraded samples specifically.
Q. I'm asking is that what it shows? It shows the difference
in the likelihood ratio between a degraded sample versus a
nondegraded sample?
A. It shows the likelihood ratios for the degraded sections,
and there are other sections for nondegraded samples.
Q. But it shows -- can you point me towards a graph, a table,
a chart that compares the two that form the basis for your
decision not to include it in the FST?
A. I don't have such a table or chart.
Q. But it's your testimony that you made those comparisons?
A. We did, yes.
Q. You made the comparisons without documenting them?
A. So when I was -- when we were developing FST, anything that
is going into the final version of the program is in here. We
did not include things that we didn't do or we didn't put in
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
226
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
the final version. This is a validation of FST as it is.
Q. OK. You didn't include them, but they exist somewhere.
A. I don't know where, but yes.
Q. Do they?
A. I can't tell you that. I don't know.
Q. Did you create them?
A. I did.
Q. Did you throw them out?
A. No.
Q. Did you save them?
A. They're somewhere on the computers that I used to use at
OCME. I did not create a formal section documenting that
decision, but --
Q. I know that. But what about the data, what did you do with
the data?
A. It's in here. The degraded samples are documented in here.
Q. But the comparison data, what did you do with that?
A. I did not create formal tables. I have already said I did
not create formal tables of these comparisons.
Q. You just looked at the --
A. I would have to look back and see exactly what is where,
but the bottom line was we did not improve the performance by
modeling degradation, so we discussed this with the
subcommittee and we decided not to model degradation in the
program.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
227
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. But FST is based upon estimated drop-out rates, right?
A. Yes.
Q. And everybody knows that with degradated samples, the level
of drop-out is higher, right?
A. Yes. And I refer back to my presentation about
underestimating the drop-out rate favors a defendant who is not
a contributor and lowers the likelihood ratio below what you
would get if you correctly specified the drop-out rate for a
true contributor.
Q. OK, but it doesn't favor all defendants, right?
A. I have explained those circumstances.
Q. And if we're using this in court, we're concerned about all
defendants, right? You said that was a concern of yours
putting a more accurate number to this, right?
A. I don't understand what situation you're saying would be
unfavorable.
Q. Any situation that's unfavorable to a defendant.
A. I presented our decision with the data supporting why we
want to underestimate the drop-out probability, and degradation
fits right into that.
Q. But let's talk about that for a second. On direct
examination you talked about there being validation studies
that fill up 20 plus binders?
A. Yes.
Q. And then summaries of those binders that are hundreds of
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
228
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
pages, right?
A. Yes.
Q. Do you really think that the members of the subcommittee
are going to read all of those binders?
A. Oh, certainly not, no.
Q. So just because something may be contained in a binder
somewhere, I mean that's meaningless unless you formally
present the issue to the subcommittee, right?
A. Right. And we did talk about degradation with the
subcommittee.
Q. But you didn't compare the results of degradated samples to
nondegradated samples to them. You didn't create -- you didn't
show them what the difference in the numbers were.
A. It's not a comparison of degraded versus nondegraded that I
think is important here. It's a comparison of if we increase
the drop-out rates that we're using when we determine that a
sample is degraded, are we able to get higher likelihood ratios
for true contributors and lower likelihood ratios for
noncontributors.
Q. And it's your position that you made those comparisons, you
didn't write anything -- or you don't have the written results
of those comparison, but based upon that that's what you made
your decision on?
A. Yes.
Q. What is a high voltage injection?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
229
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. It is a method for increasing the peak heights of a sample.
It increases peak heights.
Q. OCME has two separate protocols for high voltage
injections, right?
A. I think that there are more than two.
Q. Well, they have a separate one for high template samples
and low template samples, right?
A. Yes.
Q. And OCME's cut-off for the difference in between those two
is 200 picograms, right?
A. I don't know.
Q. As you sit here today, is it your testimony that DNA
samples -- that you are not aware of whether or not DNA samples
less than 200 picograms should be injected with a high voltage
injection pursuant to OCME's protocols? Is that your
testimony?
A. I am unaware of the protocol.
Q. OCME validated a method for determining the amount of
sampled DNA amplified in a given sample, correct?
A. Yes, or to be amplified. To be amplified. It's
preamplification.
Q. Sorry?
A. It's the amount of DNA before the amplification.
Q. Right. And that method was created based upon empirical
studies, right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
230
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. The method was created by Eric Buhl and one of his
colleagues. It's someone who is currently on the DNA
subcommittee, or was when we presented FST. He published the
method, yes.
Q. But when OCME decided to use that as part of their
protocols, it was based upon empirical studies, correct?
A. I don't know what you mean by based upon empirical studies.
Q. Do you know when OCME validated the protocols for that?
A. Before I was there.
Q. And so those protocols were not validated with the use of
FST in mind, correct?
A. No, they were just quantifying the amount of DNA in the
sample.
Q. Right, because FST wasn't created yet at that point, right?
A. Right. Of course they didn't know about it.
Q. And so when they were validated, they weren't validated for
the purposes of generating a likelihood ratio, correct?
A. They were validated for the purposes of quantifying the
amount of DNA in a sample.
Q. Is that your way of saying no?
A. It's my way of saying it's not specific to what happens
after that. There is a requirement, a SWGDAM and F.B.I.
requirement, that an attempt be made to quantify the DNA in a
sample to be used for forensic analysis.
Q. Right. So therefore at the time those methods were
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
231
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
validated they weren't being validated for the purposes of
creating a likelihood ratio, right?
MR. MCKAY: Objection. Asked and answered, and she
said she wasn't there at the time they were validated.
THE COURT: I think I understand, Mr. Strazza. You
can move on.
MR. STRAZZA: Thank you.
Q. And that method for determining the quantity or quant is
just an estimate, right?
A. Yes.
Q. In fact it's an estimate where 30 percent margin of error
has been calculated for that estimate, right?
A. On average, at the extremes, yes. In the middle of the
range, the dynamic range, it's tighter.
Q. I mean it's common knowledge that there is a 30 percent
margin of error in terms of quantification at the OCME, right?
MR. MCKAY: Objection.
A. Yes, that's often stated, but I'm clarifying.
THE COURT: Overruled.
A. I'm clarifying that the 30 percent is at the ends of the
dynamic range, so the very low samples and the very high
samples. In the middle it's not quite that variable.
Q. OK. But there are times where a quant value is determined
and it's actually 30 percent less than the true sample, right?
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
232
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
Q. And there are times when the quant value is determined and
it's 30 percent higher than the true sample, right?
A. Yes.
Q. And that's 60 percent swing, right?
A. 30 percent each way, OK, yes.
Q. And there certainly exists more accurate ways of
determining quant, right?
A. There is, yes.
Q. And quantity is measured by incorporating a florescent dye
into the DNA mixture and measuring the amount of florescence
produced, right?
A. Yes.
Q. And so when you are conducting regular PCR testing not for
use with FST you are trying to determine a quant value -- you
are trying to determine -- withdrawn.
When you are conducting a regular PCR testing and
you're not using it for purposes of FST, the exact size of the
sample isn't that important, right?
A. So the quant helps determine how much of the sample to put
in the amplification reaction.
Q. But you can make adjustments to that and fix it right away,
right?
A. What do you mean?
Q. You can dilute the sample if it's too high, or you can say,
look, there is not enough to test, right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
233
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. Yes. And that goes into the amplification, whose results
can be used for a variety of things. It's really to determine
how much DNA to put in the amplification.
Q. But what I'm getting at is the reason why a 30 percent
margin of error doesn't matter is because if it's not being
used for something like FST, the size doesn't really matter,
right?
A. No, it does matter.
Q. It's not as important.
A. I can't really answer that.
Q. When this method with a 30 percent margin of error rate was
validated, it was validated for use where a 30 percent margin
of error would be acceptable, right?
A. Sure.
Q. It wasn't validated for use with FST, right?
A. It doesn't make sense to me to think of validating a
laboratory measure for use with FST.
Q. OK, I'm going to try and explain it.
A. OK.
Q. Let's say the rubber that I want to use to make a tire has
been validated for use with bicycles. If it were known at the
time I was going through that validation process that I was
going to use that rubber to make a race car where it was going
to be going 200 miles an hour, it might not get validated for
that purpose, right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
234
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON6 Mitchell - cross
A. OK.
Q. When you say you don't -- that's what I'm getting at.
So what I'm getting at are the differences for which a
method was validated.
A. They validated it as a way of quantifying the DNA in a
sample.
Q. Right. Not for the purposes of computing a likelihood
ratio.
A. It's for the purposes of estimating the amount of DNA in a
sample.
Q. But if they get it wrong to the tune of a 30 percent margin
of error, it's not going to create something so significant as
determining somebody's innocence or guilt, right?
A. I'm not sure what the question is.
Q. I'll move on. Would you agree with me that having a broad
range within that 30 percent margin of error is fine for the
terms of regular PCR testing as long as the machine can take
that to amplify? Do you agree with that statement?
A. As long as the machine can take that --
Q. As long as it's a sufficient sample that the machine can
amplify.
A. That PCR can be done?
Q. Yes.
A. OK.
Q. Now, with FST, the quant value is used to determine the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
235
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
applicable drop-out rates that the program will apply, correct?
A. Right.
Q. And an inaccurate quantity amount will lead to an
inaccurate drop-out rate, right?
A. It's an estimate. It's our best estimate. It may not be
God's truth, but it's our best estimate.
Q. But if the wrong quantity amount is put into the FST
program, you're going get a less accurate likelihood ratio,
right?
A. If the drop-out rate is -- if the drop-out rate is not -- I
need to take a break. Can I take a five minute break and use
the bathroom?
THE COURT: Yes, sure.
All right. We will take five minutes.
(Recess)
(Continued on next page)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
236
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
(In open court)
THE COURT: Mr. Strazza.
BY MR. STRAZZA:
Q. Before you can run FST, there are certain values that have
to be input into the program by the analyst; is that correct?
A. Yes.
Q. One of those values is the number of contributors; right?
A. Yes.
Q. One of those values is the quant value; right?
A. Yes.
Q. And another one of those values is whether or not it's a
deducible mixture; right?
A. Correct.
Q. Is there anything I'm leaving out?
A. They put in the profiles --
Q. Right.
A. -- relevant to the case, but yes.
Q. When you created FST, you implement or you estimate
drop-out rates for certain scenarios; right?
A. Yes.
Q. There are different drop-out rates -- withdrawn.
When you created those rates to be used in FST, you
tested over 500,000 -- you made over 500,000 tests; right?
A. That was not in creating the mixtures for drop-out testing,
no.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
237
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. How many tests did you do for the mixtures for drop-out?
A. I would say upward of 1,500.
Q. 1,500, let's say. And some of those mixtures were
two-person, yes?
A. Yes.
Q. Some of people were two-person?
A. Yes.
Q. Some of those mixtures were three-person?
A. Yes.
Q. Some of those mixtures had smaller amounts of quant than
others?
A. Yes.
Q. Different values, as a matter of fact; right?
A. Yes.
Q. Some of those mixtures were deemed deducible and some of
them were deemed non-deducible; right?
A. Right.
Q. So the drop-out rates that you ended up with after you
conducted your study were different depending upon this those
different factors; correct?
A. Yes, they can be different.
Q. And so, for example, FST will use a different drop-out rate
for a two-person mixture at one locus than it would for a
three-person mixture at one location; right?
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
238
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. And FST will use a different drop-out rate for 50 picograms
of DNA than it would for 250 picograms of DNA; right?
A. They may be the same sometimes, but, yes, there's the
opportunity for them to be different.
Q. So my question for you is determining the one number of
contributors over another will ultimately affect which drop-out
rate that gets applied; right?
A. Yes.
Q. And the drop-out rate that gets applied will ultimately
affect the likelihood ratio that gets created; right?
A. Yes.
Q. Back to quant. Your method for determining the quant that
gets input into FST has that 30 percent margin of error rate;
correct?
A. Correct.
Q. And despite this limitation, you still chose to link
quantitation to drop-out rates?
A. Yes.
Q. Just to make sure I understand this, OCME was the only lab
in the world to do this; right?
A. To do which part?
Q. To link the quant value to the drop-out rate.
A. To my knowledge.
Q. There are other ways to incorporate the drop-out and
drop-in into likelihood ratios; right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
239
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. Yes.
Q. And the other programs that you mentioned earlier, they
don't do that; right?
A. Don't do what?
Q. Link drop-out rate to quant.
A. They do not use quant to predict drop-out rate, that's
correct.
Q. Are you aware of any other scientist in the world that does
in determining -- are you aware of any other scientist in the
world that uses that method?
A. No.
Q. Mixture ratios, the analyst has to choose a mixture ratio
before inputting that -- before running FST; right?
A. It's just a categorical decision of deducible or
non-deducible.
Q. And deducible is where you can determine there's a major
contributor; right?
A. Yes.
Q. And in that situation, one set of drop-out rates gets
applied; right?
A. Yes.
Q. And then in the other situation where you can't determine
the major contributor to the sample, another set of drop-out
rates would get applied?
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
240
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. For three-person samples, that ratio for a non-deducible
mixtures is one to one to one; right?
A. Those were the mixtures that we used to estimate the
drop-out probabilities, yes.
Q. Those are the ones that FST used for their program; right?
Not when you created the drop-out rates, but that's what FST
uses; right?
A. I don't understand.
Q. When you did your test to create the drop-out rates, you
used different choices beside non-deducible and deducible;
right?
A. No.
Q. When you did your testing on the samples, didn't you test
drop-out rates for different mixture ratios other than one to
one to one and one to one to five?
A. No.
Q. Then I want to clear this up, because then I'm
misunderstanding something.
When you did your studies to validate FST --
A. The validation or the drop-out rate estimation?
Q. There's the confusion on my part. The validation.
A. Yes.
Q. You used different mixture ratios other than one to one to
one and one to one to five for three-person mixtures?
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
241
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. And why was that important?
A. Because in casework, mixture ratios are not always one to
one to one or one to one to five, for example.
Q. So in a true case sample, that would actually be rare with
respect to a true case sample, right, one to one to one?
A. I don't know.
Q. In your expert opinion, if we're talking about a touch
sample, would you think that it would be common to have a
mixture ratio of one to one to one?
A. I don't know.
Q. You have no opinion on that?
A. Three equal contributors?
Q. For a touch sample.
A. I don't know.
Q. How about for a degraded sample?
A. I don't know.
Q. But in any event, it doesn't matter because FST for
three-person mixtures only uses one or the other; right?
A. Yes, one or the other rates are selected, yeah.
Q. One way to more accurately determine a mixture ratio would
be to look at peak heights for this purpose; right?
A. The analyst does look at peak heights in order to determine
whether a mixture is deducible or not.
Q. But in terms of assessing the more precise ratio, the
analyst does not do that with FST?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
242
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. No, an exact ratio is not estimated.
Q. But you could do that; right?
A. Probabilistic genotyping software does that. FST does not
do that.
Q. In your opinion, is probabilistic genotyping software more
accurate or create a more accurate likelihood ratio than FST?
A. It's more powerful.
Q. Is it more accepted within the scientific community?
A. I don't know.
Q. FST uses the drop-out rates for the total quantity of the
mixture; right?
A. It's based on the total quantity and the number of
contributors. So in a sense it's divided up among the
contributors.
Q. But it doesn't have a measurement for the quantity of each
individual contributor; right?
A. No.
Q. That would create a more accurate drop-out rate; right?
A. No, I'm saying it does implicitly take that into account.
Q. No, I know. That's not what I'm asking you. I know how it
implicitly does that, either the one to one to one or one to
one to five.
But if there was a way to account for a more precise
quant value for each particular contributor, that would lead to
a more accurate drop-out rate; right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
243
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. Perhaps.
Q. Do you ever do any studies on that?
A. No.
Q. In your expert opinion, do you think that's more probable
than not?
A. What's more probable than not?
Q. That you would get a more accurate drop-out rate if you
knew the quantity of the specific contributor as opposed to the
total quantity.
A. I don't know that it would be better, no.
Q. What do you think?
A. I think that -- I'm sorry. Rephrase the question, please.
THE COURT: She said she didn't know, so I'm not sure
whether -- in other words, I'm not sure what the "what do you
think" means if she didn't --
MR. STRAZZA: She's an expert, Judge.
THE COURT: Then you can then pose the question
hypothetically. I'm not sure if she's going to be able to
answer the question.
Q. FST assigns the same drop-out rate to all of the
contributors to the sample after it determines whether or not
it's one to one to one or one to one to five; right?
A. For the one to one to one samples, yes.
Q. Right. In that situation, there might not be an even -- in
a non-deducible mixture there might not be an even mixture
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
244
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
ratio in a crime scene sample; right?
A. Right.
Q. So the drop-out rate by each contributor to that sample
would be different?
A. Yes, but on average they would be the same.
Q. So you're saying that determining the drop-out rate for
each contributor to a mixture is the same as -- and then
determining it for each one and then averaging it will get you
the same number that determining the drop-out rate for the
total quant of the mixture will get you? It will get you the
same exact number?
A. The total quant divided by three would give you the average
of the three contributors.
Q. I understand that. But there exists the possibility that
one of the contributors to that sample in a true casework
sample contributed less than that; right?
A. Yes.
Q. So different drop-out rates would be applied by FST than
really should be applied, right, or that would really happen;
right?
A. We're always estimating drop-out probability. We never
know the precise truth. There may be some that are a little
higher; some that are a little lower. We estimate our best
estimate of it. And it would be fine to wish for perfect
understanding of the mixture in total, but it may not be
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
245
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
possible.
Q. Would you agree that the likelihood ratio created by FST is
an estimate based upon an estimate based upon an estimate?
A. There are several factors that are estimated that go into
the calculation, yes, and it is an estimate which -- whose
production required several estimates. This is not exclusive
to FST.
Q. And some of those estimates take it another step and rely
on more estimates; right? So it's not just one estimate
relying on an estimate?
A. Can you be more specific?
Q. Sure. You estimate the number of contributors; right?
A. Yes.
Q. To determine estimated drop-out rates; right?
A. The number of contributors is part of the estimating
drop-out rates, yes.
Q. But the estimated number of contributors determines what
estimated drop-out rate gets applied; right?
A. Yes, yes.
Q. And the estimated quant value determines what estimated
drop-out rate gets applied; right?
A. Yes.
Q. And the estimated mixture ratio determines what estimated
drop-out rate gets applied; right?
A. Yes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
246
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. And that happens across each different locus; right?
A. Yes.
Q. And then they all get multiplied by one another; right?
A. The likelihood ratios do get multiplied across the loci,
yes.
Q. So if you have a small difference that could -- a small
difference in the numbers for one of those locus, when you
multiply that by another, by another, by another, a small
number could lead to a significant difference in the likelihood
ratio; right?
A. There is some uncertainty in each parameter, but it doesn't
mean that they're all amplifying in one direction. They're in
a sense averaging each other out. You might overestimate one;
another one would be underestimated. It's a -- they are not --
I'm waiting for him to stop talking.
They are not synergistically amplifying the likelihood
ratio.
Q. But if you get the wrong estimate for one, you're going to
get the wrong estimate for the other?
A. Not necessarily, no.
Q. OK. Are you familiar with the other programs that were
discussed that create likelihood ratios in the forensic
community?
A. Yes, somewhat.
Q. Are you familiar with the term semicontinuous versus
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
247
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
continuous?
A. Yes.
Q. What does that mean?
A. Semicontinuous means something like FST where we take the
allele calls out of the gene mapper software and compute a
likelihood ratio on that. A fully continuous model is also
estimating the probability of each noncontributor genotype at
each locus.
Q. So it considers more information?
A. It does.
Q. And based upon your experience with research, considering
more information will lead to a more accurate result, right,
generally speaking, like you say?
A. It's a more powerful tool. A probabilistic genotyping
software is more powerful than FST because it's using more
information.
Q. But generally speaking, if you have more information to
consider, you're going to get a more accurate result; right?
In any kind of research that you've done, if you have more
information to consider, you're going to get a more accurate
result; right?
A. It's possible to a point.
Q. So if somebody were to say that the difference between FST
and a fully continuous -- and a more fully continuous model
were like the difference between a paper map and Google Maps,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
248
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
would you agree with that generalization?
A. No, I can't make a comparison like that.
THE COURT: That's fine. He can take it.
Q. That would be silly; right?
A. It would be hard to say whether that's an accurate
comparison.
THE COURT: Sometimes things need to be dumbed down
for people, but I'm sorry. Go ahead.
Q. Just bear with me. Let's go with that for a second. With
a paper map, right, if you were trying to determine how long it
would take you to get somewhere, you'd have to look at the
distance -- you'd be able to see the distance between two
points; right?
A. Sure.
Q. You'd have to look at the scale on the map to see what that
distance represents; right?
MR. McKAY: Your Honor, I think the witness has not
agreed with this analogy. I think this cross-examination can
be used with another witness. And it's not clear to me how
it's relevant to Dr. Mitchell's expertise in FST in particular.
THE COURT: I understand what -- let's move on to
another topic.
MR. STRAZZA: OK.
Q. What made you choose quantitation as a means to generate
the drop-out rates?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
249
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. There's general consensus that drop-out probability is
based on the amount of DNA in a sample. The methods that use
peak heights are using the heights of the peak as a proxy for
the amount of DNA in the sample. So we chose to directly
measure the amount of DNA in the sample.
Q. When you say there's a "general consensus," why do you say
that?
A. It's kind of a known -- a known fact that up to a point
more DNA in a sample has a lower probability of dropping out.
Q. But there must be a reason why no other lab in the world
uses that to determine their drop-out rates even though --
A. Well, there aren't actually that many labs applying
likelihood ratios with drop-out and drop-in in a semicontinuous
fashion.
Q. Does that mean that that method is not generally accepted
within the scientific community?
A. No, it doesn't.
Q. But everybody else chooses to do something different?
A. A lot of labs are not doing anything or are still using a
combined probability of inclusion, and many labs are moving to
the probabilistic genotyping software because those are
commercial products that are available, and that is easier for
a lab to take on than a method that -- like, for example, Lab
Retriever that was developed by an academic and does not have
product support that goes with it.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
250
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. Would the same be true for FST?
A. Well, FST wasn't available to other labs until now.
Q. Are you aware of any other labs that are interested in
using FST?
MR. McKAY: Objection.
THE COURT: Well, I'll allow it. You can answer if
you know.
A. When we were developing it, there was interest from other
labs in using it, yes.
Q. Did they ultimately use it?
A. No. It wasn't released.
Q. Now that it's released, are they pursuing that?
A. I don't know. I don't really have contact in that area
anymore.
Q. But you're on the subcommittee for validating --
A. Probabilistic --
Q. -- validation studies?
A. For probabilistic genotyping software, yes.
Q. So you're dealing with the same players in the community;
right?
A. I'm dealing with some people that are in public labs, but
others who are academics or are at NIST.
Q. By the way, your work as part of that subcommittee, part of
that work is preventing others from making the same mistakes
that OCME made during the validation process; right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
251
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. Excuse me?
Q. Your work as part of the validation subcommittee, part of
your work there is to prevent or is to learn from the mistakes
that OCME made during its validation process; right?
A. No.
Q. Are you saying OCME's validation process was perfect?
A. No.
Q. So there were some things that, if you could do differently
now, you would; right?
A. The validation guidelines that we're developing are not
based on my assessment of mistakes that OCME might have made.
Q. They're not based upon your experience at OCME?
A. They are not based upon correcting any mistakes that might
have been made by OCME.
Q. In your work there, though, if you are aware of a method
that would work better than the one you used at OCME, you would
certainly advise them on that; right?
A. What do you mean by "method that would work better"?
Q. A method of validation, the way you conducted your
validation studies or whatever you're trying to accomplish on
that committee.
A. Nothing that has gone into that document is correcting an
alleged wrong by OCME.
Q. OK. That's not really what I asked you.
Your value to that subcommittee in part is based upon
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
252
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
your experience at the OCME --
A. Yes.
Q. -- during its validation process; right?
A. Yes.
Q. So one of the things that you have to contribute to that
subcommittee is what you've learned from mistakes that were
made during the validation process at OCME?
A. I would not phrase it that way, no.
Q. Let's talk about the hidden function that was in the
original version of FST. I just want to be clear, make sure I
understood everything correctly.
When you testified on direct examination, you
testified that there were two changes to, I'll say, FST version
2.5. Is it correct that there were two changes made between
the previous version that was validated and the version FST
2.5?
A. I don't think so. I think there was one.
Q. One of the changes was where you cap off, you don't use --
you drop out a loci; right? A locus, excuse me; right?
A. If there is a locus where the allele frequencies add up to
.97 or higher, that locus is not used in the likelihood ratio
calculation.
Q. Right. And the other one you referred to as theta
correction. Those are two separate things?
A. The theta correction was in the method all along.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
253
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. But you had to make a correction; right?
A. No. It's called a theta correction, but it wasn't -- it
doesn't mean that we were doing something wrong and we
corrected it. It's a population genetics parameter called a
theta correction.
Q. But as I understand it, it's two separate things; right?
There was something with allele frequencies and then there was
something with not dropping a locus, right, two --
A. There was one thing, the cap. The .97 cap is the only
thing.
Q. But the judge asked you specifically whether or not you had
presented that to the subcommittee --
A. The theta.
Q. -- and your answer was different, if I heard correctly, for
the dropping of the locus versus the theta correction; right?
A. The theta correction was part of the methodology all along,
and it was discussed with the subcommittee.
Q. Right, but you had to make a change?
A. No, that was part of the methodology all along.
Q. Let's back up. There was a version of FST that went
online -- there was a version of FST that went online in April
of 2011; right?
A. Right.
Q. And then it went online for a limited amount of time and
then changes had to be made; right?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
254
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. Yes.
Q. What changes had to be made?
A. If the allele frequencies at a particular locus in the
evidence sample add up to .97 or higher, that locus is not used
in the likelihood ratio calculation.
Q. And is that called a theta correction?
A. No, it is not.
Q. OK. Then my apologies. I misunderstood.
So was anything done with respect to the theta
correction after FST went online in April 2011?
A. No.
Q. Now that we're talking about the same thing, you were aware
of the dropping of the locus under the circumstance you just
described. You were aware that FST was fixed to incorporate
that at the time you testified in Brooklyn?
A. Yes.
Q. And you never brought that up in your testimony; right?
A. I didn't.
Q. You are aware that that change affects the likelihood
ratio; right?
A. It can.
Q. That information was not public. The information about
that change was not public at the time; right?
A. I thought that I had mentioned it in public settings, and I
thought it was in the validation paper. And I am apparently
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
255
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
incorrect. I thought we had included it. I was not hiding it.
Q. So you thought that after you made the change, you went
back and incorporated it into the validation paper?
A. No. The validation paper came out a year after FST went
online.
Q. So you thought --
A. I thought --
Q. -- that that was updated, the information was --
A. I thought I had mentioned it in the publication, but I am
wrong. Apparently, I did not. I was not trying to hide it is
what I'm saying.
Q. Even though you testified extensively at that hearing in
Brooklyn; light?
A. Yes. It's actually not a giant factor in the calculation
of the LR.
Q. Would you agree with me, doctor, that as you sit there
right now, there are people in the scientific community who
disagree with you?
A. I would imagine. People disagree with anything.
Q. But you know of people who disagree that that is --
A. I don't actually. I don't know.
Q. Have you discussed it with any of your learned colleagues?
A. What do you mean by my "learned colleagues"?
Q. Other members -- well, let me ask you. You're on that
subcommittee with who?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
256
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. You mean the OSAC committee.
Q. Yes.
A. That is not related to FST methodology.
Q. But it's related to validating protocols for probabilistic
genotypes; right?
A. Right.
Q. If a change like that were made or if a factor like that
was incorporated into one of the programs being validated, is
that something you would advise them to include in their
validation?
A. There are things that can happen post-validation that
requires a performance check, and I would put this in that
category. Of course, if things happen during development,
those things can all be included in a validation, but
post-validation, not everything would require revalidation.
Q. Your position is something like this does not require
revalidation?
A. That's correct.
Q. Even though it could have a huge effect on the likelihood
ratio?
A. I disagree that it would have a huge effect on the
likelihood ratio.
Q. Let's talk about that. Do you agree that either side can
benefit -- either side to the numerator or the denominator can
benefit depending upon the alleles present in the mixture and
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
257
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
in the reference sample?
A. I don't understand the question.
Q. If you take a situation before the modification was made or
the change was made, let's say -- let's do this: Original FST
and then the fixed FST, OK. I'm referring to before the change
was made and after the change was made.
A. OK.
Q. Do you agree that either side can benefit -- withdrawn.
Do you agree that a different likelihood ratio gets
generated with the original version of FST than the fixed
version of FST?
A. It can.
Q. It can. How often does that happen?
A. That depends on how many alleles are in the mixture. I
don't know worldwide.
Q. Did you do any studies on that?
A. We looked at our performance check samples.
Q. That was only 12 samples; right?
A. Right, but we did comparisons to a database of 1,250
noncontributors.
Q. How did you do that comparison? Did you use the bulk
calculator or did you hand put each calculation into FST?
A. We used the bulk calculator.
Q. Did you modify the bulk calculator as well?
A. The bulk calculator is just a way of pulling in profiles to
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
258
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
use.
Q. Did you have to make any changes to the bulk calculator
when you changed the version of FST?
A. No, I believe it's one function, and it pulls in the
profiles from the bulk run.
Q. And it incorporates it into whatever version of FST that
you're using?
A. Yes.
Q. So there wouldn't be a need to change the bulk -- is it
called the bulk calculator?
A. We used it as like a way to test a set of samples or a set
of profiles at a time. We called it a bulk calculation just
because it's pulling in a set of profiles for comparison.
Q. Back to what I was saying before. Do you agree that the
likelihood ratio could change based upon the newer version of
FST either for the prosecution or for the defense?
A. Yes.
Q. And do you agree that as long as -- is it your position
that as long as the fixed evenly helps both sides after the
change, that there's no need to worry about the accuracy of it?
A. I've never said that.
Q. I'm asking you now if you agree with that statement.
A. I don't know. I don't think so.
Q. Do you think we need to be worried about the accuracy of
dropping a locus in that scenario?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
259
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
A. No.
Q. By the way, the reason you dropped that locus is because
why again?
A. Because when the theta correction, the population genetics
parameter that has always been in FST is applied, it increases
the estimates of genotype frequencies that are used, homozygous
genotype frequencies in particular. And if the total allele
frequency is close to 1, that estimate can then bring the total
genotype frequency estimate above 1, which is mathematical
nonsense.
Q. What is the effect if a suspect has those common alleles?
What is the affect on the likelihood ratio when you drop that
locus?
A. It depends on whether or not the alleles appeared in the
mixture --
Q. If they did?
A. -- and whether they repeated.
Q. If they did?
A. I can't tell you theoretically what happens with a
profile -- an imaginary profile. I would have to look and see.
Q. Do you agree that the likelihood ratio would be different
for each different comparison genotype?
A. I don't understand.
Q. What don't you understand?
A. The question.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
260
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. What part of the question, though?
A. Can you rephrase the question?
Q. Sure. The likelihood ratio between the old version of FST,
the fixed version of FST, do you believe that the change or the
difference in the two will be different for each different
genotype comparison?
A. I don't understand what you're asking.
Q. Do you agree that the differences -- do you think the
differences will always be small?
A. I don't know what you define as small, what -- I'm having
trouble theoretically answering that question.
MR. McKAY: Can you clarify differences between what?
Q. All of these questions I'm talking about the likelihood
ratio generated in the original version of FST versus the
likelihood ratio generated with the corrected version of FST.
A. The ones that we observed, the differences that we observed
were small.
Q. You only observed one sample that dropped a locus; right?
A. Two.
Q. Two?
A. Yes.
Q. But you ignored one of those two?
A. Excuse me?
Q. You ignored one of those two samples; right?
A. No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
261
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. You didn't. You used the data from both samples?
A. Yes.
Q. Let's talk about both samples. Tell me about them.
A. Well, in the document that is part of the evidence
summarizing that update and the performance check, both samples
are described.
Q. Can you explain it to the Court, because I think we passed
over one on your direct examination. I think we only talked
about one of them.
MR. McKAY: Your Honor, first of all, that misstates
the testimony. But to clarify, we can go to Government
Exhibit 16 --
THE COURT: Yes, I was just about to flip there.
MR. McKAY: -- for which there's again not a tab.
It's behind 15.
THE COURT: It's a two-page document, three-page
document, something like that, I think.
THE WITNESS: Yes, the front and back. No, it's a
two-page document.
THE COURT: OK.
BY MR. STRAZZA:
Q. Can you go over those two samples and the effect on them.
A. Yes. In the bottom paragraph on the first page, it says
two samples had one locus each that displayed such values,
meaning values where the allele frequencies added up to .97 or
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
262
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
greater. Therefore, their LR values were slightly modified as
expected. For example, for one sample where the LR at D3 was
equated to 1, the most conservative likelihood ratio was .34,
meaning the lowest of the four populations for that true
contributor was .34 previously and was .42 with FST version
2.0. So that LR went from .34 to .42. Similarly, for the
other sample, the LR was previously 3.38 times 10 to the
fourth, and with FST version 2.0, it was 3.82 times 10 to the
fourth when VWA was assigned a value of 1.
Q. I remember hearing something you wouldn't have run -- one
of those wouldn't have been run in FST; right?
A. No. These are the true contributors to the sample. Those
are the performance check samples. We also tested
noncontributors. These are real contributors. We also tested
noncontributors to these samples, and that is overall the
14,976 comparisons mentioned on the next page.
Q. OK. Tell me about that.
A. So for each of the performance check samples, we reran the
bulk comparison with the noncontributor database. And for
these two samples, there were some differences. And one is the
value that we mentioned this morning where a noncontributor --
the likelihood ratio for a noncontributor went from slightly
under 1 to, I believe, about 29. That was the example where if
this were a case, that noncontributor would actually -- that
comparison would not have been made because three of that
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
263
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
person's alleles were missing from the mixture. So that would
have been deemed no conclusions can be drawn. That's the one
where FST would not have been run in a casework scenario.
MR. STRAZZA: May I have a moment, your Honor?
THE COURT: Yes.
(Pause)
MR. STRAZZA: May I, your Honor?
THE COURT: Yes.
Q. Dr. Mitchell, you just described a scenario where it turned
into a false positive; right?
A. Yes.
Q. If three alleles are missing from a suspect profile with
comparison samples, that would not be run in FST; right?
A. According to this sample, a casework analyst confirmed
that, yes.
Q. I'm sorry. You mentioned that you performed hand
calculations as part of your FST validation study; right?
A. Yes.
Q. The purpose of that was to show that FST was calculating
the likelihood ratio correctly; right?
A. Yes.
Q. Did you perform hand calculations at all 15 loci?
A. I don't recall.
Q. Do you recall how many hand calculations you performed?
A. No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
264
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. Isn't it true that you only calculated two three-person
mixtures by hand?
A. I don't remember.
Q. Do you recall how many template quant values were tested?
A. No.
Q. Did your hand calculations reveal any kind of -- excuse me.
Did your hand calculations reveal the problem that
corrected between the original version of FST and the second
version of FST?
A. No.
Q. Were you asked at the Brooklyn Frye hearing about the
limited nature of your hand calculations?
A. I believe so, but I don't remember the specifics.
Q. And you didn't mention anything about a correction that you
made to FST at the Brooklyn Frye hearing, right, just so we're
clear?
A. No.
Q. Can a likelihood ratio at a single locus be as high as
thousand for a specific profile?
A. I don't know.
Q. Did you ever see that in any of your studies?
A. I don't remember.
Q. Didn't you show a slide to the DNA subcommittee that had a
likelihood ratio of 20,000?
A. I don't remember. Was it a real profile or something I'd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
265
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
made up?
Q. I don't know.
A. I don't know either. I don't know what you're talking
about.
Q. Can the likelihood ratio at the same locus be close to 1
for a different specific profile?
A. I don't know.
Q. Could the likelihood ratio at the same locus be as low as
one in a thousand for a --
A. I don't even know what you're talking about to begin with.
Q. If you're talking about the same -- dropping a locus -- may
I have a moment, please?
THE COURT: Yes, go ahead.
MR. STRAZZA: Sorry. Sorry.
Q. Sorry, Dr. Mitchell. I'm going to try and word this
better.
A. OK.
Q. Does dropping a locus like you would in the situation that
you -- in the corrected version of FST, in your expert opinion,
could doing that change the likelihood ratio by a thousand?
A. I would say probably not for these loci.
Q. Couldn't you have solved the problem another way?
A. Sure.
Q. Without changing the source code to FST?
A. What are you suggesting?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
266
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. Couldn't you have -- one moment, please.
Couldn't you just not enter the alleles, the problem
alleles, into the --
A. Change the evidence?
Q. No. Couldn't you not -- no, I'm never suggesting you
should change the evidence.
A. OK.
Q. If the alleles at the problem locus were not entered into
the FST worksheet, wouldn't the problem be solved without
changing the source code?
A. By changing the evidence?
Q. Well, in essence, your change is not using all the evidence
anyway, right, by dropping the locus; right?
A. We are not using that locus, yes.
Q. So when you say "by changing the evidence," that's not what
I'm suggesting. I'm saying instead of -- you're not using the
locus, so instead of doing it that way and having to change the
source code to accomplish that, couldn't you just not use the
alleles that caused the problem?
A. It sounds to me like you're suggesting changing the
evidence.
Q. That's definitely not what I'm suggesting.
A. Then I don't understand what you're suggesting.
Q. Almost done. Forget about the numbers.
A. OK.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
267
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB6HJON7 Mitchell - Cross
Q. Some basic questions. Do you think that it would be
important for the members of the subcommittee who voted to
approve FST, do you think it would be important for them to
know about the change that you made between the original
version of FST and the newer version of FST?
A. No. In consultation with our technical lead, Eugene Yant,
we determined that this was not a substantial change. That
this required only a performance check.
Q. But you thought it was important enough to incorporate into
the validation studies, right, because you thought that was
done; right?
A. I'm sorry?
Q. You testified earlier that you thought this was disclosed
in the validation studies; right?
A. I thought it was disclosed in the paper describing the
validation.
Q. And you thought it was important enough to do that; right?
A. I don't think it's as important as you're making it sound.
Q. What I'm asking you is whether you thought it was important
enough to incorporate into that paper.
A. To mention, sure.
Q. And that wasn't done?
A. No, it was not.
MR. STRAZZA: I have no further questions.
(Continued on next page)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
268
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
MR. MCKAY: Briefly, your Honor.
REDIRECT EXAMINATION
BY MR. MCKAY:
Q. Do you recall just a moment ago Mr. Strazza was asking you
some questions about how much a particular locus could change
the likelihood ratio?
A. Yes.
Q. In the examples that we walked through, the likelihood
ratios that you were talking about would be the sample
calculations?
A. Yes.
Q. Were those based on real data or hypothetical numbers that
you made up for purposes of illustration?
A. Those were hypothetical.
Q. Now, do you remember a few minutes ago Mr. Strazza asked
you a number of questions about whether the likelihood ratios
would change based on the newer or older version of FST?
A. Yes.
Q. And he asked you about tests that you performed to evaluate
whether there would be changes.
A. Yes.
Q. And you mentioned 12 samples.
A. Yes.
Q. To be clear, how many times did you compare those 12
samples to noncontributors?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
269
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
A. Each sample was compared about 1250 times to noncontributor
samples.
Q. So we are talking about thousands and thousands of
comparisons to known noncontributors?
A. Yes.
Q. Do you recall a few minutes earlier when Mr. Strazza was
asking you questions about other laboratories used likelihood
ratio programs?
A. Yes.
Q. And that there are some, indeed perhaps many, that do not
use likelihood ratio programs.
A. Yes.
Q. Can you talk about OCME generally in terms of the size and
amount of resources OCME has?
A. OCME has, as long as I've been there, had a chief medical
examiner who is very interested in research and has been able
to obtain from the Office of Management and Budget, OMB, funds
for OCME to use for research and development purposes.
Developing FST was incredibly expensive, validating it
was incredibly expensive. Most labs do not have that kind of
resource.
Q. And were there particular events in OCME's history that
produced a particular focus on evaluating DNA at OCME?
A. Yes, definitely. The September 11 attacks increased a
focus on OCME and on forensic DNA analysis.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
270
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
Q. And the resources that were devoted to this type of
analysis?
A. Yes.
Q. Do you recall Mr. Strazza early on in his cross-examination
asking you a number of questions about estimating the number of
contributors?
A. Yes.
Q. And there were a number of questions about OCME protocols
on estimated number of contributors?
A. Yes.
Q. And about an article that you contributed to that was
published in a peer reviewed journal about estimating number of
contributors?
A. Yes.
Q. As far as you know, are the protocols that OCME uses common
in the forensic community?
A. Yes.
Q. And are those protocols based in part to at least consider
some of the guidelines set forth in your peer reviewed article?
A. Yes.
Q. And to be clear, in a situation in which a mixture is
estimated to be three people, and a likelihood ratio is
calculated, does that likelihood ratio necessarily support the
prosecution or the defense?
A. No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
271
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
Q. It could go either way, right?
A. Yes.
Q. Do you remember Mr. Strazza and Mr. --
Mr. Deluca, if you could pull up the exhibits if you
have them.
Do you remember Mr. Strazza asked you a series of
questions about the degradation aspect of the validation?
A. Yes.
Q. And he was asking you whether you created a comparisons of
degraded samples to nondegraded samples?
A. Yes.
Q. To be clear, if you evaluated one degraded sample and a
different sample that was not degraded, would you expect to
have the same likelihood ratio?
A. No, they are different samples.
Q. And is that because of the degradation or is it because
they're just different samples?
A. Both.
Q. Now, what were you trying to determine when you looked at
the degradation model?
A. We were trying to determine if by adjusting drop-out
probabilities for degraded samples we could get higher
likelihood ratios for true contributors to those samples
without affecting the false positive rate for noncontributors.
Q. And so what was your finding about whether adjusting the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
272
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
drop-out rate for degraded samples in fact gave you better
separation?
A. We were not able to get higher likelihood ratios for true
contributors without increasing the likelihood ratios obtained
for noncontributors.
Q. And so as a result of that, how did you account for
degradation in FST?
A. So the decision was to include the degraded samples as part
of the validation and then to be able to use FST with degraded
samples without having a special function that would be called.
Q. Do you remember the series of questions about how, if at
all, you reported your data on degradation?
A. Yes.
MR. MCKAY: Mr. Deluca, can you pull up Government
Exhibit 7 and turn to page 443 of that PDF.
Q. And just to remind the court, what was Government Exhibit
7.
A. It was the executive summaries of the validation volumes.
Q. And so on this page here can you just read the title of
this volume of the executive summary?
A. Volume 15 summary: Estimation of Degree of Sample
Degradation and Generation of DNA Profiles from Two-Person
ID28 and ID31 Mixtures with Both Components Degraded and
Generation of DNA Profiles from Degraded, Two, Three and Four
Person Touched Item Samples.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
273
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
Q. Do you recall Mr. Strazza --
You can take that down, Mr. Deluca.
Do you recall Mr. Strazza asking you a series of
questions about whether you get a more accurate likelihood
ratio if you varied this variable or this variable?
A. Yes.
Q. And he listed some of the factors that go into the
estimation of drop-out rates, right?
A. Yes.
Q. Do you have perfect information about a case work sample?
A. No.
Q. Are you aware of another program that has perfect
information about a case work sample?
A. No.
Q. And by case work sample, I mean actual crime scene
evidence.
A. Right.
Q. Does FST purport to report perfect information?
A. No.
Q. Does it purport to report a precise likelihood ratio?
A. No.
Q. What is FST reporting?
A. It is a quantitative estimate of the strength of the
evidence.
MR. MCKAY: No further questions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
274
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - recross
THE COURT: OK.
MR. STRAZZA: Just brief follow-up, I promise.
THE COURT: Very brief.
RECROSS EXAMINATION
BY MR. STRAZZA:
Q. When you say it's just supposed to be an estimate, doesn't
it report to three significant figures?
A. I believe so.
Q. And so if it was 2.14, it would produce a result like that,
right?
A. Yes.
Q. Doesn't that make it seem like it's accurate to the 2.14
number that it's presenting?
A. It's not unreasonable to report this way.
Q. That's not what I asked you, doctor. Please. Doesn't that
seem like it's precise if you're producing a calculation like
that, 2.14?
A. I think it's a fine presentation of the results.
Q. There is no difference between 2.0 and 2.14, right, in
terms of the results of the likelihood ratio?
A. They would be interpreted in the same way; they are both in
the same range.
Q. OK. But it's just -- it's not accurate -- would you be
able to -- is it accurate enough to describe the difference
between 2.0 and 2.14?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
275
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
A. Those would be interpreted in the same way.
Q. So are you saying there is no -- the only difference is the
four categories?
A. I'm sorry?
Q. Are you saying that the only difference is when they're
interpreted in a different way in that one of the four
categories? That's the only time they're different? Is that
what you're saying?
A. No, I don't understand.
Q. When you say those would be interpreted the same way, what
do you mean by that?
A. I mean that they are both very close to one; there is not
much information there.
Q. So why list a 2.14 then? Doesn't that convey a level of
accuracy that just simply doesn't exist?
A. No.
MR. STRAZZA: No further questions.
THE COURT: OK.
REDIRECT EXAMINATION
BY MR. MCKAY:
Q. If you were testifying in court at a jury trial about FST
results, and you were asked about degree of accuracy down to
2.14, would you explain to the jury that it's an estimate?
A. Yes.
Q. Would you expect an OCME criminalist to be frank about the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
276
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
fact that that was an estimate?
A. Yes.
MR. MCKAY: No further questions.
THE COURT: All right. Thank you very much, doctor.
You can step down.
THE WITNESS: Thank you.
(Witness excused)
MR. MCKAY: Should we recall Dr. O'Connor?
THE COURT: Sure, but I'm not sure how much more --
well, let me ask the marshals what the situation is, and also
the court reporters also.
MARSHAL: We have to transport him back. He is
holding up the entire Brooklyn.
MR. MCKAY: So then perhaps we could consult with Dr.
O'Connor about scheduling and then come back and have a brief
logistics discussion with the court?
THE COURT: That's fine. Let me ask this.
Mr. Strazza, is it all right if your client -- well, check with
your client and see if he is OK with us discussing logistics so
that he can get back to.
THE DEFENDANT: Yeah, it's all right.
THE COURT: All right.
MR. STRAZZA: You heard him.
THE COURT: OK, so, yes, you should go and speak to
the doctor, and then we can talk about logistics, although --
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
277
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
and I will take a look at my calendar and see what we can
accommodate.
MR. MCKAY: Thank you, your Honor.
THE COURT: Mr. McKay, we do have the 15th, right?
MR. MCKAY: Your Honor, I think --
THE COURT: Or is that not --
MR. MCKAY: Both because the defense has three
different witnesses who are going to need to go on the 15th,
and because it seems reasonable to get Dr. O'Connor back on
while his direct is fresh in the mind, we were hoping we might
fit him at some point in the next few days in the court's
schedule. I know you have the jury trial tomorrow.
THE COURT: Yes, it's conceivable. It's just that
it's going to have to -- and again tomorrow is going to be a
full day with jury addresses and -- off the record.
In any event, it's going to be an all day affair
tomorrow. Wednesday I do have conferences, and Thursday I have
a bunch of conferences.
So I guess the question is there are chunks of time on
Wednesday, including after my last conference at 4, but there
is time in between -- but, you know, I just don't --
MR. ROSANIA: The marshals will not be happy about
that.
MR. STRAZZA: Time out. Can we go off the record for
a second?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
278
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
THE COURT: Yeah.
MR. STRAZZA: Back on the record. In terms of
estimating the length of the cross, I mean it's really going to
depend on the way it goes. Like today was very different than
I had anticipated in terms of estimating the length of the
cross. So, I just -- I don't want to say -- when we are
working with such a tight schedule, your Honor, like fitting us
in at four --
THE COURT: No, I'm just providing -- let me just say
this. I think what I was just doing is providing time, because
as I understand it, it may be that since we're on the 15th it
looks like we're scheduled to start at 11:30 or around that
time.
MR. MCKAY: So I think our concern would be if we
start at 11:30 on the 15th and the defense is calling three
separate experts, there is no chance we would finish the 15th,
and that's particularly so if we also have to do Dr. O'Connor's
cross. So if the court has a two or three hour window in the
next few days that Dr. O'Connor could come back for the cross,
and if the cross takes longer than that, we will come back
again, but I think realistically for Dr. O'Connor a two or
three hour window is probably going to be sufficient.
MR. STRAZZA: I agree.
MR. ROSANIA: I agree. I think my cross will be more
than sufficiently done.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
279
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
THE COURT: I have a 10 o'clock -- I was thinking
maybe if we start at 9 o'clock on Wednesday, break at 10 for me
to do the sentencing, come back at 11, and then we would have
from 11 until 12:45.
MR. MCKAY: That's Wednesday the 8th, your Honor?
THE COURT: That's Wednesday the 8th, that's correct.
MR. MCKAY: That certainly works for the government.
Let me double check with Dr. O'Connor.
THE COURT: So we would be starting at 9, going for
about an hour, and then starting up again at about 11.
MR. MCKAY: I will be right back, your Honor.
Your Honor, Wednesday at 9 works for Dr. O'Connor.
THE COURT: And then will you be able to -- well, all
right, Wednesday at 9 he will be able to stay also, because it
may go longer than an hour, so he would come back at 11?
MR. MCKAY: Yes.
THE COURT: OK. So why don't we plan on doing that.
So, we will have continuing of the Daubert. So we
will go for an hour and then at 11.
MR. ROSANIA: And, Dr. O'Connor is subject to cross,
meaning.
MR. MCKAY: Yes, we won't discuss substance with him.
THE COURT: I think I had said that to him when he
left the stand. That should apply. Do you want me to bring
him back in to advise him of that?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
280
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
MR. STRAZZA: No, we trust the government.
THE COURT: All right. So you are in my calendar.
Obviously the one thing that's unpredictable is if I have jury
notes I have to deal with those.
MR. MCKAY: Of course.
THE COURT: So we will handle that. I will say this,
the 15th obviously I have time, but I think did we also give
you the 16th?
LAW CLERK: No, only the 15th.
THE COURT: No, the only reason I'm saying -- well,
all right.
MR. STRAZZA: How long is your cross going to be, Tom?
MR. MCKAY: It's hard to predict these things. And my
second seat doesn't like me to tell the court --
MR. ROSANIA: It's hard to estimate.
THE COURT: So there anything else we need to discuss
right now?
MR. MCKAY: Not from the government.
MR. STRAZZA: No, your Honor.
MR. ROSANIA: No.
THE COURT: All right. Let me ask you this just with
regard to the other DNA. My recollection is that I got a
letter saying that there isn't going to be a Daubert type
challenge to that and that that will be dealt with through
cross-examination at the time of trial. Is that an accurate
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
281
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
statement?
MR. ROSANIA: You can incorporate it or not.
MR. STRAZZA: I think some Dr. Chakraborty has some
issues with it, but I think your statement was accurate. And
we can address those issues at trial.
MR. MCKAY: OK. So to be clear there is not a Daubert
motion about the hat.
MR. STRAZZA: Correct.
MR. ROSANIA: Right.
THE COURT: The only reason I'm asking is because I
know there was some testing that occurred, and that's my
recollection that as a result of the -- not as a result -- but
that that did not lead to a situation where a hearing was being
requested for some reason.
MR. MCKAY: My understanding is that there will be a
defense expert at trial who retested the hat and purports to
have gotten a different result, and so I think there will be,
if I'm right, testimony from that defense expert and perhaps
rebuttal testimony from a government expert about why the
results were different.
MR. STRAZZA: Well, right, but there may also be
testimony from Dr. Chakraborty about his opinion on that in
front of the jury.
MR. MCKAY: Right.
MR. STRAZZA: And that's a different expert than the
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
282
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
HB67JON8 Mitchell - redirect
expert who separately tested the hat.
MR. MCKAY: And I think we can discuss the parameters
of that closer to trial as long as we're talking about not
holding a Daubert hearing about that issue, right?
THE COURT: All right. I'm not -- yeah. OK. I think
I understand, but is the result of that test, is that
something, Mr. Strazza, that you've turned over or not turned
over?
MR. STRAZZA: Yes, I have turned over the entire case
file. The government is aware of who the analyst is, what the
results are. The government has everything I have.
THE COURT: OK. All right. Anything else?
MR. MCKAY: Not from the government, your Honor.
THE COURT: All right. Nothing from the defense?
MR. ROSANIA: No.
MR. STRAZZA: No.
THE COURT: We will stand adjourned. Thank you very
much.
(Adjourned)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
283
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
INDEX OF EXAMINATION
Examination of: Page
CRAIG O'CONNOR
4Direct Q. . . . . . . . . . . . . . . . . . . .
ADELE MITCHELL
84Direct Mr. McKay . . . . . . . . . . . . . . . .
187Cross By Mr. Strazza . . . . . . . . . . . . .
268Redirect By Mr. Mckay . . . . . . . . . . . .
274Recross By Mr. Strazza . . . . . . . . . . . .
275Redirect By Mr. Mckay . . . . . . . . . . . .
GOVERNMENT EXHIBITS
Exhibit No. Received
101 7 . . . . . . . . . . . . . . . . . . . . .
102 15 . . . . . . . . . . . . . . . . . . . . .
100 16 . . . . . . . . . . . . . . . . . . . . .
103 27 . . . . . . . . . . . . . . . . . . . . .
5 29 . . . . . . . . . . . . . . . . . . . . . .
104 32 . . . . . . . . . . . . . . . . . . . . .
6 37 . . . . . . . . . . . . . . . . . . . . . .
106 41 . . . . . . . . . . . . . . . . . . . . .
108 43 . . . . . . . . . . . . . . . . . . . . .
107 44 . . . . . . . . . . . . . . . . . . . . .
105A 51 . . . . . . . . . . . . . . . . . . . .
105B 57 . . . . . . . . . . . . . . . . . . . .
4 65 . . . . . . . . . . . . . . . . . . . . . .
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
284
SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300
1 90 . . . . . . . . . . . . . . . . . . . . . .
2 91 . . . . . . . . . . . . . . . . . . . . . .
3 93 . . . . . . . . . . . . . . . . . . . . . .
0 101 . . . . . . . . . . . . . . . . . . . . .
15 139 . . . . . . . . . . . . . . . . . . . .
7 153 . . . . . . . . . . . . . . . . . . . . .
8A, 8B, 8C, and 8D 161 . . . . . . . . . . . .
9 162 . . . . . . . . . . . . . . . . . . . . .
10 163 . . . . . . . . . . . . . . . . . . . .
11 164 . . . . . . . . . . . . . . . . . . . .
12 165 . . . . . . . . . . . . . . . . . . . .
16 178 . . . . . . . . . . . . . . . . . . . .
14 182 . . . . . . . . . . . . . . . . . . . .
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Top Related