84 short one so we ensure that the doctor gets -- 2 …...2019/03/11 · 84 SOUTHERN DISTRICT...

84

SOUTHERN DISTRICT REPORTERS, P.C.(212) 805-0300

HB6HJON3 Mitchell - Direct

short one so we ensure that the doctor gets --

THE COURT: Absolutely. Whenever you reach a breaking

point, just let me know, and we can take a lunch break at that

point.

MR. McKAY: The government calls Dr. Adele Mitchell.

THE COURT: Dr. Mitchell, if you could step up,

please.

ADELE MITCHELL,

called as a witness by the Government,

having been duly sworn, testified as follows:

DIRECT EXAMINATION

MR. McKAY: Thank you, your Honor.

Q. Dr. Mitchell, could you please describe your educational

background.

A. Yes. I have a bachelor's degree in math. Earned that in

1991. And then I earned a master's degree in genetic

epidemiology and human genetics. Genetic epidemiology is a

statistical genetics field. That was earned in the year 2000

from the Johns Hopkins University School of Public Health. And

then I earned a Ph.D. in human genetics and molecular biology

in 2004 from the Johns Hopkins University School of Medicine.

That was followed by a two-year postdoctoral period at the

University of Washington in the departments of statistics and

genome sciences.

Q. Can you please describe your coursework and the subject of

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

85

SOUTHERN DISTRICT REPORTERS, P.C. (212) 805-0300


your dissertation.

A. My Ph.D. coursework was within the school of medicine, and

it included about two thirds of the first two years of the M.D.

coursework, so a lot of human biology. And in addition to

that, there were courses specific to genetics, including human

genetics, model animal genetics, population genetics, and

statistics.

Q. What was the subject of your Ph.D. dissertation?

A. I was looking at uncertainty in genotype data and how that

affects downstream analyses. So, for example, if genotypes are

incorrectly called and then a subsequent association testing is

performed to try to look for variance that are associated with

a disease, the errors in the initial part can -- unbiased

errors in the initial part can in fact introduce a bias into

the association results. So I was modeling that and showing

how that can happen.

Q. What was the subject of your focus of your postdoctoral

work?

A. As a postdoctoral fellow, I developed a method that would

simultaneously estimate the genotyping error rate and the

association. So it was modeling the error so that we could

look for the association with disease without the bias that

could be introduced by the error.

Q. Have you received grant funding for your research in the

past?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

86



A. I have.

Q. Where do you work now?

A. I work for a small pharmaceutical company called Eisai,

spelled E-i-s-a-i. The company is based in Japan, but I'm in

Boston at a research site there.

Q. How long have you worked there?

A. I've worked there since February 2017.

Q. What did you do immediately before that?

A. Before that I was at Merck research labs also in Boston

from March 2014 until I started at Eisai.

Q. Did you ever work for the New York City Office of the Chief

Medical Examiner?

A. I did.

Q. When did you work there?

A. I worked there from fall of 2008 through March 2014.

Q. What were your duties and responsibilities at the OCME?

A. I led a research group there. We were focused on improving

forensic methods, both in the laboratory and analytical

methods.

Q. Is one of the things that you worked on the development of

something called the Forensic Statistical Tool?

A. Yes.

Q. Did you develop other techniques as well?

A. Yes.

Q. Just generally speaking, what were some of the other things

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

87



you worked on?

A. We worked on improving yield of DNA from a forensic sample,

improving storage, improving the profiles that we could get

from mixtures. We worked on single cell analysis, a variety of

techniques to improve forensic -- the information obtained from

forensic evidence.

Q. Did you lead any trainings while you were at OCME?

A. I did. I trained analysts in population genetics and

forensic statistics, and then once the FST was developed, I did

the training of all of the analysts in the methods behind the

technique and how to apply them to casework.

Q. Have you held any teaching positions in your life?

A. I have. I started out my career as a high school math and

physics teacher. That was from 1992 to 1999. For two years

before I worked at OCME, I was faculty at Mount Sinai School of

Medicine, and I continued on a part-time basis for another two

years after I started at OCME. I've also taught in workshops

and seminars nationally and internationally on forensic

statistics and likelihood ratios.

Q. During your time at OCME, did you win any awards?

A. I did.

Q. Without embarrassing you too much, could you tell us about

that award.

A. I won an award called the Fred Hayes Prize. It's awarded

annually to city employees who have performed in an outstanding

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

88



way in some area, and it's for all city departments. So this

was awarded to me for my work on FST.

Q. Have you testified previously about DNA?

A. I have.

Q. About how many times?

A. Three times.

Q. Were you qualified as an expert witness when you testified

previously?

A. I was.

Q. Have you ever consulted -- three times you've testified,

were you testifying for the prosecution or the defense?

A. So there was one hearing in Brooklyn where I was called by

prosecution and then I was recalled by defense in that case,

and then the other two cases, I was prosecution witness.

Q. Have you ever consulted on DNA as a defense expert witness?

A. I have.

Q. Just generally speaking, what was the nature of your work

as a defense expert?

A. I'm occasionally contacted to review a DNA case for a

defense attorney, and so it might involve examining the DNA

results and giving my opinion on the methods that were used and

the conclusions that were drawn from those results.

Q. Was there one particular case in Washington State of

interest to you?

A. There was. I worked for defense on a case where the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

89



Washington State DNA lab had produced a report that on its face

looked like it was informative in the case, and so I got the

case file and went through it and determined that the sample

was actually not -- not of high enough quality to draw any

conclusions from. So I worked with the defense attorney on

that, and we then consulted with the prosecutor on that who

ended up not using the DNA evidence once he learned of its

condition.

Q. In the DNA sample analysis in that case, does OCME generate

similar analysis of similar samples?

A. No. This sample had results at only two of the STR loci.

So OCME would never draw a conclusion from a sample like that.

Q. If you turn in that very large binder in front of you to

the beginning to the tab Government Exhibit 1, would you look

at that and see if you recognize it.

Mr. DeLuca, could you put it on the screen for her as

well.

THE COURT: Dr. Mitchell, was the work you just

described for the defendant in Washington State, was that the

work that you've done as a consultant?

THE WITNESS: Yes.

THE COURT: All right. I'm sorry. Go ahead.

Q. Do you see Exhibit 1 there?

A. I do.

Q. What is that?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

90



A. That is my CV.

Q. Does it accurately describe your education and

qualifications?

A. Yes.

MR. McKAY: Government offers Exhibit 1.

MR. ROSANIA: No objection.

THE COURT: Government Exhibit 1 is admitted in

evidence.

(Government's Exhibit 1 received in evidence)

BY MR. McKAY:

Q. Looking at pages 2 and 3, there's a long list there. What

is that list?

A. These are peer-reviewed publications on which I am an

author.

Q. Without going through every one, can you just summarize

some of the general topics about which you published.

A. Yes. So the earlier publications are -- is work that I did

as a student, the first four there. The numbers two, three,

and four are the work that I described earlier, looking at what

are the effects of undetected genotyping error on statistical

analyses. Then there are some disease-specific analyses that I

contributed to. And there's one on the top of the second page

where this was a collaboration with the San Diego Zoo. They

had a small set of endangered iguanas, and they wanted to set

up a breeding program. So I advised them on how best to pair

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

91



up the animals based on their genetics. Then following that

are some forensic publications and a few more disease-specific

publications in there.

Q. If you turn in your binder to Government Exhibit 2, can you

tell me if you recognize that document.

A. Yes. This is a paper describing application of the

Forensic Statistical Tool to casework samples.

Q. Who wrote that paper?

A. I'm the first author on that, which indicates that I did

the bulk of the work and I wrote up the manuscript.


MR. ROSANIA: No objection.


evidence.


BY MR. McKAY:

Q. Where is this article published? What publication?

A. A journal called Forensic Science International: Genetics.

Q. Is that a peer-reviewed publication?

A. It is.

Q. What does it mean to be peer reviewed?

A. When a scientist is interested in publishing some results,

they write a manuscript and send it to the editor of a journal

of interest. That editor will then do a preliminary evaluation

to determine if that may be appropriate for their journal. If

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

92



it may be appropriate, they send it to usually two experts in

the field who do quite a thorough evaluation of the methods and

the conclusions. Those experts can recommend that the

publication be admitted or accepted or they can ask for some

additional work or they can reject a manuscript. Generally, if

the two reviewers agree, the editor goes with their

recommendation. Sometimes if they disagree, the editor might

send it to a third expert in order to get a third opinion.

Q. Have you ever served as a peer reviewer for any

publications?

A. I have.

Q. What publications?

A. I've reviewed articles for several journals. One of the

main ones is this one Forensic Science International: Genetics.

But outside of forensics, I've also reviewed articles for

journals called Genetic Epidemiology, Heredity, Genetics, and

then one on statistical applications in molecular biology.

Q. This article, Exhibit 2, this was published after being

researched by individuals with expert advertise in the subject

matter?

A. Yes.

Q. If you'd turn in your binder to Exhibit 3, do you recognize

that article?

A. Yes.

Q. What is that?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

93



A. This is an article describing the validation of FST.


MR. STRAZZA: No objection.

THE COURT: Exhibit 3's admitted in evidence.


BY MR. McKAY:

Q. What this publication is this?

A. This is also in Forensic Science International: Genetics.

Q. Are you familiar with the Organization of the Scientific

Area Committees?

A. Yes.

Q. Are you a member?

A. I'm not a member of the organization, but I am a member of

one of the subcommittees of that group.

Q. What subcommittee are you on?

A. I'm on a subcommittee that was tasked with developing

validation guidelines for probabilistic genotyping methods.

Q. And to whom do those guidelines apply?

A. The guidelines apply to forensics laboratories in the U.S.

This group is organized by NIST, the National Institute of

Standards and Technology, and they have taken on the task to

develop guidelines in several different forensics areas, and

one of them is probabilistic genotyping. So several people in

the field were recruited to provide guidance and to develop

these methods for probabilistic genotyping.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

94



Q. Were you one of those persons recruited to provide

guidance?

A. Yes.

Q. Was this after you had done and published about the FST

validation?

A. This was after. I believe it started in 2014.

MR. McKAY: Your Honor, at this time I'd offer

Dr. Mitchell as an expert in human genetics, molecular biology,

forensic science, and the statistical analysis thereof.

THE COURT: OK. Any objection?

MR. STRAZZA: I would like a brief voir dire, but I

just want to write down what the government just said.

THE COURT: All right.

MR. STRAZZA: Can you say that again.

MR. McKAY: Human genetics, molecular biology,

forensic science, and the statistical analysis thereof.

THE COURT: OK.

MR. STRAZZA: May I?

THE COURT: Yes.

VOIR DIRE EXAMINATION

Q. Good afternoon, Dr. Mitchell.

A. Good afternoon.

Q. You are a forensic research scientist; is that correct?

A. I have been in the past. At this point my work is in a

pharmaceutical setting.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

95



Q. When you worked for OCME, you were a forensic research

scientist; right?

A. Yes, yes.

Q. And that's to be distinguished from a casework analyst;

right?

A. Yes, that's right.

Q. And, in fact, you never worked on actual forensic casework

in your career; right?

A. That's true.

Q. And you were specifically hired by OCME to develop the FST

program which is the subject of this hearing; right?

A. Yes, yes.

Q. And you were hired to perform the validation in that

instance; correct?

A. Yes.

Q. You had no experience with forensics prior to your work at

OCME; right?

A. Right.

Q. And I think you said this, but you don't work on forensics

now; right?

A. Correct, although I occasionally consult on forensic cases.

Q. You mentioned that you had testified in Brooklyn as a

defense witness; right?

A. I was originally called by the prosecution and then defense

recalled me.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

96



Q. Right. So I just want to be clear. I want to make sure

there's no misunderstandings.

A. Yes.

Q. The defense's position in that case was that FST was

unreliable; right?

A. I'm not sure if I would exactly characterize it that way,

but yes.

Q. How would you characterize the defense position in that

case?

MR. McKAY: Objection.

THE COURT: How does this go again --

MR. STRAZZA: Because one may think that she testified

on behalf of the defense in that case as a defense witness to

support -- it's misleading, Judge, and I just wanted to get out

what really happened in that case.

THE COURT: Why is that relevant to voir dire? In

other words, I'm just trying to figure out -- I understand the

nature of the question, but I don't understand the nature of

the question in the context of voir dire.

MR. STRAZZA: OK. I'll withdraw and revisit on cross.

THE COURT: OK.

MR. STRAZZA: We have no objection to Dr. Mitchell

being admitted as an expert in human genetics, molecular

biology, and forensic science with the understanding that's as

a forensic science researcher, not casework analyst.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

97



THE COURT: That's fine.

MR. STRAZZA: Thanks.

THE COURT: I understand.

All right. Mr. McKay, you may continue.

DIRECT EXAMINATION (Cont'd)

BY MR. McKAY:

Q. Dr. Mitchell, you mentioned this, but during the course of

your time at OCME, did you develop something called the

Forensic Statistical Tool, or FST?

A. I did.

Q. We'll talk in great detail about FST in a minute, but can

you give us to start just an overview of what FST does.

A. Yes. It is a method for putting a numerical weight on a

comparison between an individual's DNA profile and a forensic

mixture, a mixture of DNA from two or more people.

Q. Does FST tell you whether the DNA mixture contains the

defendant's DNA?

A. No.

Q. What does it express? What is the nature of the report?

A. It's a way of determining whether a mixture can be better

explained with a defendant included in the mixture or could it

be explained better if that defendant is not a contributor but

instead a randomly selected person from the population as a

contributor. So it compares two scenarios, one in which a

defendant is part of the mixture and one in which they're not,

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

98



and it asks which of these is better supported by the

evidence --

Q. What is?

A. -- and it quantifies the degree of support.

Q. When you say "quantifies the degree of support," is that a

measure of the strength of the evidence?

A. Yes. So, specifically, what it's calculating is the

probability of getting the evidence if the defendant is a

contributor compared to the probability of getting the evidence

if that defendant is not a contributor.

Q. Why did you want to develop FST?

A. It's very important to me that DNA evidence be

appropriately weighted. DNA plays a special role in forensics,

and I want to be sure that it's not given too much weight or

too little weight.

Q. Before you developed FST, do you have a sense of how OCME

analysts would have reported the findings when they analyzed a

DNA mixture?

A. Yes. In certain circumstances, they could calculate a

statistic called the combined probability of inclusion, or the

CPI. And that would be applied in a situation where drop-out

and drop-in were not suspected. Meaning, looking at a mixture

without looking at a defendant's profile, looking at a mixture

and asking, are there characteristics of this mixture that

might indicate that some alleles could have dropped out? So

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

99



things that can indicate that would be a small amount of DNA to

begin with, a lot of peaks below the threshold where alleles

are called, things like that. So if those things are not

happening and a defendant's profile could be completely

included in the mixture, so at each locus tested there are

alleles that could have come from the defendant, then this

calculation can be done. And so the result of that would be to

say this person cannot be excluded as a contributor and

X percent of the population also could not be excluded as a

contributor.

But that can only be used in those limited

circumstances. And if there's a situation where alleles that

could have come from the defendant show up at every locus

except for one, there was no way to put a number on that

result. So prior to using FST, OCME analysts and other

forensics labs as well -- it's not limited to OCME -- would

have used kind of a descriptive conclusion like this person

cannot be excluded, but there would be no sense of the weight,

who else could not be excluded. Or if a couple of alleles were

missing, they might say no conclusions can be drawn about this

comparison. Finally, if several alleles were missing, the

analyst would say this person is excluded as a contributor. So

it was this kind of levels of description about the comparison

but no numbers.

Q. How would you characterize the range within each of those

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

100



levels?

A. Those are quite a big range -- and I'll show some plots

when we look at our validation -- if comparison profiles are

categorized as either fully included in the evidence or cannot

be excluded or no conclusions or excluded. So each of those is

a pretty big range of likelihood ratios.

Q. Now, you said that FST allows you to put a number on what

previously was qualitatively described; is that fair to say?

A. Yes, that's true.

Q. Does that number that FST reports, does that purport to be

exact?

A. No, it's an estimate. And we have tried our hardest to

make sure it's not an overestimate.

Q. We'll come back to the concept of under or overestimate

later on, but first I'd like you to take a look at your screen.

Mr. DeLuca, would you pull up Government Exhibit 0.

Do you recognize that, Dr. Mitchell?

A. I do.

Q. What is it?

A. This is a presentation that I prepared describing what

underlies FST and how we validated it, how we developed and

validated it at OCME.

Q. Will this help you illustrate some of the concepts you'll

talk about today?

A. Yes, it will.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

101






evidence.


MR. McKAY: Turning to the next slide, Mr. DeLuca.

Q. What is a likelihood ratio, generally speaking?

A. A likelihood ratio is a ratio of two probabilities. The

first one is the probability of getting the data that you're

interested in, whether forensics or something else, if a

particular scenario is true. And the second probability is the

probability of getting those same results if some different

mutually exclusive scenario is true. If the likelihood ratio

turns out to be greater than 1, that's indicating that the data

are supporting the first scenario over the second, and if it's

less than 1, that would indicate that the data support the

second scenario over the first.

Q. Now, are likelihood ratios used just in forensic DNA

analysis or in other fields as well?

A. They are used in other fields, numerous other fields.

Q. Can you give an example.

A. Yes. On the next slide is an example of two competing

scenarios in a medical situation. Maybe the patient has

cancer; maybe the patient doesn't have cancer. So a test is

performed, and the question that is often analyzed in this

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

102



situation is what's the probability of obtaining these test

results if the patient has cancer versus obtaining the test

results if the patient does not have cancer?

Q. How is the comparison commonly described in the forensic

setting?

A. In the forensic setting, it's usually a comparison of

looking at a mixture and asking, what's the probability of

getting this mixture if the suspect or someone else of interest

is a contributor to the mixture versus what's the probability

of getting this mixture if this person is not a contributor and

instead an unknown, unrelated person is a contributor?

Q. How would that likelihood ratio be expressed in that

setting?

A. So on the next slide, scenario one is often called the

prosecution scenario. So it often includes a defendant.

Scenario two is often called the defense scenario where that

defendant is replaced by an unknown, unrelated person. If the

likelihood ratio is greater than 1, then the genetic data are

supporting the prosecution scenario over the defense. And if

it's less than 1, the data are supporting the defense scenario

over that of the prosecution.

Q. What are Bayesian statistics?

A. So the likelihood ratio is a component of Bayesian

statistics. And these were first developed, this way of

looking at data was first developed in the 1700s by Thomas

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

103



Bayes, Reverend Thomas Bayes. And between then and now

especially with the development of computing capabilities,

Bayesian statistics have been applied to pretty much every area

of science and beyond.

Q. And are likelihood ratios in the forensic setting, is that

a new concept?

A. No, it's not. Likelihood ratios have been used for

forensic DNA analysis since DNA analysis was introduced in a

forensic setting. The random match probability is a commonly

reported statistic for a single-source sample, and that is a

likelihood ratio. Likelihood ratios are used for kinship

calculations in a kind of a setting where we would say, what's

the probability of these two people sharing this many alleles

if they are related versus if they are not related? So that

would be a kinship setting. It was extensively used during the

World Trade Center human remains identification process because

it's a standard calculation for that. There is a software

package on the FBI website, or at least it was at the time we

were developing FST. I haven't revisited it in a couple of

years. I don't know if it's still there. But there were

methods for performing a likelihood ratio mixture analysis on

the FBI website.

THE COURT: Mr. McKay, let me ask this: The random

patch probability, was the other DNA test results that we have

in this case, do you recall, was that a random match

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

104



probability?

MR. McKAY: I do, but I don't think Dr. Mitchell has

specific --

THE COURT: I'm just asking you, actually.

MR. McKAY: So my understanding is that the other

sample, the hat, is a two-person deducible mixture --

THE COURT: OK.

MR. McKAY: -- such that you could identify the major

contributor and then apply random match probability.

THE COURT: Got it. Thank you.

BY MR. McKAY:

Q. I'll ask you, Dr. Mitchell, if you had a two-person

deducible mixture hypothetically in which you could identify

the major contributor, could you apply random match probability

to that scenario?

A. You could.

Q. You would not run FST as to the major contributor in that

scenario?

A. That's correct.

Q. Now, you mentioned earlier the CPI. Over the course of the

last -- as an alternative to the likelihood ratio method of

analyzing DNA mixtures; is that right?

A. I'm sorry. Could you repeat that.

Q. I botched that one.

Is the CPI an alternative to the likelihood ratio

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

105



method of analyzing complex mixtures?

A. In limited circumstances, yes.

Q. Over the course of the last decade, has there been any

trend regarding the use of CPI versus the use of likelihood

ratios?

A. Yes. CPI used to be a very commonly applied statistic, and

the trend is changing. In the last several years, more labs

are moving to a likelihood ratio framework.

Q. Is FST the only forensic analysis tool that uses likelihood

ratios?

A. No, there are many.

Q. Do you have some examples of other programs?

A. So there are a few others that are very similar to FST.

Examples of those would be Lab Retriever, LRmix, likeLTD, there

are a couple of others that I can't recall offhand. So those

work just like FST where they allow for alleles to drop out or

to drop in, and I can define that a little bit better in a

minute. And they have different ways of estimating the

probability that an allele might drop out or drop in. Then

there are some more complex programs that would be called

probabilitistic genotyping methods where the characteristics of

the electropherograms are used by a computer program to try to

sort out the individual profiles of the contributors. So FST

isn't doing that. It's taking the alleles that are called by

the standard genotyping software and doing its calculation just

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

106



using those, but it's not doing the separation of the

individual profiles like these other programs, these two other

programs can do.

Q. Is it fair to say that FST or programs sharing a common

approach to FST are in use by forensic laboratories worldwide?

A. Yes.

Q. I'd like to talk about the development of FST. Was it all

done in one phase or was it more than one phase?

A. Well, there were two phases to the project. The first

phase was the development of the program and the estimation of

the parameters that we were interested in, and then the second

phase was a validation or testing of the program.

Q. Let's start with the first phase. But first can you

identify the term "drop-out"?

A. Drop-out is when is somebody really is a contributor to a

sample, but yet one or more of their alleles are missing from

that sample.

Q. What is drop-in?

A. Drop-in is when there's an allele that is labeled in a

mixture that does not come from any of the true contributors to

a mixture. So that can happen -- there seems to be bits of DNA

everywhere. So an allele can be labeled even if it's not part

of a mixture made up of known individuals, and we've seen that

in mixtures that we've made using cheek swabs and blood. We

would see, if it's a high template sample, the entire profiles

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

107



of the contributors, but sometimes there will be an extraneous

allele that we don't know where it came from.

Q. That's drop-in?

A. Yes.

Q. What are some of the reasons you might see drop-out?

A. If there is not a lot of DNA in a sample, sometimes some

parts don't amplify very well, and they don't show up. Some of

the forensic loci amplify better than others. So there are

somewhere drop-out is less likely and others where it's more

likely. And it's -- at a certain level it's a stochastic

process. So sometimes one just doesn't show up by chance.

Q. Does FST attempt to account for the phenomena of drop-in

and drop-out?

A. What we did was estimate the rate at which it happens,

and -- is that what you mean by account for it?

Q. Well, you mentioned CPI.

A. Yes.

Q. Does CPI account for the possibility of drop-in and

drop-out?

A. No, it cannot be modeled in that analysis.

Q. Does FST have as a component a consideration of drop-in and

drop-out?

A. Yes, it does.

Q. Why does it incorporate that function?

A. Because that's the nature of the samples that we see in

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

108



forensics. If we have a single-source blood sample, and it's

amplified. It's very easy to analyze. But evidence comes in

all forms, from touched items to mixed saliva and blood, all

kinds of combinations. And we don't always see the entire

profile of people who contributed to the sample, and we've seen

that in mock casework samples that we've made in the lab.

Q. So drop-out actually happens?

A. Yes.

Q. When you're testing DNA from a piece of evidence, is it

possible to know what the true drop-out rate is?

A. No.

Q. So what does FST do, then, to try to estimate drop-out

rates? How do you determine estimated drop-out rates?

A. We made a large number of purposeful mixtures of DNA from

two contributors and three contributors. We used about ten

different people in various combinations to set up the

two-person mixtures and the three-person mixtures. And then we

started with different amounts of template DNA, from very small

amounts to very large am amounts, and we looked and we counted

how often did those contributors' alleles not show up. And we

looked at the different loci that are amplified for a forensic

comparison and compared those -- those loci. So some loci had

more drop-out than others and some had less.

Q. So when you make a sample of known contributors, you know

whose DNA is going into the mixture; right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

109



A. Yes.

Q. So do you know what alleles you would expect to see in the

mixture?

A. Yes.

Q. In your test results, did you always see exactly the same

alleles that you would have expected to see?

A. No.

Q. Now --

A. Sometimes a true contributor's allele or alleles would not

show up, and sometimes there would be extraneous alleles that

did not come from the people whose DNA we had put into the

sample.

Q. So that is an illustration of drop-in and drop-out; is that

right?

A. Yes.

Q. How long did that process take, estimating the drop-out

rates?

A. It was about a year to do the drop-out rate estimation and

to write the program.

Q. During that study, what factors did you see affecting

drop-out rates?

A. If you go to the next slide, we can look. The biggest

predictor of the probability of drop-out is the amount of DNA

in the sample, the starting amount of DNA. So we looked at

samples in OCME's low template range, which is 25 to 100

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

110



picograms, and then we looked in the high template range, which

is 100 to 500 picograms. And so the amount of DNA is a very

important predictor of the probability of an allele dropping

out.

The next slide, other predictors are the number of

contributors to the mixture. We looked at two-person,

three-person, and four-person. OCME eventually decided not to

implement the four-person mixtures in casework. So the number

of contributors was important and the relative contribution

from each contributor. If there are major and minor

contributors, the minor contributors are intuitively more

likely to drop out than the major contributors' alleles. So

here are some examples. The first one on the top would be one

locus at -- in a mixture that cannot be separated. So these

peaks are all about the same height. It's not possible to pair

them up based on their height.

In the second picture, the 9 and the 12 alleles are

much taller than the 11 alleles. So this is an example where

those two alleles could probably be paired together. Those

probably came from the major contributor to that mixture, and

the 11 came from the minor contributor, and we don't know what

that person's second allele is. It could be another 11 or it

could be a 9 or a 12 or it could have dropped out.

Q. Now, a moment ago you talked about how the amount of DNA

was one of the most important factors.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

111



A. Yes.

Q. Is that sometimes called quantitation or quant?

A. It is.

Q. Generally speaking, did you see that drop rates were higher

or lower as you increased the quant?

A. As the quant increases, the probability of drop-out

decreases.

Q. So in the rates that you ultimately chose to use in FST,

was quant one of the factors that went into setting the rates?

A. Yes.

Q. Did you consider any alternatives for quant?

A. We did. Early on we looked at the peak heights in the

electropherograms. So using on this slide other -- some other

groups have estimated the probability of drop-out based on the

height of the peaks that are labeled in the mixture.

Q. Why did you choose to go with quant over peak heights

ultimately?

A. OCME has a large number of electrophoresis instruments, and

we found that the peak height was less useful in predicting the

probability of drop-out across all of the instruments than the

starting amount of DNA. So we tried looking at peak height to

see if we could get a good handle on the probability of

drop-out, and it was not as predictive of the drop-out

probability as the starting amount of DNA.

THE COURT: Just to be clear, is that because of the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

jbucklet

Highlight

112



different machines that were used?

THE WITNESS: The machines were different, and we just

found that, in our hands, the quant was the better method. And

other labs may make a different decision and they might have a

different experience with peak heights versus quant in their

lab, but at OCME we found that the quant was a better

predictor.

THE COURT: OK. Go ahead.

Q. In the decision to use quant as one of the applicable

factors, was that described in the peer-reviewed publication

you wrote about the FST program?

A. Yes.

Q. And was that decision presented to the DNA subcommittee of

New York that ultimately had the sign-off to approve FST?

A. Yes.

Q. Now, based on what you described, did you determine some

empirical drop-out rates?

A. We did.

Q. Were any adjustments then made to those rates?

A. Yes. What we ended up using in the program was not the

estimates themselves but one standard deviation below the

estimate.

(Continued on next page)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

113


HB67JON4 Mitchell, direct

BY MR. MCKAY:

Q. And why did you choose to reduce or adjust it by one

standard condition?

A. So, the next slide shows a picture of the main motivation

for doing this.

So, I have two plots here, and these are both showing

likelihood ratios that were obtained from noncontributors to

mixtures, so people who were not part of the mock casework

samples that we made. We tested FST using people who really

did contribute to the mixtures and a set of people who really

didn't. So, this is -- there is a database of about 1250

noncontributors, and we ran FST using each of those people as

the suspect, if you will.

So the plot on the left shows the distribution of

likelihood ratios that we got when we were using the lower of

two possible drop-out rates. And the average there you can see

the peak there is about minus 18. That represents a likelihood

ratio of ten to the minus 18, so very, very small. So that's

good, we want noncontributors to generate very low likelihood

ratios so that they are excluded from the mixture.

On the right is a plot using a slightly higher

drop-out rate or slightly higher drop-out rates, and that

shifts the entire distribution up. So in that second plot the

average is now about minus 12. So using those higher drop-out

rates has shifted the distribution of likelihood ratios for

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

jbucklet

Highlight

114



noncontributors a bit higher. So this is showing that if lower

drop-out probabilities are used, on average LRs will be

obtained for noncontributors. And that was really important

that we keep the likelihood ratios low for noncontributors so

we're not falsely including people. So, our main motivation in

underestimating the drop-out probabilities was to keep LRs for

noncontributors low.

Q. If I may just ask a few follow-up questions on that.

A. OK.

Q. First of all you used the term underestimating the drop-out

rate. When you say underestimating, what do you mean?

A. So that phrase is really only meaningful in the context of

a true contributor. So if we have a true contributor to a

mixture, we can look at how often that person's alleles drop

out, so that would be their true drop-out rate. And on the

next slide I discuss a side effect of our decision to use lower

drop-out rates in FST.

So, we were aiming to reduce the LR for

noncontributors, but what can also happen is it may also reduce

the LRs for true contributors, and that's because if we were to

use a true contributor's exact drop-out probability in the

calculation, we would be modeling that situation as well as we

could, so we would maximize the likelihood ratio for that true

contributor.

If we use a lower probability of drop-out for that

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

115



true contributor, the model doesn't fit as well, so it tends to

reduce the likelihood ratio for that true contributor. I will

show you some plots on the next slide where we tested that.

Q. Before we get to that, may I interrupt you for one second.

So underestimate means to have a drop-out rate lower than the

true drop-out rate; is that correct?

A. Correct, for a true contributor.

Q. So that is different from lowering the drop-out rate?

A. Yes, or it's a subset of lowering the drop-out rate.

To test this statement that I'm making saying that

underestimating the drop-out rate tends to lower a likelihood

ratio one must know what is the actual drop-out rate of that

real contributor, and then to compare what do you get if you

use that versus what do you get if you use something lower.

MR. MCKAY: If we could go back to the slide we were

just looking at, Mr. Deluca. I just want to clarify a few

things, if possible.

Q. So the X axis, the horizontal axis, what is the value on

that axis?

A. Those are the exponents on the likelihood ratio. So it's

sometimes also called a log likelihood ratio. So, for example,

the minus 30 means ten to the minus 30 was the likelihood ratio

obtained for those noncontributors.

Q. So ten to the minus 30 would be a very strong exclusion; is

that right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

116



A. Absolutely, yes.

Q. And what is the zero on that X axis?

A. The zero represents a likelihood ratio of one. So that's

the point where the DNA evidence is not supporting one scenario

over another.

So, we were happy to see that this turned out to be at

the far right of the distribution. There are very few

observations to the right of zero. So, that is saying that

very few noncontributors in our validation generated a

likelihood ratio of greater than one. The vast, vast majority

are far less than one and in fact down to ten to the minus 40.

Q. So results that are left of the zero are noncontributors

for whom the likelihood ratio was less than one.

A. Yes. So those would be exclusionary likelihood ratios.

Q. And those small number of results to the right of the zero

are what you might call false positives?

A. Exactly.

Q. And when you compare the graph on the left to the graph on

the right, what does that tell you?

A. So, the graph on the left was the plot generated using the

drop-out probabilities that we use for mixtures that are

approximately one to one. So, these are lower drop-out

probabilities than the ones on the right. The ones on the

right are the drop-out probabilities used for the minor

contributor to an unequal mixture.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

117



So the ones on the left are lower, the ones on the

right are higher, the drop-out rates; and using higher drop-out

rates is shifting the noncontributor curve to the right. But

still at this point there are still very few observations above

zero, or very few likelihood ratios above one.

Q. And I think you touched on this, but the goal of

underestimating is what with respect to noncontributing

findings?

A. The goal of underestimating is to keep LRs for

noncontributors as low as possible while still being able to

generate higher likelihood ratios for true contributors. So

it's kind of a balance. If we're exclusively focusing on the

noncontributors -- we have to focus on both noncontributors and

contributors. We're trying to separate the two.

Q. To be fair, a noncontributor is someone whose DNA evidence

is not in the mixture.

A. Right, it's somebody who did not contribute any DNA to the

forensic mixture.

Q. So why is your primary concern lowering LRs for

noncontributing defendants?

A. Because these people did not leave any DNA on this sample,

whatever it is. They are not a contributor to the sample, so

they're not -- the evidence is not supporting their inclusion,

and we want to make sure that the likelihood ratio reflects

that as much as possible.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

118



Q. Does that underestimation come with a cost or side effect

with respect to true contributors?

A. It does. It can reduce the LR for a true contributor. So

maybe somebody who really did contribute to the mixture may not

generate as high a likelihood ratio as they would if we used a

higher drop-out probability, but it's a price we're not willing

to pay.

Q. And why are you not willing to --

A. I guess I didn't phrase that very well. We're not willing

to use the higher drop-out estimates because although that

would raise the LR of true contributors, it would also raise

the LR of noncontributors. So we're willing to take a little

bit of a hit on the true contributors, reducing their LR a

little bit, to try to guard against falsely including someone.

Q. Now, have you ever tried to test the argument that

underestimating drop-out lowers or reduces LRs for true

contributors?

A. Yes, that's on the next slide. We took 19 of our

validation samples, some two person mixtures, some three person

mixtures, and for each true contributor to those samples we

calculated their true drop-out rates, and we did the FST

calculation using those true drop-out rates. We then cut those

true drop-out rates in half and repeated the analysis so that

we could compare what do we get if we exactly get the drop-out

probability right, and what do we get if we underestimated the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

119



drop-out probability. So, these two graphs show the results

from that.

Q. Let me ask you first when did you do that research?

A. This was done -- this is a master's student's work that she

was using the validation samples, so they had already been

generated. I don't know exactly when she was doing this with

regard to the completed validation.

Q. Now, focusing first on the plot on the left, what do we

have on the X axis?

A. So, on the X axis or the horizontal axis, these are log

likelihood ratios, so this is the exponent on the likelihood

ratio that we obtained when we used the real contributor's real

drop-out rates.

So, for example, those two red dots on the left, those

are near minus three. That means for those comparisons, people

that really did contribute to the mixture, when we did this

analysis using their real drop-out rates we got to about ten to

the minus three LR. So there was probably quite a bit of

drop-out for those samples. Even though they were real

contributors, they actually didn't look like it because several

of their alleles had dropped out, so that's why they've

generated a low likelihood ratio.

Over on the right, so if you go up from six, there is

a red dot pretty high up. That person using their true

drop-out probability, a likelihood ratio of ten to the sixth or

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

120



one million was generated.

Q. What is on the Y axis?

A. The Y axis is a measurement of the log likelihood ratio

that we obtained when we used not their true drop-out rate but

half of that, so when we underestimated the drop-out rate what

we got.

So, again to use that top right red dot, the one that

generated a likelihood ratio of about ten to the fifth or

between ten to the fifth and ten to the sixth on the X axis,

they also generated a likelihood ratio of about ten to the

fifth or ten to the sixth using the underestimated drop-out

rates.

Q. And what are the dots?

A. So, each dot represents one true contributor, and it's

calculating the likelihood ratio using their true drop-out rate

and their underestimated drop-out rate.

Q. Why are some blue and some red?

A. The blue ones were samples amplifying OCME's standard

amplification protocol, so that's samples that are at least a

hundred picograms. The red dots that were generated from --

I'm backwards. The red dots are the standard protocol. ID28

means the Identifiler kit with 28 cycles. So the red dots are

the standard protocol. Then the blue dots are the high

sensitivity protocol. So those are the smaller amounts of DNA.

So I included that color coding to look for patterns

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

121



in one protocol versus another. But they both show the same

pattern, which is using these two drop-out rates -- either the

true rate or the underestimated rate -- the likelihood ratio is

either similar or lower when using the underestimated rates.

So, the points falling below the diagonal line are points where

the LR was lower using underestimated drop-out rates instead of

the true drop-out rates.

Q. So what is the significance of a point that is exactly on

the line?

A. A point exactly on the line would have generated a

likelihood ratio exactly the same using both of those drop-out

probabilities.

Q. And a point below the line is what again?

A. A point below the line is a point where underestimating the

drop-out rate gave a lower likelihood ratio than getting the

drop-out rate exactly right.

Q. And a point above the line is what?

A. A point above the line is where underestimating the

drop-out rate gave a higher likelihood ratio than using the

true drop-out probability.

Q. And so generally speaking what did you conclude about

underestimating drop-out rates for true contributors based on

this?

A. Generally we saw that underestimating the drop-out rate led

to a lower likelihood ratio for a true contributor than getting

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

122



it exactly right.

Q. Now, there are some examples where that was not true,

right?

A. Yes.

Q. And those are the dots that are above the line?

A. Yes.

Q. But generally speaking that conclusion held true?

A. Yes.

Q. OK. And what is the plot on the right?

A. The plot on the right is the same analysis but using the

three person samples. The one on the left was using two person

mixtures, and these are three person.

Q. And did you draw the same conclusions with respect to three

person samples?

A. We did. The results were either comparable using the

underestimated rates, or they were lower using the

underestimated rates, again because the points fall either very

close to the line or below the line.

MR. MCKAY: Your Honor, at this point I'm about to

move to a new topic which will take about 20 to 30 minutes.

Would this be a good time for the lunch break?

THE COURT: Sure. How long would you like for the

lunch break? It's now about 1:20. Do you want to come back at

about two?

MR. MCKAY: That works for us, your Honor.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

123



THE COURT: 2 o'clock? OK.

MR. MCKAY: Just timing wise, I'm about halfway

through, so if we came back at two, I would be done at let's

say 3:15 or 3:30.

Does that work on your timing?

MR. STRAZZA: Yes.

THE COURT: In other words, I think what he was trying

to ask is how long you anticipate your cross being.

MR. ROSANIA: I know. We had discussed this earlier,

and obviously that depends upon the answers to the questions,

but I think that generally works.

THE COURT: OK. All right. We'll take our lunch

break. We will stand adjourned until 2 o'clock. Thank you.

You may step down.

(Luncheon recess)


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

124



A F T E R N O O N S E S S I O N

2:00 p.m.

THE COURT: You may continue the direct examination of

Dr. Mitchell.

MR. MCKAY: Thank you, your Honor.

BY MR. MCKAY:

Q. Dr. Mitchell, so are the calculations performed by FST, are

those calculations done by hand or using the computer program?

A. The computer program does it.

Q. And how long would it take to calculate a likelihood ratio

for a complex mixture by hand?

A. It would not be reasonably possible. How long would it

take? I've done it for one locus for a few alleles, and that

can take an hour maybe, but as the number of alleles increases

and the number of loci increase it would not be reasonable to

do it by hand. It's not a difficult calculation; it's just

multiplying and adding and dividing, but just very tedious.

Q. The volume of the number of calculations to be done

requires a computer program.

A. Yes.

Q. So I won't ask you to calculate a full LR, but can we walk

through an example or simplify an example of how the math

works?

A. Yes.

Q. Go ahead.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

125



A. OK. So this is an example. Let's say we have a two person

mixture and this is an intimate sample from the victim, and

there is a suspect profile, and the victim profile is available

as well. The question of interest here would be is the suspect

a contributor to this mixture. So in this first situation that

I will show, the suspect and the victim together could

completely explain the evidence sample. So on the next slide

is a hypothetical four locus profile.

Q. Just one second. In this first example, are you going to

factor in drop-in and drop-out or no?

A. I am not. Firstly show how it's calculated without

modeling drop-out and drop-in.

So, in this evidence sample if you were to put

together the suspect and the victim profiles, you would obtain

the evidence results. And if one is not modeling drop-out and

drop-in, this must be the case in order to compute a likelihood

ratio.

The next slide, please. The prosecution hypothesis in

this example will be that the sample originated from the victim

and the suspect. And the defense hypothesis or defense

scenario will be that the sample originated from the victim and

an unknown unrelated person. So, the likelihood ratio

calculation is shown below. That PR parentheses means

probability. And the vertical bar means given. So this first

fraction here we're going to calculate the probability of the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

126



evidence if the prosecution scenario is true, or given that the

prosecution scenario is true.

We will then calculate the probability of the evidence

given that the defense scenario is true.

So in the numerator we are calculating the probability

of the evidence if the contributors are the victim and the

suspect. And in the denominator we will calculate the

probability of the evidence if the victim and an unknown

unrelated person are the contributors.

Next slide. When we're not accounting or allowing for

drop-out and drop-in, in the numerator the probability of

getting that evidence sample if the true contributors are the

victim and the suspect is one, because it's made up of exactly

what the victim and the suspect have at those loci. So if you

put them together, there is a 100 percent chance that you will

get that evidence mixture if no drop-out and no drop-in are

considered. So the numerator is easy.

The denominator we are going to replace the suspect

with a randomly suspected person and examine the situation that

the sample comes from the victim and an unknown unrelated

person.

Next slide. So, in the denominator, this unknown

contributor would have to account for any alleles in the

mixture that are not explained by the victim. So, the evidence

at the first locus is 14, 15, 16, and the victim has a 15 and a

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

127



16. Therefore, the second contributor, whoever it may be, must

provide a 14 to the mixture. And at the second locus they

would need to provide a 29, as the victim explains the 31, but

the 29 would be unexplained by the victim.

If we go to the next slide, I have listed out all the

possible genotypes of the unknown person. At the first locus,

the evidence contains a 14, 15, 16. The victim is 15, 16. So

the second contributor must have a 14. But their second allele

could be a 14 or a 15 or a 16. So, there are three possible

genotypes for a second contributor at that locus.

At the second locus the victim has a 31 allele and the

evidence has a 29 and a 31. So the unknown contributor could

be homozygous for the 29 allele, or they could have a 29 and a

31.

At the third locus, because there are four distinct

alleles and two of them could be attributed to the victim, the

other two must come from the second contributor. So at that

locus there is only one possibility for the second contributor,

and that's 8 and 11.

On the next slide I have taken each of these genotypes

and estimated population frequencies of those genotypes, so

that's the denominator of the likelihood ratio there.

In that first factor P14 squared is the estimated

population frequency of people with a 14, 14 genotype in the

population. The two times P14, P15 represents the estimated

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

128



frequency of people with a 14, 15 genotype, and the same with

the 14, 16.

Q. If I can stop you for just one second. Where do the

estimated population frequencies of people with any particular

genotype, where do those figures come from?

A. There are several options, more than several options for

estimating these frequencies. OCME developed its own

population database based on post-mortem samples from the City

of New York, and they've estimated allele frequencies for four

separate populations within New York: Caucasian, African

American, Hispanic and Asian individuals.

Q. You can continue. Thank you.

A. OK. So, at this point all that would be necessary would be

to plug in the frequencies of those alleles or those expected

genotype frequencies in each of those factors. The denominator

here -- sorry. I lost my train of thought.

Next slide. So the value of this likelihood ratio is

going to depend on the frequency of the alleles in the mixture

that match the suspect and can't be explained by the victim,

because those are the ones that we would need to draw from the

population. So, we want to know what would be the chance that

we would draw this from the population if in the denominator

when our suspect is not in the model. So if there are alleles

that are rare, that the suspect has that are in the mixture and

not explained by the victim, it would be unlikely that a

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

129



randomly chosen person would carry them, so that ends up

increasing the LR, because it decreases the probability that

this mixture would happen by chance from the random population.

A high likelihood ratio means that a mixture is

explained better with the suspect than without them, and

conversely a low likelihood ratio would mean the mixture is

explained better by a randomly chosen person than by the

suspect.

So in the next slide I have assigned hypothetical

allele frequencies here and just plugged those numbers in down

below, and in this circumstance I circled the alleles that

matched the suspect in this example, and so that using these

fairly common alleles like a 33 percent allele frequency, for

example, for allele 14 at locus 1, we end up with a likelihood

ratio of 87, just using these values.

On the next slide I show the interpretation of that

number. So the 87 means the DNA mixture found on this item is

87 times more probable if it came from the suspect and the

victim than if it came from the victim and an unknown unrelated

person. And then OCME applies a descriptive scale to these

values, and the interpretation there would be that there is

moderate support for this being a mixture of DNA from the

suspect and the victim rather than the victim and an unknown

unrelated person.

The scale is shown on the next slide, so these are the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

130



descriptors that OCME uses for likelihood ratios in the range

of one to ten. That would be limited support for one

hypothesis over the other. Ten to 100 would be moderate

support. 100 to 1,000 is strong support, and greater than

1,000 is very strong support. This scale was suggested by a

couple of publications by solid forensic scientists Bruce Weir

and John -- not John Buckleton. I'm blanking.

Q. Is it John Butler?

A. No, it's not John Butler. It's David Balding is the 2005

publication. The 1998 publication is Bruce Weir, W-e-i-r.

So if a likelihood ratio turns out to be less than

one, the reciprocal is taken, and it's reported in this same

fashion saying the evidence is providing limited support for,

for example, the victim and an unknown unrelated person rather

than the victim and the suspect. So the scale is the same

whichever way the LR ended up going.

The next slide shows what might happen if instead of

having the suspect's alleles be quite common, I have assigned

lower frequencies to those alleles just to show that alleles

that are very rare can change the likelihood ratio.

So, in this example that allele 14 which was carried

by the suspect, it would be unusual to draw that allele from a

random population, so that will increase the support for the

suspect being a contributor over an unknown person being a

contributor.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

131



Q. So using those less common allele frequencies, what was the

result that you got?

A. So this result turned out to be a little over 21 million,

and so on the next slide the interpretation would be the DNA

mixture found on this item is 21.6 million times more probable

if it is a mixture of DNA from the suspect and the victim than

if it is a mixture from the victim and an unknown, unrelated

person.

So in this example we would say that this is very

strong support for this being a mixture of DNA from the suspect

and the victim compared to the victim and an unknown, unrelated

person.

Q. Can you now do a simplified example where we factor in

drop-in and drop-out?

A. Yes. So we did not allow drop-in and drop-out in that

previous example, but now we will, but I will do just one

single allele rather than the four loci with multiple alleles.

So when we start to think about drop-out and drop-in,

we need to allow each contributor's alleles to be missing --

whether these are the known individuals or the unknown

person -- and we need to allow for extra alleles that are not

explained by the contributors in each particular scenario.

So then on the next slide I will show what I will be

modeling. So this will be a single source sample, which is not

something that OCME uses FST to analyze but just for the sake

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

132



of a simple example.

So, in this example the scenario in the numerator will

be that this DNA -- the suspect is the source of this DNA --

and in the denominator an unknown, unrelated person will be the

source of the DNA.

So we will set up the likelihood ratio in the same

way, but for each contributor or potential contributor we will

also need to ask if this person is a contributor to the sample,

did any of their alleles drop out; and also we will need to ask

if this person is the source of the DNA, are there any extra

alleles that we would call drop-in, if that person is the true

source.

So the next slide shows which locus I used. So I'm

going to use locus 3 where there was an 8 allele in the

evidence, and the suspect has an 8 and a 12. So on the next

slide I will show how I set it up.

In the previous example when we did not include

drop-out and drop-in, the numerator was 1, because if you put

those two people's profiles together, you would get the

evidence with the probability of 1.

Here the numerator will be less than one, because it's

not certain that you would get that result in the evidence if

drop-out is allowed. And we're also going to allow unknown

contributors' alleles to drop out.

So, on the next slide in the numerator the suspect is

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

133



an 8, 12 and the evidence is an 8, so the probability of

getting that evidence if the suspect is the source of the DNA

is going to be the probability that one allele from a

heterozygote drops out and the probability that no additional

alleles drop in, because that's what would need to happen if

the true contributor is an 8, 12 and the evidence is an 8.

In the denominator we let an unknown contributor have

a genotype that's made up of any combination of the alleles

seen in the mixture and any other alleles that didn't show up.

So, we will let this person have an 8 and an 8 or an 8 and

something else that we call W, or two something elses, W, W.

So, we will need to find the population frequency for each of

those and then multiply each of those by the probability of

drop-out and/or drop-in that would be required to attain the

evidence sample.

Next. So the numerator is going to be the probability

of one heterozygous drop-out times the probability of no

drop-in. In the denominator we let the unknown person have

genotype 8, 8; 8 something else; or two copies of something

else. So if that person is an 8, 8, we need to know what's the

probability that there is no drop-out from a homozygote, and

multiply that by the probability of no drop-in. Because if the

true source is 8, 8, and we see an 8, there was no drop-out and

there was no drop-in. If the source is 8 something else, one

of those alleles dropped out, so we need to know the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

134



probability of one drop-out from a heterozygote; and again no

drop-in because there are no extra alleles there.

In the final situation, if the true contributor has

two alleles that don't show up in the mixture, we need to know

what's the probability that those alleles don't drop out, but

then we need one drop-in in that example because an 8 shows up

in the mixture.

So here is what it is if we put these in as variables.

So in the numerator we have the probability of one drop-out

from a heterozygote -- that's D1 -- times no drop-in -- that's

the C zero. Then in the bottom those three terms represent an

unknown contributor with genotype 8, 8, where that homozygous

allele did not drop out and no alleles dropped in. And the

second term represents an unknown contributor with genotype 8W,

where one allele dropped out and no alleles dropped in. And

finally they can have two copies of something else, both of

which dropped out, and then the 8 would have to be a drop-in.

So it's not difficult mathematically. It can just

become very tedious to write out all of these possibilities,

especially if there are several unknown contributors in this

scenario.

Q. Now, have you written up a third example on the slides here

as well?

A. I have.

Q. And what does that example show?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

135



A. That would be example 3, the next slide, which I can go

into or I don't need to go into.

Q. Well, let me ask this. What does example 3 show compared

to what we've just seen?

A. So example 3 shows a mixture, a mixture where we're

allowing drop-out and drop-in, and we have in the numerator we

have the suspect and one unknown unrelated person, and in the

denominator we have two unknown unrelated people. So, for each

of those unknown people we would have to list out all of their

possible genotypes, estimate those frequencies and then

consider whether drop-out and/or drop-in would have had to

occur to give us the mixture that we got.

Q. In terms of the calculations that you performed in your

next example, are they the same or different from the

calculations you just did?

A. They are the same; there are just more of them.

MR. MCKAY: Unless the court feels otherwise, I think

in the interest of time we may move on from there.

THE COURT: No, we can move on as long as you don't

ask me to try and do it.

Q. So, let's move on and talk about the second -- you said

earlier there were two phases in the development of FST; is

that right?

A. Yes.

Q. And what was that second phase?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

136



A. The second phase was the validation.

Q. What does it mean -- what does validation mean in this

context?

A. Validation is a procedure performed by any forensics lab

when they are about to start using either a lab protocol or an

analytical protocol that they have not used before. So they

need to demonstrate that it functions as expected in their

hands.

Q. And do labs validate the protocols and procedures of other

labs?

A. No, a lab will only validate something that they themselves

are planning to use.

Q. The committee that you mentioned earlier, the subcommittee

that you were on, remind us what sorts of standards were you

setting on that subcommittee?

A. We were setting up guidelines for how a lab should go about

validating a probablistic genotyping software tool.

Q. So when did the validation for FST begin approximately?

A. It began midyear 2009.

Q. Approximately how long did it take?

A. About a year and a half.

MR. MCKAY: And, Mr. Deluca, if you will bring up the

slide show again.

Q. Would you explain to the court how the validation for FST

was designed and carried out.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

137



A. Yes. So the next slide shows the design. So we generated

mock case work samples of a variety of types. We created

mixtures of blood or saliva -- or DNA isolated from DNA or

saliva from two or three people. We had samples that were

handled by two, three or four people. And we had some samples

that we purposely degraded using UV light. And these samples

were processed according to the OCME case work protocol. That

protocol includes estimating the number of contributors to the

mixture, determining if this is a mixture ratio that is

approximately one-to-one, or if it's not one-to-one and a major

contributor's profile could be determined.

We then compared -- we then calculated a likelihood

ratio using FST for each of the true donors to those samples,

and we also computed FST for each noncontributor in a database

of approximately 1250 noncontributors.

So, for each sample we could tell what do the

likelihood ratios look like for the people who really did

contribute to the sample, and what do they look like for people

who did not contribute to the sample.

Q. I will ask you about the results of those tests in a

second. But first can you talk more about the manner in which

you created the samples?

A. Yes. So, on the next slide there is a description of the

different types of samples that we used.

So, we looked at both two person and three person

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

138



mixtures, and we looked at low template and high template

samples to make sure that we were covering all of those

options. And we ended up with 439 samples that fit the case

work criteria for going forward with analysis. Not all of them

did. Some of them, for example, didn't have enough loci that

amplified, or it turned out to be a single source sample, or

had too many alleles, and it didn't fit our three person

mixture criteria. So there were 439 in the end.

We ran these in FST using the assessments of the

mixtures that were made without knowing who the true

contributors were, or how many there were, or knowing anything

about the sample going into it.

Q. If I can interrupt for a second. How did you decide how

many mixtures to make?

A. Well, we just kept making more and more mixtures until it

was approaching the time that we were going to meet with the

DNA subcommittee and represent the results to them. So it was

essentially as many as we could in order to get into this

meeting where we presented the final validation.

Q. Did the scientific community have any guidelines or

guidance at this time about how many mixtures should be made

for validation?

A. There were some guidelines that were put out by SWGDAM,

which is the scientific working group on DNA analysis methods

that develops guidelines for all kinds of forensic lab

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

139



protocols, and they had recommended using at least 50 mixtures

in a validation study.

Q. If you turn in that big binder in front of you to

Government Exhibit 15, can you tell me whether you recognize

that document.

A. Yes.

Q. What is it?

A. These are the SWGDAM guidelines for validations in forensic

labs that were put out in July 2004.

MR. MCKAY: The government offers Exhibit 15.


THE COURT: OK. Government Exhibit 15 is admitted in

evidence.

(Government Exhibit 15 received in evidence)

Q. And if you turn to the second page and look down to section

3, is that the recommendation that you were referring to?

A. Yes, it is.

Q. And how many samples does SWGDAM recommend using?

A. A total of at least 50 samples.

Q. How many did OCME use?

A. In this validation we used 439 samples.

Q. Now, did many of those samples use some of the same

contributors as other samples?

A. Yes.

Q. Does SWGDAM or any other -- or did SWGDAM or any other

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

140



forensic body provide any guidance on how many different

contributors you should use?

A. No, there is no guidance.

Q. And was the fact that some of the mixtures contained the

same contributors as other mixtures disclose to the DNA

subcommittee that ultimately approved FST?

A. Yes.

Q. And what did the subcommittee say about the number of

samples tested?

A. One of the subcommittee members, John Carmedi, who is a

population geneticist said it was the largest validations he

had ever seen.

Q. Are you familiar with the program Lab Retriever?

A. Yes.

Q. And are you familiar with how Lab Retriever is validated?

A. I saw a presentation at a meeting of the American Academy

of Forensic Sciences in which one of the developers of Lab

Retriever presented a validation that they had performed using

two -- DNA from two donors that they had amplified each 20

times.

Q. So before I interrupted, you were working through the

PowerPoint.

Mr. Deluca, can you bring that up, please.

And I think you might have been on slide 52.

A. OK.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

141



Q. And continue explaining the validation.

A. OK. So the validation mixtures included touched samples --

which often are encountered in case work -- and some

nondegraded mixtures of saliva and blood, and then it also

included some mixtures where one or both of the components were

degraded by UV light.

Some of the mixtures that we used for testing were

mixtures that we had initially developed with the intention of

using them for drop-out rate estimation, but then we did not

use them for drop-out rate estimation, but we used them for

validation, and they contained mixture ratios that were

different than those that were raised to estimate the drop-out

probabilities.

The samples were processed as if they were case work

samples. So DNA was extracted, quantified, amplified,

separated and analyzed according to OCME case work protocols.

Once the mock case work sample was processed, they

were classified as suitable or nonsuitable for comparison based

on OCME case work protocols. The number of contributors to the

samples were estimated and, if possible, major and/or minor

contributor profiles were determined.

And all of this was done without knowing the true

composition of the mixture or anything about the mixture.

Q. You talked about classifying samples as suitable for

comparison. Can you elaborate on that?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

142



A. There are some things that can happen with an evidence

sample that would deem it not suitable for comparison. For

example, I mentioned earlier that I had consulted on a case

where another lab had drawn conclusions from a sample where

only two loci had results, so that would be far below OCME's

minimum number of loci to go forward with a sample, for

example. And there are other indications that could deem a

sample not suitable for comparison.

Q. These samples, was a visual analysis of them done?

A. Yes. So these are the categories that OCME previously used

to qualitatively describe the relationship between a suspect

profile and an evidence mixture. So if all of the comparison

samples' alleles were labeled in the mixture, they would say

that this person was included as a possible contributor to the

mixture. If most but not all were labeled, this person could

not be excluded. And it's a little bit fuzzy here, most if not

all, because some loci are more robust than others, so if a

comparison sample or a suspect profile, if alleles were missing

at a couple of the loci that are very robust and we expect them

to amplify, we may not be as forgiving of them missing, so that

might move it down to a no conclusions category. But if a

couple of alleles were missing at one of the loci where

drop-out is a common occurrence, then that person would not be

excluded from the mixture.

If a couple more are missing, no conclusions can be

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

143



drawn. And if several are missing, that person would be

excluded as a contributor to the mixture in a case work

protocol.

Q. Now, during the validation when a manual evaluation yielded

a conclusion of excluded, was the sample nevertheless run

through FST?

A. We did run it through FST just to see what it would look

like. In case work if a comparison person is manually excluded

from a mixture, FST is not run.

Q. And in case work, is FST run when the manual call is no

conclusions?

A. It is not.

Q. Did it used to be for no conclusions?

A. It.

Q. And at some point that change was made?

A. At some point that changed, yes. And I was not part of

implementing that change; I don't know what the rationale was

behind it.

Q. But today in practice FST would only be used if "included"

or "cannot be excluded" was the manual call?

A. Yes, that's true.

Q. But in the validation study all samples were run through

FST regardless?

A. Yes, we wanted to see how it performed on all types of

comparisons.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

144



Q. And what were the results of the performance for all types

of comparisons?

A. If you go to the next slide, so looking at the mixtures

that were categorized as two person mixtures by the case work

protocols, some of these may have been three person mixtures

where one of the contributors dropped out enough that it

reached the two person criteria. These were the four

categories. Each donor -- each true donor was categorized as

"included" or the major contributor, "could not be excluded,"

or "no conclusions," or "excluded." And then FST was run using

each of these true contributors as a suspect, and the model

that was used was the suspect plus one unknown person in the

numerator and two unknown people in the denominator. So it's a

two person mixture where we don't have a known contributor to

the mixture.

Q. So in this slide, is it right that 119 true contributors

were manually excluded?

A. Yes.

Q. So how can it be the true contributor would be excluded

after a manual evaluation?

A. If several alleles were not labeled in the mixture, that

person would be manually excluded, even though in these samples

those people really were contributors, but there was enough

drop-out that they didn't look like contributors.

Q. And if we look at the next slide, can you talk to us about

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

145



the results.

A. Yes. So, this shows the likelihood ratios or actually a

log of the likelihood ratios, so that the exponent of the

likelihood ratio on the vertical axis. So at the top is ten,

which means that's a likelihood ratio of ten to the ten. The

zero represents a likelihood ratio of one. So that's the level

where we would say this mixture is not supporting one scenario

over the other. And then results that are below zero are

likelihood ratios that are less than one, and those are the

ones that support the scenario that includes the unknown

contributor in place of the suspect, or in this case these are

all actually true contributors.

Q. So if we're looking at the plot on the left, the first of

the four boxes, what is the manual call that that corresponds

to?

A. These were contributors that were visually included in the

mixture, so their alleles showed up at all loci that were

tested.

Q. What is the heavy bar in the middle of that?

A. The heavy bar is the median result. So the median here was

about ten to the ten. But there is a spread. You can see that

one of them, one of the observations is down close to one.

This could happen in a situation where all of a person's

alleles do show up in the mixture but they are all very common

alleles, so it's not lending much support, if any, to that

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

146



individual being a contributor to the mixture based on the

likelihood ratio.

Q. And if the heavy bar is the median, what are the thinner

bars on either side of the heavy bar?

A. The bar just below the heavy bar is the 25th percentile,

and the bar just above the heavy bar is the 75th percentile.

Q. And the dotted line and the thin line that goes below it,

what do those represent?

A. So, in some cases, for example, in the graph on the right,

the second box plot, there are no dots beyond the ends of those

whiskers, so those represent the lowest value obtained and the

highest value obtained. And with this type of plot if there

are observations that are beyond one and a half times the

height of the box, above the box, those are represented by

dots, above or below the box, with a distance of one and a half

times the box height away. So those dots are still

observations; they are just kind of extreme observations.

Q. So as you move to the second column here that's labeled

CBE, what are you looking at with that box?

A. Those were comparisons for which the donor was deemed

cannot be excluded. So in those comparisons generally one or

two of the donor's alleles were missing from the mixture. So

this now the median is down from about ten to the ten to ten to

the fifth.

Q. However, in "cannot be excluded" do you have some results

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

147



that are below a likelihood ratio of one?

A. Yes. So, some of these individuals who are missing one or

two alleles, the likelihood ratio actually turns out to be

quite low, less than one.

Q. And without walking through each and every one of these

boxes, generally speaking what is the conclusion that you draw

from these charts?

A. So as we go left to right across these categories we're

going from a situation where all of the donor's alleles show up

in the mixture to more and more of the alleles are missing from

the mixture. And as would be expected, the likelihood ratios

go down as more and more of the contributor's alleles are

missing.

Q. So moving to the next slide, what does that tell you about

FST's performance relative to a quantitative assessment?

A. The likelihood ratios were consistent with the qualitative

assessments but they provided more information.

Q. And did you generate similar results for three person

mixtures?

A. We did. The next slide shows the break-down of the three

person mixtures, or these again are the mixtures that were

estimated to include three persons by case work protocols.

Q. And moving to the next slide, what were the results for the

three person mixtures?

A. And the results were similar, although for three person

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

148



mixtures on average they were a little bit lower. So, if you

recall, the two person mixtures the included category was

around ten to the tenth, and now it's down to maybe ten to the

sixth.

THE COURT: By they, are you referring to the

likelihood ratio?

THE WITNESS: Yes. Yes.

THE COURT: Go ahead.

Q. That's the median likelihood ratio?

A. The median likelihood ratio was a bit lower for the three

person mixtures than the two persons.

Q. So, overall what was your conclusion about the performance

of the likelihood ratios as opposed to qualitative assessments?

A. The likelihood ratios are consistent with the quantitative

assessments, but they provide more information and sometimes

very valuable information, such as a "cannot be excluded"

categorization sometimes generates a likelihood ratio that is

above one and sometimes it's below one. And this would be

really important information in a case. If the manual

assessment was this person cannot be excluded, sometimes the

more granular result provided by a likelihood ratio could

actually be the other way.


BY MR. McKAY:

Q. Now, was part of the validation study what you might call a

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

149



false positive study or noncontributor testing?

A. Yes.

Q. Could you describe what that entailed.

A. Yes. We wanted to know if someone who did not contribute

to the sample was tested, what did the results look like. How

often would somebody who didn't contribute to the sample

generate a likelihood ratio of greater than one or greater than

ten or greater than a thousand?

Q. So what did you do to perform that test?

A. So we used a database that we had. It's a population

database, the one we used to estimate allele frequency in New

York City. That included just over 1,200 individuals, and we

treated each of those people, one at a time, as the suspect in

all of these validation mixtures. And we computed the

likelihood ratio under the scenario that they were a

contributor compared to the scenario that they were not a

contributor.

Q. What were the results of those comparisons?

A. So overall, on the next slide we did about a half a million

comparisons with these noncontributors, and about .03 percent

of the time a noncontributor gave a likelihood ratio greater

than one.

Q. When you say .03 percent --

A. .03 percent.

Q. That's not three -- it's not 300 percent?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

150



A. It's three per 10,000.

Q. Moving to the next slide, what does this show you?

A. This shows the distribution of the likelihood ratios for

noncontributors. So these are similar to the plots that I

showed earlier where the zero on the X axis represents a

likelihood ratio of one. So there are very few observations to

the right of that. Very few likelihood ratios greater than one

were obtained. These were the two-person samples or the

samples that were deemed two-person by casework protocols. And

I'm showing you the results for the mixtures that were one to

one on the right, and the ones that were not one to one on the

left.

Q. Everything to the left of the zero is a likelihood ratio of

less than one, which is a result that favors the defense

hypothesis; is that right?

A. Yes. And on average, these were very, very low. So ten to

the minus 20 is there in the middle.

Q. It's only the data shown to the right of the zero that is

what you would call a false positive?

A. Yes.

Q. Moving on to the next, what does this one show you?

A. This is the same set of plots for the mixtures that were

deemed three-person samples by the casework protocols. So,

again, the average result is somewhere in the 10 to the minus

10 to 10 to the minus 20 range, and there are very few

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

151



observations where the likelihood ratio is greater than one.

Q. Moving to the next slide, what does this show you?

A. So the number that I mentioned before, .03 percent, the

three per 10,000, that is the frequency with which likelihood

ratios greater than one were generated for noncontributors.

That happened 163 times in the half a million tests that we

did. Fifty-six times a likelihood ratio greater than 10 was

generated. So that's a .01 percent rate. Fourteen times there

was likelihood ratio greater than 100. Five times there was a

rate greater than 1,000. And only one time there was one

greater than 10,000. And that was a situation where it was a

three-person mixture and one of the noncontributors in the

database, I believe all of that person's alleles showed up in

the mixture by chance. There may have been one missing. I

can't remember for sure.

Q. The first number, 163 --

A. Yes.

Q. -- does that encompass the 56 and the 14 and the 5?

A. Yes.

Q. The 1?

A. Yes.

Q. What does the distribution of this chart tell you about

where you'd expect to see false positives?

A. Most of our false positives were between 1 and 10,

likelihood ratios between 1 and 10.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

152



Q. So if you get a likelihood ratio that favors the

prosecution in the very strong support category, so greater

than 1,000, what was the rate of false positives?

A. That was .0009 percent. So five out of half a million.

Q. What does that tell you about the relationship between the

strength of the likelihood ratio and the possibility of a false

positive?

A. The higher the likelihood ratio, the less likely it is that

a noncontributor would generate a likelihood ratio that high by

chance.

Q. We talked earlier about the concept of a manual exclusion?

A. Yes.

Q. Remind us again what that is.

A. So the casework protocol, if an analyst looks at a mixture

and looks at a comparison profile and manually excludes that

person from the mixture, for example, if several alleles are

missing, they do not go on to generate a likelihood ratio.

That's where the analysis stops. It's an exclusion.

Q. And that's in real world, in casework?

A. That's in real world casework. We went on to compute the

likelihood ratio for all of those just to see what they looked

like, but in casework, if a sample is deemed no conclusions can

be drawn or this person is excluded, the LR is not computed.

Q. Did you go and count in this chart how many of these would

have been manual exclusions in casework?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

153



A. We did not.

Q. In theory, though, is it possible this chart encompasses a

number of samples that would have been a manual exclusion?

A. Sure, yes.

Q. In fact, would you expect that it was likely that some or

even many of these were manual exclusions?

A. I would and expect that. And some, if not many, would be

in the no conclusions category.

Q. Which under current protocols would not be run through FST?

A. Correct.

Q. In fact, a number of these samples in real life, in

casework, may not actually generate a statistic under FST?

A. That's true.

Q. If you'll turn in your binder to the tab of Government

Exhibit 7, which is the very hefty one. Do you recognize is

that?

A. Yes.

Q. What is that document or series of documents?

A. These are the executive summaries from the 22 volumes that

comprise the validation.




evidence.


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

154



BY MR. McKAY:

Q. You said this is the executive summary. This is not the

whole validation?

A. That's true. The whole validation included one or two

three-inch, three-ring binders per volume, and there are 22

volumes. These are just the summaries.

Q. When you were describing the validation process earlier,

you mentioned degradation. Can you tell us what degradation

is.

A. Degradation is when DNA is broken down by environmental

factors such as UV light or heat.

Q. Did you attempt to account for degradation in the

validation study?

A. We did. At one point we estimated drop-out rates for

degraded samples which were higher than the drop-out rates for

non-degraded samples, and then we looked to see whether using

those higher rates improved the LRs for true contributors, so

did they generate higher LRs for true contributors and did they

also maintain the lower LRs for noncontributors? And we found

that we did not improve the separation between the two groups

using this degraded module to the program.

Q. So what did you do?

A. So degraded samples are now analyzed through the standard

protocol, and we did include a set of degraded samples in the

validation as well.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

155



Q. Was what you just described about how you addressed the

issue of degradation, was that presented to the DNA

subcommittee that ultimately approved FST?

A. Yes.

Q. Was it published in your peer-reviewed article about the

FST validation process?

A. Yes.

Q. Would you say that FST is validated for use with degraded

samples?

A. Yes.

Q. How so?

A. We included degraded samples in the validation. They are

sometimes less informative than non-degraded samples; meaning,

likelihood ratios are closer to 1 than they would be if we had

a full -- full, non-degraded sample.

Q. What is the significance or what role does race play in

determining a likelihood ratio?

A. So at the forensic loci, the alleles occur at somewhat

different frequencies in different populations globally. So an

allele that might be quite common in one population could be

fairly rare in another.

Q. So the math that we were looking at earlier, when you were

plugging in allele frequencies, that may differ by race?

A. Yes. So OCME calculates the likelihood ratio four times

using its four different New York City population allele

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

156



frequency estimates, and then the value that's reported in the

casework report is the lowest of those four values.

Q. So FST generates not actually one likelihood ratio but

four?

A. Yes.

Q. But the one that is reported is which?

A. The lowest of the four. We don't know the ancestry of the

contributors to any forensic sample, so we compute it four

times and take the lowest.

Q. Did you conduct separate validation studies for the races

of each given contributor?

A. All four calculations were done for each of the validation

samples, and the values that are reported are the ones that

would be reported in casework, the lowest of the four.

Q. Were there any population geneticists on the DNA

subcommittee that approved FST?

A. Yes, there were to: George Carmody and Ranajit

Chakraborty.

Q. Did either of them express any concern to you about the

racial composition of your validation mixtures?

A. They did not.

THE COURT: At the time?

MR. McKAY: At the time.

Q. When you've been describing the manner in which FST reports

its conclusions, I think you've used the term "unknown,

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

157



unrelated." What is the significance of unrelated here?

A. In the likelihood ratio, we model replacing the suspect

with an unknown, unrelated person. In reality, the contributor

to a mixture may in fact be someone who is related to the

suspect, and so in that circumstance, the calculation -- the

population frequencies are different than they would be for a

relative of the suspect.

Q. Would you expect that two brothers would have alleles that

are more similar or less similar than a randomly selected

member of the community?

A. They would have alleles that are more similar.

Q. Did the validation study discuss at all the question of

relatedness?

A. No.

Q. Did you consider whether to incorporate relatedness in FST?

A. That was something that was on the list of things that I

wanted to do.

Q. But to be clear, FST as currently formulated does not

consider the question of relatedness?

A. That's true.

Q. You mentioned that FST issues its report. Does it specify

that it's not purporting to address relatedness?

A. It reports it as unknown, unrelated person.

Q. In a case in which the defendant made a claim or there's a

defense hypothesis that a relative was a possible contributor

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

158



or possible suspect, could defense counsel present that to OCME

and ask them to do any testing on the relative?

A. Yes. If they could obtain a DNA sample from the relative,

that could be done.

Q. Would OCME then test that DNA sample?

A. I believe that they would, but I was not involved in that

work at OCME.

Q. Has FST been peer reviewed?

A. Yes.

Q. In what ways?

A. The publications have been peer reviewed as described

earlier, and the DNA subcommittee reviewed the validation, the

development and the validation in great detail. In addition, I

and others have presented the work in progress and the final

validation and discussion of other areas of interest

surrounding OCME at -- I'm sorry, surrounding FST at national

and international forensics meetings.

Q. You mentioned the DNA subcommittee and we made reference to

it, but what is the DNA subcommittee?

A. Any new lab or analytical procedure -- any procedure that

is new to a forensics lab in the state of New York must be

approved by the New York State Commission on Forensic Science.

The DNA subcommittee handles the details of any DNA-related

procedure that's new, and the DNA subcommittee makes a binding

recommendation to the larger committee about whether or not to

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

159



approve a new DNA protocol.

Q. Who are some of the members of the DNA subcommittee at the

time that FST was considered and approved?

A. I've already mentioned George Carmody and Ranajit

Chakraborty, population geneticists. Jack Ballantyne was the

chair of the committee. He is or was faculty of University of

Central Florida and is a very well-known, well-respected

forensic geneticist. In addition, Eric Buel was on the

committee. He is the director of the Vermont Public Lab,

public DNA lab. In addition, there was a molecular biologist

by the name of Mark Batzer, and a public health epidemiologist,

an M.D., named Ann Walsh. And Charles Hirsch, the chief

medical examiner at the time, was also on the DNA subcommittee,

although he did not vote on OCME-related protocols.

Q. Is it fair to say that the subcommittee is a collection of

experts in the fields that are relevant to forensic DNA work?

A. Yes.

Q. What is your understanding of what the subcommittee's role

was or function was in reviewing a program like FST?

A. We met with the subcommittee four times over the course of

a year and a half, and the initial meeting was one where we

presented what we were interested in doing and some very

preliminary work that we had done. Then we met with them for

two progress reports where we presented our results thus far

and got feedback from them, and they at various times asked us

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

160



to do some additional work or to answer some additional

questions. Then finally, in October of 2010, we made a final

presentation to them, and they approved the method for

casework.

Q. Before these meetings with the subcommittee, did you

provide any materials to them?

A. Yes. At least two weeks in advance we sent them a summary,

including -- either exactly an executive summary or something

close to an executive summary that's presented here for them to

review prior to the meeting.

Q. What kind of materials did you present to them at the

actual meetings?

A. At the actual meetings, we had summary slides, but we also

had the material they sent them ahead of time that we could

refer to and they could refer to, to get into more detail of

what we had done.

Q. What happened at those actual meetings? What did you talk

about?

A. We talked about questions that we were working on. For

example, if we were going to implement the degraded protocol,

we talked about the numbers of samples that we were generating.

We showed the validation results as they developed. They asked

us questions about our methods as well as about our samples.

Q. You said they asked you questions. Were they substantive

questions?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

161



A. Yes.

Q. Did they have the opportunity to ask you to do additional

research?

A. They did.

Q. Did they, in fact, ask you to do additional research at

times?

A. They did.

Q. Did you also have informal interactions with members of the

subcommittee outside the context of the meetings?

A. Yes.

Q. If you turn in your binder to Government Exhibits 8A, 8B,

8C, and 8D, tell me if you recognize those documents.

A. These are presentations made by OCME to the DNA

subcommittee at the four meetings that I mentioned.

Q. Is there one presentation for each of the four meetings?

A. Yes.

MR. McKAY: Government offers 8A through 8D.


THE COURT: Government Exhibits 8A, 8B, 8C, and 8D are

admitted in evidence.

(Government's Exhibits 8A, 8B, 8C, and 8D received in

evidence)

BY MR. McKAY:

Q. If you turn to Government Exhibit 9, can you tell me if you

recognize that?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

162



A. Yes.

Q. What is that?

A. This is a letter from Jack Ballantyne, who was the chair of

the DNA subcommittee to Sean Byrne, who was the acting

commissioner and chair of the Commission on Forensic Science,

stating that they had approved FST for use with forensic

casework by OCME.




evidence.


BY MR. McKAY:

Q. Do you see that letter references an October 8, 2010,

meeting at which FST was approved?

A. Yes.

Q. Were you present for that meeting?

A. I was.

Q. Would you turn to Government Exhibit 10. Can you tell me

if you recognize that document.

A. These are the minutes from that October 10 meeting of the

DNA subcommittee.

Q. Do you see down at the bottom there does it list you as an

attendee?

A. It does.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

163






evidence.


BY MR. McKAY:

Q. If you turn to the third page, you see the bolded

paragraph?

A. Yes.

Q. Without reading the whole thing, generally speaking, what

does that paragraph report?

A. This notes that the presentation of the validation was

given by Theresa Caragine to the DNA subcommittee, and the DNA

subcommittee had reviewed and evaluated the FST and approved it

for forensic casework.

Q. Who seconded the motion to approve FST?

A. Dr. Chakraborty.

Q. After the subcommittee approved FST, was there any other

approvals required?

A. The scientific forensic commission approved FST.

Q. What is the relationship between the scientific forensic

commission and the subcommittee?

A. The subcommittee deals with the details of DNA analysis.

The commission itself does not include scientists, but rather

attorneys and judges and others -- other legal professionals,

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

164



although there is one scientist on there, Marina Stajic, who is

an OCME scientist, or was at the time. So the DNA subcommittee

deals with the details of the DNA methods and makes a binding

recommendation to the commission, although the commission has

the opportunity to ask questions or even to require additional

work if they so chose.

Q. Did members of the full commission in fact ask questions?

A. They did.

Q. Did they ultimately accept the binding recommendation of

the subcommittee?

A. They did.

Q. Were you present at the meeting at which it was accepted by

the full commission?

A. I was.

Q. If you look at Government Exhibits 11, will you tell me if

you recognize that.

A. These are the minutes from the New York State Commission on

Forensic Science meeting on December 7, 2010, at which FST was

approved.




evidence.


BY MR. McKAY:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

165



Q. If you turn to Exhibit 12, can you tell me if you recognize

that.

A. Yes.

Q. What is that?

A. This is a letter from Sean Byrne, the chair of the forensic

science commission, to Dr. Mechthild Prinz, who was at the time

the director of the Department of Forensic Biology at OCME.



THE COURT: All right. Exhibit 12's admitted in

evidence.


BY MR. McKAY:

Q. What does that letter say?

A. The letter states that FST has been approved by the

Commission on Forensic Science for use with forensic casework

by OCME.

Q. Now, did there come a time before FST was approved that

Dr. Chakraborty raised a question he had about the validation

study?

A. Yes.

Q. What was the nature of his question or concern, as best you

recall?

A. He wanted to know if having drop-out at one locus was

associated with drop-out at another locus. So if we knew there

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

166



was drop-out at locus one, did that give us any information

about the probability of drop-out at locus two, for example.

Q. How did Dr. Chakraborty raise that concerns to you?

A. He raised it during a DNA subcommittee meeting.

Q. What did you do in response to his concern?

A. I did a conditional analysis, which is a standard

statistical procedure where I included information about

whether or not there was drop-out at one locus and then looked

to see if that affected the probability of drop-out at any

other locus.

Q. Did you ultimately present that work to the subcommittee?

A. I did. And I found the drop-out probabilities to be

independent from locus to locus.

MR. McKAY: Mr. DeLuca, will you pull up Government

Exhibit 8C and turn to slide 25.

Q. What is this slide?

A. This is a summary of the testing of the independence of

drop-out rates that I presented at a DNA subcommittee meeting.

Q. If you could look at the next slide and tell us what that

is -- actually, go back one.

A. Here, I describe the question that he had asked and the

approach that I took.

Q. Now, after you presented this additional research on

Dr. Chakraborty's question, what happened next with respect to

this issue?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

167



A. He said that he had some other ideas about additional

analyses that he would like us to do.

Q. Did you subsequently speak with him about those ideas?

A. Yes. He said: Contact me after the meeting, and we can

talk about what to do. So I repeatedly attempted to contact

him. He did reply to one email saying he was about to go out

of town and for me to contact him again the next week, which I

did. He did not reply, and I repeatedly tried to call him and

was unable to get ahold of him.

Q. When you ultimately presented the validation study to the

DNA subcommittee for approval, did you describe the steps you'd

taken with respect to this issue that Dr. Chakraborty had

raised?

A. I don't remember if we -- if I specifically described that,

but he did not bring up this issue at the meeting again.

MR. McKAY: Mr. DeLuca, if you'll pull up Government

Exhibits --

THE COURT: Just to be clear, though, did he ever with

any kind of specificity indicate what he was thinking?

THE WITNESS: No.

THE COURT: OK.

THE WITNESS: No.

THE COURT: So during the meeting, he just said: Oh,

you know, I have some thoughts on this. Just give me a ring

after the meeting?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

168



THE WITNESS: Yes.

THE COURT: All right. Go ahead.

MR. McKAY: Thank you.

Mr. DeLuca, would you pull up Government Exhibit 7.

Q. Just looking at the first page, what was this exhibit

again?

A. These are the executive summaries of the volumes included

in the FST validation.

MR. McKAY: Mr. DeLuca, would you turn to page 713 of

the PDF.

Q. Dr. Mitchell, what are we looking at here?

A. This is a summary of the analysis that I did to determine

whether drop-out was independent from locus to locus.

Q. And was this summary available to the DNA subcommittee

before they voted to approve FST?

A. Yes.

Q. Was it available to Dr. Chakraborty before he seconded the

motion to approve FST?

A. Yes.

MR. McKAY: You can take that down, Mr. DeLuca.

Q. Now, you testified earlier that the calculations underlying

FST take a long time to do by hand; correct?

A. Yes.

Q. So FST has a program, a software program, to perform those

calculations?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

169



A. Yes.

Q. Are you a computer programmer?

A. I do some computer programming in my work, but I am not a

formally trained programmer.

Q. Who did the computer programming of FST?

A. Programmers that were contracted by OCME.

Q. What was your role? How did you interact with those

programmers?

A. I described what needed to be done, and as they developed

functions within the program, I would test those functions

individually to determine whether they were working correctly.

Q. How would you test those functions to determine if they

were working correctly?

A. For a small number of alleles in a sample, I could perform

manual calculations, which I did; or if it was not possible to

evaluate the entire function, they would create intermediate

outputs for me to check to confirm that they were working

correctly.

Q. When you performed those checks, did you find that you were

getting the expected outputs?

A. Yes.

Q. From your perspective, what was the goal of the computer

program or the source code underlying the computer program?

A. To compute a likelihood ratio that was not possible to

compute by hand in real time.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

170



Q. Did you want it to do the math correctly?

A. Yes.

Q. Did you care about the style of the coding?

A. No.

Q. Did the program --

THE COURT: Well, what do you mean by "style"?

THE WITNESS: I did not look at the source code and

weigh in on the way that they were naming variables or

commenting in the code or, you know, sometimes there are a

choice of several functions that one could use to do the same

procedure. So I didn't -- I didn't weigh in on what the

specific code said.

THE COURT: OK. Go ahead.

Q. Are there different code languages that one could use for

computer programming?

A. Yes.

Q. But as between any particular code language, did you have a

preference?

A. I did not.

Q. You wanted it to do the math right?

A. Yes.

Q. And did it do the math right?

A. Yes.

Q. How do you know that?

A. Because I could manually confirm that.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

171



Q. When you did the validation study, when you generated these

results that you've been walking us through, were you using

that computer program?

A. Yes.

Q. Did it generate results broadly consistent with what you

would have expected to see?

A. Yes.

Q. After the time that FST was approved for use in casework,

did there come a time that OCME detected an issue with the

source code?

A. Yes.

Q. What happened?

A. FST was put online in April 2011, and one of the first few

samples that was run generated a negative likelihood ratio,

which does not make sense mathematically.

Q. When you say "negative likelihood ratio," you mean actually

a negative value?

A. Yes. Not the negative exponent, but a negative value.

Q. Because we've been looking today at likelihood ratios

greater than 1 or less than 1?

A. Right. On the log scale the exponents are greater than

zero or less than zero, but the likelihood ratio itself is

always greater than zero because it's a ratio of two

probabilities, both of which have to be between zero and one.

Q. Were you able to figure out how it came to be that you

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

jbucklet

Highlight

172



generated a negative likelihood ratio?

A. Yes.

Q. How did that happen?

A. There are two conservative measures that OCME takes that

are standard measures that labs, many labs -- many if not all

labs use with random match probability. And those are if an

allele is extremely rare, say, one in a thousand or if it's not

observed in the frequency database that's used by the lab,

there is a standard protocol that is to assume that it was seen

five times in the sample. So to artificially inflate its

frequency to about -- in our sample would have been 1 or

2 percent. And the effect of that is a very rare allele that

is carried by the suspect and shows up in the mixture can be

quite influential in the LR because it will be unlikely to

randomly sample that from the population. So increasing the

frequency of alleles used is a way of making sure that does not

receive too much weight. It's a way of down-weighting super

rare alleles.

The second adjustment we do is if a population is

actually not one freely mixing population but instead there are

pockets of the population that kind of stick together, the

frequency with which homozygotes are found can be a little

higher than expected. So we and other forensic labs make a

slight adjustment to the expected frequency of homozygous

genotypes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

jbucklet

Highlight

jbucklet

Highlight

173



So those two things together, if there was a locus

that had alleles that added up almost to a 100 percent, when

those conservative adjustments are made, it could bump the

frequency above 100 percent, which would be nonsensical in the

calculation. So what we did was if there is a locus at which

the alleles add up to 97 percent or higher, that locus is not

used in the calculation. So what had happened in that one

sample was there were several common alleles at a locus and a

couple of rare alleles that bumped the entire frequency up to

1, and then when we did the second -- the homozygous frequency

adjustment, it gave the total genotype frequencies go higher

than 1. So we needed to avoid getting into that situation in

the future.

Q. Let me see if I can -- what did you do to avoid getting

into that situation?

A. We then made it impossible for the total genotype frequency

to go higher than 1.

Q. How did you do that?

A. By not using a locus at which the alleles added up to

97 percent or higher.

Q. So in a hypothetical situation where the alleles present at

the locus added up to 98 percent of the population, if you

applied the 3 percent theta correction factor, you'd be at

101 percent; is that right?

A. It doesn't exactly translate that way, but it's very close.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

jbucklet

Highlight

jbucklet

Highlight

174



So the adjustment that I'm talking about with the homozygous

frequency is called a theta adjustment, and it's fairly

universally applied to guard against this structured population

problem. So at a locus in the sample, if the alleles that are

labeled at a locus in the sample add up in frequency to .97 or

higher, that locus is treated as a 1 or treated as an

uninformative locus in the calculation.

Q. So when that allele frequency cap comes into play, what

kind of -- well, in situations in which it comes into play,

what can you say about the alleles that are in that mixture?

A. That they include close to -- they include, generally, all

of the common alleles at that locus. There are some loci that

have two or three very common alleles that together might add

up to close to 95 percent. So those loci often don't provide

much information anyway, because almost everyone could be a

contributor to the mixture at that locus.

THE COURT: I guess just a question I have is,

obviously, there were a lot of things in going through the

validation process that were utilized. Why wasn't this, the

procedure, the theta, whatever it is, something that was

basically looked at beforehand? In other words, there are a

lot of things that were done. It sounds like the use of the

theta was something that was done, it is commonly done, but it

wasn't factored in when going through the validation process.

THE WITNESS: We didn't see any likelihood ratios that

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

175



were negative. That was the trigger that we'd identified a

problem during the validation.

MR. McKAY: If I may follow up on that question, your

Honor?

THE COURT: Yes.

MR. McKAY: Mr. DeLuca, could you pull up Government

Exhibit 8A and turn to slide 46.

Q. So government 8A is one of the PowerPoint presentations

before the DNA subcommittee.

A. Yes.

Q. What are you describing in this PowerPoint presentation?

A. Here's a note saying in the simple examples that I showed

to the subcommittee about how the calculations were done, we

did not include this theta correction, this adjustment for the

homozygous genotype frequency. However, as we say here, we do

include this adjustment in FST.

Q. So it was explained to the DNA subcommittee that you'd be

using the 3 percent theta correction adjustment; is that right?

A. Yes.

Q. And was that also in your peer-reviewed article about FST?

A. Yes.

Q. And was it also disclosed to the subcommittee that you'd be

using this 2 percent, I believe you said, minimum allele

frequency?

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

176



Q. So the two conservative adjustments that you were going to

make were disclosed to the subcommittee; is that right?

A. Yes.

Q. But this allele cap, which had to be put in place to stop

the byproduct of these adjustments, that was not contemplated

at the time; is that fair to say?

A. That's fair.

Q. So you said in one of your earlier answers, I think, that

the loci at which this function would come into play are ones

in which there are very common alleles; is that right?

A. Yes, that's generally true.

Q. And you said that that did not provide that much

information. What do you mean by that?

A. Often those loci are not very informative because, for

example, at one locus the alleles 8, 9, and 10 together might

make up 95 percent of the allele frequencies in a population.

So often then when one is comparing, say, a suspect profile to

the mixture, the suspect has some combination of 8, 9, and 10.

So it's very common at those loci to not really learn much from

that locus because many people that are compared to that

mixture have alleles that show up in the mixture.

Q. So the discriminatory power of that particular locus is not

as high as one in which there were more rare alleles; is that

right?

A. That's true.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

177



Q. So would you expect the effect of that function of capping

the allele frequency -- what would you expect that to have in

any given case in terms of the likelihood ratio?

A. So we're treating that locus as a likelihood ratio of 1.

We're treating it as an uninformative locus in the calculation.

So, in general, I would expect that doing that would pull the

likelihood ratio for the entire profile closer to 1, so making

it less strong in either direction.

Q. Would you necessarily see that in every case, or that's

what you would suspect?

A. No, not necessarily. That would be the average result.

Q. So after that change was made to the source code, what, if

anything, did FST do to test to see what the results were?

A. So we did a performance check. We have a set of

comparisons that were run every time anything changed, and

usually the change would be something with the interface or

adding a function for research purposes or moving it to a

different server. So we would run these -- there are 12

mixtures. We would run and confirm that the output was the

same before and after the change.

Q. When you say you'd run these 12 mixtures, what would you

compare these mixtures to?

A. We would compare the true contributors to the mixture, at

least one of them. I don't know if we did all the contributors

per mixture. And then we would compare our database of

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

178



noncontributors. So we would do about 1,250 noncontributor

comparisons and at least one, possibly more, true contributor

comparisons to -- with these performance check samples.

Q. So if you have 12 samples, did you do 1,250 estimated

for -- 1,250 comparisons for each of those 12 samples?

A. Yes.

Q. Turn your binder to Government Exhibit 16 which doesn't

have its own tab. I think it's just behind 15.

A. Yes.

Q. Do you recognize that document?

A. I do.

Q. What is it?

A. This is a document that was generated summarizing the

results when we did the performance check after we implemented

the .97 cap.



THE COURT: All right. Government Exhibit 16 is

admitted in evidence.


BY MR. McKAY:

Q. What were the results of that performance check?

A. So 10 of the 12 performance check samples were unaffected,

and two of them then triggered the .97 cap. So for those two,

we summarized the likelihood ratios for the true contributors.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

179



One that had gone from .34 to .42 before and after the cap; the

other that went from 3.82 times 10 to the fourth -- excuse me,

it started out at 3.38 times 10 to the fourth before the cap

and it went to 3.82 times 10 to the fourth after the cap.

Q. So starting with the first result that changed, .34 to .42,

how would those two numbers be expressed in terms of the

likelihood ratio?

A. Those are both less than 1, and they are not qualitatively

different. They're not an order of magnitude different.

Q. What about the second, the 3.38 to the fourth and 3.82 to

the fourth?

A. The same thing, these are both times 10 to the fourth. So

they're in the same order of magnitude and a slight change in

the actual value, but not a substantial change.

Q. So in both instances, the specific number changed; is that

right?

A. Yes.

Q. But the bottom line conclusion in terms of the expression

of the strength of the evidence, did that change?

A. No.

Q. Now, when you made these 1,250 or so comparisons for each

of the 12 samples, were each -- noncontributor comparisons for

each of the 12 samples, were each of those noncontributor

samples manually evaluated to see if they were suitable for

FST?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

180



A. Do you mean following the casework protocol, whether a

person is excluded --

Q. Yes.

A. -- when they would not?

Q. That's what I'm asking.

A. So in the validation, we did something that was a little

bit different than casework protocol in this situation because

in casework, if someone is manually excluded, FST is not

computed. But we did compute FST on all of these

noncontributors because we wanted to see how it performed even

though, in a casework situation, many of those would not have

an LR computed. They would simply be excluded.

Q. So that's in the performance check. Is that also true of

the validation that you had done that you didn't do the manual

exclusions in the validation study?

A. Yes.

Q. Now, in reviewing the results of the performance check or

having reviewed them -- have you reviewed the results of the

performance check?

A. Just right here in front of me?

Q. Well, besides standing on the witness stand today, have you

reviewed the results of the performance check?

A. Yes.

Q. Are you aware of an instance in which a noncontributor

sample went from having an LR of less than 1 before the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

181



modification to the source code to having an LR of greater than

1 after the modification of source code?

A. Yes.

Q. Do you recall what the specific results were before and

after?

A. I don't know exactly what it was before, but it was

something like .5, to my estimate. And then after the cap, it

went up to about 29.

Q. Have you reviewed that specific profile?

A. I have.

Q. Based on your review of that profile, what can you say

about whether it would have been a manual -- what the manual

call would have been had that been found in casework?

A. In casework, the manual call for that sample would have

been no conclusions can be drawn.

Q. So in casework would that sample have generated a

likelihood ratio?

A. It would not.

Q. Are you aware of any other programs or methods that perform

a similar function to what we've just been talking about?

A. Yes.

Q. What example are you thinking of?

A. It's fairly common when computing a CPI, or a combined

probability of inclusion, to first examine the evidence sample

and make decisions about which loci will be included in the CPI

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

182



and which will not. This is before looking at any suspect

profile.

Q. If you'd turn in your binder to Government Exhibit 14.

Mr. DeLuca, if you'd put that on the screen as well.

Might be easier.

Do you recognize that article?

A. I do.

Q. What is that?

A. This is an article describing this very process, to look at

a forensic mixture and to make some decisions about which loci

will be included in the statistical calculation and which ones

won't.



THE COURT: All right. Exhibit 14 is admitted into

evidence.


BY MR. McKAY:

Q. Who are some of the authors of that article?

A. The first article is Fred Bieber. He is currently on the

DNA subcommittee of the forensic science commission. The

second is John Buckleton. He is one of the developers of the

STRmix, or STRmix program. Bruce Budowle is a former FBI

scientist, now an academic researcher, in Texas who used to be

a big proponent of the CPI but is now a likelihood ratio user.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

183



John Butler, who is at NIST, has done extensive research in

forensic DNA testing. And Mike Coble, who is also now at NIST

with John Butler.

Q. Is it fair to say this is a well-respected group of

forensic scientists?

A. Yes, it is.

Q. Is this a peer-reviewed publication?

A. Yes.

Q. If you look at the sixth page, I believe it is, and look at

Rule No. 1, in the bottom left corner, what are they describing

there?

A. They are describing a lab creating a locus qualifying rule,

they call it. So they say if a locus is going to be used -- a

locus is included for use in a CPI calculation if drop-out is

considered to be highly unlikely. So this is saying a lab can

use their internal protocols to identify loci in a mixture that

they do not want to include in a statistical calculation as

long as they have stated standards for making that decision and

as long as it is not made in comparison with a suspect profile.

So loci to be included in this calculation are identified. And

then if a suspect profile comes in and is compared to that

mixture, in this setting only the loci that were predetermined

to be part of that calculation will be used for comparison with

the suspect.

Q. Is that -- go ahead.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

184



A. In other words, if the suspect has an allele that does not

show up at one of the loci that they had excluded by whatever

their criteria were, it's kind of too bad for that suspect.

That potential exclusion will not be considered. The

statistics will be calculated only on the predetermined loci.

Q. Is this the function that you were describing a minute ago?

A. Well, it's a little bit different because we are -- we

would still --

Q. I think my question was unclear.

A. OK.

Q. I don't mean is this the function in FST.

A. Oh, yes.

Q. When I asked you are there other programs that perform

similar functions, is this what you were describing?

A. Yes.

Q. Now, to be clear, the allele cap, allele frequency cap

function in FST, does that come into play before consideration

of a suspect pedigree or after?

A. That -- the determination not to include a locus in the

calculation is made without considering the suspect profile,

but if a suspect profile comes in and is being manually

compared to the mixture, that locus can still be used to

exclude the suspect.

Q. So at the visual examination step --

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

185



Q. -- you would consider that locus?

A. Yes.

Q. But once the FST program runs, it mechanically excludes

that locus?

A. Treats that locus as uninformative, yes.

Q. Does it do that without consideration of whether or not the

suspect's alleles are present?

A. Yes.

Q. Are you aware that there has been litigation over whether

or not OCME should make the source code public?

A. Yes.

Q. Is it now public?

A. Yes.

Q. What was your view about whether to make it publicly

available?

A. I wanted it to be open source from the beginning. I

advocated for that.

Q. What is your understanding of why OCME resisted efforts to

make it publicly available?

A. It was a decision in the legal department, and my

understanding is that OCME wanted to reserve the right to

market it at some point if they wanted to and also did not want

to get caught up in how other labs were applying the method.

Didn't want to be responsible for ways in which other labs

might apply the method.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

186



MR. McKAY: One moment, your Honor.

THE COURT: Yes.

Q. Dr. Mitchell, in your view, are programs that analyze DNA

mixtures and generate likelihood ratios generally accepted in

the scientific community?

A. Yes.

Q. Is the method used by FST generally accepted in the

scientific community?

A. Yes.

MR. McKAY: No further questions.

THE COURT: Cross-examination.

MR. STRAZZA: After discussing it with the government,

may we have five minutes?

THE COURT: Oh, yes. Sure.

MR. STRAZZA: Thanks.

THE COURT: Doctor, you can step down. Since you've

been turned over and you're now own cross-examination, just

don't have any substantive conversations with the government

folks, OK?

THE WITNESS: OK.

THE COURT: Thank you.

(Recess)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

187


HB67JON6 Mitchell - cross

THE COURT: OK. Cross-examination.

CROSS EXAMINATION

BY MR. STRAZZA:

Q. Good afternoon, Dr. Mitchell.

A. Good afternoon.

Q. Before I start, I heard you say something towards the end

of your direct examination about -- I don't know if I got this

right -- by kind of too bad for that suspect. Did you use that

term?

A. I did.

Q. What did you mean by that?

A. In the CPI calculation when certain loci are not going to

be used in the calculation -- OCME doesn't do this. This is a

common practice in other labs -- and then a suspect profile

comes in, a suspect's alleles may not show up at the loci that

have been determined not to be part of the calculation, but

that will be ignored, and instead the calculation will use only

the predetermined loci in the probability of inclusion. And

that would be unfortunate for that defendant because their

alleles were missing at a different locus.

Q. Are you saying that those esteemed authors of that article

think that "kind of too bad for a subject" should be generally

accepted within the scientific community?

A. That was poor phrasing on my part.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

188



Q. Are you saying that a situation that could actually work to

the detriment of a suspect should be generally accepted within

the scientific community if it's not accurate?

A. No, I'm not saying that.

Q. Interpretation of DNA mixtures derived from crime scene

evidence is a major issue in forensic DNA analysis, right?

A. Excuse me, can you say it again.

Q. Sure. Interpretation of DNA mixtures from crime scene

evidence is a major challenge or issue within the forensic DNA

analysis community, right?

A. I would say it's an important issue, yes.

Q. It's what we've been talking about all day, right?

A. Yes.

Q. And mixtures arise when two or more individuals contribute

to an evidence sample, right?

A. Yes.

Q. And the contributors to an evidence sample can include

victims, suspects, etc., right?

A. Yes.

Q. And for some samples you can separate the contributors,

right?

A. Yes.

Q. And for others you cannot.

A. Right.

Q. In the situations where you can separate, the random match

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

189



possibility is conducted, right?

A. Yes.

Q. By OCME. And in the other situations where you cannot

separate the contributors to the sample, that's where a

likelihood ratio can come into play to help provide a

statistic, right?

A. Yes.

Q. And in those situations it evaluates the strength of one

scenario versus another scenario, right?

A. It evaluates support for one scenario versus another

scenario.

Q. OK. And that's exactly why you designed FST, right?

A. Right.

Q. In this case we are talking about one of the scenarios

being that the case sample that was analyzed was made up of

Dean Jones and two unknown contributors, right?

A. I am not familiar with the specifics of the case.

Q. You haven't looked at the case file in this case?

A. No.

Q. Ever.

A. No.

THE COURT: What do you mean by case file?

MR. STRAZZA: The materials that were produced by OCME

with respect to Dean Jones.

THE COURT: OK.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

190



Q. Is it your testimony that in preparation for this hearing

you never looked at the case file?

A. Yes.

Q. The FST model was designed for contributors that were

unrelated to one another, right?

A. Yes.

Q. So the likelihood ratio produced by FST would not be

suitable if the contributors were father and son, for example.

A. It does not address that scenario, no.

Q. Or brother and brother, right?

A. That's correct, it does not address that scenario.

Q. And why is that?

A. Because when the genotype frequencies are estimated, it is

using population frequencies not relative frequencies.

Q. And because relatives will have a higher frequency of the

same type of alleles, right?

A. Relatives tend to share more alleles than two randomly

selected persons from a population.

Q. And I believe you said that what you could do in that

situation is if a defendant wants to present somebody else's --

you know, a sibling, or a son or a father, they could just

present that to OCME, and OCME would test that for them, right?

A. I said I'm not actually familiar with the procedure, what

would actually happen, because I don't --

Q. So the answer is you don't know.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

191



A. I don't know what OCME's procedure is for dealing with a

situation like that.

Q. OK, I misunderstood you. I thought you said that could be

done.

A. It could be done programatically. FST could do that is

what I meant. I don't know what the protocol would be if that

came up in a case.

Q. Have you ever heard of it being done?

A. Heard of what being done?

Q. What you just described, where a suspect or through their

attorney provided somebody else's, a relative's, sample to be

tested with FST, to be compared to the case sample.

A. I don't know of a specific case, no.

Q. As you sit there, do you understand the problems with that?

A. Can you be more specific?

Q. Sure. Do you understand why a suspect might not want to

implicate one of their relatives?

A. Oh, certainly, yes.

THE COURT: Well, I mean, look, as a basic legal

matter the suspect doesn't have to put on any proof of

anything, so the issue may not necessarily be one of -- right

here we're deciding a Daubert issue which is a threshold issue

about whether or not this evidence could possibly be

considered. I think it's a separate issue. And again I'm just

thinking off the top of my head, because I didn't realize that

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

192



there was a potential argument related to this. The separate

issue then I think may relate to how relevant the actual

information would be at the end of the day for the jury. And

I'm not sure whether that's the balancing of the weight that

gets put to it or not, but I haven't sort of thought it

through.

MR. STRAZZA: I guess my position is, Judge, if we're

talking about whether something is generally accepted for an

analysis of crime scene evidence, if this issue is prevalent in

that type of situation, wouldn't that go towards whether or not

something should be will generally accepted?

THE COURT: Let me ask this, doctor. The other tools

that are used, do they somehow make or compensate or take into

account relatedness?

THE WITNESS: The CPI does not. The random match

probability can.

THE COURT: Random match probability. I understand

that, but actually what I am talking about are other programs

like FST. In other words, do you know whether they take into

account the relatedness?

THE WITNESS: I don't know whether or not they do.

THE COURT: OK, go ahead.

MR. STRAZZA: And I am sorry, I just want to make sure

I heard your Honor correctly. Was the question whether or not

some of those other programs that came up during the testimony,

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

193



whether they account for that? Was that your question?

THE COURT: Correct.

Q. And the answer is I don't know?

A. Years ago they did not. I don't know if they do now.

Q. Some of the programs that the judge is referring to didn't

exist years ago, right?

A. Right, but some did.

THE COURT: Well, when you say years ago, what

timeframe?

THE WITNESS: When I was developing FST, the LR mix

program, for example, did not account for relatives. There may

have been further work done on that program, it may do that

now, but it did not when we developed FST.

Q. How about Lab Retriever?

A. No.

Q. How about TrueAllele?

A. I believe TrueAllele can account for relatives, although it

is in a different category than FST because it does

probablistic genotyping and attempting to separate the

contributors in the mixture probabilistically, which is beyond

what FST does.

Q. And that was around when you were developing FST, right?

A. That was. I don't know if the relatedness component was

available. And at the time we were developing FST, TrueAllele

was not appropriate for low template samples.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

194



Q. Well, but wasn't relatedness one of the limitations that

was discussed and actually written about when you were

developing FST?

A. Yes.

Q. And so you mean to tell me you didn't discuss that with the

people who were using TrueAllele, and that didn't come up at

the workshops and stuff like that?

A. I believe it was mentioned at some point that TrueAllele

can do that, but I don't remember when that happened.

Q. In order for FST to be used on a sample, the sample must be

considered informative, right?

A. What do you mean by informative?

Q. Well, according to the protocols for FST there is a

requirement that the sample must be informative first, so that

was going to be my question for you.

A. I don't know what you mean by informative.

Q. OK. Did you help design the protocols for FST use?

A. Yes.

Q. Did you include the language that before a sample could be

used with the FST program that it must be informative?

A. I don't remember writing that.

Q. OK. Well, it has to be suitable for comparison, right?

A. Yes.

Q. And prior to the use of FST, OCME classified mixture

samples as included or major, right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

195



A. Um-hum.

Q. Cannot be excluded, right?

A. Yes.

Q. And that was defined as most of the known contributor's

alleles are labeled, right?

A. Yes.

Q. And then no conclusions can be drawn, right? That was

another category?

A. Right.

Q. And that was defined as several of known contributor's

alleles are not labeled but cannot be ruled out, right?

A. I guess I would make a correction. I wouldn't call it the

known contributor's alleles. I would call it a comparison

person's alleles.

Q. Thank you.

A. OK.

Q. So let's apply that to the definition and the one before

that as well, OK?

A. Yes.

Q. What is the difference between most of known contributor's

alleles labeled -- or comparison contributor's labeled -- and

several of comparison contributor alleles not labeled? Where

is the distinction there?

A. Some loci amplify better than others, so if a comparison

profiles alleles are missing at one of the robust loci, that

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

196



may not rise to the level of -- that may rise to the level of

"no conclusions can be drawn," whereas if two alleles are

missing at a less robust locus, that might be "cannot be

excluded". So, this part -- I am not an expert on that part; I

was not a case work analyst. So, the case work analysts -- who

have a lot of experience working with loci that may or may not

be more robust than another -- they are in a better position to

make these calls.

I was only making -- I was only using the calls that

case work analysts made in the validation. I did not make

those calls. The case work analysts made the calls, and then I

ran the FST and divided them up by the categories that were

given by the analysts. I was not making those calls.

Q. And in your expert opinion, are those calls subjective?

A. They are, yes.

Q. And are you aware of whether or not there were protocols in

place at the OCME for making those calls?

A. I don't know the specifics.

Q. I mean in fact what we just discussed, right, the

definitions I just read to you, are the protocols themselves,

right?

A. I don't know if those were in the case work protocols.

Because I was not a case work analyst.

Q. Would you agree that if the wrong call was made with

respect to that call, that specific issue, that a mixture might

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

197



be analyzed by -- or run through FST, that shouldn't otherwise

be? Do you understand the question? I can reword that if you

want.

A. Can you reword that.

Q. Sure. If that is not done correctly -- right? If the

analyst who is looking at more robust loci than others, if that

distinction is not made properly, that could affect whether or

not a sample ultimately gets run by FST, right?

A. Yes, yes.

Q. Do you know whether or not those guidelines or protocols

were created based upon validated studies?

A. I was not part of -- you mean to categorize cannot be

excluded, etc.?

Q. Yes.

A. I don't know.

Q. Since you weren't a part of them though, you are aware that

those guidelines or protocols were created prior to the

implementation of FST, right?

A. Yes.

Q. And so they were not created for FST, right?

A. That's true.

THE COURT: Mr. Strazza, let me ask this just in terms

of the nature of the motion. Is there -- because I know there

wasn't any briefing on this. But is there a Daubert challenge

to that determination? As I understand it, that determination

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

198



still gets made, but is there a challenge to that?

MR. STRAZZA: The issue is that when FST was validated

should it have included a new or a more appropriate protocol to

whether or not a mixture should be run in FST.

THE COURT: What you are raising is an issue relative

to FST, not an issue relative to what was done prior to that,

which I assume -- although I don't know -- Daubert challenges

had been made, and it was determined that most of the time that

evidence came in.

MR. STRAZZA: Except that FST relies upon that, and

what the witness just said is if that's done incorrectly, the

FST either is or isn't going to come into play. So the

ultimate issue that I will be arguing to the court is whether

or not that process or that issue should have been part of

FST's validation.

THE COURT: OK.

MR. MCKAY: I just note that that issue was never

raised in any prior briefing, your Honor.

THE COURT: Also, I mean, look, I don't know to what

extent I am --

The issue is --

I mean what I think the question is are those things

that experts should have looked at at the time, in other words,

those people doing the peer review, those people doing other

things. And I will have to give that some more thought.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

199



Because, as I --

Actually, doctor, do you know, was that issue raised

in the back and forth? In other words, the fact that the FST

was taking the subjective results of the -- that were generated

and then using that with the tool?

THE WITNESS: There is no inappropriate time to use

FST. It's valid for all of those categories. It's a lab

decision which samples they're going to run based on if alleles

are missing and a person is excluded, then they have chosen not

to run it on those samples. FST was validated across the

board.

MR. STRAZZA: If I may just clarify, because this is

probably going to come up in my other questions.


MR. STRAZZA: The thing is this, Judge. FST puts a

statistical weight that we're arguing should not be presented

to a jury. If there is a situation that would have -- meaning

if FST is relying on a protocol that maybe was validated for a

different purpose but not for the purpose of putting a

statistical value in front of a jury, then maybe it's something

that need to be reconsidered.

THE COURT: Well, as I understand what the doctor just

said, scientifically it was validated for each of the

categories.

What I think you're saying is that, well, did it take

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

200



into account the subjective nature of that determination in

some way.

MR. STRAZZA: Can I just give you an example? Let's

say when they were validated for different reasons, let's say

the margin of error rate was plus or minus 30 percent. Right?

And let's say that when FST was designed maybe it should

have -- and there was the 30 percent margin of error of whether

or not this evidence that's analyzed by FST is going to be

presented to the jury. Maybe the 30 percent margin of error

wouldn't have mattered presenting to a jury "could be excluded,

could not be excluded," whatever, but it sure matters when you

are putting a statistical value on that and coming up with a

number. That's my argument.

THE COURT: I understand the nature of the argument.

MR. MCKAY: Your Honor, could we have our legal

arguments after the witness is done with her testimony?

THE COURT: I'm sorry, I started this. Because I

wasn't sure again whether there was a challenge to the earlier

stuff. Because one way to -- well, never mind. Go ahead.

Continue your cross-examination.

MR. STRAZZA: I'm challenging everything, Judge.

THE COURT: No, it's just that the other stuff --

well, just go ahead.

MR. STRAZZA: I'm just finding my place.

THE COURT: Sure.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

201



Q. In order for FST to generate an accurate likelihood ratio,

there must be a number of contributors that has to be

determined, correct?

A. Yes, this is true for all likelihood ratios.

Q. For the purposes of these questions, I'm just talking about

FST.

A. OK.

Q. So in determining the number of contributors, the analysts

at OCME look at the number of labeled alleles, correct?

A. Yes.

Q. And the analyst doesn't label the alleles him or herself,

right?

A. Right, the software does.

Q. And that software is call Gene Mapper, right?

A. That's what they were using.

Q. Or was.

A. Yes.

Q. For the purposes of these questions I'm referring to when

you created FST.

A. OK.

Q. And Gene Mapper creates an electropherogram, right? That's

one of the charts that is produced after it is run through Gene

Mapper, right?

A. There is an electropherogram and allele calls, yes.

Q. And the electropherogram produces peak heights across

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

202



different loci, right?

A. Yes.

Q. And not all peak heights are labeled as alleles on that

electropherogram, right?

A. Not all peaks are labeled, that's true.

Q. And so it's only those that are above a predetermined

threshold that was set by OCME, right?

A. Yes.

Q. And that's 75 RFU?

A. Yes.

Q. And who chose that threshold?

A. That was chosen based on the validation of the software and

the equipment. There are methods published for looking at the

noise in the signal and then setting the threshold some

distance above the noise to ensure that noise is not labeled,

that only true peaks by this definition are labeled.

Q. When you say software and equipment, you are not talking

about FST.

A. That's correct.

Q. That threshold was put in place before FST, right?

A. Yes, yes.

Q. And I am assuming that -- do you know whether or not that

threshold was set based upon empirical studies?

A. I would say that it is. I was not part of those studies.

Q. Do you know though as you sit here whether or not it was?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

203



A. I have not seen it, but it is supposed to be.

Q. And were those empirical studies supposed to be validated?

A. I don't -- this is not my area of expertise.

Q. I understand. But you worked out of OCME for a period of

time, and if you can answer the question, please do; if you

can't, you can't.

A. That's outside my area.

Q. That's not what I'm asking you. Do you know whether or not

the methodology for selecting 75 RFU as the threshold was

validated? Yes or no?

A. Well, I haven't -- I have not seen the validations, so I

can't say that I know what happened in the --

Anything that's happening in the lab had to be

validated, but I have not seen the validation of this, the

selection of this threshold.

Q. And again that was created before FST, right?

A. Right.

Q. So if it were validated, it was validated for other uses

besides use with FST, right?

A. Right, for any down-stream analyses, yes.

Q. And you are aware that other labs have different thresholds

than 75 RFU, right?

A. Yes.

Q. Some are much lower, right?

A. I've heard of 50. I don't have --

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

204



Q. Have you heard of 35?

A. No.

Q. Are you aware of whether or not the software that OCME uses

now uses lower thresholds?

A. I don't know.

Q. You would agree that since the thresholds affect which

alleles are ultimately called, then the thresholds affect the

ultimate determination -- the determination of the ultimate

number of contributors that gets input into FST? You agree

with that statement?

A. Are you saying if they used a different threshold they

might come up with a different number of contributors?

Q. That's not what I was saying, but that's another -- that's

saying -- I would like to know the answer to that question.

A. It's possible. But 75 is what they use and what we used

for FST.

Q. But the threshold that is used affects -- or could

affect -- the number of contributors determined by the

analysts, correct?

A. It could.

Q. And when FST was in place, when you created it, the

protocols that the analysts were using to determine the number

of contributors that they would input into FST said that if

three alleles or more were labeled -- or if three alleles were

labeled at at least two loci that would be considered a two

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

205



person mixture, right?

A. Yes, yes.

Q. And if five alleles were labeled at two or more loci, that

would be a three person, right?

A. Yes.

Q. And if seven alleles at two or more loci, that would be a

four person?

A. I don't remember that specific one.

Q. And that's because it doesn't matter, because FST wasn't

validated to use more than three person samples, right?

A. Right, we only did three person -- two person and three

person.

Q. Was that protocol that I just described to you, was that

based upon empirical studies?

A. That that you just described is standard across the field.

Q. Do you know whether or not that was based upon any

empirical studies at OCME?

A. We did empirical studies on mixtures and looked at many

characteristics of the mixtures, and those were two of them,

yes.

Q. Before you started using that for FST, were those protocols

validated?

A. Again, I wasn't there; I don't know.

Q. You don't know? You don't know whether or not it was a

validated protocol in effect when you started creating FST?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

206



A. I was not there for that time period --

Q. I understand that.

A. -- so my answer is the same as with the analytical

threshold. I did not see that, so I can't testify to that.

Q. Dr. Mitchell, I know you don't want to testify to anything

that wasn't part of your involvement with FST, but my

questions -- I'm going to ask you respectfully to listen to the

question of whether or not you know something or not.

MR. MCKAY: Objection, your Honor.

MR. STRAZZA: Can you do that, please?

THE COURT: Well, here is -- I mean let me see if I

can get some clarity on this.

So, with regard to the validation, you hadn't actually

seen the validation papers relating to that?

THE WITNESS: Correct.

THE COURT: But based upon your experience at OCME,

and as in this field -- and by this field -- is it your

understanding that validation would have had to have -- it

would have had to have been validated?

THE WITNESS: Yes. Although I'm not sure everything

like this has to be validated. At least two loci with three or

more alleles, that's a standard definition of a two person

mixture across the field. And I don't know whether every lab

is required to empirically validate that.

Q. Would you use an unvalidated measurement to start the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

207



process of your FST program?

A. We did empirical studies on the number of contributors to

the mixtures. I don't know what they did before I was there.

Q. Are those studies included in your validation?

A. It's a published paper that I think that I saw in this

binder.

Q. What is the title of that?

A. I don't remember. Can I --


THE WITNESS: Take a look?

THE COURT: You can take a look, sure.

MR. MCKAY: If it speeds things up, she may be

referring to Exhibit 4.

THE COURT: Take a look at 4 and see if that's what

you were talking about.

THE WITNESS: Yes.

Q. Is that commonly referred to as the Perez paper?

A. Yes.

Q. Those protocols were not part of the FST validation though,

right?

A. These are characteristics of mixtures that can be used to

help determine the number of contributors to the mixture.

Q. I understand that. The question is: Was determining the

number of contributors or what's contained in that paper, was

that part of the FST validation study?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

208



A. This was part of determining the number of contributors to

the mixtures.

Q. Maybe I'm missing something. I understand that, and I

understand the results of the studies were published in that

paper.

A. Yes.

Q. But is that information contained in the validation studies

for FST?

A. Oh, no, it's not described in that same way.

Q. Because that article isn't about FST, right?

A. Right.

Q. That article was about determining the number of

contributors in general, right?

A. Yes, yes.

Q. So my question is: When you created FST -- well,

withdrawn. FST relies upon a determination of the number of

contributors, correct?

A. Yes.

Q. And in fact FST will create a likelihood ratio that is not

accurate unless a proper determination is made with respect to

the number of contributors, correct?

A. It is still an accurate comparison. The probability of

getting those alleles in the mixture if scenario one is true to

the probability of getting the alleles in the mixture if

scenario two is true, that remains valid regardless of how many

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

209



people are actually in the mixture.

Q. I know, but that's not what I asked you, Dr. Mitchell.

What I'm saying is you need to accurately determine the number

of contributors to the mixture in order to run FST, right?

A. I am saying that the calculation is still a valid

comparison even if you are wrong about the number of

contributors. It doesn't throw everything out.

Q. Let me ask, are you saying the number of contributors input

in FST is irrelevant?

A. No.

Q. But you just said that the likelihood ratio produced is

still valid if the number of contributors is wrong. Did you

say that?

A. It is still a valid comparison of the two scenarios that

you use even if the scenarios are not correct.

Q. So then why does it matter?

A. Because we do our best to come up with the best estimate

for the number of contributors that we can.

Q. Because that will affect the accuracy of the likelihood

ratio, right?

A. It may affect the relevance in a particular theory of the

case, but a likelihood ratio is still a valid calculation

comparing those two scenarios.

MR. STRAZZA: Judge, I honestly don't know if I got my

answer.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

210



THE COURT: No, I think the issue may be that this

ratio is not -- it's a probability, right? And so I think --

so while -- you know, accuracy one way or the other -- I'm not

sure if that's the way to necessarily best think about it.

In other words, so there is a range of possibility.

So, for example, if this was something that you could

empirically prove, then using the different tools should get

you just about the same results. But what is happening here,

as I understand it, is it is -- so whether it's two

contributors or three contributors, that's a probability. Am I

correct about that?

THE WITNESS: We do our best to determine whether it's

two or three contributors.

Q. But it's my understanding most respectfully that a

likelihood ratio is not a probability.

A. It is a ratio of two probabilities.

Q. But here is the thing. What if you determined that there

is four contributors to the mixture? Are you going to come up

with a likelihood ratio?

A. We didn't validate the four person mixtures in FST.

Q. So the answer is no, right?

A. Right, not with FST right now, no.

Q. So then it would seem like properly determining the number

of contributors would be relevant, to say the least, in getting

an accurate likelihood ratio.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

211



MR. MCKAY: Your Honor, I think this has been asked

and answered several times.

THE COURT: No, I will allow it.

You can answer, doctor.

A. As I said, the comparison remains valid whatever scenarios

are put in there. It may not be relevant to the case, or OCME

had decided not to go forward with the four person mixtures.

So, obviously if something was a four person mixture, that

would make a difference if it's a three person or a four person

mixture because it would be run if it was a three person and

not run if it was a four-person.

Q. So being that it's relevant in that scenario, my question

is: Was the process for determining the number of contributors

part of the FTS validation? Yes or no.

A. No.

Q. So when the process of determining the number of

contributors was validated, it was validated for a different

purpose other than to input that figure into FST.

A. Its intended purpose was for a variety of --

Q. Other than to input into FST?

A. No, including FST.

Q. That's not true, doctor, based upon what you just said,

because you said it was created -- it was validated before you

created FST, right?

A. The guidelines that you mentioned, that three alleles at

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

212



two or more loci would be called a two person mixture, that is

not something I validated on its own. I looked at all kinds of

characteristics of two, three and four person mixtures.

Q. OK, I'm going to move on.

THE COURT: OK.

Q. Would you agree that determining the exact number of

contributors to a mixture is impossible?

I will word it differently. I see your face. I will

word it differently. My apologies.

Would you agree that it's impossible to know whether

or not the analysts determined the precise number of

contributors to a mixture?

A. In a case work sample, yes.

Q. And so the analyst's conclusion at that point is just an

estimate -- is just an estimation, right?

A. It's an estimate using all of these tools that we have.

Q. And using all of those tools that you have, it's often an

underestimate, right?

A. I don't know, because I don't know the true number of

contributors.

Q. I'm going to read you a quote and tell me if you agree with

this quote or not.

"Samples must be categorized as a single source for

mixtures, and for mixtures the number of contributors must be

estimated. Characteristics of a mixture can be used to

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

213



determine whether to treat the mixture as two, three or four

people, yet due to allele sharing amongst related and unrelated

individuals, there will always be a level of uncertainty to

this determination. Using only the maximum number of alleles

observed at any locus to estimate the number of contributors to

a mixture will often lead to an underestimate."

Do you agree with that?

A. OK, that is one -- yes, that is one particular tool for

estimating the number of contributors, yes.

Q. OK. Isn't that exactly how OCME estimates the number of

contributors?

A. They use several characteristics of the mixture to estimate

the number of contributors.

Q. In addition to doing that, what else do they do?

A. There are other characteristics; they are listed in this

paper.

Q. Well, you wrote the paper, right? You were one of the

authors on the paper?

A. Yes, I'm one of the authors.

Q. So, I'm just asking you what are some of the other things

that OCME does.

A. So, they can look at the total number of alleles labeled in

a mixture. They can look at the number of loci where there are

four or more labeled alleles. For example, sometimes a three

person mixture doesn't have any loci with five alleles but it

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

214



has many loci with four alleles. So, there are other

guidelines in here that are not just about the maximum number

of alleles that appear at a locus.

Q. And are those guidelines required for determining a mixture

that gets used -- the number of contributors to a mixture that

gets used by FST?

A. Yes.

Q. Is that required?

A. That's what's used, yes.

Q. Those are the guidelines that the analysts must follow at

OCME before determining the number of contributors for FST?

MR. MCKAY: Your Honor, if I could object.

Mr. Strazza specifically objected to Dr. Mitchell's expertise

to qualify about case work at FST. We have brought in another

witness who has extensive case work experience. Some of these

questions may be better posed to that witness.

THE COURT: I guess, doctor, if you know.

In other words, are you asking what the protocols are

that they would use?

MR. STRAZZA: I mean Dr. Mitchell wrote a paper on

what should be done. Dr. Mitchell created FST. So the

question is -- and we will get to this -- but she also trained

the analysts to use FST. So, the question is: Are they

required to use that protocol that she just described to

determine the number of contributors that they use with FST?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

215



A. Yes, there are guidelines in the OCME case work protocols

describing the findings from this paper to help the analyst

estimate the number of contributors.

Q. Was that information contained in the validation studies

for FST?

A. It's a separate validation where we looked at the number of

contributors. It is not part of the FST validation, this

examining the characteristics of two, three and four person

mixtures.

Q. So the answer is no.

A. I guess, the way that you're asking the question. It's not

part of the FST validation; it's a separate validation.

Q. When you created FST, you were aware of the issue of

underestimating the number of contributors, right?

A. That can happen, yes.

Q. And you're OK with it because in your opinion that's

conservative, right?

A. That's anecdotally what we saw, yes.

Q. And in your opinion that's conservative, right?

A. What we saw is knowing the true number of contributors to

the mixture, if we ran FST using that number of contributors,

generally for a true contributor we got the highest LR. If we

ran it with fewer contributors or more contributors, it didn't

fit as well and it didn't generate as high an LR for a true

contributor as it did when we got the number right.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

216



Q. And are those studies contained in the validation

materials?

A. No.

Q. Where are the results of those studies?

A. Those were anecdotal analyses that we did that are not

contained in the validation.

Q. Are we able to view those anecdotal analyses?

A. No, but you could run them now because FST is open source.

Q. Let's talk about that very briefly.

A. OK.

Q. FST -- how long has FST been open source?

A. About two weeks I think.

Q. And that all came about after OCME tried to prevent FST

from trying to become open source, right?

A. The legal department did not want it to be open source,

that's correct.

Q. And when you say open source, did OCME put it online for it

to be open source?

A. I don't know.

MR. MCKAY: Objection, your Honor.

THE COURT: I will allow it. It doesn't matter.

Q. Do you know whether or not OCME made it public?

A. I don't know the steps of what happened.

Q. Underestimating the number of contributors as applied to

FST is not always conservative to the suspect, is it?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

217



A. I have not done rigorous studies to show that, no.

Q. And without doing a study, if a four person mixture is

determined to be a three person mixture, and the likelihood

ratio -- withdrawn.

Without doing studies, if a four person mixture is

determined to be a three person mixture, it would get run in

FST when it otherwise shouldn't be, right?

A. If it were determined to be a four person mixture it would

not be run in FST; if it were determined to be a three person

mixture, it would.

Q. And if that likelihood ratio was used against a defendant

at a criminal trial, that wouldn't be conservative, right?

A. Again, the models are still valid. The three person

comparison in the numerator and the three person comparison in

the denominator are still valid comparisons.

If it truly were a four person mixture but it met our

criteria for a three person mixture, we have no way of knowing

that it was truly a four person mixture.

Q. OK. But that could create a situation that's not

conservative for the suspect, right?

A. It could create a situation where there is a likelihood

ratio presented where in other circumstances it would not be.

Q. And that would not always be conservative to a suspect,

right?

A. What do you mean by conservative?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

218



Q. I will move on.

A. OK.

Q. The number of contributors to a mixture becomes more

uncertain as the number of contributors increases. Is that

correct?

A. Yes.

Q. And that's due to smaller amounts of each contributor's DNA

being part of the mixture, right?

A. I would say it's due to a higher chance of sharing alleles

when you have more people in the mixture.

Q. And allele sharing is when the different contributors to

the same mixture have the same alleles, so when the results are

analyzed certain contributors' alleles will be masked or

blocked out or won't come up separately, right?

A. Yeah, there would be some overlap.

Q. So they wouldn't show -- right. So you wouldn't --

A. There might be one peak that includes DNA from two

different people.

Q. So that would include a smaller amount of the number of

labeled alleles, right?

A. Smaller than what?

Q. Smaller than if all the true alleles were called.

A. Right, if all the contributors were heterozygous and they

all had different alleles, yes, that would be the maximum

possible number of alleles, yeah.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

219



Q. And obviously that could affect the determination of the

number of contributors then, because there will be less alleles

at a given locus, right?

A. Yes, but we looked at those patterns, so we know that that

happens.

Q. That wasn't my question.

THE COURT: Counsel, let the doctor finish answering,

and then you can ask your question.

MR. STRAZZA: OK. We might have a time problem,

Judge, I'm just saying.

THE COURT: Well, whatever it will be, it will be.

Q. Are you familiar with John Butler?

A. Yes.

Q. Who is John Butler?

A. He is a scientist at NIST.

Q. Well respected in the community?

A. Yes.

Q. Are you familiar with his cautionary warning regarding the

uncertainty of determining the number of contributors?


Q. Yes. Are you familiar with John Butler's statement that

less than 50 percent of all four person mixtures would leave

seven alleles at two loci?

A. I don't remember the specific numbers.

Q. Are you familiar with his position generally about the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

220



uncertainty in determining the number of contributors?

A. I'm not well versed in it. I know that it's been

mentioned.

Q. You know that he's written about it, right?

A. Yes.

Q. Would you agree with the fact that certain analysts may

disagree when determining the number of contributors to a

mixture?

A. Yes.

Q. And are you ever aware of that happening before?

A. Yes.

Q. Does it happen quite often?

A. I don't know.

Q. Are you aware -- are you aware of it happening often?

A. I am aware of at least two -- I'm aware of two

circumstances. I don't know what often means.

Q. Are you aware that there is a protocol in effect for the

situation when an analyst disagrees with another analyst in

regards to determining the number of contributors to a mixture?

A. I am aware that a protocol exists, yes.

Q. And so if a protocol exists, that's because it comes up,

right?

A. I'm sorry? I said --

Q. Why are protocols created?

A. I said I know of two specific instances where two people in

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

221



the lab disagreed about the number of contributors.

Q. Do you know why they disagreed about the number of

contributors?

A. I don't know the details.

Q. Could drop-out affect -- could drop-out -- could the

presence of drop-out cause a disagreement between analysts when

determining the number of contributors?

A. Can you --

Q. Could the phenomena of drop-out affect whether or not one

analyst thinks it's one number and another analyst thinks it's

another number of contributors?

A. One analyst could think it was more contributors with more

drop-out. Is that what you mean?

Q. Yes.

A. Yes.

Q. How about drop-in, could that affect a determination of the

number of contributors?

A. Yes.

Q. And how about allele stacking, could that affect the

determination?

A. Sharing of alleles?

Q. Yes.

A. Yes.

Q. How about stutter, could that affect?

A. It could, although it's generally not as much of an issue,

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

222



I think.

Q. And how about a degraded sample, could a degraded sample

affect the determination of the number of contributors?

A. Possibly. With a degraded sample you have less

information.

Q. Why is that? Can you explain why that is, please?

A. So the way that the forensic loci are set up, they go from

shorter to longer amplified pieces, and when a sample is

degraded the DNA is broken apart, so the longer amplicons might

be too degraded to show up. So what the profile looks like is

there may be results in any row in the electropherogram,

results at the first locus or loci, and then fewer results as

you go left to right in the electropherogram.

Q. Does that create what is called a slope effect?

A. It can, yes.

Q. And so there is more drop-out at the --

A. At the larger loci, yes.

Q. Does FST account for that?

A. FST does not explicitly model degradation, but degraded

samples can be used with FST.

Q. Well, first off, would you agree with me that degradation

is an issue with respect to crime scene samples?

A. Sure.

Q. And why would that be?

A. Degradation is often caused by environmental factors like

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

223



exposure to sun or high temperatures.

Q. And how about with touch samples, would degradation be more

prevalent in touched samples?

A. It can be.

Q. What's a touched sample?

A. A touched sample is an evidence item that does not contain

blood or saliva but is obtained -- DNA is obtained from

something that has been touched.

Q. And so --

A. I guess it could contain blood or saliva, but it's

something, an evidence item that has been touched.

Q. So with touched samples and degradation, you would agree

that the drop-out rates are different for those samples, for

those true samples, than other samples, for example, buccal

swabs or things like that?

A. Yes, they can be.

Q. Can be? Is it quite common?

A. Yes.

Q. But FST doesn't account for that?

A. It does not explicitly model it, but key degraded samples

can be used with FST.

Q. In the same fashion that nondegraded samples can be used.

There is no determination made, right?

A. Right.

Q. So, the FST worksheet has changed over time, right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

224



A. What do you mean by the worksheet?

Q. The sheet that's created that shows the different variables

that were input into the running of the program for that

specific sample.

A. The result sheet? Yes.

Q. Yes.

A. OK.

Q. That's changed over time, right?

A. Yes.

Q. When FST first started there was a section for whether or

not a sample was degraded or not degraded, right?

A. That did get printed out on the results sheet, yes.

Q. But in fact there was no way for the analyst running FST to

know whether or not the sample was actually degraded or not,

right?

A. No, that's not true. But it wouldn't be run any

differently if it were degraded or not.

Q. It was meaningless, right?

A. The reason it was there was that we had added a model for

degradation, and we were attempting to create better separation

between true contributors and noncontributors for degraded

samples, but we were not able to do that, so it was not

enhancing the program; it was not increasing the separation

between true contributors and noncontributors to model

degradation.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

225



Q. And you conducted studies when you were trying to

incorporate that?

A. We did. We did.

Q. Where are those studies?

A. In the validation.

Q. And specifically where? Which studies are those?

A. There is a section on degraded samples.

Q. And it shows what the likelihood ratios for the degraded

samples were versus likelihood ratios for nondegraded samples?

A. Yes, there is a section of degraded samples specifically.

Q. I'm asking is that what it shows? It shows the difference

in the likelihood ratio between a degraded sample versus a

nondegraded sample?

A. It shows the likelihood ratios for the degraded sections,

and there are other sections for nondegraded samples.

Q. But it shows -- can you point me towards a graph, a table,

a chart that compares the two that form the basis for your

decision not to include it in the FST?

A. I don't have such a table or chart.

Q. But it's your testimony that you made those comparisons?

A. We did, yes.

Q. You made the comparisons without documenting them?

A. So when I was -- when we were developing FST, anything that

is going into the final version of the program is in here. We

did not include things that we didn't do or we didn't put in

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

226



the final version. This is a validation of FST as it is.

Q. OK. You didn't include them, but they exist somewhere.

A. I don't know where, but yes.

Q. Do they?

A. I can't tell you that. I don't know.

Q. Did you create them?

A. I did.

Q. Did you throw them out?

A. No.

Q. Did you save them?

A. They're somewhere on the computers that I used to use at

OCME. I did not create a formal section documenting that

decision, but --

Q. I know that. But what about the data, what did you do with

the data?

A. It's in here. The degraded samples are documented in here.

Q. But the comparison data, what did you do with that?

A. I did not create formal tables. I have already said I did

not create formal tables of these comparisons.

Q. You just looked at the --

A. I would have to look back and see exactly what is where,

but the bottom line was we did not improve the performance by

modeling degradation, so we discussed this with the

subcommittee and we decided not to model degradation in the

program.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

227



Q. But FST is based upon estimated drop-out rates, right?

A. Yes.

Q. And everybody knows that with degradated samples, the level

of drop-out is higher, right?

A. Yes. And I refer back to my presentation about

underestimating the drop-out rate favors a defendant who is not

a contributor and lowers the likelihood ratio below what you

would get if you correctly specified the drop-out rate for a

true contributor.

Q. OK, but it doesn't favor all defendants, right?

A. I have explained those circumstances.

Q. And if we're using this in court, we're concerned about all

defendants, right? You said that was a concern of yours

putting a more accurate number to this, right?

A. I don't understand what situation you're saying would be

unfavorable.

Q. Any situation that's unfavorable to a defendant.

A. I presented our decision with the data supporting why we

want to underestimate the drop-out probability, and degradation

fits right into that.

Q. But let's talk about that for a second. On direct

examination you talked about there being validation studies

that fill up 20 plus binders?

A. Yes.

Q. And then summaries of those binders that are hundreds of

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

228



pages, right?

A. Yes.

Q. Do you really think that the members of the subcommittee

are going to read all of those binders?

A. Oh, certainly not, no.

Q. So just because something may be contained in a binder

somewhere, I mean that's meaningless unless you formally

present the issue to the subcommittee, right?

A. Right. And we did talk about degradation with the

subcommittee.

Q. But you didn't compare the results of degradated samples to

nondegradated samples to them. You didn't create -- you didn't

show them what the difference in the numbers were.

A. It's not a comparison of degraded versus nondegraded that I

think is important here. It's a comparison of if we increase

the drop-out rates that we're using when we determine that a

sample is degraded, are we able to get higher likelihood ratios

for true contributors and lower likelihood ratios for

noncontributors.

Q. And it's your position that you made those comparisons, you

didn't write anything -- or you don't have the written results

of those comparison, but based upon that that's what you made

your decision on?

A. Yes.

Q. What is a high voltage injection?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

229



A. It is a method for increasing the peak heights of a sample.

It increases peak heights.

Q. OCME has two separate protocols for high voltage

injections, right?

A. I think that there are more than two.

Q. Well, they have a separate one for high template samples

and low template samples, right?

A. Yes.

Q. And OCME's cut-off for the difference in between those two

is 200 picograms, right?

A. I don't know.

Q. As you sit here today, is it your testimony that DNA

samples -- that you are not aware of whether or not DNA samples

less than 200 picograms should be injected with a high voltage

injection pursuant to OCME's protocols? Is that your

testimony?

A. I am unaware of the protocol.

Q. OCME validated a method for determining the amount of

sampled DNA amplified in a given sample, correct?

A. Yes, or to be amplified. To be amplified. It's

preamplification.

Q. Sorry?

A. It's the amount of DNA before the amplification.

Q. Right. And that method was created based upon empirical

studies, right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

230



A. The method was created by Eric Buhl and one of his

colleagues. It's someone who is currently on the DNA

subcommittee, or was when we presented FST. He published the

method, yes.

Q. But when OCME decided to use that as part of their

protocols, it was based upon empirical studies, correct?

A. I don't know what you mean by based upon empirical studies.

Q. Do you know when OCME validated the protocols for that?

A. Before I was there.

Q. And so those protocols were not validated with the use of

FST in mind, correct?

A. No, they were just quantifying the amount of DNA in the

sample.

Q. Right, because FST wasn't created yet at that point, right?

A. Right. Of course they didn't know about it.

Q. And so when they were validated, they weren't validated for

the purposes of generating a likelihood ratio, correct?

A. They were validated for the purposes of quantifying the

amount of DNA in a sample.

Q. Is that your way of saying no?

A. It's my way of saying it's not specific to what happens

after that. There is a requirement, a SWGDAM and F.B.I.

requirement, that an attempt be made to quantify the DNA in a

sample to be used for forensic analysis.

Q. Right. So therefore at the time those methods were

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

231



validated they weren't being validated for the purposes of

creating a likelihood ratio, right?

MR. MCKAY: Objection. Asked and answered, and she

said she wasn't there at the time they were validated.

THE COURT: I think I understand, Mr. Strazza. You

can move on.

MR. STRAZZA: Thank you.

Q. And that method for determining the quantity or quant is

just an estimate, right?

A. Yes.

Q. In fact it's an estimate where 30 percent margin of error

has been calculated for that estimate, right?

A. On average, at the extremes, yes. In the middle of the

range, the dynamic range, it's tighter.

Q. I mean it's common knowledge that there is a 30 percent

margin of error in terms of quantification at the OCME, right?

MR. MCKAY: Objection.

A. Yes, that's often stated, but I'm clarifying.

THE COURT: Overruled.

A. I'm clarifying that the 30 percent is at the ends of the

dynamic range, so the very low samples and the very high

samples. In the middle it's not quite that variable.

Q. OK. But there are times where a quant value is determined

and it's actually 30 percent less than the true sample, right?

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

232



Q. And there are times when the quant value is determined and

it's 30 percent higher than the true sample, right?

A. Yes.

Q. And that's 60 percent swing, right?

A. 30 percent each way, OK, yes.

Q. And there certainly exists more accurate ways of

determining quant, right?

A. There is, yes.

Q. And quantity is measured by incorporating a florescent dye

into the DNA mixture and measuring the amount of florescence

produced, right?

A. Yes.

Q. And so when you are conducting regular PCR testing not for

use with FST you are trying to determine a quant value -- you

are trying to determine -- withdrawn.

When you are conducting a regular PCR testing and

you're not using it for purposes of FST, the exact size of the

sample isn't that important, right?

A. So the quant helps determine how much of the sample to put

in the amplification reaction.

Q. But you can make adjustments to that and fix it right away,

right?

A. What do you mean?

Q. You can dilute the sample if it's too high, or you can say,

look, there is not enough to test, right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

233



A. Yes. And that goes into the amplification, whose results

can be used for a variety of things. It's really to determine

how much DNA to put in the amplification.

Q. But what I'm getting at is the reason why a 30 percent

margin of error doesn't matter is because if it's not being

used for something like FST, the size doesn't really matter,

right?

A. No, it does matter.

Q. It's not as important.

A. I can't really answer that.

Q. When this method with a 30 percent margin of error rate was

validated, it was validated for use where a 30 percent margin

of error would be acceptable, right?

A. Sure.

Q. It wasn't validated for use with FST, right?

A. It doesn't make sense to me to think of validating a

laboratory measure for use with FST.

Q. OK, I'm going to try and explain it.

A. OK.

Q. Let's say the rubber that I want to use to make a tire has

been validated for use with bicycles. If it were known at the

time I was going through that validation process that I was

going to use that rubber to make a race car where it was going

to be going 200 miles an hour, it might not get validated for

that purpose, right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

234



A. OK.

Q. When you say you don't -- that's what I'm getting at.

So what I'm getting at are the differences for which a

method was validated.

A. They validated it as a way of quantifying the DNA in a

sample.

Q. Right. Not for the purposes of computing a likelihood

ratio.

A. It's for the purposes of estimating the amount of DNA in a

sample.

Q. But if they get it wrong to the tune of a 30 percent margin

of error, it's not going to create something so significant as

determining somebody's innocence or guilt, right?

A. I'm not sure what the question is.

Q. I'll move on. Would you agree with me that having a broad

range within that 30 percent margin of error is fine for the

terms of regular PCR testing as long as the machine can take

that to amplify? Do you agree with that statement?

A. As long as the machine can take that --

Q. As long as it's a sufficient sample that the machine can

amplify.

A. That PCR can be done?

Q. Yes.

A. OK.

Q. Now, with FST, the quant value is used to determine the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

235


HB6HJON7 Mitchell - Cross

applicable drop-out rates that the program will apply, correct?

A. Right.

Q. And an inaccurate quantity amount will lead to an

inaccurate drop-out rate, right?

A. It's an estimate. It's our best estimate. It may not be

God's truth, but it's our best estimate.

Q. But if the wrong quantity amount is put into the FST

program, you're going get a less accurate likelihood ratio,

right?

A. If the drop-out rate is -- if the drop-out rate is not -- I

need to take a break. Can I take a five minute break and use

the bathroom?

THE COURT: Yes, sure.

All right. We will take five minutes.

(Recess)


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

236



(In open court)

THE COURT: Mr. Strazza.

BY MR. STRAZZA:

Q. Before you can run FST, there are certain values that have

to be input into the program by the analyst; is that correct?

A. Yes.

Q. One of those values is the number of contributors; right?

A. Yes.

Q. One of those values is the quant value; right?

A. Yes.

Q. And another one of those values is whether or not it's a

deducible mixture; right?

A. Correct.

Q. Is there anything I'm leaving out?

A. They put in the profiles --

Q. Right.

A. -- relevant to the case, but yes.

Q. When you created FST, you implement or you estimate

drop-out rates for certain scenarios; right?

A. Yes.

Q. There are different drop-out rates -- withdrawn.

When you created those rates to be used in FST, you

tested over 500,000 -- you made over 500,000 tests; right?

A. That was not in creating the mixtures for drop-out testing,

no.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

237



Q. How many tests did you do for the mixtures for drop-out?

A. I would say upward of 1,500.

Q. 1,500, let's say. And some of those mixtures were

two-person, yes?

A. Yes.

Q. Some of people were two-person?

A. Yes.

Q. Some of those mixtures were three-person?

A. Yes.

Q. Some of those mixtures had smaller amounts of quant than

others?

A. Yes.

Q. Different values, as a matter of fact; right?

A. Yes.

Q. Some of those mixtures were deemed deducible and some of

them were deemed non-deducible; right?

A. Right.

Q. So the drop-out rates that you ended up with after you

conducted your study were different depending upon this those

different factors; correct?

A. Yes, they can be different.

Q. And so, for example, FST will use a different drop-out rate

for a two-person mixture at one locus than it would for a

three-person mixture at one location; right?

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

238



Q. And FST will use a different drop-out rate for 50 picograms

of DNA than it would for 250 picograms of DNA; right?

A. They may be the same sometimes, but, yes, there's the

opportunity for them to be different.

Q. So my question for you is determining the one number of

contributors over another will ultimately affect which drop-out

rate that gets applied; right?

A. Yes.

Q. And the drop-out rate that gets applied will ultimately

affect the likelihood ratio that gets created; right?

A. Yes.

Q. Back to quant. Your method for determining the quant that

gets input into FST has that 30 percent margin of error rate;

correct?

A. Correct.

Q. And despite this limitation, you still chose to link

quantitation to drop-out rates?

A. Yes.

Q. Just to make sure I understand this, OCME was the only lab

in the world to do this; right?

A. To do which part?

Q. To link the quant value to the drop-out rate.

A. To my knowledge.

Q. There are other ways to incorporate the drop-out and

drop-in into likelihood ratios; right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

239



A. Yes.

Q. And the other programs that you mentioned earlier, they

don't do that; right?

A. Don't do what?

Q. Link drop-out rate to quant.

A. They do not use quant to predict drop-out rate, that's

correct.

Q. Are you aware of any other scientist in the world that does

in determining -- are you aware of any other scientist in the

world that uses that method?

A. No.

Q. Mixture ratios, the analyst has to choose a mixture ratio

before inputting that -- before running FST; right?

A. It's just a categorical decision of deducible or

non-deducible.

Q. And deducible is where you can determine there's a major

contributor; right?

A. Yes.

Q. And in that situation, one set of drop-out rates gets

applied; right?

A. Yes.

Q. And then in the other situation where you can't determine

the major contributor to the sample, another set of drop-out

rates would get applied?

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

240



Q. For three-person samples, that ratio for a non-deducible

mixtures is one to one to one; right?

A. Those were the mixtures that we used to estimate the

drop-out probabilities, yes.

Q. Those are the ones that FST used for their program; right?

Not when you created the drop-out rates, but that's what FST

uses; right?

A. I don't understand.

Q. When you did your test to create the drop-out rates, you

used different choices beside non-deducible and deducible;

right?

A. No.

Q. When you did your testing on the samples, didn't you test

drop-out rates for different mixture ratios other than one to

one to one and one to one to five?

A. No.

Q. Then I want to clear this up, because then I'm

misunderstanding something.

When you did your studies to validate FST --

A. The validation or the drop-out rate estimation?

Q. There's the confusion on my part. The validation.

A. Yes.

Q. You used different mixture ratios other than one to one to

one and one to one to five for three-person mixtures?

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

241



Q. And why was that important?

A. Because in casework, mixture ratios are not always one to

one to one or one to one to five, for example.

Q. So in a true case sample, that would actually be rare with

respect to a true case sample, right, one to one to one?

A. I don't know.

Q. In your expert opinion, if we're talking about a touch

sample, would you think that it would be common to have a

mixture ratio of one to one to one?

A. I don't know.

Q. You have no opinion on that?

A. Three equal contributors?

Q. For a touch sample.

A. I don't know.

Q. How about for a degraded sample?

A. I don't know.

Q. But in any event, it doesn't matter because FST for

three-person mixtures only uses one or the other; right?

A. Yes, one or the other rates are selected, yeah.

Q. One way to more accurately determine a mixture ratio would

be to look at peak heights for this purpose; right?

A. The analyst does look at peak heights in order to determine

whether a mixture is deducible or not.

Q. But in terms of assessing the more precise ratio, the

analyst does not do that with FST?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

242



A. No, an exact ratio is not estimated.

Q. But you could do that; right?

A. Probabilistic genotyping software does that. FST does not

do that.

Q. In your opinion, is probabilistic genotyping software more

accurate or create a more accurate likelihood ratio than FST?

A. It's more powerful.

Q. Is it more accepted within the scientific community?

A. I don't know.

Q. FST uses the drop-out rates for the total quantity of the

mixture; right?

A. It's based on the total quantity and the number of

contributors. So in a sense it's divided up among the

contributors.

Q. But it doesn't have a measurement for the quantity of each

individual contributor; right?

A. No.

Q. That would create a more accurate drop-out rate; right?

A. No, I'm saying it does implicitly take that into account.

Q. No, I know. That's not what I'm asking you. I know how it

implicitly does that, either the one to one to one or one to

one to five.

But if there was a way to account for a more precise

quant value for each particular contributor, that would lead to

a more accurate drop-out rate; right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

243



A. Perhaps.

Q. Do you ever do any studies on that?

A. No.

Q. In your expert opinion, do you think that's more probable

than not?

A. What's more probable than not?

Q. That you would get a more accurate drop-out rate if you

knew the quantity of the specific contributor as opposed to the

total quantity.

A. I don't know that it would be better, no.

Q. What do you think?

A. I think that -- I'm sorry. Rephrase the question, please.

THE COURT: She said she didn't know, so I'm not sure

whether -- in other words, I'm not sure what the "what do you

think" means if she didn't --

MR. STRAZZA: She's an expert, Judge.

THE COURT: Then you can then pose the question

hypothetically. I'm not sure if she's going to be able to

answer the question.

Q. FST assigns the same drop-out rate to all of the

contributors to the sample after it determines whether or not

it's one to one to one or one to one to five; right?

A. For the one to one to one samples, yes.

Q. Right. In that situation, there might not be an even -- in

a non-deducible mixture there might not be an even mixture

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

244



ratio in a crime scene sample; right?

A. Right.

Q. So the drop-out rate by each contributor to that sample

would be different?

A. Yes, but on average they would be the same.

Q. So you're saying that determining the drop-out rate for

each contributor to a mixture is the same as -- and then

determining it for each one and then averaging it will get you

the same number that determining the drop-out rate for the

total quant of the mixture will get you? It will get you the

same exact number?

A. The total quant divided by three would give you the average

of the three contributors.

Q. I understand that. But there exists the possibility that

one of the contributors to that sample in a true casework

sample contributed less than that; right?

A. Yes.

Q. So different drop-out rates would be applied by FST than

really should be applied, right, or that would really happen;

right?

A. We're always estimating drop-out probability. We never

know the precise truth. There may be some that are a little

higher; some that are a little lower. We estimate our best

estimate of it. And it would be fine to wish for perfect

understanding of the mixture in total, but it may not be

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

245



possible.

Q. Would you agree that the likelihood ratio created by FST is

an estimate based upon an estimate based upon an estimate?

A. There are several factors that are estimated that go into

the calculation, yes, and it is an estimate which -- whose

production required several estimates. This is not exclusive

to FST.

Q. And some of those estimates take it another step and rely

on more estimates; right? So it's not just one estimate

relying on an estimate?


Q. Sure. You estimate the number of contributors; right?

A. Yes.

Q. To determine estimated drop-out rates; right?

A. The number of contributors is part of the estimating

drop-out rates, yes.

Q. But the estimated number of contributors determines what

estimated drop-out rate gets applied; right?

A. Yes, yes.

Q. And the estimated quant value determines what estimated

drop-out rate gets applied; right?

A. Yes.

Q. And the estimated mixture ratio determines what estimated

drop-out rate gets applied; right?

A. Yes.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

246



Q. And that happens across each different locus; right?

A. Yes.

Q. And then they all get multiplied by one another; right?

A. The likelihood ratios do get multiplied across the loci,

yes.

Q. So if you have a small difference that could -- a small

difference in the numbers for one of those locus, when you

multiply that by another, by another, by another, a small

number could lead to a significant difference in the likelihood

ratio; right?

A. There is some uncertainty in each parameter, but it doesn't

mean that they're all amplifying in one direction. They're in

a sense averaging each other out. You might overestimate one;

another one would be underestimated. It's a -- they are not --

I'm waiting for him to stop talking.

They are not synergistically amplifying the likelihood

ratio.

Q. But if you get the wrong estimate for one, you're going to

get the wrong estimate for the other?

A. Not necessarily, no.

Q. OK. Are you familiar with the other programs that were

discussed that create likelihood ratios in the forensic

community?

A. Yes, somewhat.

Q. Are you familiar with the term semicontinuous versus

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

247



continuous?

A. Yes.

Q. What does that mean?

A. Semicontinuous means something like FST where we take the

allele calls out of the gene mapper software and compute a

likelihood ratio on that. A fully continuous model is also

estimating the probability of each noncontributor genotype at

each locus.

Q. So it considers more information?

A. It does.

Q. And based upon your experience with research, considering

more information will lead to a more accurate result, right,

generally speaking, like you say?

A. It's a more powerful tool. A probabilistic genotyping

software is more powerful than FST because it's using more

information.

Q. But generally speaking, if you have more information to

consider, you're going to get a more accurate result; right?

In any kind of research that you've done, if you have more

information to consider, you're going to get a more accurate

result; right?

A. It's possible to a point.

Q. So if somebody were to say that the difference between FST

and a fully continuous -- and a more fully continuous model

were like the difference between a paper map and Google Maps,

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

248



would you agree with that generalization?

A. No, I can't make a comparison like that.

THE COURT: That's fine. He can take it.

Q. That would be silly; right?

A. It would be hard to say whether that's an accurate

comparison.

THE COURT: Sometimes things need to be dumbed down

for people, but I'm sorry. Go ahead.

Q. Just bear with me. Let's go with that for a second. With

a paper map, right, if you were trying to determine how long it

would take you to get somewhere, you'd have to look at the

distance -- you'd be able to see the distance between two

points; right?

A. Sure.

Q. You'd have to look at the scale on the map to see what that

distance represents; right?

MR. McKAY: Your Honor, I think the witness has not

agreed with this analogy. I think this cross-examination can

be used with another witness. And it's not clear to me how

it's relevant to Dr. Mitchell's expertise in FST in particular.

THE COURT: I understand what -- let's move on to

another topic.

MR. STRAZZA: OK.

Q. What made you choose quantitation as a means to generate

the drop-out rates?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

249



A. There's general consensus that drop-out probability is

based on the amount of DNA in a sample. The methods that use

peak heights are using the heights of the peak as a proxy for

the amount of DNA in the sample. So we chose to directly

measure the amount of DNA in the sample.

Q. When you say there's a "general consensus," why do you say

that?

A. It's kind of a known -- a known fact that up to a point

more DNA in a sample has a lower probability of dropping out.

Q. But there must be a reason why no other lab in the world

uses that to determine their drop-out rates even though --

A. Well, there aren't actually that many labs applying

likelihood ratios with drop-out and drop-in in a semicontinuous

fashion.

Q. Does that mean that that method is not generally accepted

within the scientific community?

A. No, it doesn't.

Q. But everybody else chooses to do something different?

A. A lot of labs are not doing anything or are still using a

combined probability of inclusion, and many labs are moving to

the probabilistic genotyping software because those are

commercial products that are available, and that is easier for

a lab to take on than a method that -- like, for example, Lab

Retriever that was developed by an academic and does not have

product support that goes with it.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

250



Q. Would the same be true for FST?

A. Well, FST wasn't available to other labs until now.

Q. Are you aware of any other labs that are interested in

using FST?

MR. McKAY: Objection.

THE COURT: Well, I'll allow it. You can answer if

you know.

A. When we were developing it, there was interest from other

labs in using it, yes.

Q. Did they ultimately use it?

A. No. It wasn't released.

Q. Now that it's released, are they pursuing that?

A. I don't know. I don't really have contact in that area

anymore.

Q. But you're on the subcommittee for validating --

A. Probabilistic --

Q. -- validation studies?

A. For probabilistic genotyping software, yes.

Q. So you're dealing with the same players in the community;

right?

A. I'm dealing with some people that are in public labs, but

others who are academics or are at NIST.

Q. By the way, your work as part of that subcommittee, part of

that work is preventing others from making the same mistakes

that OCME made during the validation process; right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

251



A. Excuse me?

Q. Your work as part of the validation subcommittee, part of

your work there is to prevent or is to learn from the mistakes

that OCME made during its validation process; right?

A. No.

Q. Are you saying OCME's validation process was perfect?

A. No.

Q. So there were some things that, if you could do differently

now, you would; right?

A. The validation guidelines that we're developing are not

based on my assessment of mistakes that OCME might have made.

Q. They're not based upon your experience at OCME?

A. They are not based upon correcting any mistakes that might

have been made by OCME.

Q. In your work there, though, if you are aware of a method

that would work better than the one you used at OCME, you would

certainly advise them on that; right?

A. What do you mean by "method that would work better"?

Q. A method of validation, the way you conducted your

validation studies or whatever you're trying to accomplish on

that committee.

A. Nothing that has gone into that document is correcting an

alleged wrong by OCME.

Q. OK. That's not really what I asked you.

Your value to that subcommittee in part is based upon

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

252



your experience at the OCME --

A. Yes.

Q. -- during its validation process; right?

A. Yes.

Q. So one of the things that you have to contribute to that

subcommittee is what you've learned from mistakes that were

made during the validation process at OCME?

A. I would not phrase it that way, no.

Q. Let's talk about the hidden function that was in the

original version of FST. I just want to be clear, make sure I

understood everything correctly.

When you testified on direct examination, you

testified that there were two changes to, I'll say, FST version

2.5. Is it correct that there were two changes made between

the previous version that was validated and the version FST

2.5?

A. I don't think so. I think there was one.

Q. One of the changes was where you cap off, you don't use --

you drop out a loci; right? A locus, excuse me; right?

A. If there is a locus where the allele frequencies add up to

.97 or higher, that locus is not used in the likelihood ratio

calculation.

Q. Right. And the other one you referred to as theta

correction. Those are two separate things?

A. The theta correction was in the method all along.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

253



Q. But you had to make a correction; right?

A. No. It's called a theta correction, but it wasn't -- it

doesn't mean that we were doing something wrong and we

corrected it. It's a population genetics parameter called a

theta correction.

Q. But as I understand it, it's two separate things; right?

There was something with allele frequencies and then there was

something with not dropping a locus, right, two --

A. There was one thing, the cap. The .97 cap is the only

thing.

Q. But the judge asked you specifically whether or not you had

presented that to the subcommittee --

A. The theta.

Q. -- and your answer was different, if I heard correctly, for

the dropping of the locus versus the theta correction; right?

A. The theta correction was part of the methodology all along,

and it was discussed with the subcommittee.

Q. Right, but you had to make a change?

A. No, that was part of the methodology all along.

Q. Let's back up. There was a version of FST that went

online -- there was a version of FST that went online in April

of 2011; right?

A. Right.

Q. And then it went online for a limited amount of time and

then changes had to be made; right?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

254



A. Yes.

Q. What changes had to be made?

A. If the allele frequencies at a particular locus in the

evidence sample add up to .97 or higher, that locus is not used

in the likelihood ratio calculation.

Q. And is that called a theta correction?

A. No, it is not.

Q. OK. Then my apologies. I misunderstood.

So was anything done with respect to the theta

correction after FST went online in April 2011?

A. No.

Q. Now that we're talking about the same thing, you were aware

of the dropping of the locus under the circumstance you just

described. You were aware that FST was fixed to incorporate

that at the time you testified in Brooklyn?

A. Yes.

Q. And you never brought that up in your testimony; right?

A. I didn't.

Q. You are aware that that change affects the likelihood

ratio; right?

A. It can.

Q. That information was not public. The information about

that change was not public at the time; right?

A. I thought that I had mentioned it in public settings, and I

thought it was in the validation paper. And I am apparently

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

255



incorrect. I thought we had included it. I was not hiding it.

Q. So you thought that after you made the change, you went

back and incorporated it into the validation paper?

A. No. The validation paper came out a year after FST went

online.

Q. So you thought --

A. I thought --

Q. -- that that was updated, the information was --

A. I thought I had mentioned it in the publication, but I am

wrong. Apparently, I did not. I was not trying to hide it is

what I'm saying.

Q. Even though you testified extensively at that hearing in

Brooklyn; light?

A. Yes. It's actually not a giant factor in the calculation

of the LR.

Q. Would you agree with me, doctor, that as you sit there

right now, there are people in the scientific community who

disagree with you?

A. I would imagine. People disagree with anything.

Q. But you know of people who disagree that that is --

A. I don't actually. I don't know.

Q. Have you discussed it with any of your learned colleagues?

A. What do you mean by my "learned colleagues"?

Q. Other members -- well, let me ask you. You're on that

subcommittee with who?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

256



A. You mean the OSAC committee.

Q. Yes.

A. That is not related to FST methodology.

Q. But it's related to validating protocols for probabilistic

genotypes; right?

A. Right.

Q. If a change like that were made or if a factor like that

was incorporated into one of the programs being validated, is

that something you would advise them to include in their

validation?

A. There are things that can happen post-validation that

requires a performance check, and I would put this in that

category. Of course, if things happen during development,

those things can all be included in a validation, but

post-validation, not everything would require revalidation.

Q. Your position is something like this does not require

revalidation?

A. That's correct.

Q. Even though it could have a huge effect on the likelihood

ratio?

A. I disagree that it would have a huge effect on the

likelihood ratio.

Q. Let's talk about that. Do you agree that either side can

benefit -- either side to the numerator or the denominator can

benefit depending upon the alleles present in the mixture and

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

257



in the reference sample?

A. I don't understand the question.

Q. If you take a situation before the modification was made or

the change was made, let's say -- let's do this: Original FST

and then the fixed FST, OK. I'm referring to before the change

was made and after the change was made.

A. OK.

Q. Do you agree that either side can benefit -- withdrawn.

Do you agree that a different likelihood ratio gets

generated with the original version of FST than the fixed

version of FST?

A. It can.

Q. It can. How often does that happen?

A. That depends on how many alleles are in the mixture. I

don't know worldwide.

Q. Did you do any studies on that?

A. We looked at our performance check samples.

Q. That was only 12 samples; right?

A. Right, but we did comparisons to a database of 1,250

noncontributors.

Q. How did you do that comparison? Did you use the bulk

calculator or did you hand put each calculation into FST?

A. We used the bulk calculator.

Q. Did you modify the bulk calculator as well?

A. The bulk calculator is just a way of pulling in profiles to

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

258



use.

Q. Did you have to make any changes to the bulk calculator

when you changed the version of FST?

A. No, I believe it's one function, and it pulls in the

profiles from the bulk run.

Q. And it incorporates it into whatever version of FST that

you're using?

A. Yes.

Q. So there wouldn't be a need to change the bulk -- is it

called the bulk calculator?

A. We used it as like a way to test a set of samples or a set

of profiles at a time. We called it a bulk calculation just

because it's pulling in a set of profiles for comparison.

Q. Back to what I was saying before. Do you agree that the

likelihood ratio could change based upon the newer version of

FST either for the prosecution or for the defense?

A. Yes.

Q. And do you agree that as long as -- is it your position

that as long as the fixed evenly helps both sides after the

change, that there's no need to worry about the accuracy of it?

A. I've never said that.

Q. I'm asking you now if you agree with that statement.

A. I don't know. I don't think so.

Q. Do you think we need to be worried about the accuracy of

dropping a locus in that scenario?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

259



A. No.

Q. By the way, the reason you dropped that locus is because

why again?

A. Because when the theta correction, the population genetics

parameter that has always been in FST is applied, it increases

the estimates of genotype frequencies that are used, homozygous

genotype frequencies in particular. And if the total allele

frequency is close to 1, that estimate can then bring the total

genotype frequency estimate above 1, which is mathematical

nonsense.

Q. What is the effect if a suspect has those common alleles?

What is the affect on the likelihood ratio when you drop that

locus?

A. It depends on whether or not the alleles appeared in the

mixture --

Q. If they did?

A. -- and whether they repeated.

Q. If they did?

A. I can't tell you theoretically what happens with a

profile -- an imaginary profile. I would have to look and see.

Q. Do you agree that the likelihood ratio would be different

for each different comparison genotype?

A. I don't understand.

Q. What don't you understand?

A. The question.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

260



Q. What part of the question, though?

A. Can you rephrase the question?

Q. Sure. The likelihood ratio between the old version of FST,

the fixed version of FST, do you believe that the change or the

difference in the two will be different for each different

genotype comparison?

A. I don't understand what you're asking.

Q. Do you agree that the differences -- do you think the

differences will always be small?

A. I don't know what you define as small, what -- I'm having

trouble theoretically answering that question.

MR. McKAY: Can you clarify differences between what?

Q. All of these questions I'm talking about the likelihood

ratio generated in the original version of FST versus the

likelihood ratio generated with the corrected version of FST.

A. The ones that we observed, the differences that we observed

were small.

Q. You only observed one sample that dropped a locus; right?

A. Two.

Q. Two?

A. Yes.

Q. But you ignored one of those two?

A. Excuse me?

Q. You ignored one of those two samples; right?

A. No.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

261



Q. You didn't. You used the data from both samples?

A. Yes.

Q. Let's talk about both samples. Tell me about them.

A. Well, in the document that is part of the evidence

summarizing that update and the performance check, both samples

are described.

Q. Can you explain it to the Court, because I think we passed

over one on your direct examination. I think we only talked

about one of them.

MR. McKAY: Your Honor, first of all, that misstates

the testimony. But to clarify, we can go to Government

Exhibit 16 --

THE COURT: Yes, I was just about to flip there.

MR. McKAY: -- for which there's again not a tab.

It's behind 15.

THE COURT: It's a two-page document, three-page

document, something like that, I think.

THE WITNESS: Yes, the front and back. No, it's a

two-page document.

THE COURT: OK.

BY MR. STRAZZA:

Q. Can you go over those two samples and the effect on them.

A. Yes. In the bottom paragraph on the first page, it says

two samples had one locus each that displayed such values,

meaning values where the allele frequencies added up to .97 or

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

262



greater. Therefore, their LR values were slightly modified as

expected. For example, for one sample where the LR at D3 was

equated to 1, the most conservative likelihood ratio was .34,

meaning the lowest of the four populations for that true

contributor was .34 previously and was .42 with FST version

2.0. So that LR went from .34 to .42. Similarly, for the

other sample, the LR was previously 3.38 times 10 to the

fourth, and with FST version 2.0, it was 3.82 times 10 to the

fourth when VWA was assigned a value of 1.

Q. I remember hearing something you wouldn't have run -- one

of those wouldn't have been run in FST; right?

A. No. These are the true contributors to the sample. Those

are the performance check samples. We also tested

noncontributors. These are real contributors. We also tested

noncontributors to these samples, and that is overall the

14,976 comparisons mentioned on the next page.

Q. OK. Tell me about that.

A. So for each of the performance check samples, we reran the

bulk comparison with the noncontributor database. And for

these two samples, there were some differences. And one is the

value that we mentioned this morning where a noncontributor --

the likelihood ratio for a noncontributor went from slightly

under 1 to, I believe, about 29. That was the example where if

this were a case, that noncontributor would actually -- that

comparison would not have been made because three of that

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

263



person's alleles were missing from the mixture. So that would

have been deemed no conclusions can be drawn. That's the one

where FST would not have been run in a casework scenario.

MR. STRAZZA: May I have a moment, your Honor?

THE COURT: Yes.

(Pause)

MR. STRAZZA: May I, your Honor?

THE COURT: Yes.

Q. Dr. Mitchell, you just described a scenario where it turned

into a false positive; right?

A. Yes.

Q. If three alleles are missing from a suspect profile with

comparison samples, that would not be run in FST; right?

A. According to this sample, a casework analyst confirmed

that, yes.

Q. I'm sorry. You mentioned that you performed hand

calculations as part of your FST validation study; right?

A. Yes.

Q. The purpose of that was to show that FST was calculating

the likelihood ratio correctly; right?

A. Yes.

Q. Did you perform hand calculations at all 15 loci?

A. I don't recall.

Q. Do you recall how many hand calculations you performed?

A. No.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

264



Q. Isn't it true that you only calculated two three-person

mixtures by hand?

A. I don't remember.

Q. Do you recall how many template quant values were tested?

A. No.

Q. Did your hand calculations reveal any kind of -- excuse me.

Did your hand calculations reveal the problem that

corrected between the original version of FST and the second

version of FST?

A. No.

Q. Were you asked at the Brooklyn Frye hearing about the

limited nature of your hand calculations?

A. I believe so, but I don't remember the specifics.

Q. And you didn't mention anything about a correction that you

made to FST at the Brooklyn Frye hearing, right, just so we're

clear?

A. No.

Q. Can a likelihood ratio at a single locus be as high as

thousand for a specific profile?

A. I don't know.

Q. Did you ever see that in any of your studies?

A. I don't remember.

Q. Didn't you show a slide to the DNA subcommittee that had a

likelihood ratio of 20,000?

A. I don't remember. Was it a real profile or something I'd

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

265



made up?

Q. I don't know.

A. I don't know either. I don't know what you're talking

about.

Q. Can the likelihood ratio at the same locus be close to 1

for a different specific profile?

A. I don't know.

Q. Could the likelihood ratio at the same locus be as low as

one in a thousand for a --

A. I don't even know what you're talking about to begin with.

Q. If you're talking about the same -- dropping a locus -- may

I have a moment, please?

THE COURT: Yes, go ahead.

MR. STRAZZA: Sorry. Sorry.

Q. Sorry, Dr. Mitchell. I'm going to try and word this

better.

A. OK.

Q. Does dropping a locus like you would in the situation that

you -- in the corrected version of FST, in your expert opinion,

could doing that change the likelihood ratio by a thousand?

A. I would say probably not for these loci.

Q. Couldn't you have solved the problem another way?

A. Sure.

Q. Without changing the source code to FST?

A. What are you suggesting?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

266



Q. Couldn't you have -- one moment, please.

Couldn't you just not enter the alleles, the problem

alleles, into the --

A. Change the evidence?

Q. No. Couldn't you not -- no, I'm never suggesting you

should change the evidence.

A. OK.

Q. If the alleles at the problem locus were not entered into

the FST worksheet, wouldn't the problem be solved without

changing the source code?

A. By changing the evidence?

Q. Well, in essence, your change is not using all the evidence

anyway, right, by dropping the locus; right?

A. We are not using that locus, yes.

Q. So when you say "by changing the evidence," that's not what

I'm suggesting. I'm saying instead of -- you're not using the

locus, so instead of doing it that way and having to change the

source code to accomplish that, couldn't you just not use the

alleles that caused the problem?

A. It sounds to me like you're suggesting changing the

evidence.

Q. That's definitely not what I'm suggesting.

A. Then I don't understand what you're suggesting.

Q. Almost done. Forget about the numbers.

A. OK.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

267



Q. Some basic questions. Do you think that it would be

important for the members of the subcommittee who voted to

approve FST, do you think it would be important for them to

know about the change that you made between the original

version of FST and the newer version of FST?

A. No. In consultation with our technical lead, Eugene Yant,

we determined that this was not a substantial change. That

this required only a performance check.

Q. But you thought it was important enough to incorporate into

the validation studies, right, because you thought that was

done; right?

A. I'm sorry?

Q. You testified earlier that you thought this was disclosed

in the validation studies; right?

A. I thought it was disclosed in the paper describing the

validation.

Q. And you thought it was important enough to do that; right?

A. I don't think it's as important as you're making it sound.

Q. What I'm asking you is whether you thought it was important

enough to incorporate into that paper.

A. To mention, sure.

Q. And that wasn't done?

A. No, it was not.

MR. STRAZZA: I have no further questions.


1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

268


HB67JON8 Mitchell - redirect

MR. MCKAY: Briefly, your Honor.

REDIRECT EXAMINATION

BY MR. MCKAY:

Q. Do you recall just a moment ago Mr. Strazza was asking you

some questions about how much a particular locus could change

the likelihood ratio?

A. Yes.

Q. In the examples that we walked through, the likelihood

ratios that you were talking about would be the sample

calculations?

A. Yes.

Q. Were those based on real data or hypothetical numbers that

you made up for purposes of illustration?

A. Those were hypothetical.

Q. Now, do you remember a few minutes ago Mr. Strazza asked

you a number of questions about whether the likelihood ratios

would change based on the newer or older version of FST?

A. Yes.

Q. And he asked you about tests that you performed to evaluate

whether there would be changes.

A. Yes.

Q. And you mentioned 12 samples.

A. Yes.

Q. To be clear, how many times did you compare those 12

samples to noncontributors?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

269



A. Each sample was compared about 1250 times to noncontributor

samples.

Q. So we are talking about thousands and thousands of

comparisons to known noncontributors?

A. Yes.

Q. Do you recall a few minutes earlier when Mr. Strazza was

asking you questions about other laboratories used likelihood

ratio programs?

A. Yes.

Q. And that there are some, indeed perhaps many, that do not

use likelihood ratio programs.

A. Yes.

Q. Can you talk about OCME generally in terms of the size and

amount of resources OCME has?

A. OCME has, as long as I've been there, had a chief medical

examiner who is very interested in research and has been able

to obtain from the Office of Management and Budget, OMB, funds

for OCME to use for research and development purposes.

Developing FST was incredibly expensive, validating it

was incredibly expensive. Most labs do not have that kind of

resource.

Q. And were there particular events in OCME's history that

produced a particular focus on evaluating DNA at OCME?

A. Yes, definitely. The September 11 attacks increased a

focus on OCME and on forensic DNA analysis.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

270



Q. And the resources that were devoted to this type of

analysis?

A. Yes.

Q. Do you recall Mr. Strazza early on in his cross-examination

asking you a number of questions about estimating the number of

contributors?

A. Yes.

Q. And there were a number of questions about OCME protocols

on estimated number of contributors?

A. Yes.

Q. And about an article that you contributed to that was

published in a peer reviewed journal about estimating number of

contributors?

A. Yes.

Q. As far as you know, are the protocols that OCME uses common

in the forensic community?

A. Yes.

Q. And are those protocols based in part to at least consider

some of the guidelines set forth in your peer reviewed article?

A. Yes.

Q. And to be clear, in a situation in which a mixture is

estimated to be three people, and a likelihood ratio is

calculated, does that likelihood ratio necessarily support the

prosecution or the defense?

A. No.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

271



Q. It could go either way, right?

A. Yes.

Q. Do you remember Mr. Strazza and Mr. --

Mr. Deluca, if you could pull up the exhibits if you

have them.

Do you remember Mr. Strazza asked you a series of

questions about the degradation aspect of the validation?

A. Yes.

Q. And he was asking you whether you created a comparisons of

degraded samples to nondegraded samples?

A. Yes.

Q. To be clear, if you evaluated one degraded sample and a

different sample that was not degraded, would you expect to

have the same likelihood ratio?

A. No, they are different samples.

Q. And is that because of the degradation or is it because

they're just different samples?

A. Both.

Q. Now, what were you trying to determine when you looked at

the degradation model?

A. We were trying to determine if by adjusting drop-out

probabilities for degraded samples we could get higher

likelihood ratios for true contributors to those samples

without affecting the false positive rate for noncontributors.

Q. And so what was your finding about whether adjusting the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

272



drop-out rate for degraded samples in fact gave you better

separation?

A. We were not able to get higher likelihood ratios for true

contributors without increasing the likelihood ratios obtained

for noncontributors.

Q. And so as a result of that, how did you account for

degradation in FST?

A. So the decision was to include the degraded samples as part

of the validation and then to be able to use FST with degraded

samples without having a special function that would be called.

Q. Do you remember the series of questions about how, if at

all, you reported your data on degradation?

A. Yes.

MR. MCKAY: Mr. Deluca, can you pull up Government

Exhibit 7 and turn to page 443 of that PDF.

Q. And just to remind the court, what was Government Exhibit

7.

A. It was the executive summaries of the validation volumes.

Q. And so on this page here can you just read the title of

this volume of the executive summary?

A. Volume 15 summary: Estimation of Degree of Sample

Degradation and Generation of DNA Profiles from Two-Person

ID28 and ID31 Mixtures with Both Components Degraded and

Generation of DNA Profiles from Degraded, Two, Three and Four

Person Touched Item Samples.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

273



Q. Do you recall Mr. Strazza --

You can take that down, Mr. Deluca.

Do you recall Mr. Strazza asking you a series of

questions about whether you get a more accurate likelihood

ratio if you varied this variable or this variable?

A. Yes.

Q. And he listed some of the factors that go into the

estimation of drop-out rates, right?

A. Yes.

Q. Do you have perfect information about a case work sample?

A. No.

Q. Are you aware of another program that has perfect

information about a case work sample?

A. No.

Q. And by case work sample, I mean actual crime scene

evidence.

A. Right.

Q. Does FST purport to report perfect information?

A. No.

Q. Does it purport to report a precise likelihood ratio?

A. No.

Q. What is FST reporting?

A. It is a quantitative estimate of the strength of the

evidence.

MR. MCKAY: No further questions.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

274


HB67JON8 Mitchell - recross

THE COURT: OK.

MR. STRAZZA: Just brief follow-up, I promise.

THE COURT: Very brief.

RECROSS EXAMINATION

BY MR. STRAZZA:

Q. When you say it's just supposed to be an estimate, doesn't

it report to three significant figures?

A. I believe so.

Q. And so if it was 2.14, it would produce a result like that,

right?

A. Yes.

Q. Doesn't that make it seem like it's accurate to the 2.14

number that it's presenting?

A. It's not unreasonable to report this way.

Q. That's not what I asked you, doctor. Please. Doesn't that

seem like it's precise if you're producing a calculation like

that, 2.14?

A. I think it's a fine presentation of the results.

Q. There is no difference between 2.0 and 2.14, right, in

terms of the results of the likelihood ratio?

A. They would be interpreted in the same way; they are both in

the same range.

Q. OK. But it's just -- it's not accurate -- would you be

able to -- is it accurate enough to describe the difference

between 2.0 and 2.14?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

275



A. Those would be interpreted in the same way.

Q. So are you saying there is no -- the only difference is the

four categories?

A. I'm sorry?

Q. Are you saying that the only difference is when they're

interpreted in a different way in that one of the four

categories? That's the only time they're different? Is that

what you're saying?

A. No, I don't understand.

Q. When you say those would be interpreted the same way, what

do you mean by that?

A. I mean that they are both very close to one; there is not

much information there.

Q. So why list a 2.14 then? Doesn't that convey a level of

accuracy that just simply doesn't exist?

A. No.

MR. STRAZZA: No further questions.

THE COURT: OK.

REDIRECT EXAMINATION

BY MR. MCKAY:

Q. If you were testifying in court at a jury trial about FST

results, and you were asked about degree of accuracy down to

2.14, would you explain to the jury that it's an estimate?

A. Yes.

Q. Would you expect an OCME criminalist to be frank about the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

276



fact that that was an estimate?

A. Yes.

MR. MCKAY: No further questions.

THE COURT: All right. Thank you very much, doctor.

You can step down.

THE WITNESS: Thank you.

(Witness excused)

MR. MCKAY: Should we recall Dr. O'Connor?

THE COURT: Sure, but I'm not sure how much more --

well, let me ask the marshals what the situation is, and also

the court reporters also.

MARSHAL: We have to transport him back. He is

holding up the entire Brooklyn.

MR. MCKAY: So then perhaps we could consult with Dr.

O'Connor about scheduling and then come back and have a brief

logistics discussion with the court?

THE COURT: That's fine. Let me ask this.

Mr. Strazza, is it all right if your client -- well, check with

your client and see if he is OK with us discussing logistics so

that he can get back to.

THE DEFENDANT: Yeah, it's all right.

THE COURT: All right.

MR. STRAZZA: You heard him.

THE COURT: OK, so, yes, you should go and speak to

the doctor, and then we can talk about logistics, although --

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

277



and I will take a look at my calendar and see what we can

accommodate.

MR. MCKAY: Thank you, your Honor.

THE COURT: Mr. McKay, we do have the 15th, right?

MR. MCKAY: Your Honor, I think --

THE COURT: Or is that not --

MR. MCKAY: Both because the defense has three

different witnesses who are going to need to go on the 15th,

and because it seems reasonable to get Dr. O'Connor back on

while his direct is fresh in the mind, we were hoping we might

fit him at some point in the next few days in the court's

schedule. I know you have the jury trial tomorrow.

THE COURT: Yes, it's conceivable. It's just that

it's going to have to -- and again tomorrow is going to be a

full day with jury addresses and -- off the record.

In any event, it's going to be an all day affair

tomorrow. Wednesday I do have conferences, and Thursday I have

a bunch of conferences.

So I guess the question is there are chunks of time on

Wednesday, including after my last conference at 4, but there

is time in between -- but, you know, I just don't --

MR. ROSANIA: The marshals will not be happy about

that.

MR. STRAZZA: Time out. Can we go off the record for

a second?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

278



THE COURT: Yeah.

MR. STRAZZA: Back on the record. In terms of

estimating the length of the cross, I mean it's really going to

depend on the way it goes. Like today was very different than

I had anticipated in terms of estimating the length of the

cross. So, I just -- I don't want to say -- when we are

working with such a tight schedule, your Honor, like fitting us

in at four --

THE COURT: No, I'm just providing -- let me just say

this. I think what I was just doing is providing time, because

as I understand it, it may be that since we're on the 15th it

looks like we're scheduled to start at 11:30 or around that

time.

MR. MCKAY: So I think our concern would be if we

start at 11:30 on the 15th and the defense is calling three

separate experts, there is no chance we would finish the 15th,

and that's particularly so if we also have to do Dr. O'Connor's

cross. So if the court has a two or three hour window in the

next few days that Dr. O'Connor could come back for the cross,

and if the cross takes longer than that, we will come back

again, but I think realistically for Dr. O'Connor a two or

three hour window is probably going to be sufficient.

MR. STRAZZA: I agree.

MR. ROSANIA: I agree. I think my cross will be more

than sufficiently done.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

279



THE COURT: I have a 10 o'clock -- I was thinking

maybe if we start at 9 o'clock on Wednesday, break at 10 for me

to do the sentencing, come back at 11, and then we would have

from 11 until 12:45.

MR. MCKAY: That's Wednesday the 8th, your Honor?

THE COURT: That's Wednesday the 8th, that's correct.

MR. MCKAY: That certainly works for the government.

Let me double check with Dr. O'Connor.

THE COURT: So we would be starting at 9, going for

about an hour, and then starting up again at about 11.

MR. MCKAY: I will be right back, your Honor.

Your Honor, Wednesday at 9 works for Dr. O'Connor.

THE COURT: And then will you be able to -- well, all

right, Wednesday at 9 he will be able to stay also, because it

may go longer than an hour, so he would come back at 11?

MR. MCKAY: Yes.

THE COURT: OK. So why don't we plan on doing that.

So, we will have continuing of the Daubert. So we

will go for an hour and then at 11.

MR. ROSANIA: And, Dr. O'Connor is subject to cross,

meaning.

MR. MCKAY: Yes, we won't discuss substance with him.

THE COURT: I think I had said that to him when he

left the stand. That should apply. Do you want me to bring

him back in to advise him of that?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

280



MR. STRAZZA: No, we trust the government.

THE COURT: All right. So you are in my calendar.

Obviously the one thing that's unpredictable is if I have jury

notes I have to deal with those.

MR. MCKAY: Of course.

THE COURT: So we will handle that. I will say this,

the 15th obviously I have time, but I think did we also give

you the 16th?

LAW CLERK: No, only the 15th.

THE COURT: No, the only reason I'm saying -- well,

all right.

MR. STRAZZA: How long is your cross going to be, Tom?

MR. MCKAY: It's hard to predict these things. And my

second seat doesn't like me to tell the court --

MR. ROSANIA: It's hard to estimate.

THE COURT: So there anything else we need to discuss

right now?

MR. MCKAY: Not from the government.

MR. STRAZZA: No, your Honor.

MR. ROSANIA: No.

THE COURT: All right. Let me ask you this just with

regard to the other DNA. My recollection is that I got a

letter saying that there isn't going to be a Daubert type

challenge to that and that that will be dealt with through

cross-examination at the time of trial. Is that an accurate

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

281



statement?

MR. ROSANIA: You can incorporate it or not.

MR. STRAZZA: I think some Dr. Chakraborty has some

issues with it, but I think your statement was accurate. And

we can address those issues at trial.

MR. MCKAY: OK. So to be clear there is not a Daubert

motion about the hat.

MR. STRAZZA: Correct.

MR. ROSANIA: Right.

THE COURT: The only reason I'm asking is because I

know there was some testing that occurred, and that's my

recollection that as a result of the -- not as a result -- but

that that did not lead to a situation where a hearing was being

requested for some reason.

MR. MCKAY: My understanding is that there will be a

defense expert at trial who retested the hat and purports to

have gotten a different result, and so I think there will be,

if I'm right, testimony from that defense expert and perhaps

rebuttal testimony from a government expert about why the

results were different.

MR. STRAZZA: Well, right, but there may also be

testimony from Dr. Chakraborty about his opinion on that in

front of the jury.

MR. MCKAY: Right.

MR. STRAZZA: And that's a different expert than the

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

282



expert who separately tested the hat.

MR. MCKAY: And I think we can discuss the parameters

of that closer to trial as long as we're talking about not

holding a Daubert hearing about that issue, right?

THE COURT: All right. I'm not -- yeah. OK. I think

I understand, but is the result of that test, is that

something, Mr. Strazza, that you've turned over or not turned

over?

MR. STRAZZA: Yes, I have turned over the entire case

file. The government is aware of who the analyst is, what the

results are. The government has everything I have.

THE COURT: OK. All right. Anything else?

MR. MCKAY: Not from the government, your Honor.

THE COURT: All right. Nothing from the defense?

MR. ROSANIA: No.

MR. STRAZZA: No.

THE COURT: We will stand adjourned. Thank you very

much.

(Adjourned)

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

283


INDEX OF EXAMINATION

Examination of: Page

CRAIG O'CONNOR

4Direct Q. . . . . . . . . . . . . . . . . . . .

ADELE MITCHELL

84Direct Mr. McKay . . . . . . . . . . . . . . . .

187Cross By Mr. Strazza . . . . . . . . . . . . .

268Redirect By Mr. Mckay . . . . . . . . . . . .

274Recross By Mr. Strazza . . . . . . . . . . . .

275Redirect By Mr. Mckay . . . . . . . . . . . .

GOVERNMENT EXHIBITS

Exhibit No. Received

101 7 . . . . . . . . . . . . . . . . . . . . .

102 15 . . . . . . . . . . . . . . . . . . . . .

100 16 . . . . . . . . . . . . . . . . . . . . .

103 27 . . . . . . . . . . . . . . . . . . . . .

5 29 . . . . . . . . . . . . . . . . . . . . . .

104 32 . . . . . . . . . . . . . . . . . . . . .

6 37 . . . . . . . . . . . . . . . . . . . . . .

106 41 . . . . . . . . . . . . . . . . . . . . .

108 43 . . . . . . . . . . . . . . . . . . . . .

107 44 . . . . . . . . . . . . . . . . . . . . .

105A 51 . . . . . . . . . . . . . . . . . . . .

105B 57 . . . . . . . . . . . . . . . . . . . .

4 65 . . . . . . . . . . . . . . . . . . . . . .

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

284


1 90 . . . . . . . . . . . . . . . . . . . . . .

2 91 . . . . . . . . . . . . . . . . . . . . . .

3 93 . . . . . . . . . . . . . . . . . . . . . .

0 101 . . . . . . . . . . . . . . . . . . . . .

15 139 . . . . . . . . . . . . . . . . . . . .

7 153 . . . . . . . . . . . . . . . . . . . . .

8A, 8B, 8C, and 8D 161 . . . . . . . . . . . .

9 162 . . . . . . . . . . . . . . . . . . . . .

10 163 . . . . . . . . . . . . . . . . . . . .

11 164 . . . . . . . . . . . . . . . . . . . .

12 165 . . . . . . . . . . . . . . . . . . . .

16 178 . . . . . . . . . . . . . . . . . . . .

14 182 . . . . . . . . . . . . . . . . . . . .

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

84 short one so we ensure that the doctor gets -- 2 …...2019/03/11 · 84 SOUTHERN DISTRICT...

Documents

Transcript of 84 short one so we ensure that the doctor gets -- 2 …...2019/03/11 · 84 SOUTHERN DISTRICT...