TOOLMARKS AND FIREARM EVIDENCE

THE BASICS

• Task is for examiner to identify characteristics of microscopic toolmarks (removing the class and subclass characteristics) and then to assess the extent of agreement in the individual characteristics in the compared toolmarks, to permit the identification of an individual or a firearm.

• WTF?

What does that mean?

• The evaluator (person who looks at the toolmark) is looking for distinctive marks caused by the tool.

• A) Did the object (e.g., a gun, a hammer, a knife, a stapler, a shovel, a shoe)

• B) Make a mark on something else (a casing, a body, a wall, the ground)

• C) That is distinctive to the object and the mark?

What does that mean, really?

• Eyeball comparison of two things and a subjective opinion about whether they match

• Expert used magic process to prove that the object (in this case, a gun) is the object that created the toolmark.

• What can we do if the magician says he (or she) used the magic process and its... A MATCH???

What can we do?

• Understand and deconstruct the magical process.• First: it’s NOT a MATCH• The science does not allow the state to claim it’s

an EYE-dentical, one hundred percent MATCH. Manning v. Mississippi, 2013-DR00491-SCT

• At best for state, expert may testify may testify at trial that it is “Consistent with”:

• E.g., Expert’s comparison and analysis of the bullet fragments established similarities that are consistent with the bullet fragments having been fired from the same firearm.

How is the analysis and comparison done?

• Evaluator may use ballistic imaging technology, but the FINAL comparison is not a machine or computer analysis

• Rather, final comparison is an EYEBALL/VISUAL COMPARISON by the firearms examiner

What is the process for comparing Bullets and/or cartridges?

• Evaluator looks for distinctive marks by the tool.

• Places toolmark into class:There are three “classes” of toolmarks that may be present:

• 1) Class characteristics: distinctive features, but shared by many items of the same type; very common. E.g., serrated edge steak knife.

• 2) Subclass characteristics: may be common to a small group of firearms that are produced by the manufacturing process. Not as distinctive as individual characteristics.

• 3) Individual characteristics: fine microscopic markings and textures that are unique to the individual tool or firearm. Most likely to distinguish the item that made the toolmark.

The Comparison Process

• Examination by microscopic comparison looking for individual characteristics “unique” to the tool or firearm.

• Intended to seem precise (Ooooh, she’s using one o’ them scientifical microscopes!!!)

• But comparison is inherently subjective (Eyeballs and judgment) and prone to confirmation bias.

How to Challenge the state’s proferredevidence

• Pre-trial:• Daubert challenge S. 907.02, Wis. Stats: not based on reliable

facts, data, science, etc.• Motion to exclude; e.g., lack of documentation• Motion in Limine and motion to limit testimony

• Independent examination of evidence:• Results of state examiner/ examination;• The process used by examiner (at lab or generally)• All of the above?

• At Trial: Cross examination of state’s expert (see above)

Pre-trial challenges

• 907.02 Testimony by experts. (Daubert)• Expert may testify in form of opinion “if the testimony

is based upon sufficient facts or data, the testimony is the product of reliable principles and methods, and the witness has applied the principles and methods reliably to the facts of the case”

• Challenge the magical process because the process is SUBJECTIVE

• Its an EYEBALL COMPARISON. There are no facts or data.

McCluskey Brief; Daubert Challenge

The grounds for this motion are: • (1) there is no reliable scientific basis for this proposed testimony, and

thus the testimony is inadmissible under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993) and Kumho Tire Co. v. Carmichael, 526 U.S. 137,119 S.Ct. 1167, 143 L.Ed.2d 238 (1999);

• (2) the testimony is inadmissible because (a) the testimony is not based upon sufficient facts or data, (b)the testimony is not the product of reliable principles and methods, and (c) the firearms examiner who performed the bullet comparison in this case has not applied the principles and methods reliably to the facts of the case;

• (3) the subjective conclusion, unsupported by statistical analysis, that a particular bullet or cartridge case was fired from a particular weapon, or cannot be eliminated as having been fired from the weapon is so weak as to lack any probative value;

No Longer Just Generally Accepted• Despite Its Long Acceptance by Courts, the Scientific Community Has Rejected Firearm

Identification as a Reliable Method of for Conclusively Establishing a Connection Between a Particular Bullet and a Particular Gun.

• Until recently, federal and state courts have admitted firearm identification almost without exception, often without even applying Daubert factors, based on the assumption that the method is scientifically valid and the fact that past courts have accepted the method time and again. See, e.g., United States v. Hicks, 389 F.3d 514 (5th Cir. 2004) (accepting reliability of toolmark and firearm identification without question); United States v. Santiago, 199 F. Supp. 2d 101, 111 (S.D.N.Y. 2002).

• We now know that the scientific community has rejected the use of firearm identification to identify a specific firearm or bullet. A recent and thorough review of the scientific literature on firearm identification by the prestigious National Academy of Sciences concluded that firearm identification cannot uniquely identify a specific gun for a number of reasons, including the fact that a firearm examiner’s conclusion of a match “remains a subjective decision based on unarticulated standards and no statistical foundation for estimation of error rates.” Nat’l Acad. Sci., Strengthening Forensic Science in the United States: A Path Forward 153-54 (2010)

• NAS Report. (hereinafter “NAS Report”), available at http://books.nap.edu/openbook.php?record_id=12589

Where to start the challenge? Discovery Demand

• Entire case file of any analyst and reviewer, including worksheets, lab notes, sketches, photographs, video and reference material and correspondence

• They won’t give it to you unless you ask. If they don’t have it, they should have it, and is subject for cross exam (I.e., no notes about process?); OR a motion to EXCLUDE….

Pre-trial Motion to Exclude: Lack of Documentation

• MOTION TO EXCLUDE FIREARMS EVIDENCE FOR LACK OF DOCUMENTATION

• Defendant XXXXXXXXXXXX, through undersigned counsel, respectfully moves this Court, pursuant to the Sixth Amendment to the United States Constitution and Dyas v. United States, 376 A.2d 827 (D.C. 1977), to exclude the proposed firearms evidence in this case. The defense requests a hearing on this motion. In support of this Motion counsel states:

• 1. XXXXXX is charged with XXXXXXXXXXXXX.

• 2. The state has informed the defense that it intends to call a firearms examiner to testify that a bullet from the crime scene and a bullet test-fired from the defendant’s gun “matched” to a reasonable degree of certainty in the area of firearms and toolmarks analysis. [GET exact words]

• 3. As part of discovery, the defense has received a single bullet/cartridge worksheet describing the bullets in question. The defense has received no other documentation relating to

the purported match between the bullet found on the crime scene and the bullet test-fired by __________.

• 4. Based on the limited amount of documentation received, the defense does not know what steps the examiner followed to reach such a conclusion or whether such steps conformed to generally accepted methods of firearms and toolmark analysis. The lack of documentation, however, falls well below documentation standards required in the relevant scientific community.

• 5. The relevant scientific community requires that documentation include depictions or descriptions of the agreement or disagreement of the projectile’s characteristics such that another qualified firearm and toolmark examiner, without the benefit of the evidence itself, can review the case record, understand what was compared, and evaluate why the examiner arrived at the reported conclusion. Such documentation is absent here.

• 6. The lack of proper documentation also denies the Sixth Amendment right for the defendant to confront the witnesses against him by effectively preventing any meaningful cross-

• examination.

• 7. WHEREFORE, for the reasons stated above, as well as any other reasons that become apparent to the Court, the proposed testimony regarding [bullet/cartridge] comparison should be excluded.

Discovery Demand: documentation of what the analyst saw

• Any and all documentation of observations that support a reported conclusion, including imaging, narrative descriptions, sketches, diagrams, charts, worksheets, or tabulation of consecutive matching striae (CMS)

• By requesting this, we are looking for what the individual examiner did, what they saw, and what they documented about how they did it.

Discovery: similarites/ dissimilarities

• Documentation of the exact points of comparison being relied upon for any firearm/toolmarkcomparison. (US v. Robinson, 44 F.Supp. 2d 1345(ND Ga. 2005)

• Documentation for any points of dissimilarity in any firearm/toolmark comparison conducted in this case, and if the dissimilarity did not result in an exclusion, an explanation as to why these points of dissimilarity did not lead to an exclusion

Discovery Demand: general procedures used by analyst

• Any Standard Operating Procedures; users manuals; or other protocols

• Here we are looking at the general procedural rules that the examiner claims they were following. I.e., lab should have protocols to follow.

• For general protocols, look to SWGGUN (Scientific Working Group for Firearms and Toolmarks): Quality Assurance Guideline. www.swggun.org

Discovery Demand: computer searches

• Results of any computer searches seeking a match with questioned evidence

• Calibration and maintenance records for all instruments and equipment used in the comparison

• Internal validation• Proficiency tests of analyst and peer reviewer, including

case file and photos (Should be double blind)• CVs• Audit documents (external and internal)

Integrated Ballistics Identification System (IBIS)

• The Integrated Ballistics Identification System, or IBIS, is the brand of the Automated firearms identification system manufactured by Forensic Technology WAI, Inc., of Montreal, Canada.

• IBIS has been adopted as the platform of the National Integrated Ballistic Information Network (NIBIN) Program, which is spearheaded by the United States Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF). NIBIN tracks about 100,000 guns used in crimes.[1] The integration of technology into about 220 sites across the continental US and its territories [2] facilitates sharing of information between different law enforcement groups. The rapid dissemination of ballistics information, in turn, allows for tracking of gun-specific information and connection of a particular firearm to multiple crimes irrespective of geographic location. A National Research Council report has found that with the NIBIN dataset, a bullet retrieved from a crime scene will generate about 10 possible matches, with about a 75-95% chance of a successful match.[1]

Back to the comparison process

• Photo of a match only shows a small (microsopic) portion of area observed during comparison

• “most, if not all comparison microscopes are now fitted with a video camera and video recorder…a video recording of the match is the only real way of demonstrating how the positive comparison was made.” Brian J. Heard, Forensic Ballistics in Court

Cross Examination of the State’s Expert

• What did we learn in discovery?• Did they document the process?• What are the protocols, procedures, or

Guidelines that they followed?• Are they aware of the procedures and

protocols that are recommended by SWGGUN, etc?

• Subjectivity of the process?

Subjectivity of comparison: art not science

• Firearm identification is an art, and not a science.• “Whatever else ballistics identification analysis

could be called, it could not fairly be called a ‘science.’” US v. Glynn, 578 F.Supp2d, 567, 570(SDNY 2008).

• Evaluator may use ballistic imaging technology, but final comparison is not a machine or computer analysis, rather a subjective visual examination comparison by firearms examiner

Cross exam

• Examiner uses own eyesight and reasoning and his/her own subjective judgment.

• Best practice is to conduct a double blind analysis.

• Best practice is that NO information is given to the examiner about the toolmark or object being analyzed.

• Giving any information can create confirmation bias

Standard is “Agreement”

• Standard is “agreement” between a specific set of toolmarks or a specific “bullet striation pattern”

• This is not an exact or precisely defined standard• Assoc. of Firearms and Toolmark Examiners (AFTE)

defines “agreement” as “significant” “when it exceeds the best agreement demonstrated between toolmarksknown to have been produced by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool.”

• WTF?

NO standard

• That is not a specific protocol about how to do the analysis.

• No universal detailed and articulated protocol about how to do the analysis

• No step by step guideline. None by AFTE, SWGGUN

• No standard for how many points of similarity, etc.

Repeatability of process

• No standards means that individual evaluators may do a comparison of the same toolmark in different ways?

• Difficult to exactly repeat the process?• Difficult to check if the examiner made a

mistake?

SWGGUN Guidelines

• In 2013, the Scientific Working Group for Toolmarksand Firearms (SWGGUN) adopted guidelines for the documentation of the examination of tools and toolmarks

• Guidelines do NOT outline a specific PROCESS• Rather, the guidelines require DOCUMENTATION of the

process used by the examiner• http://www.swggun.org/index.php?option=com_conte

nt&view=article&id=24:guidelines-for-the-documentation-of-firearm-examinations&catid=11:guidelines-drafts&Itemid=6

Guidelines for the Documentation of Firearm Examinations| |

Guidelines - Guidelines- Adopted1.0 ObjectiveThe purpose of this document is to set forth guidelines to Firearm Examiners conducting forensic examinations of firearms.

2.0 Introduction2.1 The examination of any firearm will be documented. This documentation may include the condition and physical characteristics of the firearm as received and will include information related to tests conducted with the firearm.2.2 Generally accepted forms of documentation include, but are not limited to, worksheets, laboratory notes, sketches, photographs, video and reference material.

No Known Error Rate

• Error rate is the frequency at which one deviates or strays from a correct standard

• I.e., test how often process is correct or incorrect

• “the decision of the toolmark examiner is a subjective decision based on unarticulated standards and no statistical foundation for estimation of error rates.” NAS Report, p153

Subjectivity allows for error

• “AFTE standards acknowledge that these decisions involve subjective qualitative judgments by examiners.” NAS Report, p 153

• Decisions by experts are subjective and there is a possibility that the examiner made an error?

It’s a match!

EYE-dentical!

• Sorry DA Jim Trotter: You cannot say that there is a match that is absolutely “I-dentical!”

• Cannot say that a specific gun fired a specific bullet to the exclusion of all other guns in the world

• No matter what the opinion, possibility exists that it was fired from another gun

Manning: Not an absolute match• US DOJ and FBI determined that an examiner cannot say that an “absolute” match exists: US DOJ

and FBI believe that science regarding the examinations of firearms involves human judgment

• Manning v. Mississippi, 2013-DR-00491-SCT

• “The science regarding firearms examinations does not permit examiner testimony that a specific gun fired a specific bullet to the exclusion of all other guns in the world.” “The examiner could testify to that information, to a reasonable degree of scientific certainty, but not absolutely. Any individual association or identification conclusion effected through this examination process is based not on absolute certainty but rather a reasonable degree of scientific certainty. “ “As with any process involving human judgment, claims of infallibility or impossibility of error are not supported by scientific standards.”

• May 16, 2013, US DOJ, letter.

• http://www.ncids.com/forensic/firearms/DOJ-Letter-Manning-ballistics.pdf

• Cannot say that an absolute match exists, because the comparison processes involve human judgment are vulnerable to human error