Law & Science: A Convergence in Energy Law And Policy

Michael C. Dorf: Good afternoon. My name is Michael Dorf. I am a professor here at the Cornell Law School. I will be your moderator for this panel, which is title Law & Science: A Convergence in Energy Law And Policy. The aim of this panel is to step back a little bit from the particulars of the hydrofracking questions and talk about some general questions about how law and science interact. We have three terrific panelists and so I don’t want to take up more of their time than necessary. In the order that they will be speaking are Kevin Haroff, Ted Eisenberg and Brad Karkkainen. I should just say that for me this is a great pleasure to welcome Kevin who is a double alumni of Cornell back to the Law School. Ted is of course a colleague so I need to welcome him. He welcomed me here when I joined and Brad and I were colleagues for many years together at Columbia. So it is a great personal pleasure for. Each of them will say a few words of introduction and then plunge right into it. So without further ado I am going to turn the panel over to Kevin and his powerpoint presentation and keep your questions for the end after the three of them are done I will have some very brief interchange and then will open up to the floor.

Kevin Haroff: Thanks very much. I hope everyone can hear me alright. I will do as I am told and speak directly into the microphone. It is a great pleasure for me to be here. I have had an occasion to come back to Cornell a couple of times in the last year after having been away for a long period and so it is a great opportunity for me personally to be here for the conference today. I am an attorney in private practice in California with a law firm that operates throughout the United States, as well as in Europe. My practice is primarily doing environmental legal and regulatory work in California, although I have worked on matters throughout the country including land use development projects as far away as Guam and more recently involved in disputes between the federal government and the state of Texas over the administration of the Clean Air Act Title V operating program in Texas, which was an interesting experience to say the least. One of the lessons that I have learned over the years is the importance of grounding environmental and energy policy on strong science on empirical facts as a principal of first ______*1:31:18. As a second principal where science perplex uncertainty of what those facts are, that is not a reason for letting policy develop and grind to a halt. It is just a signal to be sensitive and to be sensible about the risks that are associated with policy alternatives and making sensible decisions based upon the rigorous assessment of the risks using wherever possible good quantitative data on there, not anecdote and not speculation, which brings me to my first subset. This is a screen capture of a shot from the January 27th episode of the Daily Show with John Stewart. The shot is someone who may be familiar to you T. Boone Pickens, a very smart and very successful investor with a long history with domestic oil and gas industry. T. Boone Pickens was making an appearance on the Daily Show at that night to generally promote various ideas he has about how we as a natation can reduce our independence on foreign oil in particular and expand our development and reliance of more domestic energy and resources including in particular natural gas. Now he wasn’t on the Daily Show that night specifically to address the pros and cons of any particular development and technology, although he did take advantage of being on the show that night to talk very briefly about the risks and benefits of hydraulic fracturing and said in response to a question about whether hydraulic fracturing can cause or create potential for contamination and drinking water sources; “I have

never seen that happen and you are not talking about _____*1:33:18 and the first reader." We will be asking at the end of the program ____. “I have been here. I have fracked 3,000 wells in my life. I have never seen anything damaged.” Now I don’t want to sit here today and question the veracity of that statement or the sincerity of the person making the statement, but I do want to point out in making that statement and in hearing that statement there is a lot that we don’t know that underlies. We don’t know where those 3,000 wells were drilled. We don’t know what time frame they were drilled in. We don’t know whether they were oil wells or natural gas wells. We don’t know whether they were developed using horizontal drilling technology or something else. The point that I am trying to make is that when people advocate serious policy decisions based upon anecdotal personal experiences that is not grounded in transparent scientific assessment risk that is when we all should start worrying. That is when the red flags should come out. Which leads to my next quote, not by an oil and gas guy. This is not a quote from T. Boone Pickens. This is a quote form a very famous theoretical physicists, ______*1:34:55, who is most famous probably for his support of what is known as the Copenhagen rotation of quantum mechanics, which is somewhat beyond the scope of this presentation. But he was famous for making this quote and it is a true statement, prediction is in fact very difficult but I don’t think his point was that you shouldn’t do it. I think his point in fact, if you know something about his career was that you should do it and that is part of what science can help you do and what I would like to talk about today is risk assessment and the role that scientific quantitative risk assessment can play in making decisions about our energy policy. I would like to start with an example of one situation that we have all come to be familiar with where prediction failed. This is a spree shot from a news program showing one of the early days of unrolling or unraveling catastrophe of what we're seeing in the Fukushima Daiichi Power Plant in northeastern Japan. This is one of the initial catastrophic explosions I think in reactor number three and this has been on the news relentlessly. You're living in a hole if you haven’t seen this and it is a terrible, terrible disaster. The Fukushima Daiichi power facility utilizes six 1970s vintage general electric mark one boiling water reactors. These are huge industrial facilities. It is a very common technology. There are a number of other boiling water reactor facilities in Japan and there are many of them using essentially the same technology here in the United States. The way the news has described the events that have occurred there has been mostly to emphasize the role that a terrible earthquake 9.0 on the Richter scale and an even more devastating tsunami wave that occurred as a result of the earthquake and the impact that it had on the reactor in operations. I want to suggest to you that in fact although those were events and they did occur. They did have relevance to what has been happening at the Fukushima power plant they weren’t really the only initiating events. The more direct cause of what we have been seeing on the news was not so much those events, but it was the consequent breakdown or failure of the backup electric power system and resources to operate after the facility underwent a programmed shutdown. When the earthquake hit these plants are designed to automatically shutdown their operations, but that means that they are not producing their own power and they need to have access to energy resources in order to keep cooling systems pumping. If that is not available there are backup systems that should be in play. In Fukushima there were battery resources that should have been operating, those expired after approximately eight hours. Emergency diesel powered electric powered generators failed to start properly and to operate the cooling water system that is required in order to manage at the temperature both at the cores and at dispensed fuel rod storage are within another reactor. Those events could have been caused by lots of different things. They didn’t have to be an earthquake. It didn’t have to be tsunami. Any one a number of

different scenarios that could lead to regional blackouts on the power grid could have lead to these events, both in Japan and potentially here in the United States. We could have regional power outages caused by mechanical failures within the grid. We have seen that happen in the east coast on some occasions over time. In California we have seen occasions where we have had short term power failures or brown outs caused by essentially mismatches in power supplied demand and supply within the grid. So that was really the problem. The breakdown of the electric power arrangements that were necessary in order to keep cooling water systems operating and a whole bunch of different causes could have been what prompted those events. The designers of the Fukushima power plant did do some risk assessment back in the 1970s. They did what is called sometime scenario assessment or scenario evaluations to try to figure out what kinds of risks might be presented at the facility and in fact they did consider the potential threat of tsunami waves coming in impacting these facilities which are located in the earth underlying area close to water for a variety of reasons. The problem is when they looked at these individual scenarios, in particular when they looked at the tsunami scenario they looked at a wave that was approximately six meters high when in fact the wave that came in and washed over the facility last four meters higher than that. What they didn’t do which maybe they weren’t well equipped to do at the time for various reasons was to do a systematic quantitative and probabilistic assessment of the risks of these extreme events happening beyond the scope of the particular scenarios that they had to consider. Since the time of the assessment that were done for Fukushima our understanding of the capacity to do the sophisticated science phase of risk assessment has increased dramatically and today in a variety of different engineering and regulatory settings there is widespread acceptance of the use of techniques that are generally are described by a rubric of probabilistic risk assessment applied to any number of complex delusionary and environmental problems. Now I am going to mention what probabilistic risk assessment is a general way, but I worn you I am not a scientist. I am not a risk assessor. I am not an mathematician. I am not a statistician. I am a lawyer, but being a lawyer I do have a sense and having practiced in the area for some period of time the sense of the value of science and doing rigorous risk assessment and making sound regulatory and policy decisions and I want to come back to that principal again at the end of my presentation when I return very briefly to talking about hydraulic fracturing. Very generally, PRA analysis probabilistic risk assessment uses probability distributions to characterize both the variability and some uncertainty in different types of risk assessment, variability to find its true heterogeneity or diversity in an system or population and uncertainty being what you think it is. It is a lack of good data. Uncertainty is caused by a lack of knowledge. It is not caused by a variability of set of circumstances that are being considered by the risk assessment. It is just that you need more information, but you don’t always have perfect information and the whole point of using a probabilistic strategy towards the evaluation of risk is to recognize that in any cases data is incomplete, knowledge is insufficient and we have to make policy decisions and we have to move forward even in the face of those uncertainties and the question is really how we do it. Now again we could sit here and we could talk about some of the details and the use for example my karma analysis, which is a way of generating these complex probability distributions and making assessments. I don’t know anything about that frankly. So I am not going to bore you this afternoon by talking to you about the mechanics of that. But what I wanted to talk to you about is the fact that it exists, the fact that it is legitimate and the fact that it is used. Here in the United States as I said these techniques have been developed over the last 10, 15, 20 years and have been applied in a number of different regulatory and environmental law settings. In the

United States these techniques have been incorporated into the regulatory program specifically administered by the U.S. Nuclear Regulatory Commission. In the mid 1990’s the NRC adopted a policy specifically endorsing the use of probabilistic risk assessments in reviewing licensing determinations and applications for re-licensing by nuclear plant operators and it encourage the use of probabilistic quantitative risk assessments and evaluating the risks associated with nuclear power plant operations and interestingly and you will know from the second principal that is up on the slide, one of the reasons that the nuclear regulatory commission endorsed the use of these techniques it was a way of reducing what they call unnecessary conservatism. Inherent conservatism based in analysis of risks that don’t take into account uncertainties and therefore have to overcompensate in engineering and technical design facilities. PRI was actually viewed by the commission as a way of maybe ratcheting down a little bit about the level of conservatism that was put in to design for a nuclear power plant applications. The good news is that it was recognized as a way of being more precise, reducing both conservatism and maybe reducing also too laxing approach toward regulatory constructors. Outside the nuclear context, the use of probable... I am sorry I am skipping ahead one slide. I'm going to skip right through this one. This is just an example of how the commission encourages the use of probabilistic risk assessment on a tier way of going through the nuclear facility application process. I am going to skip that. The real point I want to say is that even outside the nuclear context the use of probabilistic risk assessment techniques has been widely used and proved by for example the U.S. Environmental Protection Agency in a number of different settings. Since the mid 1990s EPA has formerly adopted and improved the use of probabilistic risk assessments in its regulatory programs in a part of the law that is administered by the EPA that I work with more commonly which is the federal Superfund program and has been specifically acknowledged and encouraged as a tool for understanding and responding to the risks and cleaning up hazard waste sites and EPA has developed very extensive guidance documents to describe and to all risk assessors and site managers of how to go about doing that. The bottom-line is that PRI is a useful tool for assessing the variability of its safety in complex environmental settings in general. It has been recognized as such and it should be recognized whether it makes sense to do so. Including evaluating the risks that come up in the assessment evaluating different energy alternatives including but limited to hydraulic fracturing. As I said the use of PRI has been acknowledged both in regulatory settings, but it is also has been acknowledged from an engineering point of view in other settings that are very ______*1:47:16 to the kinds of situations, which we find when we are looking at proposals to conduct hydraulic fracturing today by the oil and gas industry. It has been used at a technical level to develop assessments of the risks of managed pressure drilling in offshore application, taken offshore off oil platforms. It has been used to develop quantitative assessments of risks that may be associated with the long term geologic storage and carbon dioxide, which is a technique that has been endorsed in various circles or tools that are endorsed in various circles to help up respond to the challenges of climate change, carbon dioxide not putting it in the atmosphere but putting it back into the ground. So it is recognized as a tool that can be used to develop policy decisions and then analyze risk in a regulatory setting as a tool to understand and analyze risk and complex engineering and environmental settings. It should be used as a tool to analyze and evaluate the risks associated with hydraulic fracturing and potential impacts to ground water as a result of the application of that technology. That brings me to the last point of my presentation before I'll shift over which should it and is it being used? Is it being encouraged to be used? One of the things that I did in preparing for the presentation this afternoon was to take a look at the draft study plan that was

issued by the U.S. Departmental Protection Agency just a month and a half or so ago in February. As a result of the mandate from the U.S. congress to undertake studies to evaluate the potential impacts onto water sources form hydraulic fracturing. EPA over a period of time put this plan together which intends to implement now, going forward generating initial proposals and recommendations as early as 2011 and in reading the report and thinking about the reports approach to risk assessment, one of the things that I was struck by was its sustained focus on the use of case studies and scenario evaluations. Not unlike the kinds of scenario evaluations that we saw the folks design the Fukushima plant use to not such a necessary good result. Now there may be good reasons for the reluctance of the EPA to endorse more aggressive approaches to the analysis of risks. Some of those justifications may have to do with their limited regulatory jurisdiction under the safe drinking water act, if not there are other statues and the fact that this is an area that has been left largely to regulation by state agency, there could be other reasons as well, but that doesn’t necessarily mean that it is a good idea. What I wanted to do was just end now my presentation by just referring you all to a principal regulation that is known as the precautionary principal. For those of you who have not heard that expression which is more commonly known and understood outside of the United States then in the United States let me read to you the official Wikipedia definition. That is this, if an action or a policy has a suspected risk of causing harm to the public or the environment then in the absence of scientific consensus that the action or policy is harmful, which I think is pretty much the situation we have in the case of hydraulic fracturing then the burden of proof that it is not harmful falls on those that are proposing to take the action, which in that case is the oil and gas industry. As I said the use of precautionary principal is accepted largely outside of the United States. It is an established principal in for example the regulation of chemical exposure risks and the European Union for its REACH program. Even in California we see application of the precautionary principal and some of our environmental and regulatory schemes like California’s proposition 65 safe drinking water and toxic enforcement act. So it is out there. It is accepted. We do use it here in the United States and it seems to me that it is at least worth considering a more robust application of the precautionary principal in the context or policy making decisions about hydraulic fracturing and that is my message today. Thank you very much.

Michael C. Dorf: Thank you very much Kevin we will now hear from Ted Eisenberg who is going to talk about the dangers of industry funded science.

Ted Eisenberg: So before I start I have tried to go around the room and recognize every face, which I don’t, but I don’t see anyone who has gone to Cornell Law School longer than I have. I am overwhelmed by what students have done. I don’t think we have seen anything like it in 30 years so congratulations. So the topic is law and science and one theme of each panel that I have been to is we really want to fall back on the science because we really got trust that. In the end I think everyone has said let the science determine it and we're especially trustful if that science has a U.S. in front of it, like the U.S. EDA, or the U.S. FDA that then we can do it, or perhaps if it has been an industry person that has worked a long time we can be trustful of that. So in litigations since this is law and science plaintiffs can cheat, right? If you want to see a classic example of it or I don't know if it is classic or an extreme example of it, go read _______*1:54:04 liability litigation, a Texas federal court case where the plaintiff has got expert witness, but literally they don't. The case is of _____*1:54:15. The case was a very bad incident. This is the article that I am talking about, but in the history of science I think the

modern era industry as far plaintiffs in distortive science. You can take tobacco. You can take asbestos. You can take Exxon Mobil and its research on putative damages. You can take the welding industry. I don’t know anything about the fracking industry, but I am skeptical. It doesn’t matter if the U.S. EPA says one way or the other because they can prove that as well. So the example, I want to think big about science in the sense that we really have to be skeptical. I am not sure who to trust, but I also want to think small by a very concrete example. I see a lot of gray hair in the audience or absence of hair completely and I am predicting that one in four of the people with older hair is on statins. If you are not on statins your parents are or one of your parents. Approximately one adult in four of U.S. over 65 are on statins or Lipitor is advertised. This is an ad that I took from off a t.v. screen or the Wall street Journal, Lipitor the best selling drug in the world for many years. A billion dollars a month in sales in an era of high costs reducing the risk of heart attack by 36%. This is, I don’t know if you live around here and you watch t.v., you have seen a Lipitor ad. You almost cannot believe it. They're not saying this anymore. You don’t see the big 36% up there and I think I can take some credit for that. This is the trial that so if you want gold standard science we would all like to believe that the random controlled clinical trial with results submitted to the Food and Drug Administrations is the gold standard in science. It is a lot more ______*1:56:14 than environmental stuff, which only pops up when we have a disaster. So we really believe in the regulation of science here and we really want it to be right. If any area generates good strong expectations about good science is medicine I think. This is the clinical trial that's applied the basis for the FDA approving Lipitor to be advertised to reduce heart attack risks and to be prescribed for heart attack risks. This is the article. It's called ARCT, all random control drug trials have fancy name. It was published in one of the two or three leading journals in the world ______*1:56:51, the medical journals of the world. These are the characteristics of the patients basically they had high blood pressure and at least one or two other risks. Unfortunately if you are a man that is a risk factor to part people and if you are over 65 that is a risk factor. So most men start out with two risk factors. Women it kicks in a little bit older because they tend to live longer. Lets jump right to this one it is prespecified subgroups. The primary endpoint is whether heart attacks were reduced and everything to the left as you are looking shows the drug working better than the placebo. Atorvastatin is the name of Lipitor in the scientific world. Anything to the right of that white vertical line shows the drug doing more harm than good . There is an interesting exception, the pattern, the drug doing good. It is for this tiny little small group, subgroup, known as women and what you see is that the, I mean one FDA controlled one random control trial t was the FDA based approval. Lipitor actually reduced the risk of heart attack in men by 41%. It increased the risk in women by 10%. I have never seen that advertised. I have never seen that publicized and their defense would be, well it wasn't significan'tly different. Okay but I have been put in that trial I would like to know that it is going in the wrong direction rather than in the right direction. I should say I got into this because my wife. Many postmenopausal women their cholesterol shoots up. Cholesterol shoots up in America you get put on a drug. The doctor says well we would have to test your liver before and after. That scared her and so she said does this work? So I got into the medical literature. I read every random control drug trial. There has never been, well we have to divide the wealth of primary prevention and secondary prevention. The secondary prevention is if you've already had a heart attack, you have diabetes, you had a stroke. Primary prevention is you are simply a healthy women whose cholesterol shot up and we are trying to prevent something that secondary treat. There has never been a primary prevention random controlled drug style of Lipitor that showed a heart attack benefit for women. It shows it

for men, interesting. The label for Lipitor says results for women were inconclusive, which is true if there weren’t enough women in the study. So the scientific question is does Lipitor reduce the primary end point of heart attack in females in a primary prevention context. The legal medical question is should the FDA be the exclusive monitor of drug company behavior or should we perhaps allow those really ______*1:59:43 lawyers to play a role and also regulate the medicine. So here is the question which I think we can answer as an audience. Given the uncertain results of the study for woman what does the science call out for? More study of whom? Woman and Pfizer to its credit did that. They were so proud of the study they wrote an article before they had the results. Not a misleading article, they just said hey we are doing this really important study, right and since studying heart attacks is hard because they are relatively rare events and you need to study for a long time in a lot of people they use an intermediate endpoint, arterial thickening, carotid intra-mediate thickness, IMT, right and they study and notice the last sentence the cashier, always the nice name, the cashier trial is the first randomized medical trial to examine the effects of a statin alone or combined with hormone replacement therapy, that didn’t turn out so well. The affects of the statin alone are combined with hormone replacement therapy on the structure and function of the carotid artery as early markets of arterial sclerosis and postmenopausal women with mild elevated cholesterol. The results are expected for 2007. Right? So if anyone has an internet and computer go find the published results. They never published them. Unfortunately they did register them with clinicaltrials.gov, which a lot of leading journals require registration before they even consider your article. So if you don’t register in advance you don’t get in. So if you poke around the internet long enough you will find clincaltrials.gov. Here are the results that you get here. There was no observable difference between the drug and placebo in mean change from baseline. So they did the study the science called for. They limited to woman, they had enough woman and they found no affect. They wrote no article. If you go to the Pfizer web site and look at information, try to find the results of ______*2:01:52. If you look at information for professionals, information for your doctor. Try and find the results of cashier. They have never been published. The only hint of its existence is because they made a mistake of registering it with clinical trials.gov. because they thought they were going to get a result that they wanted to publish. So what is the reaction to cashier? So I told some newspapers about this and they, you can always trust the securities market, an analysis found it and BMO capital market analyst, Robert _____(s/l Haslett) of the unpublished cashier study on the web site go to *2:02:25_____.org. _____(s/l Haslett) said the study raises questions about the IMT imaging technique, not about the trial because we know the drug works. So we get rid of that. I hope this isn’t being lied into Cornell wild, Dr .______(s/l Mantino Gato), who is the Dean of the Cornell Medical School told our lawyer that Lipitor has been proven to reduce clinical trials in primary or secondary prevention. So the cashier trial raises the question that the imaging modality doesn’t truly capture the affect of the lipid lowering agents and so I summarized there is no article publishing cashier. The medical establishment says well dismissed cashier. The simple question remains, show me a study of Lipitor primary prevention woman without heart attacks it would show that it did any good whatsoever. This drug sells a million dollars a month. It is the best selling drug in history. It will go off patent soon, but that is another issue. I will stop there because we are running a little behind. The bottom-line is I doubt that the science and the energy field is a whole lot more reliable than the science published in leading journals by highly skilled researchers. The problem is when science is founded it is very hard to trust. The downside of that is if the industry doesn’t fund it who will. It is a very

difficult question but do not trust science simply because it comes out of the U.S. FDA, the U.S. EPA or anywhere else. You have got to do it yourself.

Michael C. Dorf: Thank you very much, Ted. We will know hear from our final panelist Bradley Karkkainen.

Bradley Karkkainen: Thanks also thanks to the organizers of this talk and informative conference. I am really honored to be here. I am going to talk in just a very general way of environmental decision making under conditions of uncertainty, but first a little bit of disclaimer. I know no more about hydrofracking and the scientific issues involved that I have learned the last couple of days at this conference which is considerable I must say. I am certainly one of the most least experts in the room. So what I want to give you is just a quick detached 30,000 foot view. I, myself, to take any scientific questions or concerns to many of you. We all want science to give us definitive answers. We all want scientifically informed decision making. Sometimes science just can’t do that. Science is by its very nature always incomplete or always additional things that we could study. Science isn’t just a fix set of facts that we can pull off the shelf to be assured that we have the right answers. Science is about a disciplined process of inquiry and sometimes the science takes time. So sometimes after all the environmental assessments or environmental impact statements and indeed probabilistic risk assessments of the two sometimes we are still left with uncertainty. What do we do in that circumstance? How do we decide? What do we do in a situation with high level of uncertainty about an adverse environmental consequences. I think there are three basic approaches and at least three approaches that I am going to talk about and finally these are not based on scientific they are based on value differences. One approach is to simply proceed with a proposed activity until the environmental risks or harms are clearly established and that we're satisfied with perhaps the harms discounted by their probability recurring of the valuable of a proposed activity. This approach has a certain plausibility I think and it is sometimes dressed up in fancy labels like risk-based or an evidence-based decision making. This is probably the default approach of our society. I think it tends to be the most common approach. I know we live in a free market economy. We tend to value personal autonomy. The general presumption seems to be that anything that is not specifically prohibited or restricted by regulation ought to be allowed to perceived and if we regulate or restrict activities knowing that we have no established reasons to do so. I actually think this view underlies most of our federal environmental statutes, as comprehensive as the might be, that this burden of this thing might seem to be to industries. It sounds scientific. It sometimes dressed up in the _____*2:07:32 of best science. I think it is always the best science compatible approach in the long run. It is an approach that might work tolerably well with you know hot information environment where the unknowns or uncertain adverse affects are likely to have more magnitude, but sometimes the science is much more uncertain. It is a big complex out there and we don’t understand it perfectly and sometimes the potential environmental harms of uncertain in the great magnitude and either irreversible or extremely difficult or costly to reverse may not have adequate information of the nature and magnitude of the probability of those harms until after we have already made the decision under this approach. That circumstance, this so-called risk-based or evidence-based approach may actually prevent science from informing decision making because the science necessarily takes place after the decision is already made. I just want to emphasize that this context that the absence of a proven harm is not the same as proof of an absence of harm. I think that is a very

simple basic principal that we ought to observe whenever the claim is made that there is no proof of harm, therefore we have to proceed. This leaves some, especially in the ______*2:08:59 environmental community through recommended precautionary approach, the approach that Kevin spoke of earlier. Roughly here the approach is to assume the worse and don’t proceed until we have enough evidence that we could proceed safely, especially in circumstances of high uncertainty and potentially large or catastrophic irreversible environmental harms. We do tend to approach ______*2:09:19, but I know Ted is critical of the FDA for example. But the basic structures of the Food, Drug and Cosmetic Act essentially reflects the precautionary approach that FDA is supposed to be precautionary, is supposed to demand evidence that pharmaceutical products and _____*2:09:38 devices and so forth are safe to use before ______*2:09:41. That is a fundamental and practical application of the precautionary approach. It is probably not an approach that we can use across the board because it demands too much information and it might restrict too many activities. So adjustments should probably be confined to high impact or irreversible harms. It is sometimes criticized as unscientific or anti scientific, but I think that characterization is unfair because the precautionary approach is not itself hostile to science at all. It simply shifts the baseline of the pertinent truth and demands that we have scientific information going to the safety of the proposed activity before we would be allowed to proceed. So it places science in a different place in the decision making process. Essentially the FDA approach would show it is safe before you would proceed. There is a third approach, possibly an intermediate approach, but family of approaches, loosely characterized, as an adaptive management. Now this is not a concept that comes up much in the context in the kinds of risk assessments that we are talking about here. The basic idea here is to proceed with caution and in a structured science driven approach to systematically learn as you go and adjust your course as you learn. The idea was first advanced in the natural resource management context by scientists, particularly the ecologist was _____*2:11:21 at the University of Florida and Carl Walters, official biologist at the University of British Columbia. Paul had first developed this idea in the context of environmental impact assessments as a scientist, as an ecologist. He was being called upon to participate into an environmental impact assessments in situations of high scientific uncertainty where we often didn’t have a good baseline data or some of the underlying ecological science was incomplete or really had not been done and he was being asked to make predictions about what the environmental impacts of a proposed course of action would be in a complex ecological setting or any interacting income _______*2:12:08 of uncertainties. So Holly suggested that a more scientifically more defensible approach would be to do the best job we could with the information at hand integrating across all our elements of scientific discipline. But to treat the initial management decision as a revisable hypothesis essentially. Testing in the real world what the actual impacts would be by careful monitoring of the environmental impacts and subsequent re-assessment that would then inform subsequent decision making. Indeed Holly Walters went further and said that the initial management intervention could itself sometimes be structured as a kind of specific high scientific hypothesis designed to field test a particular hypothesis, tobacco likely effects with the information that is systematically feedback into subsequent ______*2:13:05 reference of decision making. The concept has subsequently been lively adapted in natural resource management context. Sometimes I have to admit in inadequate or constructed form or even an _____*2:13:24 form some might say to the point that adaptive management has become something of a dirty word to some environmentalist operating in those area and I have to say it is hard to point to a situation where we have actually gone adaptive management ______*2:13:44, but at least in principal I do think it is a sensible approach in some

context and with some importance caveats. First of all it is not a panacea. It won't work everywhere. In particular it won't work where we are making irrevocable or practically irrevocable commitments to activities that could have larger irreversible environmental consequences and in those circumstances I think we are much better off taking a precautionary approach and also and this is critically important in the natural resource management area it's not a substitute for rigorous ex anti-environmental assessment. I think this is where a natural resource management agencies have gone astray essentially saying we don’t need to the rigors assessment up front because we will learn more as we go along with adaptive management. Well that is not the idea at all. The core scientific idea of adaptive management is you do both. You do your best environmental assessments up front, identify the potential uncertainties and then structure the management decisions and adaptive management actions and to bring out as many of those remaining uncertainties as you can to better inform your predictive problems and better inform your decision making over time. In conjunction with that the other piece of it is related but perhaps a subtly different point is a notion that adaptive mitigation. Sometimes we just need to decide and proceed with a course of action. Often in connection with that proposed course of action we may recognize a _____*2:15:19 problem and environmental impacts that can be partially mitigated, but what we don’t do is a good job of making those mitigation actions adaptive. That is we generally just rely on the principal on the ex anti-environmental assessment, predict what kinds of mitigation measures will be needed. Predict what kind of effect they will have and I think what is called for in this kind of circumstance again a rigorous followup monitoring of the effects of the mitigation actions to ensure that they are as effective as they are predicted to be ex anti so that adjustments can be made down the line to reduce the adverse environmental impacts and to identify any other adverse environmental impacts that we didn’t identify in the first round of environmental impact assessments. So in the interest of time I better leave it right there.

Michael C. Dorf: Thanks to all the panelist. I think I am just going to ask one question and then open it up to the panel and then move onto questions from the audience and it's a question that I think cuts across each of the three presentations. So in the medical context that Ted you were talking about I think we might characterize the precautionary principal as in a sense as going back to _____*2:16:37 (s/l hypocrogies), right? First do no harm so that if there is a proposed drug that you don't know whether it has any benefit, well you know it's going to have some side effects and do some potential harm. So if you don’t know it is beneficial, don’t use it. That makes some sense in that context. But in one context where the precautionary principal is invoked, but there is no possibility of doing nothing, right? So failure to exploit some proposed new energy resource means relying to a greater extent on existing methods of obtaining energy like nuclear, like coal, like oil, all of which have their own known impacts. So in light of that the question is really for Kevin and for Brad to some extent. Does in this context does the precautionary principal, right, boil down to something like nimbyism because all you are really doing is saying don’t try this new thing because the locals don’t like it, keep doing the other thing, which has all sorts of other problems, but at least we have been doing it for years. That is a devil advocate question perhaps but designed to see what you say first.

_____*2:17:51 (s/l Kevin Haroff): No it isn’t a not in my back yard kind of approach necessarily, although I suppose it could be used to disguise that kind of attitude. I think the comments about it not being antagonist to science the precautionary principal in fact being

consistent with science in simply a general tool for making policy decisions based upon whatever data is available and simply shifting the burden of proving the outcome or what the right outcome should be. It may be that you end up with the same ultimate decision. The question is just who has got the burden of persuasion and I have kind of thought it was interesting talking and listening to your comments about adaptive management. You know I am not an expert by any means on hydraulic fracturing. I know a little bit about it, but just enough to be dangerous to some people but cheerfully don't. But it seems to me that that is exactly the kinds of circumstances that you describe were adaptive management or the traditional approach to regular risk taking is not appropriate because there is a significan't risk of irrevocable and substantial damage to ecology and resources so not to say that you don’t do it. But you don’t just do it without a very careful approach that studies the problem and shifts the burden of proof to people who want to take the action and in this case I think it really says we don’t do hydraulic fracturing. It is just we make sure that rather than assuming that just because I have seen 3,000 wells fracked I have never seen a problem. That we shift the burden of proof to the people who are proposing to do the work. Demonstrate to the satisfaction to the rest of us who have to live with the consequences and then make the decision about whether to go forward.

Michael C. Dorf: The question is are you doing the same thing with the nuclear power plants or the offshore well drilling you know, you can apply the precautionary principal to things that are already going on because they have unknown risks and ______*2:20:12.

Kevin Haroff: That is absolutely true but I think what can be done with information that is available with the risks associated with each of these different technologies is make an assessment of those risks and then make a comparison and use the results of that comparison to make your policy decisions. Quite frankly I think if you look at studies that I don’t know if people here like nuclear power or not. But if you look at studies just simply of the high impact types of risks associated with different policy alternatives like natural gas production and transmission and like oil production and transmission, like nuclear energy. Folks that have looked at the fatality rates associated with those different technologies and produce reports showing clearly the safest one of those technologies is nuclear power. So I think we can use that information and we can make decisions that are compared to relative to different technologies. It is not going to be a perfect science, but it is not an excuse not to do it.

Michael C. Dorf: Okay I think the question for you guys I have a whole bunch of cards. Yes sir and we got a microphone coming your way.

Audience member: My question again gets back to corporate person and the influence over the years that the person had question around some of our amendment rights impact on the grandfathering or the transferring of studying these drugs in your instance Ted to show the burden of proof on the people that are wanting to make a profit and minimizing over time the public’s input as a funded way of looking at those same issues and having it in a domain of the public instead of pushing it over to private industry and then the added threat of corporations using the takings clause and if and from the fifth amendment if they are not permitted to proceed with a permitted business interest and profits in the future. This is a complicated question but since the corporations have assumed the person they have those takings rights under the fifth

amendment to sue if they aren’t permitted proceed with an authorized business expenditure or business entity.

Ted Eisenburg: I think I may be too simple about their rights to sue for taking of the property law but I think the first part of your question resonated a little bit with some of the other themes which is everyone wants to study the problem. You should based the results on the best science available. I think the implications to what my sort of feelings of what I studied is that is not good enough. Because if you say study the problem and there is a strong economic interest they will study the problem and they will have very smart good people in leading journals show what they are about to do is harmless and the science of studying science and it overwhelmingly shows that results come out in favor of the funding authority. So I don’t know what the alternative is except governments to try and get around and balance the playing field. Because there is no doubt that where there is nuclear power or drugs or whatever people understandably with a strong economic interest are going to hire people who you know even unintentionally get results that they want and in the case of the drug company bury the results that they don’t want so that you never even see them. The imbalance is enormous. Brad sort of said I am not against the FDA. I don’t even think they do a bad job. The playing field is just so unbalance that the FDA’s job is hopeless. Pfizer’s advertising budget for Lipitor exceeded the entire FDA budget for regulating drugs. So there is just no way we can keep up. I think they are well intentioned they are just completely out gunned. It is the air strikes on soldiers. The planes are going to win. The soldiers are on the ground shooting up and the air strikes are shooting down. There is such an unbalanced playing field there is no way for the most consciences government agents to do their job right.

Bradley Karkkainen: Let me address the takings question. I do teach property. The constitution says nor shall private property be taken without just compensation. It doesn’t say that you need to be a person it just says it needs to be private property that is taken. So I think those are separate questions. I actually, I am very sympathetic with Ted’s position with respect to non cooperate sponsored research and I highly prized of the work that he has done in this area to expose some of the problems with it. I guess I just want to point out that there is a trade off because the government doesn’t have the resources or hasn’t allocated the resources to do the job on its own either. In the area of toxic chemical declaration for example there are approximately 60,000 or maybe 70,000 chemical that have come into widespread use in areas of industrial commercial applications since World War II. Only a tiny, tiny fractions of those have ever been subjected to even most basic screening for potential toxicity. Most of them are probably are safe, but we really don’t know and the EPA certainly doesn’t have the resources and OSHA doesn’t have the resources to do that job. So I think there are certain circumstances on which with proper controls it is appropriate to sign ad cost actually to the industry. Assign the cost to the industry not to say we trust the results of the research that they fund directly those are basically different things. Indeed European union is moving I think more in that direction through its new each chemical regulation, which basically says to the industry for any chemical profit that you are manufacturing or importing into Europe you have got to show us data how as to how it can be safely used as placing the burden on producing that information on the industry I think it is subject to the same kinds of information quality problems that Ted is talking about but I think that is going to give us a lot more information that we have gotten from the EPA with the EPA

not having the resources to study and place that entire burden on the regulatory agency. So it is kind of a difficult balancing question.

Michael C. Dorf: Okay I am going to take a question from one of the other rooms written to me on a card. It says it is Eisenberg, but I think it is actually for Haroff, but you guys can decide who it is for. The question is can probability risk assessment be used when you have a black swan event, i.e. a 1,000 year tsunami. Do you think it would of made a difference?

Kevin Haroff: There are of course limits to any standardized quantitative approach to the assessment of risk and there will always be the black swan event that nobody anticipated, nobody understood, nobody can capture or what is it, it's the Donald Rumsfeld, the unknown, the unknown. If you don’t know it and can’t know it then you are not likely to be able to capture its implications in any risk assessment. That said you are much more likely to intentionally capture the implications of a black swan event if you are doing probabilistic risk assessment where the probabilities of something occurring like the 9.0 earthquake or the 10 foot tsunami wave or at least there is a number on the chart that addressed, even if it is not something that you thought about ahead of time. So these techniques are not fail safe. They are not intended to be fail safe but they are intended to facilitate decision making in the absence of certainty and knowledge and we should use it.

Michael C. Dorf: But I assume that was not a reference to the film Black Swan? So I will take a question from the floor.

Audience member: Recently on several news networks there were at least two scientists that had said that they had left the company and I assume maybe it was GE when they were doing the nuclear power plants that are now in question in Japan which we also have here. They left the company because they felt the company wasn’t preparing for the worst case scenario. So if you could talk about that and also my biggest beef is hearing from the industry and especially the previous panels one of the lawyers who works for landowners constantly saying there is no pollution. There is no contamination. Where did they get the absolute no? Is there a name for that because that doesn’t seem like that is sound anything, sound science, or sound practicality?

_____*2:30:16 (Panelist?): Let me just say something in response to your first comment and I don’t know the personal circumstances of people that you identified some of the employes of GE or why they resigned or any of that. I think it is demonstratively true in Fukushima the case that a genuine risk that that facility faced was not captured in the climate or the operation of the facility. It is just true and hindsight is always 50/50 and that is great. What I worry about as a lesson that people may take away from what has happened in Fukushima and I try to lead a little bit to this in my presentation that quality and satisfaction of the techniques that people have available to use now, the assessment of risk, complex engineering and environmental settings has gone off dramatically since those plants were designed and built in the 60s and the 70s. In the grand scheme of things those facilities even though they are operating and many of the companies run the plants like to do as I just did a few minutes ago, tell their safety record. Those are old facilities. They are old designs and the strategies that people will employ today in engineering nuclear power facilities. I am not trying to advocate for nuclear power here by any means but the techniques and strategies of people utilize to sign those facilities and to manage

and prepare for the risks associated with unanticipated events is so much better than it was 40 years ago. We should not be deterred from taking and having an open mind in terms of evaluating nuclear power along with full range of different alternatives that we have in our contemporary energy plans.

Michael C. Dorf: I think I will take another question from the audience here.

Audience member: I just curious about the conversation about theory where Darwin would fall into this and then say what would be. Do you think he would say in order to evolve sustainability that we shouldn’t rely on the fossil fuels? Are we just delaying the inevitable?

_____*2:32:57 (Panelist?): Darwin would say let nature take its course.

______*2:33:02 (Panelist:?): Just one cautionary note on the notion that our ability to systematically evaluate the risk is so much better now than it was before. We had the best minds in the world controlled not nature but human events and having wonderful algor rhythms to study investments and finance and somehow as of 2008 our cutting edge risk _____*2:33:33 in a human environment didn’t work and I don’t thin we can get away from that. We are human. Maybe Darwin would agree with that.

Michael C. Dorf: Let me ask take it or actually I'll come back to the _______*2:33:50. Right here.

Audience member: Yes. Would it be fair for us to assume then that the gas companies probably have some studies that they are sitting on that do show adverse affect that they have not published?

_____*2:34:03 (Panelist?): How can I accuse them? There are certainly studies in the medical field that suggest unfavorable results tend not to get published. I think a practical point which probably what I think is you are going to depend on the industry to fund this because they got the money and no one is going to raise taxes to fund these things. But maybe there is a role for law to play. You take the model where the industry pays to fund things but they convince you, or the government that the people that are funding are sufficiently independent, that we can trust them. They don’t on the radio for drug talks. They don’t get consulting fees. If you convince us that they are sufficiently independent, we will let the law kick in and give you immunity from certain kinds of suits. Right? So if Merck has Vioxx research done by completely independent researchers where they have absolutely no control other than writing a check to fund it we will say you get five years of _____*2:35:01. If they do it themselves, you're strictly liable.

Bradley Karkkainen: Alright I don't want to speculate on the information that the gas industry has that they might be sitting and there certainly are plenty of examples throughout history of just that sort of thing happening. There is the whole history of tobacco for example and the information that was available to the tobacco companies. But I have to say in an environment where we place the entire burden on government to produce information before it can regulate and that is the basic structure of most of our environmental laws, that sometimes creates a disincentive for an industry to even study these questions. That is certainly true in the chemical

industry. I think what we see there is a consistent pattern of willful ignorance, if you will, because if they studies the toxicity of the chemicals that they are introducing and they turn up adverse toxicity pattern then they are compelled to report that to the EPA and that heightens the chance that it can be regulated. So we are better off simply not knowing. I think that's a well established pattern but I don’t know which of them two can't's that the gas industry falls into.

Kevin Haroff: Can I just one comment or actually two comments. One on your comment, Ted, about sophistication of risk management techniques in the financial services industry and that's, it's a well taken point, but I guess my perspective on it is the concern not so much that they were failings in the sophistication of the analytical techniques that were developed and used by financial services industry that lead to the problems with experience in the nature is the fact that risk managers and decisions makers in our financial institutions use that data while they were playing a game of craps. So it wasn’t to me so much a problem with technique. It was a problem with decision makers. It was a regulatory problem on how we allow decision makers to function in that environment. On the question over here on whether or not the oil and gas industry has reports and information and data that would be relevant to the assessment of the environment and health risks associated with the development projects. I don’t want to speculate either but I will say that I started in the first five years of my career as an oil and gas lawyer at large from a multinational oil and gas company whose name I won't mention. But I can guarantee you that companies of the sort that I am familiar with operating at the level they do have a immense amount of resources both information and financial in an access to put together data to move forward with decisions. The question in mind is not so much is there informational and financial in an access to put together data to inform decisions. The question in my mind is not so much is there information that the industry and others have developed. My guess is probably there is. My question is have we done good enough job socially as a society in putting that information together in a way that is useful for decision making purposes? I mean I think if you ask the people who promote hydraulic fracturing they will say we have years and years and years of experiences using this technology in the field and we are subject, I heard this yesterday, we are subject to a variety of state laws and regulations that require close monitoring of what we do and we generate and commence an amount of data as a result of that. Maybe not all of that data is the kind of data that you want to use to make _____*2:38:46 (s/l worse) decisions, maybe some of it is. As I said I think probably as much, if not more of the problem there is not so much lack of data. It is a lack of people putting that data together that can be useful for decision making purposes.

Michael C. Dorf: Let me refer to one of the questions from one of these cards. This person writes that the safety record of nuclear plants is tailored as good, but there is certainty that the plants will produce dangerous waste. The same is true for hydrofracking. How are these certainties, the certainties of the dangerous of the waste accounted for in probabilistic risk assessment?

_____*2:39:29 (s/l Kevin Haroff): They are properly part of the assessment. You know there are certain things that we know about risks associated with nuclear power production. I mean I suppose that the good news with respect to those risks because they are understood people have actually spent a fair amount of time and energy figuring out or at least trying to figure out solutions for dealing with them. We do have technologies for managing spent fuel and nuclear

waster. We kind of know how to do that. What we have not been able to do in my mind is overcome what are not always well founded fears about the risks associated with those _____*2:40:19 policy decisions, like building a national repository for hazardous waste in this country. We need to move away from basic policy decisions on speculation and fear and go forward and I think if we do as a society adopt that mentality a little bit more we will be making better decisions over all.

Audience member: I was interested if the industry is going to do the research. I recently and I can’t remember which study it was but a state case study that they were going to make a report but the industry apposed part of the information. So that was deleted from the EPA report and I am sorry, I apologize I can recall that, and I suspect that that happens more often than not. Much as in the medical research, if you had an independent researcher but most of his or her research or that institutions research funding money was coming from drug companies that would tend to make some of their results askew and it has happened somewhat in the past. So that I think that in looking at industry to be the one who choses the researcher. It is very difficult if that is not being move around to different people and for people not to become dependant on that source of money. How do we avoid that?

Michael C. Dorf: How do we solve that?

Ted Eisenberg: I have a better solution. The only truly independent people are of course academics. So what this calls for is massive contributions to colleges and universities so that researchers don’t have to depend upon grants from interested parties.

Michael C. Dorf: That actually leads to the next question on one of these cards. It says here Cornell stood to gain financially from losing its labs and President Skorton examined this topic which resulted in Cornell not leasing its labs. Meanwhile Penn State actively promotes gas drilling. Should studies on gas drilling that are generated by Penn State be considered less reliable? I understood that as a rhetorical question.

Ted Eisenberg: Penn State beat Cornell at wrestling. They are number one and we are number two.

Michael C. Dorf: Question over here.

Audience member: It's kind of along the same lines. Since universities have become so dependent on corporate or _____*2:43:16 by making money to give back and this is really much more of a societal problem in that we don’t value education enough to put our government money behind it. So a lot of professors are relying on this cooperate money to fund their research, which automatically makes it a suspect. So as I am going to ask the two academics on the panel, what you recommend seriously for the public to do when reading research how do we distinguish between this _____*2:43:50? I mean it is fairly odd you can dig down and find funding in some cases and in some cases you can’t?

Ted Eisenberg: Part of the answer is education. There is no reason why, what I put up, right and people pretty quickly caught it. It was something strange about woman in the ASCOT study.

But if I said here read this scientific article many people would go screaming out of the room saying I would rather have my teeth pulled out. One thing would be to perhaps to encourage interest and sophistication in science education and the like and I think this spills over into law schools that so many issues that we face today are quantitative and scientific and law schools generally focus on traditional education and the lawyers understandably rely on their own experts to form opinions whereas it might be nice to have a cadre of lawyers who could actually think independently about scientific results. There is no good answer to what you want.

Audience member: May I do a follow up? In big conferences like this it would be helpful for the moderators to introduce the fact that there are those who have been funded by industry and those who have not. It has been fairly obvious to me which ones and which ones haven’t. But that would be helpful for future conferences.

Ted Eisenberg: I will take money from anyone.

Michael C. Dorf: I have not received money anyone but I would be happy to. We've got time for one last question and I got to call this gentleman over because he is closer to the microphone.

Audience member: I just want to ask a question about the probabilistic analysis that you spoke about is in fact I have done a lot of that myself. I think I want to ask you about the risk of that. I think it really goes to you Ted on this. From a statistical point of view isn’t it true that one of the disadvantages of that kind of analysis is that you do in fact think you have thought of everything. You considered every possible outcome, but unfortunately some of them have such a low probability that the likelihood of a ten meter high wave combined with an earthquake combined failure of this and failure of this is that black swan that is way off at the end. So you don’t do anything about it. But you think you have thought about it. You think you considered it. You might have been better off if you hadn’t done that analysis and you just thought what could go wrong here and I worry a little bit once you do that kind of analysis you end up with a factor or a quotient that says this is 93% safe and you comfortable with it, while in fact that risk is still there.

Michael C. Dorf: The question is?

Audience member: That is the disadvantage of that analysis. You were saying that you have considered everything so therefore it is good on that panel. I sometimes wonder whether the fact that you can say you considered everything is in fact its weakness.

Kevin Haroff: I appreciate that and I think there is a risk of that. People you know sometimes think that if they do any kind of analysis, the fact that they have done an analysis gives them good reason to believe what the outcome will be and that is not true, but that is one point. The second point is the kind of techniques that I think people have in fact developed over years are at least better as I said then some qualitative techniques that have been used in the past, which really don’t allow you to even probabilistically to capture some of these risks. The good news is that tend or that having probabilistic risk assessment that you perform of the defence strategies that were put in place at Fukushima there would have been a probability distribution that shows that there is some likelihood, maybe a small one but it is on the graph of a ten foot wave coming

across and washing ______*2:48:10. So I think it is a tool. It is not a perfect tool. I think one of the problems with that tool and it is not just restricted to that is that what it doesn’t do a good job is capturing the interplay of multiple risks coming at you from different sources. You may do a good job at evaluating the risks associated with certain kinds of precipitating events at a nuclear power facility. But you are not going to really capture that analysis, the risks associated maybe with babies drinking radioactive water in Tokyo and maybe there are other causes that are encouraging a confluence of risks occur that really create a black swan more than anything else and those are not going to be easily captured in probabilistic risk assessment because you are not going to have inputs to design and capture.

Ted Eisenberg: I think I have a practical rule of thumb taken in, this comes about from doing home repairs or instruction. That is often the contractor will come in for a price quoted. I don’t want to go bankrupt. I have to assess the risk that he won't. So assess the risk as best you possibly can and then double it and I have never had a contractor go over that.

Kevin Haroff: That actually would of work in predicting the wave at Fukushima.

Bradley Karkkainen: Just very quickly, I think those concerns about false confidence, false precision results you are getting when you actually have a lot of very soft various variables going into the input and are common to many types of quantitative analytical ____*2:49:57 tools. For example, cost benefit analysis I think is very much the same thing and I think these phenomenon are about understood, but I don't argues for making these things useful analytical tools but not necessarily the sole criteria for decision making. Because there may be some circumstances where the quantitative risk assessment tells us that the risks might be small but you might none the less have residual concerns that might let us overrule that.

Michael C. Dorf: One thing that we can say with certainty is that we are exactly five minutes over time. So I hope you'll help me thank the panelists.

***END OF PANEL FOUR***