Pages 1 - 213 UNITED STATES DISTRICT COURT NORTHERN DISTRICT OF CALIFORNIA Before The Honorable Vince Chhabria, Judge IN RE: ROUNDUP PRODUCTS ) LIABILITY LITIGATION, ) NO. M. 16-02741 VC ) San Francisco, California Monday, March 5, 2018 AMENDED TRANSCRIPT OF PROCEEDINGS APPEARANCES: For Plaintiffs: The Miller Firm LLC 108 Railroad Avenue Orange, VA 22960 (540) 672-4224 (540) 672-3055 (fax) BY: MICHAEL J. MILLER For Plaintiffs: Andrus Wagstaff PC 7171 West Alaska Drive Lakewood, CO 80226 (720) 255-7623 BY: VANCE R. ANDRUS AIMEE H. WAGSTAFF DAVID JACKSON WOOL For Plaintiffs: Andrus Wagstaff PC 6315 Ascot Drive Oakland, CA 94611 (720) 255-7623 BY: KATHRYN MILLER FORGIE Reported By: Lydia Zinn, CSR No. 9223, FCRR, Official Reporter 2 1 APPEARANCES: 2 For Plaintiffs: Weitz & Luxenberg PC 3 700 Broadway New York, NY 10003 4 (213) 558-5802 BY: ROBIN L. GREENWALD 5 PEARL A. ROBERTSON 6 For Plaintiffs: Baum Hedlund Aristei and Goldman, P.C. 7 12100 Wilshire Boulevard, Suite 950 Los Angeles, CA 90024 8 (310) 207-3233 BY: MICHAEL L. BAUM 9 ROBERT BRENT WISNER 10 For Plaintiffs: Lundy Lundy Soileau & South LLP 11 501 Broad Street P.O. Box 3010 12 Lake Charles, LA 70601 (337) 439-0707 13 BY: RUDIE RAY SOILEAU, JR. 14 For Plaintiffs: Andrus Anderson LLP 15 155 Montgomery Street, Suite 900 San Francisco, CA 94104 16 (415) 986-1400 (415) 976-1474 (fax) 17 BY: LORI E. ANDRUS 18 For Plaintiff Sioum Gebeyehou: Law Offices of Tesfaye Tsadik 19 1736 Franklin Street, 10th Floor Oakland, CA 94612 20 (510) 839-3922 (510) 444-1704 (fax) 21 BY: TESFAYE WOLDE TSADIK 22 23 24 25 3 1 APPEARANCES: 2 For Defendant Monsanto Corporation: Hollingsworth LLP 3 1350 I Street, NW Washington, DC 20005 4 (202) 898-5800 BY: KIRBY T. GRIFFIS 5 JOE G. HOLLINGSWORTH JOHN M. KALAS 6 ERIC GORDON LASKER HEATHER ANN PIGMAN 7 STEPHANIE SALEK 8 9 Also Present: The Honorable Ioana Petrou Leonora Lynham 10 Scott Duval 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 4 1 I N D E X 2 3 Monday, March 5, 2018 - Volume 1 4 PLAINTIFFS' WITNESSES PAGE VOL. 5 6 RITZ, BEATE (SWORN) 9 1 7 Direct Examination by Ms. Forgie 9 1 Cross-Examination by Mr. Lasker 97 1 8 Redirect Examination by Ms. Forgie 165 1 9 10 WEISENBURGER, DENNIS (SWORN) 168 1 11 Direct Examination by Ms. Forgie 169 1 12 13 E X H I B I T S 14 TRIAL EXHIBITS IDEN EVID VOL. 15 297 29 1 16 510 140 1 17 544 140 1 18 586 140 1 19 1277 140 1 20 1278 140 1 21 22 23 24 25 5 1 Monday - March 5, 2018 10:02 a.m. 2 P R O C E E D I N G S 3 ---000--- 4 THE CLERK: Calling Case Number 16-MD-2741, In Re: 5 Roundup® Products Liability Litigation. 6 One counsel from each side, please step forward and state 7 your appearances for the record. 8 MS. FORGIE: Good morning, Your Honor. 9 Kathryn Forgie, F-o-r-g-i-e, for the plaintiffs. 10 THE COURT: Good morning. 11 MS. FORGIE: Good morning. 12 THE COURT: And you want to introduce your team? 13 MS. FORGIE: Yes, Your Honor. I've got Dr. -- 14 witness -- our first witness, Dr. Beate Ritz, who's sitting 15 there. And Pedram Esfandiary, who's sitting there -- standing 16 up. David Wool. 17 MR. WOOL: Good morning, Your Honor. 18 MS. FORGIE: Mike Miller. 19 MR. MILLER: Good morning, Your Honors. 20 MS. FORGIE: These three are co-leads. 21 Robin Greenwald. 22 MS. GREENWALD: Good morning, Your Honors. 23 MS. FORGIE: And Aimee Wagstaff. 24 MS. WAGSTAFF: Good morning, Your Honors. 25 THE COURT: Good morning. PROCEEDINGS 6 1 MR. HOLLINGSWORTH: Good morning, Your Honor. 2 Joe Hollingsworth, from Monsanto. And with me is my partner, 3 Eric Lasker. 4 MR. LASKER: Good morning, Your Honors. 5 MR. HOLLINGSWORTH: And then to his right is 6 John Kalas. 7 MR. KALAS: Good morning. 8 MR. HOLLINGSWORTH: And next to John is Mimi Lynham. 9 And behind me -- sitting behind me is my partner, 10 Kirby Griffis. And behind him is my partner -- 11 MS. PIGMAN: Good morning. 12 MR. HOLLINGSWORTH: Good morning. 13 -- my partner, Heather Pigman. 14 THE COURT: Good morning. 15 So as you all can see, we have a special guest, Judge 16 Ioana Petrou, from the Alameda County Superior Court. I issued 17 a notification to all State Court Judges who are handling 18 similar matters throughout the country that they were invited 19 to participate fully in these proceedings, and make these 20 proceedings part of their record in their cases. 21 Judge Petrou is the Judge who has been assigned virtually 22 all of the California State Court cases throughout the state. 23 The cases have been coordinated in her courtroom. And so she 24 is -- she's joining me here, and will be here for the duration 25 of the proceedings this week. The parties in her cases have PROCEEDINGS 7 1 stipulated that these proceedings are part of the record in 2 those cases. So welcome, Judge Petrou. Thank you for coming. 3 JUDGE PETROU: Thank you. 4 THE COURT: We also are making these proceedings 5 accessible to the public, and to other State Court Judges in a 6 couple of different ways. 7 I've told the other State Court Judges who couldn't make 8 it out to San Francisco that they can listen in on the 9 proceedings through Court Call, and so we've got Court Call up 10 and running. And I expect that various State Court Judges, off 11 and on, will be listening in on the proceedings. 12 But also we're doing video recordings of the proceedings 13 through our Cameras in the Courtroom pilot project. And both 14 sides have consented to the recording of the proceedings. 15 Monsanto requested that we not post the recordings until 16 the week of hearings is done. And we will, of course, comply 17 with that request. And so the video recordings of the hearings 18 will be up either at the end of the week or the beginning of 19 next week. And I've informed the State Court Judges that they 20 can access those recordings, as well. So I think that's about 21 it. 22 The only other thing I wanted to say to the parties for 23 purposes of tailoring your presentation, and maybe not spending 24 too much time on basics, is I just want to be transparent with 25 you about what I have read, and what I have not read so far. PROCEEDINGS 8 1 And obviously, throughout the week there will be more time to 2 do more reading in the evening; but so far I have read every 3 Expert Report. I can't promise you that I've read all of the 4 footnotes, but I have read every Expert Report. 5 I have read every brief that the parties have filed. 6 I have read the IARC Monograph, including the preamble. 7 I have not yet read the EPA reports. That's certainly one 8 of the things on my agenda to read this week. I'm trying to 9 think. 10 I think -- oh, and then all of the -- I believe I've read 11 all of the pertinent cases on causation, or all of the most 12 important ones. 13 So what I would ask is that you please work with your 14 experts to tailor your presentation accordingly, and presume 15 that amount of knowledge. 16 And, of course, there will be opportunity in the State 17 Court proceedings to supplement the record to the extent that 18 it's necessary. So with that, the plaintiffs can go ahead and 19 call their first witness. 20 MS. FORGIE: Thank you, Your Honor. Good morning 21 again. We'd like to call Dr. Beate Ritz to the stand, please. 22 THE WITNESS: Do I go up there? 23 MS. FORGIE: And, Your Honor, as a preliminary matter 24 we have put together books for the Judges, the Clerk, and for 25 Monsanto that contain copies of her PowerPoints, the Expert RITZ - DIRECT / FORGIE 9 1 Reports which I understand you already have, but they're 2 already in the book; some slides that she's going to use; and 3 copies of important medical studies that she'll be referring 4 to. I think that will make it easier for everyone. 5 So with Your Honor's permission, I'd like to pass those up 6 to the Clerk and Your Honors and to Monsanto -- or my partner's 7 going to do it. 8 (Whereupon a document was tendered to the Court.) 9 THE COURT: Thank you. 10 THE CLERK: Please raise your right hand. 11 BEATE RITZ, 12 called as a witness for the Plaintiff, having been duly sworn, 13 testified as follows: 14 THE WITNESS: I do. 15 THE CLERK: Thank you. Please be seated. Please 16 adjust your microphone. And for the record, please state your 17 first and last name, and spell both of them. 18 THE WITNESS: Yes. First name is Beate, B-e-a-t-e. 19 Last name is Ritz. R-i-t-z. 20 THE CLERK: Thank you. 21 MS. FORGIE: Can we put up slide? 22 Okay. Okay. So I did not realize that Your Honor had 23 read all of the expert briefs -- reports. I appreciate that. 24 DIRECT EXAMINATION 25 RITZ - DIRECT / FORGIE 10 1 BY MS. FORGIE 2 Q. But let me just very, very briefly go through a few of 3 your qualifications, just because I think they're important. 4 Can you state exactly what it is you do, please? 5 A. Yes. I'm an occupational and environmental 6 epidemiologist. I actually am tasked by the State of 7 California in the Center for Occupational and Environmental 8 Health at UCLA to investigate occupational and environmental 9 causes of disease. So it's really a discipline where we teach 10 our students, and we do research that involves assessment of 11 workplace hazards. 12 And my specialty has been pesticide-exposure assessment. 13 Over the last 25 years I've done very large studies in the 14 Central Valley, using the California registries; the pesticide 15 use report registries. 16 And I've worked on many different diseases, including 17 cancer. 18 I'm also currently the President -- the sitting President 19 of the International Society for Environmental Epidemiology. 20 That is a group of individuals who professionally assess 21 exposures. So that's how we define our tasks. 22 And I have published more than 260 peer-reviewed papers in 23 the area that I study; and pesticides are a big proportion of 24 it. 25 MS. FORGIE: Okay. Thank you, Doctor. RITZ - DIRECT / FORGIE 11 1 Could we see the next slide, please? 2 (Document displayed.) 3 BY MS. FORGIE 4 Q. Can you explain, please, Doctor, very basically, in 5 laymen's terms, what epidemiology is, and some of the basic 6 concepts of epidemiology? 7 A. Right. So I'm an M.D., Ph.D. So I really am interested 8 in human health. 9 So -- and the studies I do are really population-based 10 studies; meaning we are assessing in worker cohorts in the 11 population what hazards might be related to the outcomes; the 12 health outcomes. 13 So for occupational exposures, that is -- we have a lot of 14 good tools, because workers and their workplaces are usually 15 described well in terms of the agents they are using, the 16 substances, how long they've been using it, et cetera. And 17 that's one of the really important tools in epidemiology that 18 we have -- workplace exposures -- to say, Well, this chemical 19 in this kind of environment with exposure over so-and-so long, 20 we can now really link to an outcome; a health outcome. 21 So what we are doing in our studies is trying to identify 22 either hazards or protective factors that are related to human 23 health -- so my lab is really a human lab -- or better, the 24 population; the population of California, or worker population. 25 And we are trying to assess whether the risk of disease in RITZ - DIRECT / FORGIE 12 1 those who are exposed differs from the risk of disease in those 2 who are unexposed. 3 And we're generating these rates -- disease rates in the 4 exposed; and then divide them by the disease rates in the 5 unexposed. And if that ratio is one, we know that one in the 6 exposed isn't higher than the unexposed. Right? 7 And so what my students want to generate are these 8 relatives Risks/Odds Ratios. They all measure the same thing. 9 They all measure whether, in the exposed group, there is more 10 risk for the outcome than in an unexposed group that we presume 11 is as similar as possible to the exposed group, except for the 12 exposure. 13 And we use different tools to do that, but generally, we 14 have different study designs that we are dividing into cohort 15 and case-control studies; the only difference between the two 16 was, one, we are starting with the outcome, cases. And we are 17 selecting controls. And then the controls provide us with 18 exposure information, assuming that the exposure information 19 for the controls did not contribute to disease, because they 20 are controls. Right? They don't have the disease. So they 21 give us the rate of exposure among the controls that, if the 22 rate of exposure is higher in the cases, we then can say there 23 is a risk. 24 In the cohort study we go the other way. We start with 25 people who are unexposed, who are exposed, and then we follow RITZ - DIRECT / FORGIE 13 1 them over time to see what the disease rates in the two groups 2 are. And we he then calculate our ratio measures. 3 Q. Thank you. Can you explain what statistical significance 4 is, and how it is used in epidemiology, please? 5 A. Yes. So statistical significance is a very complicated 6 but easy tool in some ways. So, easy because statisticians 7 have made up rules that of .05 is the measure of statistical 8 significance. 9 However, what we are teaching our students is that with 10 p-values and statistical-significance testing, we're really 11 just assessing whether the results of our study are 12 influenced -- how they are influenced by random noise, by 13 random error. 14 What we also tell our students is that random error, yes, 15 is important. We should take it into consideration, but it is 16 really not the most important tool we have. Much more 17 important is that we are actually assessing systematic biases. 18 And systematic biases are confounding selection bias and, very 19 importantly, exposure assessment and outcome assessment that's 20 done right; that we get the information right. 21 So all of these other issues are more important than just 22 getting the p-value right, but we are using statistical 23 significance and p-values to gauge how much random error, how 24 much random fluctuation there is in our data that contributes 25 to what we are seeing. RITZ - DIRECT / FORGIE 14 1 Q. And with regard to your explanation about p-values not 2 being the be-all end-all, do you have support for that 3 proposition that you can reference for the Court, please? 4 A. Oh, yes. My teacher, who wrote the textbook on 5 epidemiology, has a whole chapter -- Chapter 10 -- that I still 6 use with my students when I teach, in which he explains why a 7 singular p-value is really not what we should be using. We 8 should be using the full data that's provided in a study to us. 9 We should use confidence intervals, p-value distributions, and 10 really assess these data to the fullest we can, and not just 11 rely on a p-value. 12 We actually have a T-shirt we wear in that class when we 13 teach it that says "No p-value allowed." 14 Q. Excellent. And have you read the deposition of 15 Monsanto's -- or one of Monsanto's epidemiologists, Dr. Rider, 16 in this case? 17 A. Yes, I have read Dr. Rider's statements, and they 18 completely agree with what I just said. 19 Q. And with regard to p-value? 20 A. With regard to the importance of p-values. 21 Q. Okay. 22 A. We use them as a tool, but they're just that. They're one 23 tool in our tool box, and not the sharpest. 24 Q. Okay. Perhaps, like me -- could we go on to potential and 25 confounding factors? Can you explain in laymen's terms, RITZ - DIRECT / FORGIE 15 1 please, what a potential confounding factor is, and what an 2 actual confounding factor is? 3 A. Right. So confounding refers to these biases. And it is 4 one of the important biases we are trying to avoid. It comes 5 from this concept of -- that we would want to know what would 6 have happened to those who are exposed if we could take the 7 exposure away from them. So, would they have had the same 8 disease risk if there hadn't been the exposure? 9 That's, of course, a counterfactual. We can never know 10 that for a group of people who has been exposed, or for on 11 individual. So what we do is we construct these comparison 12 groups. So the comparison groups are supposed to give us this 13 counterfactual rate; the rate when you have disease when you're 14 not exposed. 15 But how do we know whether these unexposed are actually 16 the right comparison group? 17 Well, we know it by trying to judge whether there's 18 potential confounding. And potential confounding comes in when 19 really there are other underlying risk factors for the disease 20 that the unexposed have, but not the exposed at the same 21 degree, so that really, the unexposed either are at higher risk 22 of the disease due to other factors, or lower risk. Right? We 23 want to make them as similar as we can. So we assess 24 confounding, and we deal with confounding. 25 The problem is there is potential confounding, where we RITZ - DIRECT / FORGIE 16 1 can argue whether a factor is really differentially distributed 2 among the exposed and unexposed, or whether it's a risk factor. 3 And there is actual confounding, which means in my study 4 this applies. And for every single study we have to really 5 very carefully -- and this is what I teach -- very carefully 6 assess whether the same factors should be considered 7 confounders or not, because you can actually muddy the pond by 8 calling something a confounder that is not. And putting it in 9 a model, you might generate confounding. You might generate 10 bias. 11 So e need to use every single bit of information we have 12 from prior knowledge to establish, first, is this a risk factor 13 for the outcome? If not, it can't be a confounder. 14 Second, is this factor also associated with/correlated 15 with influencing the exposure status? 16 Those are two criteria that need to be present for 17 confounding to happen. And the certain criterion is that it's 18 not -- that the factor is not in the pathway between exposure 19 and the outcome, so it can't be caused by the exposure, and 20 then causing the outcome. Those are the three principal 21 components or criteria that we need to check off before we say 22 it's a confounder. And it is different for every single study. 23 Q. Okay. And lastly, can you please explain what exposure 24 misclassification is, including what exposure is? 25 A. Right. So this is really what my discipline focuses on in RITZ - DIRECT / FORGIE 17 1 occupational and environmental epidemiology. And I tell my 2 students the one thing you need to learn is to assess exposures 3 right, because when we make a mistake in exposure assessment, 4 it is the same as if we have a very noisy room, and you're 5 trying to hear my voice. You will not hear my voice. You will 6 not see the -- hear the signal above the noise. 7 So if we have a lot of exposure misclassification, it's 8 basically information bias. We have a lot of noise. And there 9 is maybe a true signal somewhere. There is a voice in that 10 noise that I should be discerning, but I can't hear it. So the 11 worse I'm doing as an exposure assessor, the more likely it is 12 that I won't hear anything; that I won't find the signal. 13 We always think it's the opposite; that we hear voices 14 that tell us untruths; that there is a big signal that you know 15 is a false signal. 16 And what I really try to convey to my students is: Most 17 of the time, we're very poor instruments. Our exposure 18 assessment is no good. And if we see something -- I actually 19 get worried, because I know if I do a bad job in exposure 20 assessment, I won't see a thing. I will just -- just drown out 21 the signal in the noise. 22 Q. Okay. And finally, can you explain the difference between 23 retrospective and prospective, and how it applies to cohort 24 studies, and also to case-control studies, please? 25 A. Right. So -- so retrospective and prospective is really RITZ - DIRECT / FORGIE 18 1 how we generate our data. And when we talk about 2 retrospective, then we are -- we are saying that the exposure 3 information is in the past. And we are using tools to assess 4 what happened in the past. 5 The simplest example is we are asking people questions. 6 Right? And they have to remember, and they have to report. So 7 that's a retrospective assessment. 8 A prospective assessment is: I put a little monitor on my 9 radiation worker. And over time, I follow their exposures. 10 And I also follow them for the outcome at the same time. So I 11 follow them for health outcomes, and I measure on a daily basis 12 with a radiation measurer -- meter -- their radiation exposure 13 prospectively. 14 Of course, we have radiation meters. We don't have any 15 tools that can actually get us this type of information for 16 pesticides. 17 Q. Okay. Great. Thank you. Now let's get to the heart of 18 the matter, and talk about the relevant studies as they relate 19 to glyphosate-based formulations and non-Hodgkin's lymphoma. 20 Can I have that advanced? Okay. First of all, can 21 everybody see this? Is it -- my screen's a little small. 22 Okay. 23 Can you first explain what this is, Doctor, and how it -- 24 what it is? Just explain what it is. 25 A. Yeah. This is supposed to just be a simple visual aid. RITZ - DIRECT / FORGIE 19 1 We call it a "forest plot." 2 (Reporter requests clarification.) 3 THE WITNESS: We call it a "forest plot," like 4 forest; like the woods. 5 And so really what we have in the middle -- a blue line -- 6 is a stem. And that blue line reflects the null value or the 7 Rate or Odds Ratio of one, where the rate in the exposed is the 8 same as the rate in the unexposed. So the ratio is one. 9 Right? Same number of cases in both. 10 So if all of the red dots, which we refer to as "point 11 estimates" -- those red dots -- if they either line up on that 12 blue line, we would say the studies show that there's no 13 effect. 14 Or if these red dots fluctuate around that red line to the 15 right and the left, we would say there's so much random noise 16 and random variation between studies that, you know, on 17 average, they are null. 18 And we can kind of guess what a summary estimate across 19 all of these studies would mean. It would mean that the 20 summary estimate would be ripe at having a dot right on the 21 blue line. 22 So then we have the lines and the whiskers. And that is 23 what we refer to as the "confidence interval." In this case, 24 it is a 95 percent confidence interval. And that gives us the 25 spread of that red dot that might be due to random error. So RITZ - DIRECT / FORGIE 20 1 it's not systematic error. It's just random error evaluation. 2 And when these whiskers are above, to the right of the 3 blue line, then they're above the 95th -- then the 95th 4 percentile is -- is considered statistically significant. 5 And if they go to the other side of that line, then that 6 red dot and its confidence interval would not exclude the null. 7 So it might indicate that there is an effect, but I cannot 8 exclude random error. 9 So generally, what it helps us to do is see how all of 10 these red dots are actually lining up. And in this case, you 11 can actually see that most of these red dots are to the right 12 side. To the right side means an increased risk. To the left 13 side -- it means a decreased risk, or protection. 14 And the pattern is -- is a nice visual to kind of gauge 15 where most of the studies with their relative simple yes/no 16 exposure estimate here lined up. And they lined up to the 17 right of that blue line. 18 BY MS. FORGIE 19 Q. Now, Doctor, you teach at UCLA: University of California 20 in Los Angeles. Correct? 21 A. Mm-hm. 22 Q. And are there -- do you teach about forest plots and 23 confidence intervals? 24 And in general, the epidemiological principles that you 25 teach at UCLA -- are they generally accepted in the scientific RITZ - DIRECT / FORGIE 21 1 and epidemiological community? 2 A. Yes. That's exactly what I do. And we teach from the 3 book at UCLA that my colleague actually cowrote -- 4 Dr. Greenland -- that's considered the basic textbook in 5 epidemiology. 6 And what I cited before was the one chapter; just the 7 p-values. 8 Q. Those are the same things that you're explaining and 9 teaching to us here in the courtroom. Correct? 10 A. That's exactly it. 11 Q. Okay. And I notice that on the forest plot there is some 12 studies that appear to be on there twice. Do you see that? 13 A. Yes. 14 Q. Can you explain why that is, please? 15 A. So sometimes it is not as easy to make a decision, because 16 when you make a visual, you have to decide which study; and not 17 only which study to represent, but also studies present 18 multiple estimates, multiple dots. And you have to decide 19 which of these dots might have the best information; the most 20 information. 21 And sometimes it just helps us to outline that there is 22 more than one way to look at this data. 23 And, for example, I'm showing you the De Roos 2005, and 24 then also the Andreotti 2018 results, because these are major 25 studies. And I didn't feel -- even so they are on the same RITZ - DIRECT / FORGIE 22 1 individuals, I didn't feel it was justified to take one out, 2 and leave the other in. It's just a foundation for a 3 conversation about what these estimates mean. 4 Q. Okay. And can you explain how -- the differences or the 5 similarities between statistical significance, p-value, and 6 confidence intervals, and how they fit on your forest plot, 7 please? 8 A. Right. So all of these are completely mathematically 9 aligned with each other. 10 So these confidence intervals -- the lines with the 11 whiskers -- they represent the 95 percent confidence interval. 12 If those whiskers don't go across that blue line, it's 13 statistically significant. If they cross the blue line, it's 14 not statistically significant. 15 However, you should not just look at the widths or where 16 it lands, but also where the central estimate -- that red 17 dot -- is. What is the direction of the effect? Because any 18 study can be not having enough information, not having enough 19 statistical power to exclude random error; but they can still 20 tell you something about the direction of the effect. 21 And that's why this is plotted in this way; because it 22 shows you what the -- the general direction of the affects are 23 across all studies. 24 Q. And let's look at a couple of these studies in more 25 detail. RITZ - DIRECT / FORGIE 23 1 THE COURT: Before you move from the forest plot, 2 Dr. Ritz, if I could just ask you: Of these studies on the 3 forest plot, how many of them are adjusted for exposure to 4 other pesticides, and which ones are they? 5 THE WITNESS: It's a very good question. 6 So the problem of exposure -- of adjusting for other 7 pesticides -- actually, it is the De Roos 2003 which has the 8 "650" next to it. That is the most highly adjusted estimate. 9 It actually adjusts, I think, for 40-some other pesticides. 10 And you can see that while the confidence interval could 11 be considered wide, it still excludes the null. It's still 12 statistically significant. And the red dot is above two. So 13 it's a more than twofold risk increase. That's statistically 14 significant, and completely adjusted for every other pesticide. 15 I -- I know Anneclaire, and I would have probably argued 16 with her about this approach, not thinking that she should have 17 put 40 pesticides in there, because that's not how I teach my 18 students. 19 I teach my students you have to really consider whether 20 any one of these pesticides is a risk factor for the outcome. 21 If it's not, you shouldn't put it in the model. 22 If it is, then we can discuss whether that pesticide also 23 is related to the one under investigation -- glyphosate, in 24 this case -- meaning: Does -- do the two pesticides really 25 correlate with each other, or does one imply the other is also RITZ - DIRECT / FORGIE 24 1 used? 2 And if that's not the case, then, again, we should not be 3 throwing this into the model. 4 However, she went all out, and threw them all in the 5 model. And the effect estimate is still statistically 6 significant. 7 THE COURT: Could I ask a clarification question 8 about that? 9 I think you said that you shouldn't include the pesticide 10 in the model. You shouldn't adjust for the pesticide unless 11 you know it's a risk factor -- 12 THE WITNESS: Mm-hm. 13 THE COURT: -- for NHL. Is that right? 14 THE WITNESS: That's correct. That is the first rule 15 of assessing confounding. 16 THE COURT: So -- but what if you don't know whether 17 the pesticide is a risk factor for NHL? 18 THE WITNESS: Right. 19 THE COURT: You should not adjust for that in an 20 epidemiological study? 21 THE WITNESS: If you don't know, you have to think 22 about what you do when you put this -- 23 Then you would say, I want to try out what happens when I 24 put this in my model. 25 However, you then have -- when -- when that model then RITZ - DIRECT / FORGIE 25 1 shows a different effect estimate, we have to discuss what that 2 means. 3 So, for example, smoking causes lung cancer, I don't think 4 we want to argue about that. Right? But somebody has a 5 brilliant idea, where they say, Okay. I really think it's not 6 smoking. It's breath mints that cause lung cancer. 7 So let's ask everybody also whether they are chewing 8 breath mints. And let's not say that 90 percent of all smokers 9 take breath mints because they, you know, want to get rid of 10 that odor. 11 And when you then say, Okay. Isn't taking a -- shouldn't 12 we put breath mints -- we don't know whether it causes lung 13 cancer. Shouldn't we put it in the same model? 14 What will happen is that the effect for smoking on lung 15 cancer will happen, by definition, because you are now putting 16 two highly colinear variables into the same model. 17 Whether that means, now, that breath mints cause lung 18 cancer or not, or breath mints are just an indicator -- a proxy 19 for smoking -- you have to decide. 20 So we have to decide for every single pesticide whether it 21 is truly a risk factor for the outcome and we should consider 22 it as a potential confounder, or it's an indicator for having 23 used the pesticide under consideration. 24 THE COURT: So -- 25 THE WITNESS: It's just coming along. It's a rider. RITZ - DIRECT / FORGIE 26 1 THE COURT: So your opinion is that if we don't know 2 a pesticide is a risk factor for NHL, we should not adjust for 3 it in a study? 4 THE WITNESS: That's not -- sorry if it came across 5 wrong. No. I'm not saying we should not adjust for it, but 6 when we adjust for it, we should really be careful about how we 7 interpret what's happening to the effect estimates. Most 8 likely is that the confidence intervals widen when you do this, 9 and that the effect estimates -- if that pesticide is highly 10 correlated with the one under investigation, it is you who has 11 to decide whether it means as a confounder it's a true risk 12 factor and I should adjust for it, or it's a proxy, like the 13 breath mint. Right? 14 And nobody will take that away from us. We just have to 15 do that. 16 THE COURT: And so you mentioned De Roos 2003. Any 17 of these other studies adjust for other pesticide use? 18 THE WITNESS: The Eriksson Study adjusted for other 19 pesticides, but the estimate I'm showing here -- I don't think 20 it's the adjusted one. No. 21 So that's actually where it happened, as I recall, where, 22 when you put one or two other pesticides that are highly 23 correlated with the actual glyphosate pesticide, then that red 24 dot goes to the middle. It's 1.5, instead of 2. 25 THE COURT: And so why -- why did you include the RITZ - DIRECT / FORGIE 27 1 portrayal of the Eriksson Study on this forest plot that did 2 not adjust for pesticides, when adjustments had been made in 3 that study for pesticides? 4 THE WITNESS: Right. Because the pesticide they 5 adjusted for, MCPA, was one that was used prior to glyphosate. 6 And glyphosate was the one that followed up; that was replacing 7 MCPA. 8 So we have exactly a situation where you need to decide 9 whether you believe that MCPA is a true NHL causative factor, 10 or it's just that breath mint that tells you, Well, they used 11 MCPA. Then they, you know, had a new agent that they thought 12 was working better: Glyphosate. So they replaced MCPA by 13 glyphosate. So -- but MCPA is just an indicator for later 14 glyphosate use. So you have to make at that decision. 15 And I -- I didn't see a literature that told me that MCPA 16 was truly an NHL risk factor. 17 THE COURT: Okay. And any other studies on here 18 adjust for other pesticides? 19 THE WITNESS: Yeah. De Roos, and Andreotti. De Roos 20 2005. And Andreotti. Mm-hm. 21 THE COURT: Both of those are from the AH -- 22 THE WITNESS: AHS. Yes. 23 THE COURT: And what about McDuffie 2001? 24 THE WITNESS: McDuffie? I don't think the one I'm 25 showing is adjusted. RITZ - DIRECT / FORGIE 28 1 THE COURT: So the McDuffie did adjust for pesticide 2 use, but your portrayal of the study does not include an 3 adjustment for pesticide use. Is that right? 4 THE WITNESS: Other types of pesticide. Yes. Yes. 5 THE COURT: Okay. 6 BY MS. FORGIE 7 Q. And what about the NAPP Study? Does that adjust for other 8 pesticides, as well? 9 A. As far as I know, yes? 10 Oh, wait. The one I portrayed didn't; but they later 11 showed also an adjusted one, yes. 12 Q. And that shows an elevated Odds Ratio? 13 A. Mm-hm. 14 Q. After you adjust for pesticides in the NAPP Study. 15 Correct? 16 A. Yeah. It depends. Actually, there are different ways to 17 do this. Some authors like to just throw every pesticide in 18 the same model; and some authors just use one after the other. 19 And you then have to decide which estimate is the one that you 20 want to present or you want to talk about; but they're 21 trying -- what -- 22 It's called "Sensitivity Analysis." They are trying out: 23 Well, what happens if? 24 So they are -- they are sometimes using one pesticide 25 because they think it's maybe the more convincing one that RITZ - DIRECT / FORGIE 29 1 could have an effect on NHL, and then another one because it's 2 a common use or it's a very toxic one, and they want to see how 3 these effect estimates change. 4 That's actually how we are testing our data: How much 5 changes when I do one thing or another? 6 MS. FORGIE: Okay. So I should move this into 7 evidence. It's Exhibit 297 in the in our Exhibit List. If -- 8 MR. KALAS: No objection. 9 THE COURT: Okay. Admitted. 10 (Trial Exhibit 297 received in evidence.) 11 THE COURT: By the way, have the parties stipulated 12 to the admission of all of these exhibits? 13 MR. LASKER: As they go in, I think so. There may 14 be -- if there's something we haven't seen I don't know about, 15 but I expect we will be fine. 16 THE COURT: So then if it's something that you've 17 worked through together already, you can just do the mechanics 18 of getting them admitted afterwards, so we don't have to spend 19 the time doing it during your presentation. 20 MS. FORGIE: Okay. I will do that, Your Honor. 21 Q. So going back to the De Roos Study for a second, you 22 mentioned that it was the most highly adjusted, and that it was 23 an elevated Odds Ratio, and statistically significant after the 24 adjustments. Is that correct? 25 A. Yes. RITZ - DIRECT / FORGIE 30 1 Q. And how many -- approximately how many pesticides did the 2 De Roos adjust for in the -- as you've indicated on the forest 3 plot? 4 A. I understand it's some 40-plus pesticides. 5 Q. Okay. And for each of these studies, did you look at and 6 consider the methodology that was employed by the authors of 7 the study? 8 A. Yes. Definitely. 9 Q. And how did you go about doing that? 10 A. Well, you -- you want to know how the study design -- 11 first of all, you ask what the study design is; but then you 12 also want to know how the actual study was conducted. And so 13 what you want to know is whether the disease assessment was 14 properly done. 15 And just about all of these studies had very high, very 16 good disease assessment with cancer registries and pathology, 17 so I didn't see any problem in that regard. 18 And then, of course, the second most important thing is 19 how the exposure was assessed. And generally, all of these 20 studies are using interviews. They -- they -- 21 None of them were able to assess biomarkers or any kind of 22 records, except that McDuffie actually went back and asked 23 farmers whether they could go to the suppliers and ask the 24 suppliers about the purchasing records for pesticides, and then 25 compare the purchases for the specific farmer and what the RITZ - DIRECT / FORGIE 31 1 farmer had told them to the purchasing records. And they found 2 that they agreed very strongly. 3 And where they had disagreements, they were able to 4 actually rectify the disagreements, because the farmer that 5 year had bought the pesticides somewhere else, or not applied 6 them. 7 So -- so that is a very nice tool to use, going back to 8 records; but not everybody can do that. 9 Then the -- if it comes to interviewing, you really have 10 to consider how the interviewing was done. Was it done in 11 person, face to face? Did people go into the homes of these 12 people; give them a chance to look at their purchasing records; 13 spend time with their -- with their partners to discuss what 14 they had done in what years? 15 Or did you just give them a questionnaire that you gave 16 them half an hour to fill out? 17 And that makes a big difference. 18 There are case-control studies with maybe 400 cases and 19 the same number of controls where home visits are done; and 20 these individuals report on their use of pesticides for hours 21 in a very interactive way with the interviewer. And I would 22 consider that a much better exposure assessment than giving 23 somebody a sheet of paper with 20 pages and saying, Well, in 24 the next 20 minutes please report. Right? 25 So -- so that's the way I would assess the exposure RITZ - DIRECT / FORGIE 32 1 assessment in these different studies. 2 And then the next thing is: How much did they then also 3 ask about lifestyle factors; other pesticides; other possible 4 risk factors for the outcome, such as family history of cancer? 5 And then how did they bring all of that together in their 6 analyses? And are the analyses done adequately with the data 7 at hand, and then also interpreted in -- in a sufficient way? 8 Q. Okay. Let's go to the 2002 Hardell Study, please. 9 And you weren't asked a lot of questions in your 10 deposition about specific methodology, but Monsanto has 11 attacked the methodology of some of these studies. 12 A. Right. 13 Q. So I will walk through some of the methodology in the 14 individual studies. Okay? 15 A. Yes. 16 Q. How was the unexposed group defined in the 2002 17 Hardell Study? 18 A. Right. So the Hardell Study's one of the typical 19 case-control studies. It was a Swedish study. And Sweden, 20 just like all of the Scandinavian countries, has exceptional 21 records systems for health records, as well as for retirement 22 and occupational records. I actually have a large study that's 23 NIH funded in Denmark because of that reason. We are all 24 jealous of the Scandinavians. 25 So they are using these beautiful records systems to RITZ - DIRECT / FORGIE 33 1 identify cases. 2 And they also have population registers from which they 3 can pull the names and addresses of unaffected people. That's 4 called a "population-base selection of controls." Very well 5 done. 6 And then they go and interview these people about their 7 exposures. 8 When it comes to the analysis -- 9 So I didn't have any problems with any of that. 10 When it comes to the analysis, you are now asking 11 yourself, Well, what happens? Because we know farmers don't 12 just, you know, farm one crop. Most farmers don't just farm 13 one crop; but even if they do farm one crop, they may be using 14 different herbicides, because whatever they're trying to -- the 15 pest they are trying to get rid of -- the weed -- may get 16 resistant, or they might have recommend -- received 17 recommendations to alternate different pesticides over the 18 years to not make the weeds resistant. So generally when you 19 ask a farmer, they use more than one agent. 20 It's not like in industry, like my radiation workers, you 21 know, who are -- the one exposure they had -- they had was 22 radiation. 23 Here we have a mixture of exposures. 24 So when we want to compare what any singular chemical does 25 to an outcome, we have to consider the co-exposures; but we RITZ - DIRECT / FORGIE 34 1 also have to consider what is the true or the correct 2 comparison group to use? 3 And what Hardell chose, and later Eriksson, as well, was 4 to say, Well, what happens if we are comparing the glyphosate- 5 or phenoxyherbicide-exposed individuals to controls that did 6 not use pesticides? 7 So these are community studies. This is not like the AHS, 8 where everybody was a farmer who actually applied pesticides. 9 This was done in communities. Some people were farmers; others 10 weren't. 11 And the critique often is that, well, you're comparing 12 farmers with farmers there. If your farmer doesn't use 13 glyphosate, he probably used DDT. And, you know, that's also a 14 carcinogen. So you're comparing people who are -- who are 15 exposed to one carcinogen, to people who are exposed to the 16 other carcinogen. If you don't see anything, that's because 17 they are all exposed to some carcinogen. Right? 18 So what they tried to do here is exclude anybody who had 19 any pesticide exposure from the comparison group, and say, Now 20 we have a clean group. We have a group of individuals who have 21 glyphosate exposure, with or without -- without other types of 22 exposure, but we are comparing all of them to people who never 23 used pesticides. 24 Sounds like a good idea. 25 It's not such a good idea. And I tried to explain that to RITZ - DIRECT / FORGIE 35 1 my students, because what we do when we get a little more 2 technical -- we call it a "collider bias." And I explain that 3 to my students -- what it is, but then I also say, Well, yeah, 4 collider bias is a kind of selection bias. And you should not 5 generate the selection bias by excluding in this way people who 6 have other types of pesticide exposure from the control group. 7 But then I keep going, and say, Well, this is a potential bias. 8 Right? And they generated this potential bias. 9 But the next step is to assess: What is the quantity of 10 that bias? Because we are quantitative science. We're not a 11 qualitative science. 12 We want to know: How big is that bias? 13 And now let's estimate how big that bias would be. And 14 it's pretty simple to do in this case. All you have to do is 15 add the controls back into the comparison group who they 16 excluded. And they actually you gave you a few tables that 17 showed that data. And I was able to put that back in; redo the 18 calculations; and convince myself that the change in the 19 estimate that they are representing is within about 10 percent 20 of the estimate. 21 So the quantity of that bias is minimal, in epidemiologic 22 terms. Yes, there is bias, but it is minimal. 23 MS. FORGIE: And I should have mentioned that the 24 Hardell 2002 is Exhibit 20. 25 And Hardell 1999 is Exhibit 19. RITZ - DIRECT / FORGIE 36 1 And Eriksson 2008 is Exhibit 17. 2 And I will mention the exhibits beforehand. My apologies 3 to the Clerk and the Court. 4 Q. Okay. So then let's talk about Eriksson 2008 for a 5 minute, which is Exhibit 17. Did Eriksson 2008 analyze latency 6 periods, at all? And can you explain a little bit about what 7 latency is, and a little bit about the difference between 8 latency and exposure, please? 9 A. Yes. So we are not talking about exposures that cause 10 immediate disease or death. Right? We're talking about 11 lifelong exposures, occupational exposures that eventually, 12 when you reach a certain age, might have contributed to your 13 cancer. 14 So what we are wondering is: How long does it take to 15 initiate that cancer? 16 And generally, in cancer epidemiology we think it might 17 take at least a year in any single individual to have that 18 process develop. 19 For blood cancers -- and lymphoma is a blood-related 20 cancer -- we usually think that it could be faster than for 21 solid tumors. 22 For solid tumors, we at least want five or ten years. 23 For blood cancers, one year, two years could be a minimum 24 latency we want to see. 25 And most studies -- actually, this included -- therefore RITZ - DIRECT / FORGIE 37 1 exclude any exposures that's within one year of diagnosis. And 2 that make a lot of sense. 3 So but the next question we ask is: Is there like an 4 optimal time period that I should look at for the exposures to 5 have caused these outcomes? 6 And is that within the last ten years prior to the 7 outcome? Are those exposures more important, or are the 8 exposures more important ten to twenty years prior to the 9 outcome, or even longer? 10 And so what Eriksson tried to do is find a way to say, 11 Well, what is the optimal time period of exposure? Like, was 12 it in the forties for these farmers that are now in their 13 sixties? Was it in their fifties? 14 And that's how they then try to split up their data in 15 these different chunks -- ten years prior to diagnosis; ten to 16 twenty years; twenty to thirty years -- and estimated these 17 relative risks; these -- these risk-increase measures for the 18 outcome, according to exposure that happened in those periods, 19 with the goal to see whether -- 20 And that's another rule. If you say, Well, every exposure 21 is the same, whether it's two years prior to your cancer or 22 thirty years prior to your cancer, you're probably making a 23 mistake. Right? 24 It may be that you needed ten years of exposure, and then 25 the first cancer cell developed, but it was killed by your RITZ - DIRECT / FORGIE 38 1 immune system. You had another five years of exposure. The 2 next cancer cell developed. Your immune system was weakened. 3 That cancer cell kept going. Right? 4 So it's a process, but some of the exposures in the 20- to 5 30-year and maybe the 10- to zero-year might be not as 6 important as the one in the middle, where most of that 7 carcinogenicity really is happening, because once that cell is 8 initiated, you might have continued exposure, because you're 9 not diagnosed for another 10 years; but all of the exposure 10 that happen from the cell being initiated to you being 11 diagnosed are irrelevant, because they haven't -- the cell's 12 already there, so those exposures are not contributing anymore. 13 So when I add irrelevant exposure to relevant exposure, 14 the same thing happens: I'm muddying the pond. 15 And I'm estimating relative risks that are smaller. So in 16 order to actually estimate -- come closest to the truth, I want 17 to find the optimal period of exposure. 18 And that's what Eriksson tried to do here. And they found 19 that the optimal period of exposure prior to the outcome was 20 probably somewhere, in their data, between 10 and 20 years 21 prior to diagnosis. 22 However, it is interesting, when you read this closely, 23 the last 10 years had almost no exposure. So I -- they didn't 24 say that, but I interpreted this as is these farmers had 25 stopped farming, or stopped using pest -- herbicides. They had RITZ - DIRECT / FORGIE 39 1 absolutely no phenoxyherbicide exposure anymore. Therefore, 2 they couldn't even estimate exposure in that period. And they 3 had very little glyphosate. Therefore, the estimate for 4 glyphosate had a huge confidence interval. 5 And the huge confidence interval gives it away. There 6 must have been very little exposure to estimate anything. 7 Q. Okay. And with regard to Eriksson 2008, which is 8 Exhibit 17, Monsanto has been arguing that the 1.1 Odds Ratio, 9 which is almost a null, is the most reliable Odds Ratio for 10 glyphosate-based formulations, because it was observed during 11 the one- to ten-year latency period that you just discussed, 12 when there were no cases for exposure to other pesticides in 13 that period, and thus no confounding. 14 Do you agree with that position? 15 And if you don't, can you explain? 16 A. Absolutely not. 17 I really think they had no statistical power to estimate 18 anything in that period. And that's what that wide confidence 19 interval around the point estimate for glyphosate reflects. I 20 really think that these farmers had stopped farming, and 21 therefore there was no more exposure. So how do I estimate 22 anything in a period when nobody's using or most people are not 23 using herbicides anymore? That data just isn't there. I can't 24 estimate it. I need to go to a different study that actually 25 has that kind of data. RITZ - DIRECT / FORGIE 40 1 Q. So the data is not actually there to make that 2 determination? 3 A. No. I think it -- the estimate tells you there is no data 4 there to estimate anything for that period. 5 Q. Okay. Again, in Exhibit 17, which is the 2008 6 Eriksson Study, is there a meaningful difference between the 7 Odds Ratio for greater than 10 days of use, versus the Odds 8 Ratio for less than 10 days of use? 9 A. Yes. Actually, I really like what they did here in terms 10 of trying to get to some kind of dose-response. 11 So the days of use that -- no use at all, one to ten days 12 of use, more than ten days of use are getting at: Is there 13 actually -- 14 And what I'm seeing here (indicating) is not reflecting 15 what I'm talking about. 16 Yes. There we see it. 17 So we are seeing that, compared to those who never -- in 18 Eriksson, never used glyphosate, there are 12 cases who report 19 1 to 10 days per year use. 20 So it's not one to ten days. It's one to ten days per 21 year on average, because they were asked to report their 22 average use. 23 And then you can see that that estimate is somewhere 24 between one and two. The red dot -- the point -- is probably 25 around 1.7, from what I see. RITZ - DIRECT / FORGIE 41 1 And you can see that the lines and the whiskers are wide. 2 And the left one actually includes the one; meaning that 1.7 is 3 not -- I cannot guarantee that there isn't random error that 4 produced this estimate. Right? 5 The confidence interval includes the one. 6 However, the next estimate, more than ten days per year, 7 is greater than two. It's probably around 2.3, maybe. 8 And you can see those confidence intervals are not 9 crossing the one. Right? 10 So that's a statistically significant 2.3 -- I can't 11 remember -- two-point-something estimate for more than 10 days 12 per year use. 13 What I really like about it is you're not supposed to just 14 say, Is this the statistically significant or not? 15 You're actually supposed to now say, Well, when I go from 16 less than ten days per year to more than ten days per year, do 17 I see some kind of pattern? 18 And that's also how I teach my students. Do we see 19 patterns in the estimates? 20 I see a pattern. I see a dose-response pattern here. 21 Even so, statistically for the one estimate, I cannot 22 exclude random error. 23 Q. Okay. So this slide here is part of Exhibit 297, as are 24 all of the slides; but I'll reference them all as 297 from now 25 on, which I hope will help the Court and the Clerk. RITZ - DIRECT / FORGIE 42 1 So with regard to this slide, can you explain what 2 you're -- what this is, and what it's trying to show? 3 And also explain in more detail, please, what 4 dose-response is, and how it affects your opinions with regard 5 to causation. 6 A. Right. So these are authors who actually tried to get at 7 a dose-response. And McDuffie is very nice, because they 8 distinguish between less than one -- less than two days, or two 9 and one day per year, and more than two days per year use of 10 glyphosate. 11 Pretty much what they are saying is it's not just a yes/no 12 question whether glyphosate causes NHL, but it's a question of 13 whether you're an occasional user -- 14 You spray a day -- right? -- here and there. 15 -- or whether you are potentially a routine user, more 16 than two days per year. Every year you might be spraying 17 three, four, five days. 18 And you can clearly see here that distinguishing the 19 routine users -- the users who have a common use pattern -- for 20 them, you see that that red dot is really above two. And you 21 also see that the 95 percent confidence interval excludes the 22 null value. It's statistically significant, as I would expect 23 if there's a dose-response, or that high exposure should be 24 causing the disease at a higher rate. 25 And I don't see anything for the occasional users. So it RITZ - DIRECT / FORGIE 43 1 tells me occasional use might be okay, but don't use this 2 routinely. 3 And the estimate is more than twofold, so your risk 4 increases more than twofold if you're a routine user. 5 Q. And what does that tell you with regard to your opinions 6 about causation? 7 A. Well, this was definitely part of how my opinion was 8 formed: Going for patterns, going for dose-response in studies 9 that I considered very strong. And McDuffie and Eriksson 10 belong to those kind of studies, and so does the NAPP Study, 11 which is actually a compilation of the North American and the 12 Canadian study. McDuffie is a Canadian study. 13 THE COURT: Sorry to interrupt. Before you get to 14 the NAPP Study, could I ask one more question about the 15 Eriksson Study? 16 THE WITNESS: Please. 17 THE COURT: There was a slide that flashed across the 18 screen before this one. 19 THE WITNESS: Yes. 20 THE COURT: That was not the slide you were looking 21 for, but it mentioned multivariate analysis and univariate 22 analysis. 23 THE WITNESS: Yes. 24 THE COURT: Can you explain to me the difference 25 between those two things -- RITZ - DIRECT / FORGIE 44 1 THE WITNESS: Yes. 2 THE COURT: -- and how that relates to Eriksson; what 3 is portrayed -- 4 THE WITNESS: Right. 5 THE COURT: -- in this slide with respect to 6 Eriksson? 7 THE WITNESS: Thanks for asking. It actually makes 8 my point. And the point is when -- so the left -- the 9 univariate column shows you the relative Odds Ratios, which are 10 representing Risk Ratios here for the pesticides that were 11 found to be linked to NHL in this study. They investigated a 12 lot more, but these are the ones where the relative risks 13 actually indicated an increase. 14 And you can see that these authors are actually very 15 thorough. They are not just showing you -- 16 THE COURT: Can just to clarify, this is Eriksson? 17 THE WITNESS: Yes. 18 THE COURT: Okay. 19 THE WITNESS: This is Eriksson. 20 So they're not just showing you -- 21 The Swedish study. 22 They're not just showing you the ones where, you know, 23 it's statistically significant; but they're showing you every 24 estimate that is above one, because they want to say, Well, if 25 we want to be health protective, we should consider any RITZ - DIRECT / FORGIE 45 1 estimate that's probably above 1.5. And they actually state 2 that in their paper, because a 50 percent increase in NHL risk 3 is something we should be worried about. 4 And statistical significance is not all, because a larger 5 study may show that it becomes statistically significant. 6 So they are showing you these estimates for from 2.8 to 7 1.61. And you see for glyphosate, it's 2.02. And here, it's 8 statistically significant in a univariate analysis, because 9 that -- the confidence interval is 1.10 -- the lower one, it's 10 above 1, so that says it's statistically significant. It's a 11 twofold risk increase. 12 But here we only put glyphosate in the model. That's 13 univariate. One variable. Or one risk factor at a time. 14 And then the multivariate -- they threw them all together 15 in the model. Okay? 16 All of the ones that you see here -- tar, creosote, 17 arsenic, mercurial seed dressing, glyphosate, 2,4,5-T, and 18 2,4-D, MCPA. So the MCPA and 2,4,5-T, 2,4-D are phenoxy 19 herbicides. They are the old herbicides we used in the '60s. 20 Actually, 2,4,5-T is famous for having dioxin contamination in 21 Vietnam War. Right? 22 So glyphosate is the next one. And when you put all of 23 these together in the model, you can see on the right side 24 under multivariate what's happening. 25 All of these Odds Ratios shrink. Right? They go towards RITZ - DIRECT / FORGIE 46 1 one. They're smaller. 2 And all of the confidence intervals widen, and now pass 3 the null value. 4 This is exactly what I said when I talk about confounding. 5 If you put multiple variables into the model that are 6 correlated, and they're actually explaining that MCPA -- the 7 phenoxy -- was taken off use in Sweden, and replaced by 8 glyphosate. This is what you expect to see. 9 It doesn't mean that the true estimate is 1.5 instead of 2 10 for glyphosate. It just means my statistical model is behaving 11 in this way because I put highly correlated factors in the same 12 model. So I'm measuring the same thing five times in the same 13 person. 14 I just don't have enough information to do this correctly, 15 because what you would want is a large group of people, only 16 MCPA-exposed, only glyphosate-exposed, only tar-exposed. 17 You don't get that. Right? 18 And that's why this pattern occurs. And it also occurs in 19 every single one. 20 And you can see that arsenic now looks like it has no 21 effect; and it's a known carcinogen. It was 1.6 before. Now 22 it's 1.17. Right? But again, it's because arsenic is actually 23 one of the very -- it's one these inert ingredients that's in a 24 lot of the formulations. So it's probably the highest 25 correlated of all of what they're doing here. RITZ - DIRECT / FORGIE 47 1 And that's what I expect if it's the highest correlated, 2 because it's in many different formulations. You know, I can't 3 estimate its singular effect anymore -- I can't -- when I do a 4 multivariate. 5 And that's the drawback of these kind of analyses. It's 6 just what we live with. And we need to know how to interpret 7 this. 8 Q. Okay. With regard to the McDuffie 2001 case, which is 9 Exhibit 21, Monsanto has argued that the data -- that there's 10 issues with the data or that it's somehow flawed because 11 someone with 20 days' cumulative exposure may be placed in the 12 low-exposed category, while someone with three days' cumulative 13 exposure may be placed in the high-exposed category. 14 A. Right. 15 Q. Can you comment on that, please? 16 A. Right. With every dose scale I invent, I make mistakes. 17 I can make mistakes. This could be one mistake. I could be 18 placing some people who are cumulatively much more high exposed 19 into the group that's the low exposed, and the opposite. 20 But you see what happens when I do that? 21 I misclassify information. 22 And again, we are back to the noise-to-signal ratio. In 23 the end, if this is nondifferential -- meaning I'm doing it for 24 cases in the same way I'm doing it for controls; not 25 differential by case status -- we call it "nondifferential RITZ - DIRECT / FORGIE 48 1 exposure misclassification." What we get in the end is we 2 don't see anything. 3 So if this really happened, then I would be worried that 4 the effect estimate I'm having on my screen here is not high 5 enough, because I introduced a shrinkage towards the one. 6 Q. Okay. And did you consider, again -- or did you consider 7 that the lowest-exposure group in the De Roos 2005 AHS Study 8 includes individuals who could be categorized in the highest 9 exposure groups in both the 2001 McDuffie Study, which is 10 Exhibit 21, and the 2008 Eriksson Study, which is Exhibit 17? 11 A. Again, we use -- it's correct, if you want to say it this 12 way. However, we -- we have different scales here that we are 13 comparing. 14 We are comparing an average -- an average of 10 days per 15 year -- to a cumulative of 20 days in my life. 16 An average of more than 10 days per year could mean 50 17 days, a hundred days. We don't know -- right? -- because they 18 just grouped it into one group. So to make these general 19 statements of, Oh, the highest and the lowest across studies, 20 is not really justified. You have to really go into the data 21 and compare the exposures. 22 Also, I told you before that I'm concerned about the 23 relevant exposure period. Right? 24 If somebody had all of his exposures two years before 25 being diagnosed, it could be 200 days, because he decided to RITZ - DIRECT / FORGIE 49 1 want to be a pesticide applicator and/or to introduce GMO 2 crops -- right? -- and spray heavily every single day. Those 3 200 days could be completely irrelevant to the NHL because they 4 were two years before; while somebody else had 30 years of 5 regular applications three times a year, and he may not reach 6 the 200 days. Right? 7 So these are -- these are issues we really need to 8 consider. What is the right time period? What is the right 9 number of days? Is it cumulative? Do I -- you know, is it 10 really that I'm -- I -- every year I'm doing this; or is it 11 that I have five very intense days of spraying, I get myself 12 damaged, and from then on I can spray whatever I want? It 13 doesn't matter anymore. That one cell is starting to be a 14 cancer. Right? 15 Q. Okay. Is there anything else with regard to either of 16 these two forest plots in Exhibit 297 that you want to explain 17 further? 18 A. The only thing is that the NAPP Study really is combining 19 the North American studies that De Roos 2003 published on, and 20 some of the McDuffie data. 21 And that it is a powerful pooled study; meaning they -- 22 they got all of the data together to do these analyses. And 23 why it's powerful, is that they were not only able to assess 24 NHL as one big category. They were able to now also look at 25 certain subtypes of lymphoma. RITZ - DIRECT / FORGIE 50 1 And you see those listed here as follicular lymphoma, 2 large B cell lymphoma, and other lymphomas. And they're 3 showing that basically all of these have an increased risk due 4 to glyphosate. It's not just one subtype of lymphoma. 5 And they also show that actually the overall estimated 6 effect here for the routine users -- right? -- not the 7 occasional users. The routine, more-than-two-days-per-year 8 users -- is way above 2, and statistically significant. 9 Q. Okay. With regard to the NAPP Study, is that -- has that 10 been presented as a poster or an abstract? 11 Or in other words, do we have the complete study? 12 And can you explain how the NAPP has been peer-reviewed, 13 please? 14 A. Right. So the NAPP Study has an abstract that was 15 submitted to the ISEE conference. 16 I'm the current sitting President of that -- of that 17 society. And I was many years on the committee -- Scientific 18 Advisory Committee for reviewing abstracts for conferences. 19 And I know that we make a big effort to review every single 20 abstract for its scientific validity. And we get a lot more 21 abstracts than we can accept every year. 22 So, yes, they are reviewed. However, they are abstracts. 23 They're not a full paper. 24 I also reviewed for this the slides that were shown at the 25 conference. And so I had a personally the opportunity to RITZ - DIRECT / FORGIE 51 1 actually see the analysis and the data that they presented, and 2 that's partially what we are showing here. 3 MS. FORGIE: Okay. And I should have mentioned that 4 NAPP is Exhibit 301. Okay. 5 Q. Can we go back to the first -- to the original forest 6 plot, please? 7 Okay. And you mentioned earlier in your testimony, 8 Dr. Ritz, that most -- and you can see this -- that most of the 9 red dots -- the Odds Ratios -- are to the right of the study. 10 And then there's -- to the right of the blue line, the one. 11 And there are a few on the left; notably, the agricultural 12 health study. Is that correct? 13 A. That's correct. 14 Q. Okay. And what is the method when you have something like 15 this, where most of your forest plot is to the right of one, 16 and just a few to the left? Is there a method -- is there an 17 epidemiological methodology that you use and that you teach 18 your students how to -- as to how to analyze this situation? 19 A. Right. Yes, there is. 20 So it actually points to a qualitative difference. Right? 21 So we are interested in quantitative differences and 22 qualitative differences. And when we really see that one study 23 seems to estimate different effects from all others, we have to 24 ask ourselves: What is the qualitative difference between 25 these studies? RITZ - DIRECT / FORGIE 52 1 And here it's pretty simple. All of the other studies are 2 case-control studies. The Ag Health Study's -- Ag Health Study 3 is a cohort. So generally, we would hope that a cohort study 4 is of very high quality. And it -- it's collecting data 5 prospectively. It might be avoiding biases that otherwise we 6 might be concerned about. 7 However, the way I also teach my students is we cannot -- 8 we cannot just judge a study by the design. 9 A case-control study well done has as much if not more 10 information, sometimes, as a cohort study. 11 We are conducting case-control studies for a very good 12 reason. We are conducting them because diseases are rare. And 13 for -- and cancers are rare. We don't usually think about them 14 that way, but they are. So to assemble a large number of cases 15 takes us a lot of time. 16 A cohort study needs to be 50- to a hundred thousand 17 people, if you want to assess cancers. Any cohort study that's 18 smaller will have to follow individuals for 50 years to get any 19 kind of number that makes sense to look at for cancers; 20 especially for rare cancers. So cohorts are a challenge for 21 cancer. And everybody knows that. 22 The early cohorts were cardiovascular-disease cohorts, 23 because almost every second person develops cardiovascular 24 disease, and that's easy to study in a cohort. Cancer cohorts 25 are extremely difficult, because they have to be so large. RITZ - DIRECT / FORGIE 53 1 Since they have to be so large, they are costly. They are very 2 expensive. And they're not only expensive in terms of money. 3 They are also expensive in the time commitment you need from 4 the individuals, because you have to not only enroll them. You 5 have to be able to follow them over time. Right? You have to 6 be able to re-contact them. You have to be able to find out 7 what happened to them after enrollment; not just at enrollment. 8 What kind of exposures did they have after enrollment? 9 What kind of diseases did they develop? 10 The nice and good thing about the Ag Health Study -- they 11 knew that they could do what we call a "passive follow-up." 12 Passive follow-up is, ah, we have a cancer registry. As long 13 as we can run all of the names of the cohort members across the 14 cancer registry, we'll find those cancers. Right? We never 15 have to talk to the farmers again. We'll find their outcome 16 passively. Good design for that reason. 17 The problem is -- and the Ag Health Study is different 18 from the most famous cohort study in the U.S., the Nurses 19 Health Study, that Harvard is running. The Harvard study 20 re-contacts these nurses -- 120,000 -- since the '70s, every 21 two years. They are asking every two years what these nurses 22 are doing. Okay? And they are updating the exposure 23 information every two years, because things change. 24 Whether the women take estrogens changes. 25 Whether they take aspirin, as prescribed. RITZ - DIRECT / FORGIE 54 1 Changes. There are lots of things that change. We change 2 smoking behavior. Right? We change diet. We change pesticide 3 use. Right? 4 So the problem and the beauty and the challenge of a 5 cohort study of pesticide application is not only, How do I 6 assess the pesticides that the farmers used in the past with a 7 questionnaire, and do that accurately? How do I then also 8 document how that exposure, over time, changes? Right? 9 And that is the big challenge in the Ag Health Study that 10 I think the Ag Health Study really struggled with, because the 11 second time, it took them five years in the early '90s and mid 12 '90s to do their baseline assessment. And by the time they 13 were done with that, they realized they needed to update 14 exposure, because there was quite a lot of pesticide use 15 changing; specifically, and most dramatically, the glyphosate 16 use. 17 So they started not because of glyphosate, but in general, 18 because, you know, a good investigator does that. They want to 19 go back and see whether things change, and how they change. 20 So they tried to find these farmers again between 1999 and 21 2005. And they found 63 percent. 22 38 percent -- a third -- a third of all farmers -- 23 THE COURT: I think it was 37. 24 THE WITNESS: Thirty-seven. Yeah. Good. Yeah. A 25 third. 6.6. Whatever. Thirty-seven didn't want to be found; RITZ - DIRECT / FORGIE 55 1 had died; didn't want to answer; couldn't be found; had given 2 up farming, or didn't want to talk to the -- to the researchers 3 anymore. Right? Didn't have time to talk to the researchers. 4 So now we have a cohort in whom not -- 5 And, by the way, the beauty of the Nurses Health Study -- 6 these nurses are health professionals. That's why Harvard was 7 so smart to start that cohort -- right? -- because they are 8 committed. They are absolutely committed individuals. Over 9 90 percent have been followed for 30 years. Right? 10 That's different from, after five years, having lost 11 37 percent of your cohort. Right? 12 So these farmers, when they enrolled, weren't really sure 13 they wanted to be in this cohort. They were given a 14 questionnaire at a pesticide licensing exam. Some of them 15 might not even have been sure that they wanted to fill it out, 16 but were worried that maybe I don't get my license if I don't 17 fill it out. Right? 18 And then they go home and say, Well, that was it for me. 19 I'm not going to answer anymore questions. 20 And that is actually also documented in the large percent 21 of people who were given an additional questionnaire to take 22 home and fill out, and never sent it back in. Right? 23 So these cohort members were not as committed as the 24 nurses. And my -- my colleague from -- who was my chair for a 25 long time always -- he is the -- he is the initiator of the RITZ - DIRECT / FORGIE 56 1 Danish birth cohort of 100,000 babies. And he always said, If 2 in doubt, stay out. You do not want a cohort you cannot 3 follow. Okay? 4 You need to get committed cohort members, because you want 5 to ask them again, because things change, and you need to 6 update exposure. You need to update a lot of different things 7 about these people. 8 Q. Okay. Doctor, looking at this slide, the disadvantage the 9 cohort method and the following slide, which is the 10 agricultural health study as an example of cohort, those two 11 slides, which are both part of Exhibit 297 -- where did those 12 two slides come from? 13 A. They are from my slide deck that I use in the core 14 epidemiology teaching class that I teach every single year. 15 And I've been teaching for 20 years. And these slides are 16 .pdfs that I found from 2012. 17 Q. So these two slides, talking about the disadvantages of 18 the cohort methods, and using the Agricultural Health Study 19 cohort as an example of that, are from a 2012 teaching slide 20 that you prepared long before you got involved as an expert in 21 this litigation. Is that correct? 22 A. That's correct, because I like to use practical examples 23 when I teach my students. 24 Q. Okay. 25 A. And I was an advisor for a while on the Ag Health Study, RITZ - DIRECT / FORGIE 57 1 so I knew it quite well. 2 Q. As an advisor to the Agricultural Health Study? 3 A. Yes, as an advisor. 4 Q. Okay. And let me turn to -- well, why don't you tell me, 5 please: What are some of the issues that you saw in the 6 Agricultural Health Study, other than the 37 percent that did 7 not return the second questionnaire? 8 A. Right. So the issues -- I think it's a beautiful study. 9 I really admire my colleagues for doing it. And I think 10 there's a very big amount of useful data that they produced. 11 And, you know, I congratulate them to this wonderful study. 12 However, there are disadvantages of a cohort. And these 13 disadvantages, unfortunately, hit this study very hard when it 14 comes to glyphosate. And that is the disadvantage of having to 15 update your exposure; but it's also the disadvantage of 16 pesticide research in general, because we do not have a 17 radiation badge for pesticides. Right? So the only thing we 18 can go from is recall. 19 We have to ask. We have to go and interview these people. 20 And we have to ask them to report what they used. 21 And we can do -- and I do that in my studies. We can help 22 people with -- with visuals. We can help them by saying, Well, 23 go back and look at your purchasing records. Could you ask 24 your wife? Could you ask your son? Do you know who, you know, 25 purchased this with you? Can you reconstruct your exposure RITZ - DIRECT / FORGIE 58 1 history? 2 Because it's really like archaeology. We are trying to 3 reconstruct an exposure history for these people. 4 The Ag Health Study, having to enroll 56,000 applicators, 5 because they wanted to follow them prospectively for a rare 6 outcome -- they only had 575 NHL over 20 years in 56,000. 7 Right? It's not a huge number. 8 The case-control study started with 500 subjects -- cases. 9 So if you want to do that, you have to have an instrument 10 that people won't just get back to you and say, I'm not going 11 to do that. Sorry. I don't have the time. 12 56,000 people have to fill out 20-some pages within half 13 an hour, telling you -- reconstructing their own exposure 14 history; doing that archaeology on the spot. And that is 15 challenging. 16 And, you know, I really admire my colleagues for believing 17 in their instruments and doing it; but everybody knows -- and 18 it has been documented and -- and -- and discussed again and 19 again, what you get are failures in memory; failures in 20 reporting. Right? 21 And we see here this is how they asked it. This was a 22 bubbling questionnaire. 23 So we have Roundup®, the pesticide. Have you ever 24 personally mixed or applied? Yes? No? Easy. 25 But then: How many years have you personally mixed or RITZ - DIRECT / FORGIE 59 1 applied? 2 And you have categories. Oh, well, maybe 10 to 20 years. 3 Okay. 4 And then on -- in an average year, how much? How many 5 days did you apply? 6 So you have to say, Okay. 10 to 20 years. What was my 7 average year? Is that the last year? 8 Because most people remember things that are closer better 9 than things that were further in the past. And maybe they 10 report just that last year, and it's really not the average 11 year. Right? 12 So you're challenging people to do a lot of complex 13 archaeology of their exposure history on the spot, when they're 14 there to get their pesticide applicator license. And 15 glyphosate at the time was not sold to them as a highly toxic 16 pesticide. Right? They had concerns about pesticides other 17 than glyphosate that might -- might actually cause acute 18 symptoms. And the farmers probably experienced acute symptoms 19 from pesticides, such as organophosphates. 20 Organophosphates are derived from sarin gas. 21 Organophosphates give you flu-like symptoms. Right? 22 So somebody who comes to get this license done -- 23 licensing done for restricted-use pesticides, and then is asked 24 about 22 different pesticides, will probably remember the most 25 important pesticides they have in their mind, because they are RITZ - DIRECT / FORGIE 60 1 toxic, and they really want to avoid exposures. They want to 2 really get the application right, more than one they kind of 3 think is, you know, something that I give to my wife to go and 4 spray the weeds in the yard. Right? 5 And so it's about the attitude of reporting. It's about 6 being encouraged to report correctly, and even to remember or 7 weigh these things correctly. 8 MS. FORGIE: And with regard -- I should mention that 9 this is exhibit -- this questionnaire for the Agricultural 10 Health Study has a separate exhibit number, which is 228. 11 Q. And then with regard to -- can you explain how many 12 questionnaires were used in the Agricultural Health Study? 13 And can you explain it in light of what exposure 14 misclassification is? 15 And does the exposure misclassification and the recall 16 error apply to not just the 37 percent that responded to the 17 second questionnaire, but also to the 63 percent that did not 18 respond? I mean, yeah, that did not. 19 A. Yeah. So what we have in front of us -- 20 Q. That did respond. Sorry. 21 A. -- is actually the baseline questionnaire, because the 22 second time around, those 63 percent -- they were interviewed. 23 And that questionnaire was very different. It was a telephone 24 interview. Right? And there were prompts from an interviewer. 25 By the way, it really helps to have prompts from an RITZ - DIRECT / FORGIE 61 1 interviewer. 2 This is really the quick and dirty -- I call it -- one 3 that you give to people and say, Fill it out. Right? You're 4 here for your licensing exam. Just do it. 5 And you get it back. And the data is the data. And you 6 live with it. 7 What happens when you do that is you generate potential 8 recall error. 9 Recall error is not recall bias. Recall error is 10 something that I just misremember, misreport. You know. It's 11 not systematic. 12 Recall bias is systematic. Because I'm a case, I remember 13 it differently. Because I'm not a case, I'm remember it 14 differently. 15 This is random error of recalling or reporting. 16 The problem is this is really the enemy of exposure 17 assessment in a big way, when you are -- when this is about 18 protecting public health, because it causes nondifferential 19 exposure misclassification. A randomness in the exposure 20 misclassification. That is that signal-to-noise ratio 21 reduction. So you will not find a signal. 22 You cannot protect the public if you don't see anything. 23 And if this reporting is done wrong, and you have it -- you 24 know, whether you develop the disease or not doesn't matter; it 25 was just a random error -- the nondifferential RITZ - DIRECT / FORGIE 62 1 misclassification of that exposure makes it likely you don't 2 see anything. 3 THE COURT: I've been having a hard -- 4 That's one of the things I've been having a hard time 5 understanding, because that concept comes up in a number of the 6 reports and the supplemental reports on the AHS Study. 7 THE WITNESS: Right. 8 THE COURT: This concept of nondifferential exposure 9 misclassification moving us closer to the null, no matter what. 10 THE WITNESS: Right. 11 THE COURT: And I'm just having a hard time wrapping 12 my head around that. I was wondering if you could explain why 13 that is the case in a little more detail. 14 THE WITNESS: Yes. And you are not alone. Believe 15 me. My students have the same problem. And I show them lots 16 of examples. And, you know, they finally develop an intuition 17 when we do the numbers. 18 But the easiest way to understand it is that when you have 19 random error, you're moving people. Let's say it's just 20 exposed/unexposed. Right? 21 And somebody forgot that they used glyphosate, so they're 22 not reporting it, so you're putting them in the unexposed 23 group. 24 And somebody else misread it, and thought he was using 25 glyphosate, but it was really a different agent. And you put RITZ - DIRECT / FORGIE 63 1 him in the exposed category. Right? 2 So you're moving some people from the exposed to the 3 unexposed, and, randomly, some people from the unexposed to the 4 exposed. 5 So, in essence, you're -- you're minimizing the 6 difference. 7 That -- in disease -- that you can see in disease rates, 8 because you have contaminated both groups with the other. 9 That's it. 10 Make sense? 11 THE COURT: Mm-hm. 12 THE WITNESS: Good. 13 BY MS. FORGIE 14 Q. And does this idea of nondifferential exposure 15 misclassification -- is that something that people have been 16 aware of and published about, with regard to the Agricultural 17 Health Study? 18 A. Absolutely. So there are many, many people who are, of 19 course, in my field, aware of this. We all are. 20 But there is actually specifically groups who have been 21 addressing this with respect to the Agricultural Health Study. 22 And the first one we list here is the Gray, et al., in the 23 federal government's Agricultural Health Study critical review, 24 which was from the Harvard Risk Assessment Group. It's a 25 distinguished group of scientists who were charged and paid by RITZ - DIRECT / FORGIE 64 1 CropLife, which is an industry -- pesticide industry group that 2 asks -- issues these calls, and pays for them, to evaluate 3 potential problems with the whole Ag Health Study in general, 4 because, I mean, it's a study that costs a lot of resources and 5 money. And the government is conducting it, so we should be 6 knowing what the issues might be. 7 And they prominently state here, "Nondifferential exposure 8 misclassification" -- the one I just described -- "will produce 9 bias towards the null." Right? "Misclassification reduces the 10 power of the study to detect any genuine cause/effect 11 relationship, and will also reduce the validity of the 12 findings." 13 That's them. 14 Indeed, Dr. Acquavella, a colleague who I met, myself, 15 when I was on the panel -- on the Ag Health panel several 16 times, and also in Los Angeles -- he's a colleague who now 17 works in Los Angeles -- he was the epidemiologist for Monsanto. 18 And he would come to meetings; present on behalf of Monsanto. 19 And in the journal Epidemiology, which is actually the 20 official organ of the International Society for Environmental 21 Epidemiology -- so our journal. My society's journal. We hold 22 it in high esteem. And it's not easy to get published in this 23 journal. And he -- to his credit, he did. And he wrote about 24 exposure misclassification in the ag health pesticide study, 25 and insights comparing biomonitoring studies with what the Ag RITZ - DIRECT / FORGIE 65 1 Health Study did in terms of interviewing. 2 And his conclusion here was that there would be, 3 especially for that cumulative day of use measure -- right? -- 4 that very important one, where we're thinking we can now have a 5 dose in the Ag Health Study -- that that would have probably 6 substantial exposure-misclassification issues, because we might 7 assign these doses in the wrong way. 8 And what I just explained to you was just two exposures -- 9 not exposed/exposed -- happens also when we have a scale, 10 because all of this can go in either way, in either direction. 11 Right? 12 And so we are muddying the pond multiple times. He said 13 it. 14 And then we have Weichenthal. He's a Canadian colleague 15 who reviewed all of the ag health cancer publications in 2010. 16 He published an Environmental Health Perspectives, which is 17 actually the official journal of the National Institute of 18 Environmental Health Sciences. Again, very hard to get into 19 that journal. 20 And in this review he assessed all of the different cancer 21 outcomes across different studies, across different cancers, 22 across different pesticides, and then came to his conclusion. 23 And that conclusion also was exposure misclassification 24 undoubtedly had an impact on AHS findings reported to date. 25 And the next sentence, unfortunately, isn't on here; but he RITZ - DIRECT / FORGIE 66 1 refers back to nondifferential exposure misclassification being 2 one of the enemies we have to be aware of. 3 Q. And did the authors of the recent AHS publication, the 4 lead author being Dr. Andreotti -- 5 And the exhibit number is 12. 6 -- did they also talk about exposure misclassification? 7 A. Indeed, as any good epidemiologist should do, we limit -- 8 we list the limitations of our study. And Andreotti, just the 9 same, lists nondifferential exposure misclassification as one 10 possibilities to explain what they are presenting. So despite 11 the specific information provided by the applicators about the 12 use of glyphosate, some misclassification of exposure 13 undoubtedly occurred. 14 Given the prospective design, however, any 15 misclassification should be nondifferential, and lead to an 16 attenuated risk estimate. 17 Q. Okay. And did -- I'm going to have to move it along just 18 a little quicker, because we have some time constraints here. 19 Did the authors of this study -- they were obviously aware 20 because of these peer-reviewed publications and their own 21 comments -- the Agricultural Health Study investigators. Did 22 they make any attempt, through either Sensitivity Analysis or 23 validation studies, to correct these? 24 And can you explain very briefly what Sensitivity Analysis 25 is, and what validation studies are? RITZ - DIRECT / FORGIE 67 1 A. Right. So Sensitivity Analysis is really just playing 2 with the data you have. You're not generating new data. 3 You're just looking at your data in different ways, to try to 4 figure out whether you can assess the strengths of the bias. 5 Right? Whether -- or whether you can maybe remove a bias, or 6 can argue that, If I look at my data this way, and this is 7 still consistent, does that mean something? 8 JUDGE PETROU: Is that all post hoc analysis? 9 THE WITNESS: Post hoc. Yes. And honestly, when it 10 comes to exposure misclassification, it's really hard to do 11 that post hoc -- 12 What we really want to do -- 13 -- and to draw conclusions from that. 14 What we really want to do is go out there, and measure, 15 and then have a gold standard of measurement against which we 16 then can judge what the exposure assessment was like. 17 And I have to say, to the credit of my colleagues, they 18 spent a lot of money and effort trying to do that. They had 19 NIOSH go out there and monitor farmers in the way they were 20 applying pesticides, and then collecting urine samples from 21 them. 22 Dr. Acquavella, himself, got Monsanto to pay for one such 23 study, where he went out there and had observers observe 24 farmers in the field spraying the glyphosate; taking urine 25 samples; measuring what's in there; having them fill out the RITZ - DIRECT / FORGIE 68 1 same questionnaires that the Ag Health Study used, and then 2 comparing them. 3 And, lo and behold, what they found is that the 4 correlations were so-so. They weren't too bad, but they really 5 depended on the kind of agent. And unfortunately, for 6 glyphosate, they didn't have much luck. 7 They had much better luck with chlorpyrifos and 2,4-D, 8 where the correlations between the urinary levels and what they 9 actually observed and what the observers reported and what 10 the -- the applicator, themselves, reported -- those were are 11 .5. Not so bad. 12 But really the correlations with the urine samples for 13 glyphosate were pretty minimal. 14 And you have to remember these are also people reporting 15 within a few days of applying, so their memory is much better. 16 They also have been observed; meaning they tried their best. 17 Right? Their best behavior. 18 So what they are reporting is really short term. It is 19 the best we can do, but it certainly does not tell us whether 20 somebody reporting their glyphosate use for 10 or 20 years in 21 the past -- their baseline -- is really reflected in that 22 validity assessment with the urine samples. 23 Q. And did the -- can you explain a little bit about the 24 change of use in glyphosate that occurred during the 25 Agricultural Health Study, and how that ties in with exposure RITZ - DIRECT / FORGIE 69 1 misclassification, please? 2 A. Right. So what I can do when I cannot interview people is 3 make informed guesses. And that's what my colleagues did. 4 They said, All right. Maybe I can use the baseline information 5 I collected, no matter whether it was good information or bad 6 information; whether they remembered already -- misremembered 7 already in the beginning, or reported quite accurately. I use 8 that data and what I know about these people to predict their 9 future exposure. 10 MS. FORGIE: Let me just interrupt for a second. 11 I should say this is Exhibit 299. It's actually from the 12 recent December 12th, 2017, EPA Draft Analysis with regard to 13 glyphosate. 14 THE WITNESS: Right. 15 BY MS. FORGIE 16 Q. Sorry, Doctor. Go ahead. 17 A. So what we see here is a beautiful map of the U.S., 18 showing the estimated agricultural glyphosate use in 1994. The 19 date is very important, because 1994 is kind of in the middle 20 of that baseline assessment from the Ag Health Study. 21 And you can see -- can you see Iowa? It's near the 22 Great Lakes up there, in the middle. And it has kind of an 23 orange-y and slightly reddish color. So that was the 24 glyphosate use. 25 We see that most people use -- or most farms are covered; RITZ - DIRECT / FORGIE 70 1 meaning they're already using glyphosate. And that's before 2 GMO. Okay? 3 And in North Carolina, which is on the right, there's not 4 so much use; but Iowa is definitely covered. 5 However, now, compare that -- 6 And this is the data we're using -- what they report in 7 the baseline -- to then predict what they did the next 10 8 years, the next 20 years, if they did not answer again. 9 This is what we see in 2014. The cancer assessment in the 10 AHS -- the last cancer was recorded in 2013 from the Andreotti 11 Paper. Andreotti Paper. So between 1994 and 2014 we pretty 12 much cannot distinguish unexposed from exposed anymore, because 13 somehow every farmer must be using glyphosate; and not only 14 using it, but using it at a very high level. Right? 15 This -- this is kind of interesting in epidemiology, 16 because we also know that as soon as an exposure becomes 17 ubiquitous, it's really hard to estimate what it does, because 18 if everybody is exposed, we are now having to distinguish the 19 amount of exposure very carefully in order to say whether the 20 rates of disease are increasing. 21 And so this almost -- 22 So in 2014, Iowa, we would probably be not -- it would not 23 even be possible to estimate any risk from glyphosate anymore. 24 It's like with cell phones. Once everybody uses a cell phone, 25 we cannot estimate the risk from cell-phone use on brain damage RITZ - DIRECT / FORGIE 71 1 anymore, if there is any. So it's the same. Ubiquitous 2 exposure really is scourge of our discipline. 3 And -- but why I showed this is more to say, really, we 4 are going from 1994 to 2014, trying to estimate not only the 5 amount they are using while everybody is starting to use 6 glyphosate as if it's, you know, aspirin, and we should be 7 using it with our breakfast cereal. It is also -- they are 8 trying to estimate when this happened. Right? Because I told 9 you it is important when exposure happens; not only whether it 10 happens. 11 Did it happen within a year or two before the NHL? Did it 12 happen five years earlier? When was that change? When did 13 that exposure happen? 14 And all of that is estimated for that 37 percent. 15 Not only that, in the second round they are asking farmers 16 to now report what they used. And you would think, Oh, they're 17 asking them to report what they used since we asked them first. 18 Right? That would be logical. 19 No, they didn't ask that. 20 They asked them to report what they used in the last year 21 they farmed. 22 Okay. Now I have one year of exposure from which I then 23 extrapolate for the 63 percent that answered. And now I have 24 to extrapolate backward to their first question, and forward 25 into the future, whether that one year definitely represents RITZ - DIRECT / FORGIE 72 1 the 20-year period. Again, we are guessing. 2 Q. Is it possible to give an example, given that there's only 3 one year of information, for the 37 percent -- I mean, for the 4 percentage that answered the questionnaires? Is there any way 5 to estimate or to know how much use they actually had? 6 A. Well, we can do our best to try that, but we are ending up 7 estimating. We can definitely predict well whether or not 8 somebody used it. Right? If they -- if they tell you they 9 used soybeans or corn, it's probably GMO. It's probably 10 glyphosate. 11 How much they used, how many daze they used -- that's a 12 totally different guess. Right? 13 But I have to make that guess, in order to get the 14 dose-response. 15 Q. All right. Given your explanations with regard to the 16 Sensitivity Analysis attempts and the validation attempts to 17 correct the problems with the AHS Study, did you make a 18 determination as to how much weight, in your opinion, to give 19 the AHS Study? 20 And is there a methodological reason for the amount of 21 weight that you gave the study? 22 A. Right. So in my scientific assessment, the 23 exposure-assessment issues were so grave that for glyphosate, 24 which changed mid baseline, which wasn't updated properly, and 25 had really hundredfold increases in the time period we are RITZ - DIRECT / FORGIE 73 1 studying, where we're making so many guesses, I really have to 2 downweigh the importance of the AHS Study that otherwise I 3 really love. 4 But for a time-varying exposure, I just cannot take this 5 study serious, in terms of the science that it produced, if it 6 shows no effect, because all of the affects are drowned out in 7 the noise of exposure assessment -- exposure misclassification. 8 Q. Let me go to one other issue. And can we turn to the 9 slide that talks about protective equipment? 10 A. Mm-hm. 11 Q. Yes. This one. Can you explain -- this is, again, from 12 the questionnaire, which is Exhibit 299. 13 MR. WOOL: 298. 14 MS. WAGSTAFF: 228. 15 MS. FORGIE: 228. I'll get it right. For the third 16 time, 228. Okay. Exhibit 228. 17 Q. Can you explain what this protective equipment from the 18 AHS questionnaire -- what it is, and what it tells you about 19 the study, and about how they tried to do the intensive 20 weighting scores? 21 A. So this is a very important question, because a lot of 22 what they call their "validation studies" were actually trying 23 to see whether this information tells them something about the 24 intensity of the exposures might have gotten. 25 If I use a full moon suit, I'm probably protected. RITZ - DIRECT / FORGIE 74 1 If I sit in a tractor with a cab that has a negative air 2 flow, I'm probably protected. I can spray as many glyphosate 3 as I want. I don't breathe it. I won't, you know, get it on 4 my skin. I'm fine. 5 If I use aprons and face shields, I don't get it splashed 6 in my eyes. 7 If I don't repair equipment, I don't have it on my hands. 8 I don't, you know, eat by accident without washing my hands. 9 So we really want to get to people telling us what they 10 do, and how they do it, because that may really determine, much 11 more than the acreage or the days they sprayed, how much 12 exposure they got. And my colleagues got that right. You 13 should ask. That right? 14 However, because they only had probably 20 minutes, half 15 an hour with every individual, they did not put this question 16 after every pesticide. 17 They put this question at the end of the list of 20-some 18 pesticides that they asked about. And now they said, What type 19 of protective equipment do you generally wear when you 20 personally handle" -- 21 This is for all pesticides. 22 These are guys who are coming to get their license for 23 restricted use. And, of course, they are thinking about the 24 most toxic pesticide they're handling. Right? And they want 25 to say that they're doing this correctly. So what they are RITZ - DIRECT / FORGIE 75 1 recording here is not an -- and I really don't think it is -- 2 what they would be doing when they're spraying glyphosate. 3 This is what they're using to protect themselves from the 4 most toxic agent they've been reporting on. 5 However, this question is used for every single pesticide 6 in the same way with the same algorithm that Dr. Dosemeci 7 describes in his intensity-of-application algorithm, where we 8 used weighing factors for having used chemically resistant 9 gloves, or for disposable clothing, for face shields, to 10 downweigh the exposure reported for any pesticide, including 11 glyphosate. 12 But we don't know whether they did any of this when they 13 sprayed glyphosate. It might have been, you know, only 14 pesticide they're reporting on. We don't know. 15 Q. And how was the algorithm used to determine 16 intensity-weighted scores in the Agricultural Health Study? 17 A. Well, it plays a big roll for the intensity-weighted 18 score. That's what that score is all about. We are weighing 19 according to: How was the pesticide applied? Did you fix the 20 equipment? Did you wear a face shield? 21 Whenever that answer -- that question's answered "Yes," 22 you downgrade the exposure. So you put somebody in the 23 low-exposed group when they're reporting that. 24 If they're not reporting that, you keep them in the high. 25 Right? RITZ - DIRECT / FORGIE 76 1 So you could see all sorts of scenarios where the true 2 exposure for glyphosate -- somebody who really is highly 3 exposed landed in the low-exposed group. Somebody who was 4 low-exposed landed in the high, because they weren't reporting 5 for glyphosate; they were reporting for any other pesticide. 6 Q. Monsanto's claiming that you're criticizing this now, and 7 did in your deposition -- as a litigation expert, that you're 8 criticizing the use of this algorithm -- but you use the same 9 intensity score in your work. 10 Can you explain that? 11 A. Yes. 12 Because these really were depending on what pesticide we 13 are talking about -- right? -- because that's what they 14 actually showed in their validation study for chlorpyrifos. It 15 worked. Right? And so we use what we know for the things that 16 we think it works with. 17 However, you need to ask this for each and every 18 pesticide. You can't just generalize it across every pesticide 19 they reported. And, yes, it is an algorithm we should be 20 using, but we should be asking these questions for every single 21 pesticide. 22 Q. Okay. Can you explain briefly what a meta-analysis is, 23 and how it relates to glyphosate-based formulations and 24 non-Hodgkin's lymphoma, please? 25 THE COURT: Before you get into that, let me ask if RITZ - DIRECT / FORGIE 77 1 you think now would be a good time. We've been going for 2 about, I think, like, an hour and 45 minutes or something. 3 Would now be a good time to take a lunch break? 4 MS. FORGIE: I would love a break of any kind. 5 THE COURT: Why don't we break until about 12:35. 6 We'll resume then. 7 MS. FORGIE: Great. Thank you so much. 8 THE WITNESS: Thank you. 9 THE CLERK: Court is in recess. 10 (Luncheon recess was taken at 11:48 a.m.) 11 AFTERNOON SESSION 12:39 p.m. 12 THE COURT: Are the these (indicating) safe to eat? 13 THE WITNESS: I think so. 14 THE COURT: You can resume. 15 MS. FORGIE: Thank you, Your Honor. 16 Q. Okay. So I think we were talking -- we were just about to 17 talk about meta-analysis when we took our lunch break. And -- 18 THE CLERK: Ms. Forgie, can you turn the microphone 19 towards you, please? 20 MS. FORGIE: Oh, sorry. 21 THE CLERK: Thank you. 22 MS. FORGIE: Is that okay? Okay. Now is it okay? 23 Okay. 24 BY MS. FORGIE 25 Q. So, now, right before the break we were talking. We were RITZ - DIRECT / FORGIE 78 1 just about to start talking about meta-analysis. And were 2 there meta-analyses that were performed with regard to the 3 relationship between exposure to glyphosate-based formulations 4 and non-Hodgkin's lymphoma? 5 Can you explain just a little bit briefly what they were; 6 what they showed, please? 7 A. Right. I would like to just state briefly what 8 meta-analyses were, and why they were challenging. 9 Q. Okay. 10 A. So it's actually interesting. My professor, 11 Dr. Greenland, who wrote the book -- when he was asked what 12 would be the most challenging thing for the next decade in 13 epidemiology, his answer was, To do meta-analyses correctly. 14 And he did not have an answer on how to do this right. He said 15 this is a field of research, as a methodologist, that we really 16 need to explore. 17 And from learning from him and discussing all of these 18 issues with him over 20 years -- two decades -- what I learned 19 is there is no right way. There are different ways. 20 And there are different ways of putting data together. 21 And, yes, we want summary estimates. We want to summarize the 22 information across studies; but every single study has its own 23 flavor. And whether or not it's okay to use -- and they all 24 present different effect estimates; not just one. They usually 25 present 20, 30 different variables in one model than in RITZ - DIRECT / FORGIE 79 1 another. 2 And so we have to be very careful, as a meta-analyst, to 3 not throw apples and oranges and carrots and roots in the same 4 bucket, and say they're all the same. 5 On the other hand, if they all give us the same result, 6 we're actually pretty happy. Then there's consistency across 7 the field -- right? -- even if each study is done very 8 differently and analyzed very differently. 9 However, you have to make qualitative judgment calls in 10 terms of what you're putting into that meta-analysis. So you 11 want to have the estimates from across the studies that are 12 most similar to each other, so you can actually summarize 13 across them. 14 On the other hand, what I learned from him is what we 15 often learn more by doing a meta-analysis is what sticks out 16 like a sore thumb. So what -- what is the study that doesn't 17 fit the pattern? Right? 18 And then -- not to say that study is wrong and everything 19 else is right, but to actually learn; to learn what might have 20 gone wrong or right in that study, and then to plan the next 21 study from that point of view. 22 So we are -- we are therefore making a lot of what we call 23 "Sensitivity Analyses" around these studies, grouping them 24 according to: When was the study done? 25 What type of study design did they use? Case-control? RITZ - DIRECT / FORGIE 80 1 Cross-sectional cohort? 2 Maybe it's systematically different -- what we're seeing 3 in one design, than another. 4 Maybe it's systematically different in terms of what 5 decade the study was performed in, because exposures changed. 6 Maybe it's systematically different in terms of what other 7 risk factors they actually had in that population. Right? 8 So we're learning by doing meta-analysis. 9 And the summary estimate at the very end is just the 10 cherry on top of the ice cream, but it's not the end-all. It 11 is something that we take with a grain of salt, assuming that 12 we could actually generate that summary estimate from all of 13 the diversity that was seen in these studies. 14 Q. And did -- can you briefly describe the Schinasi, IARC, 15 and Chang and Delzell meta-analysis, please, very briefly? 16 A. Right. So what authors do is they go through the 17 literature. They're picking out the studies that they think 18 have the minimum criteria for validity. And then they pick, 19 out of those studies, the estimates they trust the best. And 20 these estimates may be adjusted for one variable in one study; 21 for another in another. But they are -- you know, they're the 22 best you can do. 23 So you're combining these. You are combining these 24 estimates, but ultimately it comes to the judgment of the 25 authors, in terms of which studies could be qualified as RITZ - DIRECT / FORGIE 81 1 fitting a meta-analysis. 2 And both meta-analyses were done with slightly different 3 studies; slightly different criteria of maybe excluding one or 4 the other. But in the end, they came up with the same effect 5 estimate -- just about the same. 6 Q. And what was -- 7 A. And the confidence. 8 About 1.3 to 1.5, depending on how you, you know, group 9 these studies. 10 And all of them were statistically significant, because 11 that is actually one of the great advantages of a 12 meta-analysis. You are now learning from studies that you're 13 pooling together; meaning you have more data, more information 14 in order to assess these effects across studies. 15 Q. And do you know if they used -- 16 Where adjusted-for-pesticide variables were available for 17 the Odds Ratio, do you know if they used in the meta-analysis 18 those adjusted-for-pesticide Odds Ratios? 19 A. Definitely. Yes. That's what they did. 20 Q. Okay. And then, Doctor, briefly, in assessing what weight 21 to give any particular study, can you tell me roughly how you 22 do this? 23 And do you decide whether or not the study had any type of 24 conflict of interest in it? 25 And how does that affect your decision as to how much RITZ - DIRECT / FORGIE 82 1 weight to give any particular study? 2 A. So is that now in a meta-analysis? 3 Q. No. 4 A. Because in a meta-analysis it's set. 5 Q. Sorry. No. I should have -- 6 A. Yeah. No. 7 Q. I should be clear. With your other -- 8 A. Right. 9 Q. With all of the other studies: The case-control, the 10 AHS -- 11 A. All of them. 12 Q. You know. Williams. Any of those. 13 A. Right. So -- so I would say in 20, 30, 35 years now that 14 I do this, I really look at every study. And I really form an 15 opinion on how valid that study is overall. And there is no 16 perfect study, as there is no perfect human being. 17 But as a doctor, I teach my students: It's not the wart 18 that's going to kill the patient. Right? It's not the tiny 19 little bit of confounding or selection bias. It is the heart 20 attack. So let's just focus in on the major problems, and take 21 care of those. 22 And if I can convince myself that there isn't a major 23 problem, then I can live with the warts, and I can live with 24 maybe hair loss. Right? 25 And the study's not perfect. And it's not the end-all of RITZ - DIRECT / FORGIE 83 1 studies, but it provides enough information for me to form an 2 opinion in the context of everything else I know. Right? 3 And so I use my sense as an epidemiologist to evaluate 4 every single epidemiologic study, but I also go beyond my own 5 field, and I read what my colleagues in toxicology do. This is 6 the beauty of the COEH, the Center for Occupational and 7 Environmental Health. I'm an epidemiologist. There's a 8 toxicologist. There's a chemist. There is a molecular 9 biologist. We are actually talking to each other. 10 And I, being trained in medicine, I love to talk to them. 11 So I love to go and discuss issues of biology, of 12 carcinogenicity. And so I cannot just close my mind and say, 13 Epi is all I do; is all I can -- 14 JUDGE PETROU: Going back to the question on the 15 table, which was what were, essentially, the criteria that 16 you -- 17 THE WITNESS: Right. 18 JUDGE PETROU: -- assess for validity? And we 19 started talking about the warts versus the heart attacks. What 20 are the heart attacks? 21 THE WITNESS: The heart attacks would be, for 22 example, if you cannot trust the exposure assessment, at all, 23 in terms of giving you the right category of exposure for 24 people. 25 Or if there is such profound confounding -- if there is RITZ - DIRECT / FORGIE 84 1 really a very strong risk factor that they just couldn't take 2 care of, and couldn't convince me that they have taken care 3 of -- 4 A lot of occupational studies that look at lung cancer 5 don't have information on smoking. Right? 6 So you would worry that, you know, these asbestos 7 workers -- really what the problem was was their smoking habit, 8 and not the asbestos. 9 So you would look at that study with a grain of salt, 10 saying, Well, unless you convince me smoking wasn't a 11 confounder -- it wasn't associated with asbestos exposure -- 12 maybe not. Right? 13 But I really need a strong reason to think that I need to 14 throw this study out, or weigh it down a lot. 15 So -- so, yeah, I use different -- what I know from 16 different disciplines that I also have some access to. I can 17 read some studies and some of the results. 18 And then I apply the Bradford-Hill Criteria, which is, you 19 know, what we do; what Dr. Austin Bradford-Hill gave us as a 20 tool; but they are not check boxes. They are really 21 instructions on how to think about science, and how to put the 22 whole picture together. 23 And it is not like, oh, this criteria isn't met; 24 therefore, there is no causality. That's not what we do. 25 We really look at the Bradford-Hill Criteria as a RITZ - DIRECT / FORGIE 85 1 guideline of how I put all of these pieces of the puzzle 2 together. And on the whole is there a picture, even if the ear 3 is missing? Right? So is that a face? Is that a human face, 4 even if I don't have the puzzle pieces for the ear? 5 And we can decide that. 6 BY MS. FORGIE 7 Q. Okay. So, Doctor, since you brought up Bradford Hill, can 8 you tell us if you performed a Bradford Hill analysis; and 9 briefly, through the various steps that you looked at, and what 10 each of them meant; and how you determined whether or not that 11 particular factor or criteria was met, please? 12 A. Absolutely. 13 So I put the studies that I read into this context, but 14 not just each study, alone, but all of the studies put 15 together. 16 So is the specificity? 17 Did they actually look at glyphosate, instead of asking, 18 Well, were you pesticide-exposed? You wouldn't believe it, but 19 a lot of studies do that. 20 None of these studies does that. They actually asked 21 glyphosate. 22 They actually make an attempt to get at glyphosate 23 exposure prior to disease onset? 24 Yes, they did. 25 So temporality has been established. RITZ - DIRECT / FORGIE 86 1 Specificity has been established. 2 Did they -- is there consistency across studies? 3 Is there a pattern? If I look at the people that I think 4 are more highly exposed versus low-exposed -- the routine users 5 versus the occasional users -- is there a pattern observed in 6 these studies? 7 Yes, it is. 8 Is it likely that we would see this just by chance? 9 Well, we have statistical significance. 10 All of these studies add up. They are all going in one 11 direction; the ones that I weighed heavily. 12 Is there biologic plausibility? 13 Yes, there is, because we have the animal studies. We 14 have the cell studies. Right? 15 So we put this together. 16 Q. Can you explain just a little bit more, briefly still, 17 what biological plausibility is? 18 And what is the biological plausibility for which 19 glyphosate-based formulations can cause non-Hodgkin's lymphoma? 20 A. Right. So in former days we thought that carcinogenicity 21 is a very simple event. And we have learned that that's not 22 the case. There are actually multiple ways of how cancer cells 23 become cancerous, and then also survive and grow. 24 And IARC actually established a whole list of criteria, 25 now that they are considering contributing to a carcinogenicic RITZ - DIRECT / FORGIE 87 1 mechanism. 2 One of them is reactive oxygen species in a cell that can 3 then attack proteins; that can reduce the capacity of a cell, 4 when there are chromosome breaks, to repair them. 5 Of course, chromosome breaks -- mutagenicity -- is one big 6 criterion; but there is also inflammation. 7 Does inflammation help these controls to develop? 8 And is the immune system sufficiently active to maybe 9 detect these cells, or not? 10 Are these people immunocompromised? 11 So there are multiple mechanisms that we are using to 12 evaluate whether there is plausible -- biological plausibility 13 to the biology that this agent might be contributing to. 14 Q. And can you very, very briefly explain how genotoxicity 15 and oxidative stress fit into the biological-plausibility 16 issue? 17 And also, is assessing biological plausibility something 18 that you've done in your 20 years of studying and publishing 19 peer-reviewed publications with regard to pesticides and 20 cancer? 21 A. Well, in fact, yes. This is -- this is very important. 22 Even so, there are established carcinogens for which we, for 23 years, didn't know what the biologic mechanism was. It was 24 just, you know, observed in humans, and believed that it caused 25 cancer in humans, such as asbestos. We didn't know how. We RITZ - DIRECT / FORGIE 88 1 are feeling much better when we can identify these mechanisms 2 that are plausibly contributing. 3 So oxidative-stress generation, genotoxicity, breakage of 4 DNA that then has to be reassembled, are these kind of 5 mechanisms that we are looking for. And that's what my 6 colleagues in toxicology actually make their money doing. 7 Right? They're testing animals. They're testing cells. They 8 are testing these mechanisms. 9 Q. And is there any peer-reviewed publications that showed 10 genotoxicity with regard to non-Hodgkin's lymphoma and 11 glyphosate-based formulations? 12 A. Absolutely. There are -- there's animal data; but more 13 importantly, there is even human data and human lymphocytes. 14 And remind -- just to remind everyone, you know, we're 15 talking lymphoma. Lymphocytes are the cells. Right? And in 16 these lymphocytes, they have -- DNA breaks have been shown to 17 occur when individuals were exposed. 18 Q. Okay. And other thing with regard to the weight that you 19 give to particular studies -- do you ever look at conflicts of 20 interest? 21 And in assessing a study, how do you do that? 22 And is that something that you teach your students at 23 UCLA, as a professor of epidemiology? 24 A. Yes. We have to, unfortunately, do that, because not 25 everybody reports the results in a way that we might want to RITZ - DIRECT / FORGIE 89 1 trust them. So we are looking for signs of, you know, conflict 2 of interest, as well as scientific fraud in everything we read. 3 And the major journals actually say very clearly what 4 those are -- the conflicts of interest. And you have reams of 5 pages that you have to fill out now for JAMA or for Science in 6 terms of what your potential conflicts of interest are, because 7 these major journals have found that it actually makes a 8 difference whether or not you have a conflict of interest, how 9 you publish. 10 Q. And how do you -- is there an acknowledgments section? Or 11 how do you go about finding out if there's a conflict of 12 interest? 13 And, just very briefly, how does that affect your 14 weighting of that publication? 15 A. Well, you have to state -- or the authors now have to 16 state whether they, you know, have a conflict of interest, 17 which could be, most of the time, a financial interest. And 18 that would then be published with the -- in the -- in the 19 scientific publication as, "Author X stated." Right? 20 But you can, of course, also see what the affiliation of 21 the authors are. And you can see patterns in publications, 22 such as -- we have some colleagues who, no matter what paper 23 they write, they never find any affects for -- 24 While a lot of other colleagues may be publishing papers 25 that actually see something. RITZ - DIRECT / FORGIE 90 1 So you're putting that into the context; into the larger 2 context of the scientific literature. 3 Q. And the methodology that you used in analyzing particular 4 studies, also documents -- EPA documents, the CARC report, 5 things like that -- is the methodology that you used in 6 analyzing these and forming your opinions in this case the same 7 that is used by colleagues in their regular course of work, and 8 that you use in your course of work, and also as a reviewer of 9 journals? 10 A. Absolutely. You have to do that. I mean, honestly, I'm a 11 scientist because I want to get to facts. I want to get to the 12 truth. I want to protect the public interest. And I don't 13 want to do this in a way that is biased. 14 And bias analysis is what I teach as a core. And I teach 15 my students because, you know, we have to avoid biases. 16 Conflict of interest is one bias. 17 Q. Okay. And have you reached some conclusions and opinions 18 in this case? 19 And can you tell me what they are, please? 20 A. Yes. After reviewing all of the scientific literature at 21 hand, I really concluded that, to a reasonable scientific 22 degree of certainty, glyphosate and glyphosate-based compounds, 23 including Roundup®, do indeed cause NHL. 24 Q. And in reaching those opinions and that conclusion, can 25 you tell me what -- RITZ - DIRECT / FORGIE 91 1 I know you've mentioned that you used the same 2 methodology, but can you tell me what that methodology used by 3 you in this case and when you're practicing as a professor of 4 medicine and of epidemiology at UCLA -- can you tell me just 5 basically what that is, in terms of literature searches, and 6 things like that? 7 A. Well, I use the methodology I learned and I teach. And 8 it's the same methodology. Whether I write a review paper, 9 whether I write a report or a grant application, I use the same 10 methodology. I try to be as thorough as I can and as unbiased 11 as I can, and distill out what I think the truth is from the 12 information I have at hand. Sometimes you can't determine, but 13 other times you can. 14 Q. And did you look at things that -- for example, the EPA 15 and the CARC report. Do you know what the CARC report is in 16 the EPA? 17 A. Mm-hm. 18 Q. Did you look at those documents, some of which reached a 19 slightly different conclusion than you did? But did you read 20 those documents anyways? 21 A. Yes, I did. 22 Q. And did you put them on your reference list? 23 A. Yes. 24 Q. Okay. And you looked at -- in terms of evaluating the 25 evidence, you looked at adjusted-for-pesticides and RITZ - DIRECT / FORGIE 92 1 unadjusted-for-pesticides Odds Ratios. Is that correct? 2 A. That's correct. 3 Q. And do you -- do you look at the ones that are unadjusted? 4 And do you also look at Odds Ratios that are not statistically 5 significant? Do they form part of your evidentiary approach, 6 and part of the weight of the evidence that you use in reaching 7 your opinions and your conclusions? 8 A. In fact, this is really important, because we spend a lot 9 of energy, a lot of money, and a lot of effort on these 10 studies. Before we throw studies out and say we cannot trust 11 them, we really want to look at them in all ways. Right? 12 And to just throw a study out because it didn't reach 13 statistical significance is really the worst thing you can do, 14 because there might be a lot of good information in that study. 15 And, again, it's quantitative. My science is 16 quantitative. It's not qualitative. 17 You want to see how big that bias could have been. And as 18 much as we are worried about confounding, confounding works in 19 both ways. You can increase the relative risk by leaving out a 20 confounder, as much as you can decrease it. So confounding can 21 make you overestimate or underestimate, if you don't take it 22 into account. 23 And again, oftentimes there are different confounders. 24 One makes -- draws the estimate away from the null; the other 25 draws it to the null. On average, they don't change it. Okay? RITZ - DIRECT / FORGIE 93 1 So if you then leave both out of your model, you're 2 actually not making a mistake; but you need to kind of evaluate 3 the data in these different ways. There is not one-fits-all. 4 You just use the best you know how to use your methods to 5 evaluate how stable your results are, and how much you trust 6 them. And that's what I do with every single study I do. 7 Q. Okay. So you looked at -- in your evaluation you looked 8 at all of the epidemiology, including epidemiology that found 9 no statistically significant Odds Ratios; elevations such as 10 the Agricultural Health Study that we have discussed at length. 11 Is that correct? 12 A. Absolutely, because you could have ten studies that are 13 not statistically significant, because each one of them was too 14 small to exclude random error; but all ten put together tell a 15 story, and a very convincing story. 16 MS. FORGIE: Okay. And, Dr. Ritz, I know this is the 17 first time that you've ever testified. In fact, I think it's 18 the first time you've ever been in a courtroom, so I hope it 19 wasn't too bad an experience. I thank you very much. 20 And I pass the witness. 21 THE WITNESS: Thank you. 22 THE COURT: If I could just ask a couple quick 23 follow-up questions regarding your opinion -- your ultimate 24 opinion. 25 THE WITNESS: Yes. RITZ - DIRECT / FORGIE 94 1 THE COURT: You talk a little bit about toxicity, and 2 you talk a little bit about the mechanistic data. And I know 3 we'll hear from other experts in much more detail about that -- 4 THE WITNESS: Right. 5 THE COURT: -- but I take it your opinion is based on 6 the totality of the that evidence; not just -- 7 THE WITNESS: Right. 8 THE COURT: -- just the epidemiology. Is that right? 9 THE WITNESS: That's very right. Yes. 10 THE COURT: And if you didn't have the other stuff -- 11 if you only had the epidemiology -- 12 THE WITNESS: Mm-hm. 13 THE COURT: -- would you have the same opinion? 14 And if -- or would it -- would it not be enough 15 information to develop an opinion? 16 THE WITNESS: I would have a harder time. I cannot 17 unlearn what I know, and undo what I -- you know. So it's 18 really hard for me to answer whether, you know, the epi, alone, 19 would make me say this; but when I put it all together, it 20 absolutely made sense to me as a scientist. 21 And I really -- honestly, that's what I do every single 22 day. I have colleagues who do Seber fish models, and 23 C. elegans, and mouse models. And I go with my results with 24 pesticides back to them to the lab and say, Could you test this 25 substance? You know. I see this in epidemiology, but my RITZ - DIRECT / FORGIE 95 1 study's the only one. I don't want to say that this is really 2 true. So can you test this in the animals? 3 And then they do. And sometimes they find something. And 4 then we keep going. Right? 5 But this story has gone on for, like, 30, 40, years. 6 There's a lot of information out there. 7 THE COURT: And your opinion that Roundup® causes 8 NHL -- is it -- is it that Roundup® is currently causing NHL in 9 the exposure levels that human beings are experiencing today, 10 or is it that Roundup® is carcinogenic, and therefore it's 11 capable of causing NHL in the abstract, or somewhere in 12 between? 13 THE WITNESS: It's probably the second, because I 14 base my opinion on the farmer studies. And we know that 15 farmers are really at the front line. Right? They're the ones 16 who have the highest exposure. And that's what I'm basing my 17 opinion on, because that's the studies we have at hand; the 18 human studies that we have. 19 THE COURT: Okay. So is that to say, then, that your 20 opinion is not that it is currently causing NHL? It's that 21 it's capable of causing NHL? 22 THE WITNESS: Currently, it's -- yeah. It's capable 23 of causing NHL. 24 THE COURT: Okay. Great. Thank you. 25 THE WITNESS: Thank you. RITZ - CROSS / LASKER 96 1 MS. FORGIE: Your Honor, he's graciously letting me 2 come back for one second. 3 I want to make sure you understood the Judge's question. 4 And thank you, Eric. 5 Q. Are you saying that currently -- is it your opinion that 6 glyphosate-based formulations can and are causing non-Hodgkin's 7 lymphoma in the community today? 8 A. It depends on what "the community" is. 9 And it's a dose question. Right? We -- we know that the 10 toxicology is in the dose. So definitely it can cause NHL. 11 What the dose is, I wouldn't venture. 12 Q. So you'd have to look at the individual to determine that. 13 A. Right. Yes. 14 Q. But in other words, your opinion is that non-Hodgkin's 15 lymphoma can and is being caused by glyphosate-based 16 formulations. Correct? 17 A. As long as there are farmers who have been using it in the 18 way that we have studied it, definitely. 19 Q. Landscapers? 20 A. Landscapers, yes. 21 Q. Other people that have been exposed to it in some way? 22 A. Yes, yes, yes. 23 MS. FORGIE: Okay. Thank you. 24 Thank you, Your Honor. 25 RITZ - CROSS / LASKER 97 1 CROSS-EXAMINATION 2 BY MR. LASKER 3 Q. Dr. Ritz, I'd like to go back to your forest plot. 4 And if you could put up Slide 23. 5 We have the forest plot that you had used in your Expert 6 Report. And it's very similar to the one you presented this 7 morning, except you added Andreotti for (inaudible). Correct? 8 A. Correct. 9 (Reporter requests clarification.) 10 BY MR. LASKER 11 Q. Andreotti to the one she presented this morning. 12 And there are a number of studies. If we can put up the 13 next slide, Slide 24. I'm going to break this up. Since it's 14 a large table, I'm going to go with the NHL only, and then 15 we'll talk about the subtypes. 16 A. Mm-hm. 17 Q. There are a number of studies in your chart that are the 18 North American case-control studies. And all of those studies 19 were then pooled into the NAPP. Correct? 20 A. Yes. 21 Q. And an epidemiologists use various statistical measures -- 22 methods to pool data together, and adjust for the fact that 23 they're using different studies to come up with one final 24 analysis. Correct? 25 A. Yes, they using statistical analyses to do that. Yes. RITZ - CROSS / LASKER 98 1 Q. And in your Expert Report at -- at Slide 27, you also 2 talked about the fact that when you have pooled analyses -- 3 Slide 27. I'm sorry. 4 Since you have the NAPP, if you were to look at also those 5 sub studies that are pooled in, it would be a double counting, 6 because it's the same data. Correct? 7 A. What are you talking about? 8 Q. I'm sorry. The NAPP is analyzing the same data as in all 9 of those North American earlier case-control studies? 10 A. Are you now saying you're conducting a meta-analysis? 11 Q. No. 12 A. Of course, you shouldn't be putting all of the sub studies 13 in the NAPP. Right? 14 Q. No. I'm sorry. If we can go back to Slide 4, all of 15 these studies, which are highlighted in yellow -- those are the 16 North American case-control studies. All of that data was put 17 into the NAPP. Correct? 18 A. Yes. It's also represented by the NAPP. Correct. 19 Q. Right. And in your Expert Report -- I'm sorry I wasn't 20 clear -- in your Rebuttal Report, you talk about the fact that 21 the NAPP Study summarizes that previous data in those 22 highlighted studies. And it is included in a meta-analysis, 23 without excluding those earlier studies. That would be double 24 counting. Correct? 25 A. If we did that in a meta-analysis, yes. RITZ - CROSS / LASKER 99 1 Q. And for meta-analysis, the normal rule -- normal way you 2 handle that is you would use that later-pooled analysis, and 3 you'd remove those earlier studies. Correct? 4 A. You would decide which studies you want to use. Right? 5 Whether you want to use the NAPP, because it adjusted for other 6 pesticides, and actually it allowed you to adjust. And that's 7 the beauty of the pooled studies. 8 It's also why I'm showing all of them in the slide, 9 because you can see that individual studies may wiggle. Right? 10 There's uncertainty. The uncertainty's pretty big, because 11 those confidence intervals are wide. When you put them all 12 together, that wiggle disappears. And the signal becomes much 13 more consistent. And that's exactly what you're looking for. 14 Q. Okay. Well, then, let's look at -- let's take out of the 15 earlier studies we have the NAPP. That's pooled. And if you 16 can put up slide 28, we then have these remaining populations. 17 And Judge Chhabria then asked you a question. And I just 18 want to make sure it's clear about which of these remaining 19 data is adjusted for other pesticides. 20 And if I'm correct, what you explained was the De Roos 21 2005 study is adjusted for the pesticides. 22 But if you can put up the next -- if you can put up the 23 Slide 29 -- 24 BY MR. LASKER 25 Q. These studies -- the data that you presented is not RITZ - CROSS / LASKER 100 1 adjusted for other pesticides. Correct? 2 A. Right. And these are the smallest studies. And that's 3 why they couldn't adjust for the other pesticides, and they 4 would weigh very little in a meta-analysis. 5 Q. Okay. And if we can go to Slide 17, this is from your 6 Expert Report. You stated that -- and, actually, you can go to 7 the earlier slide, because we're highlighting Slide 16. I'm 8 sorry -- that the most highly adjusted estimates, also known -- 9 I'm sorry. What's happening here? 10 MR. KALAS: Sixteen. 11 BY MR. LASKER 12 Q. Yeah. The most likely adjusted estimates, also known as 13 "fully adjusted models," are the estimates that adjust for as 14 many confounding variables as possible, such as adjusting age, 15 sex, race, and also time of pesticide exposure. Correct? 16 A. That's correct. 17 Q. Next slide you state, This is relevant, because it gives 18 the reader confidence that the findings are most likely due to 19 glyphosate Roundup® exposure, instead of another potential 20 cause that acts as a confounder. Correct? 21 A. Correct. 22 Q. And so if we can go back to the slides we just had, which 23 has the highlighted versions -- I think it's Slide 29. Where 24 these investigators -- and I think you explained this to 25 Judge Chhabria. Where these investigators -- where they did RITZ - CROSS / LASKER 101 1 adjust for other pesticides, these Odds Ratios that you have 2 here all went down. Correct? 3 (Reporter requests clarification.) 4 A. All go down, because it's already at null. And for 5 Nordstrom and Hardell, I have to look it up, actually, but it 6 doesn't matter. They're not statistically significant. 7 And, yes, maybe they're not adjusted, but they would weigh 8 very little overall in a meta-analysis, because they are the 9 smallest studies with the least-exposed subjects. 10 What really matters are Eriksson, and the NAPP. 11 Q. Okay. And those are the top two. For Eriksson and NAPP, 12 you have the unadjusted data here. 13 When those two were adjusted for exposure to other 14 pesticides, both of those Odds Ratios went down. Correct? 15 A. Slightly down. Yes. 16 Q. And they were no longer statistically significant. 17 Correct? 18 A. The Eriksson wasn't; but so wasn't the arsenic, and so 19 wasn't MCPA, and every other pesticide. Right? And we 20 wouldn't say that, therefore, arsenic is not a carcinogen. 21 Q. And the NAPP was no longer statistically significant? 22 A. I don't know what you're referring to. You need to show 23 me. 24 Q. Okay. We'll get to that; but so if we can -- the only 25 adjusted number we have here, then, on Slide 30, then, would be RITZ - CROSS / LASKER 102 1 the De Roos Study. 2 And we also look at the second half of your table, and 3 that was Slide 31. And this is your subtype analysis? 4 A. Right. 5 Q. And every single one of these data points that you have on 6 your forest plot -- none of these are adjusted for exposures to 7 other pesticides. Correct? 8 A. I don't think they are. And that is -- is very logical, 9 because as soon as you venture into subanalyses, it becomes 10 really hard to adjust, because you already are reducing your 11 numbers to half or a quarter. Or you can see that Orsi only 12 had 50 follicular cancers. And then you had two people 13 glyphosate-exposed. 14 How do you then adjust? You're generating null cells. 15 Your model explodes. You can't do it. 16 And I did not show these to say, Oh, there's 17 confounding -- or not. 18 What I wanted to give a visual picture of is that I'm not 19 just talking about one singular subtype of lymphoma; that it's 20 actually a pattern for a lot of the different subtypes. And 21 the larger studies allowed you to look at the subtypes. And 22 it's nice data to look at. I mean, it gives you more 23 information, but it doesn't mean that I'm saying there is 24 absolutely no confounding. 25 Again, confounding is not a qualitative issue. It's a RITZ - CROSS / LASKER 103 1 quantitative issue. 2 Do we believe there's a very strong risk factor -- some 3 pesticide that increases NHL tenfold -- and I did not put in 4 here? That's the question. 5 Q. Well, in fact, for the NAPP -- and you do have some of 6 those numbers there -- the NAPP investigators adjusted for 7 pesticides. And they found that all of those Odds Ratios that 8 you have there went down. Some of them went below one. 9 Correct? For upwards estimate? 10 A. Which estimates are you talking about? 11 Q. For the ones for the NAPP that you have on your forest 12 plot. 13 A. The one under "unspecified," or the others? 14 Q. There are four different points you have here for the 15 NAPP. You presented the unadjusted Odds Ratios. 16 When those investigators actually adjusted it for three 17 pesticides -- and we'll talk about that later -- those Odds 18 Ratios all went down. They were not statistically significant. 19 Some of them were below one. Correct? 20 A. If I generate a lot of colinearity, as we have seen in 21 that table by Eriksson, that's what I expect to see. 22 That pattern doesn't mean that the adjusted are the 23 correct estimates. I'm just seeing what happens when I put 24 highly correlated indicators for exposure -- two or three or 25 four in the same model. And if I do that with small amounts of RITZ - CROSS / LASKER 104 1 strata -- small datasets, like subtypes, then I expect that to 2 happen. What that means is another question. 3 Q. And in Andreotti -- you didn't put this on your table this 4 morning, either. Andreotti also had subtype analyses? 5 A. Correct. 6 Q. None of those were statistically significant. And many of 7 them -- and I think most of them -- were below one for their 8 point estimate. Correct? 9 A. Yes. And, I mean, we don't have to discuss Andreotti in 10 terms of subtypes, because Andreotti showed no effect. 11 Q. So why would I then expect subtypes to show an effect? 12 Actually, there was one subtype that did show an effect -- 13 even a statistically significant effect -- but I don't believe 14 it. I don't believe that statistical significance when the 15 overall effect is even below one. 16 But what does that tell us? Right? 17 The overall effect in Andreotti is .83. Does that mean we 18 should now presume that 17 percent of all NHL can be prevented 19 by putting glyphosate into our cereal? I don't think so. I 20 think there is something really wrong with that estimate. And 21 what I think it is, is residual confounding, and having the 22 wrong comparison group. 23 Q. Okay. And just so the record is clear, the Andreotti 24 finding was not particularly a significant protective. It was 25 a null finding. Correct? RITZ - CROSS / LASKER 105 1 A. It was a null finding, but on the other side of the 2 null -- 3 Q. Okay. 4 A. -- consistently, as well. 5 Q. And -- thank you. 6 And in your deposition, I want to go back to some of the 7 testimony you gave towards the end of your direct about the 8 methodology that you use in analyzing studies. And if we can 9 put up Slide 3, I think. 10 Yeah. Find 3. Slide 3. No. Slide 4. I'm sorry. 11 Slide 4. 12 I asked you the question of what steps you would need to 13 go through in analyzing an epidemiological study before you 14 could reach a conclusion in your mind that the study 15 demonstrates a positive association between an exposure at 16 interest and outcome at interest. 17 And as I walk through the answer that you gave, which is 18 the next slide -- excuse me. Next slides -- some of the things 19 that you identified for me in your deposition that you would 20 need to look at before you could reach that conclusion of a 21 positive association. So you told me that an epidemiologist 22 would need to look at the study design. Correct? 23 A. Yes. 24 Q. And epidemiologist would need to look at exposure 25 assessment of validity. We talked about that today. Correct? RITZ - CROSS / LASKER 106 1 A. Correct. 2 Q. An epidemiologist looks at outcome assessment validity. 3 Correct? 4 A. Correct. 5 Q. An epidemiologist looks at sample size, exposure 6 prevalence. Correct? 7 A. Correct. 8 Q. An epidemiologist looks for any type of bias they can 9 think of. Correct? 10 A. Correct. 11 Q. An epidemiologist does a lot of Sensitivity Analyses. 12 Correct? 13 A. Yes. 14 Q. And then, taking all of that into account, an 15 epidemiologist must be able to convince herself, no matter how 16 you look at the data, that there is a signal. Correct? 17 A. Correct. 18 Q. Okay. So let's look at the specific studies that have 19 been conducted with respect to the glyphosate and non-Hodgkins 20 lymphoma. And I'd like to start for the Agricultural Health 21 Study. And the Agricultural Health Study, I think you 22 testified, is the one cohort study that has been conducted to 23 date on glyphosate-based herbicides, and on Hodgkin's lymphoma. 24 Correct? 25 A. Correct. RITZ - CROSS / LASKER 107 1 Q. And let me put up on the screen -- this is Slide 32. 2 Putting it up. Okay. Thank you. 3 This is a table that you use from your teaching materials 4 at UCLA? 5 A. Mm-hm. 6 Q. And this is what is generally presented as in the 7 literature as the ranking of study designs by validity. 8 Correct? 9 A. Incorrect. 10 This is one paper by a good friend of mine; name of Prince 11 Lee, in Environmental Health Perspectives, where he uses this 12 table to actually say, Well, this is how we think about 13 validity, but now let me challenge you. 14 Q. Okay. 15 A. And I used this slide at the beginning of my lecture to 16 challenge my students to think about ranking this, highest to 17 lowest, and tell them why this is all wrong. 18 Q. Okay. Well, let's put up, if you can go to Slide 33, 19 but -- because I asked you this question in your deposition. 20 A. Mm-hm. 21 Q. And I asked you if this is -- the last three lines of my 22 question, I take it -- what is generally presented in the 23 scientific literature as the ranking of study designs by 24 validity. 25 And your answer was "Correct." Right? RITZ - CROSS / LASKER 108 1 A. I don't understand this question. 2 Q. When I asked you about this specific table in your 3 deposition, and I asked you the same question I just asked you 4 today about whether or not this table is what is generally 5 presented in the scientific literature as the ranking of study 6 designs by validity -- 7 A. Yeah that. 8 Q. -- said in your deposition, that was correct? 9 A. This table was something that my friend made up. It's not 10 something that you find in a textbook. 11 But yes, the ranking is what is, among epidemiologists, 12 what is often used to judge studies. 13 And this is what I teach my epidemiology students: To 14 really question and correct. Please correct this incorrect way 15 of ranking studies. 16 Q. And just so it's clear, if we can go back to Slide 32, 17 which is your table, and we have randomized clinical trials. 18 And then we have prospective cohort studies, retrospective 19 cohort studies, nested case-control studies. And those are 20 case-control studies that are actually conducted within the 21 context of a cohort study. Correct? 22 A. Yes. They are nested -- no, not within a cohort study; 23 but nested within a source population you can identify. It 24 doesn't have to be a cohort study. 25 Q. And then we talked about the case-control studies in this RITZ - CROSS / LASKER 109 1 case. And I asked you where those would fit in. And in your 2 deposition testimony you said that they would be right under 3 nested case-control studies. I can show you the deposition 4 testimony, if you'd like. 5 But a non-nested, anyway, case-control study would be 6 right below nested case-control study in this table. Correct? 7 A. If I said that, I was wrong. 8 Q. Okay. 9 A. A case-control study that's population based is just the 10 same as a nested case-control study. So maybe I didn't make 11 myself clear, and I apologize. 12 Q. Okay. 13 A. What I'm showing in this ranking, and what I teach my 14 students -- and, you know, I can bring them here, if you want 15 to -- if they get it wrong and say that a cohort study is 16 better than a nested case-control study or a population-based 17 case-control study, they don't get the points on their exam, 18 because you have to do these studies correctly. You have to 19 assess exposure validly. You have to assess outcome validly. 20 You have to think about all of the potential biases. And there 21 are many cases where any simple case-control study, done right 22 and done well, is far superior to a cohort study. 23 Q. Okay. And if we can just go back to Slide 33 for a 24 second, with this issue of what is generally presented in the 25 scientific literature as a ranking of study designs by RITZ - CROSS / LASKER 110 1 validity -- I take it your testimony is: You disagree with 2 what is generally presented in the scientific literature on 3 this point? 4 A. Not in the scientific literature. That's why I didn't 5 understand your question. I'm really sorry. 6 What is generally represented among non-epidemiologists 7 about our science that uses these observational tools -- and if 8 that is considered science, then the science isn't science, 9 because they don't understand what our science really means, 10 and how we do this. 11 Q. The public health cohort study that's a study funded by 12 the National Cancer Institute and the National Institutes of 13 Health. Correct? 14 A. National Institute of Environmental Health Sciences. 15 Q. And the AHS cohort, to date, has -- 16 (Reporter requests clarification.) 17 BY MR. LASKER 18 Q. The Agricultural Health Study cohort has to date resulted 19 in over 250 scientific publications in the peer-reviewed 20 literature. Correct? 21 A. Correct. 22 MR. LASKER: And we will -- we have an Exhibit 517, 23 which is at Tab 9, which is a listing of all of those studies. 24 And we'll move that in, Your Honor. 25 Q. In 2005 the AHS published the De Roos Study of RITZ - CROSS / LASKER 111 1 agricultural health cohort. Correct? 2 A. Correct. 3 Q. And as you put in your chart, that was a study that had 4 null finding. I think the Odds Ratio was 1.1. Not 5 significant. Correct? 6 A. Correct. 7 Q. And in your initial Expert Report in this case, you 8 criticized that 2005 study for having too short of a follow-up 9 period. Correct? 10 A. That's not what I said. I said that if you just use the 11 baseline to the outcome, that would be a short follow-up 12 period. I didn't say that that was a latency period, or 13 whatever. 14 Q. Okay. Well, but in your Expert Report you did say that 15 the follow-up period for the Agricultural Health Study 2005 was 16 6.7 years -- 17 A. Relative -- 18 Q. -- considered short latency period in cancer epidemiology. 19 That's what you wrote? 20 A. Yes, if we would only go with the baseline -- 21 If all exposure would have happened at baseline, that's 22 correct. 23 Q. But in the Agricultural Health Study, in fact, the 24 exposure went back some 15 years. So you actually had that 25 20-or-more years of exposure history. Correct? RITZ - CROSS / LASKER 112 1 A. They did retrospective exposure assessment. Just like all 2 of the case-control studies, they relied on questioning the 3 subjects, and reporting self-reporting of these exposures, yes. 4 Q. And for the 2018 study in JNCI -- the Journal of the 5 National Cancer Institute -- by Andreotti, that includes an 6 additional 11 to 12 years for follow-up of NHL onset, which 7 means that we have over 30 years of potential exposure history 8 for that study. Correct? 9 A. That -- that's not correct, because you are now saying 10 that we actually know what happened in those 20 years. And, as 11 I tried to explain before, we really don't, because we only 12 have the baseline. And then at follow-up they asked about one 13 year. And they're extrapolating over 20 years from the one 14 year of use. 15 Q. But the data that they collected in that baseline 16 questionnaire in the mid 1990s goes back potentially to 1975, 17 and when glyphosate came on the market. 18 And then we have evidence of non-Hodgkin's lymphoma onset 19 up to about 2011 or 2012. So we had the entire period of -- 20 oh, I think that's 36 or 37 years. Correct? 21 A. We don't have the same data quality for 37 years. We have 22 one data quality at baseline, which is the one where the 23 subjects remember what they used. And then we have 20 years 24 ahead of us where glyphosate exponentially in use changed. 25 What we did not have: An annual exposure assessment, RITZ - CROSS / LASKER 113 1 where we are asking 63 percent of the enrollees, based on 2 enrollees, to recall. And we are only asking them to recall 3 one year. And then we are extrapolating over a 20-year period 4 what the exact dose was. Right? 5 Q. Okay. Dr. Ritz, let me ask you this question. In the 6 2005 De Roos Study, you also criticized that study as being too 7 small, in your Expert Report. Do you recall that? 8 A. I said it wouldn't have as much weight, because it was 9 smaller than most. In terms of the sample size -- I mean, the 10 case size -- it was smaller than most of the case-control 11 studies. Yes. 12 Q. And the 2018 JNCI study, just so we're clear, with 575 13 cases of non-Hodgkin's lymphoma, and well over, I think, 450 14 cases of non-Hodgkin's lymphoma with glyphosate exposure, had 15 more exposed NHL cases than all of the glyphosate case-control 16 studies combined. Correct? 17 A. Well, we have to qualify that. Statistical power -- 18 Q. Am I correct, though, in my question? 19 A. That is not the right question to ask. 20 Q. Well, that is the question that I asked, Dr. Ritz. So if 21 you can -- 22 THE COURT: Hold on a second. Hold on a second. 23 So you do need to answer the questions that he asks you. 24 And then if you need to say something to give it context, 25 you're perfectly free to do so. RITZ - CROSS / LASKER 114 1 THE WITNESS: Yeah. Thank you. 2 BY MR. LASKER 3 Q. So if I can just repeat the question, the 2018 JNCI study 4 includes more glyphosate-exposed NHL cases than all of the 5 glyphosate case-control studies to -- correct? 6 A. Correct. 7 (Reporter requests clarification.) 8 BY MR. LASKER 9 Q. The 2018 -- 10 THE COURT: Yeah. Mr. Lasker, you should try not to 11 -- 12 MR. LASKER: I've got to slow down. I know. I'm on 13 a clock. I'm sorry. 14 Q. The 2018 JNCI study includes more exposed NHL cases than 15 all of the glyphosate case-control studies combined. Correct? 16 A. Correct. 17 May I qualify that now? 18 THE COURT: Sure. 19 THE WITNESS: So what I'm trying to say is 20 statistical power is actually an interesting animal, because 21 you have the most power at 50 percent exposure. And that's why 22 we do clinical trials with half of the population giving -- 23 getting the treatment; half of the population getting the 24 placebo. That gives us the most statistical power. 25 When we go towards just about everybody exposed, or very RITZ - CROSS / LASKER 115 1 few people exposed, that's when we have very little power. And 2 so having the most-exposed cases is not necessarily a good 3 thing. 4 BY MR. LASKER 5 Q. You also talked about in your direct examination the issue 6 of nondifferential misclassification. And Judge Chhabria asked 7 you some follow-up questions about that. Do you recall? 8 A. Mm-hm. Yes. 9 Q. And as you explained, nondifferential exposure 10 misclassification biases are reported ratios -- Rate Ratios 11 towards the null value of the one. Correct? 12 A. Correct. 13 Q. And that's because then all of the data's actually random, 14 so you're not actually measuring the exposure you're interested 15 in. And assuming everything else in the study is the same, you 16 would get an Odds Ratio of 1.0. Correct? 17 A. About. 18 Q. So to the extent that you have any nondifferential 19 misclassification, it is moving whatever your true Rate Ratio 20 is closer towards that one null? 21 A. Yes. 22 Q. And if there is no association, in fact, in a study, 23 nondifferential exposure classification actually won't change 24 the Rate Ratio, at all? 25 A. That's correct. RITZ - CROSS / LASKER 116 1 Q. And in the 2018 JNCI study of non-Hodgkin's lymphoma, they 2 report a Rate Ratio that is actually below 1.0. Correct? 3 A. Right. And you just said that you wouldn't take that for 4 granted -- for the truth, because the confidence intervals 5 include the one. So we agreed that it was one. Right? That 6 there was no effect? 7 Q. Oh, we do agree there was no effect. 8 But my question is for you: With nondifferential 9 misclassification, what you are hypothesizing took place is 10 that there is a true association out there that's somewhere 11 above one, but through nondifferential misclassification, it 12 was moved down towards the one, and, in fact, below the one. 13 Correct? 14 A. No. Incorrect. 15 What moves this estimate below the one is a little bit of 16 randomness that we agreed on before; but mostly I think it's 17 confounding. It's additional confounding, because what 18 Andreotti did differently from De Roos is change the reference 19 group. 20 Anneclaire De Roos made specifically -- when she looked at 21 her dose-response analyses, she did not compare the 22 highest-exposed glyphosate users to the nonusers. She compared 23 them to the low-exposed group. 24 And the only reason you do that is because you believe 25 there's residual confounding you cannot adjust for that RITZ - CROSS / LASKER 117 1 invalidates the use of a no-exposure group. 2 And the example is if you have individuals who never drank 3 alcohol because of religious beliefs, and you want to assess 4 whether alcohol causes esophageal cancer, they are not the 5 right comparison group. 6 It's actually -- the right comparison group are people who 7 now and then drink alcohol, have no objection to drinking 8 alcohol, but drink very little. And then look at a 9 dose-response, because you can come up with many reasons why, 10 if you use the ones who would absolutely never drink would also 11 be different from the group of people who would agree that 12 drinking is okay, or you can drink. Right? 13 There are many, many confounders you probably have never 14 measured in those who would never drink alcohol, so that's the 15 wrong comparison group. 16 In the same way, I'm sure that Anneclaire used the 17 low-exposure group because she thought about: What was that, 18 that, you know, in her study, people actually said they never, 19 ever used glyphosate. There is something we have not captured 20 about these people. 21 And interestingly, in every -- 22 BY MR. LASKER 23 Q. Dr. Ritz, I'm sorry. This question's going on very long. 24 I'm not -- the answers are going very long. I'm not going to 25 get through my questions this way. If I could just -- RITZ - CROSS / LASKER 118 1 THE WITNESS: Keep going. Sorry. 2 BY MR. LASKER 3 Q. -- point you to specifically with respect to that point 4 that you just made about what the comparison group was. And 5 this is in Your Honors' briefs, so I won't go through this cite 6 right now. 7 But in your initial Expert Report you had some criticisms 8 of the 2005 studies, because they compared to that low-dose 9 group, as opposed to no exposures. 10 And now I take it, if I understand you correctly, you also 11 have criticisms for what the 2018 study, because they did 12 compare to that no-exposure group. So either way, they do the 13 analysis -- 14 A. You've got us epidemiologists. We always criticize. 15 Right? 16 Q. I think I got that. 17 A. Critique helps us actually make our arguments, and think 18 through, you know, what the truth might be behind all of this 19 data. And we argue about it. And, you know, we -- that's 20 exactly what we do. 21 So you're right. Both are correct, or incorrect. You can 22 have the wrong reference group when you're using the never 23 users of glyphosate; wrong in the sense that that may have 24 opened the door to residual confounding. 25 However, when you use the low-exposed group, you're wrong RITZ - CROSS / LASKER 119 1 in another way. Right? 2 You're never right, unfortunately, in my science. You're 3 wrong in different ways. It's the degree of being wrong, and 4 then still making sense out of that data. And that's what we 5 do every single day. 6 Q. Okay. Well, let me ask you about that. 7 I mean, if you can put up Slide 35. And this is -- I'm 8 sorry. Not Slide 35. Slide 63. It was Tab 35. Slide 63. 9 And this is a study by Dr. Blair. And we've heard about 10 him. He has a lot of studies in this litigation. And he's 11 talking about issues with confounding and exposure 12 misclassification. And I just want to walk you through, and 13 see you if you agree with what he states in this -- in this 14 publication. 15 If you could highlight the first sentence, please. 16 Dr. Blair states that, We worry, as epidemiologists -- 17 And he's an epidemiologist. 18 -- that many potential limitations in epidemiology, 19 particularly confounding and exposure misclassification, have 20 assumed an aura of actual limitations, where it is not 21 necessary to provide any evidence that the proposed limitation 22 is present. 23 Do you see that? 24 A. Yes. Yes. 25 Q. And he then states, if you'll highlight the next sentence, RITZ - CROSS / LASKER 120 1 that, Simply the mention of a possibility of a theoretical 2 limitation is often sufficient to discount the study findings. 3 Correct? 4 A. Correct. 5 Q. And then he talks about the fact that there is debate in 6 the field. And his final sentence here is, These are, of 7 course, the situations where we should demand data; not just 8 opinions. 9 A. Correct. 10 Q. And you agree with that? 11 A. I agree with that. 12 Q. Okay. Well, let's look, then, at the data that we have 13 from these studies. And you talked in your direct about 14 problems with the questionnaire -- 15 A. Mm-hm. 16 Q. -- and the answers to the questionnaire. And the AHS 17 investigators actually conducted a study that they published -- 18 A. Mm-hm. 19 Q. -- in which they had individuals -- 20 And if you can, put up Slide 65. And this is from that 21 publication by Dr. Blair in 2002. 22 -- in which they had individuals that took the question -- 23 took the questionnaire, or filled out the questionnaire? 24 A. Twice. 25 Q. And provided their exposure information? RITZ - CROSS / LASKER 121 1 A. Mm-hm. 2 Q. And then a year later they took it again. And they 3 compared the two questionnaires -- 4 A. Right. 5 Q. -- to find out whether or not the answers that those 6 pesticide applicators were giving with respect to their 7 pesticide exposures and specific exposures -- 8 They talk about glyphosate, as well. 9 -- were consistent between those two questionnaires. 10 And the conclusion that the investigators came to was that 11 levels of agreement regarding pesticide use in this population 12 is similar to that generally found for factors typically used 13 in epidemiologic studies such as tobacco use, and higher than 14 typically reported for diet, physical activity, and medical 15 conditions. Correct? 16 A. Correct. 17 Q. That's what they state. And, in fact, the AHS 18 investigators -- 19 And Dr. Blair's testified about this. He is part of that 20 study. 21 -- specifically chose pesticide applicators, because they 22 had information based upon their research that these 23 individuals actually would have greater recall. They work with 24 pesticides all of the time. They're farmers -- would have 25 better recall of pesticides than the general population. RITZ - CROSS / LASKER 122 1 Correct? 2 A. That's correct. 3 Q. Now, you also talked a bit in your direct -- 4 A. Can I can I qualify that? 5 Q. Well -- 6 THE COURT: Sure. 7 THE WITNESS: Because this is a really important 8 study. It's called "reliability." And this is what we do in 9 epidemiology when we don't have actual measures. Right? When 10 we don't have the radiation badge. We go and ask people twice. 11 It doesn't mean when we are asking people the same 12 questions twice within a year that they don't misrecall what 13 they used 15 years before twice. They probably -- they might 14 be quite consistent in reporting, but it doesn't mean that we 15 really captured the truth. It may just mean that one person 16 who has a bad memory forgets the pesticides, and the other one 17 reports them accurately. Right? 18 All we're seeing here is that, yes, if you do this twice 19 within a year, you get about the same answers. 20 BY MR. LASKER 21 Q. You also testified during the direct about the intensity 22 algorithm that the AHS investigators use. And this is when 23 they take into account personal protective equipment, and how 24 they applied the pesticide to try and figure out how much of 25 that pesticide actually gets into their system. Correct? RITZ - CROSS / LASKER 123 1 A. Correct. 2 Q. And you talked about -- 3 Well, first of all, with respect to the 2018 JNCI study, 4 they present two dose-response analyses, one of which uses that 5 intensity algorithm for the dose-response; the 6 intensity-weighted cumulative days? 7 A. Yes. 8 Q. And they also use another -- they measure two different 9 ways. They measure dose based upon cumulative days, where they 10 don't use intensity algorithm. Correct? 11 A. They use two ways. Yes. 12 Q. And whether they use the intensity algorithm or not, they 13 did not find evidence of a dose-response. Correct? 14 A. That's correct. 15 Q. With respect to the intensity algorithm, you mentioned 16 Dr. Acquavella's study. And I want to put up -- and this is 17 Slide 75. This is a table from Dr. Acquavella's study. 18 And what Dr. Acquavella did -- and you explained part of 19 this -- was that he gave the farmers questionnaires to get the 20 same sort of information about personal protective equipment, 21 what-have-you, and figured out their intensity score. And then 22 he ranked them, one to four, based upon their intensity score, 23 with the individuals with the higher numbers having less 24 protection, and therefore a higher intensity score. And that's 25 basically how the intensity algorithm works. Correct? RITZ - CROSS / LASKER 124 1 A. Correct. 2 Q. And then he looked to see what glyphosate levels were in 3 their urine. And what they found was for the individuals at 4 that higher -- with higher intensity score, less protective 5 equipment, they had more glyphosate in their urine than 6 individuals who had the lower intensity scores. Correct? 7 A. For the highest, they see a difference. The others, they 8 don't. 9 Q. And that's -- and that's what you would want from the 10 intensity algorithm. That's adding information that is not in 11 any of the case-control studies. Correct? 12 A. This is a study where you're having urine measurements. 13 And you're asking these questions within three days of 14 applications. You're not asking people to remember 20 years of 15 use. 16 Q. I understand that. I understand that. And -- 17 A. And they they're being observed by somebody from the 18 outside while they're doing this. 19 Q. Okay. Let's move on to this issue of multiple limitation. 20 You talked a little bit about how the AHS investigators dealt 21 with the issue of the nonresponders though this second 22 questionnaire; the 37 percent. 23 Now, before we talk about what they did to deal with 24 the -- those individuals for their second-phase questionnaire, 25 the investigators also conducted analyses in their study where RITZ - CROSS / LASKER 125 1 they did not use any of the data that they generated during the 2 imputation. Correct? 3 A. Yes. They did a subgroup analysis where they threw up 4 everybody who didn't respond. 5 Q. Okay. So there's -- let's look at both of those, so we 6 understand what the investigators did. If we could put up 7 Slide 77. 8 So the first Sensitivity Analysis that they conducted -- 9 and this was actually part of the study. It was not after the 10 fact -- was published in the publication, itself. Correct? 11 A. Correct. 12 Q. The first analysis they did was they said, We have 13 complete questionnaires responses from all 54,251 individuals 14 in our study, and we're going to use that data and all of the 15 exposure information they gave going back in time. And using 16 that data, without any imputated data, we're going to see if 17 there is an increased risk ratio for non-Hodgkin's lymphoma. 18 And when they did that analysis, they found that, again, 19 they found a null result. The Risk Ratio for their 20 highest-exposures group was 0.82, which is pretty much 21 consistent with their primary finding. 22 A. That's correct. And when you look at that, you see that 23 big green arrow. That's the whole, whole time period. And 24 you're kind of shrinking those years at the top. So from 1993 25 to 2013, which is quite a long time period, they're actually RITZ - CROSS / LASKER 126 1 excluding every exposure in their time period, pretending that 2 20 years of potential exposure doesn't matter for NHL risk. I 3 think that's a pretty strong assumption to make. 4 Q. And for that reason, the AHS investigators did a second 5 Sensitivity Analysis. Correct? 6 If you can put that next slide up. And I think this is 7 Slide 82. Right? 8 So the second analysis, to address your concern, they 9 said, Let's look at the almost 35,000 people who responded both 10 to Phase 1 and Phase 2. We have actual questionnaire 11 responses. We don't have to impute anything. And let's see 12 for those 35,000 -- 34,698, to be exact -- individuals, whether 13 there is an increased Rate Ratio. 14 And again they found, looking at their highest-exposure 15 group, they had a null result. There were no associations in 16 that group that answered both Phase 1 and Phase 2. Correct? 17 A. Actually, what you just said is correct. 18 However, they are not not imputing. They're just not 19 doing the formal imputation that they did for the missed -- 20 missing -- for the missing subjects with missing data. 21 What they're doing here -- they are actually imputing, 22 because for those 34,000 that they questioned the second time, 23 they have one year of exposure assessment between their 24 baseline and 2005. 25 And then they're using that one-year information to RITZ - CROSS / LASKER 127 1 impute -- to guess backward to whatever their baseline was, and 2 then to guess forward to the time they exited the cohort, which 3 is either when they get disease, or they are still healthy at 4 the end of 2013. So they're using for those 35,000 individuals 5 this guesstimation over a 20-year period that generates 6 nondifferential exposure misclassification à la carte. 7 Q. And because of that issue that you raised -- and I don't 8 have -- I don't think I have a slide up here for you, but it 9 was in the study, as well. One of the things that the 10 investigators did is they said, Let's not go out to 2012/2013. 11 Let's bring that back to 2005. We're only going to consider 12 cancer NHL cases if they were diagnosed as of 2005, so we don't 13 have that extra time period of exposure afterwards. 14 And again for that analysis, which was published in the 15 JNCI study -- part of their publication -- they found no 16 association. Correct? 17 A. They found an alteration of 1.04. And you see how it 18 moves when you get better exposure assessment. It moves above 19 the one slowly, but surely. 20 Q. And the 1.04 was nowhere near statistically significant? 21 A. Of course not, because they still have all of the baseline 22 exposure misclassification, and ten extra years of exposure 23 misclassification where they have one year with which they 24 impute anything between baseline and 2005. 25 Q. Now let's look now at the analyses in the JNCI study that RITZ - CROSS / LASKER 128 1 did use that method in the imputation. And if I understand 2 correctly from your earlier testimony this morning, because -- 3 Well, first of all, you talked about the fact that there 4 was an increase in glyphosate use when Roundup Ready® crops 5 came into being. Correct? 6 A. There was, yes, a huge increase when the GMOs were 7 introduced. Yes. 8 Q. And that increase -- 9 And I think you explained this to me in your deposition. 10 -- was primarily due to three crops -- soybeans, cotton, 11 and corn -- where there was a rapid -- 12 A. Corn. 13 Q. -- adoption of Roundup Ready® products. Correct? 14 A. Probably. 15 (Reporter requests clarification.) 16 MR. LASKER: Roundup Ready® products. 17 Q. And the -- I think you testified earlier today because of 18 that fact that there was such a quick adoption of 19 Roundup Ready® crops, for example, a soybean farmer, somebody 20 who was a soybean farmer at Phase 1, even if you didn't have 21 their questionnaire response at Phase 2, since they were using 22 Roundup Ready® soybeans, we would know at least for ever/never 23 use, that they were using glyphosate? 24 A. For ever/never you would know this, yes, probably, because 25 the farmer could have stopped farming -- right? -- and still be RITZ - CROSS / LASKER 129 1 in the cohort, or sold his farm. 2 Q. And there are also -- we talked about this in your 3 deposition. There are specific prescriptions as to you how you 4 use Roundup® when you're farming with Roundup Ready® crops. 5 They actually have detailed procedures as to how many times a 6 year and when during a year you would use Roundup®. Correct? 7 A. There are those prescriptions, yes. 8 Q. And so a soybean farmer, if they were Phase 1, whether 9 they responded in Phase 2 or not, you would have that 10 information to be able to incorporate into your analysis in 11 trying to figure out how often they used Roundup® on their 12 crops. Correct? 13 A. You can guess it. 14 Q. Okay. 15 A. However, that may depend on, you know, how many fields 16 they have; how many different crops they have, you know; 17 whether they help out their neighbor; whether they employ 18 somebody else to spray, and they don't spray at all, 19 themselves. Right? We don't know. 20 Q. Let's talk about the North American Pooled Project? 21 A. Mm-hm. Actually, can I make one more quick comment? 22 I was very surprised when I saw in Andreotti, in the first 23 table they presented, that the group of farmers that remained 24 in the never exposed to glyphosate was larger in Iowa than in 25 North Carolina. My guess would have been the opposite. What? RITZ - CROSS / LASKER 130 1 Why are there more people unexposed to glyphosate in Iowa, when 2 we saw the maps this morning? It makes no sense. 3 So something happened there that they just can't wrap 4 their mind around or their data around. 5 Q. Okay. They raised that concern anywhere in their 6 publication? 7 A. They wouldn't. Why should they? 8 Q. Well, let's talk, then, about the North American Pooled 9 Project. And as we discussed earlier, that is that pooled 10 project that put in all of those other earlier case-control 11 studies in the U.S. and in Canada. Correct? 12 A. Mm-hm. 13 Q. And you relied -- previously this morning you testified 14 about how that data was presented at scientific -- scientific 15 conference? 16 A. Mm-hm. 17 Q. And you have slide decks. So we have all of the analyses 18 that they conducted. Correct? 19 Now at the time of your initial Expert Report when you 20 first cited to the North American Pooled Project, and you cited 21 actually at that point to the unadjusted Odds Ratios that you 22 have on your forest plot -- when you first used that study, you 23 were not aware of that slide deck and that data. Correct? We 24 talked about that in your deposition. 25 A. That specific slide deck, I wasn't. RITZ - CROSS / LASKER 131 1 Q. And you also told me at your deposition that you actually 2 did not see that presentation at the conference. Correct? 3 A. Can you show me that? 4 Q. Well, we'll have to get back to that. I think it's in the 5 record. 6 A. Yeah. 7 Q. But do you recall actually having sat in now? 8 A. No, I don't. 9 Q. Okay. And the NAPP investigators -- as I think you 10 mentioned, when they made their presentation in Brazil, when 11 they provided adjusted Odds Ratios for glyphosate and 12 non-Hodgkin's lymphoma, and they adjusted it with respect to 13 just three pesticides: 2,4-D, dicamba, and malathion. 14 Correct? 15 A. Correct. 16 Q. And when they did that, they reported an Odds Ratio for 17 glyphosate and non-Hodgkin's lymphoma of 1.13, with a 18 confidence interval of 0.84 to 1.51, which was not 19 statistically significant. Correct? 20 A. Yes, but that is the ever/never. We're not talking about 21 the routine versus nonroutine users. 22 Q. And we're going to get to that, as well. 23 And when I asked you about this adjusted Odds Ratio for 24 NAPP, the 1.13, during your deposition, you stated that you 25 could not answer whether we should use the Odds Ratio that was RITZ - CROSS / LASKER 132 1 adjusted for 2,4-D, dicamba, and malathion, or whether we 2 should use the Odds Ratio that was not adjusted. Correct? 3 A. I'm sure I didn't say that. 4 Q. Okay. Well, let's put up Slide 13. And I asked you if 5 you had greater concern -- 6 Maybe I had the question a little wrong. 7 I asked you if you had greater concern for the validity of 8 the Odds Ratio that adjusts for 2,4-D, dicamba, and malathion, 9 than for the Odds Ratios that do not. And your answer was, 10 That is a question I cannot answer, because I don't know what 11 the results would be if we did this differently. Correct? 12 A. That is correct because, I like to see the data analyzed 13 in many different ways to form my opinion. 14 Q. Okay. And you agree that if 2,4-D, dicamba, and malathion 15 are associated with glyphosate use, and within that study -- 16 within the North American case-control studies -- they were 17 reported as an independent risk factor for non-Hodgkin's 18 lymphoma -- 19 A. Have they -- 20 Q. Let me just ask you a hypothetical question first. 21 If they meet those two criteria that you talked about -- 22 (Reporter requests clarification.) 23 THE COURT: Number one, you're talking too fast. 24 And, number two, the question was a little too long and 25 roundabout. RITZ - CROSS / LASKER 133 1 So let me press the "Reset" button, and try again, as best 2 you can, to slow down. 3 MR. LASKER: I'm sorry, Your Honor. 4 Q. You testified in direct about what is needed for there to 5 be confounding in an epidemiologic study. Correct? 6 A. That's correct. 7 Q. First you need to have an exposure that is associated with 8 the exposure you're studying. That's the first thing you need. 9 Correct? 10 A. Right. 11 Q. The second thing you need is that the exposure -- that 12 second exposure; the potential confounder -- has to be a risk 13 factor for non-Hodgkin's lymphoma within that study. Correct? 14 A. Correct. 15 Q. And the third issue that you raised is that that other 16 factor can't be on the pathway towards disease. 17 A. Correct. 18 Q. Which doesn't apply with respect to pesticides. Correct? 19 A. Probably not. 20 Q. Okay. 21 A. Yeah. 22 Q. So what we're looking for -- 23 A. Although you can always argue, you know, maybe two hits of 24 what you need. Right? 25 Q. So what we're looking for to determine whether or not the RITZ - CROSS / LASKER 134 1 NAPP investigators properly adjusted for 2,4-D, dicamba, and 2 malathion, is whether they meet those two criteria: They are 3 associated with glyphosate use, and they were associated in 4 that study population -- in North American case-control 5 population -- with non-Hodgkin's lymphoma. Correct? 6 A. That's actually incorrect. 7 What the criterion for confounding is, is that the risk 8 factor has to be associated with the disease in the source 9 population, in the unexposed. 10 Q. Okay. And you are aware, I take it, that early on in this 11 litigation when we got some discovery from Dr. Blair prior to 12 your first Expert Report, even, we obtained a draft manuscript 13 from the NAPP of their analysis of glyphosate and non-Hodgkin's 14 lymphoma. And you've seen that. Correct? 15 A. I've seen that. 16 Q. And in that draft manuscript, if we can put up Slide 115, 17 was the investigators explained why they chose to adjust for 18 those three pesticides: 2,4-D, dicamba, and malathion. And 19 what they explained was they looked for the pesticides that 20 were most strongly correlated with glyphosate use. Correct? 21 A. Correct. 22 Q. And they also looked for the pesticides that are most 23 strongly associated with non-Hodgkin's lymphoma in the previous 24 studies. And these are the case-controlled studies they're 25 referring to in North America and Canada with non-Hodgkin's RITZ - CROSS / LASKER 135 1 lymphoma. That's what they state. Correct? 2 A. They say -- what are you asking me? Sorry. 3 Q. They state that they were looking for the pesticides that 4 were significantly or strongly associated with non-Hodgkin's 5 lymphoma in previous studies that were evaluated as 6 confounders. Correct? 7 A. That's correct. However, in a cohort study you don't need 8 to do that. In the cohort study you actually have the 9 unexposed. And you can see whether they are associated with 10 the outcome. Did the AHS Study show that malathion, dicamba, 11 and -- what was it? -- 2,4-D are actually causes for NHL? 12 Q. Let me just make sure I'm clear on your testimony. You 13 explained during your direct that confounding is specific to 14 the study population. Right? 15 A. That's correct. 16 Q. We're talking about the NAPP right now, which is the 17 case-control study. Correct? 18 A. Correct. 19 Q. And they looked appropriately at that study population, 20 and identified the pesticides that were risk factors in that 21 population? 22 A. That's not what I said. 23 I said in a case-control study you cannot use the data 24 from the case-control study to establish risk for risk factors, 25 because you don't have the source population. Confounder has RITZ - CROSS / LASKER 136 1 to be a risk factor in the source population. You don't have 2 that in a case-control study, by definition. So you need prior 3 knowledge. You need to know what is a risk factor for the 4 outcome. 5 In a cohort study you have the source population. You can 6 actually evaluate whether the risk factor causes the outcome. 7 If it doesn't cause the outcome in your cohort, then it's not a 8 confounder. 9 So for the AHS, which is a cohort study, we can go to the 10 data, and we can look at the data and see whether dicamba, 11 2,4-D, or malathion are predictors of the outcome, NHL. If 12 they are not, I don't have to worry about them. 13 Q. Okay. Well, you actually agreed, and you testified in 14 your direct examination, at least, for 2,4-D and malathion, 15 that those are risk factors for NHL. Correct? 16 A. Those are risk factors according to IARC evaluations for 17 cancer; possibly NHL. So we would always be concerned with 18 them being potential confounders. 19 However, in a cohort you can actually assess whether or 20 not they would be confounders, with the tools I told you, 21 because being a risk factor doesn't mean you are a confounder. 22 Q. You also have to be associated with the exposure at issue? 23 A. Yes, but in a cohort study you can actually see that. 24 Is it a risk factor for the outcome? 25 In the AHS, malathion has no effect on NHL. RITZ - CROSS / LASKER 137 1 Q. I understand. 2 THE COURT: Are you saying -- 3 Could I ask a clarification question about that? 4 THE WITNESS: Yeah. 5 THE COURT: So are you saying in -- the AHS Study 6 showed no effect of these other pesticides? 7 THE WITNESS: On NHL. 8 THE COURT: On NHL. 9 THE WITNESS: Correct. 10 THE COURT: But you've just spent a bunch of time 11 testifying about all of the problems with the AHS Study. 12 THE WITNESS: Right. 13 THE COURT: So I would assume you would conclude 14 that -- you would opine that the AHS Study's conclusion about 15 those other pesticides is also highly suspect? 16 THE WITNESS: We would be cautious about that, 17 correct. Yes. 18 THE COURT: Okay. But so you are now using the 19 AHS Study's conclusions about those other pesticides, and 20 assuming they're correct, to criticize the NAPP Study? 21 THE WITNESS: No. 22 THE COURT: Do I understand that correctly? 23 THE WITNESS: No. Sorry. That was a 24 misunderstanding. No. 25 I'm saying that in a case-control study, you have to rely RITZ - CROSS / LASKER 138 1 on prior knowledge about whether a pesticide is actually 2 related to the outcome. I can't do that within the 3 case-control study. So if IARC tells me I should be worried 4 about malathion, I'm worried about malathion. 5 In a cohort study, I can actually test that. So if, in a 6 cohort study, malathion is not related to NHL, then maybe the 7 cohort study is wrong, or maybe the malathion exposure in that 8 cohort study wasn't high enough to cause NHL. 9 THE COURT: I think -- and it's possible that I was 10 misunderstanding the questions, but I think what Mr. Lasker was 11 asking you is -- I think what he was trying to get at is: Was 12 it a good idea for them to adjust -- in the -- in the NAPP 13 analysis, was it a good idea for them to adjust for these 14 possible confounders? 15 THE WITNESS: Yes. That -- 16 THE COURT: Or another way to put it is: Are we 17 concerned that these are confounders? 18 THE WITNESS: We are always concerned that these are 19 confounders, because we know from the literature that these 20 could be NHL-causing. 21 THE COURT: Okay. So I thought I understood you to 22 be saying, We shouldn't worry if they're NHL-causing, because 23 the AHS Study says that they're not. 24 THE WITNESS: No, no, no. In the AHS Study we 25 wouldn't worry about it, because there they don't cause NHL. RITZ - CROSS / LASKER 139 1 So they also wouldn't confound the AHS Study results. 2 In the NAPP study, I would recommend that we look at them, 3 and adjust for them, and see what happens. 4 THE COURT: Okay. 5 THE WITNESS: However, it's, again, a quantitative 6 issue. How strong of a confounder are they? Right? 7 And what happens if I truly have four agents that all 8 cause NHL, and I put them in the same model? I see a force of 9 the effect for all of them. That's what we saw in that table 10 that we saw this morning. Right? All of the effect 11 estimates -- they're "splitting the variance," we call that. 12 They are all explaining a little bit of the NHL risk, but that 13 doesn't mean they're not causes. 14 THE COURT: Okay. 15 BY MR. LASKER 16 Q. And just so we're clear, when the NAPP investigators 17 appropriately did this adjustment for the pesticides, they 18 found an Odds Ratio of 1.13 that was not statistically 19 significant association. Correct? 20 A. Again, this was under the exposure model that I didn't 21 like, where they are mixing in routine and nonroutine users. I 22 think things are very different when you look at the routine 23 users. 24 Q. Okay. And let's talk about that. 25 THE COURT: Before we talk about that, is now a good RITZ - CROSS / LASKER 140 1 time to take an afternoon break? 2 THE WITNESS: Oh, yes. 3 THE COURT: Yes? 4 THE WITNESS: Thank you. 5 THE COURT: Why don't we come back at about -- want 6 to come back in about 15 minutes? 7 (Recess taken from 2:04 p.m. until 2:23 p.m.) 8 MR. LASKER: Your Honor, I've been told that I have 9 to move in Exhibit 544, which was the Andreotti 2018 Study; 10 Exhibit 586, which was the Blair 2002 Study; Exhibit 510, the 11 Acquavella 2006 Study; Exhibit -- and these are Defense 12 Exhibits. 1278 is the slide deck with the data from the 13 North American Pooled Project. And Exhibit 1277 is the draft 14 manuscript from the North American Pooled Project. 15 THE COURT: All right. No objection, I take it? 16 MS. FORGIE: No objection. 17 THE COURT: Okay. Then they are admitted. 18 (Trial Exhibits 544, 586, 510, 1278, and 1277 received in evidence.) 19 BY MR. LASKER 20 Q. Dr. Ritz, we were talking. You'd mentioned before the 21 break the other analysis that was done with the greater than 22 two days per year, less than two days per year in the NAPP 23 data. Correct? 24 A. I think that's what we talked about. Can we maybe have a 25 slide for that? RITZ - CROSS / LASKER 141 1 Q. We will, but I just want to first ask you a few questions 2 about that analysis. That analysis of days per year comes from 3 the McDuffie Study. That's where they first presented that 4 dose analysis of greater than two days per year/less than two 5 days per year. Correct? 6 A. McDuffie presented that. Yes. 7 Q. And you would I agree that the intent of this analysis is 8 not to address dose-response. Correct? 9 A. The intent of that analysis is to get at something we 10 would consider dose-response. We can't always get a beautiful 11 dose-response, unless we measure exposures very, very well; but 12 what we usually can do is distinguish between high exposure and 13 low exposure, or between routine users and occasional users. 14 Q. Okay. If you could go to Slide -- 15 A. And that's what that gets us. 16 Q. If you could put Slide 125 up. And this is from your 17 deposition, page 265, 4-18. And I asked you about whether the 18 analysis that provided in Table 8 of less than or equal to two 19 days of exposure, versus greater than two days, in your 20 opinion, does that provide evidence of a dose-response to 21 glyphosate? 22 And your answer was, The intent of this analysis is not 23 dose-response. The intent of this analysis is to distinguish 24 between types of people who use and did not use glyphosate. 25 Correct? RITZ - CROSS / LASKER 142 1 A. Well, it's what I just tried to explain. We tried to get 2 at a dose-response, but we are not capable of it because we 3 cannot measure well enough. So the best they could do was say, 4 Can we distinguish between occasional users and routine users? 5 And that's what they did with this analysis. 6 Q. And you would agree that when we talk about cancer, we 7 really have to consider chronic exposures over long periods of 8 time. Correct? 9 A. Depends. 10 Q. Well. Let me just put up on Slide 126. This was your 11 testimony in your supplemental deposition, page 168, 16, to 12 169, 1. And we're talking at this point about the 13 biomonitoring studies. And we talked about those a little bit 14 earlier. And in discussing your use of the biomonitoring 15 studies, what you explained to me then in your deposition was 16 that, When we talk about cancer, we really have to consider 17 chronic exposures over a long period of time. Correct? 18 A. This is taken out of context. It does not refer to that 19 we always need chronic exposures over a long time. I would be 20 the last person to say that, because we know that from the 21 atomic bomb, that was a one-time event. In a very short time 22 period, we had a lot of cancers. 23 Q. But for the purposes of this case, when I was trying to 24 ask you questions about the biomonitoring studies, what you 25 explained to me in explaining why you didn't find those pieces RITZ - CROSS / LASKER 143 1 of evidence important in this case, your response then was not 2 qualified. You stated that, When we talk about cancer, we 3 really have to consider chronic exposures over a long period of 4 time. Correct? 5 A. We need to consider chronic exposures; we need to consider 6 long time; but we also need to consider intense periods. 7 That was not -- this is not a statement about, you know, 8 all of the possible ways that exposures might cause cancer. 9 Q. The NAPP investigators actually conducted further analysis 10 of their data, in which they also looked not only at days per 11 year, but duration of exposure. Correct? 12 A. Would you mind showing me? 13 Q. Yeah. And I, unfortunately, do not have the slide ready 14 to pull up, but if you look in your binder -- 15 MS. WAGSTAFF: What was the depo cite from what you 16 just did before? 17 MR. LASKER: 168, 16, to 169, 1. 18 BY MR. LASKER 19 Q. If you can look in your binder, do you have -- 20 MR. KALAS: (Indicating) 21 BY MR. LASKER 22 Q. I'm sorry, Dr. Ritz. I thought you had this. 23 MS. WAGSTAFF: If you had an extra copy of those 24 slides. 25 MR. LASKER: We gave you the -- RITZ - CROSS / LASKER 144 1 MR. KALAS: We gave them a binder. 2 MS. WAGSTAFF: We don't have a binder. 3 MR. KALAS: I handed them a binder. 4 MR. WISNER: Which deposition? 5 MR. LASKER: Right now we're just putting up the -- 6 MR. WISNER: I'm looking at that. 7 MR. KALAS: Summary. 8 THE COURT: You said it was the supplemental 9 deposition. 10 MR. LASKER: I'm sorry. Okay. 11 MS. WAGSTAFF: We don't have this. 12 MR. LASKER: I will give you, Dr. Ritz, my copy. And 13 I will give them the copy that -- and at the moment -- I take 14 it back. Before I do that -- 15 MR. KALAS: Copy of that. 16 MR. LASKER: Do you have another copy? 17 MS. WAGSTAFF: Could you tell us the exhibit you just 18 had then, where it was page 1, or 268 of the depo transcript? 19 MR. LASKER: The depo transcript, again, was the 20 supplemental deposition. 21 MS. WAGSTAFF: Is that one of these? 22 MR. LASKER: Her deposition is in there. I do have a 23 deposition in there. 24 MS. FORGIE: Thank you. 25 MR. LASKER: I'm going to hand this up to you, RITZ - CROSS / LASKER 145 1 Dr. Ritz. And we'll just continue here. 2 MS. FORGIE: What exhibit number are you at? 3 THE COURT: What tab in your binder? 4 MR. LASKER: It is Tab 20 in your binder. And I'll 5 hand you a copy of that. 6 THE WITNESS: Thank you. 7 BY MR. LASKER 8 Q. And unfortunately, this slide deck is not -- doesn't have 9 numbers on the different slides; but the third slide -- third 10 slide from the end of the slide deck, which has a title, "Proxy 11 versus Self-Respondents." So you go to the end, and you count 12 back three. You will see a slide that looks like this 13 (indicating). And do you have it? 14 A. Yes. 15 Q. Okay. And so we have here the days-per-year analysis. 16 And this is the NAPP. Their numbers go down when they do some 17 adjustments from what McDuffie had. That's those -- those 18 numbers in the red and blue. 19 But the NAPP investigators -- 20 And you talked about the fact that when you pool data, you 21 can do other analyses. 22 The NAPP investigators also were able to do analyses for 23 duration of use. Correct? 24 A. They have lifetime-days as number of years times days per 25 year. Mm-hm. RITZ - CROSS / LASKER 146 1 Q. And they also have the duration, and just number of years. 2 Correct? 3 A. Yes. 4 Q. And for duration and number of years for the individuals 5 in that study who used glyphosate for a longer period of time, 6 the Odds Ratios actually were lower for longer duration of use 7 for glyphosate. Now, that's 0.94 for proxies and 8 self-respondents; and 0.78 for self-respondents only. And both 9 of them, again, are not statistically significant. Correct? 10 A. That's correct. 11 Q. And when they did this lifetime-days analysis, which sort 12 of combines both the frequency in a given year and the duration 13 over time, they end up with in their highest in their 14 higher-exposure category of either a 1.08, which is not 15 statistically significant, or a 1.06; again, not statistically 16 significant. And those -- that's the bottom row on this table. 17 Correct? 18 A. That's correct. 19 Q. And just for the record so the Court understands, proxies 20 and self-respondents versus self-respondents only -- 21 Proxies are when you had another family member who was 22 giving the exposure information. And self-respondents is when 23 there was only the person who was exposed giving the 24 information. Correct? 25 A. That's correct. I understand that's what they did. RITZ - CROSS / LASKER 147 1 Q. Okay. 2 A. So -- 3 Q. And when they looked at the information for -- that they 4 got just from the farmers, themselves, we looked at -- again, 5 we have that 1.13 Odds Ratio that we talked about earlier. If 6 they looked at the data actually that was provided by the 7 farmers, only, the Odds Ratio that the NAPP investigators found 8 was 0.95, with a confidence interval of 0.69, from 1.32. And 9 that's that top row on the right. Correct? 10 A. Yes. 11 Q. Let's talk about the Eriksson Study. 12 A. May I make a comment? 13 Q. Well, I -- Your Honor? 14 THE COURT: Go ahead, if you want to make a brief 15 comment. The plaintiffs' lawyers will have an opportunity to 16 ask you clarification questions, as well. If you want to 17 briefly make a comment or an explanation, go ahead. 18 THE WITNESS: Right. So I just would like to say 19 that it's not uncommon to see that duration doesn't matter, but 20 intensity matters, or that intensity times duration matters. 21 And we do these analyses all of the time with different types 22 of exposures assessments. And it depends on the carcinogen 23 what is most important. 24 We know for silica it is not duration of exposure. It's 25 actually intensity of exposure, meaning: Are you overwhelming RITZ - CROSS / LASKER 148 1 the lung -- the lung cells with a high dose? It doesn't matter 2 that you do a low dose over a long time. 3 So this analysis was done in order to possibly distinguish 4 between long-time smaller use, smaller exposures, and 5 shorter-period intense exposures. 6 BY MR. LASKER 7 Q. Yes. And we talked earlier about the efforts that were 8 made, at least, in the Agricultural Health Study, to try to get 9 at an intensity. They have a whole intensity algorithm that 10 tries to address that? 11 A. Correct. 12 Q. Let's talk about the Eriksson Study. Now, the 13 Eriksson Study provides -- that's not a study that was focused 14 on glyphosate, alone. It provides data for a wide number of 15 different pesticides. Correct? 16 A. They ask for quite a number of pesticides specifically. 17 Q. Okay. And they have a couple of tables. And it's a 18 little complicated, because they do an ever/never analysis for 19 each pesticide. Correct? 20 A. Yes. 21 Q. And then they do an analysis that goes number of days, and 22 they break that out into fewer days, and more days. Correct? 23 A. They have a day analysis, as well. Yes. 24 Q. Okay. For the ever/never Odds Ratio, we can put up first 25 Slide 131. And this is Table 2 from the Eriksson Study. For RITZ - CROSS / LASKER 149 1 all of their ever/never analyses of every one of these 2 herbicides that they looked at, their Odds Ratio was above one. 3 And we have those all highlighted. Correct? 4 A. That's -- that's not what I'm looking at. I'm looking at 5 the per-day table -- or is it -- which one are you looking at? 6 Q. So this is -- they have -- the ever/never is the first 7 line. And then they have the days they break down underneath 8 it. Correct? 9 A. They have it broken down by days. Yes. 10 Q. What we've highlighted is the ever/never analysis? 11 A. Yes. 12 Q. Okay. And all of those findings for all of the herbicides 13 they look at were above one. Correct? 14 A. Yes. 15 Q. And if we go to the next table, this is the other 16 pesticides that they looked at. I think this covers all of the 17 different substances they looked at in this study. And again, 18 we find for every substance that they looked at in this study 19 for their ever/never analysis, they reported Odds Ratio of 20 above one. Correct? 21 A. Yes. 22 Q. And you agree that if you have all chemicals in a 23 case-control study that have an elevated Odds Ratio, one of the 24 things you'd be concerned about is the possibility of some 25 systemic bias and maybe recall bias. Correct? RITZ - CROSS / LASKER 150 1 A. Yes, but you can address that. Here we're actually 2 looking at the patterns of per-days. And you can very well 3 distinguish patterns that actually suggest a dose-response from 4 patterns that go in opposite directions. 5 So, for example, fungicides -- you can see that the 6 slightly elevated Odds Ratio -- is in the less than 37 days. 7 And that then it goes down. So you don't just evaluate 8 ever/never, and you don't just evaluate the estimate according 9 to, you know, is it above one, or not? You want to actually 10 see what happens. 11 Q. And just to be clear, we -- you talked about this in 12 direct examination. None of these Odds Ratios are adjusted for 13 exposures to other pesticides. Correct? 14 A. I don't recall, but I imagine that these are the 15 unadjusted ones. 16 Q. Okay. And we also talked about the fact that -- 17 A. Oh, unadjusted for other pesticides. They adjusted for 18 age. 19 Q. They adjusted for age, I think -- 20 A. And sex and -- 21 Q. -- and sex, and sight; those three things? 22 A. Yes. 23 Q. Okay. And we also talked about -- I think you stated that 24 one of the things they did in this study was not really a good 25 idea, because they used as their unexposed group individuals RITZ - CROSS / LASKER 151 1 that did not have exposure to any pesticides. Correct? 2 A. Yes. 3 Q. And for each of their exposure groups, even if they 4 identify, for example, the glyphosate group as exposed, we know 5 those individuals actually have exposure to multiple different 6 pesticides; not just glyphosate? 7 A. We don't really know that; but we presume that some of 8 these individuals have multiple exposures. 9 Q. Well, we do know for glyphosate, for example, with MCPA, 10 that there was a co-exposure between those two for those 11 pesticides. People who were exposed to glyphosate were also 12 exposed to MCPA in this study population? 13 A. That's what the authors explained. 14 Q. Okay. 15 A. Because one stopped being used; the other started being 16 used by the same individuals. Yes. 17 Q. So we have -- again, going back to that issue of 18 confounding, we have an association between those two 19 pesticides, as far as whether they're used together. 20 The second question -- 21 A. Actually, that's incorrect. 22 The way I said it was they used one in one period; stopped 23 using it; and used the other one. So that's not co-exposure; 24 that's subsequent exposure, which also tells you that the first 25 exposure is actually one that has a much longer latency period RITZ - CROSS / LASKER 152 1 to act than the second one. 2 Q. Okay. 3 A. So co-exposure is different. They would do that at the 4 same time. 5 Q. Okay. So you're correct. The MCPA exposure would have 6 been earlier in time; would have a longer latency. 7 A. Right. 8 Q. A longer opportunity for non-Hodgkin's lymphoma to respond 9 to being created by that MCPA exposure than glyphosate. 10 A. Probably. 11 Q. And in the -- in the analysis -- 12 And if we can bring up -- again, this is -- I should move 13 this into evidence. Your Honor, this is Defense Exhibit 877, 14 the Eriksson Study. I'll move it into evidence now. And if 15 you can bring up the -- 16 THE COURT: I assume there's no objection. I thought 17 we were going to do -- I thought you were going to get these 18 things into evidence; the stuff that you've already agreed on 19 afterwards. Eric. 20 MR. LASKER: Okay. However you want to handle it. I 21 just want to make sure it's in the record. I don't think 22 there's going to be objections on any of these. 23 THE COURT: If you just identify the exhibit you're 24 using, as long as there's agreement ON it, you can deal with 25 the mechanics of admitting it later, so that you don't waste RITZ - CROSS / LASKER 153 1 your time doing that. 2 MR. LASKER: I will hand you, since we somehow got 3 confused on our binders -- I apologize for that. This is the 4 Eriksson Study. And if you can -- what is the page of this? 5 MR. KALAS: 1,661. 6 MR. LASKER: Page 1,661. And this is Tab 26 in the 7 binders. 8 THE COURT: Twenty-six, you say? 9 MR. LASKER: Twenty-six. Yes. 10 BY MR. LASKER 11 Q. And what the Eriksson investigators -- they -- with 12 respect to MCPA in their study is that their study confirms 13 that the phenoxyacetic herbicides as a risk factor for NHL was 14 confirmed. And MCPA in particular yields the highest Odds 15 Ratio of those different pesticides. Correct? 16 A. That's what they state. Whether, you know, that is the 17 best way of evaluating this data isn't the question. 18 Q. Okay. But we know -- so if MCPA was used earlier in the 19 same population as those people who were exposed to glyphosate, 20 so it had a greater latency, a greater time period where it 21 could cause non-Hodgkin's lymphoma -- 22 And the investigators at least also state that in their 23 population: MCPA was a risk factor for non-Hodgkin's lymphoma. 24 Correct? 25 A. That is correct for the population that farmed for a very RITZ - CROSS / LASKER 154 1 long time. 2 For the population they included that came online when 3 only glyphosate was being used, it's not correct. And I'm sure 4 there are individuals who used just glyphosate. It's just that 5 this is a mixture -- 6 Q. Right. 7 A. -- of people, because that's real life. Right? Some have 8 farmed for a very long time; have multiple exposures 9 sequentially. Others start farming; start using. And we have 10 a mixture of both. 11 Q. Okay. And you talked earlier about this issue of greater 12 than ten years before diagnosis. And that analysis -- how they 13 did the cutoff based upon how many years prior to diagnosis the 14 individual had exposure. That's one of the things you talked 15 about in this study. Correct? 16 I'm sorry. It was an issue of sort of a lag in the 17 analysis. You looked at exposure within ten years of the NHL 18 diagnosis. And then you looked at exposures that were more 19 than ten years before the diagnosis. Correct? 20 A. In this paper? 21 Q. Yes. You testified about this during the direct 22 examination. 23 A. That was the Hardell. 24 Q. Well, if you look at page 1658 to 1659, I believe it was 25 this paper (indicating). RITZ - CROSS / LASKER 155 1 A. Sixteen? 2 Q. 1658 to 1659. And that's at the top of 1659 on the 3 left-hand column. There was the latency period of one to ten 4 years. I'm sorry. I'll wait for you to get there. 5 A. I don't see that here. 6 Q. If you're on page 1659 of the paper? 7 A. Yes. 8 Q. Okay. And you look at the right -- oh, I'm sorry. The 9 left-hand column. My mistake. The left-hand column. The top 10 of the left-hand column, talking about latency period of one to 11 ten years; that top line? 12 A. Yes, yes. 13 Q. And this is the period. And I think this is what you were 14 testifying during your direct examination; that latency period 15 of one to ten years where there was no cases of exposure for 16 MCPA or 2,4,5-T or 2,4-D. And during that period, while there 17 was glyphosate exposures, and they looked just at that time 18 period, there was an Odds Ratio of 1.11. And I think you 19 explained -- and it's reflected here -- the very wide 20 confidence interval. Correct? 21 A. That's correct. Yes. 22 Q. So that was what we were talking about -- you were talking 23 about during your direct? 24 A. Yes. 25 Q. Okay. And then the point that you made was with respect RITZ - CROSS / LASKER 156 1 to the latency period of greater than ten years. And you 2 talked about the fact that glyphosate had an elevated Odds 3 Ratio in that analysis for the greater-than-ten-year latency 4 period. Correct? 5 A. Yes, it does have one: 2.26. Yeah. 6 Q. And during that same time period of greater than ten 7 years, MCPA had an Odds Ratio of 2.81. Correct? 8 A. Yes. 9 Q. And we have already talked about the fact that that is now 10 a confounder. It was associated with glyphosate. People used 11 MCPA longer ago; had a greater period of time for MCPA to cause 12 a non-Hodgkin's lymphoma. And it was identified in this study 13 as a risk factor for non-Hodgkin's lymphoma. Correct? 14 A. It was identified as a risk factor. Whether they used it 15 for longer, we don't know. We know that they used it earlier. 16 Q. Right. So there was a longer period of latency for NHL to 17 develop. Correct? 18 A. Correct. 19 Q. And you also testified about the days per -- the days 20 analysis. The greater than ten days, and less than ten days 21 analysis in this study? 22 A. Right. 23 Q. And just so the record is clear -- and we'll hear some 24 more later about trends, and how you determine whether there's 25 differences in rates between different dose groups -- but in RITZ - CROSS / LASKER 157 1 this study, they did not do any analysis that showed any trend 2 that's statistically significant of a higher Odds Ratio with 3 greater than ten days of use as, compared to less than ten days 4 of use for glyphosate. Correct? 5 A. They're not showing you a p for trend, but my 6 guesstimation would be if you did a proper trend analysis, that 7 p-value would be statistically significant; but they don't 8 present it. Yes. 9 Q. And you have not calculated that. Correct? 10 A. I'm not calculating it because I can't. Yeah. 11 Q. And again, just to be clear, this data is not adjusted for 12 MCPA or any of the other exposures that these farmers may have 13 had. Correct? 14 A. Which data is not adjusted? 15 Q. The data on days of use. 16 And if you look at Table 2 on the bottom, in the footer, 17 the very last sentence, Adjustments were made for age, sex, and 18 year of diagnosis or enrollment. Correct? 19 A. Yes. 20 Q. They didn't adjust for any of the factors. Correct? 21 A. No. 22 Q. So if there was a family history of cancer, they also 23 didn't adjust for that. Correct? 24 A. Not in this analysis, no. 25 Q. And, in fact, they didn't in any of the analyses of this RITZ - CROSS / LASKER 158 1 study. Did they? 2 The only adjustments they made, other than that one 3 multivariable analysis -- 4 A. Why would you want to? 5 Q. Well, that's a separate question, I guess, that others 6 will address. 7 A. Is it linked to exposure that your father had cancer? 8 Would you not use glyphosate? 9 THE COURT: I think we need to kind of limit 10 ourselves to letting him to ask you the questions. 11 THE WITNESS: Okay. Sorry. 12 THE COURT: Although I have to say it would be very 13 enjoyable to watch have you up there. And -- 14 MR. LASKER: Interested in doing that next time, 15 Your Honor. 16 THE WITNESS: We'll do that next time, Your Honor. 17 THE COURT: Maybe, like, late Friday afternoon. 18 MR. LASKER: I'd welcome the opportunity, Your Honor. 19 Q. Let me just turn now to the issue of meta-analysis. And 20 you had discussed that also in your direct examination. Prior 21 to the 2018 JNCI study being published, and in your initial 22 Expert Report also prior to your being aware of the ADJUSTED 23 Odds Ratios in the NAPP, you stated that it was particularly 24 important to consider meta-analyses that summarize across the 25 smaller studies. Correct? RITZ - CROSS / LASKER 159 1 A. I'm not sure that I said meta-analysis, but some kind of 2 summary estimate, because the NAPP was actually a summary 3 estimate. It's a pooled -- it's a pooled estimate; not a 4 meta-analysis. 5 Q. And you -- actually, we'll put up another: Slide 167. 6 And this is from your initial Expert Report at page 15. You 7 stated, Because many of the smaller studies had suggested 8 findings, but wide confidence intervals, it is particularly 9 important to instead consider pooled and meta-analyses that 10 summarize across these smaller studies, and not only provide a 11 much larger sample side, but they allow us to assess NHL 12 subtypes with sufficient precision. Correct? 13 A. That's correct. 14 Q. And you also stated -- and this is in your Rebuttal Expert 15 Report at page 10. You talked earlier today about the issue of 16 biases, and whether in a meta-analysis you'd also want to be 17 worried about biases in the underlying studies. Do you recall 18 giving that testimony today? 19 (Reporter requests clarification.) 20 BY MR. LASKER 21 Q. Let me restate that. I'm sorry. 22 During your direct examination, do you recall Ms. Forgie 23 asked you, and was asking you questions about meta-analyses. 24 You mentioned that one of the things that you're concerned 25 about with a meta-analysis is biases in your underlying study? RITZ - CROSS / LASKER 160 1 A. That general, yes. 2 Q. And in your Rebuttal Report, if we put this up -- Slide 3 152. And again, this is prior to the 2018 JNCI study. And you 4 were responding to a point that one of defendants' experts -- 5 Monsanto's experts -- made about the possibility of bias being 6 issued from meta-analysis. 7 In your Rebuttal Report before the JNCI 2018 study was 8 published, you stated that, While there may be an issue of 9 biases, that's only going to be a problem for meta-analyses if 10 the biases go in the same direction, which is highly unlikely 11 in practice -- is you said that. Correct? 12 A. All of the biases are exactly the same. Yes. 13 Q. Now, with the 2018 study, which is a larger study, has a 14 lower Rate Ratio than the 2005 AHS Study that was previously 15 used in the meta-analysis, and also with the NAPP data that we 16 now have, we now have adjusted Odds Ratios. And when we would 17 do a meta-analysis, if we were to use the same methodology that 18 was used in those earlier meta-analyses, the meta-relative risk 19 for glyphosate and non-Hodgkin's lymphoma would be null. 20 Correct? 21 A. I wouldn't sate state that. 22 Q. There would be no positive association, at all? 23 A. Why would you say so? 24 Q. Have you done that calculation? 25 A. No. I wouldn't do it, because I don't believe that the RITZ - CROSS / LASKER 161 1 AHS belongs into a meta-analysis and a summary estimate. I 2 believe the AHS is a study that we need to evaluate for what it 3 is. 4 For glyphosate -- it's useless for glyphosate. 5 It's a beautiful study otherwise, for every other 6 pesticide in the world that didn't change in the way that 7 glyphosate changed; but it's useless to assess the effect of 8 glyphosate currently. 9 If we keep doing the AHS for another 40 years, then maybe 10 we will be able to, because all of those Iowa farmers are 11 exposed, and maybe we can make sense of that data. 12 Q. So I understand that your view of the 2018 study explained 13 that, but I am correct if everyone was, in fact, to use the 14 Rate Ratios for the 2018 Andreotti study, and they were to use 15 the NAPP data that we've talked about that you relied upon, and 16 where we have the adjusted Odds Ratio now with other 17 pesticides, that calculation that you do in a meta-analysis 18 would result in a null finding? 19 A. I don't agree with that. It would be cherry-picking what 20 estimates you're putting into that meta-analysis -- right? -- 21 because they are presenting in the NAPP a lot of different 22 kinds of estimates. 23 Are we putting in the 1.77 for more than two days per 24 year, or are we putting in the 1.13, where we're mixing, again, 25 the people who have no exposure, or little exposure, or RITZ - CROSS / LASKER 162 1 occasional exposure, with the ones that probably have routine 2 and higher-level exposure? 3 Q. Okay. Well, let's go -- 4 A. I would suggest we don't do that. 5 Q. Let's go to Slide 148, because we talked during your 6 deposition about the methodology that was used in the two 7 meta-analyses that you testified during the direct examination, 8 which is the Chang and Delzell meta-analysis, and the IARC 9 meta-analysis. And in their analyses, those were ever/never 10 analyses -- correct? -- for meta-analysis? 11 A. I believe they were. 12 Q. And when there was a subsequent study, they would use the 13 subsequent study and not the earlier study in the 14 meta-analysis? 15 A. That's how you do it. 16 Q. And when there was a pooling of data, they would use the 17 pooled analysis and not the earlier analyses. Correct? 18 A. If there is pooled data, they would use pooled data or 19 the -- 20 Q. And -- 21 A. -- or the original ones. It doesn't matter. It should 22 give you the same result, if you do it right. 23 Q. And in the Chang and Delzell and in the IARC study, they 24 also used the Odds Ratios that were adjusted for other 25 pesticide exposures. Correct? RITZ - CROSS / LASKER 163 1 A. When they had them, they used them. 2 Q. And they used the same study that we've been talking 3 about. It's the North American case-controlled studies in the 4 U.S. and Canada. They used all of those. They used the 5 Swedish case-control studies. They used the French 6 case-control studies, Orsi, and they used the agricultural 7 studies correct? 8 A. They used those. Yes. 9 Q. And if you were to use the exact same study populations, 10 using the pooled analysis and the most updated analysis -- 11 We're not cherry picking. We're using the exact same 12 studies. 13 -- and you were to use the Odds Ratios that we now for 14 those studies, and do a meta-analysis -- 15 A. You've created a summary estimate I would not believe in, 16 because it throws out all of my scientific knowledge about 17 these kind of studies. Right? Because the one rule is if 18 there's heterogeneity, and we actually assess heterogeneity 19 with a statistical tool -- if there is a heterogeneity where 20 all studies that are case controlled -- a lot of them that are 21 case controlled are on one side of the Odds Ratio, and one set 22 is on the other. I would venture to say there is 23 heterogeneity. The rule for meta analysis not to summarize 24 across heterogeneous studies; but to group them according to 25 their heterogeneity, and learn from it. RITZ - CROSS / LASKER 164 1 That's why we have that statistical tool. And that's 2 state of the art; state of the science. 3 Q. Dr. Ritz, let me just again ask my question. And if you 4 don't know the answer, that's fine. Our expert will be able to 5 present this data because she's done this analysis, but if you 6 were to use the Odds Ratios for the most recent studies and the 7 pooled analyses doing exact same methodology that was used by 8 the meta-analyses that prior to the 2018 NCI study you were 9 citing to, and stating that those were particularly 10 important -- if you were to do that calculation now, the 11 meta-relative risk would be a null finding. There would be no 12 association there. Correct? 13 A. I'm saying you're generating a useless summary estimate 14 that I wouldn't believe. 15 Q. You don't know what that number would be. Is that your 16 testimony? 17 A. What I'm saying is there is heterogeneity. And one of the 18 rules of meta-analysis scientific rules -- you can read it up 19 in every textbook on meta-analysis -- is to assess 20 heterogeneity across study and state it clearly. 21 And when there is a lot of heterogeneity, then you have to 22 address the heterogeneity and explain it. 23 And also, if you then still venture to do -- to create a 24 summary estimate, you have to actually explain what you're 25 doing, and why you're doing it. RITZ - REDIRECT / FORGIE 165 1 Q. You offered this testimony in -- for the first time in 2 this case in your supplemental deposition after the JNCI study 3 was published and after you were aware of the adjusted Odds 4 Ratios for the NAPP. Correct? 5 A. I'm not sure I know what you're referring to. 6 MR. LASKER: The record will stand for itself, 7 Your Honor. It's in the briefing that we've provided for you. 8 I have no further questions. 9 THE COURT: Any redirect from the plaintiffs? 10 MS. FORGIE: Thank you, Your Honor. 11 REDIRECT EXAMINATION 12 BY MS. FORGIE 13 Q. Just a few questions, Doctor. To be clear, it's your 14 opinion that glyphosate-based formulations can cause 15 non-Hodgkin's lymphoma. Is that correct? 16 A. That is correct. 17 Q. And again, to be clear, it is your opinion that as long as 18 a person is exposed to enough glyphosate-based formulations, it 19 could cause non-Hodgkin's lymphoma. Is that correct? 20 A. Yes. The toxin is in the dose. 21 Q. Right. And you have to -- in order to make that 22 determination, you would have to do that on a case-by-case 23 basis by examining that person's individual exposure to the 24 glyphosate-based formulation. Is that correct? 25 A. That I -- as an epidemiologist, I looked at groups of PROCEEDINGS 166 1 people with exposure. For any individual, it's a likelihood. 2 Right? It's a likelihood of having caused the disease. I 3 can't ever say for one individual whether or not that is the 4 only cause of the disease; but for exposed groups, yes, that's 5 what we do. 6 Q. Right. And you have to look at -- to determine whether an 7 individual -- a particular individual -- has had enough 8 glyphosate-based formulation to cause NHL, non-Hodgkin 9 lymphoma? 10 A. Correct. Yes. 11 Q. Wait. Let me just finish my question. I know it's been a 12 long day and we're all tired. 13 But you have to look at that individual's medical records 14 or deposition testimony or whatever you had, to determine their 15 individual exposure. Correct? 16 A. That's correct. 17 MS. FORGIE: Okay. 18 THE COURT: But could I ask a follow-up question 19 about that. 20 MS. FORGIE: Of course. It's your court. 21 THE COURT: I may be asking the same question in a 22 slightly different way, or maybe a different question -- I'm 23 not quite sure -- but is it your opinion that in the exposures 24 some people are getting right now -- farmers -- glyphosate is 25 causing -- has caused or is causing non-Hodgkin's lymphoma for PROCEEDINGS 167 1 them? 2 THE WITNESS: If the dose is high enough, yes. 3 THE COURT: Okay. And is it your opinion that people 4 currently getting high enough doses, such that glyphosate is 5 causing non-Hodgkin's lymphoma for them? 6 THE WITNESS: I guess that would depend on what we've 7 learned about exposing ourselves. Right? If we have learned 8 enough -- if the Ag Commissioners were -- and the outreach 9 people have done their job well, we should have trained farmers 10 to avoid exposure, but we would have to find that out; whether 11 they're actually following the instructions. Right? Whenever 12 you go into a company, whenever you go into a workplace, you 13 see a lot of instructions, but whether people follow them -- 14 ask the co-worker. 15 THE COURT: But let me ask it this way. Is it your 16 opinion that some of these studies that we have discussed -- 17 McDuffie, De Roos 2003, Eriksson -- show that glyphosate has 18 caused non-Hodgkin's lymphoma in people? 19 THE WITNESS: Yes, I think they did. 20 THE COURT: Okay. 21 MS. FORGIE: Thank you for clarifying that, 22 Your Honor. 23 I don't think -- no. Thank you very much. 24 Nothing further. Thank you, Dr. Ritz. 25 THE COURT: Anything further from -- PROCEEDINGS 168 1 MR. LASKER: No, Your Honor. 2 THE COURT: All right. Congratulations. 3 THE WITNESS: Thank you. 4 MS. FORGIE: First time in a courtroom. 5 THE COURT: We'll see you on Friday afternoon to 6 examine Mr. Lasker. 7 THE WITNESS: I'll do that. 8 (Witness excused.) 9 MS. WAGSTAFF: Your Honor, would you like to take a 10 break before our next witness? 11 MS. FORGIE: I would love to take a biological break. 12 THE COURT: Five-minute break? Sure. 13 (Recess taken from 3:02 p.m. until 3:07 p.m.) 14 THE COURT: All right. Are you ready to call your 15 next witness? 16 MS. FORGIE: Yes, Your Honor. Sorry. 17 Thank you, Your Honor. I call Dr. Weisenburger to the 18 stand, please. 19 MS. WAGSTAFF: Your Honor, may I approach? 20 THE COURT: Of course. You don't need to ask. 21 THE CLERK: Sir, please remain standing. Raise your 22 right hand. 23 DENNIS WEISENBURGER, 24 called as a witness for the Plaintiffs, having been duly sworn, 25 testified as follows: WEISENBURGER - DIRECT / FORGIE 169 1 THE WITNESS: Yes. 2 THE CLERK: Thank you. Please be seated. Go ahead 3 and adjust your microphone. And for the record, please state 4 your first and last name, and spell both of them. 5 THE WITNESS: My name is Dennis Weisenburger, 6 D-e-n-n-i-s, Weisenburger is W-e-i-s-e-n-b-u-r-g-e-r. 7 THE CLERK: Thank you. 8 DIRECT EXAMINATION 9 BY MS. FORGIE 10 Q. All right. Dr. Weisenburger, welcome. 11 Can you please -- let's see. We're going to put up a 12 slide for you. And I'd like you to very, very briefly discuss 13 your résumé, and what you do, please. 14 A. Yes. I'm a hematopathologist. That's a pathologist, a 15 physician and a pathologist who studies diseases of the immune 16 system and the blood and the bone marrow, which would include, 17 of course, non-Hodgkin's lymphoma, and I've spent my career 18 really specializing on this one disease. 19 I'm currently the Chairman of Pathology at City of Hope 20 National Medical Center, which is one of the comprehensive 21 cancer centers in this country; and I've had over four years' 22 experience studying this disease, the pathology of the disease, 23 the subtypes, the genetics, the epidemiology, looking for 24 causes or etiologies, and also the clinical features. 25 And as part of the epidemiology, I was the one who was the WEISENBURGER - DIRECT / FORGIE 170 1 principal investigator of the Nebraska study that you hear 2 about in De Roos 2003, and in the NAPP, for example. 3 And I was also a principal investigator of another study 4 of farmers looking at brain and lower esophagus and stomach 5 cancer. There was also a second large case-control study in 6 Nebraska of non-Hodgkin's lymphoma which I worked very closely 7 on with one of my epidemiologists. 8 And I've also spent a lot of years working with a group of 9 epidemiologists called InterLymph. These are epidemiologists 10 from around the world who are studying non-Hodgkin's lymphoma 11 and Hodgkin's disease, too. 12 I've published over 400 papers on non-Hodgkin's lymphoma 13 and the related disorders in peer-reviewed journals, and I've 14 published over 50 papers on the epidemiology and causes of 15 non-Hodgkin's lymphoma, including studies with pesticides. 16 Q. Okay. Thank you, doctor. 17 And now, could you please explain to us what non-Hodgkin's 18 lymphoma actually is, and how it works, and the disease process 19 itself, please? 20 A. Yeah. So non-Hodgkin's lymphoma is a cancer that arises 21 from lymphocytes, which are cells of the immune system; and 22 there are different kinds of cells. There are B-cells, so you 23 hear about B-cell lymphoma, there are T-cells, so you hear 24 about T-cell lymphoma; and there are also NK-cells, which are 25 rare, here but the do cause a lymphoma they call NK-cell WEISENBURGER - DIRECT / FORGIE 171 1 lymphoma. 2 So it's a cancer of the immune system, and the B-cells are 3 the cells that produce our antibodies that protect us from 4 infections, the T cells are cells that protect us from viral 5 infections, and cancer, and the NK-cells, as well. 6 And non-Hodgkin's lymphoma is relatively common. It's the 7 sixth most common cancer in males, and it's the seventh most 8 common cancer in females. So it's much less common than many 9 of the other types. There are about 70,000 cases -- new cases 10 of non-Hodgkin's lymphoma in the U.S. every year, and about 11 20,000 patients die every year from this disease. So it not an 12 inconsequential disease. 13 One of the hallmarks of cancer and one of hallmarks of 14 non-Hodgkin's lymphoma is that there are very characteristic 15 genetic lesions that occur in these cancers. Now, these are 16 not genetic lesions that we inherit, although that could 17 happen, but for the most part, these are genetic lesions that 18 we acquire during our life, and these include chromosomal 19 rearrangements or translocations, additions or deletions of 20 genes or segments of chromosomes, mutations in genes and other 21 epigenetic phenomenon that control the regulation of the genes; 22 and these kinds of genetic lesions are characteristic of 23 non-Hodgkin's lymphoma and many other cancers. 24 The causes of non-Hodgkin's lymphoma are quite varied. 25 They're seen in patients who have inherited immunodeficiencies WEISENBURGER - DIRECT / FORGIE 172 1 or acquired immunodeficiencies like AIDS, for example. 2 Individuals who have immune dysregulation, like people on 3 immunosuppressant drugs or people with autoimmune diseases. 4 And then there are a variety of infections, such as viral 5 agents, and there are also a variety of chemical exposure that 6 increase the risk for non-Hodgkin's lymphoma. 7 And as you've heard in the previous testimony, there are a 8 number of different subtypes of non-Hodgkin's lymphoma; and 9 these subtypes, as I'll show you, correspond to the different 10 stages of maturation of the lymphocytes from an immature 11 lymphocyte to a fully mature lymphocyte, but I want to 12 emphasize that all of these subtypes are part of non-Hodgkin's 13 lymphoma. 14 Next slide. 15 So this is kind of a complicated diagram, but I'll try to 16 make it clear for you. So if you look on the left-hand side, 17 you see the bone -- it looks like a bone -- and that's where 18 the lymphocytes arise. They come from a very immature cell 19 that provides a whole bunch of different kinds of blood cells 20 and one of them is lymphocytes. 21 And so the immature lymphocytes proliferate in the bone 22 marrow, and then they migrate out of the bone marrow into what 23 we call the peripheral lymphoid organs, like the lymph nodes 24 and the spleen, and for T cells, the thymus, where they mature. 25 And as they mature, they move through the various steps of WEISENBURGER - DIRECT / FORGIE 173 1 maturation. For example, they can go through that oval 2 structure, which is called the follicle, where they can mature 3 even further; and then eventually the B-cells come out, as the 4 pink cells on the right, as either plasma cells, which are the 5 antibody-producing cells, or the memory B-cells, the cells that 6 remember what you were exposed to, and when you're exposed 7 again, they proliferate and go back. 8 And so as you look across the bottom, you can see that the 9 names of the different lymphomas are listed there, according to 10 the stage of maturation or differentiation of the B-cells. 11 So in summary, the B-cell lymphomas are all part of the 12 same disease, non-Hodgkin's lymphoma. There's a different 13 diagram for T-cells and NK-cells, which I'm not going to show 14 you, but I wanted just to show you this to give you the idea of 15 how the terminology works. 16 Next slide. 17 So I want to just take a moment to talk about -- 18 JUDGE PETROU: Before you move on to that -- 19 THE WITNESS: Sure. 20 JUDGE PETROU: -- just a couple other preliminary 21 questions. Can you tell us what is known about the latency 22 period for NHL? 23 THE WITNESS: Yeah, yes. So latency -- latency is 24 defined from the time that you have the first exposure to some 25 agent -- WEISENBURGER - DIRECT / FORGIE 174 1 JUDGE PETROU: Mm-hm. 2 THE WITNESS: -- to the time that you actually get 3 the disease. Okay? That's how we define latency. And it 4 probably differs for different agents. 5 So, for example, if you -- so for example, say if you had 6 breast cancer, and you got chemotherapy. You would be at an 7 increased risk for NHL, and that NHL would probably occur 8 fairly soon after the chemo, because chemo is very powerful 9 drugs that would cause DNA damage, and so the disease would 10 probably develop within the first five or ten years, okay? 11 On the other hand, there's a body of evidence about 12 solvents, for example, in the workplace, and with this exposure 13 to mixed solvents, the latency period is much longer. It's 14 probably in the range of 20, to 25 years. 15 So it really depends on the kind of agents that you're 16 exposed to, how potent they are -- 17 JUDGE PETROU: And the level of exposure. 18 THE WITNESS: -- how intense your exposure is, and 19 then that will really determine the latency. 20 So we don't really know for glyphosate what the latency 21 period is. We do know from the Eriksson Study that you had to 22 be exposed -- you had to have -- you had to follow the patients 23 for at least 10 years after their exposure to begin to see 24 cases. But that's about all we know about glyphosate. 25 MS. FORGIE: Does that answer...? WEISENBURGER - DIRECT / FORGIE 175 1 JUDGE PETROU: Mm-hm. 2 BY MS. FORGIE: 3 Q. Please continue, back to the next slide on methodology, 4 Doctor. 5 A. Sure. So I wanted to just talk briefly about my 6 methodology. 7 So in my analysis of this body of information and 8 knowledge about glyphosate, I used the same scientific method 9 that I -- and the same intellectual rigor that I use in my 10 daily academic practice. I didn't do anything different in 11 this analysis than I would do in another analysis as part of my 12 work. 13 I reviewed a whole variety of reports on the subject: 14 Reports from IARC from EPA, from the EFSA as well as other 15 reports. I read the industry-sponsored reports, such as the 16 ones in the Critical Reviews in Toxicology in 2016, and I 17 reviewed a whole variety of other reviews and commentaries. 18 I read a lot of -- I pooled and read a lot of the primary 19 articles from those reports, and then I did my own literature 20 searches, two or three or four times over the course of a year 21 and a half, to find any new papers or any papers I might have 22 missed, and I read those. 23 And then I will tried to synthesize all of this 24 information, and -- and weigh it, as a whole; and then 25 I applied the Bradford-Hill criteria for this evaluation of WEISENBURGER - DIRECT / FORGIE 176 1 general causation. 2 So I did a careful analysis of all the information, and 3 tried to come to the best conclusion of what I thought was 4 truth. 5 Q. Doctor, could I go back for one minute, back to the normal 6 B-cell maturation in relationship slide? 7 A. Okay. 8 Q. Okay. Looking at that slide, is there any way to 9 determine when a particular type of NHL develops in that slide, 10 or -- 11 A. No. The slide just shows how the cells mature from the 12 very immature new B-cell to the very mature B-cell that is 13 ready to go to work. 14 Q. Okay, but is it fair or accurate to say that the subtype 15 fits within here, within your diagram, depending on the level 16 of maturation of the cell? 17 A. So -- so what happens is you acquire these genetic lesions 18 over time. Some of them occur in the bone marrow. Some of 19 them occur later, in the follicular center, and then the cells 20 are arrested. They can't differentiate any further, so they're 21 arrested, and they just proliferate, and that's what becomes 22 the cancer. 23 Q. Okay. Thank you. 24 Go back to the other slide, that glyphosate-based 25 formulation, is that where we're at? WEISENBURGER - DIRECT / FORGIE 177 1 A. Right. So as you've heard from Dr. Ritz, glyphosate is 2 really the most heavily used herbicide in a variety of products 3 today, as glyphosate-based formulations; and as she told you, 4 there was a dramatic increase in the use of these chemicals 5 after the introduction of glyphosate-resistant crops in about 6 1996 and '97, which is critical when we talk later about the 7 Agricultural Health Study. 8 There have been studies of farmers, and about 60 percent 9 of farmers had glyphosate in their urine on the day of 10 application. It's in usually in parts per billion, but 11 depending on what kind of precautions they take, it can be 12 higher or lower. 13 So it shows that farmers are exposed to glyphosate, that 14 they do get a dose, that's an internal dose, of this chemical. 15 And then there are some papers that show that -- that look 16 at the cells in vitro, different kinds of cells in vitro, and 17 they've shown that there are actually toxic effects to the 18 cells in vitro at very low doses, doses below the regulatory 19 limits, so in levels of parts per million or parts per billion, 20 like we found in the urine of the farmers, or even in parts per 21 trillion in some. So even very low doses can have physiologic 22 effects on cells. 23 MS. FORGIE: Okay, and I should have mentioned -- I'm 24 sorry, I should have mentioned that the Exhibit number for 25 Doctor Weisenburger's slides is 300. Sorry. WEISENBURGER - DIRECT / FORGIE 178 1 THE WITNESS: So -- 2 BY MS. FORGIE: 3 Q. Okay, are you finished with that slide? 4 A. Yes. 5 Q. Okay. Thank you. 6 A. So I don't want to belabor the case-control studies, so 7 I'm going to give you kind of a high-powered view of the 8 studies; and if you want to talk in more details about any one 9 of them, I'm happy to, but there are basically six case-control 10 studies that were done in evaluation of pesticides, including 11 glyphosate. 12 And if you'll just focus on the risk estimates -- I'm not 13 sure you can -- can you just highlight the risk estimates? 14 Right. 15 So if you'd just focus on the risk estimates, what you can 16 see is that in five of the six studies, there were increased 17 Odds Ratios between 2 and 3, or a little more. 18 And in the one study that was negative, which was Orsi 19 number 5, it was a study with not very many cases, so it had 20 very weak statistical power. Four of the five positive studies 21 actually have statistically significant increases, of twofold 22 or more, and there are -- and of these, three of them were 23 adjusted for the use of other pesticides. 24 So you'll see the first one is Hardell, which is actually 25 the second box down, where, when the Risk Ratio was adjusted WEISENBURGER - DIRECT / FORGIE 179 1 for use of other pesticides, the Odds Ratio went from 3.04 to 2 1.85. So it was still elevated. 3 De Roos you've talked about -- 4 Q. Doctor, let me just stop you for one second, please. 5 On the Comments section, on the right of this slide, do 6 you see that? 7 A. Right. 8 Q. Where you have the stars? And it indicates that some of 9 these Odds Ratios are adjusted for significant medical 10 variables, and some are adjusted for other pesticides. 11 Do you see that? 12 A. Right. 13 Q. Okay. So in -- according to this slide, Hardell, De Roos 14 and Eriksson which are all bolded, had been adjusted for other 15 pesticides, is that correct? 16 A. That's correct. So the second -- the second one is 17 De Roos, where they did some very complex adjustments. 18 And even after all of these adjustments, the risk was over 19 twofold, with a pretty tight confidence interval. 20 Q. So to be clear, in those three studies, adjusting for 21 other pesticide use, you still have elevated Odds Ratios that 22 are statistically significant, over 2? 23 A. Well, only the De Roos was statistically significant. The 24 Hardell and the Eriksson, the Odds Ratio decreased, but it 25 didn't go to 1. It was still elevated. WEISENBURGER - DIRECT / FORGIE 180 1 Q. That's right. Thank you for the correction. Thank you, 2 Doctor. Okay, go ahead. 3 A. And then -- and then the last point I want to make on this 4 table is there were two studies that look at what we would call 5 a dose-response kind of evaluation. 6 That's the first one, McDuffie, where they looked at 7 exposure less than or equal to two days per year, versus 8 greater than two days per year. 9 And you can see in the risk estimates, for greater than 10 two days per year, the Odds Ratio was over 2, and it was 11 statistically significant. 12 Similarly, in Eriksson, they looked at exposure less than 13 10 days versus greater than 10 days, and again, they saw an 14 increase of -- over twofold increase, that's statistically 15 significant. 16 So at least two of the studies that looked at 17 dose-response actually found a dose-response. 18 JUDGE PETROU: Doctor, do you know or are you able to 19 give us any insight as to how these dates for fewer or more 20 than 2 or fewer or more than 10 were selected? 21 THE WITNESS: Well, they would, on each of the 22 applicators they would ask them either for each pesticide, or 23 usually for each pesticide they would say, well, say for 24 glyphosate, how often on average did you use it? And a farmer 25 would say, "Well, I used it three days per year, for X number WEISENBURGER - DIRECT / FORGIE 181 1 of years." So it was just three days per year, or 10 days per 2 year. It would have gone into the greater than two days per 3 year category. 4 JUDGE PETROU: No, I understand how you categorize 5 it. My question is, how do you pick two, for example, or how 6 you pick 10 as the cutoff point? 7 THE WITNESS: What some of them did -- and I don't 8 remember exactly what they did in these studies -- what some of 9 them did was they used the median exposure in the controls. So 10 they took an arbitrary number and just used it based on the 11 controls median, yeah. 12 And in Eriksson, they just counted the number of total 13 days. Yeah. 14 MS. FORGIE: Okay. 15 THE COURT: But do you know if they asked for, in 16 Eriksson, for example, do you know if they asked, you know, 17 were you exposed less than 10 days, or more than 10 days, or 18 did they ask it open-ended? 19 THE WITNESS: I think the way they would have asked 20 it would have been open-ended: On average, how many days per 21 year were you exposed to this pesticide? And the farmer 22 would -- 23 THE COURT: Or how many days total, in the case of 24 Eriksson, was it how many days total or was it how many days 25 per year? WEISENBURGER - DIRECT / FORGIE 182 1 THE WITNESS: Well I don't know what the questioner 2 said. The McDuffie one would have asked how many days per year 3 on average. 4 Eriksson, they probably would have asked, how many days 5 did you use it, or they might have taken the number of days per 6 year and multiplied it times the number of years and come up 7 with a total number of days, cumulative days. 8 I don't know the questionnaire, but that's how they would 9 arrive at that kind of data. 10 THE COURT: But your understanding is they would have 11 asked that in an open-ended way? 12 THE WITNESS: Yes. 13 MS. FORGIE: Would you like him to take the time to 14 pull out the study and look at it? 15 THE COURT: That's not necessary. 16 MS. FORGIE: Okay. 17 Q. So please continue. 18 A. So based on this data, I think there is good data to 19 conclude that exposure to glyphosate increases the risk for 20 non-Hodgkin's lymphoma. 21 And I mean, one of the criticisms of the case-control 22 studies was the possibility of -- what's the term -- bias, 23 recall bias. Recall bias. Because that's something that you 24 would -- you could see in case-control studies, versus cohort 25 studies, because the cases have the disease. WEISENBURGER - DIRECT / FORGIE 183 1 But there have been studies, for example, in our study in 2 Nebraska, Aaron Blair looked at the frequency that farmers 3 recalled certain pesticides, and they compared them -- patients 4 with disease recalled certain pesticides versus the controls 5 who didn't have the disease, and they didn't find any 6 difference. The responses were the same. The same number, the 7 same frequency. 8 So they didn't find any evidence of recall bias at least 9 in the Nebraska study. So I don't think recall bias is an 10 explanation for -- for these findings. 11 And if recall bias really was important, we would have 12 found it in the other case-control studies looking at myeloma, 13 leukemia, other solid tumors, Hodgkin's disease, and we didn't 14 find it in any of those other studies. So I don't think recall 15 bias is really an issue here. 16 So that's kind of all I want to say about the case-control 17 studies, unless you have other questions for me. 18 Q. No, that's -- 19 A. They were discussed quite at length by Dr. Ritz. 20 Q. Right. No, I think we'll continue, unless your Honors 21 have other questions on the case-control studies. 22 A. So, and then I'm going to share just a few pieces of 23 information that I think are important that sort of illustrate 24 the points I'm trying to make. 25 So this is data from the North American Pooled Project or WEISENBURGER - DIRECT / FORGIE 184 1 the NAPP project. 2 And this is some of the data that was presented at a 3 national meeting in Canada in 2015. 4 Q. Doctor, let me interrupt you for one second. Just to be 5 very clear, you're only getting -- this is an ongoing study and 6 there's some confidential information, and I want to make sure 7 that you're not putting out any confidential; everything you 8 give is public information. 9 A. This is public information that -- 10 Q. Thank you. 11 A. -- has been made available to everyone. 12 Q. Okay. 13 A. And so what this shows is, on the right-hand -- on the 14 left-hand side, days per year that the glyphosate was handled, 15 and you can see that the zero would be unexposed; and then 16 the -- greater than zero but less than equal to 2 would be the 17 days per year handled which would be sort of the infrequent 18 handlers; and then greater than two days per year would be the 19 more -- as Ritz would call it, the more -- the common users, 20 okay, the frequent users. 21 And then across the top, you've got data for all 22 non-Hodgkin's lymphoma. So let's look at that column first, 23 and what you can see is that, of course, by definition, the 24 Risk Ratio for no exposure is 1. The Odds Ratio for less than 25 or equal to a day's exposure is about equal to 1. WEISENBURGER - DIRECT / FORGIE 185 1 So there was no real increased risk for infrequent 2 exposure, but the risk for NHL increased almost twofold when 3 you looked at greater than two days per year; and the 4 95 percent confidence intervals are tight and statistically 5 significant, and the trend analysis positive, at 0.02. So 6 that's for all NHL. 7 And then if you look at the subtypes, the common subtypes, 8 follicular lymphoma is the next column, FL. DLBCL is diffuse 9 large follicular lymphoma. 10 Q. So let's just try to go, for the sake of the court 11 reporter and me, let's trying to go just a little bit slower. 12 A. Okay. 13 Q. I know you know this stuff, but a lot of the terms are new 14 for us. So thank you. 15 Let's go back to -- 16 A. Are we caught up? 17 Q. Let's go back to where we were on all NHL, all 18 non-Hodgkin's lymphoma, and let's start with that column all 19 over again, please. Will that work for you? Perfect. 20 A. Sure. So for all non-Hodgkin's lymphoma, you can see that 21 the Odds Ratio for no exposure is 1, by definition. Those with 22 the less than or equal to two days per year exposure had an 23 Odds Ratio of 0.83, which is slightly below 1. 24 Q. Okay. Again, I can see, let's go a little slower please. 25 I know it's the end of the day but, for the court reporter. WEISENBURGER - DIRECT / FORGIE 186 1 Thank you, Doctor. 2 A. Whereas greater than two days per year, the Odds Ratio is 3 1.98, nearly 2, with significant confidence intervals that 4 exclude 1, and a trend analysis, that B trend analysis, which 5 is statistically significant. 6 Q. Okay. 7 A. Basically, you have the same findings for diffuse large 8 B-cell lymphoma, and I've actually highlighted the 9 statistically significant values for you, so that they stand 10 out. 11 And then if you look across the other subtypes, for 12 greater than two days per year, you can see that they all have 13 elevated Odds Ratios between 1.5 and about 2.5, showing you 14 that you see the same -- the same kinds of numbers here. 15 They're not statistically significant, because these are 16 the smallest groups, so you wouldn't have necessarily as much 17 power to detect a significant change, but if you look at the 18 numbers they're the same across all of the subtypes, suggesting 19 that glyphosate probably causes all of the various subtypes of 20 NHL, at least the common subtypes of NHL. 21 JUDGE PETROU: Of the lymphomas, what portion tends 22 to be the diffuse large B-cell lymphoma? 23 THE WITNESS: Diffuse large B-cell lymphoma makes up 24 about a third of all of the non-Hodgkin's lymphomas. So it's 25 the most common type, actually. WEISENBURGER - DIRECT / FORGIE 187 1 And the other point I want to make here, which I should 2 have from the start, is that all of these Odds Ratios are 3 adjusted for various factors, including proxy surrogate 4 responders, and for the three chemicals that were correlated 5 with glyphosate and were known from other studies to be risk 6 factors for non-Hodgkin's lymphoma. So it's adjusted for 7 surrogates, it's adjusted for family history, for intensity of 8 use, and also for pesticides. So this is strong, compelling 9 data, when one looks at this carefully. 10 BY MS. FORGIE 11 Q. Okay. 12 A. I'm just going to say a few words about the Agricultural 13 Health Study. 14 I concur with the things that Dr. Ritz told you earlier 15 today. It's an important study. It's a large cohort study of 16 restricted-use pesticide applicators. 17 I think for most of the publications, the findings are 18 acceptable, but I think with regard to glyphosate, as she told 19 you, there are significant issues and flaws that resulted in a 20 negative finding, and that's why it's an outlier compared to 21 the other studies. 22 Q. Okay, let's remember to go nice and slow. You're doing 23 great. Okay. 24 A. Yeah. 25 Q. And I think the biggest issue and problem with the WEISENBURGER - DIRECT / FORGIE 188 1 Agricultural Health Study is this issue of exposure 2 misclassification of glyphosate use, which, as Dr. Ritz told 3 you, is non-differential, and this type of bias would move any 4 true effect towards the null. 5 And one of the other issues that I think needs to be 6 considered is that the people in this study who were exposed to 7 glyphosate had a relatively short median lifetime years of use. 8 So if you look at the study, the median years of use for 9 glyphosate was 8.5 years, and the range was between 5 and 14 10 years, which means half of the people had less than 8.5 years 11 of use. Okay? 12 And in a cohort study of people who are using chemicals, 13 usually these people use these chemicals in this occupation for 14 10, 20, or even 30 years, and so this is a relatively short 15 period of time of use, okay, which could affect the results of 16 the study, especially early on in the study. 17 The other issue is the follow-up, and the follow-up is 18 significantly longer in this update. It's 18 years, as opposed 19 to about seven years. But in these cohort studies, one usually 20 expects to follow these patients for not just 18 years, but 21 usually for 30, or 40 years, or even up to the time when most 22 of the people have died, so you have a complete story of what 23 happened, because if the median latency period is long, if it's 24 30 years or 35 years, you wouldn't see enough of the disease at 25 this kind of follow-up to really give you elevated risks. WEISENBURGER - DIRECT / FORGIE 189 1 You'd have to wait longer. 2 And so I think it's really a second interim analysis, 3 rather than a final analysis of the AHS Study; and for these 4 reasons, I think I and others have questioned the validity of 5 the findings of the AHS Study and, as a result, have given it 6 less weight than some others have, some of the other experts 7 have. 8 Q. Now, when you say you give it a little less -- you give it 9 less weight, can you explain, briefly and slowly, what you mean 10 by that, please? 11 A. Well, I've given it less weight because I have to question 12 the findings. If -- if this issue of exposure and 13 misclassification has biased the true results to the null, you 14 wouldn't find anything, and I think that's what's happened in 15 this study. 16 And so I -- so, you know, I wouldn't give it a lot of 17 validity for the same reason I probably wouldn't put it into a 18 meta-analysis, because first of all, it's a different kind of 19 study, and secondly, I question its validity. 20 Q. Okay, and when you say you wouldn't put it -- you question 21 its validity, and therefore, wouldn't put it in to a 22 meta-analysis, could you break that down and explain that more 23 fully and a little slowly, please? 24 A. Well, I think when -- Dr. Ritz covered this, but when you 25 put -- you have to select each study and decide whether it WEISENBURGER - DIRECT / FORGIE 190 1 should go in to the meta-analysis or not, based on the quality 2 of the study, the type of the study, whether you believe the 3 results or accept the results, or not. 4 And so -- and so a study like this, where one really 5 questions the validity of the findings, especially if it's a 6 negative study, one could argue that you shouldn't put that 7 kind of a study into a meta-analysis. 8 Q. Okay, and is that partly because it's a cohort; and partly 9 because of the exposure and misclassification and other 10 problems? 11 A. For me, it's more the latter. 12 Q. Okay. 13 THE COURT: On the -- on the issue of short median 14 lifetime years of use, and the latency period and all that, is 15 that a criticism that would -- or a concern that would also 16 apply to the case-control studies that you were talking about? 17 THE WITNESS: It is a criticism that could be applied 18 to the case-control studies, yes, it is. 19 THE COURT: Okay. So is that concern a reason to 20 throw out AHS? 21 THE WITNESS: Well, the main reason -- 22 MS. FORGIE: I'm sorry, I didn't hear that question. 23 I'm sorry, could you repeat it? 24 THE WITNESS: The main reason that I -- 25 THE COURT: Hold on, let me just repeat the question WEISENBURGER - DIRECT / FORGIE 191 1 so she can hear. 2 MS. FORGIE: Thank you. 3 THE COURT: The concerns that you're expressing 4 here -- I mean, you're expressing this concern about a 5 relatively short median lifetime years of use. 6 I assume your expressing that concern because it's a big 7 deal, it's not a nitpick. Is that right? 8 THE WITNESS: Right, yes. 9 THE COURT: Okay. So doesn't that apply to the 10 case-control studies as well? 11 THE WITNESS: It could. It could, except the 12 case-control studies have positive findings -- 13 THE COURT: Well, but -- 14 THE WITNESS: -- and one would have to, you know, one 15 would have to think about, well, was -- was the type of 16 exposure different in the case-control studies? 17 THE COURT: But in the case-control studies, there 18 could be other reasons why there has been association was shown 19 and, you know, there's concern about recall bias and all that 20 stuff. 21 But -- and so wouldn't -- I mean, if they also have a 22 relatively short median, why wouldn't the criticism apply 23 equally to those studies? Conversely, if you're not going to 24 apply that criticism to those studies, why could you apply it 25 to the Agricultural Health Study? WEISENBURGER - DIRECT / FORGIE 192 1 THE WITNESS: Well, the reason I'm applying it to the 2 Agricultural Health Study is that it's a retrospective and 3 prospective cohort study, and for that type of study, usually 4 the exposure time is long and the follow-up time is very long. 5 So all I'm trying to say here is, this is an interim 6 analysis of a large study; that it may be too early to actually 7 see significant effects. 8 And in fact, if you look at the Andreotti Study and you 9 look at the -- the -- the lagged analysis for 20 years, of -- 10 of follow-up, you begin to see increased Odds Ratios, 11 suggesting that, in fact, what I'm saying may actually be true, 12 because after -- in that lagged analysis, you begin to see 13 increased Odds Ratios for NHL and subtypes of NHL. 14 So all I'm saying is that it may be too early to see 15 effects in the Agricultural Health Study, and one really needs 16 to probably wait for longer years of use and longer follow-up, 17 to really be sure that this is a negative study. 18 THE COURT: Do you mind if maybe I went back to a 19 point you made earlier about recall bias -- 20 THE WITNESS: Sure. 21 THE COURT: -- with the case-control studies? 22 You mentioned that -- you stated that recall bias is 23 not -- is probably not a concern with those studies, because we 24 didn't see recall bias with respect to other chemicals and 25 pesticides. Am I remembering that correctly? WEISENBURGER - DIRECT / FORGIE 193 1 THE WITNESS: Yes. We didn't. We didn't see it for 2 glyphosate in other case-control studies. 3 THE COURT: Wait, sorry. Say that one more time? 4 I thought you were saying we didn't see it with respect to 5 other pesticides. Did I mis-hear you? 6 THE WITNESS: Well, I would say -- I'll be more 7 generalized. I don't think recall bias is a major reason to 8 explain the consistent findings for NHL and glyphosate. 9 THE COURT: And why not? 10 THE WITNESS: First of all, because we didn't see 11 recall bias in the other case-control studies of similar 12 diseases like myeloma, leukemia, Hodgkin's lymphoma and solid 13 tumors. So if you have a recall bias, you should see it -- you 14 would -- it wouldn't be selective just for NHL. 15 THE COURT: And how do we know we didn't see a recall 16 bias in those other studies? 17 THE WITNESS: Because the Odds Ratios weren't 18 increased. 19 THE COURT: Okay. You mean the Odds Ratios weren't 20 increased between people who were not exposed and people who 21 were exposed? 22 THE WITNESS: Correct. 23 THE COURT: Or people who didn't have the disease and 24 people who did have the disease? 25 THE WITNESS: Correct. WEISENBURGER - DIRECT / FORGIE 194 1 THE COURT: All right. 2 BY MS. FORGIE 3 Q. And Doctor, in case-control studies, you start with the 4 cases, correct? Whereas in cohort studies, you start with a 5 number of cases, so you need enough time for the NHL to 6 develop. Is that correct? 7 A. It's correct. Yes. 8 Q. Okay. All right. So let's go back to -- I'm trying to 9 remember what slide we were on. 10 All right. Let's go to -- 11 A. So I want to say a few words about the animal studies. 12 I reviewed all of the published literature on the animal 13 studies, including the IARC the EPA, the German review, and 14 actually even some of the published animal studies. 15 And you'll hear a lot more about this later this week, but 16 it was my conclusion that there were multiple positive 17 carcinogenesis tests in both mice and rats, and we see 18 dose-related effects for multiple tumors. 19 In one of the mouse studies, there were rare tumors that 20 were increased that you don't normally see in those animals, 21 renal tubular carcinoma. 22 And then I think it's interesting that in -- in four 23 studies in the mice, you saw malignant lymphomas, these are 24 NHLs, in the mice. So the chemicals were inducing the NHLs in 25 the mice. And it's the same tumor that we're saying the WEISENBURGER - DIRECT / FORGIE 195 1 epidemiologists study show an increased risk. 2 And then there's replication of the -- of some of these 3 tumors in the different -- in different animal studies. So 4 there was replication for hemorrhagic sarcoma, for renal cell 5 tumors, for liver tumors, for pancreatic tumors, and a variety 6 of other tumors. So there is replication. 7 Q. Okay. Well, let's go onto the next slide, the Mechanisms 8 of Lymphoma -- how do you say that? 9 A. Lymphomagenesis. It's like carcinogenesis, except for 10 lymphomas. 11 Q. Is there a difference between carcinogenesis and 12 carcinogenicity? 13 A. Well carcinogenicity is an adjective to describe a 14 chemical that causes cancer. 15 Q. Okay. Sorry. 16 A. Basically the same word. 17 Q. Okay, thank you. 18 A. So the IARC put forth two mechanisms that they thought 19 were strong mechanisms for lymphomagenesis, or carcinogenesis. 20 One was genotoxicity, which simply means that the chemical 21 damages the chromosomes, or the genes, causing these 22 translocations and rearrangements and deletions and mutations. 23 Okay? 24 And the other one was oxidative stress. And what that 25 means is that when the cells are in contact with the chemical, WEISENBURGER - DIRECT / FORGIE 196 1 there's -- there's a stress reaction; and as part of that 2 stress reaction, there's the release of what are called free 3 oxygen radicals, and those free oxygen radicals can damage 4 proteins and they can also damage DNA and cause genetic 5 lesions. So it's a sort of an indirect mechanism of causing 6 genetic lesions. 7 And one of the things I emphasized in my report was that 8 in multiple studies of human lymphocytes, they could 9 demonstrate gene toxicity of glyphosate, and the formulations; 10 and in some of those studies, it was actually at quite low 11 doses, and there was a dose effect, which is what you would 12 expect. 13 Q. And Doctor, when you say, "quite low dose," is it possible 14 to quantify that, or not? 15 A. I'd have to look in my report, but it's in my report. 16 Q. All right. Well, why don't we come back to that. Let's 17 continue and see if we can at least get through this slide 18 before we break. I think we're breaking at 4:00. 19 A. And of course, there's similar genotoxic effects we've 20 seen in other types of human cells; and there were similar 21 genotoxic effects seen in non-human mammalian cells as well as 22 in vitro and in vivo, as well as non-mammalian systems. 23 So there's a body of evidence that's pretty compelling 24 that glyphosate and the formulations are genotoxic in living 25 cells. WEISENBURGER - DIRECT / FORGIE 197 1 And then another interesting finding in some studies 2 recently was that there are certain individuals who seem more 3 susceptible to this oxidative stress pathway; and by that, I 4 mean that they have genetic polymorphisms. They inherit a 5 certain pattern of genes from their parents that might make 6 them more susceptible than their neighbor. 7 And there have been studies to look at the -- 8 Q. Let's slow down a little bit. I think my understanding is 9 we're going to be breaking at 4:00. Let's just go slow and try 10 to get slowly through this one. 11 THE COURT: If direct is almost wrapped up and you 12 need to go a little over 4:00, that's fine. 13 MS. FORGIE: Yeah, but it -- unfortunately, it's not. 14 I wish it was. 15 THE WITNESS: I think I can finish in -- I would like 16 to finish today, if I can. Let me say that. 17 THE COURT: Well, you're not going to finish your 18 cross-examination today. 19 THE WITNESS: No, no, I know that. 20 MS. FORGIE: I don't think it's going to be possible 21 to finish, so let's try to at least finish up this slide, if we 22 can, but we do have to go slowly, for the court reporter. 23 THE WITNESS: So the genetic polymorphisms result in 24 a susceptibility to NHL, and so what I'm saying is there's a 25 population of people who have these polymorphisms who would be WEISENBURGER - DIRECT / FORGIE 198 1 more sensitive. 2 BY MS. FORGIE 3 Q. And can -- 4 A. Also that sort of confirms the fact that this pathway is 5 important. 6 Q. And can you please explain what a genetic polymorphism is? 7 A. So for each of our genes, we have different sequences of 8 amino -- of -- of ribonucleotides that create a gene that's the 9 same gene, but it's slightly different, okay? So you might -- 10 your gene for G6PD, which is an enzyme, might be different than 11 mine, and I might make more of that enzyme than you do, and 12 that might protect me from something. Okay? 13 Q. Mm-hum. 14 A. So there are a lot of polymorphisms in our genes, and -- 15 and those are important in determining a whole variety of 16 different biochemical effects. 17 Q. Okay. Please continue. 18 A. Genetic variation. 19 Q. There you go, genetic variation. So genetic polymorphisms 20 means genetic variations, is that -- 21 A. Yes. 22 Q. -- have I finally got that right? 23 A. Yes. 24 Q. Okay. Thank you, Doctor. 25 A. This is just a very recently published study that I came WEISENBURGER - DIRECT / FORGIE 199 1 across actually, after my second deposition, but it speaks to 2 the fact that glyphosate induces what are called double-strand 3 breaks in cultured human lymphocytes at low doses. And I'm 4 showing you the data here. 5 The importance of this is that double-strand breaks are 6 the kinds of breaks that we see in lymphoma cells that result 7 in these rearrangements of genes, and in deletions of genes. 8 So when they culture these human lymphocytes at low doses 9 of glyphosate -- and you can see the dose in the second column, 10 the first is zero, so that's our negative control, and as they 11 increase the dose, you can see that on the -- on the third 12 column, that the mean number of cells that have a lot of 13 damage, greater than tenfold site damage, increases 14 significantly with a p-value that's significant for trend. 15 Q. Let me stop you there. Can you explain what a foci is, 16 and what those mean? 17 A. So foci just means when they looked at the cells, they 18 could see an area of genetic damage and they counted them. So 19 if they had a greater than 10 foci, or areas of genetic damage, 20 that was a highly damaged cell. 21 And then they counted the number of those cells, and so 22 you can -- what I'm presenting here is the mean percentage of 23 those cells in replicated experiments. 24 Q. Okay. 25 A. And etoposide is a known -- it's actually a chemotherapy WEISENBURGER - DIRECT / FORGIE 200 1 medication that damages DNA, and that's sort of our positive 2 control; and you can see that it had a very high number of 3 cells that had significant damage. 4 If one uses lower doses of etoposide, their findings are 5 very similar to what you see for glyphosate. 6 Q. Okay. 7 A. And then there are two studies that I think are very 8 informative with regard to DNA damage, in actual people. Okay? 9 And these are two studies that were done in South America. 10 The first one by Paz-y-Miño looked at DNA damage in blood 11 leukocytes in Ecuadorian people who were exposed to the 12 spraying of glyphosate-containing chemicals. They lived on the 13 border of Ecuador and Colombia, and Colombia was spraying all 14 of this glyphosate to get rid of the cocaine and other illicit 15 crops, and these individuals were living on the border and were 16 getting exposed to the chemicals, just as by drift, I suppose, 17 or actually being sprayed. And so what -- and so this happened 18 over a period of weeks. 19 And so after that period of time, the researchers went in 20 and drew blood from people who were exposed, who lived on that 21 in that area, and a group of people who were very similar, but 22 who lived far away and weren't exposed. 23 And then they did an assay called the comet assay, which 24 looks at DNA breaks, and what they found was that there was a 25 significant increase in what they call mean DNA migration, WEISENBURGER - DIRECT / FORGIE 201 1 which is an index of DNA damage in the exposed individuals 2 compared to the unexposed or control individuals, and the 3 p-value was highly significant. 4 So what this shows is that when people are sprayed with 5 glyphosate at pretty high doses, they actually get measurable 6 DNA damage, okay, like we've talked about. 7 Q. And can you explain what you mean by an index of DNA 8 damage? You used that phrase earlier, without explaining. 9 A. An index of DNA damage. I don't know if I used that term, 10 "index." 11 Q. Well, maybe I misunderstood. That's what I thought you 12 said, with regard to those Juarez Lopia (phonetic). 13 A. Well, in this study, the mean DNA migration is an -- it is 14 an example of DNA damage. 15 JUDGE PETROU: By "index," you mean indicator. 16 MS. FORGIE: Indicator. 17 THE WITNESS: Maybe that's what I said. 18 BY MS. FORGIE: 19 Q. Okay, sorry. Okay, so -- 20 A. And then the second study is also very informative. 21 If we go to the next slide? 22 So this was a study actually done in Colombia, again, 23 looking at glyphosate sprayed from airplanes onto crops, and 24 this one uses a different test called the binucleated cells 25 with micronuclei, which you kind of see along the left column. WEISENBURGER - DIRECT / FORGIE 202 1 And I have to kind of walk you through this study, because 2 it's complicated, but if you look at the first province, 3 Santa Marta, this was a province where they grow coffee, and 4 they don't use any pesticides. So it's organically grown 5 coffee. This is sort of your negative control group, unexposed 6 group. 7 And then Boyacà is an area where they spray pesticides, a 8 lot of pesticides, by hand. Okay? So this is an area where 9 they spray pesticides. 10 And if you look at the first box, the sort of the white 11 box in Boyacà, you can see that the DNA damage is significantly 12 higher than it is in Santa Marta. Okay, so this tells you that 13 by using pesticides, you increase DNA damage sort of at a basal 14 level, okay? 15 And in the other areas they also use pesticides, Putumayo, 16 Narino and Valle de Cauca, you can see that, if you can look at 17 the first box, each of the first boxes is significantly higher 18 than for Santa Marta, which is the control group, okay? 19 So that what says is that using pesticides will increase 20 DNA damage in blood lymphocytes, okay? 21 So in Boyacà, they did two measurements. So you see the 22 second bar, they did a measurement a month later just to see if 23 there was any change, and it was pretty much the same. Okay? 24 The next three provinces, Putumayo, Narino and Valle, were 25 characterized by aerial spraying of glyphosate. In Putumayo WEISENBURGER - DIRECT / FORGIE 203 1 and Narino, it was mainly again for illicit crops like cocaine, 2 and in Valle, it was treated -- used for agricultural processes 3 for sugarcane. Okay? 4 Q. Okay. 5 A. What they did is, shortly after the aerial spraying, which 6 is the second bar, they went in within four or five days and 7 took a blood sample, and what you can see for Putumayo and 8 Narino and Valle is the second bar is significantly higher than 9 the first bar. So what that shows is the spraying of the 10 glyphosate increased the DNA damage in that very short period 11 of time. Okay? 12 And that's the most important finding in this study. They 13 also did a third blood sample four months later, and they did 14 find significant increases in some of the places; but that 15 can't be related necessarily to glyphosate because of the other 16 pesticides that were used. 17 So again, what this study shows is that exposure to 18 significant amounts of glyphosate in people increases DNA 19 damage. 20 Q. Okay. Anything else from this Bolognesi? 21 A. No. 22 Q. Okay. 23 A. So the last slide's going to take me a few minutes. Do 24 you want me to finish my presentation? 25 Q. Well, wait a minute. The last slide being the Paz-y-Miño, WEISENBURGER - DIRECT / FORGIE 204 1 but we still have to do the whole Bradford Hill analysis. 2 A. This is Bradford Hill. 3 Q. Okay. Well, I mean the other criteria is what I'm talking 4 about. 5 Do you want to -- I don't know what the Court wants us to 6 do. 7 JUDGE PETROU: It seems like the next slide is 8 Bradford Hill criteria, at least from the slide that we have. 9 THE COURT: You can keep going for a while. 10 MS. FORGIE: Okay. 11 THE WITNESS: So I was just going to walk you through 12 how I applied the Bradford-Hill criteria based on all of the 13 information that I've shared you with over the last 45 minutes 14 or so. 15 So temporal-related -- and so I'll just walk you through 16 each of these one at a time, and we can talk about them. 17 So the first one is temporal relationship, and what that 18 means is that you have to have the exposure before you develop 19 the disease. That's the only criteria that is an absolute 20 criteria in the Bradford Hill, and it makes sense. Right? 21 Because it could -- the chemical couldn't cause the disease if 22 it occurred after you got the disease. 23 BY MS. FORGIE: 24 Q. That makes sense. 25 A. Yeah, and of course, all of the epidemiology studies WEISENBURGER - DIRECT / FORGIE 205 1 fulfill this criteria, as well as the animal studies. 2 The second criteria is strength of association, and 3 usually when you have strong associations, it's -- it's an 4 important point in strengthening your conclusion of causation, 5 and in -- and also consistency across studies. 6 And so as I mentioned to you, that the -- the case-control 7 studies for the five studies were positive, and the Risk Ratios 8 were above 2. 9 And -- and -- and I'm sorry, now I'm getting confused. 10 And most of them are statistically significant, is what I'm 11 trying to say. So there was an association that was consistent 12 and it was strong. 13 And the third criteria is dose-response, and as I 14 mentioned, in the case-control studies there were two studies 15 that did demonstrate dose-response; and of course, the animal 16 studies also demonstrated dose-response. 17 The fourth is replication of results. And as I said, four 18 of the five case-control studies were positive, four of them 19 were statistically significant. So there's replication there, 20 with the one outlier being the Agricultural Health Study, and 21 there was also replication of results in the animal studies. 22 Biological plausibility means: Does what we find make 23 sense, based on what we know about science? And as I've told 24 you, lymphoma is a genetic disease. We have shown that 25 glyphosate and the formulations cause genetic damage. They WEISENBURGER - DIRECT / FORGIE 206 1 cause the kind of genetic damages that leads to lymphoma. They 2 cause the genetic damage in lymphocytes, that are the cells 3 that become lymphoma, and they do it in people, if you expose 4 them to high amounts of -- of the chemical, as we saw in the 5 South Americans studies. 6 So I think there is biological plausibility, and the fact 7 that the animals also get lymphoma also contributes to that 8 biologic plausibility. 9 Of course, when we evaluate these kinds of information, we 10 always have to ask, are there other explanations? And we've 11 talked a little bit about recall bias, so I won't go over that 12 again, but there are other kinds of things like confounding, 13 which was talked about earlier, and others kinds of alternative 14 explanations. 15 And what I would say is that the case-control studies were 16 done by very reputable researchers. They, used a very rigorous 17 design. They were published in peer-reviewed journals. They 18 were accepted for review by EPA and by IARC and other agencies. 19 So I don't think we can just discount these important studies 20 based on some hypothetical alternative explanation. 21 What else do I want to say about that? I think that's -- 22 I'll leave it at that. 23 Q. Okay. 24 A. And then finally, disease specificity. I think one of the 25 important things here is that there is one disease that seems WEISENBURGER - DIRECT / FORGIE 207 1 to be associated with glyphosate, and that's non-Hodgkin's 2 lymphoma. All of the other studies have been negative. So 3 there seems to be a relationship between this disease, and this 4 chemical, that is not seen with other diseases, like even 5 closely-related diseases like myeloma, leukemia, Hodgkin 6 lymphoma and solid tumors. So it seems to have a specificity. 7 And then finally, coherence. I mean, I think -- all -- 8 all of the information I've told you this afternoon is 9 coherent. It fits together. 10 And we know that other pesticides, similar organophosphate 11 pesticides, cause lymphomas using the same mechanisms, the 12 genotoxicity and oxidative stress. So there's an analogy here 13 between other chemicals which have also been shown to cause 14 non-Hodgkin's lymphoma. 15 So that's kind of how I weighed the evidence, and applied 16 it to the Bradford-Hill criteria, and came to the conclusions 17 that I did, which you can show on the last slide. 18 Q. Okay. Can we put up the last slide, please? 19 So after you did your Bradford-Hill analysis, Doctor, then 20 you reached your opinions, is that correct? 21 A. Yes. 22 Q. And what did you conclude? 23 A. Well, I concluded with a reasonable degree of medical 24 certainty that glyphosate and the formulations including 25 Roundup® can cause non-Hodgkin's lymphoma in humans exposed to WEISENBURGER - DIRECT / FORGIE 208 1 these chemicals, both in the workplace and probably in the 2 environment. 3 Q. Okay, and when you say GBFs, what do you mean? 4 A. Glyphosate-based formulations. 5 MS. FORGIE: Okay. All right, I think that's it for 6 the direct for today. Unless, of course, the Court has -- 7 THE COURT: Well, I mean, maybe I will ask you 8 some -- just a couple of concluding questions, similar to the 9 ones I asked Dr. Ritz. 10 THE WITNESS: Sure. 11 THE COURT: Which is, you know, this opinion you have 12 that glyphosate and GBFs can cause NHL in humans exposed to 13 these chemicals in the workplace or environment, are you -- is 14 it your opinion that it's capable of causing NHL in humans at 15 the exposure levels they are current -- some people at least 16 are currently experiencing? 17 THE WITNESS: Yes. I believe that, yes. 18 THE COURT: Okay. So it's not just an opinion that 19 it's capable of causing NHL, in the abstract. 20 THE WITNESS: Right, and the case-control studies 21 would show you that. 22 THE COURT: Okay, and if you -- if you only had the 23 epidemiology, would that be enough to form the same opinion? 24 THE WITNESS: Well, I think it would -- it wouldn't 25 be enough to form the same opinion, but I would never look at WEISENBURGER - DIRECT / FORGIE 209 1 just one body of information. 2 But if you told me that this is all I knew, this was all 3 of the information there is, I would be -- I would be very 4 concerned, okay, and probably begin to do other studies to try 5 to see if I could understand this. 6 But probably by itself, I would say, no, but we never do 7 that kind of analysis. I mean, we have lots of other 8 information in this situation on animal studies, cell studies, 9 all kinds, even exposure in people. So I think the body of 10 evidence is strong evidence, that led to my conclusions and my 11 opinion. 12 BY MS. FORGIE 13 Q. One other question raised by that. 14 And Doctor, would it be fair to say that the standard 15 methodology both at your hospital and as what you teach is that 16 you look at the weight of the evidence, and you consider all of 17 it, you just don't teach -- you just don't look at one part or 18 another, and Dr. Ritz said it's hard to sort of unlearn what 19 you already have in your head. Is that fair? 20 A. That's correct. You have to look at all of the 21 information. You have to carefully study it, you have to weigh 22 it, you have to see how it fits together. 23 And -- and by going through that kind of analysis, you 24 come to a conclusion that -- that I think is supported by 25 scientific studies, statistically significant results, WEISENBURGER - DIRECT / FORGIE 210 1 increased Odds Ratios, et cetera. 2 Q. And is the methodology that you've used and that you've 3 been describing as you've been on the stand today, standard 4 acceptable methodology that you use in your practice and that 5 you use for peer-reviewed journals and things like that? 6 A. Yes, it is. 7 MS. FORGIE: Okay. 8 THE COURT: Okay. Great. 9 THE WITNESS: Thank you. 10 MS. FORGIE: Thank you, Doctor. 11 THE COURT: You'll be back tomorrow morning, I take 12 it, or tomorrow afternoon for cross-examination. 13 THE WITNESS: I will. 14 THE COURT: All right. You can step down. 15 THE WITNESS: Thank you. 16 THE COURT: And then before we go, let me -- I would 17 just like, if I could, to be as transparent with you as I can 18 about some of the questions I continue to have. 19 MS. FORGIE: Thank you, your Honor. 20 THE COURT: If, on the issue of adjusting for other 21 pesticide exposure -- 22 MS. FORGIE: Yes, your Honor. 23 THE COURT: -- and you know, this is -- this can be 24 explored on cross, redirect with other witnesses, I don't care 25 which witness, but I'm just telling you that this continues to WEISENBURGER - DIRECT / FORGIE 211 1 be an issue for me. 2 I still don't understand how or why it would ever be a bad 3 idea to adjust for other pesticide exposure if we are concerned 4 that there may be a link between other pesticides and 5 non-Hodgkin's lymphoma. 6 MS. FORGIE: Okay. 7 THE COURT: Or to put it another way, unless we are 8 confident that there's not a link, I don't understand why we 9 would ever think it is not a good idea to adjust. 10 And reading through the IARC Study, they seem to be of 11 that -- they seem to express that view. I mean, every time 12 something didn't adjust for other pesticide exposure, that was 13 a concern expressed by the working group. 14 So I -- I still don't -- if there is a reason not to 15 adjust for other pesticide exposure, I still don't understand 16 what that is. 17 MS. FORGIE: Well, Dr. Ritz is here. If she -- if 18 you want, she could come back on the stand and possibly explain 19 that to you. She's not going to be available tomorrow. 20 Unfortunately, she has to teach at UCLA. 21 THE COURT: Well, I think it's going to be up to you 22 how you want to sort of present that and through what witness. 23 I think we're done for today. 24 MS. FORGIE: Okay. 25 THE COURT: But that will be -- that will be up to WEISENBURGER - DIRECT / FORGIE 212 1 you to decide. 2 I also still have confusion about the NAPP Study. 3 MS. FORGIE: Okay. 4 THE COURT: When people are talking about the 5 NAPP Study, are they talking about the slides that were 6 presented at this conference, slides that were prepared for 7 this conference but not presented? What iteration of the 8 NAPP Study should we be considering here? 9 MS. FORGIE: Okay. 10 THE COURT: I'm still confused about that. 11 MS. FORGIE: Okay. I'm sorry, your Honor, I'll see 12 if I can get that clarified. 13 THE COURT: No need to apologize. I'm just telling 14 you things you should think about in the days ahead. 15 I also still have some confusion about recall bias, and I 16 have to tell you, intuitively it seems like recall bias is a 17 pretty big deal for these studies, including for the 18 conclusions about dose-response, if that's the appropriate way 19 to characterize it. Although, I guess there are some questions 20 about whether that's the appropriate way to characterize what 21 these studies did. 22 Eriksson -- and was it Eriksson and De Roos? 23 THE WITNESS: McDuffie. 24 MS. FORGIE: Eriksson and McDuffie. 25 THE COURT: McDuffie. WEISENBURGER - DIRECT / FORGIE 213 1 MS. FORGIE: Yes Your Honor. 2 THE COURT: If there was an answer given about not 3 seeing recall bias in -- with respect to other diseases, and it 4 would probably be helpful for me, at least, for somebody to 5 walk me through that, because I still don't have -- like I 6 said, intuitively, it seems like a big deal, a lot of people 7 say it's a big deal, and if it's not a big deal, I don't yet 8 understand why it's not a big deal. 9 MS. FORGIE: Okay. 10 THE COURT: So I just wanted to throw out those three 11 things. 12 MS. FORGIE: I really appreciate that, your Honor. 13 I think it will help us, given the limited time, I think it 14 will help us focus where we want the rest of the testimony to 15 go. So I appreciate that. 16 THE COURT: All right. Anything else before we 17 adjourn for today? 18 MR. LASKER: No, Your Honor. 19 THE COURT: Okay. Thank you. 20 MS. FORGIE: Thank you very much, your Honor. 21 THE COURT: So 12:30 tomorrow, is that right? 12:30. 22 MS. FORGIE: I can't remember. 23 (At 4:20 the proceedings were adjourned.) 24 25 1 I certify that the foregoing is a correct transcript from 2 the record of proceedings in the above-entitled matter. 3 4 5 March 6, 2018 Signature of Court Reporter/Transcriber Date 6 Lydia Zinn 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25