TRANSCRIPT: Meeting 4, Session 9

Roundtable

Date

March 1, 2011

Location

Washington, D.C.

Presenters

Lawrence Corey, M.D.
President and Director, Fred Hutchinson Cancer Research Center
Principal Investigator, HIV Vaccine Trials Network
Professor, Laboratory Medicine and Medicine, University of Washington

Robert M. Califf, M.D.
Vice Chancellor for Clinical Research, Duke University
Professor of Medicine, Duke University Medical Center
Director, Duke Translational Medicine Institute

Susan E. Lederer, Ph.D.
Robert Turell Professor of History of Medicine and Bioethics and Chair, Medical History and Bioethics, University of Wisconsin School of Medicine and Public Health

Dan W. Brock, Ph.D.
Frances Glessner Lee Professor of Medical Ethics, Department of Global Health and Social Medicine and Director, Division of Medical Ethics, Harvard Medical School

Eric M. Meslin, Ph.D.
Director, Indiana University Center for Bioethics
Dean (Bioethics) and Professor of Medicine, Medical and Molecular Genetics, Public Health, and Philosophy, IU School of Medicine
Director, Indiana University-Moi University Academic Research Ethics Partnership

Download the Transcript

Transcript

 

DR. GUTMANN:
So Eric, as you were saying, you had one more point.
 
DR. MESLIN:
Yes. Well this actually follows so you can consider it one point, if you want. It is a subordinate clause from the first point.
 
I can say with complete humility that virtually every commission has tried its best to see its findings implemented and with very few exceptions, I have seen that happen.
 
I would like to recommend or if I could make a wish for you is that you would focus your attention on implementing recommendations that you make. It is not that the time has passed for thinking and talking. I am a big fan of thinking and talking. That is all I am capable of doing. Maybe not so well in some instances. But I would really like to see this Commission and other bodies like it to not just come to the finish line and say we have delivered our report and we have several recommendations but to actually go through the tape and run down the road a little further and think about implementation. That is going to be the single most important thing that you can do, irrespective of what the recommendations are.
 
DR. GUTMANN:
Thank you. Let me just say so you know and the public knows as well that in the first report that we did on synthetic biology, we asked the government to report back to us in 18 months’ time what progress they had made in implementing our recommendations. And we fully intend to come back in 18 months’ time and report to the public as to what progress has been made, whether it has been full or none or somewhere in-between.
 
And so that is a very well received recommendation. Even though it doesn’t tell us what the issues are that we should focus on, it still is important because we did look at some of the — We all read the NBAC recommendations and we also recognized that there was minimal government responses to them.
 
Dan?
 
DR. BROCK:
That is a hard question, as I am sure you are aware. In particular it is hard in the context of thinking about developing country research because as has been discussed at length here this morning, the practices there need to be negotiated between local researchers from this country but also with the local communities and researchers from other countries. And so exactly what implementation would mean, who has the ability to implement because it is not simply, I think, in many cases that the feds need to pass another regulation or another law.
 
So I think implementation for many of the things in the developing country context is a good deal more complex than it is in the local context.
 
I guess if I had to think of one umbrella under which you could think about a lot of your work, it would be the notion of exploitation. I think that is, in a way, the central notion that worries people when they start thinking about developing country research.
 
Now there is a good deal of work. Wertheimer has written the best book on this. But I think most of the contentious issues could be brought under that umbrella and it also is an umbrella that has got a lot of confusion in the literature about it. I am on Zeke’s side about undue inducement, for example. But so that might be a way of framing the underlying uneasiness that people have and that you want to respond to.
 
DR. GUTMANN:
Thank you. Susan.
 
DR. LEDERER:
What I would say is that I think one critique of bioethics that has grown has been the narrowness of its focus. And in some ways informed consent and I know this is a caricature but hold with me, you know, it was like a band-aid for all the problems with, and it turns out not to resolve them, not surprisingly.
 
So I guess if I had one thing to suggest, it would be to broaden the focus on the socioeconomic realities. I mean, for example, I don’t want to make another requirement but listening to the conversation earlier sort of like, I mean some of those plans for global research sound fabulous but the question to me is not if the money runs out, it is when. And it seems to me if we look at the past, a lot of bad decision making comes when the financial realities change dramatically and there is not contingency plan.
 
So I mean, I would like to put in a plea for planning not just for aftercare of the subjects but, you know, some sort of plan for when your funding source dries up completely and what are you going to, you know, how are you going to scramble? I mean, maybe people do take that into account but certainly they did not do so historically.
 
DR. BROCK:
I think the Gates Foundation has been focusing on that very issue with regard to HIV funding.
 
DR. GUTMANN:
Right. Thank you. Rob.
 
DR. CALIFF:
If I had my way about it, you all would define the attributes of the ecosystem that are most desirable. And I actually think they are pretty common. And if you think of the ecosystem as research participants, research sites, a national oversight of research sites in a country and then a global federation to oversee those, I think it is entirely possible now that there would be concordance among people in different countries about the desirable attributes that could be put forward as a measurable goal that no one could achieve in any country immediately but could be measurable as progress.
 
And then I would also put in my plea, I always put in to my friend Jeremy Sugarman here, —
 
DR. GUTMANN:
Yes.
 
DR. CALIFF:
— empirical measurement of what people are doing and what they believe I think needs to be an attribute of the system that needs to be built in.
 
DR. GUTMANN:
So if there were a set of attributes, do you think it would be possible to have some set of measurements to judge how well a proposal met those attributes?
 
DR. CALIFF:
I do and I want to be clear here that I am not saying that you can do everything through measurement. People often misquote Deming as saying you can’t improve it if you can’t measure it. What he actually said was the most important things can’t be measured but you should measure what you can.
 
And yes, I think if you define the qualities that were most desirable for each of those elements like an informed public or research participants, those things are actually empirically measurable and I think you would see much better implementation if you dealt with the system in that way.
 
DR. GUTMANN:
Right. Thank you very much. I am going to open it up for Commission members to pick your brains, okay? And your experience and expertise. John.
 
DR. ARRAS:
Thanks again. One of the realities on the ground that we have heard about is the off-shoring of biomedical research. A large proportion of the CDC’s and NIH research budgets go to research conducted overseas. I know that about 50 to 60 percent of the research done by U.S. pharmaceutical companies is now off-shored.
 
This raises, I think, interesting questions about subject selection. A lot of the research that you have been describing, you know, has been research done overseas for the benefit of people there. A lot of the research that I have been reading about lately, largely done by pharmaceutical companies, has been off-shored with the specific intention of basically doing research in a more financially congenial or legally congenial environment.
 
So very briefly, if you could talk to the issues of the promise and perils of off-shoring.
 
DR. GUTMANN:
Eric first and then Rob.
 
DR. MESLIN:
Well, the alliteration promise and peril is a little challenging but it is fun to try.
 
I think the perils are pretty obvious when you talk about exploitation and the selection of sites on the basis of lowered barriers, the kinds of things we were talking about before. There are reasons beyond just human subjects protection and why going somewhere with a lowered barrier is a bad idea. It is bad economically. It is bad politically. It is bad for a whole bunch of reasons. But what we don’t tend to think about is what the opportunities are for going elsewhere. And if we can get out of our mind the idea of that model of the opportunistic pirate who is going elsewhere and think that there may be reasons of international partnership and collaboration where we are now seeing not only more studies going on elsewhere, but foreign investigators have increased dramatically as a function of both FDA-sponsored studies and PIs or co-PIs at the NIH. So there are great advantages to “off-shoring.”
 
I do want to say one other thing. Let’s ensure that it is on the record. We have been talking all morning about international research as a function of the north/south or the economically developed versus developing country diet. And I think that is a substantial focus of ethics debate. But there are significant discussions going on between north/north. I mean, my own home country of Canada has an interesting conversation with the U.S. all the time about why it has to satisfy FDA and NIH rules when the Tri-Council policy is arguable as good as or better than. So this is not simply rich country/poor country. The discussion needs to include all of the above.
 
DR. GUTMANN:
Thank you. Rob.
 
DR. CALIFF:
I mean first of all I would just like to attack the notion that there is any such thing as a U.S. medical products company anymore. All of these companies are global. Many of them are telling me they don’t see the U.S. as their future market at all. Their device company is developing new devices with no intention of ever marketing the devices in the U.S. because they see it as a declining market with a declining economic scope.
 
The promise, of course, is having research done in places and neighborhoods that are pertinent to those places and neighborhoods as opposed to doing the research in one country like the U.S. and then telling the Chinese what the dose of a drug is, never having studied it in China. I think that is obvious.
 
But I think one of the things that bothers me the most right now is off-shoring of NIH-funded research that is intended to inform Americans about American clinical practice being done simply because we cannot get the research done in the U.S. It is too expensive and it takes too long to get the answers. That is a peril which is not a peril of off-shoring. It is a peril of a cause of off-shoring that we need to pay attention to.
 
DR. GUTMANN:
Could you follow up on your own statement and answer what should be done about that?
 
DR. CALIFF:
I think we have to, I think the U.S. system is in as much or more need of reform than the other countries that we are talking with, which is why an international commission is a good idea. We have as much to learn from other places.
 
You know, when we do cardiovascular trials funded by the National Heart, Lung, and Blood Institute, we simply cannot get them done in the U.S. even for questions that are most pertinent to U.S. practice because we are not a good environment to do it.
 
DR. GUTMANN:
Say more about that, though. Because? What would make a difference in making us a better environment?
 
DR. CALIFF:
If it were lower costs with less bureaucracy and less hurdles that are focused purely on what appears to those of us that do research as being paperwork that has no useful purpose with almost no empirical evaluation of which of the rules and regulations are actually beneficial.
 
To put in the one plug, it was mentioned that I am the co-chair of CTTI, which is an FDA academic public-private partnership.
 
DR. GUTMANN:
Right.
 
DR. CALIFF:
We are systematically trying to address those issues by empirically measuring what works and what doesn’t. Most of what we do probably has little to no effect on better research.
 
DR. GUTMANN:
Okay, thank you. Nita.
 
DR. FARAHANY:
I want to return to some of the issues that were raised by the NBAC report and to understand both where you would suggest that we go from that report and understanding some of the differences based on the landscape that you present of changes.
 
So first the NBAC report, while much of it may not have been implemented, the issues that were directed to the government, many of the recommendations were addressed at researchers rather than at the government. So not about paperwork but instead about norms of conduct and ethical standards for research would seem just as applicable today as they did at the time.
 
zNevertheless, many of those have also not been implemented and the kind of norm of conduct and ethics of conduct that that report highlights may not be fully embodied in the research practices today, particularly when there are these growing international partnerships that are developing.
 
So I was hoping first you could speak to what extent you think recommendations from a body like this can be effective when directed at researchers or directed at this kind of norm of conduct. Second, identifying what changes you think or what additional recommendations beyond that would be relevant in light of the ten points that you highlighted or really the nine you got through, that you highlighted about the changing landscape and what new kind of issues it raises relative to the recommendations that were already issued.
 
DR. MESLIN:
Thanks very much. I mean, it is a bit odd to try and reflect on what a report written basically 11 years ago, whether it still has any relevance because the context as I think we have all been seeing has changed substantially.
 
The issue of how to influence researchers is itself a great subject of empirical and scholarly study. How do you affect the behavior of investigators, physicians or anyone presenting them with a set of ethical commitments without having the tools for them to implement them and to show why it is useful is to me a fool’s errand.
 
I do think, however, that there are examples both in the testimony I gave and in other materials available, fairly widely available, that if you give investigators and researchers generally the tools to understand what ethical issues are, the training that they need and the proof that what they do will both ensure that their research gets completed in a timely fashion without undue burden, they are actually quite sympathetic to being given that information. It has to be done in a fairly simple way. I don’t mean simplistic. I mean simple way. We have seen through the —
 
DR. FARAHANY:
Could you just to speak to what you mean by tools?
 
DR. MESLIN:
Sure. Three tools. One is training and education. We have actually moved quite far down the road in training investigators about research ethics. It used to be you had to go online and take an NIH course, get 70 percent and satisfy your institutional requirements. We are now making some progress along those lines. The cooperative institutional training initiative, the online module program run out of the University of Miami and the University of Washington, now has probably a thousand institutional subscribers around the world. There is some evidence that it is actually having an impact. So that is tool number one.
 
Tool number two, which was alluded to in the NBAC report, and I think would be very helpful if you wanted to push this ball down the road to the NIH, is providing the administrative and financial tools to foreign countries to use indirect costs at a rate commensurate with what U.S. institutions do.
 
I could just give you one example. At Moi University in Kenya, they aren’t allowed to use any of their indirect cost recovery to support their institutional research ethics committee. They have to do it out of regular funds. Not allowing an institution the tools to carry out the review that they are actually required to undertake is something that frustrates researchers and sends them elsewhere. So that is two tools.
 
DR. WAGNER:
It is not just Kenya.
 
DR. MESLIN:
Oh, I only speak of the country I know a little bit.
 
DR. WAGNER:
You know the U.S. pretty well.
 
May I jump in on a question?
 
DR. GUTMANN:
Jim.
 
DR. WAGNER:
I have a question for all of you and it has to do with informed consent, which I realize is one of Christine’s seven sacred elements of ethical research. But I wonder about rethinking it in the context of this particular mission that we have in the following way.
 
At the end of the day, one of our responsibilities as researchers is to ensure that we shift some of that responsibility to informed consent to the subject. Right? I mean, ultimately that is one of the things that we are supposed to do.
 
I have a concern that gets to Dan, your comment about exploitation and Susan your comments about broadening the focus on socioeconomic realities that really the ethical purpose, the moral purpose for informed consent is not to secure the consent so that I can go forward, but it is to inform.
 
I worry about the capacity of some places where it will be important and already is important to do research of the individuals there actually to be fully informed. And how do we measure that and how do we go about ensuring that we can inform in places where, you know, rates of literacy are under five percent and where perfectly adequate social behaviors and social understandings for that community may lack the, I hate to use the word sophistication because it sounds pejorative, but for this purpose may lack the sophistication to enable us to actually get informed subjects, not just check off on informed consent.
 
DR. GUTMANN:
Well, Dan you have written a lot about informed consent so why don’t you try to answer that?
 
DR. BROCK:
Unfortunately, I am afraid that is correct.
 
A similar question can arise not even in the developing world context where medical students, for example, when you start talking about informed consent with them, there is always a certain number of skeptics in the group who say, well patients, often they are talking about patients as opposed to research subjects, well they can’t really understand it as well as we do. They would have to go to medical school and do a fellowship and so forth to really understand it. And the right response to that I think is you have to think about what it is that in that case a patient, but in other cases, a research subject, needs to understand. There will certainly be many things that the investigator understands that the research subject doesn’t understand but also has no need to understand.
 
DR. WAGNER:
Let me stop you there. I am just curious. Please, who makes that judgment? Is it the pharmaceutical company that is sponsoring the research? Do we look to third parties to do that? Who makes that?
 
DR. BROCK:
IRBs typically step in to this, in terms of what needs to be a part of the consent process and often the forms but and there is reasonable disagreement about this.
 
I think what a subject wants to know and for that matter what a patient wants to know is basically what is going to happen to me. What are they going to do and what is going to happen to me, and how might that matter in my life?
 
Now, I think that is something that even unsophisticated and not very educated people can understand if the information is given to them, I am inclined to say in an understandable form, but nevertheless.
 
So I think if we think about what we want people either in clinical medicine or in research to understand as a part of their giving informed consent, then I think people are more capable of it than I guess I took your question to suggest.
 
DR. GUTMANN:
Anita? Oh, Susan, please. Please. I’m sorry.
 
DR. LEDERER:
I guess I would agree about the much greater capacity of ordinary people to understand what is relevant to them in decision making.
 
I guess my concern, I mean, I am reminded that what I consider to be the first informed consent law passed in this country was actually in 1957 in Arkansas and it had to do with ensuring that people got the right color blood during a time when everything was racially segregated. And I worry that informed consent is used when you say shift, I think oh, you know, putting the — It is obviating responsibility. It is abandoning the potential subject or patient. And so it is, one could look at sort of the swing back from autonomy beneficence, you know, we have shifted a great deal over the course.
 
DR. WAGNER:
Fine. I respect and appreciate that and particularly for the comments you make on our shores. And maybe I am only concerned about a very small population, perhaps as Guatemalans were at that time, but we continue in this day and age to have peoples for whom their societies, among whom their societies do not have even words, even literate societies. Come on, the Tibetan community in exile is a great example where as they learn science and as they learn medicine, names have to be translated phonetically before they can even be described.
 
I was talking a little earlier with Lawrence who had to leave us about northern Ethiopia with the same sort of situations. He told me in some of their studies they actually stage dramatic presentations and act out what it would mean. I think that is laudable and I think it is wonderful. I think the fact that it has occurred to someone that that is necessary demonstrates that there will be some populations for which we shouldn’t just say oh I think they understand better than you think they do.
 
So I just wanted to push back a little.
 
DR. BROCK:
Just to push back once more, I think that we may have to decide that there are certain contexts in which we can’t do ethical research because we can’t get adequate consent. And you know, we have already decided there are certain things that we don’t want to do, for example, in prisons and in other contexts or with people who are — We can’t do some of the things that Alzheimer’s researchers would like to do by way of brain biopsies and so forth for a variety of reasons. And that could be a part of what you do here is to identify exclusionary conditions, I suppose.
 
DR. GUTMANN:
Good. Thanks for that exchange. Anita.
 
DR. ALLEN:
Thank you. I was struck by a couple of comments that Dan Brock made and I want to follow-up on by asking questions though to Susan Lederer, Dr. Lederer and Dr. Califf.
 
For Dr. Lederer, I mean, you gave this stunning range of examples of unethical, immoral, unjust medical practice and research in the past and suggested there may even be more atrocities we haven’t discovered yet.
 
DR. LEDERER:
I’m sure there are.
 
DR. ALLEN:
You are sure there are. Okay.
 
So what I want to ask you in light of Dr. Brock’s comment that he is sort of an optimist, you seem to be more of a realist, but I want to ask you is there any specific recent event or development which in your view signals, at least signals, a clear departure from the past level of disrespect for non-white, non-American vulnerable populations in research? Is there anything that you think gives the public a ray of hope about the departure from the past and a new thinking about these populations? That is my question for you.
 
For Dr. Califf, you emphasized in your remarks the expense of doing research in the United States. And Dr. Brock made an interesting comment about how if we were to address all of the ancillary needs of developing countries, it would actually increase the cost of research in those countries. I want to try to put these two thoughts together.
 
So maybe the U.S. is actually not too expensive and maybe the developing world is actually not too cheap.
 
DR. GUTMANN:
Susan.
 
DR. LEDERER:
I don’t think you should look to historians for optimism. You are much safer with the philosopher.
 
You know, we had a conversation earlier like when he said he was going to offer the observation that Guatemala couldn’t happen again, I guess today. I’m not so sure that there aren’t circumstances in which it would not happen. But what we can hope for is that the response would be dramatically different and that it wouldn’t be sanctioned in the way that Dr. Cutler, who was not a fringe person by any sense, a very central person with connections to all the important syphilologists that they apparently overlooked it. But is there a historical event or historical development which you think might give us a basis for optimism?
 
FEMALE PARTICIPANT:
You mean other than Presidential Commissions?
 
DR. LEDERER:
That’s a good start. Okay.
 
DR. GUTMANN:
Maybe, I mean, this is a time because just let’s be — Since the question was posed specifically, could Guatemala happen again, there is a list, isn’t there, of things that would prevent Guatemala from happening again. That is, you can’t go in — This was U.S. government-funded research that took a prison population, tried to inject an active syphilis bacteria in it and so on.
 
So I am sure all of you could make a list of things that have happened since then that would prevent something just like this from happening the way Guatemala happened. And if not, I would like to know why not.
 
There are other things. I assumed that, Susan, you were speaking a bit more broadly or metaphorically in warning us that two things. One is that there will be things that happened in the past that come uncovered that we now don’t know about. And secondly that we shouldn’t be complacent. And Dan would agree with this, and if you don’t please tell us. We shouldn’t be complacent that nothing unethical can happen from here on in. Okay? Is that —
 
DR. LEDERER:
Yes, I was calling attention to the fact that we were going to have things —
 
DR. GUTMANN:
These are real people’s lives who were affected. So I want to make sure we are very careful about what we say could and couldn’t happen from here on in.
 
DR. CALIFF:
Wow. I mean, that was a great question. My only comment on it is I don’t think there is a single event. But just in my career, it is night and day what you could do in the “good old days” with no one knowing about it. But there is no 100 percent guarantee. There still will be bad things that will happen.
 
My response to your question is that it is not — I mean my beef is not just that the U.S. system is too expensive but also not the highest quality. So that is what I call a bad buy when I talk with our business school. More expensive and not the best quality.
 
And I think the way to think about it is this. Clinical research, doing a clinical research project is like a construction project in a way, in that it is very labor intensive. It takes people who devote professional time and energy. Each of those people has a salary. And as long as there is a salary differential even at exactly the same level of expertise, until global economics takes care of the problem which I am pretty sure will happen over the next 20 or 30 years, the same piece of work in the U.S. is going to cost a lot more than a piece of work in another place.
 
And so with all good intention we add another layer of oversight that creates the need for another FTE, it makes it even harder to get the research done on a relative basis in the U.S. compared to the other countries, even if they implement exactly the same system that we have.
 
Having said that, I agree with you or the premise that the proper place is probably the middle ground there. I am just saying that the proper place is probably not we are on a pedestal and everybody else needs to come up to what we do.
 
I think we need to examine what we are doing. I see the UK is just done on a national basis and that is a report worth reading for anyone who hasn’t read it. What they have said is we have gone too far. We have got to strip ourselves of things that are not effective in bureaucracy, get more efficient, and then everybody else should try to meet that same standard. That’s my view on it.
 
DR. GUTMANN:
Eric.
 
DR. MESLIN:
I actually agree with the point and it actually gets — I want to be a little bit provocative when asking could it ever happen again. This is the only time I will mention the other NBAC report focusing on human subjects research.
 
We still have an incompletely covering federal oversight system. So when you say could this ever happen again, the answer is of course it could if it was conducted by researchers who were not federally funded or who had no intention to market or license a drug or a device by the FDA. It could happen today. It might be happening today. That may not be the sound bite you want for the meeting but the point is if you are concerned about whether Guatemala could happen again, which of course is several studies and not just one, could happen again in the United States, the answer is, it probably could but you might not know about it. We might not know.
 
But could it happen within — under the umbrella, under the Common Rule? Less likely. Those agencies that are not signatory to the Common Rule? Probably not because they almost all follow it in spirit anyway but we still don’t have complete harmonization between the FDA and the Common Rule.
 
So before we — I mean I am a complete believer in your question. None of us should sit here and say could it happen here and answer well, it might. Ethically it ought never to happen. We have protections in place but we have a leaky system.
 
DR. GUTMANN:
That is very important. That is why I asked the question, so we get out on the table what is it that can and cannot, even though obviously we don’t know everything that is happening. So you are saying it can but not under — it would not be governmentally funded research but it could happen if it weren’t.
 
Nelson.
 
DR. MICHAEL:
So you have described a framework. Many of you have described a framework that goes from investigators, community engagement, goes all the way up through national regulatory or normative bodies and to an international something that would exist that would actually change this debate from what my Nigerian colleagues would say just grammar to actually action.
 
So I would like your opinion about how that could actually happen. Adult male circumcision, one of the three RCTs done in Kenya, I know you know these studies well because they were done just south of where you were, took the Minister of Health of Kenya how many years before they went into a period of being able to actually deploy that modality.
 
Some of that comes, what Susan I think has picked up on maybe comments I made earlier to Dr. Corey about what donors are going to be able to sustain is going to be the second pillar.
 
So, in the long run we all agree that access to care is laudable and maybe that finds somewhere between the best standard of care and local standard of care. If you raise the local standard of care to something that is in between, that is going to make potentially that kind of engagement less sustainable, especially if there is no PEPFAR number three, if the President’s malaria initiative doesn’t get reauthorized, etcetera. I think in the current environment we need to be sanguine about those kinds of realities.
 
So the two questions to you really are how do we actually a commission like this, how do we actually do something, in your view, that isn’t just another report that forms some sort of framework that is sustainable that could actually make sure that we attenuate the possibility that another Guatemala happens.
 
Could we go down the line?
 
DR. GUTMANN:
Eric, you on first. We are going to go down — This is the last question before we go into public comments. So I would ask each of you to answer it, please.
 
DR. MESLIN:
I actually think that something that has been proposed ever since the President’s Commission back in the day proposed and that is a single unified domestic policy in the United States. That would be probably the first important step to get very clear where does the U.S. stand on these issues.
 
DR. BROCK:
The President’s Commission that I was connected with 30 years ago produced about ten different reports and on quite a diverse range of subjects. Some of them we were asked to do by the President and some of them we did on our own initiative because we thought we should.
 
And in thinking about who were they addressed to and what kind of impact would they have, the answers were very different as one went from one report to another. I think the most influential report that we did was the one on deciding about life-sustaining treatment. That was not one that we were asked to do. We did a study on the definition of death and we found that everyone was more interested in life-sustaining treatment, so we took that up.
 
What was the intended audience for that? Well quite a diverse audience with the courts who quoted us at length in many decisions. It was physicians. When I went back to Brown after being in Washington at that point, I went into a case conference at Rhode Island Hospital and the risk manager walked in carrying our report and saying, they say it is okay so it is okay. Now, that is not a very good philosophical argument. But nevertheless, it was a piece of the kind of influence we hoped to have.
 
And so I think you probably should think broadly about how you might want to influence a behavior in this area and the diverse ways you might do that.
 
DR. GUTMANN:
Susan?
 
DR. LEDERER:
Very briefly I guess it would be to create a coherent more unified transparent policy. I think that I don’t often agree that sort of regulations are unnecessary but I think they could be certainly more less opaque to many people.
 
DR. GUTMANN:
Rob.
 
DR. CALIFF:
Just to be consistent, if you can find the magic of defining actionable attributes and ways of measuring them to be built into the system, and as a case example, to do the opposite of whatever was done, the consent system we have in the U.S. where we are all dealing with 15-page, single spaced consent forms and there is ample documentation that everything was covered and yet, if you do the measurement piece and you actually call people after they have given consent and ask them what they know about it, it is very little.
 
Think of actionable attributes that can be measured and then ways of making it a living system that evolves based on the measurements.
 
DR. GUTMANN:
So thank you all again. Thank you very much.
 
[Audience Applause]