Can AI be used in preparation for examination for discovery?
In law, where time is of the essence and precision is key, embracing technology can be a game-changer. Breedon Litigation prides itself on being at the forefront of integrating cutting-edge technology into our practice, believing that it not only saves time but also reduces costs for our clients. Recently, I wanted to push the limits of what I thought technology, and in particular AI, could do. Could ChatGPT assist me in the preparation of my upcoming examinations for discovery? Honestly, I was skeptical, but the results surprised me. In this blog post, I will discuss my experience with ChatGPT and what I learned along the way.
ChatGPT and Ethical Considerations
One of the first challenges I faced was ensuring that my use of ChatGPT adhered to the ethical standards of client confidentiality and data privacy.
In navigating this ethical landscape, I was guided by the Technology Guideline provided by the Law Society of Ontario (“LSO”). The Guideline stresses the importance of employing technology in client service functions, advocating for its use to enhance service delivery in a conscientious, diligent, timely, and cost-effective manner. All good things.
However, in seemingly the same breath, the Law Society cautions the need for lawyers to maintain confidentiality and privilege when using electronic means of communication. This involves being vigilant about the risks of disclosure, employing security measures such as encryption, and securing information on cloud services. Lawyers are also advised to develop law office management practices to guard against inadvertent discovery or disclosure of confidential information transmitted electronically.
So, essentially LSO says: use technology but make sure not to sacrifice confidentiality when doing so. Helpful.
I decided to be overly cautious in interpreting these guidelines, particularly because I find
ChatGPT’s privacy and data collection policies to be rather opaque. I ensured that all information fed into the system was either public record, devoid of any identifying details, or non-private in nature. As you will see, this definitely limited the way I could use ChatGPT, but safe is better than sorry.
ChatGPT as a Virtual Expert
I started my examination prep how I usually do – review any preliminary expert opinions and see whether there is any helpful literature that can guide my questioning of the defendant.
I ended up finding an article from a reputable medical journal that was applicable to my client’s case. I read it and came to my own conclusions about what it meant and how It helped us. However, I decided to turn to ChatGPT for additional insights. I uploaded the article to a new chat and asked ChatGPT general questions related to my client’s case.
“Patient Y presented to Dr. X with the following symptoms. Based on the provided article,
what diagnosis should Dr. X have provided to Patient Y? Provide the page number from
the article where you sourced your information from. If you cannot answer this question
using information in the article provided, do not guess.”
“If Dr. X did an examination of Patient Y and noted the following, would the diagnosis of Z
still have been appropriate?”
“What about the location of the lesion that Dr. X noted; if it were located here vs here,
would that make a difference in the ultimate diagnosis?”
The results were pretty great. ChatGPT not only provided information that aligned with our expert’s opinion and my review of the article, but also was able to engage in a dynamic dialogue with me, allowing me to delve deeper into the matter. It’s worth noting that this level of interaction was possible thanks to ChatGPT 4’s advanced capabilities, which are a significant leap from its predecessor.
The Art of Prompt Engineering
Coming off the high of using ChatGPT as expert, I turned to the meat of my examination prep – prepping my questions for the defendant. Each lawyer has their own way of doing this. Some will simply list the areas they want to ask about, while others will write out each question verbatim. My style falls into the latter category. I find that the more I prepare for my examinations (and trials for that matter) the better able I am to be flexible and creative on my feet. It is what works for me, and I am sticking with it.
The downfall of writing each question is the time it takes to do it. My hope was that ChatGPT could help me reduce the time I spent on this task, allowing me to focus more on strategy than on question formation.
And so, I opened a fresh chat and put in the following prompt:
“I am a lawyer. I represent Patient Y in a medical negligence case against Dr. X. Patient Y
alleges that Dr. X misdiagnosed his medical condition. What questions should I ask in the
examination for discovery of Dr. X?”
I then sat back and waited for what I assumed would be the perfect answer.
Not so much.
The questions that ChatGPT provided me were not bad. Not bad at all. But they were very general, often repetitive, and not comprehensive enough for a discovery. Obviously though, right? Look at the question I gave to ChatGPT. I am not sure what I was thinking would happen with such a threadbare prompt.
My experience highlighted the importance of ‘prompt engineering’ – the skill of crafting effective prompts to elicit the most useful responses from ChatGPT. I learned quickly that I had to be much more specific in the prompts I provided. Ideally, I would have uploaded a memo or chronology to ChatGPT that gave it the context necessary to really deep dive into the case and develop questions. I did not do that though because all of my work product had information in it that identified my client and the nature of the case.
I did end up engineering prompts that were detailed enough to guide ChatGPT but vague
enough to maintain confidentiality. Examples of this were:
“Dr. X is a specialist in this type of medicine. What questions should I ask him in examination
for discovery that elicit his experience in this area?”
“Patient Y saw Dr. X on this day. At the appointment, Patient Y described the following
symptoms. Dr. X recorded in his notes that he did an examination of Patient Y and noted
the following. What questions should I ask Dr. X about his examination of Patient Y?”
“Dr. X diagnosed Patient Y with the following. What questions should I ask about this
“Patient Y consented to treatment based on the diagnosis and medical advice provided
by Dr. X. What questions should I ask about the conversation that Dr. X had with Patient Y
about the diagnosis and recommended treatment?”
“Re-word the following question to be simpler and easier to understand”
In the end, I got questions that, while still general, were much more specific and targeted than
before. I ended up using about 25% of the questions generated by ChatGPT verbatim, which I
thought was pretty good. Many other questions I had to rejig, but still found them useful in creating the overall line of questioning.
In the future, I may try creating a memo or timeline without identifying information. It will be
annoying I am sure, to have to give aliases to the various characters in the story. But this will allown me to upload the memo to ChatGPT which I am sure will help in prompting the creation of questions with more specificity. With that said, even with a memo, prompt engineering is a skill. It takes practice to get it right. Don’t be discouraged if, at first, you don’t get the output from ChatGPT that you were looking for. The problem might be you and not ChatGPT.
Understanding ChatGPT’s Limitations
While ChatGPT proved helpful, it clearly has its limitations.
Prompt engineering was often frustrating and time consuming (though I do think, over time, this does get easier).
The questions ChatGPT generated for the examination for discovery, though insightful, were
The Benefits: Beyond the Blank Page
The most significant benefit of using ChatGPT was overcoming the initial hurdle of the blank page. Instead of starting from scratch, I was able to refine and customize the questions ChatGPT generated. This not only saved time but also helped me understand the case better and identify relevant areas of questioning.
I have also talked a lot about prompt engineering. Though it was often frustrating to provide a
prompt that did not elicit the answer I was looking for, the process of generating prompts turned out to be a valuable exercise in case analysis. You really need to understand what is important about your case and what is relevant about a particular line of questioning in order to develop good prompts. Like I had hoped, when done correctly, using ChatGPT allowed me to spend less time on forming the perfect question, and more time on understanding the facts, reviewing the law, and forming a discovery strategy.
Looking Ahead: Incorporating ChatGPT in Future Preparations
Would I use ChatGPT again for preparing for discoveries? Absolutely. It proved to be a time-saving tool, and I ended up using about 25% of ChatGPT’s suggestions verbatim. ChatGPT’s ability to rephrase and format questions was incredibly helpful. And, most importantly, in developing prompts, I was able to spend valuable time thinking through the case and what I wanted to achieve at discovery.
My experience with ChatGPT in preparing for this examination was overall positive. No need to fear – ChatGPT is FAR from replacing lawyers. But, when used wisely, AI can significantly augment the legal preparation process, saving time and enhancing the quality of work.
Visit our Insights page for more articles, news and events – https://breedon.ca/insights/