Copper Dome Chronicle: Using AI To Draft Legislation
Welcome to a special edition of the Copper Dome Chronicle, sponsored by Advocatus. We strive for substantive writing with brevity, inspired by the book Smart Brevity. It’s one of the best books about communications and writing we’ve read in years.
Today’s edition is 955 words or less than a 4 minute read. Today we share the results of an experiment using Bard by Google, a generative artificial intelligence (AI) chatbot, used to draft legislation. Note: this post was written by a human, not an AI chatbot. Thanks for reading and sharing; we welcome your feedback and commentary!
Experiment
Advocatus asked Bard five prompts for legislation in South Carolina:
Write legislation providing a state holiday for election day in South Carolina.
Write legislation eliminating compulsory attendance laws in South Carolina.
Write legislation creating a Governor's School for the Skilled Trades in South Carolina.
Write a budget proviso prohibiting the use of Bard by Google on state owned devices in South Carolina.
Write a joint resolution amending the South Carolina state constitution to eliminate the position of the Lieutenant Governor in South Carolina.
For context, these are not any potential bills representative of an Advocatus client nor is any prompt a statement of support or opposition to the underlying policy. We simply wanted to see how well Bard worked. The answer: surprisingly good.
High Quality Outputs
Prompt 3: Write legislation creating a Governor's School for the Skilled Trades in South Carolina
Advocatus Rating: 8/10
This prompt was moderately challenging as Bard was asked to create a new school and there was no previous South Carolina legislation to use. There are three governor’s schools in existence so those statutes may have aided Bard.
The drafts produced were very impressive. All drafts contained a purpose section, an oversight entity, prescribed powers, student admission policy, funding, and an enactment clause. The drafts were a little light on some details that would likely be included, such as employee (teacher) policy, facilities, transportation, and a clause that the school is subject to federal and state school safety and health requirements. For a piece of legislation creating something out of nothing, Bard’s Draft 1 and Draft 2 were excellent first rough drafts.
Prompt 1: Write legislation providing a state holiday for election day in South Carolina.
Advocatus Rating: 7/10
This prompt was the easiest test because it is legislation that has been filed in the South Carolina General Assembly (S.8 in 2021) that is publicly available using any search engine. All three drafts produced the correct output, though the formatting of the text (underline and strikethrough) was not properly used. Additionally, each draft provided some persuasive language about why legislators should support the bill, which could help advocates when creating messaging and narratives.
Quality Outputs
Prompt 4: Write a budget proviso prohibiting the use of Bard by Google on state owned devices in South Carolina.
Advocatus Rating: 5/10
Bard was right on target with Draft 3: “No state funds shall be used to purchase, lease, or otherwise acquire Bard by Google, or any other large language model or artificial intelligence product or service, for use on state-owned devices.” Draft 1 was average but could have been subject to a germaneness point of order (i.e. Senate Rule 24): “No state owned device shall be used to access or use Bard by Google.” Draft 2 was off-the-mark and more fitting of a resolution.
Proviso drafting requires know-how and insight into South Carolina’s rules of germaneness. Bard may eventually learn by using other provisos available online, but for now Bard is at best an easy tool to produce a proviso rough draft. Humans will need to revise the output and it may be less time consuming to simply look at other existing budget provisos for inspiration.
Low Quality Outputs
Prompt 2: Write legislation eliminating compulsory attendance laws in South Carolina.
Advocatus Rating: 3/10
This prompt was the most challenging to rate. Usually when eliminating law a piece of legislation would say “Section XX-X-XXXX is repealed.” But that is not what Bard produced. Instead it amended the existing code section regulating compulsory attendance and simply stated some variation of “There is no requirement for compulsory school attendance in this State.” Bard earns points for finding the correct code section but in our experience, this is likely not how legislative counsel would draft such a bill or amendment. However, if the intent is to create new law instead of repealing existing law, this prompt could be viewed as successful.
Prompt 5: Write a joint resolution amending the South Carolina state constitution to eliminate the position of the Lieutenant Governor in South Carolina
Advocatus Rating: 1/10
This prompt proved to be difficult for Bard. The first failure point was formatting, as none of the drafts included ballot language (see S.95 as an example). Then Bard produced an incorrect draft that maintained the Office of the Lt. Governor but without any powers (Draft 2). While some of the persuasive text was usable for messaging, as a tool to produce a rough draft of legislative text Bard failed with this prompt.
Summary
Overall, Bard’s outputs were surprisingly good from our experience; we would rate its usage 7/10. Writing legislative text is a fairly specialized skill. Each state has its formatting, word preferences, and chamber rules that must be taken into account when drafting legislation. For transparency, we’ve uploaded a PDF file of the prompts and copied the text; the only alteration to the text was font style and size.
We’ll try some other AI experiments this year to see how the technology is progressing. In our opinion, generative AI has a use case as a first draft “writer,” which will save time for advocates, lobbyists, legislators, legislative staff, and agency staff. Additionally, we know and will see more generative AI used to generate advocacy content: correspondence to government officials, social media posts, news releases, guest columns, and other communications. Some federal and state lawmakers in other states have called for transparency if a communication - such as a TV ad - uses generative AI. The National Conference of State Legislatures has a webpage devoted to AI legislation. Whatever one’s position may be on the uses of AI, something everyone can agree upon is more, not less, AI is on the horizon.
