Documentation is a crucial part of the AI development process. It ensures that the AI behaves ethically and is transparent to both stakeholders and auditors.
The aim of this study is to identify the requirements for good AI documentation and the limitations & challenges associated with this process. In addition, we want to show how a suitable approach can be implemented.
Documentation Requirements
Documents, like invoices, legal contracts, receipts, and other documents, are an integral part of enterprise business processes. However, data locked in these documents creates bottlenecks for information processing and impedes digital transformation.
Consequently, enterprises need solutions that can unlock the value of these documents. AI is a promising technology that can help solve these problems.
Aside from the complexity of AI, there are many other challenges associated with integrating it into an existing system. Especially in sensitive and regulated use cases (e.g. pharmaceutical production or commercial banking), compliance requirements often represent a significant adoption barrier.
To get an insight into how AI is documented in these use cases, interviews were conducted with practitioners from commercial banks and consultancies. Additionally, two AI specialists from software companies were interviewed.
Documentation Guidelines
Suitable documentation guidelines for AI are essential to ensure fairness, accountability and transparency (FAT) when AI is used in regulated environments. However, current standards from software engineering are not suited to collect the required evidence.
Interviewees in this study point out that the additional technical details of AI development as well as the performance of the AI need to be documented. This is to ensure that potential biases that could distort the decision-making process of the AI are detected earlier and that they do not recur.
In addition, interviewees also highlight the need to document important design decisions. They argue that this would help to ensure that the AI does not make decisions that are unintended or illegal in a given context.
Moreover, the research suggests that companies deal with AI documentation differently depending on the risk level of their use cases as well as on their AI adoption. This is particularly true for the prepared adopters, which include one interviewee from the banking and finance industry as well as two interviewees from software companies.
Documentation Process
Documentation is a central process in software engineering. It is an essential source of audit evidence and can help ensure accountability of AI applications.
Suitable documentation also forms the basis for internal and external audits of AI (Ellul et al., 2021; Raji et al., 2020). However, the lack of appropriate documentation guidelines is a significant barrier to AI adoption in practice.
Our study explored the current state of AI documentation in regulated use cases by conducting semi-structured interviews with four interview partners from banks, consultancies, and software companies. We identified four types of use cases:
Documentation Tools
Software documentation is essential for ensuring that your users have the right information about your product and can easily use it. It can also help your developers create and maintain high-quality code that reflects your business needs.
Creating software documentation can be time-consuming and complicated, so it’s important to find a tool that makes the process quick and easy. Luckily, there are several tools that can help you do this.
For example, ChatGPT is a great AI tool that can automatically generate documents like user stories, PRDs, API docs, and other technical documents. However, it’s important to keep in mind that AI will never replace human input and editing.
Document AI can process your documents to extract structured metadata and other valuable information, such as text and layout details. There are many processor types, including Optical Character Recognition (OCR), form parsing, splitting, classification and entity extraction. Each processor is designed for a different task and can handle text and layout data for specific document types.