AI Governance: Safeguarding Trust and Data Privacy

Updated September 2024

Artificial Intelligence (AI) is continuing to impact nearly every industry by enhancing efficiency, automating tasks, and unlocking new opportunities. However, with great power comes great responsibility, and as the regulatory landscape continues to evolve, it’s essential for organizations to remain steps ahead by establishing a robust AI governance program. From Europe’s AI Act to United States (US) federal and state legislative proposals, one commonality in the new legal and regulatory landscape is the need to establish an AI governance program to ensure responsible and ethical use of AI. By implementing clear guidelines, policies, and oversight mechanisms, organizations can mitigate risks associated with bias, privacy breaches, and unintended consequences while fostering trust among stakeholders. Today, we'll dive into the key steps and considerations when establishing a comprehensive AI governance program.

Review Standards and Regulatory Obligations

Before you embark on your AI governance journey, it's crucial to review your organization’s legal and regulatory obligations as well as applicable AI standards. This includes a meaningful review of the United States National Institute of Standard and Security (NIST) AI Risk Management Framework (AI RMF), the playbooks and underlying roadmaps. The AI RMF applies a risk-based approach and organizations should apply it based on their respective role, anticipated use, size, and complexity. In doing so, the AI RMF traces the entire development or product AI lifecycle. It is a great resource for organizations to utilize when establishing a sustainable AI governance program. The first step highlighted in the AI RMF is establishing Governance - this means creating policies, processes, and procedures based on three key actions:

• Number 1: Map. You should consider why you are using AI in the first place – what's its purpose? Who will be using it? What laws do you need to follow? And what are people's expectations around it? You should also think about where your organization will be deploying AI systems.
• Number 2: Measure. This involves developing metrics for measuring risks associated with your AI system as well as evaluating how effective your current controls are at managing those risks. Regular assessments should be conducted so that any necessary updates can be made along the way.
• Number 3: Manage. You should prioritize identified risks from both mapping out purposes/expectations/settings (Map) and measuring effectiveness (Measure), responding promptly to them, and effectively managing them overall.

Define Use Cases and Set Clear Objectives

An important area to also consider is your organization’s AI use cases. Organizations should ask - are you planning to use AI for data tagging, identifying personally identifiable information, creating more structure to the treasure trove of unstructured data you might have? Or instead, are you planning to use AI to make automated decisions about a person’s health condition, benefits, loan application or employment. Your use case of AI will determine whether you’ll likely have additional requirements and legal obligations versus others.

Having clear objectives and a grasp around your organization’s AI use-cases is crucial when it comes to developing an effective AI governance program. Clear objectives help in several ways. They enable you to prioritize what aspects of AI governance are most important for your organization and ultimately allow you to focus resources on areas that will have the greatest impact. In short, they provide a basis for evaluating whether your AI governance program is meeting its intended goals and if any adjustments need to be made along the way.

Assemble a Cross-Functional, Diverse Team

AI governance is not the responsibility of a single department; it requires cross-functional collaboration and diverse stake holder input. Why is this important? Well, each department brings unique knowledge and skills to the table. Legal professionals versed in data privacy can assess compliance with regulations and protect against potential legal risks. Trust & compliance experts can establish guidelines to adhere to industry standards and ethical principles. IT and security specialists possess technical expertise necessary for implementing AI systems securely, while data scientists and engineers can contribute their analytical capabilities in understanding complex algorithms and models. By building a governance program that involves an array of individuals from different teams across your organization, you can develop an adaptive AI governance program that can easily evolve alongside technological advancements rather than being limited by siloed perspectives, which may hinder agility when responding to the rapidly changing regulatory landscape. It also can foster transparency throughout the organization regarding how decisions related to AI are made and what policies will need to be issued internally and externally. In doing so, this cross functional team ensures the development of an effective AI governance program.

Develop AI Governance Policies and Guidelines

Creating comprehensive AI governance policies and guidelines is crucial to effectively managing the use of AI within your organization’s operations. Whether your AI system is built in-house, or you are utilizing a third-party AI service provider, policies set clear boundaries for personnel to follow. When developing AI policies and guidelines, consider including key components such as:

• Data Privacy and Security: Define how personal and sensitive data will be handled. Define what specific privacy and security controls should apply to your use of AI. If using an AI third-party service provider, consider the privacy and security certifications offered by that provider.
• Ethical Guidelines: Establish principles to ensure AI applications are used responsibly and avoid biases.
• Compliance: Ensure adherence to relevant industry regulations and standards.
• Transparency: Define how decisions made by AI models can be explained to users. If using an AI third-party service provider, confirm whether it will train on your content and, if yes, can you revoke consent for it to do so.
• Accountability: Clarify roles and responsibilities for AI governance.

It’s important to also think about how your AI policies are communicated throughout your organization. Consider utilizing various channels for communicating policies such as email announcements, intranet portal, or even dedicated training sessions. It’s equally important to create an open-door culture where personnel feel comfortable asking questions regarding policy clarification or seeking further guidance when needed. Try identifying a key leader in your organization to assist you in your communication efforts. By having leaders champion your AI policies, you are able to communicate how important these policies are to the organization. Remember, different individuals prefer different modes of receiving information; so, using multiple channels increases the chances of reaching all personnel effectively.

Conclusion

As the impact of AI continues to grow across industries, it is becoming increasingly crucial for organizations to be proactive in establishing a robust AI governance program. By addressing your organization's use of AI through clear guidelines, policies, and oversight mechanisms, you can effectively mitigate risks associated with its implementation. A well-defined AI governance program ensures that ethical considerations are taken into account when developing and deploying AI technologies. It also helps maintain transparency and accountability throughout the entire process. With a strong focus on responsible AI practices, organizations can harness the full potential of this transformative technology while minimizing any negative consequences or unintended outcomes.

At Box, we’ve done the work to establish and implement a meaningful AI governance program. To learn more, we encourage you to read the Box AI Principles and Box AI: Acceptable Use Policy & Guiding Principles. As the AI public policy landscape continues to evolve, our customers can continue to trust Box in supporting their efforts in meeting the highest bar in data privacy, security, and compliance.