Imagine your organisation deploys an AI system to screen job applicants. The system is efficient, fast, and reduces the workload for your HR team. But what if, unbeknownst to you, the AI is unintentionally favouring candidates from certain backgrounds while excluding others? Without proper governance, such biases can go unnoticed, leading to legal risks, reputational damage, and a loss of trust among employees and customers.
AI governance is the framework that ensures AI systems are developed, deployed, and used responsibly. It’s not just about avoiding risks—it’s about building trust, ensuring fairness, and aligning AI with your organisation’s values and legal obligations. For leaders and professionals, understanding governance is key to leveraging AI’s benefits while mitigating its potential harms.
Emotional intelligence (EI) plays a critical role in AI governance. Governance isn’t just a technical or legal checkbox—it’s about understanding the human impact of AI systems. Leaders with high EI are better equipped to:
Empathise with stakeholders: Recognise how AI decisions affect employees, customers, and the broader community. For example, if an AI system is used to allocate resources, emotionally intelligent leaders will consider how those decisions are perceived and their potential to create inequities.
Foster a culture of responsibility: Encourage teams to take ownership of AI systems, ensuring they are designed and used ethically. This includes promoting psychological safety, so employees feel comfortable raising concerns about AI biases or unintended consequences.
Navigate ethical dilemmas: AI often presents complex ethical questions, such as balancing efficiency with fairness. Emotionally intelligent leaders can weigh these trade-offs thoughtfully, considering both the data and the human implications.
To govern AI responsibly, organisations should adopt a set of core principles. These principles provide a foundation for developing policies, processes, and accountability mechanisms. Below are the most critical ones:
AI systems must be designed to treat all individuals equitably, regardless of their background, race, gender, or other characteristics. This requires:
Diverse datasets: Ensure training data represents the full range of users and scenarios the AI will encounter.
Bias audits: Regularly test AI systems for discriminatory outcomes, using both technical tools and human review.
Inclusive design: Involve diverse teams in the development process to identify blind spots and ensure fairness.
Example: A financial institution using AI to approve loans must audit its system to ensure it doesn’t disproportionately reject applications from minority groups. If biases are found, the system must be retrained or adjusted.
AI systems should not be "black boxes." Stakeholders—whether employees, customers, or regulators—need to understand how decisions are made. This includes:
Clear documentation: Maintain records of how AI models are built, what data they use, and how they arrive at decisions.
Explainable AI: Use models and tools that can provide understandable explanations for their outputs, especially in high-stakes areas like hiring or lending.
Communication: Be transparent with users about when and how AI is being used, and what its limitations are.
Example: If an AI system is used to evaluate employee performance, managers should be able to explain how the system works and what factors it considers. This builds trust and allows employees to address any inaccuracies.
Organisations must take responsibility for the actions and decisions of their AI systems. This means:
Clear ownership: Assign accountability for AI systems to specific teams or individuals, from development to deployment and monitoring.
Redress mechanisms: Provide channels for users to challenge or appeal AI-driven decisions, such as a rejected job application or loan denial.
Legal compliance: Ensure AI systems comply with relevant laws and regulations, such as GDPR in the EU or the Equality Act in the UK.
Example: If an AI-powered hiring tool is found to discriminate against certain candidates, the organisation must take corrective action, such as retraining the model or discontinuing its use.
AI systems often rely on vast amounts of data, some of which may be sensitive. Governance must ensure that:
Data protection: Personal data is collected, stored, and processed in compliance with privacy laws (e.g., GDPR).
Minimisation: Only the data necessary for the AI’s purpose is collected and retained.
Security: AI systems are protected against breaches, tampering, or misuse, with robust cybersecurity measures in place.
Example: A healthcare provider using AI to diagnose conditions must ensure patient data is anonymised and securely stored, with access restricted to authorised personnel only.
AI should augment, not replace, human judgment. Governance frameworks must include:
Human-in-the-loop: Ensure humans review and validate AI decisions, especially in high-stakes or sensitive areas.
Continuous monitoring: Regularly assess AI systems for performance, biases, and unintended consequences, with human oversight to intervene when necessary.
Training and awareness: Equip employees with the knowledge and skills to use AI responsibly and ethically.
Example: In a customer service setting, AI chatbots can handle routine queries, but complex or emotionally charged issues should be escalated to human agents for resolution.
Implementing AI governance doesn’t have to be overwhelming. Start with these practical steps:
Create a clear, organisation-wide policy that outlines your principles, roles, and responsibilities for AI use. This policy should align with your organisation’s values and legal obligations. Include guidelines for:
Approval processes for new AI systems.
Data collection, storage, and usage.
Bias testing and mitigation.
Transparency and explainability requirements.
Form a cross-functional team—including legal, HR, IT, and leadership representatives—to oversee AI governance. This board should:
Review new AI projects for compliance with ethical and legal standards.
Monitor existing AI systems for biases, risks, or unintended consequences.
Provide guidance and training to employees on responsible AI use.
Schedule periodic audits of your AI systems to ensure they remain fair, transparent, and compliant. Audits should include:
Technical testing: Use tools to detect biases, errors, or performance issues in AI models.
Human review: Involve diverse teams to assess the real-world impact of AI decisions.
Stakeholder feedback: Gather input from employees, customers, and other stakeholders to identify concerns or areas for improvement.
Ensure all employees—especially those working with AI—understand the principles of responsible AI use. Training should cover:
The basics of AI and how it works.
Ethical considerations, such as fairness, transparency, and accountability.
Practical skills for identifying and mitigating biases in AI systems.
Your organisation’s AI policies and governance framework.
Governance is most effective when it’s embedded in your organisation’s culture. Encourage employees to:
Ask questions: Promote a culture where employees feel comfortable raising concerns about AI systems.
Take ownership: Empower teams to take responsibility for the AI tools they develop or use.
Collaborate: Break down silos between technical, legal, and business teams to ensure AI is developed and used ethically.
A global retail company implemented an AI-driven system to personalise marketing offers for its customers. The system analysed customer data, including purchase history and browsing behaviour, to tailor promotions and recommendations. While the system boosted sales, the company soon received complaints from customers who felt the offers were invasive or discriminatory.
Upon investigation, the company discovered that the AI system was inadvertently targeting certain demographic groups with less favourable offers. For example, older customers were receiving fewer discounts on high-value items, while younger customers were being steered toward trendier, but more expensive, products. This raised concerns about fairness and age discrimination.
The company took the following steps to address the issue:
Paused the system: Temporarily halted the AI-driven marketing to prevent further harm.
Conducted an audit: Reviewed the system’s training data and algorithms for biases. They found that the data used to train the model was not representative of all customer groups, leading to skewed outcomes.
Retrained the model: Updated the training data to include a more diverse range of customer profiles and behaviours. They also adjusted the model’s objectives to prioritise fairness alongside profitability.
Implemented oversight: Introduced a human review process for AI-generated offers, ensuring that a team of marketers and ethicists validated the system’s outputs before they were deployed.
Communicated transparently: Informed customers about the changes and invited feedback on the new system. They also provided an option for customers to opt out of AI-driven personalisation if they preferred.
The revised system not only reduced biases but also improved customer satisfaction. By prioritising fairness and transparency, the company rebuilt trust with its customers and demonstrated its commitment to responsible AI use.
Imagine your organisation is considering deploying an AI system to automate performance reviews for employees. The system will analyse data such as project completion rates, feedback from colleagues, and self-assessments to generate performance scores and recommendations for promotions or development opportunities.
Question: What governance principles and practical steps would you implement to ensure this system is used responsibly? Consider the following:
How would you ensure the system is fair and inclusive?
What transparency measures would you put in place?
How would you maintain accountability and human oversight?
Take a moment to reflect on how you would address these challenges in your own organisation.
