When the tools get smart, the rules must too!
Artificial Intelligence (AI) has quickly evolved from a futuristic concept into a powerful force reshaping industries, and cyber security is no exception. With AI now enabling both advanced defence systems and sophisticated attacks, organisations face a critical need to rethink how they govern risk and maintain compliance in this fast-changing environment. This is where AI in GRC (Governance, Risk, and Compliance) becomes essential, offering both new opportunities and significant challenges in a fast-changing environment.
As Yuval Noah Harari notes in Nexus, humanity increasingly finds itself in the position of the Sorcerer’s Apprentice – conjuring technologies we may not fully understand or control. The image of Mickey Mouse overwhelmed by an enchanted broom he set in motion is more than just a cartoon; it’s a cautionary metaphor for the unintended consequences of unleashing autonomous systems.
Navigating the Risk
In cyber security, this risk is especially real: the same AI tools designed to protect can also be used to deceive, infiltrate, and cause harm. Against this backdrop, traditional Governance, Risk, and Compliance (GRC) frameworks must evolve to meet the challenge.
As AI systems become more autonomous, particularly in the field of cybersecurity, these philosophical concerns are no longer just abstract. Real-world breaches have already demonstrated AI’s dual nature: its capacity to both safeguard and undermine. In Hong Kong, scammers used deepfake technology to convincingly impersonate a company’s CFO, instructing an employee to transfer $25 million to a fraudulent account. Similarly, North Korean operatives employed AI tools to assume fraudulent IT roles, infiltrating multiple companies and exfiltrating sensitive data. AI-generated phishing attacks, tailored with extraordinary precision, are bypassing traditional security measures, further illustrating AI’s ability to facilitate adaptive, fast-moving attacks that challenge conventional defences.
The Evolving Threat Landscape: AI’s Role in Cyber Security Breaches
AI-driven cyber attacks are becoming more common and complex. Beyond deepfakes and phishing, AI systems are now being used to automate the discovery of network vulnerabilities. These AI tools can rapidly scan vast amounts of data, identify weaknesses and exploit them far faster than human hackers. This shift is particularly concerning for organisations relying on traditional security measures. Firewalls, intrusion detection systems and antivirus software are increasingly ineffective. AI-powered systems can bypass these defences, making it harder for cyber security teams to detect and mitigate attacks in real time.
For example, in AI-enhanced phishing, attackers no longer rely on sending generic emails to thousands of people in the hopes of catching a few. Instead, they use AI to craft highly personalised, context-aware emails that mimic the writing styles of trusted colleagues or executives. The sophistication of these attacks makes it difficult for individuals to spot fraud, especially when the emails are tailored to reflect their daily work patterns or recent interactions. This is just one example of how AI renders traditional defence mechanisms less effective.
Another example is a malicious PowerShell script, suspected to have been AI-generated, deployed in targeted attacks. The use of AI in malware development introduces new techniques that bypass conventional security measures, highlighting the urgent need for advanced cyber security strategies to combat AI-driven threats.
The reality is that traditional GRC frameworks, designed for manual processes and human decision-making, are ill-equipped to handle fast-paced, AI-driven threats. As AI continues to evolve, organisations must reconsider how they approach GRC in the context of cyber security.
The Governance Challenge: Ethical and Regulatory Risks in AI Deployment
AI’s growing influence in GRC brings with it ethical and regulatory challenges. As AI systems become more integrated into decision-making processes, issues like algorithmic bias, lack of transparency and accountability emerge.
AI systems are often described as “black boxes,” making decisions based on complex algorithms that even experts may struggle to understand. This opacity is particularly concerning when AI systems are used to make decisions that affect individuals or organisations, especially in sectors like hiring, lending and security.
For example, AI systems used for fraud detection or risk management may unintentionally reinforce existing biases if trained on biased data. If the training data reflects historical inequalities, the AI may perpetuate or even amplify these biases, raising important questions about fairness and accountability, especially in areas like finance, healthcare and criminal justice, where AI decisions have significant real-world consequences.
From a regulatory standpoint, the rapid pace of AI development is outpacing the creation of laws and frameworks to govern its use. While regions like the European Union have made strides with AI-specific regulations, such as the AI Act, the regulatory landscape remains fragmented.
Without global standards, organisations must navigate a confusing array of local and international laws, making it difficult for GRC professionals to ensure AI systems comply with all relevant laws and ethical guidelines.
AI as a Tool for Strengthening GRC: Bridging the Compliance Gap
Despite these challenges, AI offers substantial potential to strengthen GRC frameworks. AI-driven tools can improve compliance by automating the enforcement of policies and providing real-time insights into compliance status.
One powerful example is Retrieval-Augmented Generation (RAG), an AI-powered assistant that answers policy-related questions in natural language. With RAG, employees can instantly access accurate, context-specific answers to questions like, “Can I connect my work laptop to public WiFi?” or “Can I share this document with a third party?” This reduces the risk of compliance violations due to misunderstandings and improves efficiency.
Moreover, AI can help organisations proactively identify risks and vulnerabilities. By continuously learning from historical data and adapting to new threats, AI can flag potential issues before they escalate. For instance, AI could identify emerging cyber security risks based on patterns in attack data or unusual system behaviour, allowing GRC teams to take preventive measures before a breach occurs. This proactive risk management is something traditional systems cannot provide.
To fully realise these benefits, however, GRC teams must ensure that AI systems are integrated in ways that uphold transparency, fairness and accountability. AI’s decision-making processes must be explainable and there must be mechanisms to monitor and audit the system’s performance. Organisations should also ensure that their AI systems are continually updated to reflect the latest regulations, ethical guidelines and best practices.
The Road Ahead: Governance and Control of AI in GRC
As AI continues to evolve, its integration into GRC frameworks presents both significant opportunities and substantial risks. Leaders in GRC must develop new strategies to govern AI systems, ensuring they are used responsibly, ethically and in compliance with relevant laws and regulations. AI has the potential to transform how organisations approach risk management and compliance, but it must be handled with care. This involves setting up strong governance structures that prioritise transparency, fairness and accountability, ensuring AI systems align with organisational ethical standards.
In the next blog in this series, we will explore how RAG-based bots can “bring policy back to life,” providing organisations with real-time access to critical policy information while enhancing compliance efforts. By leveraging AI responsibly and strategically, GRC leaders can better manage risks, streamline compliance and foster a more agile and secure organisational environment.
Author: Subodh Chettri
Bibliography
- Harari, Yuval Noah. Nexus: A Brief History of Information. London: W. W. Norton & Company, 2020.
- Christian, Brian. The Alignment Problem: Machine Learning and Human Values. W. W. Norton & Company, 2020.
- Russell, Stuart. Human Compatible: Artificial Intelligence and the Problem of Control. Viking, 2019.
- European Commission. “Artificial Intelligence Act.” 2021.
- Bostrom, Nick. Superintelligence: Paths, Dangers, Strategies. Oxford University Press, 2014.
- Lomas, Natasha. “Hackers Use Deepfake AI to Mimic Voice of CEO in $25 Million Fraud Attack.” TechCrunch, 2021.
- Zetter, Kim. “North Korea’s Alleged Cybercriminals Used AI to Get IT Jobs and More.” Wired, 2021.
- Shwartz, Laura. “AI-Generated Phishing: A Growing Threat.” Financial Times, 2022.
- Proofpoint. “Hackers Deploy AI-Written Malware in Targeted Attacks.” BleepingComputer, October 2024.
Further Reading:
How AI can be used in Cyber Security Threat Detection, Investigation and Response