On 27 November 2023, the UK National Cyber Security Centre published Guidelines for Secure AI System Development. These were developed together with the US’s Cybersecurity and Infrastructure Security Agency and have been endorsed by agencies from 17 other countries. The Guidelines point out that as well as existing cyber security threats, AI systems are subject to new types of vulnerabilities including the use of the AI to perform unauthorised actions or to extract sensitive AI model information.

The Guidelines use the term “AI” to refer specifically to machine learning, which is defined as applications that:

• involve software components (models) that allow computers to recognise and bring context to patterns in data without the rules having to be explicitly programmed by a human; and
• generate predictions, recommendations, or decisions based on statistical reasoning.

The Guidelines apply whether the AI system has been created from scratch or built on top of tools and services provided by others, and whether based on models hosted by an organisation or by making use of application programming interfaces.

The Guidelines provide recommendations in four areas, namely, secure design (e.g. understanding risks and threat modelling), secure development (e.g. supply chain security and documentation), secure deployment (e.g. protecting infrastructure and models from compromise, threat or loss and developing incident management processes) and secure operation and maintenance (e.g. logging and monitoring and update management). In each case, the AI provider should implement the most secure option by default but, where risks cannot be mitigated, the provider should inform users down the supply chain of such risks and how to use the AI system safely.

Since providers often incorporate software, data, models or remote services from third parties into their own systems, it is the providers of AI components that are expected to take responsibility for the security of users further down the supply chain.

For more information, click here.