Artificial Intelligence
Cloud Security Alliance: Best Practices for Securing AI Systems
The Cloud Security Alliance (CSA), a not-for-profit organization whose mission statement is defining and raising awareness of best practices to help ensure a secure cloud computing environment, has released a new report offering guidance on securing systems that leverage large language models (LLMs) to address business challenges.
Aimed at system engineers and architects along with privacy and security professionals, the Aug. 13 report, titled “Securing LLM Backed Systems: Essential Authorization Practices,” describes LLM security risks associated with designing systems, outlines design patterns for extending LLM system capabilities and explores pitfalls in authorization and security.
CSA said that while there has been rapid adoption of LLMs to tackle diverse business problems, guidance and best practices are scarce, especially when external data sources and using the LLM for decision-making processes are involved.
“System designers can use this guidance to build systems that utilize the powerful flexibility of AI while remaining secure,” said CSA.
One easily digestible takeaway from the sprawling report is a list of fundamental principles and best practices — some tailored specifically to LLM systems — that CSA used for its recommendations:
- Output Reliability Evaluation: LLMs may produce unreliable results; their use should be carefully evaluated depending on the criticality of the business process involved.
- Authorization: The authorization policy decision and enforcement points should always reside outside the LLM to maintain security and control.
- Authentication: The LLM should never be responsible for performing authentication checks. The authentication mechanism used in the broader system should handle these checks.
- Vulnerabilities: Assume LLM-specific attacks like jailbreaking and prompt injection are always feasible.
- Access: Enforce least privilege and need-to-know access to minimize exposure and potential damage control lapses.
Those five items provide the basis for “best practices and considerations” and “pitfalls and anti-patterns” for a number of architecture design patterns for LLM-based systems, including:
- Retrieval-Augmented Generation (RAG) Access Using a Vector Database
- RAG via API Calls to External System
- LLM Systems Writing and Executing Code
- LLM-Backed Autonomous Agents
Autonomous Agents
The latter is one of the hottest areas in AI right now as it allows LLMs to go beyond simply responding to user prompts to actually take independent actions based on those prompts. While extremely useful, such authority comes with concerns and risks.
“Agent-based frameworks are still in their infancy, and many challenges and limitations remain when building agents,” the report said. “Some of these challenges include having to adapt a role to effectively complete tasks in a domain, being able to do long-term planning, reliability or knowledge limitation. LLM agents also face the already mentioned challenge of non-determinism, which, while beneficial for generating creative ideas, poses risks in scenarios requiring high predictability.”
