Security Considerations – Experiments & Development

Guardrails – The Foundation of Safe AI Systems

Bo / April 4, 2026

Guardrails – The Foundation of Safe AI Systems What Are Guardrails? Rules and constraints that prevent AI systems from operating outside intended boundaries. Core Guardrails: 1. Scope Limitation Only give access to tools when absolutely necessary 2. Authentication Restrictions Require identity verification before interaction 3. Data Access Boundaries Clearly define what each tool can access and do 4. Input Validation Ensure all inputs are safe and expected 5. Tool Usage Restrictions Design tools with narrow, specific purposes 6. Approval Workflows Require approvals for sensitive actions 7. Testing Continuously test the agent for vulnerabilities Final Thoughts AI agent security is not a single feature—it’s a system of layered protections across: Data Tools Identity Interactions The safest agents are designed with minimal exposure, strict controls, and continuous oversight.

Spoofing & Impersonation Risks

Bo / April 4, 2026

Spoofing & Impersonation Risks What Is Spoofing? When a user pretends to be someone else and the agent accepts it without verification. High Risk When: Users self-identify (name/email) No identity verification exists Sensitive data is provided based on claims Prevention: Implement authentication before access Use trusted identity systems Authentication Methods: Username & Password SSO (Single Sign-On) OAuth

AI Consultant: Training and Fundamentals, Security Considerations

Tool Misuse & Unauthorized Actions

Bo / April 4, 2026

What Is Tool Misuse? When users manipulate an AI agent to use tools (APIs) in unintended ways. Risks: • Unauthorised data access • Execution of unintended actions High Risk When: Tools have broad access Instructions lack restrictions No authorization checks exist Prevention: • Use strict and explicit instructions • Limit tool capabilities (narrow scope)

AI Consultant: Training and Fundamentals, Security Considerations

Agent Interaction Vulnerabilities Explained

Bo / April 4, 2026

Agent Interaction Vulnerabilities Explained What Are Agent Interaction Vulnerabilities? These relate to how the agent manages users, permissions, and actions. Main Risks: • Cross-session data leakage • Tool misuse • Identity spoofing Cross-Session Data Leakage What Happens Different users accidentally share the same conversation history. Causes: Poorly designed conversation IDs Non-unique identifiers like: Name + surname Email address Result: One user accesses another user’s chat history and personal data Prevention: • Automatically generate conversation IDs • Ensure IDs are truly unique

AI Consultant: Training and Fundamentals, Security Considerations, Uncategorized

Knowledge File Vulnerabilities & Data Protection

Bo / April 4, 2026

Knowledge File Vulnerabilities & Data Protection What Are Knowledge File Risks? These risks arise from sensitive information stored in files that unauthorized users may access. Personally Identifiable Information (PII) Sensitive data includes: Names, addresses, phone numbers Social security, passport, driver’s license Home address, IP address Bank account and credit card numbers Date of birth Biometric data (fingerprints, facial recognition) Medical records and health information 3 Types of Knowledge Leakage 1. API Data Leak Occurs when your agent sends knowledge file data to third-party providers. Examples of providers: OpenAI Anthropic Google Risk Factors: Direct use of LLM providers Free-tier services (often use data for training) Different legal jurisdictions Weak data retention guarantees Why It’s Risky All data passed to external providers is subject to their policies. It may be: Logged Stored Used for training Prevention: Avoid storing sensitive data in knowledge files Use business-tier LLM providers with retention guarantees 2. Accidental Data Exposure High Risk When: Knowledge files contain PII Instructions don’t restrict data sharing No authentication or access control is implemented 3. Memory Injection What It Is Attackers insert malicious instructions into knowledge files that the agent treats as legitimate. High Risk When: Files are stored in shared locations Multiple editors have access Changes are not tracked Files include examples of agent reasoning/actions Prevention: Restrict file access Implement content review processes Use read-only file formats

AI Consultant: Training and Fundamentals, Security Considerations

Understanding AI Agent Security Fundamentals

Bo / April 4, 2026

Understanding AI Agent Security Fundamentals Introduction AI agents are powerful—but with that power comes responsibility. Security must be intentionally designed, especially when agents process files, access tools, and interact with users. Two Main Vulnerability Categories • Knowledge File Vulnerabilities • Agent Interaction Vulnerabilities