Short reads exploring specific aspects of post-quantum, autonomous AI, and on-chain security. New pieces roughly monthly per shift.
MCP servers are commonly see as semi-trusted, with tool lists and descriptions reviewed upon install. However, these lists can be changed or malicious, making MCP servers a potential supply chain risk.
AI agents' reasoning traces are supposed to describe how the tool decided to run tools and generate output. However, they aren't as reliable or accurate as they seem.
Most AI security discussions focus on the threats of compromised AI systems. However, an AI agent that is confidently wrong can be just as dangerous.
"Human in the loop is the most common method of managing the security threats of AI systems. However, this is often just security theater.
Do you really trust what your LLM or AI agent is telling you? Model weights, context, and outputs can be manipulated in many ways, introducing lies and biases.
Often, prompt injection attack coverage and defenses focus on direct prompt injection attacks. In agentic workflows, indirect prompt injection is a major risk with the potential for cascading effects.
Prompt injection is the most famous attack against AI systems, but it's not the only one. Memory poisoning introduces persistent changes to AI agent by targeting saved state.
AI guardrails are often (over)sold as a defense against prompt injection and other AI threats. While useful, they're often ineffective and aren't enough for security.