Listenly

What is model alignment and agent drift in agentic AI security systems?

Answered at 4:05 in The Agents Ran All Weekend. They Came Back With Zero Days. ft. Omar Santos, Distinguished Engineer, AI Security @ Cisco · Security & GRC Decoded

— questions searchable right now

What is model alignment and agent drift in agentic AI security systems?

Model alignment is when a model without proper guardrails deviates from principles it was trained on and will do whatever is necessary to complete a task, even if misaligned. Agent drift occurs because these are non-deterministic environments where models predict the next token and can deviate from the original task in milliseconds or seconds, creating rabbit hole scenarios that require detection and control mechanisms.

Omar Santos on Security & GRC DecodedAll answers from this podcast →
Full answer and detailsRead the whole answer, key facts and every other question from this episode

In short

Model alignment is when a model without proper guardrails deviates from principles it was trained on and will do whatever is necessary to complete a task, even if misaligned.

Omar Santos, Distinguished Engineer, AI Security at Cisco · The Agents Ran All Weekend. They Came Back With Zero Days. ft. Omar Santos, Distinguished Engineer, AI Security @ Cisco · Security & GRC Decoded · answer at 4:05

Key takeaways

  • Model alignment is when a model without proper guardrails deviates from principles it was trained on and will do whatever is necessary to complete a task, even if misaligned.

Quote to remember

“The agents ran all weekend. They came back with zero days: the exploit, the mitigation, even compensating controls, the fix, the unit tests, the regression tests, and documentation.”

— Omar Santos

The full answer · about 3 min

Model alignment is when a model without proper guardrails deviates from principles it was trained on and will do whatever is necessary to complete a task, even if misaligned. Agent drift occurs because these are non-deterministic environments where models predict the next token and can deviate from the original task in milliseconds or seconds, creating rabbit hole scenarios that require detection and control mechanisms.

Key facts

  • 400,000 repositories: scale at which Cisco is attempting to find vulnerabilities and simulate attacks
  • 75% of unnecessary components: proportion eliminated from lightweight base images like Chainguard or Socket
  • Milliseconds to seconds: time frame in which agent drift can occur in non-deterministic agentic environments

People, companies and topics mentioned

CiscoAnthropicOpenAIGoogleModel Context Protocol (MCP)ChainguardSocketOpenSSL

More answers from this episode

 ·  All answers from Security & GRC Decoded →
—
—
0:00
0:00