Join GW Engineering's Department of Computer Science for the next installment in their Spring 2026 Colloquium Series! The talk titled "Securing AI Systems in the Real World" will be given by Postdoctoral Researcher at the University of California, Santa Barbara, Yigitcan Kaya!

Abstract 

AI systems have become part of critical infrastructure, powering malware detection, vulnerability analysis, and customer-facing services. While security benchmarks report strong gains in attack robustness and detection accuracy, these improvements often fail to translate into reliable protection in real deployments, where systems face adversaries, distribution shift, and operational constraints. Bridging this gap is the central focus of my work at the intersection of AI and systems security. Guided by a measurement-first philosophy, I study AI systems in deployment conditions to uncover hidden failure modes, vulnerabilities, and broken defensive assumptions, and translate these insights into practical defenses. 

This talk presents two recent works demonstrating how ignoring deployment context causes AI defenses to rely on assumptions that break in practice. First, I examine the emerging ecosystem of web-based AI chatbots built on third-party plugins. I show that widespread implementation flaws invalidate model-level prompt-injection defenses, allowing attackers to inject privileged instructions with success rates far exceeding predictions from existing threat models. Second, I study security classifiers such as malware detectors, which are traditionally evaluated assuming each input has a single correct label. In practice, labels are often context-dependent, reflecting organizational policy, workflow, and risk tolerance. I formalize this mismatch as policy discrepancy, an overlooked source of real-world failure, and introduce a framework that adapts models to operational context using post-deployment feedback.

I conclude by outlining future directions for emerging security challenges in the era of agentic AI, where attackers increasingly leverage AI offensively, and defenders must reason under security ambiguity to ensure trustworthy system behavior.

Bio

Yigitcan Kaya is a postdoctoral fellow at UC Santa Barbara, where he works with Giovanni Vigna and Chris Kruegel. He received his Ph.D. from the University of Maryland, College Park. His research lies at the intersection of trustworthy AI and systems security, with a focus on understanding how AI systems behave under real-world conditions in security-critical applications. His work has appeared at flagship venues in machine learning (ICLR, ICML) and security (IEEE S&P, USENIX Security), and has been featured by outlets including MIT Technology Review and VentureBeat. His research is supported by a U.S. Intelligence Community Postdoctoral Fellowship and an Amazon Research Award.