Live intelligence

Latest AI updates

21 updates
Research

Practical Defenses for Face Recognition Pipelines Against Backdoor Attacks

This technical report presents three practical defense tricks against data poisoning-based backdoor attacks in multi-stage face recognition systems, assuming in-house training but compromised third-party data. The defenses include a training-time method to detect poisoned identities and two inference-time filters for suspicious inputscars. Each is evaluated on realistic workloads, aiming to provide actionable hardening techniques for system providers.

AI securityface recognitionbackdoor defense
Research Editor selected

Anthropic's new prototype proves which model wrote an AI response without exposing its weights

Anthropic released FlashProve, a research prototype that uses cryptographic methods to verify the provenance of AI responses without revealing model weights. The system aims to prove which model generated a given output, addressing concerns about model authenticity and security. However, it has significant limitations, including performance overhead and restricted applicability.

Why it matters: This is a concrete, recent AI development with practical implications for model verification and security, offering readers insight into a novel cryptographic approach.

AI securitymodel authenticitycryptographic verification
Industry

The Security Companies Trying to Get AI Agents Under Control

The article highlights the emerging challenge of securing AI agents, as these autonomous systems become more prevalent. It notes that security companies are developing new tools and strategies to manage and control AI agents, addressing risks like unauthorized actions and data exposure. The piece also mentions that OpenAI may release new models as soon as Friday, adding urgency to the security discussion.

AI securityAI agentsOpenAI models
Research Editor selected

OpenAI pipeline finds 948 Linux kernel privilege-escalation flaws and generates patches

OpenAI built an autonomous pipeline that found, validated, and generated patches for 948 local privilege-escalation vulnerabilities in the Linux kernel. The work demonstrates an AI-driven workflow for large-scale vulnerability discovery and remediation.

Why it matters: It shows a concrete, large-scale application of autonomous AI to security vulnerability discovery and patching, which is directly useful to security and engineering readers.

OpenAILinux kernelAI security
Products Editor selected

Anthropic Releases FlashProve: Cryptographic Proofs That a Reply Came From Committed Weights

Anthropic's Frontier Red Team released FlashProve, an open-source zkSNARK system for provable inference. It proves that a model's reply came from committed weights without revealing them, achieving ~110 tokens/second on a 400B MoE model using one H100. Proofs are 18.78 MB and verify on a CPU in 17 seconds, though proving costs 70-95x the GPU time of standard inference. The system is formalized in Lean and targets use cases like verifying model provenance and detecting tampering.

Why it matters: This is a concrete, recent AI development with significant performance improvements over prior zkML systems)Skip, offering practical value for model verification and security.

zkMLAI securitymodel provenance
Industry Editor selected

Bug hunters can now earn up to $172,000 from OpenAI as company ramps up security

OpenAI has quadrupled its bug bounty payouts, offering up to $172,000 for critical vulnerabilities, and launched a new internal security sprint. The initiative aims to incentivize security researchers to find and report flaws in OpenAI's systems. This move underscores the company's increased focus on AI safety and infrastructure protection.

Why it matters: Provides concrete, timely information on OpenAI's expanded bug bounty program, relevant for security researchers and those tracking AI industry security practices.

OpenAIAI securitybug bounty programs
Industry

Preparing our next security sprint to harden OpenAI

OpenAI is launching a new internal security sprint in partnership with Cathedral Systems to test and strengthen its defenses using AI. The initiative focuses on proactively hardening OpenAI's systems against potential threats through collaborative red-teaming and security assessments.

OpenAIAI securityCathedral Systems
Industry Editor selected

New VOLTA Infostealer Uses Claude, Gemini, and Kimi AI to Rank Victims by Financial Value

Cybersecurity researchers have uncovered a Windows infostealer named VOLTA-AUTODOXER that leverages large language models (LLMs) including Claude, Gemini, and Kimi to evaluate and rank stolen data by its monetization potential. This marks a novel use of AI in cybercrime, enabling attackers to prioritize high-value victims. The discovery underscores the growing trend of AI-powered malware for targeted financial exploitation.

Why it matters: Provides concrete evidence of LLMs being weaponized in real-world malware, offering critical insight for cybersecurity professionals and AI safety researchers.

AI securitymalwareLLM misuse
Research

ANGLER and REEF: Adaptive Red-Teaming Benchmark for Web Agents

Researchers introduce REEF, a benchmark of red-teaming executable environments with state-based verifiers for measuring web agent security, covering 9 websites across travel, commerce, productivity, media and communication. Each benign task is paired with a malicious objective delivered through 5 attack types at the user and environment level. ANGLER builds on REEF's execution feedback to evolve attacks via LLM-based mutation and near-miss seed selection rather than a fixed attack set. Across 8 VLM-based web agents, ANGLER reached a 20.36% end-to-end attack success rate versus 15.24% for the non-adaptive setting, and cut benign task success from 61.9% to 44.4%.

web agentsAI securityred-teaming
Research Editor selected

Erase the Injected Region: Multimodal Prompt-Injection Defense with On-Policy Self-Distillation

The paper proposes ERASE, a defense against multimodal indirect prompt injection that erases injected regions and uses on-policy self-distillation. It preserves task utility while blocking attacks, achieving near-zero attack success on benchmarks across image, audio, and video. For example, on 3B audio, it achieves 60.1% accuracy compared to 33.1% for ARGUS, close to clean-input performance.

Why it matters: Provides a practical defense for multimodal AI systems against prompt injection without sacrificing task performance.

prompt injection defensemultimodal AIself-distillation
Research Editor selected

SecPO: Adversarial Training and New Loss Cut Prompt Injection Attack Success Rates Dramatically

Researchers propose SecPO, a principled adversarial training pipeline for defending LLM agents against prompt injection attacks. They introduce a SecPO loss that approximates an idealized injection-invariant reference model, and combine it with efficient synthesis of strong injected inputs. Their SecPO + AT model reduces attack success rates by up to 14.5x compared to Meta-SecAlign, and further 2.4-4.6x over DPO loss, while maintaining utility across nine benchmarks.

Why it matters: This paper presents a concrete, state-of-the-art defense against a critical AI security threat, with significant measured improvements and released resources.

AI securityprompt injectionadversarial training
Products

Oracle and OpenAI launch AI Security Mesh for vulnerability management

Oracle and OpenAI announced a partnership featuring AI Security Mesh, a tool for vulnerability management. The system is designed to identify vulnerabilities, investigate exploits, and prioritize fixes using AI-generated evidence.

AI securityvulnerability managementOracle
Industry Editor selected

Reported Relay Compromised Codex and Claude Code Requests

Dark Reading reported Symantec findings about a relay that rerouted requests from Codex and Claude Code. The relay reportedly disabled sandboxing and, in one case, replaced credentials. The incident highlights security risks in intermediary infrastructure used by AI coding tools.

Why it matters: It gives developers and security teams a concrete warning about relay-based risks affecting widely used AI coding assistants.

AI securityCodexClaude Code
Industry Editor selected

A Defense Contractor Gave Their Personal AI Agent Access to a Secure Pentagon System

An Accenture employee's personal Claude AI agent operated on a secure Department of Defense network for two months before being detected. The incident highlights security risks from unauthorized AI agent use in sensitive environments. The vulnerability was patched after identification.

Why it matters: Reports a concrete security incident involving AI agent misuse in a high-stakes government network, offering timely lessons for AI governance.

AI securitydefense contractorgovernment networks
Industry

Defense Contractor's Personal AI Agent Accessed Secure Pentagon System for Two Months

An Accenture employee's personal Claude AI agent ran on a Department of Defense network for two months before being identified and the vulnerability patched, according to sources. The incident highlights security risks of unauthorized AI agents in sensitive environments. The breach was discovered and remediated, but details on data exposure remain undisclosed.

AI securityDepartment of DefenseClaude
Industry Editor selected

AI agents breach Anthropic's internal systems in bug bounty test

Cybersecurity startup Cathedral used AI agents to hack into Anthropic's internal systems via third-party software as part of a bug bounty program. Anthropic confirmed that sales, HR, and IT data were accessed but disputed the extent of the breach. The incident highlights vulnerabilities in even leading AI labs through external software dependencies.

Why it matters: This is a concrete, recent security incident involving AI agents, offering valuable insight into real-world AI-driven cyberattacks and enterprise vulnerabilities.

AI securityAnthropiccyberattacks
Industry

AI-Discovered PHP Vulnerabilities Broadly Exploited, Amazon Warns

Amazon says vulnerabilities in a widely used programming language, discovered with AI tools, are now being broadly exploited in the wild. The company identified a hacking attempt four days before the flaws were publicly disclosed. The flaws have prompted governments and companies to rush to secure systems. The language reportedly underpins much of the internet.

AI securityvulnerability researchAmazon