Advent of Cyber 2025 - Day 8 - Hacking AI


🎄 Day 8 – Agentic AI Hack

Restoring SOC-mas in Wareville

Introduction — When AI Goes Rogue

Artificial Intelligence has evolved rapidly—from simple chatbots responding to predefined prompts to agentic AI systems capable of planning, reasoning, and executing actions autonomously.

In Wareville, this evolution has gone wrong.

The Christmas Calendar AI agent has been compromised by Sir BreachBlocker III. Instead of marking Christmas on December 25th, the system now displays Easter, throwing the entire SOC-mas operation into chaos.

With McSkidy missing, the only option left is to investigate, understand, and exploit the AI agent’s weaknesses to restore Christmas.

🎯 Objectives

In this challenge, we aim to:

  • Understand how agentic AI systems function

  • Identify weaknesses in autonomous AI design

  • Exploit insecure AI tool usage

  • Restore the correct holiday configuration

  • Recover the final flag

🤖 What Is Agentic AI?

Agentic AI goes beyond traditional conversational models. These systems can:

  • Plan multi-step tasks

  • Use external tools or APIs

  • Adapt based on execution results

  • Perform actions autonomously

While powerful, agentic AI introduces new attack surfaces, especially when internal reasoning or tools are exposed to users.

📘 Large Language Models — The Foundation

Agentic systems are typically built on Large Language Models (LLMs).

Capabilities

LLMs can:

  • Generate human-like responses

  • Follow instructions effectively

  • Perform logical reasoning

Limitations

LLMs:

  • Have no real-time awareness

  • Can hallucinate information

  • Cannot act independently without tools

Because LLMs rely on pattern prediction, they are vulnerable to:

  • Prompt injection

  • Jailbreaking

  • Tool misuse

To enhance reasoning, frameworks like Chain-of-Thought and ReAct are used.

🧠 Chain-of-Thought & ReAct — How AI Agents Reason

Chain-of-Thought (CoT)

CoT allows models to reason step-by-step internally.
However:

  • Errors can accumulate

  • Reasoning may be incorrect

  • Internal thoughts should never be exposed

ReAct (Reason + Act)

ReAct improves reliability by combining:

  • Reasoning (thinking through the problem)

  • Actions (calling tools or APIs)

This design enables AI agents to:

  • Fetch real-time data

  • Execute tasks dynamically

  • Update decisions based on results

Unfortunately, when poorly implemented, this exact mechanism becomes exploitable.

🛠️ Tool Usage in Agentic AI

Modern AI agents can call developer-defined tools or functions.

Example capabilities include:

  • Searching logs

  • Modifying system states

  • Updating configurations

If tool access is not properly restricted, attackers can:

  • Discover internal functions

  • Extract sensitive tokens

  • Trigger privileged actions

This is the vulnerability exploited in Wareville.

🎅 The Exploit — Restoring SOC-mas

Accessing the Calendar

Navigate to:

http://10.49.140.12

Observed issues:

  • December 25 marked as Easter

  • A chatbot controlling calendar logic

  • A visible thinking / reasoning panel

⚠️ Exposing internal reasoning is a critical security flaw.

Step 1 — Observing the Leak

Sending a simple greeting reveals the AI’s internal chain-of-thought, exposing how it reasons and which tools it considers.

Step 2 — Attempting a Direct Fix

Asking the agent to reset December 25 to Christmas fails — but the reasoning output leaks function names.

Step 3 — Enumerating Functions

Prompting the agent reveals available internal tools:

  • reset_holiday

  • booking_a_calendar

  • get_logs

This confirms tool enumeration via prompt injection.

Step 4 — Token Discovery

Calling reset_holiday fails due to missing authorization.

The agent’s reasoning indicates the presence of an access token.

Step 5 — Log Extraction

Using the get_logs function and refining prompts forces the agent to expose sensitive internal data.

📌 Leaked Token

TOKEN_SOCMAS

This demonstrates how poor prompt filtering and excessive transparency can compromise security.

Step 6 — Executing the Reset

With the valid token, the privileged function is executed successfully:

  • December 25 is restored to Christmas

  • The system confirms the correction

  • The final flag is revealed

🎁 Final Flag

THM{XMAS_IS_COMING__BACK}

✅ Mission Complete

SOC-mas has been restored, and Wareville’s calendar is back to normal.

This challenge highlights:

  • The dangers of exposing AI reasoning

  • The risks of unrestricted tool access

  • Why agentic AI must be carefully secured

🔐 Key Security Takeaways

  • Never expose chain-of-thought publicly

  • Restrict and validate tool execution

  • Separate reasoning from user-visible output

  • Treat AI agents like privileged system components

🚀 What’s Next?

If you found this challenge interesting, explore Defending Adversarial Attacks, where you’ll learn how to:

  • Secure LLM systems

  • Prevent prompt injection

Comments

Post a Comment

Popular Posts