AGENTIC SOFTWARE ANALYSIS

Code in. Analysis results out. Fully autonomous.
Islem Bouzenia (CISPA) · Cristian Cadar (Imperial College London) · Michael Pradel (CISPA)
ELLIS Unit Stuttgart
AnalysisAgent reject: try again 1 create isolated environment 2 install the analysis tool 3 prepare + build your project 4 analyze the project judge and validate LLM fully autonomous: every step end to end, recovering from failures on its own evidence only
2
unknown bugs already found
(masscan, radare2)
94%
analyses completed,
human-verified
36 min
average time
per analysis
$0.41
average cost
per analysis

AThe task, and why it is hard

Software analysis examines a program automatically, to uncover:

crashes and security flaws
bugs, leaks, bad pointers
performance bottlenecks
structure: who calls whom

But before any analysis can run, someone must set it up by hand.

! ! ! isolated environment containers 1 install AFL++ and LLVM compilers 2 recompile curl build systems 3 example inputs + flags tool manual 4 run + monitor know-how 5 triage the findings expertise 6 missing libs build breaks hangs false alarms analysis

BInside AnalysisAgent: each step, and its artifact

1Sets up the environment
→
DOCKERFILE · written by the agent
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND=noninteractive LLVM_VERSION=13
RUN apt-get update && apt-get install -y build-essential \
    clang-13 llvm-13 llvm-13-dev llvm-13-tools cmake bison flex \
    libboost-all-dev python3-pip zlib1g-dev libcap-dev \
    libgoogle-perftools-dev libsqlite3-dev libtcmalloc-minimal4
RUN pip3 install lit wllvm && update-alternatives --install \
    /usr/bin/clang clang /usr/bin/clang-13 100
... abridged
2Installs the analysis tool
→
TERMINAL · built from source, smoke-tested
$ git clone https://github.com/stp/stp && cd stp/build \
    && cmake -DNOCRYPTOMINISAT=ON .. && make -j8 install
$ git clone https://github.com/klee/klee-uclibc && ./configure \
    --make-llvm-lib --with-llvm-config=llvm-config-13 && make -j8
$ cmake -DENABLE_SOLVER_STP=ON -DENABLE_POSIX_RUNTIME=ON \
    -DKLEE_UCLIBC_PATH=/work/klee-uclibc /work/klee && make -j8
↻ 14 of its 29 steps went to dependency battles (TCMalloc, SQLite3, uclibc)
$ klee --version   →   KLEE 3.1 ✓
3Prepares the project
→
TERMINAL · masscan compiled for analysis
$ export LLVM_COMPILER=clang && make -j8 CC=wllvm
CC src/main.c ... CC src/massip-parse.c ... CC src/rawsock.c ...
ld: cannot find -lpcap   ↻ installs libpcap-dev, retries
$ extract-bc bin/masscan
masscan.bc ✓   entry point: massip_parse_ipv4
4Runs the analysis
→
RESULTS · produced on the real masscan code
$ klee --libc=uclibc --posix-runtime --only-output-states-covering-new \
    --max-time=120 --sym-args 1 3 16 masscan.bc
KLEE: output directory is "klee-out-0"
KLEE: WARNING: undefined reference to function: ioctl ...
✓ 28 completed + 172 partial test paths
klee-out-0/: test000001.ktest ... test000028.ktest · run.stats
✓Judge validates the evidence
→
VERDICT · evidence accepted
VERIFIED 
✓ real project · ✓ artifacts exist · ✓ matches reference
29 steps · $0.39 · 12 min

CAnalysisAgent beats today's best agents

average over 4 AI modelsbest AI model (Gemini-3-Flash)
RAG-Agent
writes setup scripts from retrieved docs
10%
23%
Mini-SWE-Agent
popular general-purpose coding agent
37%
63%
ExecutionAgent
prior agent for building + testing projects
57%
77%
AnalysisAgent (ours)
purpose-built for software analysis
79%
94%

AnalysisBench: 7 analysis tools, 10 projects, 35 tasks in total (C, C++, Java). All successes verified by hand; gaps significant (padj < 0.001).

100%
of tasks AnalysisAgent claims solved (Gemini-3-Flash)
94%
verified by humans: the smallest claim-to-reality gap of any agent tested

DAnalysisAgent found 2 real bugs

386,193
tests generated by KLEE
8.3×10⁷
inputs executed by AFL++
masscan, an Internet-scale port scanner
NEW BUG 
$ masscan -c crash.conf --echo
[-] CONF: bad MAC address: router- = 80
Bus error (core dumped)   # fatal crash
radare2, a code-inspection toolkit: wrong math
NEW BUG 
$ rax2 '1<<64'
0x1                       # correct answer: 0

The bottleneck was never the analysis tools. It was the setup.

AnalysisAgent removes it: professional software analysis, end to end, verified by evidence.

Read the paper Get the code + data