chat-with-arxiv
Fail
Audited by Gen Agent Trust Hub on Feb 15, 2026
Risk Level: HIGHPROMPT_INJECTIONEXTERNAL_DOWNLOADS
Full Analysis
- PROMPT_INJECTION (HIGH): The skill implements a Question Answering system that is susceptible to Indirect Prompt Injection (Category 8). \n
- Ingestion points: In
examples/paper_content_processor.py, thedownload_and_process_papermethod retrieves PDFs from external URLs and extracts all text content usingPyPDF2. \n - Boundary markers: In
examples/paper_question_answerer.py, thegenerate_answermethod interpolates this raw text into the LLM prompt using only a simple 'Context from research papers:' header as a delimiter. This lacks robust isolation or instructions to ignore embedded commands. \n - Capability inventory: The agent uses the extracted content to synthesize findings and answer user questions, which allows an attacker to influence the agent's persona, reasoning, and output by uploading a malicious paper to ArXiv. \n
- Sanitization: No filtering, escaping, or validation is performed on the extracted PDF text before prompt interpolation. \n- EXTERNAL_DOWNLOADS (MEDIUM): The skill performs automated network downloads of external PDF files from non-whitelisted domains. \n
- Evidence: In
examples/paper_content_processor.py,requests.get(pdf_url)is used to fetch content from URLs provided by the ArXiv API. ArXiv is not a whitelisted source in the security policy, and the lack of domain validation for thepdf_urlcould allow connections to malicious sites if the metadata is manipulated.
Recommendations
- AI detected serious security threats
Audit Metadata