security-property-based-testing
Security Property-Based Testing
When to Use
- Testing that security invariants hold across many generated inputs
- Fuzz-testing parsers, validators, and input handlers
- Verifying authentication/authorization boundaries
- Testing cryptographic implementations for correctness
- Finding edge cases in input validation logic
When NOT to Use
- Testing specific known attack payloads (use unit tests)
- Load testing or performance testing
- UI testing or visual regression testing
Core Concept
Property-based testing generates thousands of random inputs and verifies that security properties always hold:
For all inputs X: security_invariant(process(X)) == True
Frameworks by Language
| Language | Framework | Install |
|---|---|---|
| Python | Hypothesis | pip install hypothesis |
| JavaScript/TS | fast-check | npm install fast-check |
| Rust | proptest | cargo add proptest --dev |
| Go | rapid | go get pgregory.net/rapid |
| Java | jqwik | Maven/Gradle dependency |
Python (Hypothesis) Examples
Input Validation
from hypothesis import given, strategies as st
@given(st.text())
def test_sanitizer_removes_script_tags(input_str):
"""XSS property: output never contains <script>"""
result = sanitize_html(input_str)
assert "<script" not in result.lower()
@given(st.text())
def test_path_traversal_blocked(user_path):
"""Path traversal property: resolved path stays within base dir"""
try:
resolved = resolve_user_path(user_path)
assert resolved.startswith(BASE_DIR)
except ValidationError:
pass # Rejection is safe
Auth Boundaries
@given(st.sampled_from(["admin", "user", "guest"]),
st.sampled_from(["/api/admin", "/api/user", "/api/public"]))
def test_auth_boundaries(role, endpoint):
"""Authorization property: guests cannot access admin endpoints"""
response = make_request(role, endpoint)
if endpoint.startswith("/api/admin") and role == "guest":
assert response.status_code == 403
Serialization Roundtrip
@given(st.dictionaries(st.text(), st.text()))
def test_serialize_roundtrip_safe(data):
"""Serialization property: roundtrip preserves data without code execution"""
serialized = safe_serialize(data)
deserialized = safe_deserialize(serialized)
assert deserialized == data
JavaScript (fast-check) Examples
const fc = require('fast-check');
test('SQL parameterization prevents injection', () => {
fc.assert(fc.property(fc.string(), (userInput) => {
const query = buildParameterizedQuery(userInput);
// Property: query never contains unescaped user input directly
expect(query.params).toContain(userInput);
expect(query.sql).not.toContain(userInput);
}));
});
Security Properties to Test
| Property | Description |
|---|---|
| No injection | Output never contains unescaped control characters |
| Path containment | File paths stay within allowed directory |
| Auth enforcement | Protected resources reject unauthorized access |
| Idempotent validation | Validating twice gives same result |
| No information leak | Error messages don't vary by secret values |
| Constant-time comparison | Comparison time doesn't depend on input |
| Roundtrip safety | Serialize/deserialize preserves data exactly |
Best Practices
- Start with the most security-critical invariants
- Use custom strategies that generate attack-like inputs
- Combine with coverage tracking to find untested paths
- Save failing examples as regression unit tests
- Run with high example counts in CI (at least 1000)
More from elizaos/eliza
nano-pdf
Edits PDF files using natural-language instructions via the nano-pdf CLI. Supports modifying text, changing titles, fixing typos, and updating content on specific pages. Use when the user wants to edit a PDF, modify PDF content, update PDF text, fix a typo in a PDF, change a PDF title, or rewrite part of a PDF page.
30wacli
Send WhatsApp messages to other people or search/sync WhatsApp history via the wacli CLI (not for normal user chats). Use when the user asks to send a WhatsApp message, text someone on WhatsApp, search WhatsApp chat history, sync WhatsApp conversations, backfill message history, or forward a file via WhatsApp to a third party.
27nano-banana-pro
Generate or edit images via Gemini 3 Pro Image (Nano Banana Pro). Use when the user asks to create an image, generate a picture, produce AI-generated artwork, edit a photo, compose multiple images, or upscale an image to higher resolution. Supports text-to-image generation, single-image editing, and multi-image composition using the Gemini API.
27obsidian
Work with Obsidian vaults (plain Markdown notes) and automate via obsidian-cli. Use when the user asks about notes, vault management, PKM, knowledge base organization, wikilinks, or personal knowledge management in Obsidian.
25session-logs
Search and analyze session logs (older/parent conversations) stored as JSONL files using jq and rg. Use when the user asks about prior chats, previous conversations, conversation history, what was said before, session costs, token usage, or tool usage breakdown across past sessions.
24discord
Use when you need to control Discord from Otto via the discord tool: send messages, react, post or upload stickers, upload emojis, run polls, manage threads/pins/search, create/edit/delete channels and categories, fetch permissions or member/role/channel info, set bot presence/activity, or handle moderation actions in Discord DMs or channels.
24