llama-cpp

Malicious tool definition detected

## Server Modes ### llama-server ```bash # Basic server ./llama-server \ -m models/llama-2-7b-chat.Q4_K_M.gguf \ --host 0.0.0.0 \ --port 8080 \ -c 4096 # Context size # With GPU acceleration ./llama-server \ -m models/llama-2-70b.Q4_K_M.gguf \ -ngl 40 # Offload 40 layers to GPU ``` ## OpenAI-Compatible API ### Chat completions ```bash curl http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "llama-2", "messages": [ {"role": "system", "content": "You

Malicious tool definition detected

## When to use llama.cpp **Use llama.cpp when:** - Running on CPU-only machines - Deploying on Apple Silicon (M1/M2/M3/M4) - Using AMD or Intel GPUs (no CUDA) - Edge deployment (Raspberry Pi, embedded systems) - Need simple deployment without Docker/Python **Use TensorRT-LLM instead when:** - Have NVIDIA GPUs (A100/H100) - Need maximum throughput (100K+ tok/s) - Running in datacenter with CUDA **Use vLLM instead when:** - Have NVIDIA GPUs - Need Python-first API - Want PagedAttention ## Quick st

Tool passed security scan

Tool passed security scan