name: Inference skill_id: inference description: Guides model-serving and runtime-inference decisions across local, remote, and packaged deployment paths. version: 1.0.0 source_kind: elephant-builtin aliases: ["model inference", "serve a model", "run inference"] trigger_phrases: ["help me serve this model", "set up inference for this model", "run local inference"] keywords: ["inference", "serving", "latency", "throughput", "deployment", "runtime"] category: mlops

Inference

Use this built-in skill when the user needs to run, serve, benchmark, or operationalize model inference.

Core rules

• Separate model choice, hardware fit, serving stack, and request pattern.
• Make latency, throughput, memory, and quality tradeoffs explicit.
• Inspect the real deployment target before recommending a stack.
• Prefer reproducible launch and benchmark steps over hand-wavy performance advice.

Default workflow

1. Identify the model, hardware, traffic pattern, and latency target.
2. Choose the serving path that fits the deployment constraints.
3. Validate startup, request shape, batching, and resource usage.
4. Benchmark and tune only after the baseline path is stable.

Guardrails

• Do not imply a model will fit on hardware you have not checked.
• Do not conflate prototype notebook inference with production serving.
• Do not quote performance numbers without workload context.