docker model run

DescriptionRun a model and interact with it using a submitted prompt or chat mode
Usagedocker model run MODEL [PROMPT]

Experimental

This command is experimental.

Experimental features are intended for testing and feedback as their functionality or design may change between releases without warning or can be removed entirely in a future release.

Description

When you run a model, Docker calls an inference server API endpoint hosted by the Model Runner through Docker Desktop. The model stays in memory until another model is requested, or until a pre-defined inactivity timeout is reached (currently 5 minutes).

You do not have to use Docker model run before interacting with a specific model from a host process or from within a container. Model Runner transparently loads the requested model on-demand, assuming it has been pulled and is locally available.

You can also use chat mode in the Docker Desktop Dashboard when you select the model in the Models tab.

Options

OptionDefaultDescription
--debugEnable debug logging

Examples

One-time prompt

docker model run ai/smollm2 "Hi"

Output:

Hello! How can I assist you today?

Interactive chat

docker model run ai/smollm2

Output:

Interactive chat mode started. Type '/bye' to exit.
> Hi
Hi there! It's SmolLM, AI assistant. How can I help you today?
> /bye
Chat session ended.