HARDWARE / SIZING
The right machine.
For your real workload.
A model that starts up is not necessarily a service several people can use. Before buying, you need to observe the quality, the memory and the response time under the planned conditions.
Reviewed by Saphan · 7 September 2026
Describe the workload before the configuration
Prepare an example of a short request, a representative document and a difficult case. State how many people must use the system at the same time and how long they can wait. Reading a long file does not impose the same constraints as sorting a few lines.
Also say whether the system has to read images, transcribe audio or query a document base. The conversation model is sometimes only part of the total load.
Memory is not just the model file
The weights, the intermediate computations, the context and simultaneous requests all use resources. A quantized version can reduce some requirements, but its effect on quality and speed must be tested with the chosen engine. The number of parameters alone is not enough to predict the final experience.
For a given tool, check the hardware it supports and its options in the corresponding documentation. The Ollama FAQ covers, among other things, settings for context, concurrency and loading; their effects depend on the configuration.
The test to ask for before buying
- Run the examples with the model and software versions recorded.
- Observe the first result, the complete output and its quality.
- Replay with several requests and a longer document.
- Check how it holds up over time: memory, temperature, noise and waiting time.
- Restart the system and restore a configuration in a test environment.
An isolated throughput figure does not prove that your scenario will be comfortable. Ask for the measurement conditions: context, quantization, load and the steps included.
Think about where the machine will live
An office machine, a rack-mounted server and a device deployed in the field have different constraints. Network, cooling, physical access, backup and maintenance must all be planned for. Keeping a documented margin is more useful than choosing the biggest hardware without a scenario.
Saphan can start from the hardware you already have, compare the options and define a suitable configuration. A cloud add-on can also be considered when occasional capacity does not justify a permanent purchase.