Can we reasonably use the InferenceMAX benchmark dataset to get a Wh per query figure?
  • troed@fedia.io
    2·
    22 hours ago

    Yeah this one isn’t very difficult. My GPU isn’t using more power running a model in Ollama than when playing games, so, a query needing 30 seconds of GPU power needs as many Wh as playing a game for 30 seconds …