continuum::runtime::DecodeParams Struct Reference

#include <kv_prefix_cache.hpp>

Public Attributes
std::string	op_name
float	temperature = 1.0f
std::int32_t	max_tokens = 0

Member Data Documentation

max_tokens

std::int32_t continuum::runtime::DecodeParams::max_tokens = 0

op_name

std::string continuum::runtime::DecodeParams::op_name

temperature

float continuum::runtime::DecodeParams::temperature = 1.0f

---

The documentation for this struct was generated from the following file:

• include/continuum/runtime/kv_prefix_cache.hpp