*Cries in 18GB unified memory*
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
| Equivalent to 14GB of VRAM and 4GB of system RAM given the limits I assigned to the GPU. EDIT: we be getting lots of big (≥20B), small (≤4B) models these days but very little in between [link] [comments] |
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.