Tried the q2 ggml and it seems to be very good! First tests make it seem as good as airoboros, which is my current favorite.
this post was submitted on 09 Aug 2023
13 points (100.0% liked)
LocalLLaMA
2207 readers
26 users here now
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
founded 1 year ago
MODERATORS
agreed, it seems quite capable, i haven't tested all the way down to q2 to verify but i'm not surprised