LocalLLaMA

2207 readers

27 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup@sh.itjust.works

pax@sh.itjust.works

noneabove1182@sh.itjust.works

I'm having a fantastic time with this model. (huggingface.co)

submitted 9 months ago by Stopwatch8200@lemmynsfw.com to c/localllama@sh.itjust.works

6 comments fedilink hide all child comments

I've been using TheBlokes Q8 of https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B, but now this one (https://huggingface.co/Weyaxi/OpenHermes-2.5-neural-chat-7b-v3-1-7B) I think is killing it. Has anyone else tested it?

top 6 comments

sorted by: hot top controversial new old

[–] noneabove1182@sh.itjust.works 3 points 9 months ago* (last edited 9 months ago) (1 children)

Hmm had interesting results from both of those base models, haven't tried the combo yet, will start some exllamav2 quants to test

What's it doing well at?

quant link for anyone who may want: https://huggingface.co/bartowski/OpenHermes-2.5-neural-chat-7b-v3-1-7B-exl2

[–] Stopwatch8200@lemmynsfw.com 2 points 9 months ago* (last edited 9 months ago)

I haven't tried neural-chat, but the combined model seems to be better (anecdotally) than OH2.5/Mistral at following instructions, reasoning, some of the overall quirks with llama.cpp seem to be ironed out with it too.

[–] MigratingtoLemmy@lemmy.world 2 points 9 months ago

What hardware are you running this off of?

[–] rufus@discuss.tchncs.de 2 points 9 months ago (1 children)

Seems quite intelligent. But I'm currently trying storywriting and that seems to be the one thing it doesn't excel at. It mostly stays being very abstract and summarizing the plot instead of playing it out.

[–] noneabove1182@sh.itjust.works 2 points 9 months ago

Yeah this seems less focused on creativity, there's a lot of really good models out there tuned for story telling that will far exceed generalized SoTA models

[–] mixtral@sh.itjust.works 1 points 9 months ago

How to download it for MLCChat(Android)?