I got cancelled too and chose Hetzner instead. Will not do business with a company that can't get their filters working decently.
vluz
joined 1 year ago
Lovely! I'll go read the code as soon as I have some coffee.
I do SDXL generation in 4GB at extreme expense of speed, by using a number of memory optimizations.
I've done this kind of stuff since SD 1.4, for the fun of it. I like to see how low I can push vram use.
SDXL takes around 3 to 4 minutes per generation including refiner but it works within constraints.
Graphics cards used are hilariously bad for the task, a 1050ti with 4GB and a 1060 with 3GB vram.
Have an implementation running on the 3GB card, inside a podman container, with no ram offloading, 1 vcpu and 4GB ram.
Graphical UI (streamlit) run on a laptop outside of server to save resources.
Working on a example implementation of SDXL as we speak and also working on SDXL generation on mobile.
That is the reason I've looked into this news, SSD-1B might be a good candidate for my dumb experiments.
Oh my Gwyn, this comment section is just amazing.
Goddammit! Don't tell that one, I use it to impress random people at parties.
Not joking, although I understand it seems very silly at face value.
Dark Souls 3 PvP specifically SL60+6 at gank town (after pontiff).
It used to be my go-to wind down after a work day.
It made me smile and actually relaxed me enough to go to bed and sleep, especially after a hard day.
HateLLM will be a smash. /s
That's wonderful to know! Thank you again.
I'll follow your instructions, this implementation is exactly what I was looking for.
Absolutely stellar write up. Thank you!
I have a couple of questions.
Imagine I have a powerful consumer gpu card to trow at this solution, 4090ti for the sake of example.
- How many containers can share one physical card, taking into account max memory will not be exceeded?
- How does one virtual gpu look like in the container? Can I run standard stuff like PyTorch, Tensorflow, and CUDA stuff in general?
Don't trust brave, never will.
view more: next ›
Got one more for you: https://gossip.ink/
I use it via a docker/podman container I've made for it: https://hub.docker.com/repository/docker/vluz/node-umi-gossip-run/general