My own server? YOLO
At work? Grafana, KOBS, Victoria Metrics, Jaeger, OpsGenie, ...
this post was submitted on 17 Feb 2024
78 points (98.8% liked)
Selfhosted
39194 readers
410 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 1 year ago
MODERATORS
My own server? YOLO
I can't figure out whether there's a monitoring tool called YOLO or you don't monitor anything.
Now I am intrigued to develop one that is called YOLO.
But just in case: no, I don't monitor my server. If I notice something not working, I ssh into the machine and check what's up. I don't want to deal with another zoo of services for the monitoring part.
You are me
Yes.
This is the first time I’ve heard of Victoria Metrics. It looks like it has a similar use case as Prometheus, is that correct? If so, what made you or your team choose one over the other?
IIRC it had better performance than Prometheus. We also ditched Elasticsearch in favor of ClickHouse to keep up with log ingestion.
My clients when they text me the server is down.
This has the same energy as my spouse yelling at me because jellyfin went down
Or my partners greeting me in the morning "Home assistant went down again, so the lights are all manual"
Thankfully that one is mostly solved.
So damn accurate ahhaha
"Huh weird, I tried to use and it's not working. Welp, guess I better fix it..."
I'm a huge fan of Netdata, very configurable and monitors just about anything you could want. Great interface and alerts too - https://www.netdata.cloud/
Same been running netdata for years. They're monetizing now where it used to just be free. Good for them, it's a great product. And it's foss
I was looking for something free that I could host on my machine but thanks, I didn't know about it
Netdata is free and can be run standalone. Just install it and do not configure the cloud integration. You can see your dashboard on localhost:19999
Oh that's neat, will take a look! Can you run it on docker?
As others stated, you can run and access the interface locally (or setup your own reverse proxy) for free. Their Cloud dashboard is also free for up to 5 nodes. They recently added a flat-rate "Homelab" plan as well, if you want to remove the limit. It's all quite usable for $0 otherwise though!
Netdata 100%
It feeds my itch for more data than I know what to do with and it's presented in one of the cleanest ways I've ever seen for so much info.
load more comments
(4 replies)
Node exporter on hosts, OpenTelemetry collector to scrape metrics and collect logs, shipping them to Prometheus and Loki, visualising with Grafana.
Day job is for an observability platform where we heavily encourage the use of (and also contribute) to the OpenTelemetry collector project, hence my use of it.
Try VictoriaMetrics. Basically the same feature set as Prometheus, but so much more resource friendly for homelab scale. I store some metrics for 12 months now, because it's easy.
load more comments
(2 replies)
I just use homepage as my homepage :D
I can see simple CPU/RAM/storage stats and got widgets for almost all services, one of them is portainer so I can see if any service is stopped (most of them are running in docker). Also few services send notification on error or update
I know its not really a monitoring tool, but it works well enough for me
Zabbix for agent / snmp based statistics.
Uptime Kuma for up/down states with a webhook notification into Discord so I get instant alerts on my phone when one goes down.
How has nobody in this thread said check_mk yet?
It's free, you host it yourself. It's built off of nagios, compatible with nagios plugins, supports snmp or agent based checks. It can email, SMS, slack or discord you when something breaks, you can write your own custom checks in any language that can output to a local console... I could never imagine even looking for something else.
+1 for check_mk.
It's got a scriptable config file that begs for automation like mgmtConfig and it does SNMP. For me, that's it. SNMP->MQTT->SNMP next year.
load more comments
(1 replies)
At home, libreNMS. Just SNMP everything.
For work, whatever the tool of the day is from management.
I've been using uptime Kuma recently and it's great but works better outside of docker.
Inside docker I'd get a lot of false down positives from I assume docker throttling the checks.
Plus it works with email, telegram, and matrix chat alerts. I monitor all my clients sites with it, and it's bullet proof behind caddy.
For light touch monitoring this is my approach too. I have one instance in my network, and another on fly.io for the VPSs (my most common outage is my home internet). To make it a tiny bit stronger, I wrote a Go endpoint that exposes the disk and memory usage of a server including with mem_okay and disk_okay keywords, and I have Kuma checking those.
I even have the two Kuma instances checking each other by making a status page and adding checks for each other's 'degraded' state. I have ntfy set up on both so I get the Kuma change notifications on my iPhone. I love ntfy so much I donate to it.
For my VPSs, this is probably not enough, so I am considering the more complicated solutions (I've started wanting to know things like an influx of fali2ban bans etc.)
I just do web hosting for clients sites and use Kuma to monitor uptime and SSL certificates.
Ive got multiple Kuma's running as well.
At home, nagios, at work colleagues. (I finally escaped the admin rat race)
Grafana set up to run on the server locally, then I connect to it via SSH forwarding. Then I can view all kinds of metrics in my browser in a neat interface.
I liked Grafana a lot, but I can't monitor things like zfs pools with it right?
I don't know as I don't use zfs pools, but a simple search led me to this https://grafana.com/grafana/dashboards/15362-zfs-pool-metrics/
Nevermind then! Will take a look at it ^^
I like monit. It’s simple to setup and pretty flexible.
I used it as well until I found out I could just do it with systemd. https://www.baeldung.com/linux/systemd-service-fail-notification
Zabbix
Second Zabbix. Been using it for years and it just works.
Adding my vote for Zabbix. It was a bit of a bear to set up and I had to write custom scripts to install the agents with TLS settings that were secure enough for me, but once it's all set up it's amazingly easy and intuitive to use and incredibly customizable.
Monitorix or Netdata.
Prometheus and Altertmanager
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| MQTT | Message Queue Telemetry Transport point-to-point networking |
| SSH | Secure Shell for remote terminal access |
| SSL | Secure Sockets Layer, for transparent encryption |
| TLS | Transport Layer Security, supersedes SSL |
| VPS | Virtual Private Server (opposed to shared hosting) |
5 acronyms in this thread; the most compressed thread commented on today has 16 acronyms.
[Thread #521 for this sub, first seen 17th Feb 2024, 17:25] [FAQ] [Full list] [Contact] [Source code]
Btop and logwatch with logrotate. I use healthchecks to check if the server is unreachable and it notifies me.
I use collectd and graphs on my openwrt router. It can even use data from mqtt-connected thermometers and gather metrics from other collectd instances.
view more: next ›