this post was submitted on 05 Mar 2024
27 points (93.5% liked)

Selfhosted

39987 readers
779 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
27
How do you manage your server files? (reddthat.com)
submitted 8 months ago* (last edited 8 months ago) by legoraft@reddthat.com to c/selfhosted@lemmy.world
63 comments fedilink hide all child comments
 

I'm currently debating on how to manage files on my servers. I have a jellyfin and a minecraft server on which I need to add, remove or download files quite often. I don't really want to use scp for everything, so I was wondering what everyone uses.

Edit: I'm looking for a gui solution, but a somewhat automated process of backups etc. is also nice

top 50 comments
sorted by: hot top controversial new old
[–] kevincox@lemmy.ml 12 points 8 months ago (1 children)

IDK what OS you are on but on Linux most file managers have support for remote filesystems. SFTP (SSH-FTP, not to be confused with FTPS which is FTP-secure) is ubiquitous and if you use scp then you already have SSH set up.

If you need Windows support it is more of a pain. You may need to set up Samba or WebDAV and permissions can suck. But you can also download a third-party file browser that supports remote protocols.

So basically SFTP, and I fairly regularly just use a graphical file manager when I am doing one-off operations.

[–] legoraft@reddthat.com 3 points 8 months ago (2 children)

I'm using debian, so sftp would be an option, do you use a graphical client?

[–] kevincox@lemmy.ml 6 points 8 months ago (1 children)

Right now I am just using nautilus (default GNOME file manager) but in past I was using Thunar (default XFCE file manager). I'd be pretty surprised if whatever file manager you are currently using doesn't support SFTP out of the box. Typically you can just enter something like sftp://myhost.example into the location bar. They may also have a dedicated network connection section with a wizard to add it.

[–] legoraft@reddthat.com 1 points 8 months ago (1 children)

I'm considering this, as I can see by your example, you can add a domain name to the server. How would you go over doing this?

[–] kevincox@lemmy.ml 2 points 8 months ago (1 children)

You should look into DNS or maybe mDNS or even local DNS aliases.

[–] legoraft@reddthat.com 1 points 8 months ago

I will take a look, thank you very much!

[–] SomeBoyo@feddit.de 2 points 8 months ago

nemo for a gui, or midnightcommander after logging in via ssh for managing/moving files on the server.

[–] Illecors@lemmy.cafe 10 points 8 months ago (1 children)

NFS comes to mind, naturally.

I remember some years ago scp had a big issue, can't recall what, though. But that made me have a look at rsync, and I've been using that ever since. Flags are a bit atteocious, but I've aliases rsync -avz status=progress to copy and it's been happy days. One other benefit - incremental copy. Helps in cases where a copy procedure had been stopped for whatever reason.

[–] kevincox@lemmy.ml 2 points 8 months ago

I wouldn't really recommend NFS unless you need to remote mount as a "true filesystem" with full support for things like sockets, locking and other UNIX filesystem features or you need top performance. It is so difficult to do authentication and UID mapping that it typically isn't worth it for simpler use cases like "add, remove or download files".

scp can be slow with large numbers of small files. rsync is much better at that and can do differential transfers if you need that. Since rsync can also run over SSH it can be very easy to just use it as a default.

[–] Darkassassin07@lemmy.ca 10 points 8 months ago (1 children)

Linux?

I just use sshfs to mount ssh shares and move files between them like any other folder.

Same with samba shares (windows).

[–] Klajan@lemmy.zip 1 points 8 months ago

You can do the same on Windows (mount ssh shares), there is an Fuse like project for Windows: WinFsp.

[–] eletes@sh.itjust.works 9 points 8 months ago

Sftp using the KDE file browser Dolphin. Keeps it simple for my monkey brain as if I'm accessing the drive locally.

[–] ptz@dubvee.org 8 points 8 months ago (4 children)

NFS, SMB, and/or sshfs

  • NFS for my media shares between servers (Emby, MPD, etc)
  • SMB for accessing from workstations
  • sshfs when I need a quick, one-off file session (or if I'm accessing remotely)
[–] matcha_addict@lemy.lol 2 points 8 months ago

Sshfs is also a good candidate if for whatever reason you want security / encryption

load more comments (3 replies)
[–] MangoPenguin@lemmy.blahaj.zone 8 points 8 months ago (1 children)

Plex/Jellyfin is automatically managed by Sonarr/Radarr so I don't touch those.

But for game servers I use Pterodactyl which has a nice WebUI to manage the server and its files, and has automated backups.

[–] legoraft@reddthat.com 2 points 8 months ago

pterodactyl looks really neat, will definitely look into that. I have a manual system for my media library, so I want to add the directories with artwork and movies manually to the directory which jellyfin reads.

[–] crony@lemmy.cronyakatsuki.xyz 7 points 8 months ago (1 children)

I tend to use rsync 99% times.

[–] legoraft@reddthat.com 4 points 8 months ago (1 children)

how would you do that with a large media library?

[–] crony@lemmy.cronyakatsuki.xyz 2 points 8 months ago* (last edited 8 months ago) (1 children)

I would just use rsync on the whole folder.

Rsync is smart enough to be able to only send files you didnt' have previously like a regular mv command.

Or you can sync the whole directory and have it also delete removed files.

Edit: If you wan't something to automatically "sync/backup" the data, why don't you try and use syncthing? P2P syncing solution that might be exactly what you need in your hse case.

[–] legoraft@reddthat.com 2 points 8 months ago (1 children)

I'm now considering syncing my minecraft world with syncthing, I already use it for some things but don't know why I didn't think of doing that.

On the other hand, if I have a 100+ gb media library, it seems kinda over the top to also have it fully copied on my local machine. Do you do this?

[–] crony@lemmy.cronyakatsuki.xyz 2 points 8 months ago (1 children)

When it comes to media like my music library and comics for example I just use rsync to move new ones to the server every time I collect some new (I have about 60gb of it)

[–] legoraft@reddthat.com 2 points 8 months ago (1 children)

so you basically have a copy of your media library on a local machine?

[–] crony@lemmy.cronyakatsuki.xyz 1 points 8 months ago (1 children)

No, I only have new files on my local machine, and as I collect new stuff I move them to my server with rsync and just remove it onthe local machine.

Rsync is more than smart enough to, if you keep same folder structure to easilly move the files to the server without you having to manually copy other each file.

[–] legoraft@reddthat.com 2 points 8 months ago (1 children)

sounds good, do you have any docs on how to do that?

[–] crony@lemmy.cronyakatsuki.xyz 1 points 8 months ago (2 children)

Don't really have some rn. Will make a page on my wiki and link it to you later with some basic examples.

load more comments (2 replies)
[–] matcha_addict@lemy.lol 5 points 8 months ago (1 children)

Can you tell us what's your issue with scp? It'll help us make better recommendations.

[–] legoraft@reddthat.com 3 points 8 months ago (5 children)

I mostly want some sort of graphical way, I'm often moving a bunch of loose files and seeing them is a lot easier for me when transferring

[–] vegetaaaaaaa@lemmy.world 4 points 8 months ago* (last edited 8 months ago) (1 children)

sftp://USERNAME@SERVER:PORT in the address bar of most file managers will work. You can omit the port if it's the default (22), you can omit the username if it's the same as your local user.

You can also add the server as a favorite/shortcut in your file manager sidebar (it works at least in Thunar and Nautilus). Or you can edit ~/.config/gtk-3.0/bookmarks directly:

file:///some/local/directory
file:///some/other/directory
sftp://my.example.org/home/myuser my.example.org
sftp://otheruser@my.example.net:2222/home/otheruser my.example.net
load more comments (1 replies)
[–] matcha_addict@lemy.lol 3 points 8 months ago (1 children)

I would recommend you mention "graphical" or "GUI" in your main post. Most suggestions to be assuming CLI.

Some suggestions mentioned NFS / SSHFS. Those would allow you to use any GUI file manager for moving stuff over.

Sorry I can't help more. Don't have a lot of knowledge in gui tools

load more comments (1 replies)
load more comments (3 replies)
[–] drkt@lemmy.dbzer0.com 3 points 8 months ago

rsync for backups. SMB / NFS for things I need on a daily basis. SSH FTP (use whatever client you want) for things I need rarely but don't wanna bother with SCP and I especially don't bother ever bother setting up vsftpd for 'real' FTP.

[–] blotz@lemmy.world 3 points 8 months ago* (last edited 8 months ago) (1 children)

Nemo/most file explorers have a remote server option. For nemo, "Files > Connect to server". Once configured, you should be able to view your server files from your file explorer and uploading files should be as easy as drag and drop. I'm pretty sure nautilis has something similar too!

It's not the best for security because if someone gets your computer, they can access your server through the file explorer. But it's fast and convenient if you are already using your file explorer.

[–] kevincox@lemmy.ml 2 points 8 months ago

Generally you can even just type it into the location bar, no need to find a specific dialog. For example if I type CTRL+l then sftp://myserver.local into Nautilus it pops up a username+password dialog (or just logs in using my keys).

[–] jaykay@lemmy.zip 2 points 8 months ago

Wouldn’t FileBrowser work? Idk if you’re on docker or not though

[–] halm@leminal.space 2 points 8 months ago (1 children)

SCP?

[–] legoraft@reddthat.com 2 points 8 months ago (1 children)

sorry, should've clarified: secure copy, it's an ssh kind of way of copying files to a server

load more comments (1 replies)
[–] originalucifer@moist.catsweat.com 2 points 8 months ago

docker, persistent shared storage

[–] gravitywell@sh.itjust.works 2 points 8 months ago (2 children)

I use samba for normal usage and rsync for backups.

load more comments (2 replies)
[–] cmnybo@discuss.tchncs.de 2 points 8 months ago

I use rsync and SFTP to transfer files to a remote server and NFS over the LAN.

[–] jlh@lemmy.jlh.name 2 points 8 months ago (1 children)

scp is one of the best options. You could also try sshfs if you want something more interactive.

load more comments (1 replies)
[–] normalexit@lemmy.world 2 points 8 months ago

It's not very sexy, but I just use SFTP and filezilla. Filezilla is available for all the major operating systems, and an ssh server is basically required anyways.

[–] slazer2au@lemmy.world 1 points 8 months ago

Being the weirdo I am I have 2 things I use.

WinSCP to get that classic FTP look of file transfers and VS Code when working on smaller compose files.

load more comments
view more: next ›