BermudaHighball

I'd love to know if anyone's aware of a bulk metadata export feature or repository. I would like to have a copy of the metadata and .torrent files of all items.

I guess one way is to use the CLI but this relies on knowing which item you want and I don't know if there's a way to get a list of all items.

I believe downloading via BitTorrent and seeding back is a win-win: it bolsters the Archive's resilience while easing server strain. I'll be seeding the items I download.

Edit: If you want to enumerate all item names in the entire archive.org repository, take a look at https://archive.org/developers/changes.html. This will do that for you!

It seems like 6 or 7 years ago there was research into new forms of storage, using crystals or DNA that promised ultra high density storage. I know the read/write speed was not very fast, but I thought by now there would be more progress in the area. Apparently in 2021 there was a team that got a 16GB file stored in DNA. In the last month there's some company (Biomemory) that lets you store 1KB of data into DNA for $1,000, but if you want to read it, you have to send it to them. I don't understand why you would use that today.

I wonder if it will ever be viable for us to have DNA readers/writers... but I also wonder if there are other new types of data storage coming up that might be just as good.

If you know anything about the DNA research or other new storage forms, what do you think is the most promising one?

The more that content on the web is "locked down" with more stringent API requests and identity verification, e.g. Twitter, the more I wonder if I should be archiving every single HTTP request my browser makes. Or, rather, I wonder if in the future there will be an Archive Team style decentralized network of hoarders who, as they naturally browse the web, establish and maintain an archive collectively, creating a "shadow" database of content. This shadow archive is owned entirely by the collective and thus requests to it are not subject to the limitations set by the source service.

The main point is that the hoarding is not distinguishable from regular browsing from the perspective of the source website, so the hoarding system can't be shut down without also giving up access to regular users.

Verification that the content actually came from the real service could probably be done using the HTTPS packets themselves, and some sort of reputation system could prevent the source websites themselves from trying to poison the collective with spam.

Clearly, not all of the collected data should be shared, and without differential privacy techniques and fingerprint resistance the participating accounts can be connected to the content they share.

Has anything like this been attempted before? I've never participated in Archive Team, but from what I read it seems similar.

In the past, most software I used was paid and proprietary and would have some sort of limitation that I would try to get around by any means possible. Sometimes that would be resetting the clock on my computer, disabling the internet, and other times downloading a patch.

But in the past few years I've stopped using those things and have focused only on free and open source software (FOSS) to fulfill my needs. I hardly have to worry about privacy problems or trying to lock down a program that calls home. I might be missing out on some things that commercial software delivers, but I'm hardly aware of what they are anymore. It seems like the trend is for commercial software providers to migrate toward online or service models that have the company doing all the computing. I'm opposed to that, since they can take away your service at any time.

What do you do?