827
The Internet Archive is under attack, with a popup claiming a ‘catastrophic’ breach
(www.theverge.com)
This is a most excellent place for technology news and articles.
Again, isn't that the site's prerogative?
I think there should at least be a recognized way to opt-out that archive.org actually follows. For years they told people to put
in robots.txt, but they still archived content from those sites. They refuse to publish what IP addresses they pull content down from, but that would be a trivial thing to do. They refuse to use a UserAgent that you can filter on.
If you want to be a library, be open and honest about it. There's no need to sneak around.