No Stupid Questions

Everybody is blaming SEO, which is true - but Google is also hamstrung by walled gardens.

Before Facebook, most content posted to the web was open. It could be viewed by anyone without logging in. Reddit even uses this paradigm.

But then Facebook started putting everything behind their account login and suddenly, Google can no longer spider a significant amount of the conversation going on on the Internet - and it can't link you to it either, because the link would be dead if you weren't a logged-in Facebook user. And of course it's not just Facebook.

This is why appending site:reddit.com has come into fashion in the past couple years. Reddit, being open, viewable without a login, is a fantastic source for finding people who are talking about exactly what you're searching for.

And it's another reason why Meta is cancer: all the conversations going on about whatever problem you are experiencing that made you do a search in the first place, if they exist in private groups on something like Facebook - they are useless to you and useless to anyone but the members of that private group. We are losing our giant public knowledge base because capitalism.

You can’t just write an essay like that and not tell us what terms you used for your searches

The signal to noise ratio has seemed particularly out of wack with Google lately. The amount of blog spam SEO nonsense that crops up into the top 4 results has been pretty noticeable.

I’m not sure it’s entirely a Google thing. Reddit’s decline has made it harder to find quick answers for, “My washing machine’s making this weird string of beeps?” Niche hobbies moving from forums to Discord chats means, “How do I safely remove a keycap without damaging the switch?” is becoming a pinned message in a server you have to hear about via word of mouth. Basically any technology troubleshooting topic has moved from a blog post / forum to a YouTube video. And a 10 minute long one at that. Gotta hit those higher ad tiers.

For what it’s worth, I’m starting the new year off giving Kagi a try. It’s a startup trying to make a paid search engine work. You get 100 free searches to give it a try. After that it’s $5/mo for 300 searches, or $10/mo for unlimited. I’m not sure I’ll sign up for it just yet, but it seems pretty nice. No ads, custom components for things like Stack Overflow and Reddit, and some other nice touches for people who care about search. Their image search actually has a “View Image” link in addition to the “View Page” link. It’s hard to quantify how “good” a search result is, but I’ve been pretty impressed with it so far.

I've finally switched to DuckDuckGo because of this. Even though only about two months ago I said here somewhere that it's garbage. Google just managed to convince me that they're more garbage.

Google was really valuable before web services were so monopolized and consolidated like they are now. It's almost more useful to use the specific websites search function for many things now. Before this, you could run searches and it would have all these personal and small websites indexed. Oh look, here's a guy who lives his whole life as Peter Pan and has a website about it, cool... now it's just a profile on some social media site same as anyone else.

I refuse to believe you haven't been able to find a Hollywood movie after an hour? That sounds more like an issue with you than Google

What happened is SEO got good and money got made and fortunes got made and greed has taken over.

The internet today is the equivalent of the first and last 10 pages of the old yellowbooks. Why do you think AAA Auto is called what it's called?

While it's fun to bash on Google, this might have been a more productive discussion if you had provided your search query and perhaps a sample of the results

The biggest issue I have is that half my results come back as videos. Video results should be in the video tab. I don't want to watch a half hour long video just to find out how to make a healing brew in ark.
One paragraph would convey the information 10x faster than any video could

I was trying to Google "Best way to shave your head with low or no water pressure" because I was staying somewhere rural for a bit and my razor kept clogging.

All I got were straight razor blog spam and dozens of other completely unrelated shit.

I tried the shake it in a bowl method, 1/10 razor still clogged with hair.

I just registered an account here specifically because I've noticed it a ton recently and I wanted to reply to this since it's been on my mind. From my experience, google's quality has been going down in general for a while now, but very recently (the last few months or so?) it hasn't been just unusable in a figurative sense, it's been quite completely literally useless to the point of basically being broken.

I really wish I could remember some specific examples of what I was searching for, but I've had more than one experience where it felt like if it couldn't find something on reddit or wikipedia (which I usually have to give it some assistance anyway with the site: filter), it was like that thing just didn't exist. It was just pages and pages of what looked like fake AI generated articles that were only maybe slightly adjacent to the topic I was searching for. If it happens again or I can remember a specific case I might try to update my response.

Disclaimer: I use bing 50% of the time depending on which browser profile I have open. No real specific reason here, just that I didn't bother updating the search engine settings on all profiles. Ironically, bing, which I had always regarded as inferior, does manage to give better results in some cases, but even still I feel like the quality has (somehow?) managed to go down as well.

Lately I've been trying to use mojeek, which (to my understanding) unlike other sites like DDG actually has its own crawler whereas most alternatives are just frontends for google/bing. The results are kind of wonky a lot of the time, but at least it's not so much fake unrelated garbage.

I do have an adblocker on all the time. Perhaps that's related. Maybe I shouldn't be surprised that my experience is so shitty given that I'm clearly not their target audience, if we're just talking about advertising.

Just this morning I noticed that ChatGPT (which I usually hate using) was giving me better results than google. Not just in a little way, the experience was about 100x better. Theory: they're trashing their search engine product to try to force people onto their "AI" products. Probably not that far-fetched. If they really want to push one product over the other you can either make one product a lot better than the other or make the other product a lot worse.

willing to bet google is garbage now because of all the AI-run “blogs” that post unhelpful idiotic filler “articles” on every topic under the sun

edit: i despise this shit so much that i made this dissection of a bullshit AI article: https://i.imgur.com/Hr1wffj.png

Google search failed to even find a hollywood movie, even after 1 hour of attempts.

That's just not believable. What was your search criteria?

It got so bad, I mainly use duckduckgo (95%) as of about two months ago

I've heard the theory that it's LLM-generated spam content ruining the remaining results. There's presumably just so many webpages with heaps of garbage text now, that search engines need to aggressively filter anything that looks remotely like spam, including lots of legitimate content.

I do find it kind of terrifying, too. It's happened a few times now that I remember some event from a year ago or so, sometimes even being relatively certain what the title of an article was, and I just can't find anything about it. As if it had never happened.

This is why

The long and short of it - Google search was designed at a time when the web was in its infancy. Basically just text and a few images.

Fast forward to today, and reddit is the only one that still allows its data to be crawled.

As media has become more social (basically all of it) the walled gardens prevent you from even viewing content without an account.

Every platform wants you to be searching inside their service.

Google is useless.

duckduckgo has been working well imo

Google search failed to even find a hollywood movie

Do you understand what a difficult problem this is though? You're searching for a movie without knowing the title, the release year, the studio, the actors, or anything else.

The medium you actually want to search is the entire back catalogue of Hollywood movies. And, we're talking the movies themselves -- not text, but motion pictures, audio and video. Finding a way to search audio-visual content is extremely challenging because you effectively need a computer to "watch" the movie and understand it.

Failing that, a second-best way to accomplish what you want is to search the movie scripts that were used to film the movie. That's a much easier problem in that they're text. But, it's a hard problem because the movies, the scripts, etc. are all owned by Hollywood studios who are notoriously against any new technology they don't control, that changes the paradigm in any way, etc.

If that isn't possible, the only remaining way of doing this task is to search through the web for commentary about the movie. For a big movie that made millions and has tons of reviews you might have some luck, because there might be a body of text that reflects what happens in the movie. You're basically relying on reviewers / discussions translating the audio-visual medium of the film into text that the search engine can find and index. But, you need enough discussions of the movie to make that possible.

A user here actually recognized your description of the plot and identified the movie as "John Dies at the End". Again, without relying on someone who has seen the movie, can you imagine how hard this would be for a search engine to do? It would have to watch and listen to something in an audio-visual medium, and understand what it saw enough to form a plot summary. Instead, you were lucky enough to come across a human who had seen and remembered the movie.

But, the movie you were searching for shows why it was so hard to find. This is a 2012 movie that grossed $141,951 according to IMDB, with an opening weekend of $12,467. This movie made $0.1 million, meaning almost nobody saw it. If you had known that Paul Giamatti and Clancy Brown were in it, you probably could have found it relatively quickly by searching their IMDB pages. But, as an aside, it's pretty amazing they did a movie that was made on such a tiny budget. Normally just getting one actor like that would blow through hundreds of thousands.

Anyhow, I think what has happened is that SEO has become better, walled gardens have blocked off Google from indexing huge areas of the web, and, most importantly, people's expectations have become much higher. Back when John Dies at the End was released, nobody would have expected to be able to find a movie based on searching for a vague description of the plot, unless they were using the exact right keywords and expected to find reviews using those keywords.

The kinds of things major search engines can do today are frankly like magic. You can search for a vague description like "actress who was in the movie with the blue people", and holy shit, of the text links, Avatar's Wikipedia page is the first one, and Zoe Saldaña's is the second. I mean, just stop for a second and think about how amazing that is.

It's on purpose.

You spend longer IN Google, so you see more Google ads, on a Google platform, so Google gets a bigger cut of the pie.

It's the same reason Google started summarizing Wikipedia (or other highly rated results) on its search results where possible. Why they built basic functionality (timers etc) into their search engine.

This is what capitalism does. A constant battle of finding the lowest quality to price ratio. Everything will naturally gravitate to the shitiest cheapest version of itself.

Google still works in languages other than English, like my workaround has been to just search in Estonian and I'll usually actually get better results and like zero AI content (AI sucks at Estonian, can't even get grammar right). So if you wanna use Google learn an obscure language.

I would be very interested to know exactly what you were searching for.

I have noticed this. I have a few searches that I do regularly, and over time I've watched the results get less and less relevant for the same keywords.

One of the more recent searches was for a set of data I had been building. I had the keywords from my notes, and when I went to search for it again, using the same keywords that found it the previous times, it was no longer a result. I knew the dates of one event in particular, so I narrowed to that, and still google served me results for ten years before the specified date range. A bit more fine tuning, and Google continued to serve the same results, all not even remotely close to what I was after, and results that were found even as recently as last week are not longer there.

Google has been useless since they started "customising" search results for individual users/browsers. That was what, ten years ago?

If they've found a way to make their web search even worse, I have to applaud them for winning the race to the bottom.

Are there search engines that still work?

Qwant, Mojeek, Startpage, Ecosia. You could look for trustworthy SearchX instances too. Even Duckduckgo is better than Google (meaning better than nothing).

"We need better training data for our AIs. Let's introduce some random scramble into search results, and when users have to hunt through the list and pick what they actually wanted instead of the top result, we can use those data to train the AI how to respond to those words when they come up in AI prompts."

-- a Google exec, probably?

Google’s search results have definitely declined in recent years. It’s why I’ve mostly been using Perplexity for searching now.

No, can't say I had issues like that.

And I will say that while I think Google Search has become poisoned by fake/AI results, it's actually marginally better on Google than on something like DDG. It feels like all major search engine scraper developers just gave up on hte cat-and-mouse of blocking shit content and slowly it's all succumbing to endless SEO bullshit. 1995 Altavista all over again ;_;

Duckduckgo has gotten good enough that they're being more brave with ads: the first several results are always ads for me now, such that I usually have to scroll to get ito good results. I don't begrudge the ads; ddg doesn't track users, and ads are how they fund the service.

Lately, I've switched my default engine to a good searx instance. When I'm not looking for a business, it gives me better results. However, when I am loojing for products or services, DDG is better. DDG seems to prioritize commercial interests, either intentionally or not. I suspect it has something to do with SEO; maybe searx ignores a lot of that.

I also find that Bing is providing better results than Google, lately.

Finally, here's one of the best search engine resources I've come across recently:

Search Engine Party