Comments on: Does Anything Really Disappear from the Internet?

By: Marie

Marie — Thu, 17 Nov 2005 00:09:30 +0000

Deleting the URL from your blog program, doesn’t necessarily delete it from the net.

Tip: To “delete” content from places like Bloglines’ cache, replace the words on the page with new words, or a dot, or something else. But, do it quickly – before you post the number of posts (lastn=”_”) specified in your feed file. I hope that makes sense.

Anyway, when Bloglines makes its next pass, it’ll cache the new words and the old words will be gone. In theory, this should work with Goggle’s bot, too.

By: Mike

Mike — Wed, 16 Nov 2005 21:45:03 +0000

Google Concurring Opinions. (.) You’ll notice a date on the 4th line of the first result. In green font it reads “Nov 14, 2005.” That’s the last time Google crawled your site. Generally Google crawls popular (as measured by incoming links/page rank) and reguarly updated blogs every 2 or 3 days. So you have at least a day to keep something you posted out of Google.

Now, as someone noted, if you publish an RSS feed, you have a shorter lead time. It depends on how long it takes from various RSS readers to pick up you feed. Bloglines (rough estimate) usually takes from 1 to 3 hours to pick up a feed. So if you write something you regret, but delete it quickly, you have only a short time before your Bloglines subscribers read it.

Of course, as your blog gets more popular, there is a good chance that at any give time, more than one person will be reading your blog. So, putting aside technical issues to answer your question… Concurring Opinions ain’t no empty forest.

By: John Armstrong

John Armstrong — Wed, 16 Nov 2005 18:16:04 +0000

I used to blog back in college before the word was invented. Had to hard-code the HTML myself. That site is mostly dust in the wind. I’ve only ever been able to find the splash page archived anywhere, imploring surfers to look on my works and despair.

Could a site be up for more than a year and be scoured from the net anymore? I doubt it.

By: Paul Gowder

Paul Gowder — Wed, 16 Nov 2005 17:00:52 +0000

It always surprises me that the engines seem to obey robots.txt, but they do. I don’t know why, unless they think it will insulate them from liability.

I have successfully destroyed files from the early days of the internet, but today…

By: Stephen M (Ethesis)

Stephen M (Ethesis) — Wed, 16 Nov 2005 14:53:23 +0000

I’ve actually spent time trying to run things down that went away. Especially comments on blogs that have been deleted, etc.

The archive cycle, especially a while back, was not an hourly or even a daily one.

By: brewster kahle

brewster kahle — Wed, 16 Nov 2005 08:05:54 +0000

Yes, you can stay out of the wayback machine on archive.org and out of most search engines– you put a robot exclusion on your website. you can also just stay out of the wayback machine and in the search engines etc. search on “robot exclusion” and you will find the magic incantations.

-brewster

By: Michael Froomkin

Michael Froomkin — Wed, 16 Nov 2005 07:38:40 +0000

Sure it can vanish... if you didn't send it anywhere (e.g. via RSS), and if no one came to read it in the interim. For unpopular unlinked sites that can be a very long time. For a site like this one...well, just hope you didn't ping any sites to come and do updates, or have the bad luck to be visited by a googlebotor other robot, not to mention a person who kept a copy). As a practical matter, how long the 'window of forgiveness' may be depends on your traffic...and luck. But sure, lots of old stuff is gone forever, and new stuff too can vanish un-archived, especially if you get to it quickly enough.

[And if your sever is on Unix, when a file is deleted/changed it is much more erased than on Windows.]