Your final paragraph is very helpful, since I'm flailing in the same way for the same reason. I have bots and spiders blocked because I don't want a bunch of strangers showing up from all over the web -- but as my list of memories grows to where I need search engines just for them, maybe it's worth it. I'll never have enough links to my journal to get a PageRank above 827 or so anyway, so what's to worry about?!
If you unblock spiders, will Google crawl all the way back and index all the public entries? Or only the ones that are posted after the unblocking?
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
no subject
Your final paragraph is very helpful, since I'm flailing in the same way for the same reason. I have bots and spiders blocked because I don't want a bunch of strangers showing up from all over the web -- but as my list of memories grows to where I need search engines just for them, maybe it's worth it. I'll never have enough links to my journal to get a PageRank above 827 or so anyway, so what's to worry about?!
If you unblock spiders, will Google crawl all the way back and index all the public entries? Or only the ones that are posted after the unblocking?