Caffeine, spiders and robots, oh my!
Do you remember to ask Google to index your website when you make changes on it?
No? Well, it’s not the end of the world, you might not need to.
It just helps sometimes and in this day and age with Brexit and other challenges - we could all do with a little bit of extra help.
SEO - we all want to be in the top drawer of the World Wide Web filing cabinet…
As you already know, Google is a humongous search engine. Its indexing system is like an extremely efficient PA with a huge filing cabinet where we all wish we could be filed in the top drawer so we were the easiest to find.
If you’re already on that first page of Google search, congrats! No matter the journey you took, the expense and clever marketing - well done, it’s not easy.
BBC computers, dancing and curiosity…
I wish I had listened in my computer studies lessons at school instead of writing notes to people in my class. My parents decided to choose computer studies as one of my options - I wasn’t interested. I was a dancer with choreography on my horizon but they thought if I sat playing on a BBC computer it might be useful for my future.
They ruined my options! I'm not bitter.
However, I’m interested in computers now and curious about how Google looks at my website. In particular - I’m curious about caffeine, spiders and robots.
Google will index your website automatically but when they are ready unless prompted. You can request Google to index your website each time you make changes but even then you go into a queue. Websites that are well established, that have great content and give value to their readers tend to get a more regular automatic indexing by Google.
From my understanding...
Googlebot is the search bot used by Google to index websites
Caffeine is Google’s advanced indexing system
Caffeine uses little crawlers to index websites
The busy little crawlers are called spiders
Spiders scatter around when they visit sites and they use the links on the website to discover other pages
The software that Google uses is especially drawn to new websites, changes to existing websites or dead links. The Google spiders love to climb all over fresh content - blogging is a great way to keep fresh content on your site
404 pages should be dealt with as soon as possible - spiders don’t like them and they leave a bad taste in Google’s mouth
Google have webmaster tools to give website owners choices of how Google crawls their websites
You can request that Google sends its group of overworked spiders to re-crawl your website
You can opt out of spider crawling and use a file called robots.txt. The robots.txt is an exclusion standard used by websites to communicate with web crawlers and other web robots
Google's crawlers take note of key signals such as fresh content and keywords on websites
Forever curious and open to a conversation about Google indexing or similar.
If you would like to use my content writing services please email me at email@example.com or call me on 07800887857.