.Google.com's Gary Illyes discloses Googlebot gathers hyperlinks rather than "observing" all of them, negating main documentation.Googlebot accumulates hyperlinks rather than following all of them in real-time.Gary Illyes discussed this details in Google.com's 'Search Off The File' podcast.Google.com's records opposes the actual crawler habits.