The most important news of the last month for all website traffic was the Google’s confirmation that both “Discovered – Currently Not Indexed” and “Crawled – Currently Not Indexed” Status could Last forever” in the Google Search Central. In other words, they may crawl a webpage, but they may choose not to index it and unless interdicted, they may never come back to it again.
Till now, this would have typically happened only if the Googlebot was unable to index, because your site was coded in some obsolete language, or was extremely slow, or you had no-index, or a block on the page. As of last year, “Googlebot may decide that some crawled pages on the domain aren’t worth indexing and decides not to index them by looking for a pattern in their URL, meta-information, or content.”
The recommendation by Google is to ensure that the pages have fresh titles and fresh content. In case of shows, here is a general example of a time when DWG helped address a title freshness issue for one of our art clients on ARTdynamix™:
The venue has a repeating annual one-day program, and chooses to promote the program by year and program name, e.g. alphabet-soup-series-2021 page and alphabet-soup-series-2022. In this case, since the 2021 page had been crawled and the meta titles for both pages were alphabet soup series, despite clear differences in content – because some of the content was pulled from the previous year’s page -the bot chose not to index the page.
To resolve this problem, we first 301 redirected the 2021 page to the 2022 page to ensure that the appropriate information reaches the consumer. Second, we requested an indexing of the 2022 page which took a while but was eventually done. Once the 2022 page was indexed, we removed the 301 redirect from the 2021 page. And finally, we modified the title tag for the 2021 page, renamed it redirected the old link, and removed the duplicate content, in case the bot returns to look at these pages. Of course, the time is worthwhile to make sure the page is found!
We encourage you to use more descriptive meta titles, for example if 2021 alphabet soup series is about Diversity, Equity & Inclusion then the meta title of the Diversity-Equity-Inclusion-series page should be 2021 Diversity, Equity & Inclusion program for 2021 alphabet soup series, and for the Disability-Justice-Series page(2022 alphabet soup series) the meta title may be Disability Justice program for 2022 alphabet soup series.
This rule is applicable if you are a presenting organization or have shows of your own and choose to have topical series included and is not necessarily the same for those membership organizations that hold annual conferences, or if you have e-commerce or content site. Googlebot is smart enough to tell the difference and treats them differently.
But no matter what kind of site you have, at the end of the day, you should have an indexing strategy in mind, and create a well-optimized site map and avoid duplicate content.
–Nami
Credit to Google updates, think, search, and tools