Skip to content

Documentation sites ​

Your documentation site is the first thing Overdesk grounds a draft in. You give it an address, Overdesk reads and indexes the site in the background, and drafts can quote your articles and link to them. Drafting stays paused until at least one site is set.

Add your documentation ​

  1. Open Settings → Knowledge base.
  2. Under Documentation sync, enter the home page of your documentation in Docs site URL(s). To add more than one site, separate the addresses with commas.
  3. Select Save on that card.

Only admins can change these settings. Saving an address starts a crawl at once, and the site is read again every night after that. To start a crawl at any time, an admin selects Sync docs on the Knowledge base page.

What Overdesk reads ​

Overdesk treats the address you enter as a starting point. It follows the links on that page, and on the pages they lead to, as long as they stay on the same site and under the same address. Enter https://docs.example.com/product-a/ and Overdesk reads the pages under /product-a/ and nothing else on that site.

A few things follow from that:

  • Enter the home page of your docs, not a single article. The crawl begins at the starting page, and what gets stored is the pages it leads to.
  • By default Overdesk follows links up to 2 clicks away from the starting page. Max crawl depth raises that for sites whose articles sit deeper.
  • The site has to be public. Overdesk reads it the way a visitor would, without signing in.
  • A page needs a title and some real text to be kept, and a crawl stops at 5,000 pages per site.
  • Pages your visitors write are skipped: search results pages, comment pages and feeds, and any page a redirect takes off your site. Comment sections are left out of the pages Overdesk keeps, so a visitor's comment never becomes part of your documentation.

Fine-tuning the crawl ​

Four optional fields sit below the address. Leave them blank unless something looks wrong. Whatever you set applies to every site in the list.

FieldUse it when
Link selectorWhole sections of your docs are being missed. Point it at the links Overdesk should follow, such as your docs navigation.
Content selectorNavigation or footer text is ending up in your knowledge base. Point it at the element that holds the article text.
Title selectorArticle titles come out wrong. Left blank, Overdesk uses the page heading.
Max crawl depthArticles sit more than 2 clicks from your docs home page. Expect a longer sync.

The three selectors are written as CSS selectors, for example nav a, article or .page-title.

When a site can't be reached ​

If you list several sites and one can't be reached, the others still sync, and the run on the Runs page carries a warning in Details saying how many failed. If none can be reached, the run fails and says so there. Pages Overdesk already holds stay in the knowledge base either way.

When a page leaves your site ​

When you take a page down, Overdesk takes it out of your knowledge base too, so drafts stop quoting an article you've removed. A page is removed once two complete crawls in a row don't find it.

  • A crawl that was cut short removes nothing. That covers a site that couldn't be reached, a page that failed to load or was refused, and a crawl that stopped at the 5,000-page limit. When every site was reached and a failed page or the limit holds pages back this way, the run on the Runs page says how many in Details. A page your site reports as not found or gone doesn't cut a crawl short: that is the site saying the page left.
  • A page that still answers but isn't kept keeps its last stored copy. That covers a page that now redirects somewhere else, or comes back without a title or with almost no text.
  • When you change your docs address, or narrow the crawl with a lower Max crawl depth or a tighter Link selector, the pages that fall outside it leave the same way, after two complete crawls.
  • If a crawl would remove more pages than it found, or more pages from one site than it found on that site, Overdesk removes none. The run on the Runs page then carries a warning in Details saying what it held back and why. That usually means the site changed shape, a redesign or a new navigation, rather than lost its pages. Check the selectors above, and the next complete crawls settle it.
  • A page removed by mistake comes back on the next crawl that finds it.

A docs address is always required ​

Drafting checks for a documentation site address before every draft, and other sources don't take its place. Even when your help center articles sync on their own, keep at least one address here. See Help center articles and Point Overdesk at your docs.