Related Links selection

Links for pages are selected to maximize relatedness subject to the constraint that every page receives the defined number of related incoming links in your configuration per Module. The relatedness of a pair of pages is computed as the semantic similarity of the page text.

When a page does not have enough related pages, or a page has not yet been indexed, the API returns links to randomly selected pages.

You can check the number of related links being selected for each Module by reviewing your Configuration Modules number of links.

Uptime and latency

The API is built on standard AWS services and as of May 27, 2022 we have had no major outages.

Average response time: 150ms
Uptime: 99.9%

Requests rate limits

The API endpoints are not restricted by request rate limits; however, we encourage keeping the requests under 20 requests/second per endpoint. Updating data for a set of 10k pages will be done in less than 10 minutes.

When using the API endpoints to get data by batches, the API users should plan their jobs accordingly, considering the number of pages and the data update period, which could vary from one day to one week. Hitting the API on every final user request is discouraged.

Caching

Server-side rendering with caching is strongly recommended. The results can be cached using the endpoint URL with query string parameters as the key.

We perform a recrawl and reprocessing of the links every 24 hours, it's recommended to cache the endpoint responses with a TTL of around 24 hours. The input parameters can be used as the key.