Michael Bromley 3322a5633e docs: Fix scraper config		2 年前
..
.env.example	2645afb855 docs: Add complete plugin tutorial	2 年前
config.json	3322a5633e docs: Fix scraper config	2 年前
readme.md	2645afb855 docs: Add complete plugin tutorial	2 年前

Scraper

This directory contains the config to run the Typesense Docsearch scraper.

First you need to copy the .env.example file and fill in the values for your Typesense instance.


### Local usage

To populate a locally-running Typesense instance, change the `config.json` file to point to the locally-running docs site:

json { "start_urls": [

"http://host.docker.internal:3000/"

], "sitemap_urls": [

"http://host.docker.internal:3000/sitemap.xml"

], "allowed_domains": [

"host.docker.internal"

] }


Then run:

bash docker run -it --env-file=.env -e "CONFIG=$(cat config.json | jq -r tostring)" typesense/docsearch-scraper:0.7.0 ```