scraper

package

v0.0.0-...-ca1e246 Latest Latest Go to latest Published: Jan 29, 2025 License: MIT Imports: 15 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/hahaclassic/algorithm-analysis

Documentation ¶

Index ¶

Variables
func BenchmarkScraper(baseURL, saveDir string, numOfRepeats, maxPages int)
type Scraper
- func New() *Scraper
- func (s *Scraper) Start(baseURL, dirPath string, maxWorkers, maxPages int) int

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	ErrCreateDir   = errors.New("scrapper: can't create directory")
	ErrLoadPage    = errors.New("scrapper: can't load page")
	ErrSavePage    = errors.New("scrapper: can't save page")
	ErrExtractURLs = errors.New("scrapper: can't extract urls")
)

Functions ¶

func BenchmarkScraper ¶

func BenchmarkScraper(baseURL, saveDir string, numOfRepeats, maxPages int)

Types ¶

type Scraper ¶

type Scraper struct {
	// contains filtered or unexported fields
}

func New ¶

func New() *Scraper

func (*Scraper) Start ¶

func (s *Scraper) Start(baseURL, dirPath string, maxWorkers, maxPages int) int

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL