robotstxt/robotstxt_test.go at master - GitHub
The robots.txt exclusion protocol implementation for Go language - robotstxt/robotstxt_test.go at master · temoto/robotstxt.
temoto/robotstxt: The robots.txt exclusion protocol implementation for ...
This is a robots.txt exclusion protocol implementation for Go language (golang). Build. To build and run tests run go test in ...
TV Series on DVD
Old Hard to Find TV Series on DVD
samclarke/robotstxt: Go robots.txt parser - GitHub
Robots Parser Build Status Coverage Status Go Report Card GoDoc. A robots.txt parser written in Go, based on the Node.js robots-parser package.
robotstxt/robotstxt.go at master · temoto/robotstxt - GitHub
Package robotstxt implements the robots.txt Exclusion Protocol // as specified in http://www.robotstxt.org/wc/robots.html // with various extensions.
google/robotstxt-spec-test - GitHub
txt Specification Test. This project aims to allow for any implementation of a robots.txt parser to be tested and have some confidence that it ...
rimiti/robotstxt: Robots.txt parser and generator - Work in ... - GitHub
A robots.txt parser and generator written in Go, based on samclarke robotstxt package. It currently supports: User-agent: Allow: Disallow: Sitemap: Crawl- ...
robotstxt/google_test.go at master - GitHub
The robots.txt exclusion protocol implementation for Go language - robotstxt/google_test.go at master · temoto/robotstxt.
grobotstxt is a native Go port of Google's robots.txt parser ... - GitHub
grobotstxt is a native Go port of Google's robots.txt parser and matcher C++ library. Direct function-for-function conversion/port; Preserves all behaviour ...
Analyze and Test robots.txt Files on a Large Scale - advertools
Bulk robots.txt Tester . This tester is designed to work on a large scale. The robotstxt_test() function runs a test for a given robots.txt file, checking ...
Commit upstream/1.1.2+git20221109.1.1fe1f4f - robotstxt ...
install: curl -sfL https://install.goreleaser.com/github ... robotstxt_test.go less more. 170, 170, assert.True(t ... git fetch origin master. 7, 7, git stash. 8, 8 ...