kfir/Web_Crawler_API
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
Web_Crawler_API/public/robots.txt

3 lines
24 B
Text
Raw Normal View History

first
2023-05-30 09:56:38 +00:00
User-agent: *
Disallow:
Reference in a new issue Copy permalink