README.md

This commit is contained in:
Kfir Dayan 2023-04-17 19:51:48 +03:00
parent c5d5a6ac72
commit daa1f363c8

View file

@ -16,4 +16,5 @@ Post a JSON object to the crawler with the following format:
"url": "http://www.example.com", "url": "http://www.example.com",
} }
The crawler will then crawl the given url and store the results in a database and assets in a file system. The crawler will then crawl the given url and store the results in a database and assets in a file system
`crawler_assests/www.example.com/`.