diff --git a/README.md b/README.md index 68655f8..b8f6a57 100644 --- a/README.md +++ b/README.md @@ -1 +1,19 @@ -# Crawing && Serving # \ No newline at end of file +# Crawing & Serving # + +The crawler is a simple crawler that crawls the web and stores the results in a database and assets in a file system. The server is a simple server that serves the results of the crawler. + + +## Crawler ## + + + +### Usage ### + +Post a JSON object to the crawler with the following format: + + `domain.com/crawl` + { + "url": "http://www.example.com", + } + +The crawler will then crawl the given url and store the results in a database and assets in a file system. \ No newline at end of file