Interlinker: a real time web crawler
Available at
@paoloose/discmaths/project3What if you can see how the web is created?
It's impossible to map the entire internet, but what if we can generate an small portion of it?
How it works?
- You seed the crawler with a url ('the origin')
- It will start crawling that page, looking for links (
<a/>tags) - The links found will be added to a queue
- It will repeat the process for each link in the queue
You can feed multiple origins to the crawler, and it will crawl them in parallel!
Start and paste a link in the search box to see it in action!