|
|
95e4b27bc3
|
Added timeout
|
2021-05-08 21:07:00 +02:00 |
|
|
|
a09bc1da54
|
Check if file size small enough
|
2021-05-08 20:58:00 +02:00 |
|
|
|
9fa95aba03
|
Also parse non-index urls
|
2021-05-08 20:31:44 +02:00 |
|
|
|
0462614a22
|
Cleaned logging
|
2021-05-08 19:58:05 +02:00 |
|
|
|
5ca786b1ae
|
Added exception handling during get
|
2021-05-08 18:15:32 +02:00 |
|
|
|
a9a07aabea
|
Don't load already visited sites
|
2021-05-08 18:00:49 +02:00 |
|
|
|
3866b2be0f
|
Wrote first version
|
2021-05-08 17:43:31 +02:00 |
|
|
|
b3e781bd98
|
Initial commit
|
2021-05-08 14:56:51 +00:00 |
|
