Donate

WebClient Slow In Crawling Or Web Scraping A Website In C#

Here's a tip i got from stack overflow on webclient slow on web crawling.
1
2
3
ServicePointManager.DefaultConnectionLimit = int.MaxValue;  
 ServicePointManager.MaxServicePoints = int.MaxValue;  
 ServicePointManager.MaxServicePointIdleTime = 0; 
I simply changed the default connection limit and max service points
to a numeric value. Then, the crawling starts to speed up.

Greg

Comments

Donate

Popular Posts From This Blog

WPF CRUD Application Using DataGrid, MVVM Pattern, Entity Framework, And C#.NET

How To Insert Or Add Emojis In Microsoft Teams Status Message

Bootstrap Modal In ASP.NET MVC With CRUD Operations