Crawling and Download Data from Non-SSL WebPages

The code below will help you to crawl the entire page into single variable.The code majerly depends on WebClient class which is in System.NET namespace. Creating object of which will get you access to methods such as object.DownloadData which will crawl the entire webpage for the URL provided.

byte[] aReqHTML;
string myString = null;
ArrayList a = new ArrayList();
WebClient objwc = new WebClient();
aReqHTML = objwc.DownloadData(url);
UTF8Encoding utf8 = new UTF8Encoding();
myString = utf8.GetString(aReqHTML);
return (myString);

In this set of code the same WebClient object is used to download Data i.e. Images,FIles from the Webpage.And the Downloaded file is saved on the desired location on the local system.

Uri path = new Uri(TextBox2.Text);
WebClient wc = new WebClient();
string a = "Image";
wc.DownloadFile(path, Server.MapPath(a) + ".jpg");