I am developing a website that allows the users to enter a URL of the web page for promotions. I need some help on coding that would scan the given URL and downloads all HTML Markups placed inside a
Here is the concept I'm using
protected void ScanLinks(string url)
{
WebClient client = new WebClient();
string html = client.DownloadString(url);
HtmlTag tag;
HtmlParser parse = new HtmlParser(html);
while (parse.ParseNext("div", out tag)) //need to modify here
{
string htmlMarkups;
if (tag.Attributes.TryGetValue("class", out htmlMarkups)) //and here
{
TextBox2.Text = htmlMarkups;
}
}
}
What changes I need to make in above coding to rid out. Please suggest its alternative way also.

Sam HobbsPosted Oct 20, 2014, 7:32 PM
Richard, I am not sure how to respond to "retrieve a link containing what looks like hidden data". I am not sure how familiar you are with HTML. If I understand what you are asking then you would be able to easily answer the question yourself if you were familiar with HTML.
Note that Abhimanyu is using WebClient to download the HTML. My article Introduction to Web Site Scraping describes how to parse HTML but it gets the HTML from a WebBrowser control. I am not sure where your HTML is coming from.
I suggest not using the HtmlParser class. It might have bugs and limitations and if so then it could result in making more work for you. My article describes using the parser written and supported by Microsoft. It will take time to learn but it will end up saving you time overall.
Richard ArnoldPosted Oct 20, 2014, 6:19 PM
You referred to another post of yours, i.e. "get classes using the DOM"
I searched for this but didn't find it. Can you provide me with the link?
Also, will getting the class using the DOM allow me to retrieve a link containing what looks like hidden data, i.e.
This is a download link, which if I right-click on it and choose "Inspect Element (Q) I'm able to plainly read the desired information, but I need to programmatically read this link with its href info.
Thanks for your help.
Sam HobbsPosted Feb 27, 2012, 1:20 AM
Also, you might have a problem using WebClient because many web sites execute scripts in the page that adds more content to the page. If that is done then you will not get the additional content using WebClient.
For the benefit of others, note that this question was also asked in Reading HTML Markup using C# in ASP.NET.
Sam HobbsPosted Feb 27, 2012, 12:15 AM
I am not familiar with HtmlParser so I can't help you with the other part. This question is actually two separate questions; the "className" instead of "class" secret applies to other parsers too, such as IE.
I am familiar with IE and I know how to do it using IE (SHDocVw and mshtml).