Webclient doesn't support frames

Question

Hello people! This is my first post here. I am currently developing an application that is supposed to get data from a website, get some product information, then place it in a database etc.

I am using webclient but the url I want to retrieve from only returns part of the page and with the message "your browser does not support frames".

How can I get the "full" page downloaded so I can parse the html(string) using my RegEx expressions? In other words get the same html as I get when I manually use the right-click and view source on the url?

I will paste some info below. Thanks in advance for all help.

WebClient fileReader = new WebClient();
fileReader.Credentials = credentialCache.DefaultCredentials;
fileReader.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
Stream data = fileReader.OpenRead(url);
StreamReader sr = new StreamReader(data);
string html = sr.ReadToEnd();
infoTextBox.AppendText(html);

Mauro Balsano · Answer

Hi Joakim, how did you solve exactly the problem? I've the same problem, and I try to solve it with your suggestions, but without success... So, can you post code of your solution? Thanks a lot.

Joakim Astrom · Answer

Alright. I've come up with a solution. It's not a great one but maybe it will help someone. I use the WebBrowser object and manually login to the site. Then I use the specific webpages and use the browser object to get the "stuff". As I said, the WebClient object doesn't return all the data and gives a "your browser doesn't support frames" message. Clues? If you use the WebBrowser object like me be sure to "catch" the returned page in DocumentComplete event or you will be confused :) If anyone has tips for automatic login or a clue as to why I can't download it all in the WebClient or HttpWebRequest objects please let me know! Thanks. Cheers!

Joakim Astrom · Answer

Thanks for the effort! But this is where I get the problem. The return string of that code you provided (using the directlink which is the link with the product info I am interested in) is just this: frame name = "header" scrolling = "no" noresize target = "main" src="/europe/corporate/jsp/navbar/c_headernavbar.jsp?printMarkup=n" frame name="main" src="/corporate/PageFinder.jsp?pageID=&pagePath=/login/failure" marginwidth="0" marginheight="0" scrolling="auto" noresize target="_self" This page uses frames, but your browse r doesn't support them. I can't paste exactly how the html looks in here but maybe you get the picture? Seems to be two problems. One, being logged in, and two, my browswer client doesn't support frames!? The whole "body" of code is sort of cut off. But when I manually use the directlink and view source (being logged in off course) I get all the generated text/code/info I am looking for. What am I missing? Thanks in advance.

Kirtan Patel · Answer

private void btnDownloadSourceCode_Click( object sender, EventArgs e) { WebClient wc = new WebClient (); string Source= wc.DownloadString( "http://se.ingrammicro.com/corporate/corporate.jsp" ); textBox1.Text = Source; }

Joakim Astrom · Answer

Example of a "direct-link": http://se.ingrammicro.com/search/datasheet?skuNumber=1355556&source=search/europe/corporate/jsp/navbar/c_headernavbar.jsp?printMarkup=n

Joakim Astrom · Answer

..for what it's worth :). http://se.ingrammicro.com/corporate/corporate.jsp

Joakim Astrom · Answer

Well hello and thank you :). I can give you the url but unfortunatelly it requires login and that information is not mine so I can't give it to you I'm afraid. That is the other problem I have. I thought I could login manually and then run my scraper but it seems to require login from the code. Ideas for that? Anything else I can provide you that will help?

Kirtan Patel · Answer

friend, what webpage url you are trying to download ?? tell me the page url may be i can help you :)

Joakim Astrom · Answer

...and sorry about the chunk of text. I don't know how to edit it so it doesn't remove new lines etc. Possible?

Webclient doesn't support frames

Answers (9)