How to search HTML for key substrings

alec

I have a several blocks of html (full sources for websites) currently they are stored in a dataTable and I want to search each of them for image urls and key text then extract each of these pieces of data to there own string to then be filed in the DataTable (and eventually be put into the database). currently I have no code as this is a bit beyond me but my guest at how to do it is to use something to find groups of characters signifying the beginning and end of the substring I want and then using SubString(int, int) to extract it but this idea is all I have please help!

6 Replies

Know the answer? Post it — somebody with the same question will find it here.

Sam Hobbs15y ago
Are you familiar with the eBay Developers Program? It is likely it will help and if so then it is much better to use it. You can do most anything with it, except a few things such as sniping that eBay does not like.
alec15y ago
Hi sam I would like to know how to do this. and I wish I was getting the data from eBay they would give it willingly alas ebay is an in house acronym that as far as I can tell is meaningless
Sam Hobbs15y ago
I am not sure of what you are doing, but it is highly likely that I would do it very differently. I would put the HTML into a HTML document and then use the existing parser to get the data from the HTML. The existing parser is the exact same thing used by IE to parse HTML. Are you interested in knowing how to do that?

Note that there is probably a much easier way to get the data from eBay.
alec15y ago
this code fetches the page source (its html) and then splits it about the characters I know begin the image name and end the image name and then searches the string[] for all possible image name strings using regex as you sugested

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Net;

namespace extractImageUrl
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}

private void button1_Click(object sender, EventArgs e)
{
HttpWebRequest myWebRequest = (HttpWebRequest)HttpWebRequest.Create("");

myWebRequest.Method = "GET";

// make request for web page

HttpWebResponse myWebResponse = myWebRequest.GetResponse() as HttpWebResponse;

StreamReader myWebSource = new StreamReader(myWebResponse.GetResponseStream());

string html = string.Empty;

html = myWebSource.ReadToEnd();

string MakeBlocks = html;

string[] Blocks = html.Split('/','"');

string find = ".jpg";

string[] posibleImages;

foreach (string Block in Blocks)
{
if (System.Text.RegularExpressions.Regex.IsMatch(
Block, find, System.Text.RegularExpressions.RegexOptions.IgnoreCase
)
)
{
string[] AddBlock = {Block};

posibleImages = posibleImages + AddBlock;
}
else
{
//do nothing
}
}

}
}
}
alec15y ago
after a serious googling session I think the best thing to do might be string.splint() and split first about the characters that I know start it taking the second half then splitting about the characters I know end it and taking the first half also I have thought of a way to do it iteratively with substring() and Regex but that is slow and horribly complicated
Vulpes15y ago
For anything complicated like that, the first port of call would be regular expressions (.NET Regex class) which are far more powerful (albeit slower) than the methods in the string class.

If it's XHTML rather than plain HTML, you might also be able to use LINQ to XML if the urls etc. can be found in predictable places.

If you can post a (hopefully shortish) sample, we might be able to conjure up some regular expressions you could use.