Search files and folders from any website URL

Hi,

I am trying to find the files from the URL path folders.that URL may be any website. for example

Input: http://www.dreamincode.net/forums/topic/77912-ftp-download-delete/

First need to parse that URL and check it out,is there any log files or document files are there within that folder. That URL is not belongs to local path, it may third party site like www.google.com/ or something else.

In our example, that folder may forums , topic and 77912-ftp-download-delete

*.log or *.doc like that..

that searching process going till end of the URL. I tried some of the way but i cant get the solutions. if any body knows the solution, please update here..

Thanks in advance,

Regards,

sasi

Here is a ten minute code from mind and google. I hope this will help you :)

private void button1_Click(object sender, EventArgs e)
{
    List result = new List();
    Uri myUrl = new Uri("http://www.dreamincode.net/forums/topic/77912-ftp-download-delete/");
    string clean = myUrl.Scheme + "://" + myUrl.GetComponents(UriComponents.HostAndPort, UriFormat.UriEscaped);

    for (int i = 0; i < myUrl.Segments.Length; i++ )
    {
        clean += myUrl.Segments[i];
        result.AddRange(ParseSite(clean));
    }

    // tada! result has all your files, hope so ;)
}

private string[] ParseSite(string url){
    List result = new List();
    string data;

    // get the data with httprequest, in data , not in mind now ;)

    MatchCollection m1 = Regex.Matches(data, @"(.*?)",
        RegexOptions.Singleline);

    foreach (Match m in m1)
    {
        string value = m.Groups[1].Value;

        Match m2 = Regex.Match(value, @"href=\""(.*?)\""", RegexOptions.Singleline);
        if (m2.Success){
            string file = m2.Groups[1].Value;
            if (file.EndsWith(".doc") || file.EndsWith(".log") /* or whatever you want */ )
            {
                // if file starts with http:// it absolue else you have to re-add the host here, easy task ;)

                result.Add(m2.Groups[1].Value);
            }

        }


    }
    return result.ToArray();
}

1 Reply

Know the answer? Post it — somebody with the same question will find it here.

Benjamin Kemner15y ago
Here is a ten minute code from mind and google. I hope this will help you :)

private void button1_Click(object sender, EventArgs e)
{
    List result = new List();
    Uri myUrl = new Uri("http://www.dreamincode.net/forums/topic/77912-ftp-download-delete/");
    string clean = myUrl.Scheme + "://" + myUrl.GetComponents(UriComponents.HostAndPort, UriFormat.UriEscaped);

    for (int i = 0; i < myUrl.Segments.Length; i++ )
    {
        clean += myUrl.Segments[i];
        result.AddRange(ParseSite(clean));
    }

    // tada! result has all your files, hope so ;)
}

private string[] ParseSite(string url){
    List result = new List();
    string data;

    // get the data with httprequest, in data , not in mind now ;)

    MatchCollection m1 = Regex.Matches(data, @"(.*?)",
        RegexOptions.Singleline);

    foreach (Match m in m1)
    {
        string value = m.Groups[1].Value;

        Match m2 = Regex.Match(value, @"href=\""(.*?)\""", RegexOptions.Singleline);
        if (m2.Success){
            string file = m2.Groups[1].Value;
            if (file.EndsWith(".doc") || file.EndsWith(".log") /* or whatever you want */ )
            {
                // if file starts with http:// it absolue else you have to re-add the host here, easy task ;)

                result.Add(m2.Groups[1].Value);
            }

        }


    }
    return result.ToArray();
}