Sentence parsing

direhack

Hi everyone,
I have a problem with parsing a text file into my program. The input is in a text file with sentences. Each sentence can either be on a separate line, or it can be Following the previous sentece on the same line. What I need to do is to read the whole file and then cut it up into single sentences that I can put in String[] array. For example, a file can be like this:

This is a sentence. This is another sentence.
This is a third sentence.
Fourth sentence.

However, the trick is that I want my program to read sentences that have periods inside them that do not terminate that sentence, for example:

This sentence will end at 3 p.m. on February 20th.

I was looking into regular expressions and just string.indexof() methods, but I cannot get far with them.

Thank you for all your help.

2 Replies

Know the answer? Post it — somebody with the same question will find it here.

Scott Lysle18y ago
This seems to work pretty well.

private ArrayList SplitSentences(string sSourceText)

{

   string sTemp = sSourceText;

   ArrayList al = new ArrayList();

   string[] splitSentences = Regex.Split(sTemp, @"(?<=['""A-Za-z0-9][\.\!\?])\s+(?=[A-Z])");



   for (int i = 0; i < splitSentences.Length; i++)

   {

      string sSingleSentence = splitSentences[i].Replace(Environment.NewLine, string.Empty);

      al.Add(sSingleSentence.Trim());

   }

   return al;

}

Example use:

private void button5_Click(object sender, EventArgs e)

{

   ArrayList al = SplitSentences(txtParagraphs.Text);

   for (int i = 0; i < al.Count; i++)

      //populate a list box

      lstSentences.Items.Add(al[i].ToString());

}
Alan18y ago
It's difficult to come up with a competely water-tight solution to this but try this approach (.Net 2.0 or later) which assumes that all sentences will begin with a capital letter and that all sentences, except the first, will be preceded by a dot and a space:

using System;
using System.IO;
using System.Text.RegularExpressions;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
       Regex r = new Regex(@"\. [A-Z]");
       string text = File.ReadAllText("direhack.txt");
       // replace newline characters with a single space and remove any final spaces
       text = text.Replace("\r\n", " ").TrimEnd();
       // find all occurrences of ". " followed by a capital letter
       MatchCollection mc = r.Matches(text);
       List temp = new List();
       string sentence = null;
       int startIndex = 0; // index of start of next sentence

       foreach(Match m in mc)
       {
          if (m.Index > 1)
          {
             // parse out the preceding sentence
             sentence = text.Substring(startIndex, m.Index - startIndex + 1);
             temp.Add(sentence);
             startIndex = m.Index + 2; // update startIndex
          }
       }
       // add last sentence in text
       sentence = text.Substring(startIndex);
       temp.Add(sentence);

       string[] sentences = temp.ToArray(); // convert to array
       // check it worked
       foreach (string s in sentences)
       {
          Console.WriteLine(s);
       }
       Console.ReadKey();
    }
}