Monitoring Forum Posts and Sending Email Notifications with Python

Introduction

Staying up-to-date with the latest forum posts can be time-consuming, but what if you could automate the process and receive email notifications whenever new posts are added? In this tutorial, we'll explore how to monitor forum posts and send email notifications using Python. We'll utilize web scraping techniques, the BeautifulSoup library for HTML parsing, and the built-in smtplib module for sending emails. Let's get started!

Prerequisites

Before we begin, make sure you have the following prerequisites in place:

Python installed on your system
Basic knowledge of Python programming
Access to an Outlook/any mail account or SMTP server.

Setting up Python Environment & Dependencies Installation

Let's start by setting up our development environment. Open your preferred text editor or integrated development environment (IDE) and create a new Python file- notification.py

Our code requires the beautifulsoup4, requests, and bs4 library for HTML parsing. To install it, open your terminal or command prompt and run the following command.

pip install beautifulsoup4, requests, bs4

Implementation

In our code, we'll continuously monitor a forum page for updates and send email notifications whenever new posts are detected. Let's go step by step through the code.

First, we need to import the required libraries. Add the following import statements at the beginning of the notification.py file.

import requests
from bs4 import BeautifulSoup
import time
import smtplib
from email.mime.text import MIMEText

Next, we'll define some variables to configure our email notification settings. Update the following variables in the code.

# Update with the respective settings
smtp_server = 'YOUR_SMTP_SERVER'  
smtp_port = 587 
smtp_username = 'YOUR_EMAIL_ADDRESS'
smtp_password = 'YOUR_EMAIL_PASSWORD'  
recipient_email = 'RECIPIENT_EMAIL_ADDRESS'  

temp = None
interval=60 #seconds

# URL of the forum page
url = 'https://www.c-sharpcorner.com/forums/'

Make sure to replace 'YOUR_SMTP_SERVER', 'YOUR_EMAIL_ADDRESS', 'YOUR_EMAIL_PASSWORD', and 'RECIPIENT_EMAIL_ADDRESS' with the appropriate values.

Now, let's define the core functionality of our code: monitoring the forum page and sending email notifications.

We'll create a function named check_for_updates() that will scrape the forum page, check for new posts, and send email notifications when necessary. Here's the function implementation.

def check_for_updates():
    global temp
    response = requests.get(url) 
    soup = BeautifulSoup(response.content, 'html.parser')  

    div_forum_posts = soup.find('div', {'id': 'forum_posts'})  
    ul_latest_article = div_forum_posts.find('ul', class_='latestArticle threads') 
    if ul_latest_article and ul_latest_article.text != "No Latest Questions!" and ul_latest_article != temp:
        content = ul_latest_article.get_text().strip()  
        temp = ul_latest_article
        send_email_notification(content)  # Send email notification
    else:
        print("There are no latest questions")

In this function, we use the requests library to send a GET request to the forum page specified by the url variable. Then, we parse the HTML content using BeautifulSoup. We locate the specific <div> and <ul> elements that contain the latest forum posts.

We perform some checks to ensure that there are new posts available. If new posts are found, we extract their content, store it in the content variable, and call the send_email_notification() function.

Let's implement the email notification functionality in the send_email_notification() function.

def send_email_notification(content):
    msg = MIMEText(f"There is a new question on the forum: {content}")
    msg['Subject'] = "New Question on the Forum"
    msg['From'] = smtp_username
    msg['To'] = recipient_email

    try:
        with smtplib.SMTP(smtp_server, smtp_port) as server:
            server.starttls()
            server.login(smtp_username, smtp_password)
            server.send_message(msg)
        print("Email notification sent.")
    except smtplib.SMTPException as e:
        print("Failed to send email notification:", str(e))

In this function, we create an email message using the MIMEText class from the email.mime.text module. We set the subject, sender, and recipient email addresses. Inside a try-except block, we establish an SMTP connection with the server, log in using the provided credentials, and send the email message.

Once we've implemented the necessary functions, it's time to run the code and monitor the forum page for updates.

At the end of the script, add the following code to continuously check for updates.

while True:
    check_for_updates()
    time.sleep(interval)

This code runs an infinite loop, calling the check_for_updates() function at the specified interval (in seconds) using the time.sleep() function. Now run the code using Python notification.py.

The output will be as follows.

python notification.py

Output of Python notification.py

Conclusion

You've successfully built a Python script that monitors the forum page for updates and sends email notifications using SMTP. By automating this process, you can stay up-to-date with the latest forum posts without having to constantly check the website.