Building a Conversational Chatbot Using Mastra AI

AI isn’t just a buzzword anymore—it’s become an important part of today’s software. Whether you’re building a chatbot, a virtual assistant, or a tool that works with documents, you need solid frameworks to handle the work behind the scenes.

Over the last week, I was working on one of our projects when a new challenge came up—something many of us in tech eventually face:

Building a chatbot that can actually understand questions and respond with useful, relevant information.

The project lead brought in a requirement to transform our existing AI utility into a fully functional chatbot—quickly and with minimal turnaround time—so the client could interact directly with the system and get meaningful answers.

We considered several frameworks. Python-based tools like LangChain, Rasa, and Semantic Kernel each have their own strengths, but then the team introduced me to something new that felt surprisingly lightweight and promising: Mastra AI.

Mastra AI is an open-source TypeScript framework that makes it easy to build smart agents with memory, tool integration, and step-by-step workflows. What really stood out to me was how developer-friendly it is—you get a built-in chat UI where you can test your agent, inspect its memory, connect tools like Azure Cognitive Search, and debug everything locally before deploying.

What is Mastra.ai?

Mastra.ai is an open-source TypeScript framework that streamlines the development of intelligent agents and AI workflows. It stands out for its developer-friendly features, including a built-in chat UI for testing, memory inspection, debugging, and local integration with tools like Azure Cognitive Search.

Whether you're building chatbots, automation systems, or multi-agent networks, Mastra helps you move from prototype to production with ease. It offers workflow orchestration, memory management, RAG, observability, and seamless integration with vector databases and cloud platforms.

More than just a toolkit, Mastra is a developer-first framework that bridges experimentation and deployment, making it easier to build scalable, intelligent applications quickly and confidently.

Project Structure

src/
  mastra/
    agents/
    tools/
    workflows/
    index.ts
.env
package.json
tsconfig.json

Connecting Mastra to an LLM

One of the key strengths of Mastra is its LLM-agnostic architecture. You can plug in any LLM model using a compatible TypeScript LLM package, making it easy to integrate providers like OpenAI, Anthropic, or even open-source models. This flexibility ensures that you're never tied to a single vendor and can choose the best model for your specific needs.

Example with Open AI and Azure Open AI

import { Agent } from "@mastra/core/agent";
import { openai } from "@ai-sdk/openai";
import { createAzure } from '@ai-sdk/azure'

// Create an agent with Open AI
const agent = new Agent({
  name: "assistant",
  instructions: "You are a helpful assistant.",
  model: openai("gpt-4o"),
});

// Create an agent with Azure Open AI
const azure = createAzure({
  apiKey: process.env.AZURE_OPENAI_APIKEY,
  resourceName: process.env.AZURE_OPENAI_RESOURCENAME,
});

Initializing an Agent

import { openai } from '@ai-sdk/openai';
import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';
import { LibSQLStore } from '@mastra/libsql';
import { weatherTool } from '../tools/weather-tool';
import { createAzure } from '@ai-sdk/azure'

const azure = createAzure({
  apiKey: process.env.AZURE_OPENAI_APIKEY,
  resourceName: process.env.AZURE_OPENAI_RESOURCENAME,
});

export const weatherAgent = new Agent({
  name: 'Weather Agent',
  instructions: `
      You are a helpful weather assistant that provides accurate weather information and can help planning activities based on the weather.

      Your primary function is to help users get weather details for specific locations. When responding:
      - Always ask for a location if none is provided
      - If the location name isn't in English, please translate it
      - If giving a location with multiple parts (e.g. "New York, NY"), use the most relevant part (e.g. "New York")
      - Include relevant details like humidity, wind conditions, and precipitation
      - Keep responses concise but informative
      - If the user asks for activities and provides the weather forecast, suggest activities based on the weather forecast.
      - If the user asks for activities, respond in the format they request.

      Use the weatherTool to fetch current weather data.
`,
  model: azure('gpt-4o-mini'),
  tools: { 'weatherTool' },
  memory: new Memory({
    storage: new LibSQLStore({
      url: 'file:../mastra.db', // path is relative to the .mastra/output directory
    }),
  }),
});

Initializing the Mastra Framework

import { Mastra } from "@mastra/core";
import { PinoLogger } from "@mastra/loggers";
import { LibSQLStore } from "@mastra/libsql";
import { weatherWorkflow } from "./workflows/weather-workflow";
import { weatherAgent } from "./agents/weather-agent";

export const mastra = new Mastra({
  workflows: { weatherWorkflow },
  agents: { weatherAgent },
  storage: new LibSQLStore({ url: ":memory:" }),
  logger: new PinoLogger({ name: "Mastra", level: "info" }),
});

Key Features

1. Unified LLM Access

Mastra allows you to connect to multiple LLM providers using a single interface. Switching from GPT to Claude or Gemini becomes a simple configuration change—no SDK rewrites, no integration headaches.

Use Case: Test the same chatbot logic against different LLMs and decide which gives the most accurate and relevant results.

2. Agent Memory and Tooling

One of the things I really like about Mastra is how it handles memory. Agents can remember things like user preferences, past conversations, and important facts—so responses feel more personalized and in context, even across sessions. It supports both short-term and long-term memory, with features like semantic recall and customizable memory handling.

Mastra also makes it easy to plug in your own tools or connect to external APIs and data sources. Whether it's fetching live data or triggering actions, you can extend your agent’s capabilities as needed. And with support for the Model Context Protocol (MCP), you can even share tools across platforms or scale them up without starting from scratch.

Put simply, Mastra gives you a solid foundation for building smarter, more useful AI agents that actually understand and act on what users need.

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';

const memory = new Memory({
  options: {
    lastMessages: 1,
    semanticRecall: false,
    workingMemory: {
      enabled: true,
      template: `
# Todo List
## Active Items
- Example (Due: Feb 7 3028, Started: Feb 7 2025)
  - Description: This is an example task - replace with whatever the user needs
## Completed Items
- None yet
`,
    },
  },
});

export const todoAgent = new Agent({
  name: 'TODO Agent',
  instructions: 'You are a helpful todolist AI agent. Help the user manage their todolist...',
  model: openai('gpt-4o-mini'),
  memory,
});

3. Deterministic Workflow Engine

One of the standout features I’ve worked with in Mastra is its Deterministic Workflow Engine. It gives you full control over how multi-step AI or automation processes are structured. You can define each step clearly, run them in sequence or parallel, and even handle things like branching logic, pausing, and resuming—without losing track of what’s happening.

What really impressed me is the deterministic nature of it. For the same inputs, you’ll always get the same outputs, which has made debugging and scaling in production environments a lot more predictable and stress-free. And because it’s all type-safe, you catch potential issues early on during development. It’s definitely helped us build more reliable, traceable AI workflows.

import { Step, Workflow } from '@mastra/core/workflows';
import { z } from 'zod';

const myWorkflow = new Workflow({
  name: 'my-workflow',
  triggerSchema: z.object({ inputValue: z.number() }),
});

myWorkflow
  .step(new Step({
    id: 'stepOne',
    outputSchema: z.object({ doubledValue: z.number() }),
    execute: async ({ context }) => ({ doubledValue: context.triggerData.inputValue * 2 }),
  }))
  .then(new Step({
    id: 'stepTwo',
    outputSchema: z.object({ incrementedValue: z.number() }),
    execute: async ({ context }) => ({ incrementedValue: context.steps.stepOne.output.doubledValue + 1 }),
  }))
  .commit();

4. RAG (Retrieval-Augmented Generation)

Mastra’s approach to RAG has been super helpful—it grounds LLM responses in your own data, which really boosts accuracy and reliability. With built-in support for chunking, embedding, and retrieval across vector stores, I was able to process documents and run semantic search with just a few lines of code. It’s a great fit for any project that relies on custom knowledge.

import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";
import { PgVector } from "@mastra/pg";
import { MDocument } from "@mastra/rag";

// 1. Initialize and chunk document
const doc = MDocument.fromText("Your document text here...");
const chunks = await doc.chunk({ strategy: "recursive", size: 512, overlap: 50 });

// 2. Generate embeddings
const { embeddings } = await embedMany({
  values: chunks.map((chunk) => chunk.text),
  model: openai.embedding("text-embedding-3-small"),
});

// 3. Store and query in vector DB
const pgVector = new PgVector({ connectionString: process.env.POSTGRES_CONNECTION_STRING });
await pgVector.upsert({ indexName: "embeddings", vectors: embeddings });
const results = await pgVector.query({ indexName: "embeddings", queryVector: embeddings[0], topK: 3 });

console.log("Similar chunks:", results);

5. Built-in AI Evaluation Tools

Mastra provides built-in AI evaluation tools to measure agent quality using metrics like summarization, content similarity, and tone consistency. These tools return normalized scores (0-1) and can be easily added to your agent for automated, model-graded evaluation. Here’s a code snippet to add built-in evals to an agent:

import { Agent } from "@mastra/core/agent";
import { openai } from "@ai-sdk/openai";
import { SummarizationMetric } from "@mastra/evals/llm";
import { ContentSimilarityMetric, ToneConsistencyMetric } from "@mastra/evals/nlp";

const model = openai("gpt-4o");

export const myAgent = new Agent({
  name: "ContentWriter",
  instructions: "You are a content writer that creates accurate summaries",
  model,
  evals: {
    summarization: new SummarizationMetric(model),
    contentSimilarity: new ContentSimilarityMetric(),
    tone: new ToneConsistencyMetric(),
  },
});

6. Developer Experience That Actually Delivers

Mastra is how developer-friendly it is right out of the box. It comes with built-in UI components and kits that make it easy to spin up, test, and iterate on AI agents and workflows quickly.

You get a local playground to experiment in real time, along with REST APIs and type-safe client SDKs that make integration and prototyping feel seamless. Setting up a new project, managing agents, and visualizing workflows takes minimal effort—and that speed really matters when you're trying to move from concept to production without getting stuck in setup hell.

The Trade-Offs You Should Expect

1. Best Fit for TypeScript/Node.js Teams

Mastra is built with TypeScript and Node.js at its core, which makes it a natural choice for teams already working in the JavaScript ecosystem. If you're comfortable with TypeScript, you’ll find the framework intuitive and easy to extend.

That said, it’s not the best fit for teams who primarily work in Python or other language stacks. While you can still interact with Mastra via its APIs, the native developer experience really shines when you’re working directly within a TypeScript codebase. So if your workflow revolves around Python tools like LangChain or Rasa, there might be a steeper integration curve.

2. Slight Learning Curve, But Worth It

When I started with Mastra, the custom concepts like agents, workflows, RAG, and evals took a bit of getting used to, especially coming from a Python-heavy background. But once the core ideas clicked, the structure made everything feel more organized and scalable. There’s a learning curve, but it’s worth it for building reliable, production-grade AI systems.

3. Tied to the Vercel AI SDK: A Double-Edged Sword

In my experience, Mastra’s reliance on the Vercel AI SDK offers a lot of flexibility out of the box, especially for LLM integrations. But it also means you're somewhat tied to the SDK’s abstraction layers and update cycles. It’s great when everything works smoothly, but you’ll need to stay in sync with upstream changes.

4. Still Growing: Early-Stage Ecosystem

Mastra is relatively new, so its ecosystem and community aren’t as mature as more established frameworks like LangChain or Rasa. That means fewer plugins, third-party integrations, and community-driven examples—at least for now. That said, the core team is active, and the framework is evolving quickly.If you’re comfortable working with emerging tech, it’s an exciting space to be part of early.

5. Limited Out-of-the-Box UI

While Mastra provides a helpful local dev interface, it does not offer a production-ready frontend. If you’re building an end-user-facing application, you’ll need to create your own UI or integrate with an existing frontend framework.

Mastra.ai brings a fresh, developer-first approach to building intelligent agents and orchestrating AI workflows. From integrating multiple models to debugging and evaluating outputs, it simplifies the complex without getting in your way.

If you’re exploring Mastra or working on something similar, let’s connect! I’d love to share what I’ve built, hear your thoughts, and swap ideas on pushing the boundaries of AI development together.