Building an AI Agent from Scratch
An upcoming series in The Computist Journal.
Hey! I've been radio silent for the last couple months or so, dealing with lots of work, classes, family stuff, and life. But also, thinking a lot about AI and LLMs.
One of the things that's kept the most busy is designing and supervising around five LLM-based projects or apps of different scopes and scales, dealing with lots of different ways to gather data, interconnect systems, and make LLMs use them and reason about them.
I've also been teaching a lot of AI courses lately, from introductory lessons for non-experts to graduate courses on advanced LLM theory and applications.
All of this has got me thinking about the most straightforward way to teach you how to leverage the power of language models to build powerful applications.
And this is what I've come up with.
How to Train your Chatbot - Refactored
In recent years, large language models (LLMs) have revolutionized how we interact with technology. From generating human-like text to answering complex questions, these models are reshaping the landscape of AI applications. But how do we go beyond simple chatbots and build intelligent agents capable of deep research, document understanding, and web-based knowledge retrieval?
This is the question that motivates this new series: How to Train your Chatbot.
LLMs are powerful, but raw language generation alone doesn’t solve every problem. To build truly useful AI applications, we need to combine language models with structured reasoning, external knowledge sources, and multi-step workflows. This is where agent-based systems shine. They orchestrate multiple components, tools, and data sources to perform complex tasks autonomously or semi-autonomously.
Imagine an AI assistant that not only chats but can:
Search the web for up-to-date information
Parse and analyze large document collections
Answer nuanced questions by synthesizing multiple sources
Perform deep research tasks with iterative refinement
Work quietly in the background to solve complex problems, generating code snippets, creating graphs, and automating workflows without interrupting the user experience.
This is what I want to teach you how to build.
What to expect from this series
Over the course of about 10 articles, we will embark on a hands-on journey from the ground up:
Starting Simple: We’ll begin with a minimalist chatbot built using Streamlit a fantastic Python framework for creating interactive apps quickly.
Adding Intelligence: Next, we’ll integrate LLMs to generate meaningful, context-aware responses.
Asynchronous Communication: We'll code a robust, production ready asynchronous communication pipeline using redis that can support multiple clients and conversations.
Agentic Architectures: We’ll evolve our app into an agent-based system, capable of calling external APIs, managing state, chaining tasks, and working asynchronously in the background.
Document Handling: We'll learn how to ingest, index, and query large document sets effectively, and store them in a production ready vector DB.
Web Search: We’ll add live web search to keep our agent informed with real-time knowledge.
Question Answering & Reasoning: We'll build a sophisticated QA pipeline that combines language understanding with retrieval, capable of answering multi hop questions.
Coding: We'll add coding capabilities to our agent to enable the solution of structured problems that require sophisticated reasoning.
Deep Research: Finally, we’ll explore how to create multi-step research agents that plan, gather, synthesize information, generate code, and produce visualizations autonomously.
Each lesson builds on the previous one, ensuring you understand the concepts and code at every step. By the end, you’ll have a robust, extensible agent-based app that you can adapt to your own projects. All built with open source, state of the art technology and using the most robust prompt techniques currently available.
Who is this for?
This series is designed for developers, researchers, and enthusiasts who:
Have some familiarity with Python;
Want to go beyond “black-box” LLM usage and build structured AI workflows;
Are curious about how to combine language models with external tools and data; and
Appreciate clear, practical tutorials with code and explanations.
That's it. No previous experience with LLMs or AI needed.
Let’s get started
In the next article, we’ll dive right into building our first simple chatbot app with Streamlit. No complicated setup, just pure hands-on coding to get you familiar with the basics.
If this looks exciting for you, hit that subscribe button to receive all upcoming articles in your inbox.
See you soon!