What Is a Large Language Model? How Chatbots Actually Work

A large language model (LLM) is the technology behind modern AI chatbots — it is, at heart, a very large version of the prediction idea from the machine learning article in this series, applied to text. Given some text, it predicts what word is most likely to come next. Repeated over and over, one word at a time, that simple prediction is enough to write essays, hold conversations, and answer questions.

1.The Core Idea: Predicting the Next Word

Underneath a chatbot's reply is a model trained on an enormous amount of text — books, articles, websites, and more — learning to predict, for any given piece of text, what word (or part of a word) is statistically most likely to follow. That's the entire task the model is trained on. It never explicitly "knows" facts the way a person does; it has learned extremely detailed patterns about which words tend to follow which other words, across a staggering range of topics and contexts.

2.Tokens: How Text Becomes Numbers a Model Can Use

Models don't work with letters and words directly — they work with numbers. Text is first broken into pieces called tokens, which might be a whole word, part of a word, or a punctuation mark, and each token is converted into a number the model can process mathematically.

Notice that "rewarding" was split into two tokens. Breaking uncommon or longer words into smaller familiar pieces lets a model handle new or rare words it hasn't seen as a whole, by combining tokens it already recognises.

3.Generating a Reply, One Token at a Time

When a chatbot writes a response, it isn't planning the whole sentence in advance. It predicts one token at a time: given everything written so far, it calculates a probability for every token in its vocabulary, picks one (usually the most likely, sometimes with a bit of randomness for variety), adds it to the text, and repeats the entire process again for the next token.

This process, repeated hundreds of times for a long reply, is why longer responses take visibly more time to generate than short ones — the model is genuinely doing the same prediction step over and over, token by token.

4.Attention: How a Model Tracks Context

Predicting the next word well requires understanding context — in the sentence "The trophy didn't fit in the suitcase because it was too big," knowing whether "it" refers to the trophy or the suitcase requires weighing the whole sentence, not just the words immediately before "it." LLMs use a mechanism called attention, part of an architecture called the Transformer, which lets the model weigh the relevance of every earlier token against every other token when producing its prediction — effectively letting it "focus" more on the words that matter most for the current decision, regardless of how far back they appeared.

This is the same layered, weighted structure introduced with neural networks in the previous article, adapted specifically to handle the order and relationships within language rather than fixed lists of numbers.

5.Training in Two Stages: Pretraining and Fine-Tuning

Building a modern chatbot generally happens in two broad stages:

• Pretraining — the model is trained on a vast amount of general text, simply learning to predict the next token extremely well. This stage produces a model that is fluent but not yet well-behaved as an assistant.
• Fine-tuning — the pretrained model is further trained on more targeted examples of good, helpful responses, often refined using human feedback (a process commonly called RLHF, Reinforcement Learning from Human Feedback) to make it more useful, honest, and appropriate to talk to.

Pretraining is what gives a model its broad knowledge and fluency; fine-tuning is what shapes it into something that behaves like a helpful assistant rather than simply continuing text in any direction.

6.What LLMs Can — and Can't — Do Well

Understanding an LLM's real limitations is essential to using one responsibly.

• Hallucination: because a model is predicting plausible-sounding text rather than looking up verified facts, it can state incorrect information confidently and fluently — always worth verifying anything important independently.
• Knowledge cutoff: a model's core knowledge comes from the text it was trained on up to a certain date, so it won't natively know about events after that point unless it's specifically given that information.
• No true understanding: an LLM doesn't experience or comprehend meaning the way a person does — it's recognising and continuing extremely sophisticated statistical patterns, which is why it can occasionally fail in ways a person with real understanding wouldn't.
• Inherited bias: like any model trained on real-world data (as covered in the previous article), an LLM can reflect biases present in its training text.

7.Getting Better Results: The Basics of Prompting

• Be specific. Clear, detailed prompts give the model more relevant context to predict from, generally producing more useful replies than vague ones.
• State the format you want. Asking for a list, a table, or a specific length shapes the pattern the model continues.
• Ask it to check its own work for anything factual and important, or verify independently — remembering the hallucination risk above.
• Break big tasks into steps, the same way breaking a problem into a clear algorithm (from the earlier article) produces better code.

A Closing Thought

A large language model can feel like it's "thinking," but underneath every reply is the same mechanical loop: turn text into tokens, predict the most likely next token using patterns learned from enormous amounts of training text, add it, and repeat. It's a remarkable amount of capability to emerge from one repeated step — and understanding that step is what lets you use these tools well, instead of trusting them blindly.