Billions of pages exist online, yet a search takes a fraction of a second. This guide explains what a browser does, how a search engine crawls, indexes and ranks pages, and how to search, judge and protect yourself online, with diagrams and worked examples.
You type a few words, press search, and within a second a ranked list of pages appears from a web with billions of pages. It feels like magic, but it is the result of careful preparation done long before you typed anything. Two different tools are involved, and people mix them up all the time: the web browser, which displays pages, and the search engine, which helps you find them. This article explains what each one does, follows a page from the moment you click until it appears on screen, and then opens up the search engine to show how it crawls, indexes and ranks pages. You will also learn how to search better, judge what you find, and protect your privacy.
Helpful background: What Is the Internet, Really? explains how data travels as packets, which this article builds on. You can read this one on its own, too.
Think of a huge library. The browser is like the door, the reading desk and your eyes: it is how you enter and read whatever you choose. The search engine is like the library's catalogue and its helpful librarian: it tells you which shelves are worth visiting. You need the first to read anything, but you do not need the second if you already know exactly where to go.
| Web browser | Search engine | |
|---|---|---|
| What it is | A program or app installed on your device | A website (and service) run by a company |
| Main job | Fetch web pages and display them | Find and rank pages that match your words |
| Where it runs | On your phone or computer | On the company's servers in data centres |
| Examples | Chrome, Safari, Firefox, Edge | Google, Bing, DuckDuckGo |
When you type words into a browser's address bar, the browser does not search anything itself. It sends your words to the search engine that is set as its default, and that search engine does the finding. You can change the default in your browser settings, and you can use any browser with any search engine.
Opening a web page involves a surprising number of steps, and a good browser does all of them in a fraction of a second. First, learn to read the address you type. It is called a URL (Uniform Resource Locator). Here is an example address, using example.org, a name reserved for examples:
https://www.example.org/articles/cloud.html?lang=en#intro
After you press Enter, the browser works through these steps:
One common belief needs correcting. A padlock (or similar icon) beside the address means the connection between you and the site is encrypted. It does not mean the site is honest or safe. Criminals can also use HTTPS on fake sites, so you must still check the address carefully and be wary of unexpected requests for passwords or money.
A search engine cannot scan the entire web every time someone types a query. That would take far too long. Instead, its work is split into stages. Google's own guide to its search system describes three stages, and other search engines work in broadly similar ways: crawling, indexing and serving results (which includes ranking). The first two are done in advance. Only the last happens while you wait.
Crawlers (also called spiders, robots or bots) are automated programs that visit a web page, read it, collect every link on it, and then visit those linked pages in turn. Repeating this over and over lets them discover enormous numbers of pages. A site owner can also hand over a list of pages, called a sitemap, to help crawlers find everything.
Crawlers cannot see everything. Some pages are not reachable because nothing links to them. Others need a login, such as your email inbox, so crawlers cannot open them. Site owners can also place a text file called robots.txt at the top of their site to ask crawlers not to visit certain areas. It is a polite request that well-behaved crawlers follow. It is not a lock, so it should never be used to protect private information.
After a page is crawled, the search engine works out what it is about and stores a summary in a giant database called the index. Not every page that is crawled gets indexed, because pages may be duplicates, have very little content, or be marked by their owners as "do not index".
The index works like the index at the back of a textbook, except that it is built automatically. Rather than listing pages and what is on them, it lists words and which pages contain them. This is called an inverted index, and it is the reason a search takes a fraction of a second: the engine looks up your words in the index instead of reading pages.
Imagine a search engine has crawled only three pages:
Step 1: Tidy the words. Engines commonly change everything to lower case and ignore very common words such as "is", "and" and "a" (called stop words).
Step 2: Record where each word appears.
| Word | Pages |
|---|---|
| solar | 1, 2 |
| energy | 1, 2 |
| clean | 1 |
| wind | 2 |
| panels | 2 |
| coal | 3 |
| fossil | 3 |
| fuel | 3 |
Step 3: Answer a search. For "solar energy", look up both words. "solar" is on pages {1, 2} and "energy" is on pages {1, 2}. The pages that have both are {1, 2}. Page 3 is never even touched.
Step 4: Order them. Both pages match, so ranking must decide. Page 1 uses "energy" twice in a very short sentence, which hints that it is closely about the topic, so a simple engine might put it first.
Limits of this model: a real engine also understands that "photovoltaic" is related to "solar" and that "run" and "running" are forms of one word. Modern engines try to match meaning, not just exact words.
For common searches, thousands or millions of pages may match. Ranking decides which come first. Search engines use many signals together and change them regularly, and they do not publish exact formulas, partly to make cheating harder. Google states that it does not accept payment to rank a site higher in its ordinary (organic) results. Commonly discussed signals include:
The idea that links act as votes is the heart of PageRank, a method described by Sergey Brin and Lawrence Page at Stanford University in 1998. Their paper explains that using the link structure of the web helps calculate a quality ranking for each page. Modern ranking uses far more than links, but the central idea is still useful for understanding why well-linked pages tend to rise.
Four pages exist: A, B, C and D. The links are A→C, B→C, D→C and C→A (an arrow means "this page links to that page").
Step 1: Count incoming links. C has 3. A has 1. B and D have 0.
Step 2: Give each page the same starting score. With 4 pages, each starts at 1 ÷ 4 = 0.25, so the total is 1.
Step 3: Each page shares its score equally among the pages it links to. A, B and D each link to one page (C), so each hands over all of its 0.25. C also links to just one page (A), so it hands over all of its 0.25.
Step 4: Add up what each page receives.
Check: 0.75 + 0.25 + 0 + 0 = 1, so no score was lost.
Reality check: the real method repeats this sharing many times and also includes a "random jump" factor so scores settle fairly. It is one signal among many, and it can be fooled, which is why search engines fight link schemes and spam. This example is deliberately simplified for learning.
A results page is more than ten blue links. You may see:
Two people who type the same words may see different results. A search for "best pizza" shows local places that depend on where you are. Language, device, time and, in some cases, your past searches and account settings can also change the order. Researchers disagree about how strongly personalisation traps people in an information "bubble", so it is wise not to assume it either does or does not matter. The practical advice is the same: if a topic is important, look at more than one source and, when you can, try a different search engine or a private window.
Good searching is a skill. These tips work in most major search engines, although the exact symbols can differ slightly:
"light-independent reactions".jaguar -car.site:.edu or site:nasa.gov.Goal: find reliable information about the animal called the jaguar for a school report.
| Try | Search | What happens and why |
|---|---|---|
| 1 | jaguar | Too broad. The word names an animal, a car brand and more, so results are mixed. |
| 2 | jaguar animal habitat | Better. The extra words show which meaning you want. |
| 3 | jaguar -car habitat diet | The minus sign removes car pages, and "diet" narrows the topic. |
| 4 | jaguar habitat site:.edu | Limits results to educational sites, which are often (but not always) more careful sources. |
Last step: do not stop at the first result. Open two or three pages and see if they agree.
A high ranking means a page matched many signals. It does not prove the page is true. Anyone can publish on the web, so you need to check. Ask:
A powerful habit is lateral reading: instead of staying on the unfamiliar page, open new tabs and search for what other trusted sources say about the page and its author. Fact-checkers use this approach because it quickly shows whether a site can be trusted.
Using the web leaves traces. A search engine may record your searches, and websites can see your IP address and details such as your device and browser type. Cookies and similar tools can be used to follow your activity from site to site. Many services use this data to personalise results and show adverts, and the rules differ between companies and countries.
A private or incognito window is often misunderstood. It mainly stops your own device from saving your history, cookies and form entries after you close the window. It does not make you invisible. The websites you visit, your school, workplace or internet provider can still see activity. It is useful on a shared computer, but it is not a shield.
Sensible habits: review privacy and cookie settings, keep your browser updated, be careful what personal details you type into forms, and sign out on shared devices. Read more in Digital Footprint & Online Privacy and Cybersecurity Basics. If you are curious how services store your data, see Cloud Computing.
Search engine optimisation (SEO) means improving a website so that search engines can find and understand it, and so that it ranks well for the right searches. Honest SEO includes writing clear, original and accurate content, using descriptive titles, making the site fast and mobile-friendly, and earning real links. Dishonest tricks, such as stuffing a page with repeated keywords or buying links, go against search engine guidelines and can cause a site to be ranked lower or removed. In the long run, being genuinely useful is the best strategy.
Try each question first, then tap it to reveal the answer. A scored multiple-choice quiz follows below.
A browser is an app that fetches and displays web pages, while a search engine is a website that finds and ranks pages matching your words.
Because crawling and indexing were done in advance. A search only looks up your words in the prepared index instead of scanning the web.
"wind" is on page 2 and "solar" is on pages 1 and 2. Only page 2 has both, so page 2 matches.
C has three incoming links, each passing it a score of 0.25, so it receives 0.75. A has only one incoming link, from C, so it receives 0.25.
It tells you the connection to the site is encrypted. It does not tell you that the site itself is honest or safe.
Any two of: the websites you visit can still see your visit; your school or workplace network can still see activity; your internet provider can still see activity.
"Be the search engine" (no devices needed): give groups three or four short paper "web pages" containing a few sentences. Ask each group to build an index on a sheet of paper (word → page numbers), then answer search questions from other groups using only the index. Finish by asking how they would rank pages that all match.
Search challenge: give students a question with an ambiguous word and ask them to improve their search wording in at least three steps, noting what changed each time.
Lateral reading practice: choose an unfamiliar website and have students find what two independent sources say about it before deciding whether to trust it.
Safety note: remind students not to type personal details such as full names, addresses or school information into search boxes or forms, and to ask a trusted adult if something online makes them uncomfortable.
This article is original EDUSAMBAM writing. The description of crawling, indexing and serving follows the public documentation below, and the PageRank idea follows the original research paper. The tiny index and link examples are invented for practice, and the diagrams are original EDUSAMBAM schematics. Other ideas (URLs, DNS, HTTPS, cookies, browser engines, private browsing) are widely taught web fundamentals explained here in our own words.
Search tools and their features change often, so check each service's current settings and policies yourself. This article does not recommend one search engine or browser over another.
10 questions. Select an answer for each, then submit to see your score instantly.