If you have ever typed a half-remembered phrase into a site’s search box and got back exactly what you were thinking of, there is a decent chance Elasticsearch was doing the work behind the scenes. It powers search at Wikipedia, GitHub, Netflix and a long list of smaller sites that simply got tired of slow, dumb database queries. This article walks through what Elasticsearch actually is, how it stores and finds things, and which of its functions matter most in day-to-day use.
So what is it, exactly?
Elasticsearch is a distributed search and analytics engine built on top of Apache Lucene. Lucene has been around since 1999 and is a very good, very low-level library for full-text indexing. The trouble is that using Lucene directly means writing a lot of Java and thinking hard about things like sharding and replication yourself. Elasticsearch wraps all of that in an HTTP API that speaks JSON, handles clustering for you, and scales horizontally by adding more machines.
Two things make it different from the database you are probably already running:
- It is built for text first. Relevance scoring, stemming, synonyms, typo tolerance and phrase matching are core features, not bolted-on extras.
- It is near real-time. A document you index now is searchable about a second later, across however many nodes you have.
A quick note on licensing, because people ask. In 2021 Elastic moved away from Apache 2.0, which prompted AWS to fork the project as OpenSearch. Since 2024 Elasticsearch has also been available under the AGPL. For practical purposes the two projects are still very similar, and most of what follows applies to both.
How data is organised
The vocabulary trips people up at first, so here is the short version. A document is a single JSON object, roughly the equivalent of a row. Documents live in an index, which is the equivalent of a table. Each index is split into shards, and each shard can have replicas on other nodes. Shards are what let an index grow beyond a single machine, and replicas are what keep the cluster answering queries when a node falls over.
Indexing a document is a single HTTP call:
PUT /articles/_doc/1
{
"title": "Why our coffee machine keeps breaking",
"body": "After the third repair in a month we finally called the manufacturer...",
"author": "Rita",
"tags": ["office", "coffee", "hardware"],
"published_at": "2026-10-12T09:30:00Z"
}You did not have to define a schema before doing that. Elasticsearch looks at the values, guesses the field types, and creates a mapping on the fly. Dynamic mapping is handy for prototyping, but in production you almost always want to define the mapping yourself, because the guesses are not always what you want. A string, for instance, gets mapped as both text (analysed, for full-text search) and keyword (stored as-is, for exact matching and aggregations). That doubles the storage for every string field whether you need it or not.
PUT /articles
{
"mappings": {
"properties": {
"title": { "type": "text", "analyzer": "english" },
"body": { "type": "text", "analyzer": "english" },
"author": { "type": "keyword" },
"tags": { "type": "keyword" },
"published_at": { "type": "date" }
}
}
}The inverted index, or why it is fast
A relational database answering WHERE body LIKE '%coffee%' has to read every row and scan every body. Elasticsearch does the opposite. When a document comes in, the text fields are run through an analyzer, which chops the text into tokens, lowercases them, strips stop words, and usually reduces them to a stem (“breaking” becomes “break”). Each token is then stored in an inverted index: a sorted list of terms, each pointing to the documents that contain it.
Searching for “coffee” is now a dictionary lookup rather than a table scan. The same analysis is applied to the query, so a search for “broken machines” will match a document that says “machine keeps breaking”. You can see exactly what an analyzer does to a string with the analyze API, which is one of the more useful debugging tools when results look odd:
POST /articles/_analyze
{
"analyzer": "english",
"text": "The machines were breaking constantly"
}
// tokens: "machin", "break", "constantli"Querying
Queries are written in a JSON structure called Query DSL. It looks verbose compared to SQL, but it composes well, and once you have built a few queries the shape becomes second nature. The most common building block is match, which runs a full-text search against one field. multi_match extends that to several fields and lets you boost some over others. Here, a hit in the title counts three times as much as a hit in the body:
GET /articles/_search
{
"query": {
"multi_match": {
"query": "coffee machine repair",
"fields": ["title^3", "body"],
"fuzziness": "AUTO"
}
}
}The fuzziness setting is what gives you typo tolerance. “Coffe machne” will still find the document, at a slightly lower score. Every hit comes back with a _score, calculated by default with BM25, which rewards terms that are rare across the index and appear often in a given document, while penalising very long documents so they do not win just by containing more words.
Real queries usually combine several conditions, and that is what bool is for. It has four clauses: must (required, affects score), should (optional, boosts score), filter (required, does not affect score) and must_not. The distinction between must and filter matters more than it looks. Filters are cached and skip scoring entirely, so anything that is a yes/no condition, like a date range or a category, belongs in filter.
GET /articles/_search
{
"query": {
"bool": {
"must": [
{ "match": { "body": "repair" } }
],
"filter": [
{ "term": { "tags": "hardware" } },
{ "range": { "published_at": { "gte": "now-1y" } } }
],
"must_not": [
{ "term": { "author": "spam-bot" } }
]
}
}
}Aggregations
This is the part of Elasticsearch that often gets overlooked by people who only think of it as “search”. Aggregations are the equivalent of GROUP BY, but they run in the same request as the query and against the same filtered result set. That is how faceted navigation works on shopping sites: the counts next to each brand or size are a terms aggregation on whatever matched the current search.
GET /articles/_search
{
"size": 0,
"query": { "match": { "body": "repair" } },
"aggs": {
"by_tag": {
"terms": { "field": "tags", "size": 10 }
},
"per_month": {
"date_histogram": {
"field": "published_at",
"calendar_interval": "month"
}
}
}
}Setting size to zero tells Elasticsearch you do not want the documents themselves, only the buckets, which makes this kind of request very cheap. There are bucket aggregations (terms, ranges, histograms, geo grids), metric aggregations (avg, sum, percentiles, cardinality) and pipeline aggregations that operate on the output of other aggregations. Nest them and you can answer questions like “average order value per country per week” in one round trip.
Other functions worth knowing about
Suggesters. The completion suggester is built for search-as-you-type and is backed by a separate in-memory structure, so it answers in a few milliseconds. The term and phrase suggesters are what sit behind “did you mean” features: give them a misspelled query and they return the closest terms that actually exist in the index.
Highlighting. Ask for it and each hit comes back with fragments of the matching field, with the matched terms wrapped in tags of your choosing. Saves you from doing it badly with regular expressions on the client.
Synonyms and custom analyzers. You can define your own analysis chain per field: a tokenizer followed by any number of token filters. A synonym filter that maps “laptop” to “notebook”, an ASCII folding filter so “café” matches “cafe”, an n-gram filter for partial matching. Analysis is where most of the quality of a search experience is decided, and it is worth spending time on.
Geo queries. Index a geo_point and you can filter by distance, bounding box or polygon, and sort by proximity. “Shops within 5km, nearest first” is a few lines.
Vector search. More recent versions support dense_vector fields and approximate nearest-neighbour search, which is how semantic search on embeddings is done. You can mix it with traditional keyword scoring in the same query, which in practice tends to beat either approach on its own.
Bulk API and ingest pipelines. Indexing one document per request is slow. The bulk endpoint takes thousands at once. Ingest pipelines let you transform documents on the way in, extracting fields, parsing dates, enriching from another index, without touching your application code.
Where it fits, and where it does not
Elasticsearch is excellent as a secondary store that sits alongside your database of record. You keep writing to MySQL or Postgres, push a copy of each document to Elasticsearch, and route search and analytics traffic to it. The gains are big: queries that took seconds in SQL come back in tens of milliseconds, and the results are actually ranked by relevance rather than by whatever order the rows happened to be in.
It is a poor choice as a primary database. There are no transactions, writes are eventually consistent across replicas, and schema changes to existing fields generally mean reindexing everything. It also wants a fair amount of RAM; running it on a tiny VPS next to your web server tends to end in tears. Managed offerings from Elastic, AWS and others take most of the operational pain away if you would rather not babysit a cluster.
If you take one thing from this article, let it be this: the magic is not in Elasticsearch itself but in how you analyse your text and shape your queries. Get the mapping right, put conditions in filters, boost the fields that matter, and you will have a search that feels like it understands what people meant to type. Get them wrong and you have a very fast way of returning the wrong thing.