SoccerOracle

I watch a lot of football. Before big matches I’d end up digging through Sofascore and FBref, trying to piece together what might happen. All of it was historical, and none of it was a model. So I built one.

The player similarity side came from a separate itch. I’d watch someone play and wonder who plays like them in other leagues, and nothing surfaced that well.

This is V2. V1 was basically a script: no real database, no structure. I showed it to someone, they pointed out how messy it was, and they were right. After my internship I rebuilt it from scratch with a proper database, an async job queue, CI/CD, and a clean line between the API and the ML layer. It’s still a deliberately scoped version. There’s far more data to pull and a richer model to build, which is where this write-up ends.

Match predictions take real work, so they go through a Redis queue and a separate worker process. Player similarity is a lookup over vectors computed ahead of time, so it goes straight from the API to the database. Everything persistent lives in one Postgres instance on Supabase: relational data, player embeddings, and the similarity search itself.

The frontend is React with TanStack Query, talking to a FastAPI backend. When it asks for a match prediction, it doesn’t get a prediction back. It gets a job ID.

The route does three things and stops: it generates a UUID for the job, pushes the job onto a Redis queue, and returns the ID. That takes milliseconds. The client then polls with that ID until the result is ready, so no request ever sits waiting on the model.

A prediction isn’t instant. For every match the system fetches the last five matches for each team from API-Football, averages them, pulls head-to-head history, and assembles 80 features before the model even starts. Done inside the request, the user’s connection would sit open and frozen through all of it.

The bigger problem is scale. Picture El Clásico: thousands of people want the same prediction at the same moment. If every request ran the whole pipeline, the server would repeat identical, expensive work thousands of times over and fall over. None of that work needs repeating. It’s the same match and the same prediction every time.

So the heavy lifting moves off the web server. Redis holds a list that acts as the queue: the API adds jobs to the left end with lpush, and the worker takes them off the right end with brpop. Push left, pop right, so jobs come out in the order they went in.

The worker is its own process that does nothing but wait for jobs. brpop is a blocking call, so while the queue is empty the worker isn’t spinning or burning CPU checking over and over. It’s parked until Redis wakes it the moment a job lands. Then it fetches the data, builds the features, runs the model, stores the result, and goes back to waiting.

The worker’s first real job is to turn a fixture into something a model can read. For each team it takes the last five matches and averages the stats across them to capture current form, then adds what the two sides have done against each other.

That’s 80 features, and they break down as: 19 form stats per team, 4 from the head-to-head record, and 38 from their most recent meeting. Eighty is not a designed number. It’s everything the API gives that could be turned into a number, which is its own kind of limitation.

The model is LightGBM, a gradient-boosted decision tree algorithm. It builds a sequence of small decision trees, each one focused on correcting the mistakes of the trees before it, and the prediction is all of them added together. It’s fast to train and it’s strong on exactly this kind of data: rows and columns of stats.

It predicts 18 match statistics, such as goals, shots, corners, possession, saves and passes, for both teams. Rather than one model predicting all 18 at once, scikit-learn’s MultiOutputRegressor trains 18 independent models, one per stat. If the goals prediction looks off, I can inspect the goals model on its own instead of untangling one giant model doing everything.

Training data comes out of the same five matches the predictor uses. For each matchup the most recent meeting becomes the target, the 18 stats to predict, and the four before it become the inputs. One matchup, one row: 80 numbers in, 18 out. That’s about 280 rows in total, split randomly 80/20 into training and test.

The bar the model has to clear is deliberately dumb: always predicting the average for that stat. If a trained model can’t beat guessing the mean on held-out matches, it hasn’t learned anything, however good the RMSE looks in isolation. New versions are tracked in MLflow and only get promoted if they beat the current model by at least 12% on the tracked validation metrics, so a run that lands slightly ahead by luck doesn’t churn production.

An RMSE of 0.87 sounds high until you remember that goals are rare and partly random. Professional models struggle there too. I treat it as an honesty check rather than a number I expect to get near zero. The winner rate comes from full Premier League and La Liga matchdays, where a blind guess on a three-way outcome gets about a third right.

When the worker finishes, it writes the result to Redis with a one hour TTL, a time to live after which the entry deletes itself. The first person to request El Clásico triggers the model. The next thousand people who request the same match never touch it: they get the finished prediction straight out of Redis.

The expiry also keeps memory under control. Old predictions clear themselves out instead of piling up forever, and an hour is fresh enough for a match preview.

Similarity starts with player stats: up to four seasons, 2022 through 2025, across more than 10,000 player profiles from API-Football.

The hardest problem in the whole project was here. Player names are inconsistent across data sources. Arabic names get transliterated differently, accents come and go, and abbreviations vary by league. I matched names with fuzzywuzzy and a tuned similarity threshold, and that threshold is a real trade-off. Too low, and different players with similar names get merged. Too high, and obvious matches get missed. Some edge cases still need resolving by hand.

To answer “who plays like this player,” every player becomes an embedding: a list of numbers that describes their playing style, where players who play alike end up with similar lists. Once every player is a vector, the question becomes maths about which vectors sit closest together.

Per-90 puts everyone on the same footing, so a player on 800 minutes compares fairly with a starter on 2,500. Season decay says recent form matters most:

Position z-scores turn each stat into how far above or below average the player is for their role, measured in standard deviations. Raw numbers mean different things by position: 60 passes per 90 is ordinary for a midfielder and exceptional for a striker.

Position weights then emphasise what matters for that role. Attackers weight goals at 2.0 and tackles at 0.3, and defenders flip it, with tackles and interceptions at 2.0 and goals at 0.3.

Finally, PCA compresses the 17 weighted stats down to 12 dimensions, with a separate fit for each position group. That vector is stored in a player_embeddings table with pgvector.

Why compress at all? Because the stats overlap. Goals, shots and xG all rise and fall together, so in the raw 17 dimensions “is a shooter” gets counted three times and shooting quietly dominates every comparison. Compression is how one trait goes back to counting once.

It’s also worth being clear about what PCA does, because the name suggests picking. It doesn’t pick 12 of the 17 stats. It builds 17 brand new axes, each one a blend of all 17 stats, ranks them by how much they tell players apart, and then the top 12 are kept and the bottom 5 dropped. Stats that move together collapse onto a shared axis, which is exactly the double counting going away.

A similarity search is a single SQL query against pgvector, ordered by cosine distance. Cosine similarity compares the direction two vectors point in and ignores their length: a similar style means the vectors point the same way.

That matters because of how stats scale. Take a prolific winger and a mid-table winger with the same profile shape: lots of dribbles, high xG, few tackles. Their vectors point the same way, but one reaches much further out.

As a sanity check I ran nearest-neighbour queries on Ballon d’Or shortlists and on players I watch every week. The results passed the eye test: stylistically similar players clustered together, not just stars with similar minutes.

This was my biggest learning leap. It’s where I first connected model serving, vector databases and backend API design as one system rather than separate concepts: data ingestion, feature engineering, embedding generation and prediction serving working together, with caching deciding whether response times stayed reasonable.

I’ve built several projects since, but SoccerOracle is where I went from following tutorials to architecting something end to end on my own.

Player-level data in the predictions. Right now the model only sees team form and has no idea who’s actually playing. If a key player is injured or the manager rotates the squad, it can’t know. That’s the most impactful thing to add, and it’s the heart of a V3: lineups and injuries feeding the match model.

Scheduled scraping instead of per-request API calls. Every prediction currently fetches recent matches from the external API. That’s slow, it costs API credits, and at real scale the bill grows fast. I’d scrape FBref or Sofascore on a schedule, daily or after matchdays, and store it in Supabase, so a prediction becomes a database query. The API would only be a fallback for data I don’t have yet.

Richer data and a bigger model. More matches and leagues to choose from, and more stats per team as inputs, which goes hand in hand with a model that can use them.