At Fiatbitcoin a worker collects metrics from our own Bitcoin node every ten minutes: mempool size, number of waiting transactions, fee rates, hashrate, difficulty. Over time that builds a decent time series, one row per measurement in a table. At some point the obvious question comes up: when is one of these values actually unusual? When is a note or a push worth it, and when is it simply Tuesday evening?
The cheap answer is a fixed threshold. Mempool above 200,000 transactions, alarm. The problem with that shows after a week: the mempool breathes in a daily and weekly rhythm. Sunday evening is reliably fuller than Tuesday morning, fees pick up at certain times. A flat threshold flags every predictable peak as an anomaly and stays silent on an untypical spike that happens to stay under the line. What you actually want is a band that knows the normal rhythm, and an alarm only when a value breaks out of that band.
First the simple way
The honest entry point into something like this is not a foundation model but a rolling z-score: mean and standard deviation over a window, and anything beyond two or three sigma counts as unusual. That is a few lines of code and perfectly sufficient for many series.
It fails precisely on the seasonality. Mean and spread over a flat window do not know the daily rhythm. The Sunday peak lifts the mean, inflates the spread, and in the end the band is either so wide that nothing stands out anymore or it flags half the week. You can repair this with seasonal decomposition, STL and relatives, but then piece by piece you are rebuilding a forecasting model anyway. At that point I wanted to know how far a ready-made model carries, one that brings seasonality along out of the box.
What TimesFM is
TimesFM is a foundation model for time series published by Google. Decoder-only, built much like a language model, but trained on sequences of numbers instead of text. The point for my case: it works zero-shot. You feed in a historical series and get a forecast for the next steps back, without training the model on your own data. And it returns not just a point value but quantiles, so a band with uncertainty. That band is the actual gain for anomaly detection.
The catch: TimesFM is a PyTorch model. Elixir has no library for it. The reflex would be a separate Python service with an HTTP interface, a second deployment artifact, a network hop per call. That is what I wanted to avoid.
Python in the BEAM with Pythonx
pythonx embeds a Python interpreter directly into the running BEAM VM. No separate process, no HTTP, no serialization layer over the wire. The Python environment is set up via uv, the model is kept as a global variable in the interpreter.
Because the Python GIL enforces serial execution anyway, all model access runs through a single GenServer. Loading is lazy, on the first call, so the app boot does not wait on several gigabytes of model checkpoint. The core is the Python snippet that fetches the forecast:
arr = np.asarray(series, dtype=np.float32)
point_forecast, quantile_forecast = model.forecast(horizon=int(horizon), inputs=[arr])
{
"point": [float(v) for v in point_forecast[0]],
"quantiles": [[float(q) for q in step] for step in quantile_forecast[0]],
}
On the Elixir side that is a GenServer.call which trims the context to the model’s maximum size and decodes the result:
def handle_call({:forecast, series, horizon}, _from, state) do
state = ensure_python(state)
case state.globals do
nil ->
{:reply, {:error, "Modell konnte nicht geladen werden."}, state}
globals ->
context = Enum.take(series, -@max_context)
h = horizon |> min(@max_horizon) |> max(1)
{result, _} =
Pythonx.eval(@forecast_code, Map.merge(globals, %{"series" => context, "horizon" => h}))
{:reply, {:ok, Pythonx.decode(result)}, state}
end
end
That was the whole bridge. A GenServer, a lazily loaded model, a function that takes a list of floats and gets back a point forecast plus a quantile band.
Do not trust the model, measure it
The quantile output is a list of columns per forecast step. Which column is which quantile I could have taken from the docs. For something a later alarm logic builds on, I wanted to see it. So I fed in a synthetic sine series and looked at the output for one step:
num quantile columns: 10
quantile vector: [97.49, 97.00, 97.20, 97.23, 97.23, 97.27, 97.30, 97.31, 97.37, 97.52]
point: 97.27
Ten columns. Column 0 falls out of line (97.49 is higher than its neighbours), columns 1 through 9 rise monotonically. That is the telling structure: column 0 is the mean, columns 1 through 9 are the quantiles 0.1 to 0.9. And the point value 97.27 is exactly column 5, the median. That nailed the band down without guessing: lower bound column 1 (q0.1), upper bound column 9 (q0.9). That note now sits right in the module doc so it does not get lost.
Holdout-1: the actual anomaly logic
The idea is simple. You take the series, hold back the last observed value, let the model predict a band for exactly that point from the history before it, and compare. If the actual value lies outside the q0.1 to q0.9 band, it is unusual. The nice part is that the band is seasonal. The expected Sunday peak lies inside the band, an untypical swing mid-week does not.
In code that is less than twenty lines:
def check(series) when length(series) < @min_points, do: :insufficient
def check(series) do
{context, [actual]} = Enum.split(series, -1)
case Forecaster.forecast_quantiles(context, 1) do
{:ok, %{"quantiles" => [step | _]}} ->
low = Enum.at(step, @q_low)
high = Enum.at(step, @q_high)
cond do
actual > high -> {:anomaly, %{actual: actual, low: low, high: high, direction: :above}}
actual < low -> {:anomaly, %{actual: actual, low: low, high: high, direction: :below}}
true -> :normal
end
{:error, _reason} ->
:insufficient
end
end
Enum.split(series, -1) separates the last point from the rest. The rest goes into the model as context, the band comes back, the comparison is a cond. I verified this against the real model: a seasonal base series with a matching endpoint yields :normal, the same series with a spike at the end yields :anomaly with direction: :above, a series that is too short yields :insufficient.
From a hit to a visible note
The rest is wiring with what already stands at Fiatbitcoin. An hourly worker calls the agent, which walks the three structure-rich metrics (transaction count, mempool size, fastest fee) and writes each hit as a note into the internal knowledge base, the same one the morning briefing and the notifications draw from. On the agent dashboard the detector gets its own widget with a run button, and on the insights page a section appears that shows the anomalies of the last 24 hours, with an arrow up or down and the expected band in plain text.
Same model, a second use: the fee corridor
Once the forecaster stood as a GenServer, the second use was almost free. The anomaly detector forecasts one step ahead and compares. For the fees I wanted several steps, the expected band of the fastest fee rate over the next hours. forecast_quantiles takes the horizon as an argument anyway, and the quantile band was already there. The only new thing was turning the per-step quantiles into a corridor.
corridor =
steps
|> Enum.with_index(1)
|> Enum.map(fn {step, i} ->
%{
minutes: i * @step_minutes,
low: Float.round(Enum.at(step, @q_low), 1),
median: Float.round(Enum.at(step, @q_median), 1),
high: Float.round(Enum.at(step, @q_high), 1)
}
end)
There is a deliberate boundary here too. I forecast the fees, not the price. The Bitcoin price is close to a random walk, a point forecast on it would be theatre and would invite users to bet on it. The fee rate, on the other hand, carries daily and weekly structure, because the mempool fills during active hours. That is exactly where a seasonal band means something, and the corridor answers a concrete question: is it worth waiting a few more hours before sending a transaction.
A band alone does not answer the question though, it only shows it. So I stretched the horizon to twenty-four hours and turned the corridor into a verdict: take the cheapest window by median, and recommend waiting only when it is both at least 2 sat/vB and at least 20 percent cheaper than sending now. The double threshold keeps the recommendation from nagging about half a saved sat that nobody acts on anyway.
def recommend(corridor, current) do
cheapest = Enum.min_by(corridor, & &1.median)
savings = current - cheapest.median
rel = if current > 0, do: savings / current, else: 0.0
wait? =
savings >= @min_abs_saving and rel >= @min_rel_saving and cheapest.minutes > @step_minutes
# ... verdict as a map
end
On the node dashboard the corridor sits as a shaded band next to the current fee estimates, the q10 to q90 band as an area, the median as a line over twenty-four hours, and above it the verdict in plain text: “Cheapest window in about 6 h at around 4 instead of 19 sat/vB right now.” When waiting is advised, a vertical line marks the cheap window directly in the chart.
The nice part about this split is testability. recommend is a pure function, corridor in, verdict out, no model call. That made it the first piece of this whole thing I could cover with an ordinary unit test: a few hand-built corridors, the expected verdict next to them. The model-touching code around it still needs the real model, more on that below. The same forecaster bridge, built once, now carries two features.
Lessons Learned
Three things that worked, and one where I have to be honest.
1. Pythonx makes the second runtime cheap.
I was set on a separate service with HTTP and got an embedded interpreter instead. A GenServer that simply accepts the GIL’s serial constraint, a lazily loaded model, done. For a model that forecasts a few series per hour, that is exactly the right amount of apparatus. A separate service would have been more operations for no gain.
2. Verify the model empirically, do not trust the docs.
Measuring the column order of the quantiles via a synthetic series cost five minutes and ruled out a class of silent errors. Had I guessed the indices and swapped them, the band would have been upside down and the detector would have produced plausible-looking nonsense without ever crashing. For anything a decision logic builds on, the look at the real output is worth it.
3. Holdout-1 is simple and exactly right.
The temptation with a foundation model is to think complicated. The actual logic is an Enum.split and a cond. Hold back the last point, predict the band for it, compare. It needs no more than that, and because it is so little, you can read it and trust it.
4. What I would sharpen: the operational edges and the tests.
This is where the compromises sit. The model lives as several gigabytes in the same BEAM that also serves node, jobs and LiveView. The very first call loads the checkpoint and blocks a job worker for minutes while doing so. For a single-node setup that is acceptable, in a larger arrangement I would isolate the model. For forecasting I deliberately stay with the structure-rich network metrics and leave the Bitcoin price out, because it is close to a random walk and a point forecast on it would be eyewash. At most the uncertainty band would have value there. And honest about testing: I have since split out the pure decision logic and cover it with ordinary unit tests, for the fee verdict for instance, corridor in, verdict out. The model-touching functions still hang on the live model though, I verify them against the real model instead of with a test. The clean way would be to put the forecaster too behind a swappable interface, the way the existing embedding detector already does, and test the check function deterministically with that. It is on the list.
If a particular spot interests you more, the GenServer’s lazy loading mechanism, how the anomaly note links to other agents in the knowledge base, or why I landed on the 24-hour window on the insights page, write it in the comments. I am happy to go deeper.
Kommentare