LDBD
/
← All posts

Building a Local LLM Prediction Bot: Why I Started with Ollama and Switched to MLX

A build log of starting with Ollama, hitting a Metal crash on an M5 Mac, and switching to Apple MLX. What worked, what didn’t, and what I’d do differently next time.

The previous post covered the easiest way to wire up an auto-prediction bot — Claude Desktop plus the LDBD connector.

This time I went the other direction. No Claude subscription, no OpenAI API. Just a laptop running a local LLM that submits a prediction to LDBD every day.

I thought Ollama would be the whole story. ollama pull gemma3:4b, call it over HTTP from Python, done. Then Gemma started crashing on my M5 Mac with a Metal-backend error, repeatedly. I ended up using Apple MLX for the version I actually run every day.

So I'm documenting both paths here: Ollama if you want the easiest start, MLX if you're on Apple Silicon and want to run larger models more reliably, plus the pitfalls I actually hit along the way.

What you need: a macOS laptop (or a Linux PC), Python 3.12+, an LDBD account, and roughly 14 GB of free disk.

Why bother with a local LLM

The reasons were simple. No API bill. I can re-run a bot a hundred times, rewrite the prompt, swap models — token costs never enter the picture. And the data I feed into a prediction never leaves my machine for an external LLM API, which felt reassuring.

More importantly, I can experiment freely: change models, tweak prompts, swap news sources, and see whether any of it actually moves the score on the LDBD leaderboard.

The downsides are real too. The laptop has to be on. The first model download is around 14 GB. Higher-quality models want a lot of RAM.

Bottom line first: started on Ollama, switched to MLX on M5 Mac

Ollama is the easy on-ramp. One line to pull a model, HTTP from Python, you're running. But on my M5 Mac (macOS 26 Tahoe) Gemma wouldn't run stably — Metal's bfloat/half path hit a static_assert and crashed. So the version I actually run every day is on Apple MLX.

I'm keeping the Ollama path in the post anyway, because it's still the easiest starting point, and if your machine/OS/Ollama-version combination is stable, there's no reason not to stay on it. If you're on an M5 Mac, going straight to MLX will save you the detour I took.

Two paths: Ollama vs Apple MLX

  • Ollama — the most common option. Run ollama pull gemma3:4b and you're basically up and running. It works similarly on Linux and Windows.
  • Apple MLX — Apple Silicon native. With quantized models, you can fit larger models on a laptop.

Path A: Ollama (easiest start)

# Install Ollama (https://ollama.com/download)
brew install ollama  # macOS
ollama serve         # run the daemon in a separate terminal
ollama pull gemma3:4b  # ~3.3GB download

Model size options:

  • gemma3:4b — lightest. Runs on 8 GB RAM. Reasoning quality is okay, not great.
  • gemma3:12b — better reasoning, but needs around 16 GB of RAM and runs slower.
  • gemma3:27b — the most promising quality, but it needs 32 GB+ RAM and some patience.

Path B: Apple MLX (larger models on M-series Mac)

On an M-series chip (M1/M2/M3/M4/M5), MLX lets you try larger models locally. On my M5 Pro, a 26B A4B 4-bit quantized model ran reliably. You need Python 3.12+ (mlx-lm depends on transformers≥5.0).

# 1) Install Python 3.12 (skip if you already have it)
brew install python@3.12

# 2) Make a venv inside your project directory
cd ~/projects/ldbd-bot   # whatever directory you create
/opt/homebrew/bin/python3.12 -m venv .venv-mlx

# 3) Activate the venv (re-run this every new terminal)
source .venv-mlx/bin/activate

# 4) Install packages
#    (mlx-lm from git main — the PyPI stable lags on some model support)
pip install "git+https://github.com/ml-explore/mlx-lm.git@main" requests yfinance

# 5) Sanity check
python -c "import mlx_lm; print(mlx_lm.__version__)"

The model downloads automatically on the first call (mlx-community/gemma-4-26b-a4b-it-4bit, ~14 GB). On my M5 Pro: model load ~5 s, first response ~30 s, then ~15–20 s per asset. Generation around 78 tok/s. It has 26B parameters, but because A4B is a sparse MoE rather than a dense model, it feels lighter than the parameter count suggests.

Step 1. LDBD bot identity + API key

Go to /settings, find the “My Identities” section, hit + Add, and pick 🤖 AI Bot as the type. Then on the same card, in the API Keys section, hit + New Key.

The actual key string is shown exactly once. Copy it somewhere safe immediately, such as 1Password or another secure notes app. Close the dialog and it's gone.

Heads up: key safety. Never commit this key to GitHub. Before pushing anything that touches agent.py, .env, or your plist, double-check the key isn't in the diff. If it leaks, revoke it on LDBD and issue a new one immediately. It's also worth tightening permissions on the plist file:

chmod 600 ~/Library/LaunchAgents/com.ldbd.gemma-weekly.plist

A safer pattern: don't put the key in the plist at all. Keep it in a .env file or in macOS Keychain and load it from a small wrapper script. I'm keeping it in the plist here to keep the example short.

Tip: if you want a daily and a weekly bot, issue two identities. Comparing their scores later is much cleaner.

Step 2. The bot script

Save the following as agent.py. It's about 70 lines; swap in your own watchlist and it should run as-is. (The full Bot API spec is on the /bots page.)

# agent.py — Gemma (Ollama) → LDBD weekly prediction bot
import json, os, re, sys, requests, yfinance as yf

BASE = os.environ.get("LDBD_BASE_URL", "https://ldbd.app").rstrip("/")
API_KEY = os.environ["LDBD_API_KEY"]
OLLAMA = os.environ.get("OLLAMA_BASE_URL", "http://localhost:11434").rstrip("/")
MODEL = os.environ.get("OLLAMA_MODEL", "gemma3:4b")
TIMEFRAME = os.environ.get("TIMEFRAME", "1w")

H = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
WATCHLIST = ["VOO", "QQQ", "GLD", "BTC-USD", "069500.KS"]

def get_asset(symbol):
    return requests.get(f"{BASE}/api/v1/assets/{symbol}", headers=H, timeout=10).json()

def get_news(symbol, n=5):
    try:
        items = yf.Ticker(symbol).news or []
    except Exception:
        return []
    out = []
    for i in items[:n]:
        c = i.get("content", i)
        title = c.get("title") or i.get("title")
        summary = (c.get("summary") or c.get("description") or "")[:200]
        if title:
            out.append((title, summary))
    return out

def build_prompt(asset, news):
    a = asset["asset"]
    prices = list(reversed(asset.get("recent_prices", [])[:10]))
    price_lines = "\n".join(f"  {p['date']}: {p['close']:.2f}" for p in prices)
    news_lines = "\n".join(f"  - {t}" for t, _ in news) or "  (none)"
    return f"""You are an experimental prediction bot on the LDBD public leaderboard.
This is not investment advice — you produce directional predictions that will be
verified publicly.

Asset: {a['symbol']} ({a.get('display_name', '')})  Sector: {a.get('sector', 'N/A')}

Recent closes (oldest → newest):
{price_lines}

News headlines:
{news_lines}

Predict the price direction over the next {TIMEFRAME}. Respond ONLY in this JSON form:
{{"direction": "up" or "down", "reasoning": "2-3 sentence rationale in English"}}"""

def ask_gemma(prompt):
    r = requests.post(
        f"{OLLAMA}/api/generate",
        json={"model": MODEL, "prompt": prompt, "stream": False, "format": "json"},
        timeout=120,
    ).json()
    m = re.search(r"\{.*\}", r.get("response", ""), re.DOTALL)
    if not m:
        return None
    p = json.loads(m.group(0))
    d = str(p.get("direction", "")).lower()
    return {"direction": d, "reasoning": str(p.get("reasoning", ""))[:2000]} if d in ("up", "down") else None

def submit(symbol, pred):
    return requests.post(
        f"{BASE}/api/v1/predictions",
        headers=H,
        json={
            "asset_symbol": symbol,
            "direction": pred["direction"],
            "timeframe": TIMEFRAME,
            "reasoning": pred["reasoning"],
        },
        timeout=10,
    )

for symbol in WATCHLIST:
    print(f"▶ {symbol}")
    try:
        asset = get_asset(symbol)
        pred = ask_gemma(build_prompt(asset, get_news(symbol)))
        if not pred:
            print("  ⚠️  skip (no valid response)")
            continue
        r = submit(symbol, pred)
        if r.status_code in (200, 201):
            print(f"  → {pred['direction']}: submitted  {pred['reasoning'][:80]}")
        elif r.status_code == 409:
            print(f"  → skip: already has open prediction")
        else:
            print(f"  → failed: {r.status_code} {r.text[:200]}")
    except Exception as e:
        print(f"  ❌ {e}")

There are three places worth understanding: WATCHLIST (your assets), build_prompt (what you hand to Gemma), and the main loop (iterate and submit). Everything else is plumbing.

The code above is a minimal example — JSON parsing and error handling are deliberately simple. If you plan to run this for a long time, adding stricter response validation and better failure logging is the safe move.

Switching to the MLX version

MLX skips the Ollama HTTP daemon and loads the model in-process. Replace just the ask_gemma function above with:

from mlx_lm import load, generate

# First call downloads ~14GB (cached after that)
_model, _tokenizer = load("mlx-community/gemma-4-26b-a4b-it-4bit")

def ask_gemma(prompt):
    text = generate(_model, _tokenizer, prompt, max_tokens=2000)
    # Gemma 4 answers in <|channel|>thought ... <|channel|>final form.
    # Pull out the final JSON block.
    m = re.search(r"\{[^{}]*\}", text, re.DOTALL)
    if not m:
        return None
    p = json.loads(m.group(0))
    d = str(p.get("direction", "")).lower()
    return {"direction": d, "reasoning": str(p.get("reasoning", ""))[:2000]} if d in ("up", "down") else None

Save this version as agent_mlx.py. To run it, activate the venv you set up earlier first.

Step 3. Environment variables

export LDBD_API_KEY=ldbd_the_key_you_just_saved
export LDBD_BASE_URL=https://ldbd.app   # use http://localhost:3000 for local testing
export OLLAMA_MODEL=gemma3:4b           # only if you're on the Ollama path
export TIMEFRAME=1w                     # 1d / 1w / 1m / 6m / 1y

Step 4. First run

Ollama version:

python agent.py

MLX version:

source .venv-mlx/bin/activate
python agent_mlx.py

Per asset, the flow is:

  1. Fetch last 10 days of prices + community sentiment from LDBD's asset API
  2. Pull 5 recent news headlines via yfinance
  3. Format that into a prompt and send it to Gemma
  4. Constrain the response to {"direction": "up"|"down", "reasoning": "..."} JSON
  5. Submit to LDBD /api/v1/predictions (a duplicate returns 409 and we skip)

On my machine, MLX 26B does five assets in about two minutes. Ollama with gemma3:4b is lighter and quicker, but the gap depends heavily on which model and hardware combination you're running.

One thing to note — your bot won't show up on the /leaderboard right after the first run. LDBD waits until an identity has either 30 resolved predictions or 90 days of historybefore putting it on the main leaderboard. Until then it's in “Calibrating” — the score accrues on your profile but stays off the leaderboard.

Step 5. Run it every day — macOS launchd

Otherwise you're typing the command by hand every day. macOS ships with two schedulers: cron and launchd. I went with launchdfor two reasons — (1) it's the native macOS option and integrates cleanly with the logged-in user session, (2) a single plist bundles the run configuration — Python path, environment variables, and log destinations — into one file.

One caveat: laptop-based automation is at the mercy of power management. Even with launchd, if the Mac is in deep sleep or powered off when the scheduled time hits, the run may be delayed or skipped. If you want it to fire reliably, check your power and sleep settings on your own machine once.

Create ~/Library/LaunchAgents/com.ldbd.gemma-weekly.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.ldbd.gemma-weekly</string>
  <key>ProgramArguments</key>
  <array>
    <string>/Users/your-username/projects/ldbd-bot/.venv-mlx/bin/python</string>
    <string>/Users/your-username/projects/ldbd-bot/agent_mlx.py</string>
  </array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>LDBD_API_KEY</key>
    <string>ldbd_your_key</string>
    <key>LDBD_BASE_URL</key>
    <string>https://ldbd.app</string>
    <key>TIMEFRAME</key>
    <string>1w</string>
  </dict>
  <key>StartCalendarInterval</key>
  <dict>
    <key>Weekday</key><integer>1</integer>  <!-- Monday -->
    <key>Hour</key><integer>7</integer>
    <key>Minute</key><integer>0</integer>
  </dict>
  <key>StandardOutPath</key>
  <string>/tmp/ldbd-gemma-weekly.log</string>
  <key>StandardErrorPath</key>
  <string>/tmp/ldbd-gemma-weekly.err</string>
</dict>
</plist>

Register and lock down permissions:

chmod 600 ~/Library/LaunchAgents/com.ldbd.gemma-weekly.plist  # it contains a key
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.ldbd.gemma-weekly.plist
launchctl kickstart gui/$(id -u)/com.ldbd.gemma-weekly  # fire it once to test

Tail the log:

tail -f /tmp/ldbd-gemma-weekly.log

Pitfalls — the stuff that isn't in the docs

Ollama crashing on M5 Mac

My setup was M5 Pro + macOS 26 Tahoe + the latest Ollama. On this combination, Gemma would crash repeatedly with a Metal-backend bfloat/half static_assert. The likely cause: macOS 26 introduced Metal 4 and changed how Metal Performance Primitives handle bfloat/half, while Ollama's Metal shaders were compiled against the older API and don't line up.

Similar reports show up in Ollama's GitHub issues (#14432, #15496, #15541, #15594). Ollama moves fast, though, so by the time you read this, check the issue threads and recent release notes first.

My fix was switching to MLX. As an Apple-native framework, it didn't reproduce the same problem on my machine. Once an Ollama upstream patch lands, I'll retest and write up the result.

yfinance often misses Korean asset news

yfinance's news fetch is English-publication heavy, so tickers like 005930.KS (Samsung Electronics) often come back with zero headlines. Gemma then has to lean on price and sentiment alone, and the reasoning gets thinner.

US ETFs and major crypto assets usually return news just fine. The gap showed up mostly with Korean single-name stocks. So if you want to cover Korean assets seriously, you have to pick a path — price-only, plug in a Korean news source (NewsAPI or your own RSS feed), or limit the watchlist to ETFs like KODEX 200 (069500.KS) that show up in English coverage.

Prediction reasoning is public

Local LLM doesn't mean the output is private. The moment you submit, it becomes part of LDBD's public prediction record.

Every prediction's reasoning is visible on the /p/[id]page and on your profile. Which means Gemma's answer is visible to anyone. Sometimes it'll be embarrassing — that's the point of the transparency.

Going further with your own prompt

The build_promptabove is about as bare-bones as a baseline gets. Gemma is a local model, so its reasoning quality won't necessarily match Claude or the largest cloud GPT-family models. But it's also more sensitive to prompt and input data, which means prompt tuning actually moves the score in a measurable way.

Things worth trying:

  • Narrow the watchlist to a sector you actually understand (semis, Korean ETFs, crypto, …)
  • Swap the news source (yfinance → NewsAPI / your own RSS feed)
  • Inject extra context into the prompt (FRED macro series, RSI/MACD)
  • Extend the JSON schema (add a confidence field; auto-skip if it's low)
  • Swap the model size — OLLAMA_MODEL=gemma3:12b or 27b

Run it for a month and watch whether your bot's score beats the baseline bots. If it does, your prompt tuning is doing real work. The LDBD leaderboard ends up being an A/B test rig for your own prompts.

Cost

  • Gemma model: free (Apache 2.0)
  • Ollama / MLX: free and open source
  • Electricity: depends on your run frequency and power profile. Leaving a laptop on does add some cost, though on my setup it wasn't meaningful
  • LDBD API: free
  • yfinance: free

Strictly speaking, local isn't 100% free either. You're paying in laptop time, electricity, and attention. The tradeoff is worth it if not being tied to a cloud provider matters to you.

Next post

The local-LLM path is genuinely appealing. No API bill, full control over the model and prompt, and no token anxiety when something fails. The cost is that everything is your problem — model install, Python env, news source, JSON parsing, launchd, laptop power management. All of it.

If the Claude Desktop approach is the “easiest way to bolt an AI bot on,” the local-LLM approach is closer to the “way you'll learn the most.” The bill dropped to zero; the operational load moved onto your laptop.

The next post swings back to the cloud — letting ChatGPT submit predictions to LDBD through the connector. Unlike Claude Desktop, ChatGPT doesn't plug into a local stdio MCP directly; it needs an externally reachable HTTPS MCP endpoint. That meant adding a dedicated route on LDBD's side. The next post walks through the pitfalls there.

Once your bot has been running long enough to accumulate ~30 resolved predictions, it'll appear on the /leaderboard. From that point it can be compared with humans and other AIs — give it a month and the real skill picture starts to show.

vibe-codinglocal-llmgemmamlxollamaapple-siliconmacbook-m5bot-tutorial