👓 Omi (prototype)
Developer prototype for Omi capture hardware. This prototype is a backend that runs on a PC you control, connects to an Omi DevKit 2 over Bluetooth LE, transcribes audio locally with Whisper, and analyzes each transcript with a local Ollama model.
Status: prototype. There is no store build or release. The source is public at companionintelligence/CI-OMI , and you run it from source on a PC you control. It is also not connected to Companion Memory or Hub: today it is a standalone service with its own REST API, WebSocket, and event log. It doesn’t write to Companion Memory and it isn’t a Hub app. See Limitations for what isn’t built. To follow progress or ask questions, join the Discord .
Overview
Omi is an open-source wearable line. The prototype targets two developer kits: the Omi DevKit 2, which streams audio over Bluetooth LE, and the omiGlass DevKit, an ESP32-S3 with a camera. The backend runs on a “companion PC” on your network. It scans for the device over Bluetooth, reads the device’s audio stream, transcribes it with the open-source Whisper model on that PC, asks a local Ollama model to label each transcript (topic, tone, intent, entities, urgency), and logs the result as a JSON event you can read over a REST API or a live WebSocket.
Transcription and analysis run on the companion PC. The first run downloads the Whisper model, and the Ollama models must already be pulled. The service also has an optional Omi API client (OMI_API_KEY, default https://api.omi.me) for Omi’s own cloud, and it accepts webhooks that Omi’s cloud can send to it. Those are separate from the local pipeline.
Key features
- Bluetooth LE audio — scans every 10 seconds for BLE devices whose name contains “omi”, “devkit”, or “glass”, connects, and starts an audio handler for DevKit devices.
- Local transcription — Whisper runs on the companion PC. The default model is
base, set withWHISPER_MODEL. - Local analysis — Ollama labels each transcript with a topic, tone, intent, entities, urgency, and short summary. The default chat model is
llama3.2:3b. - Event log and API — events are stored in a SQLite database on the PC. Read them with
GET /api/events, watch them on the/ws/liveWebSocket, or send events in through/api/webhooks/conversation,/api/webhooks/audio, and/api/webhooks/transcript. - Required API key — every endpoint except
GET /healthneeds anX-API-Keyheader.
Supported hardware
| Device | Connection | Status |
|---|---|---|
| Omi DevKit 2 | Bluetooth LE to the companion PC | Audio capture works end to end in the code, with the caveats in Limitations |
| omiGlass DevKit | Bluetooth LE discovery | Discovered and connected, but camera capture isn’t wired up |
There is no phone relay and no Wi-Fi setup mode for the pendant in the repository. The companion PC talks to the device directly over Bluetooth, so it needs a Bluetooth adapter and must be within range of the device.
Companion devices compared
Omi is one of several Companion devices. Each one captures or shows something different, and they connect in different ways; this table records only what each device’s own page states.
| Device | Audio in | Camera | Output on the device | Link to the Hub |
|---|---|---|---|---|
| Omi (prototype) | Bluetooth LE audio from an Omi DevKit 2; Whisper on a PC you run | Bluetooth LE to a PC running the prototype. Not linked to Companion Memory or Hub | ||
| Mentra Live | Voice through MentraOS; speech-to-text on a server you provide | Photos on request, opt-in | Display when the glasses have one; spoken replies; red privacy LED | MentraOS Cloud to the bridge's public HTTPS address; Companion Memory through the bridge |
| Even Realities G2 | Touchbar long-press; Whisper (the Web Speech fallback uses the phone mic) | Green-on-black MicroLED HUD, five lines per page | Bluetooth LE via the Even Hub app; the app calls your agent-hub over HTTP | |
| Pebble | Pebble Dictation (a cloud speech service), on watches with a microphone | E-ink display | Bluetooth to the phone's Pebble app, which calls Companion Memory over HTTPS | |
| Sipeed MaixCAM | Optional audio recording | On-device models on the camera's NPU | Wi-Fi; JSON detection events over HTTPS to Companion Memory | |
| Home Assistant Voice PE | Mic array, on-device wake word; Whisper in Home Assistant or Companion | Speaker (Piper or Kokoro TTS) | Ethernet or Wi-Fi, via Home Assistant (Wyoming, TCP 10700) |
Setup for each lives on its own page: Mentra Live, Even Realities, Pebble, Sipeed MaixCAM, and Home Assistant Voice PE.
Use cases
- Experiment with capturing conversations on your own hardware and transcribing them locally.
- Feed transcripts and events from a DevKit into your own tools through the event API.
Architecture
The CI-OMI prototype is open source on GitHub at companionintelligence/CI-OMI . It connects Omi wearable hardware over Bluetooth LE to a local Python FastAPI service on a companion PC, transcribing audio with Whisper, analyzing transcripts with Ollama, and storing structured events in a local SQLite database.
flowchart TD
DK["Omi DevKit 2 / omiGlass"] -->|"BLE Stream"| BLE["BLE Device Manager<br/>10s scan loop"]
BLE --> AudioPipe["Audio Pipeline<br/>16-bit PCM handler (Opus stubbed)"]
BLE -.->|"Placeholder in prototype"| VisionPipe["Vision Pipeline<br/>Frame Extractor (MediaPipe / YOLO)"]
AudioPipe --> STT["Speech-to-Text<br/>Local Whisper (WHISPER_MODEL=base)"]
VisionPipe -.-> STT
STT --> NLU["NLU & Intent Extraction<br/>Local Ollama (llama3.2:3b)"]
NLU --> EventLog[("Local SQLite Event Store")]
Webhooks["Omi Cloud Webhooks<br/>/api/webhooks/conversation · audio · transcript"] --> EventLog
EventLog --> API["FastAPI REST & WebSockets<br/>Port 8000 (X-API-Key guarded)"]
EventLog -.->|"Planned (not wired in prototype)"| MemoryIngest["Companion Memory Ingest<br/>POST /v1/events"]Subsystems and pipelines
- Bluetooth Low Energy (BLE) ingestion: Scans every 10 seconds for BLE peripherals matching
omi,devkit, orglassand attaches an audio handler for Omi DevKit 2 devices. In the current prototype, incoming audio bytes are processed as raw 16-bit PCM (Opus packet decoding is a placeholder), andomiGlasscamera capture is not yet wired to feed frames into the vision pipeline. - Local transcription and NLU analysis:
- Audio processing: Feeds captured audio buffers into local Whisper (
WHISPER_MODEL=baseby default) on the companion PC. - Computer vision (scaffolded): Includes MediaPipe and YOLO / Moondream (
moondream:1.8b-v2-fp16) modules for frame analysis, thoughPOST /api/capture/imagecurrently returns a placeholder response. - Intent extraction: Sends each transcript to local Ollama (
llama3.2:3b) to extract topic, tone, intent, entities, urgency, and a summary.
- Audio processing: Feeds captured audio buffers into local Whisper (
- Local SQLite event store, API, and authentication boundary:
- Analyzed events and inbound webhook payloads (
/api/webhooks/conversation,/api/webhooks/audio,/api/webhooks/transcript) are written to a local SQLite database and served overGET /api/eventsand the/ws/liveWebSocket. - Every route except
GET /healthrequires the local service key (OMI_COMPANION_API_KEY, sent asX-API-Keyor?api_key=on WebSocket handshakes). This key is distinct from the optional upstreamOMI_API_KEYused by the Omi cloud client (https://api.omi.me). Forwarding events from SQLite to Companion Memory (POST /v1/events) is planned but not implemented in this prototype.
- Analyzed events and inbound webhook payloads (
Setup
You need a PC with Python 3.8 or newer, at least 8 GB of RAM (16 GB recommended), and 10 GB of free disk space. Linux is recommended. macOS and Windows are also listed.
Get the source
Clone the public repository:
git clone https://github.com/companionintelligence/CI-OMI.git
cd CI-OMIRun the setup script
chmod +x scripts/setup.sh
./scripts/setup.shThe script creates a Python virtual environment, installs the backend’s requirements, creates the data/ folders, and writes a .env file that includes a generated OMI_COMPANION_API_KEY.
Install Ollama and pull the models
Install Ollama as described in Installing Ollama, then start it and pull the two models:
ollama serve
# In a second terminal:
ollama pull llama3.2:3b
ollama pull moondream:1.8b-v2-fp16Prepare the hardware
Power on the Omi DevKit 2 so it advertises over Bluetooth as “Omi”.
Start the backend
./run.shThe prototype is not a Hub app, so it has no Hub hostname. The API listens on port 8000 of the PC that runs it. Open http://localhost:8000/docs for the interactive API reference.
Usage
Authenticate
Every endpoint except GET /health requires the key from .env:
export OMI_COMPANION_API_KEY=$(grep OMI_COMPANION_API_KEY .env | cut -d= -f2)
curl -H "X-API-Key: $OMI_COMPANION_API_KEY" http://localhost:8000/api/statusIf no key is set, the service answers every guarded request with 503. It never falls back to open access.
Start capture
curl -X POST -H "X-API-Key: $OMI_COMPANION_API_KEY" http://localhost:8000/api/startList connected devices with GET /api/devices and read logged events with GET /api/events.
Watch events live
The /ws/live WebSocket streams events as they happen. Browsers can’t set headers on a WebSocket handshake, so pass the key as an api_key query parameter. Other clients should send the X-API-Key header instead, because query parameters end up in proxy logs.
Limitations
- Not connected to Companion Memory or Hub. Nothing sends transcripts or events to Companion Memory. There is no Hub app listing, no Hub pairing, and no Omi source in Companion Memory.
- Audio decoding is a placeholder. The audio processor treats incoming bytes as raw 16-bit PCM. Decoding the device’s Opus stream isn’t implemented.
- No camera capture. The vision code exists, but nothing feeds it frames, and
POST /api/capture/imagereturns a placeholder response. - No web UI, speaker diarization, or retention settings. The service is API-only. The event store can delete events older than a set number of days, but no setting or API calls it.
- Most of the plan is unbuilt. The repository’s own implementation checklist has 6 items checked and 255 unchecked.
Troubleshooting
Ollama connection failed
Check that Ollama is running with curl http://localhost:11434/api/tags. Start it with ollama serve.
No device found Make sure the DevKit is powered on and advertising, that Bluetooth is enabled on the PC, and that the device isn’t already connected to another host. The scan looks for names that contain “omi”, “devkit”, or “glass”.
Every request returns 503 or 401
The service refuses guarded requests until OMI_COMPANION_API_KEY is set, and rejects requests without a matching X-API-Key header. Run ./scripts/setup.sh or ./run.sh, which generate a key and write it to .env.
Port 8000 is already in use
The service always listens on port 8000 on all network interfaces. The API_HOST and API_PORT settings in .env aren’t read. Find the process that holds the port with lsof -i :8000 and stop it.
Logs The service logs to the terminal where you started it.
Links
- GitHub repository (companionintelligence/CI-OMI) — open-source repository
- Omi official site
- Companion Memory
- Mentra Live