🎭 MetaHuman

A face for a local AI, built in Unreal Engine. A photoreal character in a 1970s mainframe-room set, designed to talk with you. It is designed to run against chat and speech servers you host yourself, and to stream to a browser with Pixel Streaming or run as a native Windows app built from the same source.

Status: source only. There is no packaged build. The v0.1.0 pre-release on GitHub has no installer, container image, or binary, only GitHub’s automatic source archives. Its release notes say the project has not been built, packaged, or run end to end as part of that release, and the repository has no CI workflows. To use it, you build it yourself with Unreal Engine 5.8, under Epic Games’ terms. It isn’t a Companion Hub app and isn’t in the Hub app store.

Overview

CI-MetaHuman is an Unreal Engine 5.8 project. The companion’s code is a reusable plugin, Plugins/CICompanion, written in C++. A chat backend answers, a speech service speaks, a transcription service listens, and the character’s face is driven to match.

The character in the project is a Reallusion Character Creator avatar named Nia, whose persona is Eliza. The repository keeps the MetaHuman name from an earlier design that used an Epic MetaHuman, and the Epic MetaHuman plugins stay optional in the project file.

You can reach the character two ways:

Pixel Streaming — a Linux container renders the Unreal app and streams video and audio to a browser over WebRTC. The player is served on port 8090.
Native Windows — the same source builds a native Windows executable that draws to a local window.

What the repository says it does

These behaviors come from the project’s README and docs. They were not verified for the v0.1.0 release.

Area	What it provides
Conversation	Chat goes to an interchangeable backend: native Ollama (`/api/chat`) or any OpenAI-compatible `/v1/chat/completions` server, such as LM Studio, Lemonade, or vLLM, chosen by configuration.
Voice	Replies are spoken with Kokoro text to speech, and Whisper transcribes speech, both through an OpenAI-shaped HTTP server such as AMD Lemonade. Each turns on only when you set its server address; see Requirements. Lip-sync is timed from the synthesizer’s phoneme durations, with documented fallbacks.
Face and eyes	Reply text drives a sustained facial expression over the mouth movement, with involuntary blinking and eye movement that differs while the character listens, speaks, and thinks.
Memory	An opt-in setting, `MHLLM_MEMORY=1`, writes conversation transcripts to disk in plain text, with no expiry. It isn’t connected to Companion Memory, the personal memory product.
Delivery	A Linux container streams to a browser with Pixel Streaming, or the same source builds a native Windows executable.

Technical architecture

flowchart TB
    accTitle: How MetaHuman turns a typed or spoken question into an animated reply
    accDescr: Text or microphone input goes to a chat and cognition backend. Speech services turn the reply into audio. Emotion and response state, and phoneme-timed lip-sync, drive the face of the Unreal avatar. The avatar is delivered as a Windows app or through Pixel Streaming.
    Input["Text or microphone input"] --> Backend["Chat and cognition backend<br/>Ollama or OpenAI-compatible"]
    Backend --> Voice["Speech services<br/>Kokoro and Whisper"]
    Backend --> Emotion["Emotion and response state"]
    Voice --> Face["Phoneme-timed facial animation"]
    Emotion --> Face
    Face --> Unreal["Unreal avatar"]
    Unreal --> Delivery["Windows app<br/>or Pixel Streaming"]

Plugin: Plugins/CICompanion holds the companion code, including the avatar and face, the chat clients, voice, and the UI, plus an automation test suite. Source/MetaHumanLocalLLM is a near-empty host module.
Content: Content/ holds the set and the character. Deployment/author_diorama.py authors most of it at build time.
Deployment: Deployment/ holds the Dockerfiles, the compose files that define the signalling, TURN, and sign-in proxy services, and a sidecar that returns phoneme durations from Kokoro.
Configuration: every endpoint and credential comes from environment variables, so one image serves every environment. Docs/CONFIGURATION.md lists them all.

Requirements

Need	Detail
Unreal Engine	5.8, which you obtain from Epic Games under Epic’s terms. The C++ also compiles on 5.5, but the project’s content packages need 5.8 to cook
Epic image access	For the container build, link your GitHub account to Epic Games, accept the Unreal Engine EULA, and run `docker login ghcr.io`
A GPU host	A Linux host that can render with Vulkan. NVIDIA hosts encode with NVENC, AMD hosts encode VP8 in software, and a CPU-only fallback is slower. The README says an NVIDIA GPU isn’t required
Docker	Docker with Compose v2, for the container route
A chat server	Ollama (default `http://127.0.0.1:11434`, model `llama3.2`) or any OpenAI-compatible server
Speech services	Kokoro and Whisper behind an OpenAI-shaped server such as AMD Lemonade. There is no default address. Set `KOKORO_SERVER_URL` and `WHISPER_SERVER_URL` in `Deployment/.env`, where `.env.example` leaves both blank. With the first blank, and `PIPER_SERVER_URL` blank too, the character is mute and the app logs a warning. With the second blank a stub replaces the transcriber, so the microphone does nothing. The conversation test script can run without them with `--llm-only`
A sign-in client	For the sign-in proxy, an OIDC client registered in Companion Portal, the cloud control plane behind CI Account: an issuer URL, a client ID, and a client secret. The README says this sign-in gate has never worked; see Limitations
TURN credentials	A `TURN_USER` and `TURN_PASSWORD` for the WebRTC relay. `.env.example` ships a placeholder password to replace

Deployment/docker-compose.face.yml is the other way to get speech addresses. It supplies http://127.0.0.1:13305 for both, but it also points the chat at the authors’ own endpoint and model, http://127.0.0.1:11435 and qwen3.8:27b, unless you set OLLAMA_BASE_URL and OLLAMA_MODEL.

Build and run the container

The repository’s Deployment/README.md documents this path. Nothing here was verified for the v0.1.0 release, and that guide says the container route has not been built or run end to end: the first docker build is also the first Linux cook.

Get Epic image access

Link your GitHub account to Epic Games and accept the Unreal Engine EULA. Then sign in to the registry:


docker login ghcr.io

Copy the two environment files


cd Deployment
cp .env.example .env
cp .env.edge.example .env.edge

.env holds the app settings. .env.edge holds the two secrets for the sign-in proxy. Deployment/README.md lists the values that have no usable default: the Companion Portal issuer URL, client ID, and redirect URL, and TURN_USER and TURN_PASSWORD, in .env, and the proxy’s client secret and cookie secret in .env.edge. Also set KOKORO_SERVER_URL and WHISPER_SERVER_URL in .env, as the speech services row describes.

Build and start

On an NVIDIA host:


docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up -d

On an AMD host:


PROJECT_DOCKERFILE=Deployment/Dockerfile.amd \
  docker compose -f docker-compose.yml -f docker-compose.amd.yml up -d --build

The first build also runs the initial Linux cook, which the deployment guide describes as long and multi-gigabyte. If KOKORO_SERVER_URL and PIPER_SERVER_URL are both blank, the character is mute: the audio track connects but carries silence, and the app logs a warning that names both.

Open the player

The deployment guide tells you to browse to the sign-in proxy on port 8095, which sends you to Companion Portal to sign in, and to keep the Pixel Streaming player on port 8090 internal. The README reports that the sign-in gate has never worked and that port 8090 is open on the tailnet, which is a Tailscale private network. Read Limitations before you expose either port.

For the native Windows build and the test scripts, see the README’s “Build and test (Windows)” section and Docs/BUILD-AND-DEPLOY.md.

Configuration

Set these in the environment. The README shows these example values. They aren’t all defaults: Deployment/.env.example leaves KOKORO_SERVER_URL, WHISPER_SERVER_URL, OLLAMA_BASE_URL, and OLLAMA_MODEL blank.

Variable	Example	What it sets
`OLLAMA_BASE_URL`	`http://127.0.0.1:11434`	The Ollama address
`OLLAMA_MODEL`	`llama3.2`	The chat model
`KOKORO_SERVER_URL`	`http://127.0.0.1:13305`	The speech synthesis server
`WHISPER_SERVER_URL`	`http://127.0.0.1:13305`	The transcription server
`WHISPER_MODEL`	`Whisper-Base`	Required by the OpenAI request shape. A blank value omits the field
`MHLLM_MEMORY`	`1`	Opt-in. Writes conversation transcripts to disk

Limitations

No packaged build. You build the Linux container or the Windows executable yourself, and a first build is long. The deployment guide says the container route has not been built or run end to end.
No working sign-in. The README says the OIDC sign-in gate has never worked and that port 8090 is open on the tailnet, a Tailscale private network. The deployment guide says to keep 8090 internal and to expose only the sign-in proxy on 8095. Until the gate works, keep the host on a private network.
One viewer at a time. The roadmap lists a single Unreal process, one streamer, and one conversation.
No head motion or gestures. The character doesn’t nod, tilt, or gesture, and it can’t act on the world, because tool calling isn’t built.
Little observability. Turn latency and failures are visible only in the container logs.
Not connected to Companion Memory or Hub. The chat backend is an Ollama or OpenAI-compatible address you configure. CI_HUB_URL is a historical name for that address, and the deployment README says the Hub isn’t in the chat path.
The docs disagree about the engine version. The project file, the Dockerfile, and Docs/ARCHITECTURE.md target Unreal Engine 5.8. The top-level README and Docs/README.md still say 5.5 for the shipped build.

License

Companion Intelligence’s own code and content are free for noncommercial use under the PolyForm Noncommercial License 1.0.0 . That license doesn’t cover Epic Games’ Unreal Engine or MetaHuman assets, which stay under Epic’s terms, or third-party content in Content/, which keeps its own license. See the repository’s NOTICE.md .