GatewayCrate

One model API, the providers you choose

GatewayCrate

Open-source, self-hosted AI gateway.

Route applications to configured local and cloud model providers through a common API. Powered by LiteLLM, with persistent configuration and PostgreSQL-backed administration, virtual keys, and spend tracking.

View on GitHub

Open source · MIT licensed

Get the book on Amazon

The Self-Hosted AI Builder's Guide Practical guidance for deployment, security, backups, and everyday operation.

Applications → gateway → providers

Choose your routes.
Keep the controls.

APPLICATION ROUTE

OpenAI-compatible API → configured provider

WITH POSTGRESQL

Admin UI · virtual keys · budgets · spend tracking

  • Local models
  • Cloud providers
  • Virtual keys
Illustrative route. Provider requests may leave your server and incur charges; a self-hosted gateway does not make cloud inference local.

A focused service for your applications

A common API for the models you configure.

Put model routing and administrative controls in one service. Use local endpoints, approved external providers, or an explicitly configured mix. Your providers perform the model inference.

Images are automatically built and published through public GitHub Actions workflows. Powered by LiteLLM.

Start with the database-backed setup

Configure your routes.
Keep access deliberate.

Use the maintained Compose setup when you want the Admin UI, virtual keys, and spend tracking. It includes PostgreSQL without publishing its database port.

  • Docker and Docker Compose on a supported Linux host
  • A configured local model endpoint or provider credentials for model requests
  • A text editor to configure the env file and loopback port binding

Full setup guide on GitHub

For access from other computers, follow the maintained HTTPS, authentication, and network-exposure instructions.

  1. Get the maintained configuration

    git clone https://github.com/hwdsl2/gatewaycrate.git
cd gatewaycrate
cp litellm.env.example litellm.env

    Edit litellm.env to add your selected provider credentials or a reachable local endpoint. External providers may charge for requests.

  2. Bind the gateway to loopback

    services:
  litellm:
    ports:
      - "127.0.0.1:4000:4000/tcp"

    In docker-compose.yml, replace the existing litellm service’s port mapping with this value. Keep the other services, settings, and volumes. Then run docker compose up -d and check docker compose logs.

  3. Retrieve the master key and list routes

    gateway_master_key="$(docker exec litellm litellm_manage --getkey)"
curl http://127.0.0.1:4000/v1/models \
  -H "Authorization: Bearer $gateway_master_key"

    This local administrative check can return an empty model list until routes are configured. Use database-backed virtual keys for applications; do not put the master key in browser code or share it with users.

Part of a larger toolkit

Build your Self-Hosted AI Stack

GatewayCrate is also available as part of Self-Hosted AI Stack. Deploy local models, chat, document processing, voice, and AI tools together, or choose a smaller configuration for your needs.

Explore Self-Hosted AI Stack

Connect the services you need

Serve your own local model

InferCrate runs Ollama models behind API-key authentication. Connect it as a local backend when you want model inference on infrastructure you operate.

Explore InferCrate

Before you deploy

Is GatewayCrate free to use?

The GatewayCrate project is open source under the MIT License. You provide hardware or cloud hosting and pay associated costs. Models, dependencies, and external providers retain their own licenses and terms.

Does GatewayCrate run a language model?

No. It is an API gateway. A configured backend, such as InferCrate or an external provider, performs inference. The gateway itself does not need a GPU.

Are cloud-provider requests private and local?

Requests to external providers leave your server and follow those providers’ terms and data practices. A local-only route requires a configured local backend and an application that uses it.

Do the Admin UI and virtual keys need PostgreSQL?

Yes. Use the included Compose setup or another correctly configured PostgreSQL database. A minimal standalone container does not by itself provide the database-backed management features.

Can it start without provider API keys?

Yes, but the model list may be empty. Configure a reachable local endpoint or supported provider credentials and model routes before making chat requests.

Which key should my application use?

Use an appropriately restricted virtual key where possible. The master key has administrative access and should be kept private. Virtual-key creation, restrictions, and budgets require the database-backed setup.

The image includes optional aggregate usage counts. See the usage-count documentation; set LITELLM_DISABLE_USAGE_COUNTS=1 to disable them.

Choose your providers. Make every route intentional.

View on GitHub