About

The project and the lab

HybridInference is developed by the Harvard MadSys Lab at the Harvard John A. Paulson School of Engineering and Applied Sciences, and published under the MIT license. The routing system at its center, RouteWise, is the subject of the lab’s EuroSys ‘27 paper and ships separately as the MIT-licensed llm-routewise library; this gateway is its production reference integration.

The people behind the project are visible where the work happens: the contributors page lists everyone who has landed a change.

See it running

FreeInference is the lab’s own deployment of this gateway. A deployment’s identity — branding, team, sponsors, user-facing documentation — lives with the deployment, in its overlay and on its own site, not in this repository. That separation is why these pages describe no particular deployment and name no team: a fresh clone comes up as your gateway, not ours.

Getting in touch

Bugs and questions go to the issue tracker; patches follow the contributing guide.