Vibe Coding DiscoverAnyJev
View on GitHubTurn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating)
Python library that turns an open LLM into a calibrated decision model by reading typed questions from one next-token prefill. L0 removes position/label bias with zero labels; L1/L2 add temperature or closed-form heads for real probabilities, with shipped Qwen3 heads.
Use Cases
routing requests between categories with calibrated probabilitiesgating risky or irreversible tool calls in agentsscoring task completion on a 5-bin scalethreshold-based auto-decision with human fallbackreducing answer order-flip bias from raw logitsconverting uncalibrated LLM confidence into usable probabilitiesserving per-question decisions at lower latency than full forward passescollecting labels from a review queue and auto-fitting decision headscalibrating LLM classifiers with 100-500 labelsonline recentring of heads after prompt rewordingreplacing a fine-tuned decision model without trainingbenchmarking decision calibration across models and tasks
Built With
- Language
- Python
- Frameworks
- PyTorch · Hugging Face Transformers · vLLM · SGLang · NumPy · scikit-learn · Datasets · pytest · ruff · setuptools
Tags
calibration · decision-making · llm · logits · uncertainty-quantification · training-free · probabilities · typed-decisions · system-one · inference · evaluation · temperature-scaling · hidden-states · human-in-the-loop · qwen3 · probability-thresholding