Autoresearch
Run single-GPU model experiments with a coding agent, then compare changes under a fixed training budget.
- Development Workflows
- Agent Workflows
- Performance
Andrej Karpathy
Andrej Karpathy publishes practical ways to work with language models, from single-GPU experiments run by coding agents to a Markdown wiki an agent can maintain over time.
About Andrej Karpathy
Karpathy’s public work spans language model training and the workflows that make models useful. Autoresearch gives a coding agent a training script to modify, a fixed time budget, and an evaluation metric. The resulting log and Git history let a researcher compare proposed changes with measured outcomes.
LLM Wiki takes a different approach to persistent knowledge. A file-capable agent reads source documents, writes linked Markdown pages, and updates existing pages when new evidence changes an earlier conclusion. Karpathy published the design as a Gist. Readers can use it to specify a wiki in their own workspace.
Autoresearch: Agent-Run Training Experiments
The upstream project shows how an agent can iterate against a fixed training budget. Its experiment instructions specify what to change and how to record successful and failed runs.
Explore Other Publishers
Meet other people and teams publishing open tools, projects, and Agent Skills.
Frequently Asked Questions
What does Autoresearch actually run?
A coding agent edits the training script, runs short model-training experiments, and logs the results. The default comparison metric is validation bits per byte.
What do I need to try the upstream experiment?
The official setup calls for a single NVIDIA GPU, Python 3.10 or later, and uv. The README lists community ports separately.
Which agents does Autoresearch document?
The README names Codex and Claude. Start the agent in the repository and give it the experiment instructions after confirming a baseline run works.
How do I compare Autoresearch results?
Keep the data, evaluation setup, GPU, and per-run budget fixed. Compare validation bits per byte and check the change history before accepting a result.
What did Karpathy publish as LLM Wiki?
He shared a design for building and maintaining a personal wiki with an agent. The original Gist describes the file layout, update routine, and role of source material.
Does LLM Wiki require Obsidian?
Karpathy uses Obsidian to browse the wiki. An agent can maintain the underlying linked Markdown in a workspace where it can read the sources and edit the wiki.
Build a Knowledge Base with the LLM Wiki Pattern
Read how source documents, linked pages, and agent updates fit into a persistent Markdown knowledge base.