Intro topic: Running

News/Links:

Book of the Show

Tool of the Show

Topic: World Models

Making decisions with AI
- Action-Value (called a Q model): What is the long-term value of making a decision at a position
- Policy: What action should I take (must be a distribution)
- Value (called a V model): What is the value of a position (depends on policy)
- Advantage/Disadvantage: difference in value given two policies
- When advantage is +, do that more.
Model-Free
- Look at the current situation and suggest an action
- Run that action in the real world and measure the effect
- Use that measurement to suggest better actions next time
Model-Based
- Observe rollouts (sequences of situations) and learn the dynamics
- Choose an action, use your dynamics model to measure the consequence
- Potentially do MPC (try many actions and choose the best)
World Models
- Observe many many rollouts and learn a full forward model (how to create the input in the future)
- Train a policy & value inside the world model
- Deploy the policy and fine-tune based on the real world