Skip to main content

RL Model Training

In short

The process of running the RL agent through historical data to learn a trading policy

Training the RL model means running the AI through years of historical market data thousands of times, letting it learn from its mistakes. Like a chess AI playing millions of games — it gradually figures out what works.

RL model training involves iterative episodes where the agent observes market states, takes actions, and receives rewards. Training uses vectorized environments for speed. Hyperparameters (learning rate, batch size, clip epsilon) significantly impact convergence and final policy quality.

Related concepts

  • Reinforcement Learning — RL is how an AI learns to make decisions by trying things and getting rewards or penalties. Think of training a dog — good moves get treats, bad moves get nothing. The AI keeps adjusting until it finds the best strategy.
  • PPO Algorithm — PPO is one of the best algorithms for training AI agents. It learns slowly and steadily instead of making wild policy changes that could destabilize training. Think of it as a careful student who improves step by step.
  • Walk-Forward Validation — Walk-forward validation trains the AI on old data, then tests it on the next period — then repeats. This mimics real-world conditions and prevents the AI from 'memorizing' the training data instead of learning real patterns.
  • Sharpe Reward — Instead of rewarding the AI just for making money, we reward it for making money efficiently relative to risk. An AI rewarded with Sharpe ratio learns to avoid unnecessary risks and build consistent returns.