Skip to main content

Reinforcement Learning

In short

AI learns by trial and error to maximize a reward signal — applied to trading

RL is how an AI learns to make decisions by trying things and getting rewards or penalties. Think of training a dog — good moves get treats, bad moves get nothing. The AI keeps adjusting until it finds the best strategy.

Reinforcement learning trains an agent to take actions in an environment to maximize cumulative reward. In trading, the agent observes market state, takes buy/sell/hold actions, and receives a reward based on portfolio performance. Unlike supervised learning, RL discovers strategies through interaction.

Related concepts

  • PPO Algorithm — PPO is one of the best algorithms for training AI agents. It learns slowly and steadily instead of making wild policy changes that could destabilize training. Think of it as a careful student who improves step by step.
  • Reward Function — The reward function is the AI's score card. If we reward just raw returns, the agent might take huge risks. Our agent is rewarded for Sharpe ratio — returns relative to risk — which incentivizes consistent, risk-adjusted gains.
  • Observation Space — The observation space is everything the AI trading agent can see before making a decision. It might include price data, technical indicators, sentiment scores, and portfolio state. More useful data = better decisions (up to a point).
  • Action Space — The action space defines what choices the AI can make. A simple agent has 3 actions: buy, sell, hold. A more sophisticated one can trade any fraction of its portfolio. More actions = more flexibility but harder to train.