Skip to main content

Reward Function

In short

Defines what the RL agent is optimizing — the score it tries to maximize

The reward function is the AI's score card. If we reward just raw returns, the agent might take huge risks. Our agent is rewarded for Sharpe ratio — returns relative to risk — which incentivizes consistent, risk-adjusted gains.

The reward function determines agent behavior. Naive return maximization leads to excessive risk. Risk-adjusted rewards (Sharpe ratio, Sortino) train more robust strategies. Reward shaping (intermediate rewards for position quality) can accelerate learning but risks reward hacking.

Related concepts

  • Reinforcement Learning — RL is how an AI learns to make decisions by trying things and getting rewards or penalties. Think of training a dog — good moves get treats, bad moves get nothing. The AI keeps adjusting until it finds the best strategy.
  • Sharpe Reward — Instead of rewarding the AI just for making money, we reward it for making money efficiently relative to risk. An AI rewarded with Sharpe ratio learns to avoid unnecessary risks and build consistent returns.
  • PPO Algorithm — PPO is one of the best algorithms for training AI agents. It learns slowly and steadily instead of making wild policy changes that could destabilize training. Think of it as a careful student who improves step by step.
  • Observation Space — The observation space is everything the AI trading agent can see before making a decision. It might include price data, technical indicators, sentiment scores, and portfolio state. More useful data = better decisions (up to a point).