Skip to main content

Sharpe Reward

In short

Using Sharpe ratio as the RL reward signal — trains risk-adjusted strategies

Instead of rewarding the AI just for making money, we reward it for making money efficiently relative to risk. An AI rewarded with Sharpe ratio learns to avoid unnecessary risks and build consistent returns.

Using Sharpe ratio as the reward signal trains agents to maximize risk-adjusted returns. This prevents the agent from taking extreme concentrated bets. The reward can use rolling Sharpe (more responsive) or episode-level Sharpe. Sortino-based rewards can further reduce downside risk.

Formula

Sharpe Reward = (Rp - Rf) ÷ σp over training episode

Related concepts

  • Sharpe Ratio — Sharpe ratio measures how much return you get for every unit of risk you take. A Sharpe of 1.0 means for every 1% of risk, you earn 1% return above the risk-free rate. Higher is better.
  • Reward Function — The reward function is the AI's score card. If we reward just raw returns, the agent might take huge risks. Our agent is rewarded for Sharpe ratio — returns relative to risk — which incentivizes consistent, risk-adjusted gains.
  • Reinforcement Learning — RL is how an AI learns to make decisions by trying things and getting rewards or penalties. Think of training a dog — good moves get treats, bad moves get nothing. The AI keeps adjusting until it finds the best strategy.
  • PPO Algorithm — PPO is one of the best algorithms for training AI agents. It learns slowly and steadily instead of making wild policy changes that could destabilize training. Think of it as a careful student who improves step by step.