# Rewards Model Explained

* **Reward Modelling** is an essential part of making high-quality models.
* Best-of-N sampling is the main part of InstructGPT loop
* In fact, if you read the OpenAI's paper carefully, PPO out-of-the-box only achieves best-of-2 performance!
* Contrary to PPO (Proximal Policy Optimization), Best-of-N sampling is **simple**, **robust** and **high-performant**

{% embed url="<https://arxiv.org/pdf/2303.06135.pdf>" %}
In depth explanation
{% endembed %}

**Figure below shows the architecture for best-of-4 sampling with reward**

<figure><img src="https://1857108475-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjXeBfUiM9SFkYfrZF6FA%2Fuploads%2FdSROJ50Ej5bW6WmSqFg7%2Fdownload.png?alt=media&#x26;token=edf5da32-4aa0-464d-aae6-acf5d0a487d6" alt=""><figcaption></figcaption></figure>

## Keep in mind these parameters

<figure><img src="https://1857108475-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjXeBfUiM9SFkYfrZF6FA%2Fuploads%2FLUGA7LFnMKAtRuMlrQ2r%2Fimage.png?alt=media&#x26;token=b9eb2381-51fc-4502-ae63-66ee2d0bea1a" alt=""><figcaption></figcaption></figure>
