Joy is a deep satisfaction with God that permeates all other areas of our lives. We have joy because we have him, and our hearts—even on the difficult days—enjoy God’s goodness, his beauty, and the beauty he has placed around us.
If God were to literally forget our sins, as in erasing them from His memory in the way that humans forget trivial details, this would introduce serious theological problems.