rtxtEditor.SelectionColor = FontDialog1.Color

Or if you don't want to use fontdialog,

rtxtEditor.SelectionColor = System.Drawing.Color.Yellow