What in the material causes the highlights to be different colors?
Specular highlights occur when light is reflected at the surface in the mirror-like direction.
This wikipedia article https://en.wikipedia.org/wiki/Specular_highlight#Microfacets explains why plastic spectral highlights look white. Plastic spectral highlights reflect the color of the light source because "plastic is made up of tiny beads of color suspended in a clear polymer". Light gets reflected against this clear coating instead of the actual color beads.