Who remembers colors better? Men or women? Studies show that it is women who have the better color memory.
The theory for this difference in the sexes is based on evolution, and species survival. When early man was busy outside the cave searching for red meat (hunter activity), the women were out searching for red berries (gathering). Men developed the ability to track moving objects; women on the other hand learned to tell the difference between the edible red berries and the poisonous red berries. This is also why you don't run into many color blind women (the trait for color blindness is located on the X chromosome, and since women have two X chromosomes, they are less likely to have abnormal color vision on both - color blindness occurs in 0.4 percent of women, and 8 percent of men).
Of course, color memory can be affected by factors other than gender as well. Age and color familiarity (using a variety of colors on a regular basis, perhaps in a work environment) also play their roles, and some colors are easier to remember than others. Therefore a young male artist would most likely outmatch yellows with an eighty year old, retired, female lawyer in a color skill test.
Interestingly, the time delay between being exposed to a color and being asked to match that color from memory does not have a significant effect on memory. If it did, then each harvesting season, early man would have had to learn the poisonous fruits all over again by trial-and-error.