Actually...
Food coloring can definitely change the flavor of food. Red, purple and black all seem to have a mildly bitter aftertaste. Also, I use a lot of natural food colorings in my work and, since they're derived from turmeric and things like that, they often retain some of the flavor. I assure you it's quite weird to taste a cake with turmeric-laced icing.