Im relatively positive removed from play cards are still public knowledge as long as they were removed "face-up", otherwise, you could not verify that your opponent actually DID remove a light and dark to special summon a chaos monster, or an earth type when summoning 2 Gigantes (as an example).
As another example, I have a misprinted Goblin Attack Force. The Earth Attribute has been replaced with the "Star" symbol used to show monster level.
If you did not know that GAF was an Earth-type, and only had a glance at the symbol, it would look like I removed a "Fire" attribute for Gigantes. As game play progresses (and my opponent is slightly forgetful), maybe they would like to verify that indeed GAF is not "Fire" and that it just looks different. Would that then mean that regardless of his confusion, my removed from play is no longer questionable?