The colors that make a person up could be shifted to a part of
light that goes BEYOND the rainbow. Lower than Red or higher
than Violet. Everything could work exactly as it would for a
visible person, but humans would be unable to see that person at
all. You would not see a shadow because the invisible person
would not affect ANY of the rainbow colors, and shadows that
humans see are only made when rainbow colors are blocked. If you
don't block rainbow colors, you don't cast shadows. But would an
invisible person be able to see another invisible person? Maybe:
if their eyes could also see a bigger rainbow than we can see.