The artwork, which organisers claimed to be the first in Europe, had been added to Madeira Drive to mark LGBT Pride Month.
Brighton and Hove Council said the rainbow was scrubbed away because it was an artwork that had never been intended as an official crossing.
Some residents had called for the crossing to become a permanent feature.
Neil Sykes, who owns The Modern World Gallery on Madeira Drive, said the response to the rainbow had been "astonishing" with visitors coming from across the country to see it.
He also claimed the road needed a new crossing because of a lack of safe points for pedestrians to cross...
What utter tosh.