As the son of Russian-Jewish parents he grew up under the influence of klezmer and Russian folk music and sang in the synagogue. He became rang the bell at the Episcopalian Church St. Thomas on Fifth Avenue in New York, where for 15 minutes each day he let predetermined church songs ring out on one of the few large, playable glockenspiels in North America.
Subsequently he often improvised hours long and arouse the interest of the artists and musicians of the New York avant-garde scene like Tony Conrad, Moondog, La Monte Young and John Cale (Velvet Underground), who successively hung out in the church and who listened to him as attentively as they did critically, for the competition among those artists devoted to the invention of new music was intense. On account of the minor commercial success of his music he switched for to visual arts.
In the mid-1990s Palestine released some of his earliest musical works. They became a source of inspiration for a new generation, including Nice Cave and Thurston Moore of Sonic Youth. Solo albums followed, as did numerous collaborations, like with Mika Vainio and Pan Sonic, Michael Gira (Swans), Tony Conrad, Keith Rowe, Terry Jennings, Rhys Chatham, Simone Forti and Mama Baer.
He gave live performances again, for which the largest and the loudest concert piano in the world (Bösendorfer Imperial) was perfectly suited to him.