He was born in Houston, Texas in 1940. Felder, Wayne Henderson, Joe Sample, and Stix Hooper founded the group while in high school in Houston. The Jazz Crusaders evolved from a straight-ahead jazz combo into a pioneering jazz-rock fusion group, with a definite soul music influence. Felder worked with the original group for over thirty years, and continued to work in its later versions, which often featured other founding members.
Felder also worked as a West coast studio musician, mostly playing electric bass, for various soul and R&B musicians, and was one of the in-house bass players for Motown Records, when the record label opened up operations in Los Angeles, California, in the early 1970s. He played on recordings by the Jackson 5 such as "I Want You Back" and "The Love You Save," for Marvin Gaye and Grant Green. He also played bass for soft rock groups like America and Seals and Crofts. Also of note was his contribution to the John Cale album, Paris 1919, and Billy Joel's Piano Man and Streetlife Serenade albums. He was one of three bass players on Randy Newman's Sail Away (1972) and Joan Baez Diamonds & Rust. Felder also anchored albums from Joni Mitchell and Michael Franks.
His solo album, Secrets, which prominently featured Bobby Womack on vocals, reached No. 77 in the UK Albums Chart in 1985. This album featured the minor hit, "(No Matter How High I Get) I'll Still be Looking Up to You", sung by Womack and Alltrinna Grayson.