Fantasy Records was founded by brothers Max and Sol Weiss in 1949 in San Francisco. They had previously operated a record-pressing plant before forming the Fantasy label. One of the label's first signings was Dave Brubeck, and soon they were shipping 40,000 to 50,000 copies of Brubeck recordings a quarter, making large profits for the company.
Brubeck reportedly encouraged the Weiss brothers to sign other contemporary jazz performers, including Gerry Mulligan, Chet Baker, and Red Norvo, but he himself left the label to join Columbia Records. Fantasy's first subsidiary label, formed in 1951, was Galaxy Records, named in honor of a contemporary fantasy and science fiction magazine. Fantasy was for many years the largest independent jazz record label in the world. A second sub-label, Scorpio Records, was started in the mid-1960s.
American film producer Saul Zaentz joined Fantasy in the mid-1950s, and later became instrumental in forming the Fantasy empire of labels. Zaentz purchased the label from the Weiss brothers in 1967, and soon after signed Creedence Clearwater Revival (CCR), fronted by former Fantasy warehouseman John Fogerty. The band became Fantasy's most successful act.
Zaentz used the CCR profits to build an empire of mostly jazz labels from the early 1970s on. He had picked up Debut Records as a wedding present from bassist Charles Mingus to which were added Good Time Records, Prestige Records (in 1971), Riverside Records and Milestone Records (both 1972), Stax Records (1977), Contemporary Records (1984), Specialty Records (1991), Pablo Records (1987), Takoma Records and Kicking Mule Records (both 1995).
In 2004, Fantasy was sold to a consortium led by Norman Lear and merged with Concord Records to create a new company called Concord Music Group. While some operations are still located in Berkeley, the record label is now headquartered at the Concord Music Group location in Beverly Hills, California. Saul Zaentz died in 2014.