Hefner was known for leading a life of excess, hosting countless celebrity parties and wild orgies every chance he could get.
But while many simply took the porn magnate as a bit of a character - an eccentric millionaire who was often seen wearing a red robe and smoking a pipe - the truth was reportedly far darker.
Holly Madison lived in the mansion between 2001 and 2008, before she ended her relationship with Hef.
Speaking in a new documentary about the sinister side of the Playboy mansion, the 42-year-old opened up about her 'isolating' experience.
She said: "The reason I think the mansion was very cult-like, looking back on it, is because we were all kind of gaslit and expected to think of Hef as, like, this really good guy.
"You started to feel like, 'Oh, he’s not what they say in the media — he’s just a nice man'."
She added: "Another thing that reminds me of a cult is how it was so easy to get isolated from the outside world there.
"You had a 9.00pm curfew, you were encouraged to not have friends over. You weren’t really allowed to leave unless it was, like, a family holiday."
Madison also said that Hefner forced her to quit her job as a waitress.
"He said it made him jealous, and he would appreciate it if I quit my job," she explained. "So, instead, we were given $1,000 a week as an allowance."
The revelations are made in upcoming documentary series Secrets of Playboy, which promises to lift the lid on Hefner and the Playboy mansion.
The series investigates reports of drug use, sexual abuse and even bestiality that allegedly took place at the world-famous property and other Playboy venues.
It will also contain exclusive interviews from key insiders, including Hefner’s ex-girlfriends Madison and Sondra Theodore.
Theodore, now 65, recalled orgies at the mansion five nights a week, which became ‘protocol’.
She also witnessed first-hand how his sexual demands would turn nasty, and admitted he ‘scared me at the end... you couldn't satisfy him. He wanted more and more and more’.
"The group sex was at least five nights a week," she added.
"They had a protocol. He liked to direct and you didn't segue away from it because you could tell it irritated him."
Theodore also said Hefner was a prolific drug user who would 'send her out to buy cocaine numerous times', and the sex 'broke me like you'd break a horse'.
Secrets of Playboy will premiere on A&E on 24 January
Featured Image Credit: Alamy
- Hugh Hefner's Ex-Employee Says Drug Use At Playboy Mansion Was 'Necessary Evil'
- Hugh Hefner's Ex Says Dog Became Cocaine Addict In Playboy Mansion
- Holly Madison Says Hugh Hefner Took Unsolicited Photos Of Playmates To Control Them
- Former Playboy bunnies reveal the dark side of having unprotected orgies with Hugh Hefner