Michael Richards remembers learning he was the product of a sexual assault

Michael Richards speaks openly about his life in his memoirs Inputs and outputs, is out June 4, and he doesn’t sugarcoat any of the trauma – including the discovery that he was lied to about his father’s identity his entire childhood.

The His field The star’s mother always claimed that his father had died in the war, but as he grew older he realized that this did not add up. She told him that his father had died in a car accident.

Eventually he tried to track down a man he thought might be his father. He urged her to tell him the truth, and she confessed that his conception was the result of her sexual assault.

Richard’s mother wanted an abortion, but it was illegal, so she initially gave him up for adoption. However, she later changed her mind and raised him as a single mother. The surname Richards was made up and the actor, now 75, was devastated.

Richards says the truth hit him hard and left him feeling unlovable his entire life. He also says he carried anger inside him for a long time.

“I had to accept that I was unwanted and that my mother wanted to get rid of me,” he says.

“This is definitely something I’ve had to explore over the years to figure out how my anger stems from a feeling of inferiority. I have a hot temper, and it stems from not being unwanted, not being accepted, not being understood, not being good enough to be liked or even loved,” Richards continues.

The comedian says his insecurity has led him to turn down opportunities. “I turned down the offer of a star on the Hollywood Walk of Fame. I felt like I didn’t deserve it,” he says. “I turned down Saturday Night Live twice because I didn’t feel good enough.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

This feeling of not being good enough continued when he played Kramer in His field.

“I felt every week during His field I was never really satisfied with my performance and thought I could always do better. And the higher the stakes, the harder the pressure to always be good became. And accepting myself, even when my audience loved me and the awards, recognition and offers were pouring in – that felt almost impossible at times, far too overwhelming,” he says.

“I thought, ‘I don’t like myself as much as they like me. They wouldn’t like me if they knew the real me, the person behind the character they’re laughing at,'” Richards continues.

Inputs and outputs hits stores June 4th and can be pre-ordered now wherever books are sold.

To read more about Michael Richards, pick up the latest issue of PEOPLE on newsstands now.

If you or someone you know has been sexually assaulted, please contact the National Sexual Assault Hotline at 1-800-656-HOPE (4673) or go to Rainn.org.