The TV presenter, 40, regularly discusses hot-button issues on This Morning, which she has co-hosted with Phillip Schofield since 2009.
However, in an extract from her new book Reflections, seen by The Sun, Holly admitted drawing criticism for her beliefs sometimes isn't worth it.
Candid: Holly Willoughby has revealed she sometimes waters down her personal opinions to avoid controversy and cancel culture
She wrote: 'To fit in with other people's expectations of me, I have ended up simplifying myself and swallowing my voice so that I stay in the pigeonhole I'm expected to sit in.
'This can mean watering myself down. The moment you have an opinion or come down hard on one side of a debate or the other, in somebody's eyes it will be seen as controversial or wrong.
'I err on the side of caution because I want to go home and be a mum and focus on other things in my life.'
Holly added that to go out on a limb and say what you think can be 'scary' so it's easier to use other people's opinions.
Career: The TV presenter, 40, regularly discusses hot-button issues on This Morning, which she has co-hosted with Phillip Schofield since 2009
However, the presenter said every time you do that, you end up not knowing what you want anymore.
It comes after Holly joked her 't**s are around [her] knees' in a candid chat on Fearne Cotton's Happy Place podcast.
She said: 'We are heading towards that kind of menopausal time, skin is changing, hair is changing, body's changing - even having had a baby the shape of your body completely changes.
'I think that it's really important in those moment rather than look down and go, 'Oh my god, look at everything changing, I'm getting old, I'm getting more wrinkled. Oh my God, my t**s are around my knees.'
Opinions: However, in an extract from her new book Reflections, Holly admitted drawing criticism for her beliefs sometimes isn't worth it
'Instead of looking down, you've sort of got to look up with it and go, 'God, my body was able to carry a baby and my boobs