Boy George

has insisted

Madonna

is "too full of herself."

The Culture Club legend — who insisted "every gay man loves Madonna and there's no one gayer than me" — has reflected on several less than positive meetings with the

Queen of Pop

over the years, insisting her unfavorable memories of the first time they crossed paths couldn't have happened like she recalled.

Writing in his memoir

Karma: My Autobiography

, he said while Madonna remembers him "head to toe in Westwood", he was exclusively wearing Sue Clowes at the time.

Reflecting on a later meeting in New York City, George continued: "Madonna arrived with Sean Penn and pretended she didn't see me. Madonna is everything it says on the tin but she adds new ingredients every day. I know for a fact

she's too full of herself

to even mention me.

"She once said, 'Boy George was mean to me in the eighties and he's still mean.'

To be fair, I haven't really been given the opportunity."

The 62-year-old

Karma Chameleon

hitmaker insisted he is still a fan of her work, while he isn't so certain she shares his "sense of humor."

He said: "Not knowing anything about her I would assume she'd have issue with the word 'some'... I have a sense of humor, I don't think Madonna does, despite some of the clothes she wears (LOL )."

George admitted he loves "all the drama" and "pop star behavior", hailing her brand as "genius".

However, he wrote: "But outside of that, you know, it's how you act. You know, because you want, as I say in the book, you want your artists to be weirdly beautiful and strange and all that, but you also "I want them to be kind of ordinary."

Despite their apparent feud over the years, George

thinks they could become pals one day

.

He said: "Like Bette and Joan, we should have been friends. There's still time."