With over 7 billion people in the world, even if you have 1 million fans, the odds are 7,000 to 1 that a listener won't enjoy your music.
If you hear a song and don't like it, it doesn't mean that it's bad. It means you're not the right audience for the song.
Focus on finding the right audience for your song and let the haters hate. Practice the Golden Rule of Songs:
Treat other songs as you'd want your song to be treated.