I have never listened to anyone who criticized my taste in space travel, sideshows or gorillas. When this occurs, I pack up my dinosaurs and leave the room.
--Ray Bradbury, Zen in the Art of Writing: Essays on Creativity
The phrase "dream zappers" didn't exist back when Ray Bradbury was writing this book, but clearly he understood the concept.
Get away from those who criticize you, who drag you down and distract you from your goals, who mock you for the things that interest you. Find room in your life for people who support your creative pursuits. They're out there. Keep putting your work out there and you'll find them.
In short, don't indulge negative people, get away from them. Pack up your dinosaurs and leave.