Some of my fiction sci fi characters are ESP sensitive and the term is used a lot. Capitalized ESP stands out in my sentences like yelling so I am using esp,lower case. Is that acceptable? Will a potential publisher object filing my novel in the trash?
ESP stands for "Extrasensory Perception". It is an abbreviation. The correct way to write it is therefore in All Caps: ESP, similar to how one writes NASA, USA and DNA.
A potential publisher wouldn't trash your novel for a capitalisation error, but it is quite possible they wouldn't understand what you mean by 'esp', similar to how one might look at a misspelt word, and be unable to figure out what it was supposed to be.
As stated in the other answer, ESP should be in all caps. One way around this problem is to come up with a term that people would use in every day usage to refer to someone with this ability. This happens all the time in science fiction and fantasy (and, it should be said, in real life). Alfred Bester, Isaac Asimov, and Robert A. Heinlein used to refer to Espers for people with ESP powers, and I think Phillip K. Dick used the term "teeps" for people with telepathic powers in "The Hood Maker" ... there are many examples to draw from.
If it is referred to as "ESP" specifically, it should be capitalized. Reading "esp" lowercase feels like reading a shortened "especially" in a text message. But you might avoid an overabundance of ESPs by allowing the narrator and/or characters to call it something else, especially if there are many of them that have it. They might have "powers," a "sixth sense," "radar," or use another story-specific noun/verb like "esper" (pronounced ess-purr) as mentioned in other answers, to refer to ESP or those with ESP in the story. You could even use the context of your sci-fi world to avoid "ESP"... if there's an invented language, name it in that, translate it once, and use your name from then on. Or, the history of a discovery or technology that explained or enabled the individuals with ESP might be discussed, and from its name derived another for the phenomenon itself.