Technically, the name "Star Trek" is not and cannot be copyrighted. It is "trademarked", which legally is an entirely different thing.
A "copyright" protects the exact words or images used in a published work. You cannot take a copy of, say, "Lord of the Rings", put your own name on as the author, and publish it. But you certainly can write your own fantasy novel that others might think resembles Lord of the Rings. You can also include quotes from "Lord of the Rings" in your own work under the "fair use doctrine", but that's getting off in a different direction from your question.
A "trademark" protects a name or title. It's completely different from a copyright. Copyright law specifically says that you can't claim a copyright to a name, phrase, or slogan, etc. But you can claim a trademark.
Usual disclaimer: I am not a lawyer. But in general, if you make up a word that never existed before, you have a very strong claim of trademark. If you use existing words, your claim is weak. If those words are purely descriptive -- an example I recall reading in a legal discussion was "Main Street Grocery Store" -- your claim is VERY weak. If they're used in a different context from the literal meaning, your claim is stronger. Like "Apple Computers" has a decent claim because their computers are not particularly related to literal apples.
So for example, if you wrote a story where the characters use a "frambar drive" to travel between planets -- "frambar" being a word that I just made up -- and someone else wrote a story where he said they used a frambar drive, you'd have a strong trademark claim. But if you said they used a "star drive", your claim would be pitifully weak, because the words "star" and "drive" have been around for hundreds of years. You could make a claim to trademark if you were the first to using those two particular words together, but it would still be a weak claim.
A trademark can be registered with the government, but it doesn't have to be. You can take someone to court for stealing your unregistered trademark. The value of registering is that it is evidence that you really were the first to use the mark. Without a registration, you have to get into arguments in court about who used it first.
So specifically: I haven't checked, but I would be surprised if the people who produce Star Trek have not registered that name as a trademark. When they first made the show their claim would have been weak because "star" and "trek" were existing words being used with their common meanings, but they've built up such a brand association over the years that I'm sure a court would respect that.
Terms like "hyperdrive" and "warp drive" are used so commonly in science fiction that anyone pressing a trademark claim would have a tough battle. I wouldn't hesitate a moment to use those.
A word like "Vulcan" might be tricky. The word "Vulcan" is the name of a Roman god, certainly not invented by Gene Roddenberry. The name was used in astronomy for a planet that was speculated to be inside the orbit of Mercury, so the Star Trek people can't even claim first use for name of a planet. I think they'd have a hard time winning a lawsuit over someone else using that for the name of a planet. But if they brought such a suit, I'm guessing they have a lot more money to spend on lawyers than you do, so is it worth fighting?
In general, I wouldn't use someone else's made-up names. I wouldn't write a story with Klingons or Mimbari, etc. Common phrases like "star ship", "hyperdrive", etc, you can use freely. I've never heard of someone successfully suing over use of words like "transporter", that would be a hard fight to win but maybe in context.
And all of THAT said, I heartily agree with What that you are far better off to write an original story than to devote a lot of effort to seeing how much you can get away with stealing from someone else.