It really depends on your story. Sometimes a conlang (constructed language) will look silly, but sometimes it's almost necessary.
For instance, I'm working on a story in which alternate dimensions are a very real thing, and Earth (which once was a great dimension to trade with for metal, something that's scarce in most other dimensions) has become an obscure topic since most of its mage traders died in a magical collapse. Since English, in these worlds, has achieved about the same status as, say, Latin and Greek have in ours: once prevalent, but now only spoken by a select few--in this case, scholars and mages.
With this setup, it would be ridiculous if there wasn't a language to write in its place occasionally. A few of my main cast speak it, so that's not so much a problem. But it would have been absurd if scholars and mages, the few among the many, were the only people they ran into or had to talk to.
Also, it gives something to the world. You can have a lot of fun with a conlang. Mine sounds kind of like German, and is very rhythmic: most of the words are three syllables, with the stress on the second. Can we say fun to read aloud?
It's a lot of work, though. You have to decide how far you want to go--if you just want enough for a few characters to speak occasionally, then just do that. Don't make a whole language if you don't need to.
If you decide you want to, make sure you've read Limyaael's stuff. There are some rookie mistakes that you want to avoid, because anyone who's ever taken a language class can spot them immediately.
http://www.livejournal.com/users/limyaael/459655.html
http://www.livejournal.com/users/limyaael/460892.html
http://www.livejournal.com/users/limyaael/462195.html
http://www.livejournal.com/users/limyaael/463028.html