If you're thinking of something that offends nobody, and which people feel better after hearing, you're most of the way there.
Everyone laughing is quite an ambition, and I'm not sure it's possible. The trick is that if your humour has a target, that target will be the first to laugh - laughing with someone is always funnier than laughing at them. Displaying genuine affection for the object of any jokes (even if the affection isn't part of the joke itself) will let you get away with a lot more than otherwise and the humour won't have to be innocent or soft. Think of Waldorf and Statler in "The Muppet Show" - downright abusive some of the time, but they're there every week.
Also, the humour shouldn't only flow one way. In "Cheers" when Norm and Cliff would joke with Woody (again, genuine affection there), it was always funniest when he ended up looking smarter than them.