When we read about the reality of a constructed universe we very quickly adopt the concepts and ideas that that reality contains.
You don't need to explicitly spell out a size by numbers, and unless you are talking about some higher authority type of scenario then it would strike me as being unrealistic that equals would refer to each other by number.
If numbers are simply a label to identify different sizes, then a name for each would be much more natural, and feel less degrading to the reader.
It sounds like it would be similar to how we see and discuss race, and I would imagine that refering to different races by numbers would be confusing (my mind would constantly be saying 'now, was 5 Chinese or Cherokee' whereas a a name is more likely to stick.
make what you write easy for your reader to follow and understand, Whether you call them gigantor. asdfa, or bob the reader will quickly understand their place in your world if you explain it to them.
Finally though, I'd say write your story, if it makes sense in your head to use numbers, then go with that! Listen to critiques when you think its something that you actually want to change. Get it written, then figure out what to do with it from there. Ideas such as that can be good things to tackle in the first rewrite when you've found out whether it works or not.