...or the unbiased approach rooted in probability theory.
I am proposing this answer for the following reasons:
- the OP seems to have some reserves in considering experience as a valuable weighting function.
- I want to address the case in which most of the target readers are of the uninformed / unexperienced type
Let's consider readability to be a quantity r, such that given a population of individual readers we can define: i) the average readability R and ii) its standard deviation S.
You can now ask a certain number n of random people, selected without bias from the population of interest, to read your text and give their estimate of readability.
The central limit theorem says that the average of these estimates converges to the mean readability for the population as n increases.
It also says that the convergence rate is . That is, if asking one person only yields a standard deviation (think of as variability in the response) of S, asking n people and averaging their response yields a much lower standard deviation .
Best results if you ask people to rate your readability with a number on a fixed scale, e.g. from 0 to 10.
In just a few reads you could get a decently accurate estimate of what the larger population of readers would say about the readability of your text. And you could ask just anyone.
For instance, asking 4 people, the average estimate has only half the standard deviation compared to asking just one person. If you can ask 8 people, you have reduced the error in the estimated readability to about 30% of asking just one person.