Re: Does type checking for all context property sets. (issue4654090)

[Carl Sorensen]

On 7/8/11 2:48 AM, "address@hidden" <address@hidden> wrote:

> On Jul 7, 2011, at 6:38 PM, Han-Wen Nienhuys wrote:
> 
>> On Wed, Jul 6, 2011 at 4:19 AM,  <address@hidden> wrote:
>>> Reviewers: ,
>>> 
>>> Message:
>>> I tried this with my local branch and I noticed no slow-down (I'm sure
>>> there is one, but it certainly is not prohibitive).  It gets rid of any
>>> potential segfaults from bad property sets in the layout block.
>> 
>> Can you do a real measurement ? Have the code print out timestamps at
>> start & end of translation, and run a 10 times with and without.
>> 
> 
> Note that there is an outlier in the no context property checking.  I've shown
> means with and without the largest/smallest dropped.
> 
> Time of translation (just before the call to lilypond-main and start of
> interpretation and just before the message "Interpreting..."), 10 runs with
> context property checking (in microseconds)
> [261539, 261617, 261597, 261030, 265018, 260641, 282217, 262489, 262616,
> 265436]
> mean: 264420.0
> mean w/ outliers dropped: 262667.75
> 
> Time of translation, 10 runs with no context property checking (in
> microseconds)
> [258837, 259777, 261231, 261915, 261306, 259702, 260661, 260903, 262827,
> 317991]
> mean: 266515.0
> mean w/ outliers dropped: 261040.25

According to my statistics, there are no outliers (i.e., no points that are
more than 3 standard deviations away from the mean.

Whether one eliminates the "outliers", or whether one does not, the
conclusion is the same.  There is no statistically significant difference in
translation time between the two versions. (The version with context
checking has a slightly smaller mean in this data set, but the difference is
not even close to being statistically significant  -- 1/3 of the smaller
standard deviation, 1/9 of the larger).

The statistical conclusion has to be "no slow-down".

Thanks,

Carl