Sunday, April 27, 2008

Stats and other schtuff

Hmmn. Spent the last day reading up on stats - great way to pass the time :blink:

In any experiment there are 2 possible types of error: systematic and random.

Systematic:

Ergomo isn't factory calibrated properly
Offset is incorrect for temperature
k- factor in Ergomo is incorrect

Systematic errors affect all samples collected.

Random:

This run had more/less wind than that run
Goofy couldn't hold his aero position in this run or that run
Height up or down the bank was not constant
Too close behind another cyclist on the track and run was draft affected
Temperature varies between trials
Left/Right leg imbalance changes total reported watts

Random errors affect some samples collected.

Because the systematic errors affect everything the same way, we can probably discard them if we are only looking to show a significant difference between 1 helmet and another. In this case systematic errors might be important if we really want to know the difference at [i]exactly 200 real watts[/i], not what the test Ergomo unit reports as 200 watts.

Random errors are much more sinister. Maybe the difference between 2 helmets is due to solely random error, and not because 1 is more aero than another. This we don't want.

Still thinking ...

Friday, April 25, 2008

Aero testing helmets - comments

whodesigns:

Hi Fishboy,

Interesting thread, I have done similar stuff in the past and also spent time in wind tunnels. There are a lot of uncontrolled variables in outdoor testing, hence the increased use of wind tunnels.

I have a few questions:

How do you verify the calibration of the Ergomo, in my experience they can be a bit random?

How are you sampling the data?

What is your calculated total error for each run?
Aero testing is very individual. It has been my experience that a helmet may be fast on one person but slower on another.


Fishboy:

Yep Ergomos can be a bit random, and in fact there are studies (at the Saris web site) that confirm this; there is also no verifiable external test you can perform like on an SRM to ensure accuracy.
The BB unit in question is new and was properly installed to spec. It performs accurately in high climb tests predicting speed vs. weight on certain grades.
Every test I've done allows offset calibration to take place after stabilisation of temperature - all the above tests were conducted with the same offset value. So yes, it is possible that the measurement instrument is not as accurate as it could be, but short of heading off to the tunnel, it should be capable of producing some results. In fact this was one reason for trying these tests - could you actually see a difference between certain types of equipment/position without having to resort to a tunnel.

The data is sampled by looking for sections of constant speed and velocity, where the initial and final velocities are the same. Because it is on a velodrome, the data chosen must be more than 400 metres in length to ensure that at least one full lap of the 'drome is completed. This ensures that if there is some slight wind all angles into and away from the wind are covered (all tests so far conducted with no wind). Average power and speed are recorded and plotted.

By total error do you mean TE = Bias + Z*SD? And what value would you recommend for Z?

And yes, aero [i]can [/i]be highly variable. What works for one person might not work for another. I've stated that a few times through this discussion. But in general terms it is still applicable. The fact that the test data shows an improvement of 0.5 to 1.0 kmh at 200 watts by using an aero helmet is an important finding that people without access to their own tunnel and R&D staff would be interested to know. I've achieved that same result 4 or 5 times now over the course of the last year with testing, and results backed up with racing, so I'm pretty comfortable that this is on the money. And if the choice is aero wheels at $2500-3500 or a helmet for $200 that achieves the same benefit, again, I think people would be interested to know.

There is so much crap that goes on with aero testing manufacturers claiming this much time with this and this much time with that. I'm convinced that most manufacturers don't actually have that much idea what is actually going on (some notable exceptions), but the reality is that saying something is 'tunnel tested' is good for biz. For every tiny bit of data I've collected from every wind tunnel test that I've been able to find, the test result has often depended on who was running the test. As Cervelo says: "We're happy to finish second in tunnel tests paid for by someone else."

whodesigns:

This is so true, you know what, the really interesting data never gets published, the best stuff I have seen in the tunnel will never get published.

Don't even get me started on CFD!!!!

By total error I mean the sum of all the errors, often this is bigger than the difference you are trying to measure so it becomes uncertain if you are really seeing a trend or not. Repeat trials can help, as can use of a control.

As I said it gets really hard to control the variables especially outdoors, so apart from the Ergomo, there is frontal area, rolling resistance, ambient conditions etc etc.

Congrats on the time and effort you have spent on this.

Tuesday, April 22, 2008

Aero testing #3 - results

[quote=fishboy,Apr 24 2008, 01:02 PM]Test number three this morning. Limar Chrono 05 v Giro Advantage 2 with taped vents. Subjectively, don't think taping has helped improve the situation, but need to run the data.
[right][snapback]392346[/snapback][/right]
[/quote]
Better get to the LHS (local hardware shop) and get some duct tape.

Turns out that with the vents taped the Giro Advantage 2 performs exactly the same as the Limar. Kind of surprised me - how can 5 pissy little vents on the front of the helmet result in 0.9-1.0 kmh slower speed at 200 watts??? :blink: Should repeat this test with vents taped and de-taped to make sure.

So the first rule of aero helmets would seem to be: [quote]Buy aero helmets without vents in preference to ones with vents.*[/quote]
And the second is: [quote]If you have vents on an aero helmet, tape them.[/quote]

* Sure, if you think you're going to have a hot head, get one with vents, and ride that bit slower.

Here's the data:

[img]http://www.efirst.com.au/rob/fishboy/TapedGirovLimar.gif[/img]

Also interesting to note that this test with done with different clothing to last time - looser (but warmer) jacket which is not skin tight. This jacket is responsible for dropping the speed at 200 watts from:

37.1 and 38.1 kmh for Advantage 2 and Chrono (test 1)
37.1 and 37.9 kmh for Advantage 2 and Chrono (test 2 - same clothes as test 1, skin tight)

to

36.0 kmh for both helmets (test 3) - looser jacket, non skin tight

Aero testing #2 - similar results to earlier test

[quote=fishboy,Apr 21 2008, 09:28 PM]
Here it is folks:
[img]http://www.efirst.com.au/rob/fishboy/helmetR1-R4.gif[/img]
[right][snapback]391319[/snapback][/right]
[/quote]

I've just realised that the original test done between the Chrono and the Advantage is almost the same results as the second test.

The first test had average kmh speeds at 200 watts of: 37.1 and 38.1 for Advantage and Chrono
Second test at 200 watts: 37.1 and 37.9 kmh for Advantage and Chrono

So even though first trial used a disc and H3, and the second 36 spoked training wheels, the magnitude of the difference between the helmets was almost identical.

This is very good from a repeatability point of view, not so good from an aero wheel point of view - why spend $3500 on wheels if a $200 lid does the same job!!!!

Monday, April 21, 2008

Aero testing #2 - final results

I've been riding in the wrong helmet most of the season... :(

At least that is what the data says.

The simple assumption that new = better isn't necessarily so.

Here it is folks:


Orange and pink are the Specialised Decibel (least aero/non aero helmet).
Next best is the red circle of the Giro Advantage 2 (more aero).
Best is the Limar Chrono 05 in green (most aero).

I did have 2 outlying data points over 46 kmh, but I omitted them because it added more noise - probably cause power delivery at those higher speeds is a bit hit and miss (it's hard to control the application of 430 watts in a controlled manner - at least for me! - and hard to sustain that power for a long enough period that the speed stabilises.

So time savings; how much:

200 watts

KM......Decibel...Advantage2...Chrono
Av KMH.....36.6......37.2......37.9

time on course (hh:mm:ss)

KM......Decibel...Advantage2...Chrono
20......0:32:47...0:32:15...0:31:40
40......1:05:34...1:04:31...1:03:19
90......2:27:32...2:25:10...2:22:29
180.....4:55:05...4:50:19...4:44:58

savings (hh:mm:ss)

KM......Decibel...Advantage2...Chrono
20......0:00:00...0:00:32...0:01:07
40......0:00:00...0:01:03...0:02:15
90......0:00:00...0:02:23...0:05:04
180.....0:00:00...0:04:46...0:10:07
Seems amazing that the choice on an aero lid can save you over a minute in a Sprint race, and more than 10 in an Ironman...

Remember this is based on 200 watts - if you are faster than this, it is even more important to have an aero lid!

Grain of salt time:

Are the results accurate? You tell me. Carefully designed experiment - careful test protocol, careful evaluation of results. As accurate as I can get.

Repeatable? Possibly. The whole experiment needs to be done again, and maybe a third time before we can see that it is repeatable and therefore 'admissable as evidence'.

Believable? Why is the Giro slower than the Chrono? Well the wind tunnel eyeball test confirms:

- The Giro has front vents, the Limar has none
- The Giro is taller and wider at the front than the Limar

both these factors should hurt the Giro

but

the Giro has a longer tail, and side burn covers that should make it lower drag than the Limar.

Would you get the same time savings if you used these helmets?

Possibly. This data is based on a 188cm dude on a P3C in a pretty aero position. If you aren't like that then the magnitude of the savings might be different. In general terms you are likely to have lower drag and faster times with the same helmets as shown above.

Improvements:

- More helmets
- Indoor venue (although wind less than 10kmh doesn't seem to have affected control runs to a measurable extent, particularly in the important business zone of 36 to 41kmh)
- Wind tunnel comparison to validate??

Any other suggestions?

fishboy ;)

Sunday, April 20, 2008

Aero testing #2 - results

Ok Got the control runs analysed.

And its good - both separate runs with the non aero helmet are very close together!
So the concern about the extra wind in the second trial seems not to be a problem.

Here's the graph:
[img]http://www.efirst.com.au/rob/fishboy/SpecialisedDecibel.gif[/img]

Certainly the area between 32 and 41 kmh is very similar from both trials.

Will get to analysing the other two aero helmets soon, but probably not today.
The process is a bit tedious and if I'm gunna do more of this I think I might write a program to do the analysis. 5am swim training .... :unsure: :sleepy:

Saturday, April 19, 2008

Aero testing #2 - results

OK, Second test completed this morning. No results yet - will try to get online today.

Managed to get a full run of speeds for:

1. Specialised Decibel (non aero)
2. Limar Chrono 05 (aero)
3. Giro Advantage 2 (aero)
4. Specialised Decibel (non aero)

Unfortunately there was a bit more wind at the end of the test, will see if that affects the data. Dead calm at the start. Runs 1 and 4 should be very close together on the graph. If not ... maybe indoor velodrome time ;) . Also used spoked training wheels instead of disc/H3 to try to prevent any wind on that gear lowering CdA.

Subjectively, at speeds over 40 the two aero helmets felt significantly easier to maintain speed than the non aero lid. Max speed was about 50kmh.

All the other parts of the test were OK - constant Ergomo offset at beginning and end, consistent use of the selected speeds, consistent riding position (head up looking 15 m in front of the bike, still shoulders held in same position, knees just brushing the top tube all the time, same seat position, stayed away from drafting other track riders).

Thursday, April 17, 2008

Aero testing #2 - results

Here's the test data (piccy): Still working on the real results (time consuming :( )

Aero testing #2

Tomorrow mornings weather is looking OK (calm, but probably cold again).

I think I'll try another test to try to improve on the first set of results (since I'm not sure I believe them...)

[b]Improvements:[/b]

[b]3 helmets: [/b]
Specialised Decibel (Non aero)
[img]http://www.claremontcycle.com/photos/helmets/07HelmetDecibel.jpg[/img]

Limar Chrono 05 (aero)
[img]http://www.tfn.uk.com/acatalog/Limar%20Chrono%2005%20Aero%20Helmetmed.jpg[/img]

Giro Advantage 2 (aero)
[img]http://www.worldcycling.com/graphics/00000002/GIRHAD08-ti.jpg[/img]

[b]Test protocol:[/b]

Ergomo offset 30 min after bike is in ambient

Speeds:
24
28
32
36
40
44
max!

Run order (to save a bit of time, recovery on slower speeds, cause you need to try to keep power delivery really even). This time the speeds will be taped to the bars to make sure the X-axis data points are the same for each helmet.

32
36
40
28
44
24
max!

(so that's a run of >60 sec for each speed for each helmet, 21 runs total)

Ergomo offset afterwards (hopefully same offset).

If it turns out that these two trials are OK, and we can see real differences, then it is worth taking the trouble to collect a set of helmets together and do a more extensive test.


BTW both Otter and Jester PM'd me - obviously keen to add further performance enhancement to each arsenal at their disposal. :lol:

Wednesday, April 16, 2008

Aero testing #1 - results

[b]Results:[/b]

Holy Cow Batman - we do have some results. But before we get all excited, maybe we need to RETEST, just to see if the results stays the same (ie is the experiment is repeatable). Wouldn't hurt to tighten the test protocol so the numbers are the same eg. 26, 32, 36, 38, 40, 42, 44, max.

Here is the graph of the data:
[img]http://www.efirst.com.au/rob/fishboy/limar%20v%20giro.gif[/img]

Shows about a 12 watt reduction in drag at 40 kmh by using pink helmet, compared to orange helmet. And guess which helmet takes an additional 6-10 seconds in transition (ie get on your head)?

Now who wants to hazard a guess as to which is Limar Chrono 05 and which is Giro Advantage 2?

I'll post the answer tomorrow to allow some guessing time.

Aero testing #1

Protocol:

Location: Carnegie Velodrome
Date/Time: 16-Apr-2008 0645
Weather: Clear, calm, 8 deg C

Weather Moorabbin airport:

Time................Deg...C.Dir...KMH....KMHGust...hPa
07:30am..........6.8.........N......7...........7.........1029.2
07:00am..........5.8.........N......4...........6.........1028.8
06:30am..........4.9.........N......4...........6.........1028.4
06:00am..........5.9.........N......7...........9.........1028.1

Bike + rider weight: 91.7 +/- 0.5kg

Ergomo Power meter
Stabilised at ambient temperature for 25 min before offset
Offset calibrated before test: 983 offset value
Offset calibrated after test: 983 offset value (no change)

Description:

Ride at constant power and speed for 2-3 laps of the velodrome
Choose range of speeds from 25 to max capable KMH eg.
25, 28, 30, 34, 38, 40, 42, 44, max.

Ride on blue line of track (ie same distance up the bank) to prevent changes in potential energy.

Change aero equipment and repeat test using same protocol above.

Download power file, isolate sections of speed with same initial and final speed > 60 seconds in duration, calculate average watts in this section of data.

Plot data using fancy spreadsheet (Coming soon).

Results:

What? You want results? Just kidding - working on these right now and will post when completed (soon).