Rockie wrote: ↑Thu Aug 16, 2018 11:49 am
B208 wrote: ↑Thu Aug 16, 2018 9:55 am
I have written rubrics that evaluate both motor and cognitive skills in great detail.
Engineer right?
Nope. You're thinking of Headley, (who extends a cordial invitation to come over to ..'s for a chat).
Rockie wrote: ↑Thu Aug 16, 2018 11:49 am
Even in a controlled simulator environment no two exercises are the same.
By this I assume that you mean no two instances of the
same exercise are the same. The only thing that changes between iterations of a simulator ex are the pilot, (either a different pilot or a pilot that has hopefully learned something from the previous iteration). Any changes in the outcome of the ex in a simulator are due to the crew. This is fortunate given that what we want to measure is the pilot's ability. Detailed, well defined standards and rubrics will provide accurate discrimination with regard to just how good the pilot is.
Rockie wrote: ↑Thu Aug 16, 2018 11:49 am The only way to bridge that gap is to use subjective judgement, which circles us back to the word "good".
No. The first step in evaluating performance is to write a detailed description of ideal performance, and then to write detailed descriptions of increasingly less ideal performance. The number of levels of performance you want to describe is up to you. Once you have written all of these detailed descriptions you place them in a ranked table called a rubric. You compare the observed performance to the described performance in the rubric. The more detailed the rubric, the less subjective the evaluation is.
To put this into terms you can understand, the current flight training evaluation rubric is not designed to differentiate how 'good' someone is. It is essentially designed to differentiate between inadequate, adequate, good and perfect. Don't misunderstand me; I'm not knocking the current rubric. It does the job for which it was intended, determining if someone is safe to have a license.
Rockie wrote: ↑Thu Aug 16, 2018 11:49 amIs "good" a level 3? What if a guy gets a 3 in difficult circumstances but the next guy gets a 4 in ideal circumstances? .
That's easy. If you want to determine how 'good' a pilot is have them perform the same exercise under increasingly difficult circumstances. The further they get before they fail is a direct measure of how good they are. As has already been pointed out, the current flight training rubric is not set up to do this, nor does it need to be set up to do this.
Rockie wrote: ↑Thu Aug 16, 2018 11:49 am
The best pilots I've ever seen mixed good judgement in all circumstances with hand flying skills commensurate with the job. Lack either and you shouldn't be flying airplanes. There are people who boast about being the best stick (and maybe they are), but I would never get in an airplane with them because they utterly lack judgement. Conversely great judgement won't help you if you can't keep the airplane going where it's supposed to.
Well done. You finally arrived at the position where I started. This proves that you are, in fact, trainable.