Motion in flight simulator

[louis_grenier]

http://www.cqfayul.com/com/simulator-motion.html

Flight Simulation Motion Systems:
A Pseudoscience Imposed on Air Carriers


Airlines are asked to spend billions of dollars on questionable
functions and gadgets to train their pilots in flight simulators.
There is a historic opportunity for regulators to rethink the certification
requirements of commercial pilot training programs.



Jean LaRoche, FRAeS
Director of Research & Development
CQFA Chicoutimi College of Aviation

Gavan Lintern, Ph.D.
Monash University Accident Research Centre
Melbourne, Australia


(January 27th, 2021) Despite billions spent and half a century of research, there is no proof, however small, of any benefit related to simulator motion for the safety of the travelling public. The last burden air carriers need today are useless operational costs. If it is true that life will not be the same after the pandemic, the time has come for regulators to align regulatory requirements and pilot training standards with proven scientific evidence.

http://www.cqfayul.com/com/simulator-motion.html

[Scuderia]

A plausible thesis but a hasty, spotty essay.

The charge of pseudoscience reflects on the authors, too.

[Heliian]

I guess they're not familiar with flight sim classifications?

There are plenty of stationary training aids but their realism ends with no motion.

The whole point of full motion sims was to avoid having to spend even more money on training in real aircraft.

The references in it are ancient, it's 2021, have they been to a Sim?

[ahramin]

Publish or perish. With very few exceptions, nearly all academic disciplines suffer from the need to publish papers regardless of whether or not the author has anything to say.

[digits_]

There are plenty of stationary training aids but their realism ends with no motion.

Isn't that in part because the non-motion sims aren't built up to the same standards as a full motion sim (excluding the motion software). I've always noticed quite a bit of a cockpit quality difference between the no motion sims and the full motion sims. The feedback forces in a motion sim seem more realistic than in a non-motion sim, which in itself doesn't have anything to do with motion.

If I think back to my sim visits (704 style aircraft), then I don't really disagree with the notion that the motion part is mainly a useless expense. The sims were very accurate during cruise flight, in which motion doesn't really matter. During ground ops, most motion was felt, and even the full motion sims at that point are fairly lacking. As long as you can't maintains a constant >1g force in a sim, it's doomed to be inaccurate.

The motion seems to help people to think the sim is real, because it moves, but the accuracy of the movements is still lacking in comparison to a real airplane.

[Big Pistons Forever]

I thought the purpose of motion was to make the sim instructor nauseous so he/she will end the sim session early

Seriously though IMHO vastly improved visuals have made motion less important. Traditional hydraulic motion bases are expensive and maintenance intensive, however the new generation of electric rams make the motion base simpler and less expensive.

I know there is some interesting research been done on how much motion is really needed. There is developing evidence that a high fidelity wrap around visual system with a motion base that has about 25 % of the range of travel of a conventional full flight simulator can achieve essentially identical pilot experiences

[Canoehead]

Weak research paper with questionable credibility. I'm sure the (first) author runs an excellent ACP course, but I'm unable to find any information on him except that he has written a few papers on airline-related topics. How much training/checking time in heavy airplanes and Level D hardware does he have?

Way too many studies in this paper are quoted from the days of pulley and piano wire platforms. If he went for a ride in a 7000XR series CAE sim, he may have a different idea of how good the fidelity is (although until you've flown both sim and airplane, you can't really understand it). Electric motion platforms have much lower maintenance costs and better fidelity. FFS training programs have already been reduced to bare minimums with the augmentation of CPTs/IPTs. Most of the travelling public don't realize that a (insert type here) pilot's first flight in the airplane happens on a revenue flight. That only happens because of those advances in technology, specifically visuals and motion. You think the airlines don't know the cost of flying a 787 up to YYB to do circuits? That's the tradeoff; anything less than FFS training will require burning Jet-A (and all of the cascading operational costs of flying the airplane). That's simply not an economical option.

Finally, I take exception to this statement in his article:

Furthermore, the expense associated with the deployment of high-fidelity, full-motion simulators draws resources away from other important instructional concerns. Quality of instructors can have considerable impact on training outcomes, yet little effort is put into ensuring the quality of instructors. Typically, pilots become instructors because of a combination of seniority and preference. Little work is put into ensuring that those who can become skilled teachers are selected and then trained for the job. Most significantly, despite the fact that instructors have an important role to play in teaching skills such as Crew Resource Management, studies of simulator fidelity have ignored the role of the instructor.

That's an incredibly unfounded and uneducated statement. Not saying that there aren't some questionable instructional technique and quality issues within most training departments, but to say "little effort is put in...". I refer back to my opening paragraph.

[ami6car]

All,

Good day. I am the first author.

Some of you seem to have solved in 5 minutes (granted, 5 minutes each) a problem 50 years of very serious research haven't been able to solve. Despite all their means and efforts, even simulator manufacturers admit they have never been able to come up with valid data on this issue. If I could, I would copy here the 100's messages I received from their engineers, in total agreement with the article. Some were brave enough to post their opinion in professional groups in Linkedin. If it were as clear and linear as some of you here seem to indicate, don't you think they would have published their result once and for all? Wouldn't this be worth the Nobel Prize?

Canoehead, our article is a white paper, not a research article. It is about better training, not cheaper training. It cites rock-solid work from world-class research centres, including the work of FAA's Judith Bürki-Cohen. Her final word on motion is not only recent (2011), but is what the regulators needed to begin acting on this.

And they did.

Motion for pilot training is dead. EASA will become the first regulator to remove the requirements for motion cues from all initial and recurrent Type Rating training. Other CAAs have NPA in the mill. Canada plans to issue new guidelines in December 2021 and yes yes, you will be able to say your piece. Well-managed airlines have recently purchased FTDs by the dozen and received Level-D-like training credits. Evidence-Based Training is also bringing all sorts of changes in the same line, such as V1 Cut assessments once every 3 years.

You can call me anytime to tell me you disagree, hopefully based on data, facts. If you drop by my training centre in Dorval, I'll provide you the face mask and all the coffee you can drink. I am not hiding. I certainly don't know everything in aviation and I will be thrilled to learn from you.

By the way, I like motion too. It's completely useless, but I love it.

Regards and best wishes for the hard times

Jean LaRoche, FRAeS
Montreal

[ami6car]

That's an incredibly unfounded and uneducated statement. Not saying that there aren't some questionable instructional technique and quality issues within most training departments, but to say "little effort is put in...". I refer back to my opening paragraph.

Canoehead, since you seem to know, please post here the number of training captains who received a structured Train The Trainer course, in the world.

I will stand corrected.

Thank you

Jean

[Canoehead]

All of the ones at my company have.

[ami6car]

All of the ones at my company have.

"Incredibly unfounded and uneducated" you said. Please answer the simple question, thank you.


Jean LaRoche
Montreal

[Canoehead]

I have no idea worldwide, I can only speak for my operation.

Your generalization about little effort being put in to ensure instructor quality seems to be quite blunt and rather broad. I agree on the first 10 words of the following quote, but the rest of those 3 lines certainly don't apply where I work. I suspect that in North America, most Part 121/CAR 705 training department managers and instructors would disagree with this too.

Quality of instructors can have considerable impact on training outcomes, yet little effort is put into ensuring the quality of instructors. Typically, pilots become instructors because of a combination of seniority and preference. Little work is put into ensuring that those who can become skilled teachers are selected and then trained for the job.

While an Instructor Rating isn't required to be an instructor in these environments, almost all that I know have been civilian or military instructors, bring line-training experience, or bring other training backgrounds with them to the position (for starters).

I'll have a look at some of Judith Bürki-Cohen's work sometime.

[ami6car]

I agree with you Canoehead that Part 121/CAR 705 North American carriers have worked hard to improve their training departments and I am glad that you are part of this, and that you can influence positively your fellow pilots. Around the world, safety board reports however tend to indicate it remains rare and needs to be addressed. Canada will likely address this officially as part of the current work to modernize commercial pilot training and testing. The last sentence you quote above actually comes from citations found in several accident reports with fatalities. We didn't make it up. TransAsia 235 is not the only example, as you know. I will add that this is my 31st year training experienced training pilots for initial Approved Check Pilot delegation. I actually created the ACP course training standard for TCCA in 1989. I think it qualifies as descent experience.

Last point that was intended in the paragraph you didn't like: I'm sure we will agree that instructors, not devices, teach. Yet, the role of the instructors is never mentioned in transfer studies. We feel we need less hardware, and more liveware.

Happy training

Respectfully


Jean LaRoche
Montreal

[KissPlusOne]

I'm not an airline guy, but why not go to VR simulators? I've done it once on FS2020. Even though the graphics were lower, and the screen was jumpy, it was a much more realistic experience. I even found myself anticipating the lurch as the plane comes to an abrupt stop after slamming on the brakes (I don't fly for accuracy in the game. I fly like a crazy person because I know I'm not going to die.)

Combining a motion platform with a cockpit setup and VR headset seems like the way to go. Save money on enormous screens and get a MUCH more realistic feel for it all.

[ami6car]

Sorry KissPlusOne, but "combining motion platform with..." simply doesn't yield the training benefits the industry always wished for. It was sexy for sure. But it never transferred to the actual aircraft.

[Bede]

All,

Good day. I am the first author.

Thanks for weighing in here.

Am I correct in that this is intended as an opinion essay as opposed to a peer-reviewed academic article? Nothing wrong with that as long as everyone understands the limitations.

I have no data whether or not your thesis is correct, but I think that if you want to change regulator's minds, you'll need a lot more hard data to disprove a long standing paradigm.

[ami6car]

Hi Bede,

Yes your are correct. Our paper points towards peer-reviewed studies and ideally, you could follow up with FAA's Judith Bürki-Cohen 2011 paper, which is recent and very well done (like all JBC's work). She also gives an extensive list of references. Lot's of fun ahead. The reason why our white paper lists studies as far back as the 60's and 70's is to show that the issue was well known a long, long time ago. It is always very interesting to learn how they conducted the studies.

We deliberately released this paper via internet because scientific publications don't always reach the operations. Despite the large number of studies published in the past, all over the world, several large carrier chief pilots and regulators told us last week they had never heard/read about this. By all means Bede, don't trust us _AND_ follow up with proper scientific sources, such as the one cited above.

By the way, EASA is already proposing to remove the requirements for motion for type rating and recurrent training. It will be up to the air carriers to ask their respective regulator why they made (and keep) motion mandatory.

Regards from Montreal


Jean LaRoche, FRAeS

[Canoehead]

I will add that this is my 31st year training experienced training pilots for initial Approved Check Pilot delegation. I actually created the ACP course training standard for TCCA in 1989. I think it qualifies as descent experience.

Last point that was intended in the paragraph you didn't like: I'm sure we will agree that instructors, not devices, teach. Yet, the role of the instructors is never mentioned in transfer studies. We feel we need less hardware, and more liveware.

As Bede said, thanks for weighing in. And I would like to apologize for the general question of your experience. My statement of the paper being of an "uneducated" source was also hastily written- we may disagree on the content of your paper; however, it was not fair to say you're uneducated. That's obviously not the case. As a long-time poster on this board, I try hard to refrain from mudslinging (way too common here), and when I post, I typically only do so on subjects that I have direct knowledge or experience with.

That being said, I think you and your colleagues are misguided on the importance of motion in the airline (704/705/121 etc) training and operational environments. I certainly will have a look at some of these studies you’ve mentioned when I get time. In the meantime, here are some of my limited thoughts on the subject. I've not been doing this for 31 years, however I was an instructor at the community college level for a significant period, a training captain in my 703 life, (just a driver in 704) and in 705 everything through to an ACP-A (no HF instructing though; my CRM skills are lacking obviously ). Only 15 or so years of instructing in various capacities and on a few types.

One of the biggest challenges is training low time pilots new to the airlines. Programs were previously designed for newhire pilots showing up with 3000 to 5000+ hours with multi-turbine experience. Recently (up until Covid) it was becoming common to have 250-hour pilots start a course (sometimes less) at my company. There has been tweaking done to programs and supervision, and without a doubt, especially for these pilots, motion is a requirement. The training in fixed base devices is absolutely necessary to teach systems and procedures, however the introduction of motion to the candidate enhances critical aspects to the learning and evaluating in a large airplane. Removing it would make achieving certain tasks and objectives prior to going flying unattainable. Completing tasks while sitting in a stationary environment may be straightforward for many. However, add motion to the equation, even if it is only a psychological player for some, and it creates an entirely new challenge. Have you ever had to operate in such intense moderate to severe turbulence that you couldn't program the FMS for the new arrival and runway ATC just gave you while you're mere miles from the start point of the new STAR? I have- and I can create a hell of a rough ride in the sim to simulate that environment (if that were a syllabus exercise). I’ve seen many pilots progress through CPT/IPT events and get to the FFS part of the program, and they can’t get through. It doesn’t happen often, but I’ve seen it, and without the added complexity of motion and associated training, would we assume those struggling pilots would otherwise pass and should be ready to fly the airplane? Not a chance.

As far as recurrent training, there are certainly some exercises that could potentially be done without motion (ie: FMS hold programming, or engine starting etc.). But how do you expect that approach to stall, windshear, EFAV1 or inflight engine shutdowns are trained effectively without motion? I can tell you that even though the fidelity of the motion during some of those events is based on interpolated data and test flying (and therefore not exact), there is an abundance of cues that come out of that as a result of the vestibular and visual trickery going on that we call "motion". Pilots rely on ALL of their senses; smell, sight, touch (motion) and sound (please engage sim manufacturers on better sound quality). Taste- ask about the horrible coffee on the break.

The idea that pilots are more and more just autopilot babysitters and FMS programmers is certainly what many think we are. Good pilots still understand "seat-of-the-pants" flying. Even in a FFS I can feel uncoordinated flight, or if the airplane is turning the wrong way on a hold entry or the sinking feeling during a windshear encounter. FAA Directive 2 requirements have added in some incredible new tools to simulator software, and motion is an integral part of those maneuvers.

You had mentioned sim manufacturer engineers sending in many emails agreeing with your findings; are any of them experienced rated pilots in heavy aircraft? I've worked with some simulator design engineers. While they have brilliant minds, and some are very talented, I have found that they understand 1s and 0s very well, but some can be obtuse and even daft when it comes to theory of flight or even common-sense issues. I've had several discussions where I've had to say "I don't give a rat’s behind about what the data says or what you say 'should happen'... I fly the airplane and I am telling you that THIS is how it works". No disrespect to engineers, technicians or researchers... but the SME on how a simulator behaves or feels (motion) is going to be a type rated pilot on that type (preferably test pilots for the nitty gritty).

Manufacturers have vastly improved simulators- I don't think there is any doubt there. Several years ago I took part in collaborating with a simulator manufacturers R&D people on instructor ergonomics; they observed 2 of my sessions and asked me to provide feedback on everything from the seat-belt release button to lighting to the idea of sitting at a "mag-lev based IOS" when they master that technology. As for motion, they have certainly improved from the hydraulic actuators of the past to the current electric systems.

As a further relevant topic for discussion, I would be interested in your opinion or any research you've done on the idea of Computer Based Training versus classroom training. There are many considerations to be made around a company's choice on how to proceed with that. In the interest of safety, proficiency and the almighty balance-sheet, I'd love to see some research done on that now that we are several years into that method of learning which seems to be the norm at most operators. I have my own opinions on that; perhaps something for another thread.

Regarding instructors and their ability to teach (either as facilitators or instructors), I'm in agreement with you on the need for better and more "liveware vs. hardware" (great way to say it by the way). The blanket statement about quality in your paper struck a nerve with me because at my company, we have excellent, high quality instructors. I'm not naive enough to think we don't have a few "quality issues", but on the whole they are excellent. They can work with 250-hour kids and 25,000-hour senior pilots old enough to be their parent. All the while working with them in the same crew, and they connect with both. Continuous improvement is vitally important and is something my company's training department takes seriously. Contrary to what you said in your paper, a LOT of work is put into ensuring that those who can become skilled teachers are selected and then trained for the job. Maybe not everywhere, but in North American and Western European environments, I think your statement is inaccurate.

So again, my apologies for my earlier tone. However, with all due respect I feel that the idea that “motion is useless, just fun, or a money grab (or waste)” is extremely short-sighted and inaccurate. It's certainly an opinion that psychologists, engineers and MBAs can have. But it's wrong. As a type-rated pilot and instructor, I have one too. It just happens to be the correct one.

[ami6car]

Hi Canoehead,

Nice to read your post. Lots of very good questions, which I also asked myself many times. To make sure we understand each other, I am not a scientist by any means. I come from operational aviation, from being a flight instructor, to flying the Arctic, to 705 domestic then overseas pilot. The rating I am the most proud of is my Flight Instructor Rating. Having worked two decades with a WWII pilot, known to be one of the fathers of aviation psychology, I was lucky to learn about transfer of learning. Canoehead, I certainly don't wish to discount any rational argument from fellow instructors, no matter what they are.

"Scientists have been concerned with the training effectiveness of simulators of all kinds. The fundamental test of that is of learning transfer. The concern is not with how well students can fly the simulator but rather, how well they can fly the airplane having had prior experience in the simulator. On a number of issues, those two do not line up in the way we might expect. Motion has proven to be one of those issues." (Lintern, 2021)

I am sure that your experience shows that students learn how to control a simulator more quickly if it has motion. This is well documented and accepted. It is often called Quasi Transfer of Learning (a name introduced by Gavan Lintern, Cognitive Engineer) But subsequently, they do not show any differential advantage in the airplane. That has been a consistent finding across the board. Your example below regarding turbulence confirms this even further. This finding even extends to Rotary Wing aircraft. A report out of Fort Rucker, sponsored by the Army Aviation center, on motion versus no motion training for Rotary Wing aircraft, showed no transfer advantage for motion training on a rather extensive range of flight tasks. But when we wrote the article, we decided to focus on commercial airline type rating.

(...) however the introduction of motion to the candidate enhances critical aspects to the learning and evaluating in a large airplane. Removing it would make achieving certain tasks and objectives prior to going flying unattainable.

You touch the exact point. More often than not, adding motion while training makes the trainee's life easier. Of course, there are limits to this, such as, whenever Gs should be a concern.

Have you ever had to operate in such intense moderate to severe turbulence that you couldn't program the FMS for the new arrival and runway ATC just gave you while you're mere miles from the start point of the new STAR? (...........) and without the added complexity of motion and associated training, would we assume those struggling pilots would otherwise pass and should be ready to fly the airplane? Not a chance.

I operated in moderate turbulence many times. Any pilot who landed in Torbay did. I have to rely on science however. Few topics, if any, have been more studied than simulation motion. There isn't a single study supporting your point above. On the contrary, there are tens of thousands of airline pilots flying in turbulence today who, like myself, were trained in motion simulators without any scripted turbulence.

As far as recurrent training, there are certainly some exercises that could potentially be done without motion (ie: FMS hold programming, or engine starting etc.). But how do you expect that approach to stall, windshear, EFAV1 or inflight engine shutdowns are trained effectively without motion? I can tell you that even though the fidelity of the motion during some of those events is based on interpolated data and test flying (and therefore not exact), there is an abundance of cues that come out of that as a result of the vestibular and visual trickery going on that we call "motion". Pilots rely on ALL of their senses; smell, sight, touch (motion) and sound (please engage sim manufacturers on better sound quality). Taste- ask about the horrible coffee on the break.

The whole aviation training world changed after AF447. Can we comfortably assert that simulator motion helped the AF447 pilots? How exactly is simulator motion helping the recovery of unusual attitude today? Those rather painful questions led EASA to abandon mandatory sim motion for type rating and recurrent training. The EASA NPA is out as I write these lines. You may submit your comments before March 31st.

You had mentioned sim manufacturer engineers sending in many emails agreeing with your findings; are any of them experienced rated pilots in heavy aircraft? I've worked with some simulator design engineers. While they have brilliant minds, and some are very talented, I have found that they understand 1s and 0s very well, but some can be obtuse and even daft when it comes to theory of flight or even common-sense issues. I've had several discussions where I've had to say "I don't give a rat’s behind about what the data says or what you say 'should happen'... I fly the airplane and I am telling you that THIS is how it works". No disrespect to engineers, technicians or researchers... but the SME on how a simulator behaves or feels (motion) is going to be a type rated pilot on that type (preferably test pilots for the nitty gritty).

Hahaha. I worked a lot with them to, in Canada and Australia. I tend to react the same way. I swear! But many are very good, and some are also active pilots as well. But at the end of the day, I can't do what they do. So we need each other.

As a further relevant topic for discussion, I would be interested in your opinion or any research you've done on the idea of Computer Based Training versus classroom training. There are many considerations to be made around a company's choice on how to proceed with that. In the interest of safety, proficiency and the almighty balance-sheet, I'd love to see some research done on that now that we are several years into that method of learning which seems to be the norm at most operators. I have my own opinions on that; perhaps something for another thread.

I would like to see such study as well. I have not done any research because I am not a scientist and will never be one. I have produced over 100 hours of asynchronous aviation eLearning. This is the core of my daily work. Here's a sample of what I do. Made recently. It's public.

https://vimeo.com/461924337

In March/April 2020, I completely recreated the Approved Check Pilot course in English and French, for hybrid synchronous/asynchronous delivery. 28 hours (each language) in total. Since CQFA began doing distance learning in 1997, this is the first time we use this hybrid approach. I will tell you Canoehead that I was very pessimistic, but Covid-19 didn't leave me much options. TC re-certified the course in May 2020. Last week I gave my 11th distance-learning ACP course. I was in the sim all day today with initial candidates to complete their training. I'll admit: I don't see any difference. The applicants are as sharp as ever. The final test will be when enough ACP applicants will have gone through the initial monitor.

I'm in agreement with you on the need for better and more "liveware vs. hardware" (great way to say it by the way).

That is inspired from the SHELL model.

The blanket statement about quality in your paper struck a nerve with me because at my company, we have excellent, high quality instructors. I'm not naive enough to think we don't have a few "quality issues", but on the whole they are excellent. They can work with 250-hour kids and 25,000-hour senior pilots old enough to be their parent. All the while working with them in the same crew, and they connect with both. Continuous improvement is vitally important and is something my company's training department takes seriously. Contrary to what you said in your paper, a LOT of work is put into ensuring that those who can become skilled teachers are selected and then trained for the job. Maybe not everywhere, but in North American and Western European environments, I think your statement is inaccurate.

I still back this statement and would prefer discussing it over a coffee. I know some departments are fantastic, of course. I am convinced yours is and it's great. Frankly. But our paper isn't Canadian. It was published in four official ICAO languages and generated comments from 21 countries so far. Still, after our last exchange, I called the training people of two "National Ops" airlines. Both replied today, saying that less than 10% of their training captains had structured Train the Trainer courses. As I said previously, at the end of this year, Canada will likely propose changes to Training-Captain training requirements. Of course, some Canadian 705 carriers take this very seriously. One of them is about to become the first EBT operation in the country. But both of us can also name flagship airlines around the world with a definite 0%. I believe the biggest impact on aviation safety today would be to invest in training pilots, "from the ground up" no pun intended.

I appreciate very much your dedication and time to make this exchange possible. I always learn a lot when I interface with professionals.
I know you will like reading the previously cited Bürki-Cohen 2011 reference. You should download the EASA NPA and consider submitting your comments before March 31st.

The FFS sold today are fantastic devices. The day a solid transfer of learning study will document an advantage of training with motion, I promise, I will change my mind and write about it.

Simulator early tomorrow morning. Time to go.

Regards



Jean LaRoche, FRAeS
Montreal