Everyone is a #$%^&* expert. I just received a long dissertation from an “expert” who “as retired high time international airline pilot, I feel qualified to give you a look at the practices in an airliner cockpit.” He seems to have all the answers. He feels that modern cockpit automation has diminished pilots’ stick-and-rudder skills to a dangerous level. The solution, he says, is “a pilot should hand fly the jet from take-off to 10,000 feet on departures and from 10,000 feet to landing at least twice a month. This will be enough to maintain a satisfactory level of hand flying.”
Do you really think that will do it? Back in 1988 I helped Delta formulate an automation philosophy for pilots to have a strategy for dealing with the then new automation available to us. Since then the controversy has not abated. On the one side, automation naysayers proclaim that they need all the practice that they can get and that they can do a better job of controlling the airplane than the computers. The automation cool-aid drinkers (I am one of these) say that pilots must have manual flying skills available, but some things are better left to the computers. To reconcile these diverse strategies, the regulators and operators have mandated (promulgated) many rules for the pilots to obey. For example, if the runway visual range (RVR) for an approach is less than 4000 ft., and an autopilot and/or flight director is available, then it must be used. That sounds very reasonable to me. On the other hand, when demonstrating proficiency at my semi-annual simulator torture, a hand-flown, single-engine approach looms. A little manual flying practice seems to be in order, as the aforementioned expert asserts. Yes, but how much and what type? More to come.
Jan 23, 2015
Back in the early ’80s, I was flying the B-727 when the then highly automated B-737-200 came out. I quickly volunteered to take a more junior position so I could experience the automation. It had a mode control panel, dual autopilots, and a Performance Data Computer, very advanced for the day. Some of the pilots had difficulty managing even that low level of automation. When the B-767 came out, it ushered in a whole new era of automation. Senior pilots were flocking to the higher pay, but many had difficulty in the checkout. Delta backtracked somewhat and started the “Introduction to Aviation Automation” (IAA) program, designed to bring the dinosaurs up the the computer skills of the kids. Fortunately for the dinosaurs, Delta allowed them to fly the 767 like they had done in the 727, if they chose to do so. That, of course, missed the whole point of increasing safety with automation.
On the other side lay the lazy, the geeks, and the arrogant. They never turned the stuff off. “Gear up, flaps up, autopilot on,” was the mantra heard in many cockpits. Later that was followed by “Gear Down, Flaps down, autopilot off, aren’t I wonderful.” Maybe so, but I couldn’t really know if they could fly or not. I once saw a new 757 pilot get cleared for a visual approach from 10 miles out from Louisville, a rare opportunity for those of us condemned to fly the instrument approach most of the time. He proceeded to type his way around the approach with the mode control panel. A sadly lost chance for that manual practice that we covet.
Too little or too much use of the automation obviously can result in less safety and efficiency. Where is the sweet spot where automation reduces cognitive workload when it is needed, and provides guidance accuracy that humans are unable to achieve?
Jan. 24, 2015: THE HISTORY
So what did we come up with in 1988? The need for an automation philosophy became obvious after Delta’s “Summer of Shame.” In a short period Delta pilots narrowly avoided a midair due to a pilot error over the North Atlantic, landed at the wrong airport (no injuries), crashed a 727, and was still smarting from a L-1011 crash. I attended both the Delta 191 and Delta 1141 NTSB hearings as the ALPA Human Factors representative. At the 1141 hearing, the NTSB Investigator In Charge (IIC) was really mad at Delta for not complying with the 191 findings and recommendations. That’s when the airline really took action. The Vice-president of flight operations (later Senior Vice-president of operations), Harry Alger, selected Captain Reuben Black to lead a group that was to create a Cockpit Resource Management (later Crew Resource Management) program for the airline. Alger committed a huge amount of resources from a notoriously penurious company. Black was the consummate diplomat and led by extracting the best that was within each of us. Delta had never hired an outside consultant for anything. We were funded to hire the late Bob Helmreich and his NASA/UT group, the late Earl Wiener at the Univ. of Florida, and Phillip Hackman, a well-known psychologist from Boston.
Delta needed fixing. As a new hire, I was introduced to line flying at the Houston, Texas base. “We are a captain’s airline,” was never so true as it was there. What that really meant was that management was unwilling to devote the resources to develop a meaningful set of operations procedures to guide us. In that void, the cowboy captains ran amuck. Their logic ran something like this: “Since we haven’t had any accidents, the crazy stuff we are doing must be OK.” I was flying fighters in the Air National Guard at the time and thought I was bulletproof, but they scared me. Their luck ran out in 1985. The fix, was a good flight operations manual, better communications, and most of all, a fully participating crew, committed to doing what was best for the passengers. Imagine the wailing and gnashing of teeth when the old guard was called upon to change. “Holding hands in the hot tub,” they called the class. They planned to co-opt the program and never change. A pilot could not fail a check ride for CRM. However, an angry First Officer might fail to remind the Captain that one of his illegal habits would cause him a technical failure. The younger pilots got it. Management’s choice was just to allow the dinosaurs to retire in time, or to force compliance. It was a bloodbath. Simulator check rides were repeated, line checks were incomplete, pay was lost, it was serious.
In this tumultuous environment, the company wanted to form a policy on the use of automation. After all, Southwest had bought automation and disabled it. American bought automation and demanded that it be used. Of course, the FAA had previously promulgated rules that required the use of automation. Little argument existed over the notion that Category II approaches were better flown by the autopilot. No argument existed over the notion that Category III – autoland approaches were a spectator event for the pilots.
The problems facing us were similar to the ones we have today: Mode confusion, automation surprises, over dependence/loss of manual skills, and increased workload trying to use the automation that was poorly understood. Here’s what I came up with:
1. There is an optimum level of automation for every situation.
2. If the automation is increasing workload, use a lower level.
3. Pilots must be proficient at all levels of automation.
4. Both pilots should be fully aware of the state of the autoflight system at all times.
Optimum levels include allowances for using a low level to maintain proficiency. Maintaining proficiency at all levels of automation requires one to self-manage their training program. That is, enough manual flying to be able to handle the aircraft when the autopilot says, “I quit.” Both pilots fully aware means that secret typing in the Flight Management System is not allowed.
As of my retirement, this philosophy remained largely intact after eighteen years. Some good stuff was added:
One pilot should maintain outside visual awareness.
Both plots should maintain a comfortable workload distribution and situational awareness.
The pilot flying must compare the performance of the autoflight system with the flight path of the aircraft.
Most of us would agree that the aforementioned ideas are pretty good ones, but this devil is in the implementation. What with modern aircraft using Controller to Pilot Data Link Communications (CPDLC) the assigned altitude is automatically set in the mode control panel. How could you screw that up? Well, we still have altitude busts. We still have B-777s crashing for lack of manual flying skills. Correct me if I am out of date, but the last time I checked there had NEVER been a fatal accident following a autopilot-coupled Category III approach. Thus the dilemma: the autopilot can do it better than I can at the moment, but might fail unexpectedly.
I have experienced cognitive overload. I have frequently felt that I was at maximum workload. For instance, on a manually flown approach: intercepting a localizer, slowing, and configuring (and talking on the radio when single pilot). If a surprise had happened, I would have had to prioritize and do something later. Automation is a wonderful workload reducer in that situation. Asking your fellow pilot for help works too.
I have questions:
How does this apply to flying single-pilot aircraft?
How can I maintain manual flying proficiency when my boss says always use the autopilot?
How can I trap and mitigate the inevitable errors?
The weather is 700 ft. ceiling and 2 miles visibility. Is overall safety enhanced by flying the approach manually today?
Is there room (or need) for different styles of resource management?
Jan. 26, 2015
Flying IFR in a single pilot airplane can be very busy. I remember taking my instrument rating check ride, a long time ago. The examiner asked me to hold at an intersection with a single VOR. I must have done it OK, because I passed. I wouldn’t try to do that today for all the tea in China. I think that is partly because of my concept of risk and partly because I’m moving a little slower nowadays. Fast forward to a naive kid flying a F-106. It was so easy to get behind that airplane that we were all trying to think of something to do to get ahead (of stay up with) the situation. Things don’t happen that fast in an airline cockpit, but it was good preparation for flying the Bonanza. I have owned that airplane for less than a year and have shot approaches down to minimums twice.
I wouldn’t have been there if I did not have a good autopilot. What exactly are my manually flown minimums? I don’t have a formal a set of rules. I know I’m OK with VFR on top and punching through an undercast down to 500/1. What if the autopilot had failed enroute to a destination at minimums? I would go to the alternate. What if the approach coupler failed on the way down to a minimums approach? It’s always an exciting experience when the autopilot hands you the airplane unexpectedly. If the approach were stabilized and no other factors were worrying me, I would continue. If I were on downwind leg and this happened, well that’s a tough call. Proceed to alternate? Ask for an extended downwind? Admit to an equipment failure to ATC?
There are good reasons that corporate and airline flying is safer than General Aviation (GA). Maintenance, procedures, proficiency, management, and nearly every aspect of the operation receives more focus than GA gives it. Is that to say that we GA pilots should emulate corporate and airline operations? If you can afford to trick out your avionics package like a 777, I say, “Go for it.” Bloody likely though. What we could do is adopt an automation/workload philosophy more like theirs. Statistically speaking, it would certainly be a good idea. If one were to embrace this concept, then there is a wealth of knowledge that draws from their experience and the procedures that were created in response to it.
Feb. 3, 2015
For those of us who believe that Air Asia 8501 and Air France 447 crashes are similar, note the following:
DATE: December 10, 2014
AD #: 2014-25-51
Emergency Airworthiness Directive (AD) 2014-25-51 is sent to owners and operators of Airbus Model A318, A319, A320, and A321 series airplanes. Background:
The European Aviation Safety Agency (EASA), which is the Technical Agent for the Member States of the European Community, has issued Emergency Airworthiness Directive 2014-0266-E, dated December 9, 2014 (referred to after this as the Mandatory Continuing Airworthiness Information, or the MCAI), to correct an unsafe condition on all Model A318, A319, A320, and A321 series airplanes.
The MCAI states: An occurrence was reported where an Airbus A321 aeroplane encountered a blockage of two Angle of Attack (AoA) probes during climb, leading to activation of the Alpha Protection (Alpha Prot) while the Mach number increased. The flight crew managed to regain full control and the flight landed uneventfully.
When Alpha Prot is activated due to blocked AoA probes, the flight control laws order a continuous nose down pitch rate that, in a worst case scenario, cannot be stopped with backward side stick inputs, even in the full backward position. If the Mach number increases during a nose down order, the AoA value of the Alpha Prot will continue to decrease. As a result, the flight control laws will continue to order a nose down pitch rate, even if the speed is above minimum selectable speed, known as VLS.
This condition, if not corrected, could result in loss of control of the aeroplane.
To address this unsafe condition, Airbus ***[has] developed a specific Aircraft Flight Manual (AFM) procedure, which has been published in AFM Temporary Revision (TR)No. 502.
For the reasons described above, this AD requires amendment of the applicable AFM [to advise the flight crew of emergency procedures for abnormal Alpha Prot.
Well, it appears to me that in the A-320, if the computer thinks that the aircraft is stalled, the nose is coming down, and there is not much the pilot can do about it. I flew the A-330 simulator. Delta was considering the purchase of them and wanted to know if ALPA would go for it or not. It was pleasant to fly, when everything was working. I asked for a scenario in which I had to maneuver to avoid a midair. The aircraft was so gentle in responding to my inputs that we had a simulated collision. The takeaway for a Bonanza pilot trying to be safer is to choose an airline that buys Boeing. I’m just not buying that these very experienced pilots just went stupid under pressure. Remember the cheapest group to blame in a crash is the pilots.
Feb. 18, 2015
AOPA’s Tom Haines weighs in (Pilot Magazine)
One of my favorite aviation publications, “AOPA Pilot,” Feb. 2015 issue contains an article by Tom Haines. He comments that back in 1912 when Sperry developed the first autopilot, some of the pilots probably declared the contraption was for wimps. “Automation can improve safety and reliability, increase situational awareness, and increase redundancy. Yet, despite the progress, we continue to see a high level of accidents where pilots seem too dependent on automation.” From another source, I read about the last moments of Air France 447. After the static system failed, the automation set the throttles at an acceptable place for level flight. If the pilots had just held the nose level for a while, things might have turned out differently.
A few years ago I was asked to comment on Turkish Air 1951 which crashed at Amsterdam Schiphol Airport in 2009. They accepted a slam-dunk approach. With the throttles at idle, they didn’t catch an autothrottle computer failure. The aircraft crashed short of the runway for lack of thrust. I believe that there is a parallel to the Asiana 214 crash in San Francisco bay in 2013. Although they had visual conditions, it was a slam-dunk and lack of thrust became the cause.
I had very little excitement during my airline career. However, the most gut-wrenching feeling I ever had, was the second or two that I thought I had waited too long to push up the throttles, near the ground. Why would a pilot be caught in that situation? When we transitioned to jets, we were always told, “Keep the throttles ‘way up, because jets don’t accelerate as fast as piston engines. ” Good advice, but if you were able to push the throttle(s) up at the last minute, you saved fuel and looked “cool.”
The late Neil Anderson, the famous Convair/General Dynamics test pilot, who conducted many of the F-16 early tests, was my mentor when I was learning to fly airshows in the T-33. I miss him greatly. He told me a story about himself and the MiG-21. He was “the pilot” when the F-16 was sold to Egypt. President Mubarak asked Neil if there was anything he wanted. Neil replied that he would like to fly the MiG-21, a mysterious aircraft back then. Next morning, there was the aircraft waiting for him, no briefing, no checkout, just a crew chief and a General to answer questions. Asked about his plan, Neil said he wanted to conduct a couple of “touch-and-gos.” The General asked, “What is touch-and-go?” Neil explained that after landing, the pilot advances the throttle and takes off again. The General nodded in acceptance. Upon returning from his flight, Neil did exactly as planned and advanced the throttle, except nothing happened! Halfway down the 12,000 ft. runway, the engine reached 60%. With 2,000 ft. to go, 75% was indicated. After exiting the departure end of the runway and destroying nearly a mile of brush, the engine reached full thrust and the aircraft became airborne. Leaving the landing gear down, he landed and taxied up to the ramp, where the General and crew chief waited. The flaps, gear doors, and belly had been severely damaged, a tribute to the durability of the MiG. The General asked, you guessed it, “Mr. Anderson, did you enjoy your flight?”
Well, the point of this discussion is that jet engines accelerate slowly and our lack of understanding and practice of thrust management may be more detrimental to safety than getting rusty in our manual maneuvering skills.
Feb. 22, 2015
(Copy of Correspondence from Bud Landacre, commenting on the Neil Anderson story)
Interesting article. I was always an advocate of hand flying the aircraft. I remember a quote by on old mossback published in an article in BCA magazine regarding automation. He said ” I can’t fly worth a damn any more but I can type 40 words per minute”. I was riding as an observer in the Lear simulator as the crew flew a coupled ILS approach to minimums. As the crew dutifully monitored the autopilot flown approach the sim operator input an aileron trim malfunction (full left aileron trim). The autopilot did a masterful job of flying the airplane with a not too noticeable control wheel offset. Can you guess what happened when the captain clicked off the autopilot at about 150 ft agl in min vis conditions?
Feb. 22, 2015
Here’s me on a recent trip to the Texas Hill Country.
Feb. 23, 2015
I’m hoping to find consensus on the use of automation in aircraft. Looking at the discussion and comments suggests one thing that we can agree on is that automation can do some things better than human pilots.
According to the FAA, we must use the autopilot when cruising in RVSM airspace. The Reduced Vertical Separation Minima was introduced to pack more airplanes into the 28,000 ft. to 41,000 ft. airspace. Formerly aircraft were assigned altitudes two thousand feet apart. With RVSM that restriction is reduced to one thousand feet. However, to be allowed into that airspace, the aircraft and autopilots must be certified to be very precise. This sounds reasonable to me.
I remember a long flight from Taipei to Portland in the MD-11. As the copilot, I had flown the previous leg; it wasn’t my turn to fly. When all three autopilots failed, it became my leg. The Captain wouldn’t or couldn’t manually fly those long hours going home. I confess that my altitude-holding precision diminished after a while. Yes, the autopilot is better at that.
Regulations tell us that when the runway visibility is less that 3/4 mile, we must use an autopilot if it is available. Also, we must use three autopilots, autothrottles, and automatic landing if the visibility is less than 1,200 ft. Some Category III certified aircraft can land in weather as low as zero ceiling and 75 meters horizontal visibility. These numbers vary across airlines, where their operations specifications are approved by “the authorities” and become regulatory. Again, I have no problem with these regulations. In my 50 years of flying, I have conducted or more precisely, watched the autopilots conduct, only five approaches in zero-ceiling conditions. Three of them were in Zurich. To qualify to do this, the aircraft must be equipped with three autopilots, dual autothrottle computers, dual radar altimeters, and a slew of other gadgets which must all be working properly. One important gadget is GPWS, Ground Proximity Warning System, which compares GPS position to a worldwide terrain database to assure us that we aren’t going to run into a mountain. The pilots must be trained and current, and the aircraft must have conducted an autoland recently. If any of the required equipment fails early in the approach, better weather must be found elsewhere. Closer to the ground a failure such an one of the autopilots is merely disregarded and the approach is continued with two, since a missed approach from low altitude is considered more risky. The Captain must fly the approach. The First Officer (copilot) makes the callouts. The mode changes are among the most important of these. Near the ground the aircraft goes into align mode for crosswind correction. “Flare arm, flare capture, touchdown, and rollout,” are the only human sounds in the cockpit . If any of these mode changes don’t happen at the proper altitude, I press the TOGA (takeoff-go around) button and we’re outta there. As the aircraft passes fifty feet from the ground, the copilot says “Flare Arm,” and the radar altimeter announces height every ten feet in its human female voice . “Fifty, …., forty, …, thirty – flare Capture, …., twenty, …., ten, …., touchdown, …, rollout. I still see nothing out the front windscreen. Derotation begins and the nose wheel is gently landed as I bring the engines into reverse thrust. As the nose lowers, I begin to see the runway centerline lights, but only a few since the visibility is so low. The alignment is perfect – it always is. The wing spoilers deploy automatically, the brakes slow us automatically, with out any skidding, I disconnect the autopilots as we slow down. I am instructed by tower to continue to where the runway lights intersect the SMEGS or Surface Movement Guidance and control System lights. Since the tower controllers can’t see us, we receive guidance and control with lights embedded in the taxiways. Chuck Yeager might be able to do that without getting wet armpits, but I can’t.
If there’s anyone out there who likes to do this without an autopilot, I would like to hear from you. Absent any disagreement, I would like to establish Automation Tenet No.1: Automation can do some things better than human pilots.
Mar. 3, 2015
You might notice something a little strange about the F-102 in this photo. I took some of these from the rear seat of a T-38 while TDY at White Sands Missile Range (WSMR) in New Mexico in 1974. I’m not sure if I took this one or not. The airplane is one I frequently flew while at Keflavik, Iceland. The point, of course, is that no one in in it. No discussion of humans vs. computers in aviation would be complete without getting to UAVs, or Unmanned Aerial Vehicles, or drones. The caution with which we conducted these flights might be indicative of the process whereby we entrust airliners to one or zero pilots in the future.
This is a QF-102, a “man-rated” airplane. That is, it still had all the controls for a pilot in it. It could be flown with or without a pilot. Later versions, called the PQM-102 had no place for a person in the cockpit. Earlier drones, such as the QF-104 were controlled from an aircraft (usually a T-33) flying nearby. The QF-102 was controlled from the ground via a huge radar dish that was not perfectly reliable. Consequently, WSMR safety insisted that we have a device that would destroy the aircraft if we had a Loss Of Signal (LOS) for more than a few seconds. Harsh stuff. The purpose for the drone was to test our new missiles with a full-scale target. Previously, we had done so by shooting at a Ryan Firebee with radar and infrared augmentation. That was a small aircraft and no one knew if the accuracy of our tests would “scale up” to real fighters. Thus the QF program. It was a sad day that I watched “my” airplane that I had flown so much in Iceland, get shot down by an AIM-9 missile. Better to die being productive that be chopped up, I suppose.
Later during some of the scheduled shots at the aircraft, the pilots were fully ready to fire more than just the test missile at the drone if it were to have a LOS problem. None ever escaped, to my knowledge. That was a far cry from modern UAVs. Nowadays, GPS and Satellite communication permit lethal drone airstrikes in Afghanistan, conducted by pilots in a bunker in Houston. With this new-found reliability, is it only a matter of time before airlines go to a single pilot? The B-777 have a software program in it that sounds an alarm if no switch or button has been actuated for a few minutes. Sometimes when there was an unusually long distance between reporting points over the Pacific, it would fire. Had we been asleep, it would have awakened us. I think the next big controversy will be whether or not to allow a single pilot during the cruise portion of a flight, while retaining two or more for other times. That’s going to be a tough sell for airline management, remembering AF 447. I admit that my prejudice may come from the desire for job security. However, retaining two pilots at all times has some obvious safety benefits. Imagine a single pilot on a B-797 in the future who expires unexpectedly. The airplane detects this and summons the other (sleeping) pilot. Meanwhile some previously unknown software failure mode commands 23 snap rolls. Not a pretty thought. Forgive my ramblings. What I intended to say is: Are there some things that pilots can do better than the automation?
March 6, 2015
Bill Gross commented: I think it might be profitable to look into the events at 3 Mile Island. The PA Nuclear Power plant that almost lost it in the late 1970s. From what I recall the plant operation was almost totally automated to the point that the operators lost focus on what was going on. When indicators started showing the plant was having problems, these ignored because the operators lost that focus.
I think he is exactly right. Many disciplines are experiencing the controversy that we have in aviation. This goes beyond automation. It is a crisis in responsibility. Whether your title is Captain, Nuclear Power Plant Supervisor, or Backup Quarterback, if you are being paid to be ready to do a job, but seldom are called upon to do it, discipline is required. That may be internally or externally enforced. Pilots are usually the first at the scene of the accident, often in a fatal manner. Perhaps this is the reason that most of them who I know, generate this discipline internally. Not so for the backup quarterback. I he does a lousy job, he can take his millions and go home.
March 17, 2015
The computer says it will make it to Tokyo just fine, climb aboard.
In my last post I asked what things pilots do better than the automation. I’ll start by suggesting that pilots taxi, takeoff, prioritize, adapt, fly the visual approach, and evaluate risk better than the computers. Also, and this is a biggie, we check for reasonable answers. Before I retired, we were still putting the flight plan into the Flight Management System manually. Maybe it’s being uploaded by now. When we did that, we would check the total distance against the printed copy. If it was very far off, the legs page usually revealed that we had a typo which resulted in the magenta line going off the page to some faraway place then back to the next waypoint. We are very good at finding and mitigating those unanticipated failure modes, both self-inflicted and computer generated.
We still taxi, takeoff, and fly the visual approach with little automation. During my B-777 checkout, I was asked to fly a V1 cut without the automatic rudder input. I asked, “Why do I need to practice multiple emergencies (loss of an engine, and loss of the autorudder)?” The answer was that it’s not really much of an emergency if you can fly an engine-out departure like a normal one. I will take this as a compliment: an engine failure during takeoff is still too risky to allot to the automation. I don’t know why we haven’t implemented auto-taxi. Maybe it’s because the low-risk situation doesn’t warrant the expense. I used to say after turning off the runway, “It was a good flight and I don’t see how we could possibly be killed between here and the gate.”
Any comments about what we humans do best? Why aren’t we practicing those things, like visual approaches? Hello, management, are you listening?
June 12, 2015
I just read my June issue of “Soaring” Magazine. In it Rick Eriksen comments on this subject in “Today’s Pilots Depend Too Much on Technology, adding to Human Error.” He says, “Planes fly the same way they flew back when I was flying in the 1970s, ’80s, and ’90s. And we made it work with much more reliability than they do with today’s over-the-top automation.”
I have to disagree in that statistics clearly show that automation has made us safer. I do agree that many of us have checked our brains at the door and let the automation run. However, if we return to rule #3, “pilots must be proficient at all levels of automation,” and we agree that is a good idea, then how do we convince reluctant managers and pilots that it is a good idea? Mr. Eriksen says, “Maybe it is time for the FAA to start to begin to differentiate between flying time and computer time.” That’s a good idea for managers, because training and check rides are so closely monitored. How about the lazy pilot: “Gear up, flaps up, autopilot on. Aren’t I a wonderful pilot?” And then there’s the pathetic airline pilot typing their way around a visual approach with the mode control panel. I think it was an extreme dependence on the autothrottles that nailed the Asiana pilots, after all, the airplane hit the runway.
I regret that many of us in general aviation do not maintain our manual flying skills sufficiently. However, it is completely unacceptable that airlines and other professional pilots fail in this regard. The fix is simple: the regulatory authorities demand that manual flying skills and visual approaches be demonstrated on rating and recurrent check rides. If they tightened down the screws on this, pilots would be out there practicing.