I think that most any part of a loop is a high AoA situation, particularly the vertical up. 4 seconds of letting go there will end very badly in most types. What I have observed is the most new pilots are uncomfortable pulling enough G to complete a maneuver as intended. Not enough G may not see it through, and then a botched recovery is necessary.What's the danger of letting go of the controls in a high AoA situation for 4 seconds?
From my limited aerobatic experience, and more comprehensive unusual attitude recovery experience, I have observed that most pilots tend to let the plane fly them in such circumstances. Indeed, some recent testing in a modified Grand Caravan actually required "abandoning" the controls for twenty seconds in high speed and specified conditions of flight, and that was un nerving.
Abandoning the pedals in my taildragger for a second on the runway will result in being off in the weeds for sure.
I agree that in gentle cruise flight, inattention is tolerable (apparently 20 seconds in a Caravan!). Hopefully aerobatic pilots hold themselves to a higher standard of attention, and have the training to know what correct attention should be for maneuvering. From my experience flying airwork in both the Aerobat, and a regular 150, they both fly the same, and are equally capable, other than the Aerobat gives the pilot a lot more room to recover after getting it wrong. Aerobatics are a risk, poorly executed, and botched completion/recovery aerobatics are a big risk...
I perceive a risk of tailslide with an incomplete loop, or other maneuvers where the plane is pointed vertically, and the next element delayed too long. When I was a new PPL, my instructor took me in the school Citabria. I was poorly prepared for the aerobatics which we did that day, and in hind sight, think of that "training" to be more detrimental than beneficial to my piloting attitude. Later in life, I was trained by an aerobatic pilot who did have appropriate experience (and it was only an Aerobat).
But as I have said, I heartily encourage competent aerobatic experience for all pilots, just for awareness, and self confidence for unusual attitude recovery.