As with many discussions on the web the debate around skeuomorphism and flat design has often missed the point on both sides. Much of the conversation has centered on the purely aesthetic or on localized issues of function such as affordances. However, underlying the shift in style is a change in the media of both web and mobile design.
The Web
The web is now expected to work on a continuous range of screen sizes, pixel resolutions and user contexts. Evolving expectations have coincided with new technical tools such as CSS 3 and vastly improved support for typography. Both the hardware and software elements of the medium have changed significantly over the last several years. A new aesthetic style has naturally emerged as designers have been learning how to deploy new technical tools to meet evolving user contexts and expectations.
Android and Windows “Metro”
Android and the Windows “Metro” have always used a mostly-flat aesthetic style. I am not aware of any public statements made by the design teams behind these products indicating how and why this choice was made. While we can't know for sure, I believe Marc Edwards is correct in his speculation that the decision was driven by technical constraints, specifically the need to support low-end GPUs. The technical constraints likely to underly these designs are also likely responsible for the choice to use square corners, a choice which has been relatively pervasive in flat designs even in places where this constraint does not apply.
iOS 7
This year at WWDC Apple revelead a significant redesign of iOS. Many designers pundits have pointed to iOS 7 in claiming that Apple is no longer a design leader, that they are now following the design trend established elsewhere. Superficially there is some merit to this argument in that skeuomorphic ornamentation and light effects are largely absent but there are significant differences in both the aesthetics of iOS 7 and the motivation underlying these choices.
Rene Ritchie called iOS 7 the birth of the dynamic interface. The potential of modern hardware to provide an even more fluid experience has motivated1 many of the changes Apple is making to iOS. The dynamic interface moves skeuomorphism from ornament to function. Instead of a heavily textured skeuomorphism that relies on pre-rendered lighting effects we have a dynamic skeuomorphism brought to life by a physics engine. This new design language embraces animation (often physics-driven) as a means to communicate spatial stability and consistency. On-screen content moves and behaves in a fashion similar to real-world objects. The new illusion is that the screen is a window into a virtual world with depth and perspective.
The shift from a static to a dynamic illusion has significant implications for the aesthetic choices Apple has made in iOS 7. Previous versions of iOS use pre-rendered lighting effects with a consistent light source to produce an illusion of depth. This illusion is broken when objects move with physical realism. Rendering similar effects dynamically as part of the primary user interface on mobile devices is prohibitively expensive2. Dynamic light and shadows may also be distracting to the user and would conflict with the principles of deference (to content) and clarity Apple has adopted.
Of course a consistent light source for the interface is still necessary. Apple has chosen the light source most natural to the medium of screen design: a backlight. While Apple has often favored very bright and bold colors, the choice of a backlight is probably partly responsible for the somewhat controversial icon colors. The specific colors chosen resemble the backlit mylar signs often seen in movie theaters. These signs may have been an obvious source of inspiration if Apple's designers started to think explicitly in terms of backlighting.
Conclusion
Design should always be about more than aesthetics and fashion. Great designers begin with a deep understanding of the potential and the limitations of their medium as well as clear user-focused design goals. Specific aesthetic choices often naturally emerge from (or are implied by) this beginning. Form follows function.
We are designing for devices and screens that are much different than those of just a few years ago. This necessitates thinking differently about our designs which obviously results in different choices. It is essential for us as screen designers to understand how and why a new trend in style has emerged in response to a changing medium.3
While I believe Apple's primary motivation is an improved user experience, others have observed that there are also significant strategic benefits to Apple in moving to a dynamic user interface. This interfaces pushes current hardware to its limits and will continue to do so for several years as Apple increases the dynamism over time. This will motive customers to continue upgrading their devices regularly and make it difficult for competitors to copy in devices with less capable hardware. Of course these strategic benefits will only be realized if the Apple is successful in delivering a superior experience that is highly valued by actual users.
Sophistacted dynamic lighting effects are acheivable on mobile devices in specific contexts such as games but would not be feasible for the entire operating system UI.
The long shadow fad is an example of a stylistic variation based on a superficial view of the broader trend towards less ornamental design. As it is a purely aesthetic fashion it is not likely to last very long.