In "Should the iPhone 5 get a larger screen?" (CNET
January 21, 2011 11:15 AM PST) David Carnoy asks if the iPhones 3.5 inch screen should be made larger to compete with 4 inch Android phones. He comments that some of the screens larger than 4 inches (such as the 4.3 inch EVO are "bulky"). When teaching web design for mobile devices to students in 2006 I had to consider how large a screen to design for. The larger the screen, the easier it was for the students to design for.
So I considered what ultimate limit there was on the size of a mobile phone screen. I decided the limiting factor was the size of the human hand, this was partly because the phone had to be comfortably held in the hand and also fit in a pocket (and pockets are sized to fit hands). But what size is a hand. In a flash of inspiration I considered the size of the typical credit card: 85.60 × 53.98 mm (standardised as the ID-1 format).
Allowing for margins, the height of a credit card is also about the same as the width of a newspaper column (1.83 inches or about 47 mm). That would suggest a reasonable amount of content could be displayed on a portrait format screen that size.
Credit cards are also about the same size of a business card. This seem reasonable, as both are designed to be held in the hand and fit in a wallet, which fits in a pocket. The wallet has to be a little larger than the cards it accommodates. So the size of a credit card seemed to be a natural maximum size for a hand held, pocket size device. This has a diagonal size of 101 mm (4 inches) and supports Mr. Carnoy's view that devices with screens bigger than 4 inches are clumsy.
ps: There is speculation in the Wikipedia about the origin of the size of a credit card. I suggest it derives from the human hand.