This pattern goes back to some of the earliest surviving codexes, and crosses cultural boundaries.
I think the answer rests in the technology itself. A codex was stronger if the pages were stitched up along one of their longer sides. Furthermore, readers quickly found it was easier to hold and use a book if that line of stitching was vertical, supporting the pages and providing a symmetric reading surface.
That format carried over to printed books, which in the West settled into standard sizes (folio, quarto, octavo) based on standard sheets of paper. Even publications that aren’t bound, such as newspapers, usually have a vertical aspect ratio.
Among the exceptions are many picture books, especially from mid-20th-century America: The Little House, The Runaway Bunny, Where the Wild Things Are, and so on. Those share the horizontal aspect ratio of cinema, selected to fill our binocular vision. In fact, when picture books present single images on full spreads, as they’ve increasingly come to do, they almost all have horizontal aspect ratios.
While some trim sizes are more expensive than others, book publishers can choose from an infinite range of aspect ratios. In contrast, an electronic device offers a screen of one defined size. Its proportions might differ from one device to the next. A device might be flippable to offer two ratios (say, 3:2 or 2:3). But if creators or publishers want to take advantage of the full screen, they have to adapt to its dimensions.
Generally the default setting for digital books is still based on a vertical aspect ratio, and that for digital videos on a horizontal aspect ratio. Adaptations of illustrated books seem to be caught in the middle, some going one way and some the other, some trying to offer both. Eventually we’ll probably have a standardized aspect ratio for screens, and designers will start with those dimensions in mind.