In http://bugs.mapwindow.org/view.php?id=1844 a large JP2ECW/ERMapper JPEG2000 file of 1.2GB is listed.
Opening this file takes several minutes. But when it is opened zooming and panning is really fast.
It seems for zooming and panning the overviews are used or some other intelligent optimization.
When looking at the load process of these large files optimizations might be possible.