This rule is here because it allows the use of tricks such as those described in the appendix on Autodetection of Character Encodings. We recognize that although the Web provides a method for a server to tell the client what kind of encoding is being used, sometimes it breaks down, and sometimes there's no server (like when you're reading something straight off a disk).
In these situations, everything works much better if entities give the processor some help in figuring out how things are encoded.
Back-link to specCopyright © 1998, Tim Bray. All rights reserved.