That may be caused by a corrupt font.
The Marlett font is used to display the close, open, minimize, and other window control buttons.
Microsoft has a knowledge base article on this at
http://support.microsoft.com/support/kb/articles/q134/8/61.asp

This article is aimed at windows 95, you should be able to use the system file checker to restore it in windows 98.
Click start | run and key sfc.exe and click OK.