I'm gonna go ahead and ressurect this thread because I am also having issues playing Betrayal in Antara.
I have succesfully installed Windows 3.1 on DosBox, and have fully installed betrayal in Antara succesfully as well.
However, whenever I try to start the game I only get as far as a black screen, and the sword cursor popping up before DosBox freezes and exits automatically.
I am not quite sure what is happening as you said it ran fine for you under Windows 3.1 but any troubleshooting would be especially helpful.