So with advice from the group, the floor was insulated. Thanks to the suggestion of the insulating bricks under the firebrick floor. The plywood removed (wasn't part of the structure). Insulation around the dome, then stucco over it. Added a wire mesh over the insulation before stucco. Probably could have been a little thicker.
The second floor made the oven more shallow. It is holding heat pretty well. After a couple more low heat fires it will be time to try cooking!