Duxford is a great day out. You will need a full day to see all the aircraft there. I have flown into East Kirkby a couple of times and again it was a great experience to hear the roar of the four Merlin engines from a few metres away. Old Buckenham is not far from Norwich and has a few old guns etc. from its days as a USAF base in WW2. There are some interesting aircraft in the hangar including a Piper Cub in USAF colours, which is owned by an actor called Martin Shaw. Another good air museum is the Yorkshire Air Museum, at Elvington near York, but that is a little further to fly to.
I'm sure some others can recommend some more great places.