The firearms act is open to interpretation and different Police forces interpret it there own way. I'm an FAC holder and therefore have known a lot of people fall fowl to there area's Police forces idea of what the firearms act means.
As far as I know, any firearm has to be in a cover in a public place. I sometimes ride my bike to the shooting range with my rifle in a sleeve on my back and a box of ammo in my pocket. If I get stopped I'm not breaking the law, the rifle is covered and I'm licensed to carry it.
A deactivated gun is still a firearm, hence you need a certificate to prove if can't fire, and you don't need a certificate for an imitation firearm because it's not or never been a firearm. So I would say as long as your deac is covered, it's legal.
Like Ian says, you might find yourself defending this in court depending on what area you live in. That's if you're lucky..! In days of old, you may be stopped by the Police and a friendly Bobby will go through your paperwork, it's all in order and off you go. But if a member of the public calls the Police and says I've just seem a man riding a motorbike with a pistol in his belt, you'll have the armed response unit out. They've been known to shoot first and ask questions later..! and in today's climate, have got away with every mistake they've made. Like the bloke in London who was shot for carrying a table leg! and the armed response vehicle that was called out to a 10 year old playing with a purple plastic water pistol in a playground! I also know a bloke who went to a Royal mail workers fancy dress party, as Rambo with a plastic toy gun that had a bright red plastic end on the barrel. Someone called the Police and said I've just seen a man with a gun go into this house, as he walked home from the party. His phone rang saying this is armed Police outside your house, to which he thought it was his mates messing about and told them it's late, so f**k off. The door came in and he was dragged out.
Ok, scaremongering aside, if you're riding dressed as a DR on a 1940's bike and don't stop and walk into a shop or a garage with a pistol on your belt, you'll be ok 
email (option): horror@blueyonder.co.uk
After the dunblane incident I got phone calls from all over the world, turns out BT had issued me a number just previously used by a firearm specialist in my town & he hadn't updated his press contacts. The Guy is Jan A Stevenson, he went on to help put together the white paper on firearms after Dunblane, he also is a keen "anything ww2" enthusiast, he drinks at my local & if anyone would know, he would know, but I bet he would simply say, regardless of the rules, don't be anywhere with anything that looks like a firearm that a member of the public or officer of the law could mistake you for being a threat, because you are likely to be taking a risk.
Edit, I didn't see your post horror before posting mine, ....
Incidently, I have a few Percussion pistols & they are antiques really but when I checked with my police firearms Guy at Chelmsford about ownership he excitedly said "no, they can't touch you, for having those & you can make your own shot" its not exactly what I expected from a copper
Something like this during the Olympics could be problematic. ...
I hope the brakes are good on that bike, as the gun looks like it could have quite a bit of recoil when fired!! 
email (option): rickandjo@pullen14.freeserve.co.uk
Yes I worded it badly & had deleted by time I read your reply.
I read this " until you fire them and therefore effectively "reactivate" them as firearms." & thought you meant it came back into the 97 act, but maybe you didn't mean that.
I'm going to talk to Jan about some of these points if he's in the pub next time, he did after all write some of the 97 act & he's a good laugh, he's from Alabama originally, his name was supposed to be Ian but the vicar read it wrong & they kept it as Jan
just to add my pennies worth i was told that i couldnt wear a webley revolver unless i belonged to a reenacters club by a few shows that i emailed also i have replica 1851 naval colt which is deactivated but it is classed as a cap gun because of the little caps used to activate the powder funny old laws 
email (option): roger.beck@node6.com
I think the upshot of all this is that it's probably best not to go down the gun route at all? Just out of interest, I've been having a look at the prices of deactivated guns at some dealers on line and I think the cost is probably reason enough not to get one! 
email (option): rickandjo@pullen14.freeserve.co.uk
Obsolete calibre firearms, are bullet sizes that are not commercially available, so therefore no ammunition can be bought. There is a list of calibre that this applies to.
Muzzle loading firearms come under the antique section and are legal to hold without a license as long as it is an original. I don't know off hand what the cut off year is, but if you have a newly make muzzle loading firearm, it will have to be on your license.
You can make ammunition for obsolete calibre firearms and if you have 1 round of ammunition, it becomes an illegal firearm unless put on a license.
I shoot a Martini Henry breech-loading rifle which is an obsolete calibre, so it has to go on my license. I've done well with it in competitions at Bisley, winning both the "Mad minute" trophies at the 2 meetings that year. I fired 14 rounds in a minute, all hitting a target at 200 yards.
It is of course a BSA (Only cus Norton never made guns) made in 1889 by BSA&M Co
Does that make it the oldest BSA here
email (option): horror@blueyonder.co.uk
It appears that the person that told you that must have thought it was a replica, not deactivated, to own a blank firer or replica gun you must be a member of an appropriate club such as the MVT as these are covered by the VCR act, or apparently it must be painted red (I think it was red)To own a deactivated gun it must only be covered in public which as everyone has said is just "common sense" so as not to cause alarm and make sure that you keep the certificate handy just to prove it as such if stopped.
email (option): davmax@ntlworld.com
I have a deactivated Enfield revolver, but since I'm in Canada, the rules are likely different, though just as confusing.
I can't buy a plastic replica of the Webley or Enfield, as that is classified as a "prohibited device". Nor can I, without a Restricted Possesion and Acquisition License, purchase or own an active Enfield or Webley revolver. HOWEVER, I can buy without restrictions or license, a deactivated revolver. From a foot away, it looks as active and dangerous as it's previous life before deactivation. Yes, it came with a certificate to declare it as deactivated. I can buy a Bren, Sten or any other deactivated machine gun without restriction too, but those are only able to be owned in their active form, by somebody with a grandfathered version of the Firearms Acquisition license. Couldn't buy any automatic active machine gun now in any case even if I wanted one, unless it has been irreversibly modified to semi-auto status- in which case it is classed the same as a lee-Enfield rifle, non-restricted! (but you still need a Possesion and acquistion license to own one) .
But as one guy told me, "Start waving that around in public and it will get you shot by the cops just like a real gun would!
email (option): vinver#ns.sympatico.ca
Regulations for black powder guns are less stringent in the UK..but you do need an 'explosives license' to have black powder!..Ian
email (option): ian@wright52.plus.com
The .38 Enfields I was looking at were welded solid and seemed very expensive and it's seems like even more if you can only look at it in your own living room, with the lights off and the curtains drawn!
email (option): rickandjo@pullen14.freeserve.co.uk