Well that will be the source of the water which is getting into the oil.
It does not take much to make the oil milky
Get a nice big bottle of the cheapest oil you can find then go for a nice long ride & change the 90ml of oil when you get home .
Repeat as often as necessary till the oil comes out clean, horrid thing to have to do on a nice sunny weekend .
Better still would be a new felt oiled with light machine oil before installed to prevent it adsorbing water
Apparently fabric treatments like ScotchGuard or even a water repellant like WD40 does a good job of keeping the water out of the felt.

Thanks for these ideas Trevor. It isn't all together yet, I can get some WD40 up there behind the clutch. Might even take the clutch off (again)to get a better shot. Wifey currently ransacking the laundry cupboard for Scotchguardy stuff.
It's meant to rain for a while here (about bloody time even if it does cut down the riding) so I've got the opportunity to do it all.