And so to Hopton-on-Sea. Like many other explorers and pilgrims, I have not had a very good experience of this church. It is never open, there is no keyholder notice, and its aspect is so forbidding that this alone is probably enough to put some people off of trying any further. I do not know anyone who has visited it on a casual basis during the last ten years or so and managed to see inside. Back in 2008, when I had first come this way, it had been on a grey day in early March. As you may imagine, my hopes were not high. Most East Anglian parish churches are open all day, every day, but in the characterless hinterland of Great Yarmouth, with its suburbs pinned against the sea by the traffic-choked A12, they all seem to be kept locked. And Hopton, I have to say, seemed to me a dismal place on this gloomy, drizzly day. I felt my heart sink as we walked up the path to the church, the great padlock already visible on the south doors. St Margaret was built in the 1860s by that great eccentric Samuel Sanders Teulon, one of the most prolific architects of the 19th century. He seems to have ploughed a fairly lonely and idiosyncratic furrow, but at this distance the quasi-Norman touches he put to his buildings lend a certain jollity to an otherwise somewhat austere face. This is his major work in Norfolk, although he was also responsible for Brettenham and the near-rebuilding of Rushford on the other side of the county. St Margaret has a powerful central tower, but transepts which seem to fall away to nothing, as if they had once existed but had been demolished. And perhaps that impression was Teulon's intention.

This new church was built to replace the medieval St Margaret, which burned down in 1865. The old St Margaret was out in the fields, and as so often in Norfolk the opportunity was taken to build a new church nearer to the centre of population. At the time, this was a tiny village of just 250 people, but the rise of the English seaside resort coupled with the industrial expansion of Yarmouth has meant that this parish now has several thousand of residents, not to mention the hundreds of holiday-makers who come to stay at the holiday camps on the seaward side of the parish. It may come as a surprise to learn that Hopton-on-Sea was actually in Suffolk until the border was moved in 1974. Because of this, we need to turn to The Buildings of England: Suffolk to read Pevsner's observation that the glass in the chancel is by William Morris and Burne-Jones, about 1881... beautiful and peaceful after Teulon's architecture.

But would we actually be able to see it? A notice behind the barred grill raised my hopes of a key for a moment, but all it said was Sadly, we have had to lock this porch. The shelter and protection we had hoped it would offer has been abused. It was signed by the Vicar. Well, I don't know, but this notice seemed to tell me more about the Vicar of Hopton than it did about the people. And I wondered who the notice meant when it said we. The PCC? The congregation? The churchwardens? There was something a bit smug about it, as if the people of Hopton had had their chance, but they had blown it. Maybe it is just me being eternally optimistic, but I would like to think that people who live in a place like Hopton would be just as deserving of somewhere in which to experience a spiritual quietness and a sense of the numinous as people living somewhere pretty. Or, indeed, as people living in Ipswich, a much-maligned town, but one where all the town centre churches are open or accessible every day. And the crime rate in Hopton cannot really be that much higher than it is in central Ipswich - can it?

I came back in September 2010, on the occasion of the Historic Churches bike ride. It was towards the end of a reasonably successuful day: I had visited some twenty new churches, mainly in and around Great Yarmouth, and I was now heading south to catch a train back to Ipswich from Lowestoft. I knew that Hopton was taking part, and I was pleased to see a yellow bike ride poster outside of the church gates. I leaned my bike up against a wall, and hurried rather breathlessly up the churchyard path.

Well, the church was locked. I couldn't believe it for a moment. I set off around the building, to see if there was another way in, and then found the nice lady who was signing cyclists in tucked away in the parish room extension on the north side. I mentioned to her that I had found the church locked. "Yes dear", she said. "We didn't want to go to the bother of opening up the church." So, no change there then. However, she was very happy for me to have a look inside, and let me in through the vestry. "Take as long as you like dear", she said.

The most striking impression on entering St Bartholomew is quite how much smaller it feels on the inside than it looks outside. The Transitional-style double lancets do not allow much light inside, and so it is rather gloomy, but this rather suits Teulon's architecture, with his chromatic brick patterns and sombre furnishings. And then you turn to face the chancel, and at once the world comes to life.

The north and south windows contain Edward Burne-Jones' figures of the virtues: Faith, Hope, Charity and Humilty. These are probably among the most familiar of his figures, having been used first at Oxford in 1871. The east window is perhaps less harmonious, depicting Christ's Resurrection above sleeping Roman soldiers as angel musicians look on.

The High Victorian feel of the chancel offsets these works perfectly, with an ornate stone lectern supported by a thoughtful angel, and a pulpit that might have been designed for an Anglo-catholic cathedral in the Colonies. The fabulously ornate organ in the north transept presumably dates from the building of the church. The south transept, reinvented as a Julian chapel, is less successful, not least because of the glass in the three lancets. It would always be difficult for modern glass here not to jar in comparison with the treasures in the chancel, and although there is a quiet simplicity to Paul Quail's work here, it feels like an intrusion.

The nave is really nothing at all in comparison with the chancel, but this is exactly as it should be, and no doubt what Teulon intended. The relatively small marble font matches the lectern and pulpit, and that is about it. There is some truly execrable modern glass in a south side window - what can they have been thinking of? - but I soon found myself wandering up to the chancel again. It occured to me that churches which were kept locked all the time are, according to Church Watch, more likely to be broken into than those which are open during the day, and I wondered if that made these extraordinary windows more vulnerable. The nice lady told me afterwards that they are insured for a million pounds, but of course they could never be replaced.

The best protection against vandalism for an urban church is an open door and a constant flow of people. This doesn't happen overnight. It requires a culture to be built up, a culture of welcome on the part of the church, and a sense of ownership on the part of the people. The great majority of villages and towns throughout England have parish churches which are open every day. It is not just a blessing to the parish, but also a blessing to the church community, who are able, in a very real spiritual sense, to fulfill Christ's Works of Mercy. Or as St Paul said, and he puts it so much better than me, Let brotherly love continue.  Do not neglect to show hospitality to strangers, for thereby some have entertained angels unawares.