There are some churches that you always remember. This may be for a variety of reasons: a historical treasure perhaps, or simply the feel of the place (what my friend Phil calls the wow factor), or the setting, or a confrontation you had with the keyholder, or any number of other reasons. Ingoldisthorpe, pronounced in-g'l-thorp, I will always remember simply for being beautiful. At first, things weren't promising. The long Norfolk coastline, more than eighty miles long, is home to hundreds and hundreds of open churches, for miles inland. Very few are kept locked, and even then there are keyholders (I can only think of three churches within a few miles of the Norfolk coast that are locked without keyholders). But here, not far from the busy resorts of Heacham and Hunstanton, we found St Michael, which in itself is no mean feat, with its doors barred.

There was a keyholder notice, but I was already pretty disorientated by finding the church. The suburbs of Ingoldisthorpe twist and turn in a convoluted manner, and the church is quite hidden from the road. We found a sign, but it seemed to lead into someone's private drive. I suspect that the church was once in open fields, but that these were sold off for housing development. Now, the narrow churchyard is surrounded on all four sides by private gardens. In fact, the keyholder is not far off, but be aware that in early June the gardens were already overgrown enough to obscure the sign that bore the name of the house.

But, in all honesty, few Norfolk church keys are more worth getting than this one, for St Michael contains what is probably the best collection of late 19th and early 20th century glass of any small church in East Anglia. It is like stepping inside a jewel. And a living jewel at that; for some unaccountable reason, the revision of Pevsner describes St Michael as redundant. Well, it certainly isn't, and there is a real sense of a working church.

Before you get to the glass, the font is a curiosity - it was once a square Norman font, but somebody, probably in the 13th century, thought that it would look better octagonal. It doesn't, I think, but that is what we are left with. The pretty screen is also a fine medieval survival. Under the tower is a set of three brasses of Agnes Bigge and her parents, in all their puritan glory in the early years of the 17th century. If I had been that ugly then I do not think I would have wanted a brass made of me, but there they are, warts and all.

St Michael was almost entirely rebuilt in the late 14th century, when Dec was becoming Perp, and is a harmonious textbook example of this moment. And then, in 1855, George Pritchett came along. He was commissioned by the Rector, William Beckett, to refurbish the church, and this was done very well. It provides the setting for the jewels. Incidentally, Pritchett also rebuilt the neighbouring Rectory for the fabulous cost of £2,000, about half a million in todays money. In the 1850s, many churches were built for less.

The bulk of the glass was commissioned from the O'Connors, and no expense was spared. This was augmented over the following seven or so decades. Frederick Preedy did the fine south aisle east window, which depicts Christ walking on the water, the Ascension, and the raising of Lazarus, the last of these is rather chilling. Also in this aisle is work by Robert Bayne, featuring William Morris's daughter May as a model.

All of it is of the highest quality. Apart from the work of the O'Connors, the most interesting moments are to be found in the chancel and the north aisle. The chancel north window depicts, in its upper lights, scenes from the life of Thomas Beckett, son of the Rector. He emigrated to New Brunswick, and died there in 1863. We see him in a photographic style sitting at his desk, and lying with his gun in the woods. Another image shows the ships in Saint John harbour.

In the north aisle is a superb composition of 1934 by Christopher Webb. It depicts Faith and Hope in one window, and to the left is Charity. Charity forms the largest of the three figures. She protects two children, including a girl with the bobbed haircut of the time. The child holds a spray of may blossom, a sign of rebirth.

I thought how poignant this image would have been in the mid 1930s, at the height of an industrial and agricultural depression, with the rise of Fascism in Europe, a time of strikes and hunger marches. I think this window is marvellous. You can see images of it, and many others, below.

That Ingoldisthorpe was fully in the Anglo-catholic tradition can be deduced from the altar and reredos. Even if the tradition has gone, these survive. St Michael is a perfect example of the Church of England at the height of its powers, at the summit of its artistic integrity, and should not be missed.