Hopefully incorporating lessons learned from last week's catastrophic ViLoR failure!
I wondered if that was where the problem occurred. Do we know why it went wrong? I thought each station had a studio back to Birmingham and another to London to avoid this sort of issue.
A BT engineer damaged fibres in Oxford in just the right way to cause a fault condition that had not been predicted and mitigated. There was a further issue which took out the fallback of routing the analogue side of the studio directly to air.
In a ViLoR studio there are two mixers, one is for local sources and one which is a control surface for the remote playout and Outside Sources.
It seems a variety of lash ups were put into play to get something on air, generally involving ISDN. There were stories of CDs from people's cars being used at some stations.
Reading between the lines, I think some stations may have been able to get the analogue half of one of their ViLoR studios on air, others apparently had to use a different studio entirely. I wonder if they might have lashed up some OB kit, perhaps in their NCA studio where there is an ISDN line? Or if they have a more conventional production studio?
It may be one of those Omnishambles where a freak combination of individual issues leads to a big screw up - there was enough resilience built into the system to cope with several failures at once but a specific combination of issues made the whole lot fall over.
It may be one of those Omnishambles where a freak combination of individual issues leads to a big screw up - there was enough resilience built into the system to cope with several failures at once but a specific combination of issues made the whole lot fall over.
Ah an lucky series if circumstances. You can only build so much resilience.
To add to the fun, they had another issue on the day - it appears ENPS is cloud hosted too.
Think that depends on your definition of cloud. It could have changed since I was last involved in it - but ENPS was based around multiple servers around the UK, with individual programmes and stations hosted on a specific server, though that server may not have been sited locally. The backups for each area were hosted on a different server. There were also 'reflectors' that ensured running order updates from a specific server were received by machines actively working to that area, but not to areas 'just looking' at another server's content. (i.e. if you were working on Points West, your running orders were on a BBC West server, along with those of the local radio stations in that patch, and your PC looked at a BBC West reflector so you got instant running order updates. If you were working at Look East, you could open running orders on the BBC West server, and look at the Points West running order, but you wouldn't necessarily see updates happen instantly)
You could call this 'cloud' hosting - and it may be the servers are now off-site rather than on BBC premises, and may be 'virtual' running on commercially hosted VMs (though ENPS has a lot of information security stuff around it - so I'm not sure that is the case)
Dare I mention the sustained use of a full 6:30 opt for Look East West viewers is becoming increasingly regular? Tonight was a very separate programme as it focussed on the collapse of Monarch but last week was much closer to a STV style arrangement where stories were pooled in the second half of the programme, and reporters featured on both opts, albeit at different times and responding to slightly different questions.
On another note, the Look East Facebook page has rebranded to BBC News East of England, something that's been seen in a number of other regions too - they keep a separate Twitter though, which does look a bit strange on the surface, but is presumably as the counties have their own feeds as part of BBC local radio stations. The Facebook URL features on the title card of Look East and whilst they've updated it on some, the old address still regularly pops up.
What are people's opinions on the current sofa presentation of Look East? It seems a bit of a hash to me. Surely after nearly 10 years they are due a complete revamp.
Also, it must be a big anniversary coming up for Stewart and Susie 20 years as the main duo.
What are people's opinions on the current sofa presentation of Look East? It seems a bit of a hash to me. Surely after nearly 10 years they are due a complete revamp.
Also, it must be a big anniversary coming up for Stewart and Susie 20 years as the main duo.