"Open Formats" are a way of saying "Publicly Available Standards". The difference between these formats, especially ODF, and a format like DOCX, for example, is that anyone can easily get access to and understand the data structure of such documents. This means that any company could easily make a program that correctly opens or produces ODF documents.
Document formats directly relate to who can get access to information. South Korea has a company called Hancom that is a Microsoft Office replicate, except with better support for Korean. Although Hancom does support saving documents to DOCX and ODF formats, they also invented their own, called Hangul Word Processor (HWP), that is DOCX modified just enough to not work with MS Word. So what is the problem? Well, HWP can only be opened with the Hancom Word Processor or the viewer. The viewer is free, but available only for Windows. The problem then becomes that you have to use a version of Windows to view the document, and if you want to edit the document, you have to buy a copy of Hancom Word. In other words, if you are running OSX or Linux you cannot communicate. If you have any other office suite installed, you cannot communicate. This means that the only people Koreans can communicate with via HWP is other Koreans. Unfortunately, it is a national standard, which means that they have a huge problem communicating internationally. Best case, foreigners will pirate a copy of Hangul Word Process to view/edit the documents. Worst case, they wont bother opening the document at all.
The UK's move is brilliant for the simple fact that more people can see what they are publishing (regardless of their computer setup), while at the same time potentially saving the Government some money.
Some groups have seen this as a move to boot out Microsoft, but I did not see it that way. They could still use Microsoft products, they are just making it easier for others who don't use Microsoft to actually get access to information. Governments should try to improve communication nationally and internationally. And making documents available in easy to access formats is a step in the right direction.