Archiving Over 2000 Website Per Year
The UKGWA service archives each and every UK Government Website on a pre-defined schedule. All the archives are stored in a multi region storage architecture. The archive are made
Archive Hundreds of Social Media Accounts
The UK Government has chosen Social Media as its primary communications channel, the Social Media accounts for all Government departments is captured in near real-time to ensure an accurate record is kept.
Big Data & Full Text Search
The current data set in excess of 150TB and billions of documents and is growing all the time. Our ability to index such huge volumes of data at speed enables MirrorWeb to deliver faceted full text search across the whole UK Government Web Archive.
The MirrorWeb UKGWA Service is;
Easy to Use
Intuitive front end website and intelligently designed management portal.
We give TNA a 99.9999999999% guarantee on all data stored with us.
By using AWS Route 53 & AWS Cloudfront we deliver a lightning fast service.
MirrorWeb ISO9001 & ISO 27001 Certified so TNA know they are getting the best.
All data is replicated across multiple availability zones across multiple regions and multi factor auth is standard on everything.
Great Customer Support
24/7/365 UK based support to ensure that any problems are dealt with as soon as they are reported.
The National Archives (TNA) is a non-ministerial government department. Its parent department is the Department for Culture, Media and Sport of the United Kingdom. It is the offical archive of the UK government and for England and Wales; and "guardian of some of the nation's most iconic documents, dating back more than 1,000 years."
TNA is the official archive and publisher for the UK government, and for England and Wales. Its role is to collect and secure the future of the government record, both digital and physical, to preserve it for generations to come, and to make it as accessible and available as possible. It is based in Kew, South West London. More information on TNA can be found at: www.nationalarchives.gov.uk
The UK government’s use of the web to engage with the public and to disseminate information has expanded at an enormous rate in the last decade. It is now probably the most common medium through which the citizen interacts with the state. By archiving government websites, TNA ensures that, through the UKGWA, a vital part of the open digital record is captured and preserved. By doing this, the UKGWA has become a core Service in addressing the challenges posed by the increasing number of digital records.
The UKGWA captures the record itself, the provenance of the record, the original context in which it was is published and, to the best of its ability, replays the record through the original playback mechanism associated with it.
Even though the well-known search engines can index, and have indexed, portions of UKGWA content, we know that it is very incomplete. The UKGWA therefore needs its own search service. Since its initial development in 2009, our search has been serving thousands of users per month. As the size and complexity of the UKGWA has grown, so have the expectations of our users who demand a reliable, comprehensive and intuitive search service. We must be able to deliver this and realise the potential of the UKGWA as a resource.
In the 13 years since it was founded, the UKGWA has grown to over 100TB in size, consisting of about 4 billion objects. It has a large and varied user base accounting for approximately 1.5 million visits per month.
User needs analysis has shown that, while the UKGWA is heavily used and well thought of, the Service must develop in a number of respects to remain fit for purpose. It is essential that UKGWA can adapt to capture the varied government record, while at the same time provide innovative and efficient tools to interrogate and reuse the collection. This means that the Service must give access to each resource at the individual level, but also provide access to the data in aggregate, opening new use case opportunities for the resource.
At the heart of the Service is the need to provide continued access to our existing collection, while at the same time providing excellent capture and replay solutions that can cope with changing web technologies. It is recognised that this is an enormous challenge and so, in order to address its challenges, potential suppliers must be able to innovate as well as exploit developments in the field of web archiving.
As a technology partner MirrorWeb has an excellent relationship with AWS and deploys a distributed server infrastructure across multiple availability zones. The AWS compute technology gives us unparalleled technological capability and ensures we can archive and reply hundreds of websites at anyone time. We use AWS S3 to store all data and provide a 99.999999999% SLA. We encrypt all data in transit and at rest and maintain multiple copies in disparate geographical locations. Using Elasticsearch clustering technology gives our clients a powerful search tool that returns enriched results across all datasets that are then filtered using faceted search.
As a cutting edge digital archiving and big data technology company based in the Manchester we have attempted to create a solution that the TNA will find exceeds the technical requirement and thinks outside the standard management and delivery model. We have designed the architecture to create a scalable environment that will grow with the UK Government's digital strategy.
While we use hybrid and pure cloud service providers to give us unparalleled compute power we believe that the UK Government archived data should be stored in UK mainland facilities controlled by UK companies. With this in mind our technology, hardware and network partner provides UK mainland facilities that are CESG approved and connected to the PSN.
To enable MirrorWeb to deliver the level of service that the TNA requires for the UKGWA we have formed several strategic partnerships. Each of the partners has been chosen due to their expertise and experience of the technologies that will be used to deliver the UKGWA service.