Day 26, data moving challenge

Random jottings to try to re-factor the thinking process on a complex subject.

How to migrate data from one environment to another while also managing to:

• Maintain equivalent security of access control.
• Resolve previous bad practice in currently configured access control where this has occurred.
• Improve the management of access control for admins of those areas.
• Identify files with embedded links to other files.
• Archive data that should be kept - in read-only form, and with access control applied where required.
• Delete data that is no longer needed - for data security management reasons, and to ease management.
• Provide a template, flatter, structure, with role (group) based access rather than explicitly by user.
• Keep the template simple, without forcing a regimented and inflexible system on people.
• Ensure the new system is manageable, scalable, and usable.
• Provide a simpler method for accessing files that don’t require multiple drive mappings.
• Identify files/folders that may have system and automated processes associated with them.
• Ensure the performance characteristics are improved, or at least equivalent of current.
• Catch and resolve further problems and conflicts as discovered.

The number of Terabytes of data is in double figures.  There are many many millions of files ranging from small, to ordinary, and to large.

There are more than 160 defined areas containing data, each of these managed by one, two, or multiple people.  Up to 300 people will need to be fully appraised of what needs to be done, they will also need to discuss this with the people they represent. Some areas will need a great deal more work than others  There will be other external dependencies.