7 steps for a successful file migration project
File data migrations do not always rhyme with peace of mind...
Demanding users, complex environments, productivity downtime, data loss risks, rising budgets… Migration projects multiply the challenges and today it's often not enough to be a data storage expert. A successful migration strategy requires a powerful and dedicated migration tool and a well-prepared and executed plan.
And any delay to deploying new file storage/NAS servers means the volume of files to migrate just keeps on growing. It can become a vicious circle that nails your whole project to the starting blocks.
Atempo teams stand by your side before and during the migration processes meaning you can switch to a new storage or consolidate existing storages.
Here are our 7 steps for a successful file migration:
***
1 - Identify internal workflow needs
Technical challenges should not mask the main objective for the file migration: answering the needs of the end users.
What's more, getting your professional teams on board guarantees that storage best practices are respected and evolve.
The storage manager translates each business workflow into a list of technical requirements. This means teams can understand the type of storage or NAS migration that is needed:
- a single massive import migration performed as a one shot job;
- a long-term, large-scale synchronized migration;
- or a consolidation operation for data stored on several different storage devices.
Deliverable: A document which lays out all requirements drawn up with IT teams and end users. The resulting spec will be the reference document for the final migration delivery.
***
2 - Map out the migration project
The IT teams keep working on the ins and outs of the project and in particular:
- the actual nature of the data. Migrations of unstructured data often involve text and multimedia file, search results, IoT logs… each with their specific challenges based on their size, number, organization and ownership.
- the volume of data to migrate. Increasingly migration projects involve hundreds of terabytes or even petabytes of data.
- and the complexity of the IT environment: heterogeneous storage silos, file formats and network protocols; the need to transition to a new storage vendor...
The rigor required to succeed this stage of preparation will largely define the success or failure of the migration project.
Deliverable: An in-depth evaluation of project risks and a list of migration requirements defined by the organization (data governance, targeted norms, cartography of current and future data end users…).
***
3 - Prepare data for migration
It might sound obvious but this must be done prior to the actual start of the migration! Whether it's a full migration or a data consolidation, now is the time to clean up those file systems. What does this entail?
- analyze the storage structure of the source data: number of files per directory, links between files, … ;
- correct the storage architecture if necessary;
- eliminate identical copies and legacy defects - it's never a good idea to transfer redundant or harmful data;
- etablish file format and naming conventions;
- decide the migration ground rules and priorities to guarantee that data be transferred to the right location at the right time and that the data is protected and secure during the migration;
It's also the moment to schedule the migration: which period is best for the organization, import frequencies, times of day to run the migration... For this it's a good idea to involve the end users and aim to limit the impact on their activities. Finally, we should automate a maximum number of operations, to limit the risk of error.
Deliverable: Data sources ready to be migrated, a spreadsheet listing all source and target destinations, a priority schedule to migrate directory groups and files, and a document targeting end-users referencing file naming and storage address conventions.
***
4 - Choose the right tool for complex data migration projects
The software available for migration projects is exploding right now. But most tools simply do not have the capacity to manipulate several petabytes and billions of files between complex and heterogeneous storages. Other solutions are simply too expensive and only suitable for very large corporations. 
Atempo's Miria for Migration places migration between storages within reach of all organizations and all IT environments.
- Miria adapts to many file systems and preserves ACLs (Access Control Lists).
- Miria can handle several petabytes of data in complete security.
- Dell / EMC Isilon / ECS, Qumulo, Lustre, DDN GRIDScaler, IBM Spectrum Storage, Panasas, cloud storages (S3, AWS, Azur), object storages. Miria is fully vendor agnostic and enables genuine and successful migrations between different storage platforms.
- Mira automates migration processes along with the generation of pertinent reports throughout the entire operation.
- Thanks to the deployment of on-demand Miria Data Movers, the solution ensures rapid, evolutive, and multi-storage migration.
Miria for Migration ticks the boxes regarding the preparation phase, rapidity of execution and reliability for the migrations of large volumes of data in complex environments.
Deliverable: Atempo experts handle complete end-to-end migration project steps.
***
5 - Perform a data migration pilot test
Don't take our word for it: the test bed is part and parcel of the migration procedure recommended by Atempo. Data management experts define the perimeter within which a real time migration runs.
This is also the ideal moment to perform security backups and other risk control measures, to eliminate any data loss risks. Atempo has a range of additional backup tools for Miria for Migration that are fully adapted to large file volumes.
Deliverable: Validation of the chosen solution and preparation of the actual migration.
***
6 - Schedule your file migration project
Most migration solutions are unable to reduce the time required to undertake the complete migration task. In addition, the bandwidth used for end user storage needs is often reduced during migration with a genuine production impact. Managing this simultaneous migration and impact on the professional activity can multiply risks of downtime and data loss.
Miria for Migration adopts an innovative approach. It works with incremental cycles:
- Migration runs during set times only. The use of the bandwidth can be limited for example during working hours and pushed to maximum or near maximum in other periods (Miria is quite capable of saturating a 10 GB network for example). File transfers are accelerated in line with production constraints.
- Files on the source storage remain accessible for analysis or modification during the complete migration duration.
- At each automatic migration cycle, Miria detects files that have been modified on the source storage. A cycle of incremental synchronizations with the target storage runs until the two storages converge.
The automation and data flow management enable the cutover from old to new storage in just a few hours, sometimes even in minutes following the convergence.
Deliverable: Storage and NAS migrations with no impact on internal productivity.
***
7 - Ensure the success of data migration
No file migration project is fully over until there has been an in-depth analysis of the target storage. This means:
- a complete inspection of the target storage by a storage manager and the migration consultants;
- a series of tests of the migrated data, using the test scenarios defined in advance of this project (tree structures, ACLs, metadata…);
- and an evaluation of end users satisfaction, using the specification document defined on the launch of the migration project.
Miria for Migration generated a set of automatic reports during the entire migration process. Evaluating the ultimate success of each project just got a whole lot simpler!
Deliverable: An operational storage destination, a validated data migration job within time and budget constraints = happy end users!
***
Learn more about Miria ? Attend an upcoming webinar:
***
Read more:
