How to Submit Data Migration Requests, Communicate Data Securely, and What to Expect During a Data Migration Project
Types of Data that can be Migrated
The Import Tool in Compyle can be used for creating brand new participants or organizations but not updating existing records or adding survey/note records. Who imports the new participant records is up to the customer requesting the bid.
- Data that can be migrated
-
New Participant records or Organization records
- Updates to existing Participant/ Organization records
- Survey forms and survey form instances
- Note type forms and note type form instances
- Group assignments
-
- Exceptions - data that can't be migrated
- Attached files
- Standard notes not entered in a custom note type form
- Record edit history and other system logs
- Participant photos
Migration Project Overview and Phases
Data migration is a professional software service that Clear Impact offers at our current standard rate for professional services. It includes project management as well as developer effort to successfully migrate the data, and the project manager will work directly with the customer technical contact to throughout the data migration process. It is essential that there is a designated person on the customer's team who can answer questions and make decisions about data adjustments and migration timing.
Scope of Work
With a data migration project, we start by determining the scope of work needed for the project, which typically takes five to ten business days from the request for a bid.
This includes estimating the time for any combination of the following services:
- Building the forms to migrate data into
- Editing/cleaning up the data for consistency with Compyle forms
- Migrating participant records via the Participant Import tool
- Connecting survey/note records to Compyle participant records
- Mapping migration data to Compyle form fields
- Migrating the data
Initial Testing
As part of the scope of work, we may conduct initial proof of concept testing. If so, we will we document the steps to create and connect each of the different kinds of records. This includes data transformations and formulas to connect related data sets. Our goal is to increase efficiency and automate wherever possible so that later phases of the migration will go more quickly.
Migration Mapping, Form Build, & Testing
As part of the migration project, a data crosswalk, or mapping document, will be created to identify the source data and where it will go in Compyle. If form building is part of the project, the forms will need to be built with an awareness of the data that they will receive. Additional data transformations and formulas may be needed to adapt the data to the new forms if those forms differ from the original.
Prior to migrating the data to the live site, there will be a test migration to a testing environment which will include a copy of the customer's site. Once the initial migration test data is in place on the test server, we will review the records to make sure that survey instances, survey responses, and analytics were generated correctly with the right data for the right people and timeframes.
Final Migration Data
After the test migration is verified, we will confirm the live data migration data and begin the work of migrating the full data to the live site using the steps and transformations documented in the testing phase. This includes testing and verifying the data on the live site internally.
The migrated survey and note records will be reviewed:
- record counts will be verified against migration row counts
- a set of records will be reviewed to confirm that data went into the correct fields for the appropriate survey instance and participant/organization
- Compylations will be created to verify analytics data entries
Submitting a Data Migration Request
Contact support@clearimpact.com to request a data migration bid and start the process. If you are working a Clear Impact consultant or implementation specialist, you are welcome to contact them instead of the helpdesk.
Sharing Data Securely - Adding the Data Migration template in Scorecard
To safely and securely communicate sample data and the actual data to migrate, we use a Scorecard template in your Scorecard instance.
Steps:
1. Open Scorecard.
2. In the Tools menu, select Templates
3. When the Quick Start Template Library opens, search for "migration."
4. Select the Clear Impact Data Migration template.
5. Enter a Tag for the new Scorecard Objects, if desired.
6. Click the Install Selected Scorecards button.
Migration Complexity & Scope of Work Estimates
In order to migrate the survey forms and note forms for tracking participant activity, engagement, attendance, services, assessments, touchpoints, etc in Compyle, we have to create the data collection structure records, other "records about records", and analytics records that Compyle typically generates as information is gathered through the survey/note workflow.
Without these additional system entries about the migrated data, those records will not be accessible in the client folders or reportable in Compyle's compylations. For example, all notes and surveys have to be linked to a point-in-time data collection instance for reporting by time frame.
There are multiple layers to the information about surveys/notes, and they have to be created in a particular order because they build on one another and relate to each other. As we have added functionality to Compyle, the underlying structure has grown as well, and there are newer areas of the system that we have to account for now.