Notes to SharePoint Migration
On this page we explain how we migrate content of an IBM Notes application to a new (non Domino) application. As an example we replace a Notes-based DMS with a SharePoint Online substitute from Portalsystems. This is based on our proven Lotus Notes to SharePoint migration approach. The process of Lotus Notes Migration to an alternative application is complex. This is because you need to carefully map the Notes content with the fields in the target application since the way an application stores and maintains information is never similar. This mapping task is very time consuming and a precision job in order to make the content migration 100% accurate and error free.
IBM Notes application migration to SharePoint
Mapping the Notes database content with the new application
Once you know witch application is going to replace the Notes application you must migrate the content from the Notes database to the target application. You need to map the Notes content in such a way that it lands exactly at the right spot in the target application. The crucial step is to design the mapping. The migration tool that does the actual migration is not yet important at this stage.
Below is an example of a mapping table that we use. This simple tool suffices since it tells us exactly what to do with the Lotus Notes data.
The way the data is stored in the Notes application likely differs completely to the target application which is obviously not IBM Notes/Domino. For example; the Notes database might use response documents to store certain content which are not present in the target application. Also, the latter might store the files in a dedicated list while Notes stores files in Notes rich text fields in the main documents.
Adjusting Notes database content prior to the migration
To make the Notes content suitable for migration often you need to alter certain metadata values stored in the Notes documents. In our projects we provide our customers an easy way to adjust certain data fields in the Notes application prior to the migration. The migration tool will then use the added field values in the Notes documents and migrate them. The advantage of this approach is that the person doing the actual content migration does not have to worry about the details since they have all been taken care of.
To facilitate this approach we add several views to the source Notes database (we call them ‘Migration Views’) that contain buttons to manipulate these specific field values for multiple documents at once. Our client can set new (including empty) values for selected fields, and even retract a newly set field value if a mistake has been made. Below image provides an example (highlighted in green).
In the example below, we add the possibility to update a Sub Sub Process.
Please note that when we add a new migration field (in green), we don’t touch the original field value in the Notes documents. We will configure the migration job in such a way that it uses the green field values instead of the original field value.
We also add a button to select batches of documents that are ‘ready for migration’ to enable a step-wise migration.
The successive views actually form a small workflow: Select Notes documents for migration -> Modify fields -> Migrate -> Validate -> Archive migrated documents -> Completed.
The correct functioning of these views requires a lot of consultation with our customer. However, experience has shown that it is worth the time investment to implement these changes up front in the (Notes) source system rather than after the migration in the target system. It is certainly not mandatory to use the migration views, we will jointly determine with our customer if it adds value.
We fix the last modification date/user value prior to the document modification in these views.
Migration environment
In the example on this page we demonstrate a Lotus Notes migration to SharePoint. The following elements need to be taken into account:
- The Domino/Notes environment (Source)
- The SharePoint environment (Target)
- Migration views to modify values prior to migration
- The Quest Migrator for Notes to SharePoint software (‘Quest MNSP’) to move the Notes data to SharePoint
- The SQL Server software to resolve the Notes document link targets on SharePoint
Notes/Domino environment
You can migrate data from server-based databases as well as local Notes databases. From a performance perspective we advice to migrate data from a local replica or copy. However, under certain circumstances this is not an option. For instance, when a database stores file attachments in DAOS.
When the Lotus Notes to SharePoint migration takes place from a local source, we replicate the current server-based database to the local migration machine. After that, replication for the local database is disabled. All these tasks are done in this local database.
SharePoint environment
The standard rule is to set-up a separate SharePoint Site for each Notes database. Up front you need to determine if the Notes database is suitable for migration to the specified SharePoint Site or if adjustments are necessary. SharePoint specialists do the configuration and provisioning of the SharePoint environment in consultation with our customer.
Migration Views
In the Lotus to SharePoint migration preparation, you need to make decisions, for example on what fields you want to migrate. Once this is clear our customer prepares an Excel table containing the mappings between the Notes fields and the available SharePoint columns. Also you may add comments to each mapping. For instance, if there are new SharePoint columns that do not exist in the Notes database. Also, if there are columns that should have a default value or a value different from the mapped Notes field, etc.
Quest Migrator for Notes to SharePoint
Lialis has vast experience with several Lotus Notes to SharePoint migration tools. We prefer using the Quest Lotus Notes Migration tool. Depending on the number of document types that you migrate, we create one or more migration jobs within the Quest Migrator. We test these jobs on a few Notes documents, taking into account all exceptions, modifications, rules and agreements as described above.
Below you find an example of such a migration job. First you define the source data. Notice that the value for Process is initially looked up from the field set with the Migration Views action. If this field is not available (i.e. if the Process value wasn’t modified in the Migration Views), the software uses the value from the original field as source value.
If the source data definition is ready you map them to the target columns that you import into the Quest Notes Migrator from the SharePoint site:
For each mapping, we have to check whether the source data type is compatible with the target data type. For instance, you cannot map a source datetime field to a SharePoint Text column. Additional manual operations may be needed. Such as, actions to fit a multi-value field into a single line text field and vice versa, set default values, translate values, etc.
If test migration results are not yet perfect, we’ll continue to make adjustments until the output is exactly in accordance with the specifications of our customer. If that is the case, we will set the Notes production database to read-only. Subsequently, we will perform a full Notes to SharePoint migration of the database.
Before usage we will configure the Quest Notes Migrator for SharePoint software in consultation with our customer. Configurable items are: User Mapping, Blocked Files and Link Tracking.
User Mapping
The Quest migrator provides a method to map users with Azure/SharePoint accounts. This is essential to populate Names columns in SharePoint with SharePoint users instead of Notes users. For instance, the Created/Modified columns. In the picture above, the setting ‘Preserve Created/Modified identities’ will populate these columns with the correct SharePoint identities. It will only do this when you configure the user mapping correctly. We need to do a profound test to make sure that the mapping method includes all users. We can use a Domino Directory or a text list with internet addresses as a source to find the corresponding SharePoint login. Often, you can only make a SharePoint user mapping on the basis of the Notes Common Name (as you can see in below example).
↓ 
When you cannot map a Notes user account to a SharePoint user, the migrated value may be empty. This may be a user that has created/updated some Notes documents and has left the company. Generally, an empty user field in SharePoint will not be a problem. However, if it concerns mandatory user fields, for example the Modifier field, you must enter at least one name. For these cases we reserve a so-called ‘Default’ user. This user will substitute the non-existing user. You must register the default user in SharePoint. A commonly used name for this account is ‘Migration Account’. This default user is to ensure that every (mandatory) SharePoint Names column has at least a value.
The downside of substituting non-existent users with a default user is that you lose history (for example, the Last Modifier information). However, it is possible to still record these non-existent users in the target system. However, it will be in the form of a text field containing the Notes names of these users.
Blocked Files
Another aspect we need to take into account in the configuration is file size and extensions. SharePoint has some limitations. By default only files of 50 MB or less are migrated, and many file extensions are blocked.
Generally, we tend to clear all the extensions from the block list (i.e. migrate any type of file) and also clear the maximum file size. The migration computer hardware (memory) will determine whether the software migrates a large file or not. Error messages relate in most cases to memory issues due to extremely large files. There is always the option to manually move these files to SharePoint. This is workable as long as there are not too many.
Link Tracking
Notes documents may contain document links that point to other Notes documents in the same or in a different database. When we migrate documents with links, by default these links are not updated. They will still point to other Notes documents. This may be rather pointless when the Domino system is shut down. There are alternative options for dealing with these links, as you can see in the image below.
Using the Link Tracking Service is the most advisable option. This service ensures that you convert each original Notes link target to the new target location in SharePoint. If the linked targeted item does not come across in the migration the link will end up being a ‘dead’ link. The Quest Lotus Notes to Office 365 migration software also handles dead links.
To enable Link Tracking, we must install a SQL server (Express will do). We can do this on the migration workstation itself.
SQL Server
In principle we only need a SQL Server when you enable the Link Tracking Service in the Quest migrator. The software allows setting up a Link Tracking database in SQL. As a result you register every Notes document that we migrate in this database.
↓
However, we recommend to always set up a Link Tracking database in SQL. Although you may not find links in the first few databases you are migrating, you may find them in subsequent databases at a later stage in the project. If a database contains links to a migrated database in an earlier stage, and you forgot to register these documents in the SQL database, it is not possible to resolve these links to the new target location in SharePoint.
When all Notes databases have been migrated, we run the Link Finalization script from Quest. This process checks in the SQL database if the linked target documents are migrated, and replaces the link target with the new SharePoint location. If the software can’t find them, there is an option to replace them with an alternate URL (e.g. to a dead-link page). An alternative is to simply replace them with a direct link to Notes. However, in that approach these links will not work in most SharePoint targets.
Only until all databases are migrated and the migration project is complete, you can remove SQL (and the document link tracking database) from the workstation.
Usually, a direct value migration from a Notes field to a SharePoint column is sufficient. If you need to replace values for a specific field with another value, we can use the mapping table. The same applies to fields that are not present in the source. However, there may be new field values that you cannot apply easily. For example when the target system uses a different category structure.
Conclusion
A Notes to SharePoint migration is a complex process. However, our decommissioning application approach which is a result tens of Notes application migrations to SharePoint over the past decade, will make the task less daunting. We offer several solutions to avoid the common migration pitfalls. Surely, a solid preparation is key. Our migration process, checklists and templates will help you prepare your project. These highly increase the chances of a migration result that will make your end users happy. So, in case you have a need for a IBM Lotus Notes to SharePoint migration, our migration specialists are here to assist.