NDDM Technical Manual

To achieve the best scanning result in the shortest time frame, we advise you to carry out the below listed scan sequence.
Each step in the table below is explained in detail in this post.

Except for ‘readers/authors’ scan, all scan tasks are carried out by the Domino server. Scanning may also be done by the Notes client at much lower scan speeds though.

Supported Notes and Domino builds

NDDM runs a scheduled agent on Domino servers to carry out the scan. Therefore NDDM has been tested extensively on the newest versions of all relevant Domino server major releases: R6, R7, R8, R9, R10 and R11; on both Windows and Linux operating systems, 32 and 64 bits servers. We expect lower Domino servers versions will not pose any problems. However, if you run lower Domino server builds (R4, R5), we advise you to contact us so we can give these versions a test run. Please consult the following list with Domino versions/platforms that we tested NDDM on.

Summary of tested configurations:

Windows/32
Release 9.0.1 FP8|February 23, 2017 Build 405
Release 9.0.1 October 14, 2013 Build 405
Release 8.5.3 FP6|November 21, 2013 Build 390
Release 7.0.2 September 26, 2006 Build 265
Release 6.5.5 November 30, 2005 Build 198

Windows/64
Release 11.0 .1 FP2|October 20, 2020 Build 452 *
Release 10.0.1 FP6|September 24, 2020 Build 450 *
Release 9.0.1 FP10 HF747|July 24, 2020 Build 405
Release 8.5.3 FP6 HF3175|April 28, 2017 Build 390

Linux/32
Release 9.0.1 October 14, 2013 Build 405

Linux/64
Release 11.0.1 FP2|October 20, 2020 Build 452 *
Release 11.0 November 25, 2019 Build 451 *
Release 10.0.1 November 29, 2018 Build 450 *
Release 9.0.1 October 14, 2013 Build 405 *

*) notes.ini setting required: LS64BITCCALLOUTPointerSupport=1

The required notes.ini settings for some of the 64-bit servers is a fix that has been applied because of 32/64 bit incompatibility. This setting is necessary for the relevant server to perform a user activity scan on a database. When an argument is passed by reference, the C function receives a 32-bit pointer to the value area. API calls from LotusScript on a 64-bit Domino server will fail (crash) when an 64-bit pointer is expected.

For example: when a variable is a pointer that is passed by reference, this works perfect on a Domino 32-bit server when this variable is defined in LotusScript as a Long. However, the same situation on a 64-bit Domino server – where the variable is defined in LotusScript as a (64-bit) Double – makes the server crash.

In order to be able to return the pointer in double, you need to set environment variable LS64BITCCALLOUTPointerSupport=1

You can set this variable by directly editing the server’s notes.ini, or by issuing a console command on the server:

set config LS64BITCCALLOUTPointerSupport=1

A server restart is not necessary after this notes.ini change. It will take effect immediately.

After adding the Notes.ini setting, the Agent Manager task has to be restarted on the Domino server.

During our extensive testing, we learned which servers this setting should apply to. If a user activity scan is initiated on such a server – and this setting is not found in the server’s notes.ini – this is reported in the scan log on the Task document. The database scan will continue its run, but the user activity scan task is skipped.

Furthermore, if a scheduled User Activity scan task is run from a server that may very well run without problems, but was not tested by us, we don’t allow the agent to run the User Activity scan anyway. Other scan tasks are allowed though. The Task document log will tell you that the User Activity scan is not possible on this server.

This also applies to non-tested 32 bit servers. As mentioned above, we tested the newest versions of all relevant Domino server major releases from R6 up to R11. Only on these Domino servers, the server-based agent is allowed to do a User Activity scan. If one of your servers is not allowed to do the User Activity scan, there are 2 options:

Option 1: run a User Activity scan from the client (background agent). This option is recommended.

Option 2: ask a Notes developer to adjust the script library called ‘slPlatformInfo’ to include your server. When you choose this option, please be VERY careful, and make sure you run a test before you apply this on the production server. You can also contact Lialis to perform this test and adjust the script.

If for whatever reason the agent is interrupted during the scan (for example by an unexpected error, or an automatic Domino server reboot or crash), the Notes database that the agent was processing at the time of the interruption is skipped when the NDDM scan agent starts again. Just in case the scan of this database has caused a server crash, omitting this database in the successive run prevents the server from crashing again. The skipped database is logged:

Domino server performance impact

All scanning tasks are carried out by the Domino server on its own local Notes databases by a NDDM Lotus Script agent. See image below:

Domino server based scan of the Notes database user activity. The impact on the Domino server is zero because the scan is very fast. For example scanning 100.000 Notes databases will take less than 10 minutes (depended on server speed).

Domino Server based scan of the Notes database content. The impact on the Domino server performance is average; not high and not low. The agent will scan each Notes document in all Notes databases on the Domino server running the scan. The scans are very fast because the Domino servers scan their local Notes files. However, a single Domino server scan can take up to a few days depending on the amount of Notes files on the Domino server.

NDDM has been designed in such manner that multiple Domino servers will run their own local scans in a parallel operation, resulting in enormous scan speeds. So, scanning 1 Domino server will take the same amount of time (a day or 2) as scanning 10 or even 100 Domino servers.

The Domino server agent manager must allow long agent run times of at least 6 hours (12 hours is even better). If the Domino agent manager cuts off the NDDM scan agent there is no problem. The NDDM scan agent will pick up the Notes database scan in the new run at the exact moment it was interrupted in the previous run.

A Domino administrator can adjust the NDDM scan agent, simply by adjusting the Agent schedule. For example to set the NDDM agent to only run at night time.

With NDDM we have scanned many Notes databases, from small to extremely large. The database in the example below holds 115,087 Notes documents and the NSF file size is 51 GB. The Notes content scan of this particular NSF file took 30 minutes.

01/06/2021 05:57:09 PM GMT – Start Database Analysis
01/06/2021 05:57:09 PM GMT – Start scanning Design Elements
01/06/2021 05:59:25 PM GMT – Finished scanning Design Elements
01/06/2021 05:59:26 PM GMT – Start scanning content (incl. scan links)
01/06/2021 06:27:51 PM GMT – Finished scanning content (incl. scan links)
01/06/2021 06:27:51 PM GMT – Finished Database Analysis

Here is an example of a small Notes database of 400 MB holding 876,430 Notes documents. The scan took 15 minutes.

01/06/2021 01:23:47 PM GMT – Start Database Analysis
01/06/2021 01:23:47 PM GMT – Start scanning Design Elements
01/06/2021 01:24:46 PM GMT – Finished scanning Design Elements
01/06/2021 01:24:46 PM GMT – Start scanning content (incl. scan links)
01/06/2021 01:38:26 PM GMT – Finished scanning content (incl. scan links)
01/06/2021 01:38:26 PM GMT – Finished Database Analysis

The scan speed depends mostly on the amount of Notes documents and to a lesser extend on the Notes database file size.

The total Notes database content scan speed also depends heavily on the Notes database setup and the OS server performance. We have managed to scan 280 GB in one hour. In this one hour, 4,823,000 Notes documents where scanned by one particular Domino server. If 100 Domino servers carry out the scan, the total scanned GBs and Notes documents can by multiplied by 100.

Setup and Configuration

When you read this technical manual, we assume you have received the NDDM Notes database file from Lialis. It’s not possible to download an evaluation version.

Open the local NDDM Notes database with your Notes client and make a Notes database copy (with your Notes client) to the main IBM Domino (HUB) server that replicates with all other Domino servers. Make sure to include the documents in your copy. Do not copy the ACL.

Open the new NDDM Notes database copy on the first Domino server and continue.

ACL – First adjust the ACL. Make sure all Domino servers have manager access with delete permissions. Also add all administrators and viewers if applicable. You might want to add people to view the content with read permissions, or even set the default access to Reader. Add the signer ID with manager can delete permissions.

Signing – NDDM will run unrestricted scheduled agents on all Domino servers to be scanned. This means you have to sign NDDM with a Notes ID that is allowed to run unrestricted agents on each Domino server. In most environments, the ID of the Notes administrator has these privileges. You can also use a certain Domino server ID file to sign NDDM. But no worries, NDDM will report if the ID does not have enough permissions, and it has the means to adjust it later on in the scan phase.

Configuration

NDDM holds multiple Settings views listed in the left navigator where the NDDM configuration must be set prior to scanning.

Excluded Files and Folders – First exclude all Domino system files and folders from the database import. You may also want to exclude the mail folders. We already made a pre-selection of system files that we are aware of. We do this because it’s a waste of resources to scan large Domino system databases like the log.nsf. It is up to you to add additional folders and files to the exclude list prior to carry out any scanning. Because Linux servers are case-sensitive, the file/folder names entered here are case-sensitive as well.

You may use the button “Add additional files/folders” to paste files and folders to quickly setup the exclusions.

It is not a problem when some system databases were accidentally imported. You can delete these system database documents from the NDDM, and you can then improve the exclusions list after the Domino server import has been completed.

Users and servers to exclude from the user activity scan – Here you must add users and servers to exclude from the Notes database user activity scan. This step is mandatory, because filtering away all Domino servers, signer ID and names of administrators and developers will result in a much more accurate list of activity of real users. Generally, it’s not interesting to know what servers and admins are doing in a Notes database. So add as many names now. Later in this manual you will be guided how to improve this list.

Try to add as many known Domino servers and ‘unreal’ users to the exclude list. See the sample below:

You may use the button “Add additional servers/users” to paste Server and user names from Domino groups to quickly setup the exclusions.

NDDM Search keywords – The strings listed in this view will be used for the design string scan. In the setup phase we leave it for now.

NDDM Servers –  The roll-out part of NDDM allows you to create replicas of the NDDM Notes database to all (or a selection of) Domino servers in the Domino domain that you are planning to scan with NDDM. Your Notes client will create the new replicas in the Notes client background. This may take a short while depending on the amount of Domino servers you are actually going to scan and network bandwidth.

You may also use the Domino Adminp process to create NDDM replica’s.

You need to make sure that all Domino servers you are going to scan are setup with a so-called ‘pull push replication’ to the main or HUB Domino server. Normally this is the case in Domino environments. Check this anyway just to be sure.

If you have Domino cluster servers, you only need to install NDDM on the main Domino cluster server.

When all NDDM replicas are created with NDDM or via Adminp, please press the button Check NDDM Replica to make sure the NDDM file is present on each Domino server.

You may assign Keys to each group of Domino server, for example to indicate global regions where the Domino servers reside in.

NDDM – Send Errors to – If scan errors occur on a database, a button will appear on the Database info document that allows you to send the error message to the specified address.

NDDM – Targets – Lialis NDDM has a set of Target calculation rules that will assist you to calculate the Targets automatically when the scan is completed. Please refer to the section Target analysis on this page for details.

NDDM runs 4 scheduled agents, of which 1 agent need adjustments in the Notes Designer Client prior to running.

You don’t need to make changes to the first 3 agents.

Agent DBDISCOVERYANALYSISONSERVER is the agent that does all of the import and analysis work. This agent needs to be able to perform restricted operations with full administration rights. This will prevent a lot of database access errors. No changes needed.

Agent EXPORTONSERVER creates a CSV export of all databases that are included in the NDDM tool. To enable the agent to save the CSV file on a drive, this agent should be allowed to perform restricted operations. No changes needed.

Agent ProcessMailingQueue sends out email messages to database owners. To make sure the message is only sent once from one server, and to provide a single reply address, this agent must run on one specified server, please adjust the Domino server. Choose 1 Domino server where this agent runs on like the HUB Domino server.

Please also adjust the run on behalf of the reply user:

This is also described in section ‘Batch-wise send out mailings to end users for information’.

Agent RefreshReaders is only to be used when migrating from NDDM Release 410 or older to release 411. This agents adds a reader field to all content to protect it from unauthorized access by end users when using the survey questionnaire features.  To run this agent open the agent in the Notes designer client, change the Run on Domino server to a Domino server holding NDDM and enable the agent. Then wait for the Domino server to start it and let it do the work.

The Domino server log will display a message like this:

09/23/2021 01:36:19 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: Agent RefreshReaders started
09/23/2021 01:36:23 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 8000 documents to process…
09/23/2021 01:36:26 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 7000 documents to process…
09/23/2021 01:36:30 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 6000 documents to process…
09/23/2021 01:36:33 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 5000 documents to process…
09/23/2021 01:36:36 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 4000 documents to process…
09/23/2021 01:36:38 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 3000 documents to process…
09/23/2021 01:36:41 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 2000 documents to process…
09/23/2021 01:36:44 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 1000 documents to process…
09/23/2021 01:36:46 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: 0 documents to process…
09/23/2021 01:36:46 PM AMgr: Agent (‘RefreshReaders’ in ‘Development\NDDM\NDDM_MASTER_4.nsf’) printing: Agent RefreshReaders finished

You may consult the hidden view (ReadersCheck) to see if all documents in NDDM received the reader field updates. The view should show 0 documents.

When the agent is finished please disable the agent, its no longer needed.

The NDDM Notes views are set to NOT refresh automatic. This has been done to speed up the NDDM scan process. Please press F9 in most of the NDDM views when you are working in NDDM!

At this stage all selected Domino servers have a replica of the Lialis NDDM Notes database. You do not need to open all NDDM replicas. NDDM has been designed in such a way that you can initiate all scan work in 1 NDDM Notes database on your main central HUB server. With task documents, each Domino server is instructed to carry out its own scan work. You only need to wait for the results to be replicated back to the central NDDM Notes database.

You might want to check the Domino server agent manager task settings in the Domino server documents. Scanning may take a few hours or days depending on the size. It is recommended to increase the agent manager run time to 12 hours for example. If the scan takes more time, then there is no problem: NDDM will continue the scan at the point where it was interrupted in the previous scan.

You also might want to create new Domino server replication connections documents between the HUB server and all the Domino servers to be scanned, where only the NDDM Notes database is replicated (pull push) in the shortest time interval of 5 minutes for example. This will speed up the work considerably.

Follow the order of steps as described below carefully.

Scanning – Import Notes databases. In this step, the Domino server adds a Database Info document for each database found. To make sure that no .nsf file is omitted, only information that can be retrieved regardless of database access is recorded (server, file path and -name). This task will only take a few seconds to complete.

Select one, multiple or all Domino servers. Use the Key to filter away Domino servers.

The scheduled tasks can be seen in this view:

Now wait until the Domino servers have carried out the import and the import task is completed. Each database on the server now has a Database Info document in NDDM.

A Domino server will run this Notes database import task in a few seconds. The only delay is waiting on the agent manager and the scheduled replication.

Note: if there are multiple new/in progress task documents for the same server, the scan agent will always pick the newest Task document to run on, which is the first displayed in this (unsorted) view. So make sure that you only create a second task document AFTER the server completed its first task.

Ideally, use the ‘Scheduled On Domino’ option for all scan tasks to perform. Domino servers agents process data much faster than client-based agents. However, in certain cases you might want to run a task manually on the client. You can do so by using the ‘Manual On Client’ option in the navigator .

You can always re-run this step. The agent will simply add new Notes databases when they are found.

If you delete all NDDM content you can run this task also to start from scratch.

Make sure you improve the NDDM exclusion list and delete newly excluded files from NDDM simply by deleting the Notes db records in NDDM.

Scanning – User activity. When the Notes database import – conducted in the previous step – is ready, it is time to carry out the Notes database user activity scan. In this scan, the excluded Domino servers and Admin user names are skipped, thus not reported and not used in the analysis phase of the scan.

You want to know the real end users activity. Domino server and Administrators user activity is irrelevant

This manual will assist you to carry out the scan multiple times until only real end users remain. The separation of real end users from servers or system users is crucial for determining if a Notes database is still (actively) used in your organization.

As mentioned before, also the user activity scan should preferably be performed by the scheduled server agent.

Press the menu “Analyze Databases > Scheduled On Domino > User Activity > Servers” to display the task screen.

Select “Analyze User activity” (default). A good approach is to start with 1 Domino server holding the majority of the Notes databases because we have to extend the list “Users and servers to exclude from the user activity scan”.

Besides collecting all Notes database user activity details per year, NDDM will also collect all user activity details per user. The ‘User Activity after Date’ box can be used to list all active users and their activity after the specified date. By default, the date is set one year back into the past. You can adjust this date if you like to.

NDDM will also collect the top x users in this time period. These 5 (for example) most active users can be used to communicate with, using the NDDM email tool (explained later on in this guide). You may change the top x number users if you like.

When the date and/or top x users fields are blank (or 0), these settings are omitted during the scan. All activity is then included in the activity per user and the top x active users.

Press the button “Initiate task(s)”. The scheduled scan tasks can be seen in this view:

When the scan is completed open view ‘Database Info’:

Scroll to the right and sort the column ‘Last Read By’.  Zip through the content and identify the first non real end user name (for example a Domino server, Admin, agent signer or Developer name).

Simply open the Notes database scan document and press this green marked link. A pop-up will appear where you can select the non end user name(s) to add to the exclude list.

The user name(s) will be added to the exclude list of NDDM.

Carry out this exclude task for all non end user names found in the user activity scan in the column ‘Last Read By’.

Do the same with the column ‘Last Write By’. Go through the content and identify Domino server names, Admin names, agent signer names and Developer names in order to find all names that are not the real end users.

At this stage you have excluded all non end users for the first Domino server. Now you must do a new server based user activity scan for the same Domino server. There is a big chance that new non end users will pop up in the columns ‘Last Read By’ and ‘Last Write By’. Exclude these names now and redo the user activity scan a few times until the 2 columns ‘Last Read/Write By’ only report real end users.

When the first Domino server only reports end users and all non end users are excluded, you can start scanning the other Domino servers for user activity. It is likely that you will find a few more non end users which you must exclude. When you exclude more names, you should do a re-scan of the user activity on the Domino servers where non end users where found. After a few new user activity scan operations, only real end users should be listed in the columns ‘Last Read/Write By’. Once again:

You want to know the real end users activity. Domino server and Administrators user activity is irrelevant

One remark on the user activity scan: “na” (not available) can indicate that – although user activity recording is enabled – there are no user activity entries, or that all user activity is designated to users that are excluded. In the latter case, it can be concluded that the Notes database is not used by real end users.

These are the possible scenarios regarding User Activity scanning:

Scanning Notes database user activity with the Notes client and exclude all non end users will take a day max.

It is crucial the user activity scan does not show non end user names. You may improve the results later in the scan process

Scanning – Notes database content.

Normally the Notes database content scan is done after the user activity scan. The Notes database content is executed by the Domino servers in parallel, because this task will process a lot of Notes data.

Instruct the Domino servers to carry out the scan and sit back for the results to appear in the central NDDM Notes database.

You want to discover the dates when the Notes database content was last modified

You may select one, multiple or all Domino servers.

Do not enable Deep Scan Doclinks yet. We will do this later on.

Wait until the view “Scheduled Scan Tasks\All” shows that all servers are ready with the content scan.

A few important remarks on the Notes database content scan:

• In the content scan, the Users/Servers exclude list is NOT applied, because it is very interesting who has last modified the content; a Domino server, a signer ID or a real person. The exclude list is only used in the user activity scan.

A Domino server will run a Notes database content scan task in a few hours or a few days, depending on the database size and number of documents. You must wait for the scheduled replication. The Domino servers will run this task in parallel, not sequentially.

A sample Notes database content scan result:

The Last Doc Created/Modified (By) data of the scan is very important to determine what to do with the Notes database in the step ‘target analysis’, which is explained further below.

At a later moment you may re-scan all or selected Notes databases to update the scan details, using the options in the left menu.

For the few Notes databases that encountered problems during the Domino server based scan, you can apply a manual scan. You can find these Notes databases by – for instance – checking the ‘#Forms Used’ column:

Before you manually scan these Notes databases, make sure to enable Full Access Administration in the Domino Admin client.

Now select these Notes databases and click on ’02. Analyze Databases\Manual On Client’.

Scanning – Notes database content scanning issues.

The Domino server based scanning agent runs under a same Notes ID that was used to sign the NDDM. Its possible that this ID does not have access to the complete content of some Notes databases. This sometimes happens when – for instance – an ACL Role is used in reader fields on the Notes document.

When the agent was unable to scan all content, the Notes database is highlighted in red in the view. For example:

The form shows the number of documents that could not be scanned (marked with a green square in the image below).

To fix this, make sure the scanning signer ID has all the roles in the Notes database ACL. If this does not fix the issue, the problem may be caused by something else. A Domino administrator needs to step in to implement a fix resulting in a complete scan of this particular Notes database. However, if the scan target for this particular database is already beyond doubt, a complete scan for this database is not needed.

View ‘Database Read Protected’ displays all Notes databases where the scan has discovered protected Notes documents.

To get round the protected Notes documents issue, run a manual content scan of these databases with the Full Access Administration option in the Notes admin client enabled. By doing this, you can access most protected documents, resulting in a much better scan result. The image below displays the steps to be taken with regards to scanning read-protected documents.

NDDM has these 2 views to report Notes databases where the user activity or the content scan has not been completed

Make sure these 2 views are empty, then all Notes databases are scanned properly.

Scanning – Logging of scan errors

On the bottom of each NDDM Database Info document, a scan log is displayed.

If an error occurs during a scan, this error is listed in this Analysis Log field:

For the User Activity scan, additional warnings and info is logged as well:

• Warning: User Activity recording not enabled for this database
• Info: User Activity recording is enabled for this database, but no activity was found

In the Errors views we have created multiple views where the scan errors are grouped.

In case of scheduled tasks, additional logging is displayed on the task document itself. This logging will also list errors that can not directly be traced back to a specific database.

The Scheduled Scan Tasks\All view displays the number of errors found during the scheduled scan. These do not include the Warning and Info messages mentioned above.

Additional (fatal) errors are logged in the Agent Log view. These errors mostly correspond to the error on the Database Info and/or the Task form, but sometimes this is not the case.

This logging capacity in NDDM makes it a lot easier to track down the cause for any error. Note, however, that in the compiled version, the error does not display the correct script line number.

Found excluded files

Suppose during the scan you have excluded certain Notes databases that where scanned in a previous scan.

NDDM offers an easy way to find newly excluded Notes databases.

Press this action in the main Database info view

When the Notes agent is ready navigate to this view and simply delete the found records from NDDM because these files are recently excluded form the scan.

In Root

The view In Root displays all files that are in the Domino server root folder, normally this folder only holds system Notes databases you might want to exclude from the scan and delete from NDDM.

Mark Duplicate excluded files

This action will identify double excluded file names so you can delete double excluded file names from NDDM.

Now the scan is ready. You have collected a huge amount of details on all Notes databases which now have to be analyzed.

In this phase, NDDM will try to classify all Notes databases into one of these 3 main categories:

• Replace – Notes database is actively in use
• Archive – Notes database is only read
• Delete – Notes database is not used at all

Lialis NDDM has a set of Target calculation rules that will assist you to calculate the Targets automatically.

For each Notes database the content scan and the user activity scan are reviewed by above listed target definitions. NDDM will calculate in which group (Replace, Archive or Delete) the particular Notes database fits.

The Targets setup we have supplied in your NDDM version may differ from the examples we give in this guide. The Targets setup we supply are based on the latest knowledge we have gained in our projects.

Lialis will always guide clients in the target calculation phase because of its complexity and importance

First you have to decide the target cut-off date; for example 1 or 2 years in the past.

In below Diagram we have simplified the cut-off date approach we use in NDDM.

This way you can adjust the cut-off date easily on all targets

Important – make sure you restore task 320 to 15 July 1970 !!!!!!!

The problem is that there are always Notes databases that do not fit nicely in one of the 3 categories (Replace, Archive or Delete). This is the main reason why there are more than 3 target calculation documents provided in NDDM by Lialis.

Lets look at the Replace target definition document in more detail.

Target number is used later on in the process when we worry about Notes database replicas on multiple Domino servers.

Target short can be Replace, Archive or Delete. You may rename these if you prefer.

Target description has a very detailed naming of this particular target.

Availability it is possible to enable 3 options:

The blue marked criteria are applied to the Notes database content scan results.
The green/yellow marked criteria are applied to the Notes database user activity scan results.

• No Documents – when the Notes database has 0 documents
• No Activity – when the user activity is “na” (means there are no real users found after applying the exclude list on the user activity)
• Not scanned – when the user activity is not enabled in the Notes database

In the following fields you can set if this target is applicable. For example if the last document creation date is after a certain date.

Last doc created, Condition: Created after Date: 02-03-2020
Last doc modified, Condition: Modified after, Date: 02-03-2020
Last read date, Condition: Read after, Date: 02-03-2020
Last write date, Condition: Written after, Date: 02-03-2020
Reads, Condition: Larger, Number: 10
Writes, Condition: Larger, Number: 10

All criteria must apply to the particular Notes database scan results before the target is applied.

The target calculation may be carried out by the Domino server or by the Notes client. Its better to let the Domino server carry out this task because the Domino server will do it much faster compared with the Notes client.

In the view “Database Info” press the button Analyze Targets > All docs (scheduled).

A new server task will be listed, wait for the server to carry it out.

With the targets supplied, we try to achieve that all Notes databases are targeted. However, this might not be always the case.

In the rare case that a target cannot be calculated, the below columns are empty in the main view Database Info.

On the Notes document you can follow the target calculation decisions made by the system. This will help you understand the target calculation and it will help you finetune the target calculation.

Lialis support is always included to assist you in the target calculation and fine-tuning

If a Notes database is replicated across multiple Domino servers, NDDM will calculate which Notes database replica is used the most, based on the available usage data. This particular replica will then be assigned the Winner role. The calculation mechanism is not very complex.  The hub server icon (see image below) indicates that this particular Notes replica is the winner database. If a Notes database is not replicating, it will always get this winner role.

When all Notes databases with the winner role are migrated, or closed, or otherwise dealt with, you can be sure that all Notes databases in the whole environment are migrated and the all Domino servers can be decommissioned.

You may change the target from archive to delete for example. In this case enable the tick box Set Manually to prevent next target runs to overwrite your manual target settings.

In the NDDM views, Notes databases where the target has been set manually, are identified with this icon.

NDDM supplies 3 views to display only the Delete, Archive or Replace winners databases

It is recommended to use these 3 winners views to carry out batch ‘ACL Close’ operations or to send out emails with the NDDM survey tool for example.

A Notes database can have replicas on multiple Domino servers. Notes content is replicated between replicas, but Notes user activity is not replicated. This means that – with respect to the example of the Sales development database displayed below – 6 replicas of the same Notes database will be targeted differently on each Domino servers: 1 replica is targeted with Delete, 4 databases are targeted with Archive, and 1 database is targeted with Replace.

This real live situation happens regularly, because Notes content is often modified on only one or maybe just a few of the many replicas. In this example, the so called ‘winner target’ will become ‘Replace’ because this target overrules the lower targets ‘Delete’ and ‘Archive’.

You might have wondered why the targets have a number. Well, this number is used to calculate the master winner target. In above example, 1 Notes database gained the target 2570 Delete , 4 Notes databases got the target 3560 ‘Archive’,  and 1 database got the target 4590 ‘Replace’. The replica with the highest target number is assigned the winner status, which in this example is 4590.

Note that the numbers of the targets you received from Lialis may differ from the the numbers being used in this manual.

In the project execution, the Notes databases with the winner targets will be dealt with. The Notes database from this example will have to be replaced. When all winner Notes databases are either deleted, migrated/replaced or archived the project is completed.

In future reporting, only the winner Notes databases should be used and reported, because they stand for the whole Notes database collection.

A good way to study and review the targets is to export the NDDM content to Excel (client-based, selected documents) or CSV (server-based, all documents in view) as illustrated below.

The client based export might be very slow in large Notes environments, in this case use the second option (server based CSV).

Export client based:

In Excel it is easy to review the target calculations made by NDDM, by having a close look at the columns that have the target definitions. Simply make the deletions in the Excel columns in the same manner as defined in the Target settings document and you will understand how NDDM does the target calculations for you.

Note that the columns we use in the target calculation are colored in Excel.

Export server based CSV

A new export task has been created to be carried out by the Domino server.

When the task is completed please find the csv file in this view

When the amount of data that has to be exported is very large (larger then 32.000 lines) NDDM will split the export files in multiple parts (export_000.csv and export_001.csv)

Save the CSV file to your PC

In Excel do this:

Navigate to the CSV file

Press load in the import screen

And all data in NDDM is exported to Excel.

The Excel export provides additional information that is not displayed in the NDDM Notes views. For example, a listing of the design forms that the Notes database contains. Sorting on the forms column will give a great insight in Notes databases with a similar design. Notes databases that contain forms with the same name indicate that chances are high that these databases have a similar design, and thus may be grouped together, even if the template names are different.

Note that the CSV Export is done by scheduled agent called EXPORTONSERVER, that should be allowed to perform restricted operations as described in section ‘Scheduled Agents’

NDDM has below free fields to be freely used by our customers to organize the Notes app decommissioning.

These fields are displayed in the views

NDDM allows clients to import data into these fields using an Excel file import.

The example excel file to be used for this import is located in the Notes database using document.

Store this excel file to your computer.

The import will only work when the Domino server name holds the certificates and the file path is correct.

Use below action to carry out the Excel file import into NDDM.

The import will run in the Notes client background, so please be patient for this import task to complete

These imported values are also present in the server based CSV export.

NDDM offers a way to find certain Notes database design strings. In this example we search for the design strings “@MailSend” and “.Send” to find out which Notes databases are sending out emails with for example workflow links that point to documents in other databases. Listing these Notes databases is important if you plan to do a Notes email migration first, since the database workflow will likely no longer be functioning.

Finding out these details only matter for the replace winner Notes databases. Hence first go to the view Replace winners.

Select all Notes databases in this view and carry out the scan instruction as displayed below, to find the design strings “@MailSend” and “.Send”.

The scan will be run in the Notes client background and will take a few minutes per Notes database.

An example of the found strings:

If a sub-string (case insensitive) is found, the complete line of code is returned as a result.

Another way to find out if a database is sending mail, comes from the content scan: If documents are found containing a SendTo and/or CopyTo field, the addresses in these fields are listed on the Database Info document. Note that this listing is up to a max number of characters. The goal is to get an impression of how important the mail functionality is for this application, rather than obtaining a complete list of addresses.

When you combine all this information, you get a relatively good insight in the mail functionality and whether it is still in use, for example:

The Notes database holds memo forms with people in the SendTo field:

The memo form has been used recently:

The Notes database holds design elements capable to send out workflow emails.

The ‘Database Design strings found’ view displays a selection of the Notes winner databases where the design strings (.Send & @MailSend) were found. Also the number of names found in SendTo and CopyTo fields is displayed in this view:

Note: only if you ran the ‘Scan design on keywords’ on a database, and only if this scan returned results, the database info document will be visible in this view.

NDDM offers a deep Notes document link scan. Normally, you would only run this on Notes databases with the target Replace HUB.

Go to the ‘Replace winners’ view, select all Notes databases and press the menu option ‘Selected Databases’ highlighted in the picture below:

A server-scheduled task will now scan all document links found in each document in all databases. Each link will be tested whether it is a dead link or if it is pointing to an existing target. Note that for each server involved in this scan, a separate Task document is created:

When the scan is completed:

A result of the Deep Notes document link scan is displayed in the image below

NDDM offers a readers and authors field scan. Normally this scan is only carried out for the Notes databases with the target Replace and archive because for Notes databases with the target Delete the scan results are not relevant.

The scan results will be listed in this view.

An example of the results is displayed below. Note that the names displayed per field, is a collection of all the names that were found on all documents created with the specific form.

You may scan all Notes databases on all servers for reader and author field details.

Press this scan action:

Or you may carry out this reader and author field scan on certain Notes databases to save scanning time.

For example, select all replace winner databases

Choose this scan option

Select below scan option to carry out the scheduled scan reader and author field scan task for selected Notes databases

Its possible to scan a few Notes databases for reader and author fields using the Notes client, but this scan is very slow and we recommend to use the server based scan approach.

NDDM supplies great Notes database ACL adjustment functions that can be applied bidirectionally to multiple Notes databases in one go. It is easy to close the access for regular users for all Notes databases that have been targeted to 1400 Delete. If needed, this can be undone with one mouse click.

First, configure the ACL lists. The “Edit ACL Lists” function allows you to configure ACL entries to be added when the ACL is adjusted by NDDM. For example, always add the servers and admin groups.

The entries listed under “Unchanged ACL entries” are not updated during the batch process where NDDM is changing ACL’s on multiple databases. Make sure to add the standard ACL Manager groups, server names and administrators to this list. Also include the members of groups that you add to the ACL. When a Notes database is ‘closed’, keeping these entries in tact ensures proper replication between Domino servers and Admin access later on.

NDDM will only change the ACL on 1 Notes database replica when multiple Notes db replicas are selected to be changed!!!!! This prevents ACL replication errors later on.

In the example below, the Admin is going to put the selected Notes databases with the target 2570 Delete to read only.

Its a good practice to set Notes databases with the target “2570 Delete” to read only, and give users a few weeks to react as they are not able to modify the content in Notes. Then you know that the Delete classification should be adjusted.

The ‘Status’ view gives details on ACL closure status.

The NDDM Notes form displays details on all ACL changes made with NDDM.

Changing back ACL changes made with NDDM is easy using the appropriate action menu:

Reverting access is logged as well.

When the targets are applied accurately, it is time to reach out to the users of the Notes database to get confirmation of the calculated target for example.

You need to give the end users access to NDDM so they can answer the questions you send them, for example via an ACL wildcard, see image below.

With above permissions end users will not get any access to the content of NDDM because all content is protected with reader fields. The image below shows the NDDM user interface to end users who do not have the Admin role.

NDDM has a nice way to send (questionnaires) emails in batch, based on predefined templates and specific data from your scan that can be used in your project.

To configure a mailing, click in the menu on ‘Mailings’.

You may send the mailing to the following 5 groups of users.

Owner is retrieved from this field in NDDM.

Unique users after is retrieved from this field in NDDM.

ACL Managers is retrieved from this field in NDDM. All servers in this group are excluded from the mailing.

Unique users is retrieved from this field in NDDM.

Most active users is retrieved from this field in NDDM.

To start the communication we suggest to use the Most active user option to start with.

The email subject can be set.
All end users will be listed in the BCC field of the email. Because of this reason the SendTo field has to be filled in.

The body of the email can be set.

The 2 options at the bottom of the body are important.

Add link to Database Info Document – this option will send the user an email with a document link to the NDDM document where the questions can be answered. You may use this method if the end users will receive a few emails for a few Notes databases they are requested to answer the questions in NDDM. The insert bookmark options (for example %DbAllDocLinks%) is supported. The email send is displayed below linking to the NDDM document with the questionnaire.

The link will link directly to the NDDM document with the questions.

Add link to questionnaires list – this option will send the user a NDDM Notes db view link where the user can view all NDDM questionnaires directed to him. You may use this method if the people will get multiple or many Notes database questionnaires in NDDM. The insert bookmark options (for example %DbAllDocLinks%) is NOT supported. The email send contains a link to the questionnaires view.

The questionnaires view is displayed below. It lists 1 questionnaire in this example.

The user may open the questionnaires 1 by 1 and answer the questions. The view will display what questions have been answered.

The bookmark options (for example %DbAllDocLinks%) can only be used for questionnaire option “Add link to Database Info document”. An example is listed below.

The email result of bookmarks is listed below. You can see that all variables are filled in with the actual values from the NDDM scan.