LDMI Technical Manual

 

To achieve the best scanning result in the shortest time frame, we advise you to carry out the below listed scan sequence.
Each step in the table below is explained in detail in this post.

Preparations	Carefully read the section “Supported Notes and Domino builds” on this page. Certain Domino server releases on certain OS need a notes.ini settings LS64BITCCALLOUTPointerSupport=1. This required notes.ini settings, for some of the 64-bit Domino servers, is a fix that has been applied by LDMI because of a Domino 32/64 bit incompatibility. This setting is necessary for the relevant server to perform a Domino server based user activity scan on a Notes database.
Setup and roll out	Install the LDMI Notes application on the first Domino HUB server Set the ACL of the LDMI Notes application Sign the LDMI Notes application with an ID that has manager access to all Notes mail databases and that is allowed to run unrestricted agents (allow restricted operations) Open the LDMI configuration, set the Domino Mail directories to scan, exclude all Domino server and Admin names Use LDMI to create LDMI replicas to all Domino servers to be scanned in your Domino domain Enable the scan agents in LDMI with the button on the Configuration document
First and successive user activity scans	Wait for replication to deliver the results to LDMI on the HUB server After the first activity scan, you can exclude all discovered non-real end user accounts (e.g. admin, signer, system, server accounts). During successive activity scans, accounts previously excluded will not be found. However, new non-real end user accounts may surface. Exclude these as well. Continue excluding non-real end user accounts until only real end users are found in the user activity, or the scan returns na (‘not applicable’) for certain Notes databases (this means the user activity does not hold real end users).
Target calculation	Set the target calculation cut-off date, for example 1 or 2 years in the past (as per your choice)

 

Supported Notes and Domino builds

LDMI runs a scheduled agent on Domino servers to carry out the scan. Therefore LDMI has been tested extensively on the newest versions of all relevant Domino server major releases: R6, R7, R8, R9, R10 and R11; on both Windows and Linux operating systems, 32 and 64 bits servers. We expect lower Domino servers versions will not pose any problems. However, if you run lower Domino server builds (R4, R5), we advise you to contact us so we can give these versions a test run. Please consult the following list with Domino versions/platforms that we tested LDMI on.

Summary of tested configurations:

Windows/32
Release 9.0.1 FP8|February 23, 2017 Build 405
Release 9.0.1 October 14, 2013 Build 405
Release 8.5.3 FP6|November 21, 2013 Build 390
Release 7.0.2 September 26, 2006 Build 265
Release 6.5.5 November 30, 2005 Build 198

Windows/64
Release 11.0 .1 FP2|October 20, 2020 Build 452 *
Release 10.0.1 FP6|September 24, 2020 Build 450 *
Release 9.0.1 FP10 HF747|July 24, 2020 Build 405
Release 8.5.3 FP6 HF3175|April 28, 2017 Build 390

Linux/32
Release 9.0.1 October 14, 2013 Build 405

Linux/64
Release 11.0.1 FP2|October 20, 2020 Build 452 *
Release 11.0 November 25, 2019 Build 451 *
Release 10.0.1 November 29, 2018 Build 450 *
Release 9.0.1 October 14, 2013 Build 405 *

*) notes.ini setting required: LS64BITCCALLOUTPointerSupport=1

The required notes.ini settings for some of the 64-bit servers is a fix that has been applied because of 32/64 bit incompatibility. This setting is necessary for the relevant server to perform a user activity scan on a database. When an argument is passed by reference, the C function receives a 32-bit pointer to the value area. API calls from LotusScript on a 64-bit Domino server will fail (crash) when an 64-bit pointer is expected.

For example: when a variable is a pointer that is passed by reference, this works perfect on a Domino 32-bit server when this variable is defined in LotusScript as a Long. However, the same situation on a 64-bit Domino server – where the variable is defined in LotusScript as a (64-bit) Double – makes the server crash.

In order to be able to return the pointer in double, you need to set environment variable LS64BITCCALLOUTPointerSupport=1

You can set this variable by directly editing the server’s notes.ini, or by issuing a console command on the server:

set config LS64BITCCALLOUTPointerSupport=1

A server restart is not necessary after this notes.ini change. It will take effect immediately.

During our extensive testing, we learned which servers this setting should apply to. If a user activity scan is initiated on such a server – and this setting is not found in the server’s notes.ini – this is reported in the scan log on the Task document. The database scan will continue its run, but the user activity scan task is skipped.

Furthermore, if a scheduled User Activity scan task is run from a server that may very well run without problems, but was not tested by us, we don’t allow the agent to run the User Activity scan anyway. Other scan tasks are allowed though. The Task document log will tell you that the User Activity scan is not possible on this server.

This also applies to non-tested 32 bit servers. As mentioned above, we tested the newest versions of all relevant Domino server major releases from R6 up to R11. Only on these Domino servers, the server-based agent is allowed to do a User Activity scan. If one of your servers is not allowed to do the User Activity scan, you could ask a Notes developer to adjust the script library called ‘slPlatformInfo’ to include your server. When you do this, please be VERY careful, and make sure you run a test before you apply this on the production server, because an incorrect adjustment may crash the server. You can also contact Lialis to perform this test and adjust the script.

If for whatever reason the agent is interrupted during the scan (for example by an unexpected error, or an automatic Domino server reboot or crash), the Notes database that the agent was processing at the time of the interruption is skipped when the LDMI scan agent starts again. Just in case the scan of this database has caused a server crash, omitting this database in the successive run prevents the server from crashing again. The skipped database is logged:

LDMI supports IBM Verse email. A Lotus Notes client is needed as well, the version of the Lotus mail client is not really important because the Domino server will do all the scan work.

Domino server performance impact

All scanning tasks are carried out by the Domino server on its own local Notes databases by LDMI Lotus Script agents. See image below:

Domino server based scan of the Notes Mail database user activity. The impact on the Domino server is zero because the user activity scan is very fast. For example scanning 100.000 Notes mail databases will take less than 10 minutes (depended on server speed).

LDMI has been designed in such manner that multiple Domino servers will run their own local scans in a parallel operation, resulting in enormous scan speeds. So, scanning 1 Domino server will take the same amount of time (a day or 2) as scanning 10 or even 100 Domino servers.

A Domino administrator can adjust the run schedule of the agents by simply opening the agent in Notes Designer and adjusting the schedule. Below is an example of the LDMI CreateTasks agent, responsible for issuing all work tasks for the Domino server to carry out. This is explained later on in this post.

Setup and Configuration

You can download LDMI from here

[email-download-link namefield=”YES” id=”4″]

 

Open the local LDMI Notes database with your Notes client and make a Notes database copy (with your Notes client) to the main IBM Domino (HUB) server that replicates with all other Domino mail servers. Make sure to include the documents in your copy because LDMI holds a few settings documents you need. You may copy the ACL because we added the default Domino server and admin groups to the ACL.

Open the new LDMI Notes database copy on the first Domino server and continue.

ACL – First adjust the ACL. Make sure all Domino servers have manager access with delete permissions. Also add all administrators and viewers if applicable. You might want to add people to view the content with read permissions.

Signing – LDMI will run scheduled agents on all Domino servers to be scanned. This means you have to sign LDMI with a Notes ID that is allowed to run agents on each Domino server. This ID must also have read access to all Notes mail databases. In most environments, the ID of the Notes administrator has these privileges. You can also use a certain Domino server ID file to sign LDMI. LDMI will report if the ID does not have enough permissions.

Configuration – Go to the view ‘Configuration & Roll-out’ and press the button ‘New’ to create a new configuration.

Although it is possible to create multiple Configuration documents, only the 1st configuration document in the view is the active one.

The configuration form has 3 parts which will be described in the following sections.

Mail Directories to include from the scan – First set the folders where the Notes mail files are stored. You may set or type multiple folders. Use a new line for the next folder.

 

Users and servers to exclude from the user activity scan – In the new configuration form you may add users and servers to exclude from the Notes database user activity scan. This step is important, because filtering away all Domino servers, signer ID and names of administrators and developers will result in a much more accurate list of activity of real users. Generally, it’s not interesting to know what servers and admins are doing in a Notes database. So add as many known Domino servers and ‘unreal’ users to the exclude list now. See the sample below:

After each User Activity scan, you can add more ‘unreal’ users to the exclude list, as described in the above table under ‘First and successive user activity scans’.

Configuration of the Roll-Out – The roll-out part of the LDMI configuration allows you to quickly create replicas of the LDMI Notes database to all (or a selection of) Domino servers in the Domino domain that you are planning to scan with LDMI. Your Notes client will create the new replicas. This may take a short while depending on the amount of Domino servers you are actually going to scan.

Now press the ‘Rescan servers’ button to find all running Domino servers.

It will show that LDMI is placed on the first Domino server and you will be able to create new replicas on all running Domino servers that hold Notes mail files.

The replicas are created in the background, as you will see on the replicator page of the Notes client.

You need to make sure that all Domino servers you are going to scan are setup with a so-called ‘pull push replication’ to the main or HUB Domino server. Normally this is the case in Domino environments. Check this anyway just to be sure.

If you have Domino cluster servers you only need to install LDMI on the main Domino cluster server.

Start/Stop Agents

After saving and re-opening the configuration document, you will see a button at the bottom of the document. Press this (green) button to start the scanner Notes agents. Now the process will go fully automatic, all you have to do is sit back and wait for the results to replicate back to the main LDMI Notes database on your hub server. The scan steps are explained in the next section.

You can find the configuration document in the view ‘Configuration & Roll-Out’. If there is somehow more than 1 configuration document in this view, be aware that only the first one is used by LDMI.

The LDMI Notes views are set to NOT refresh automatic. This has been done to speed up the LDMI scan process. Please press F9 in the views when you are working in LDMI

At this stage all selected Domino servers have a replica of the Lialis LDMI Notes database. You do not need to open all LDMI replicas. You only need to wait for the results to be replicated back to the central LDMI Notes database.

When the agents have been enabled in the configuration the scanning will happen fully automatic in the following sequence.

Scanning – CreateTasks agent

The CreateTasks agent will create new task documents in LDMI.

1 task document which instructs all set Domino servers to scan all mail files and log the scan details in LDMI.

1 task document which instructs the HUB server to calculate the targets of the IBM Mail Verse Notes files.

You need to set the Domino server which is going to do the target tasks, use for example the HUB server. Only 1 Domino server must do this task, does not matter which server you choose.

The Notes Mail file scanning is carried out by the agent DBDISCOVERYANALYSISONSERVER

The target calculation is done by the agent DBTARGETINGONSERVER

Now wait until the Domino servers have carried out the tasks. When the tasks are done they are listed in the Tasks Completed view.

A Domino server will run this Notes database user activity scan task in a few seconds. The only delay is waiting on the agent manager and the scheduled replication.

If you delete all LDMI content you can run this task also to start from scratch.

Scanning – Logging of scan errors

On the bottom of each LDMI Database Info document, a scan log is displayed that also lists database-specific scan errors, if any were encountered.

Additional logging for the scheduled agent run is displayed on the task document. This logging also lists errors if any were encountered during the scan.

Then there is the Agent Log view, where additional (fatal) errors are logged:

This logging capacity in LDMI makes it a lot easier to track down the cause for any error. Note, however, that in the compiled version, the Error line does not display the correct number.

Now the scan is ready. You have collected a huge amount of details on all Notes databases which now have to be analyzed.

In this phase, LDMI will try to classify all Notes databases into one of these 3 main categories:

• Active – Notes mail database is actively in use
• Read only – Notes mail database is only read
• Delete – Notes mail database is not used at all

Lialis LDMI has a set of Target calculation rules that will assist you to calculate the Targets automatically.

For each Notes mail database the user activity scan are reviewed by above listed target definitions. LDMI will calculate in which group (Active, Read only or Delete) the particular Notes mail database fits.

The Targets setup we have supplied in your LDMI version may differ from the examples we give in this guide. The Targets setup we supply are based on the latest knowledge we have gained in our projects.

Lialis will always guide clients in the target calculation phase because of its complexity and importance

First you have to decide the target cut-off date; for example 0,5, 1 or 2 years in the past.

This way you can adjust the cut-off date easily on all targets by pressing the button “Set Cutoff date”.

Lets look at the Replace target definition document in more detail.

Target number is used later on in the process when we worry about Notes database replicas on multiple Domino servers.

Target short can be Replace, Archive or Delete. You may rename these if you prefer.

Target description has a detailed naming of this particular target.

In the following fields you can set if this target is applicable. For example if the last document creation date is after a certain date.

Last read date, Condition: Read after, Date: 13-03-2020
Last write date, Condition: Written after, Date: 13-03-2020
Reads, Condition: Larger, Number: 10
Writes, Condition: Larger, Number: 10

All criteria must apply to the particular Notes database scan results before the target is applied.

The target calculation is carried out by the Domino server.

In the rare case that a target cannot be calculated, the below columns are empty in the main view Database Info.

If a Notes mail database is replicated across multiple Domino servers, LDMI will calculate which Notes database replica is used the most, based on the available usage data. This particular replica will then be assigned the HUB role. The calculation mechanism is not very complex. The hub server icon (see image below) indicates that this particular Notes replica is the HUB database. If a Notes database is not replicating, it will always get this HUB role.

When all Notes databases with the role HUB are migrated, closed, or dealt with otherwise, you can be sure that all Notes databases in the whole environment are migrated and all Domino servers can be decommissioned.

With LDMI its easy to export all content to Excel as illustrated below. Simply select all documents and press the Export button.

In Excel, it is easy to review the target calculations made by LDMI, by having a close look at the columns used by the target definitions.

LDMI supplies great Notes database ACL adjustment functions that can be applied bidirectionally to multiple Notes databases in one go. It is easy to close the access for regular users for all Notes databases that have been targeted to 1400 Delete. If needed, this can be undone with one mouse click.

First, configure the ACL lists. The “Edit ACL Lists” function allows you to configure ACL entries to be added when the ACL is adjusted by LDMI. For example, always add the servers and admin groups.

The entries listed under “Unchanged ACL entries” are not updated during the batch process where LDMI is changing ACL’s on multiple databases. Make sure to add the standard ACL Manager groups, server names and administrators to this list. Also include the members of groups that you add to the ACL. When a Notes database is ‘closed’, keeping these entries in tact ensures proper replication between Domino servers and Admin access later on.

The ‘Database Status’ view gives details on ACL closure status.

The LDMI Notes form displays details on all ACL changes made with LDMI.

Changing back ACL changes made with LDMI is easy using the appropriate action menu:

Reverting access is logged as well.