Fully Automated Notes Codebase Analysis and Documentation
A Lotus Notes (or IBM Notes, HCL Domino) application can consist of numerous design elements, some of which include code that enables the application to perform its functions for users. These applications can become quite complex, making it challenging to understand how different Notes design elements interact with one another. An IBM Notes application can include agents and script libraries, with some agents being triggered from forms, while others are activated by user actions. A Domino developer will be familiar with these concepts and the intricacies involved.
Goal
The goal is straightforward:
I want to build a tool that analyzes the design of a Notes application and generates a detailed Word document about the Notes app functionality, making it easy to understand how the Notes app operates. This tool should be fully automated, capable of processing one or multiple Notes applications at once. Additionally, the output document should be user-friendly, allowing non-Notes developers to adjust and fine-tune the generated Word document as needed.
Workflow
Notes DXL export
Our Notes Domino application scanner can quickly export all design elements from multiple HCL Domino applications in a very short amount of time. The image below displays all agents for a specific Notes application, with forms, views, and other design elements included in the export as well.
The export generated by NDDM is shown in the image below. It is a raw DXL export of the Notes design elements, which isn’t immediately readable, but it contains all the necessary information we need.
Upload data to the cloud
We will use Quest NMSP to upload the NDDM Notes DXL records to a SharePoint list. The reason we use a SharePoint Online list is that Make.com has a SharePoint module. Other approaches are possible because the NDDM DXL upload to a cloud location is straightforward, with no special data alterations occurring in this step.
The SharePoint list:
DXL Processing with make.com, Mongo DB and OpenAI
The previous steps of getting Notes DXL exports into SharePoint are very straightforward. Now, the complex part of the workflow begins, which is fully automated using Make.com.
One of the goals is that non-Domino developers should be able to steer and fine-tune the data processing. This is achieved using Make.com. Make.com is a no-code automation platform that allows users to connect various applications and automate workflows without requiring programming skills. It provides an intuitive drag-and-drop interface for designing automation scenarios, enabling seamless data transfers and integrations between different cloud services and applications.
In Make.com, we have multiple scenarios that handle the entire workflow. At a high level, the following scenarios are needed:
Transfer the DXL data from SharePoint to MongoDB
We use MongoDB because it can hold very long text strings, and its supported by Make.com.
In this scenario, MongoDB is populated with the DXL information and data labels such as the NSF title, file name, and other relevant metadata (from the SharePoint list).
After this step, SharePoint is no longer used. All data processing will be handled in MongoDB from this point forward.
Notes DXL analysis
This is the core part of the workflow, where multiple scenarios in Make.com will send Notes form, agent, view, and other DXL data from MongoDB to OpenAI for analysis. OpenAI will process this data to derive insights, perform pattern recognition, or generate relevant outputs based on the data.
An OpenAI prompt example is listed below.
You are a Lotus Notes application developer. I will feed you raw DXL export of a Notes form.
{{22.DesignCodeDXL}}
Get me a listing of all subforms on the form. I want to know if the subform is computed or not, if its computed give me the formula, if its not computed tell me the subform name.
An example how you have to reply:
Computed subform or not, if not computed the linked subform name, if its computed the formula.
Only reply me the results, dont add your introductions
The approach is straightforward: the raw DXL is sent to an OpenAI prompt, where we ask OpenAI a specific question, such as “Give me a listing of all subforms on this form.” OpenAI excels at understanding DXL and will provide an accurate response. The reply to this question is then stored in a dedicated field in MongoDB.
For analyzing a complete Notes database, we will ask OpenAI multiple/many questions related to forms, agents, views, and more, such as:
– Generate me a list of the fields on the forms, including the text in front of the fields.
– Which agents are scheduled? Please provide the schedule details.
– Please tell me which subforms on this form are computed, or which are not computed. If they are computed, provide the formula.
– How important is this form? Rate it in percentage and explain why.
All answers will be stored in dedicated fields within MongoDB records.
The fun part is that the person performing the analysis can easily add new OpenAI questions to the Make.com scenarios and store the data in MongoDB.
Below is an example of a MongoDB record, where you can clearly see the fields of this particular form.
To analyze a single Notes codebase DXL, we will run multiple Make.com scenarios to analyze all parts of the Notes app, as shown in the image below.
The final step is to retrieve the data from MongoDB and generate a document. This document will summarize all the insights gathered from OpenAI’s analysis of the Notes forms, agents, views, and other elements, providing a comprehensive overview of the entire Notes application.
Document generation
At this stage of the workflow, all the data we need is stored in a MongoDB collection across many fields. This data is now retrieved from MongoDB, organized into new variable arrays, and then added to a new Google document, which is the final step in the scenario.
This allows us to generate a structured document containing all the insights and analysis from the previous steps.
Result
See the text below for an example output. Please keep in mind that this is only a very short example of the data generated by the setup described in this post.
Notes database title: Homepage
File name: Development\ABC\homepage.nsf
Forms
Number of forms in this NSF: 44
Form: CommentA
Summary:
This form is a response form in a Lotus Notes application. It allows users to create comments in response to a document. The form includes fields for the comment author, subject, content, and other details. There are also actions available such as editing, forwarding, moving to folder, and categorizing the comment. The form includes formatting and styling options for the content of the comment.
Importance:
80% because this form is used to create the main documents.
Fields:
Field Name: Author_D, Field Type: Text (Computed for Display), Multiple Values: No, Formula: @Name([CN]; Author)
Nested subforms summary:
DocumentHeader
ResponseSubform
WebFileUpload
Versioning
Nested subforms details (formulas):
Subform 1: Not Computed, Name: DocumentHistory
Subform 2: Not Computed, Name: DocumentHeader
Subform 3: Computed, Formula: ResponseSubform
Subform 4: Computed, Formula: @If(@ClientType = “Web”; “WebFileUpload”; “”)
Subform 5: Not Computed, Name: Versioning
Form actions:
Action name: DokumentEdit, Shared action: No Action formula: @Command([EditDocument]) Action properties: Hide when formula: @IsDocBeingEdited = @True
Action name: DokumentClose, Shared action: No Action formula: @Command([FileCloseWindow]) Action properties: Hide when formula: @IsDocBeingEdited = @False
Processing time and costs
In this section, we will only cover processing costs. Project consultancy and software license costs (Quest NMSP and Lialis NDDM) are not listed here; please contact us for an offer.
Moving Notes DXL data via SharePoint to MongoDB is very fast, and the processing costs (including the Make.com scenarios and MongoDB) are relatively low.
Because the Notes DXL data sets can be quite large, only OpenAI model 4.5 can be used, as this model is very smart and supports large datasets. (We experienced “content too large” errors when using OpenAI 3.5 models, by the way.)
The OpenAI processing cost for a single Notes database can range from 25 euro to several hundred euros, depending on the number of design elements.
Running the complete Make.com scenario on a single Notes database will take several hours. The generation of the document will take a few minutes.
Security
This model only processes the Notes application codebase; the Notes content is not processed. Clients should consider OpenAI’s data privacy policies.
The MongoDB is stored on Amazon, and the client can choose the geographical region. Make.com data centers are located in the EU and the USA, and the client can choose their preferred region.
Concluding
The nice approach of this setup is that the person analyzing the Notes app has full control over how the output document is constructed and the data it contains, simply by altering the Make.com scenarios (no code). This person does not need to be a Domino developer, though understanding some basic Notes codebase principles is required.
OpenAI is the key component in this setup and has proven to be very accurate in retrieving difficult answers from the DXL input. While the costs of running OpenAI Notes app design analysis are not low, they are still many times lower compared to human consultancy, where qualified Domino developers are necessary.
Next steps
Extending this approach can be achieved in many ways. All the Notes database design codebases from multiple databases can be stored in a central cloud repository, and this comprehensive data can be utilized in various ways. It’s possible to vectorize the data in MongoDB to enable proximity searches and connect the MongoDB database to a large language model (LLM) chatbot for question and answer interactions.
In this post, we focused on IBM Notes applications, but the same approach can also be applied to non-Notes applications and many other parts of a company’s IT infrastructure. It’s really exciting stuff!
Thanks for reading.
