ADDI

note

This guide assumes you have an IBM Cloud account that is accessible. It also assumes you have the ibmcloud cli installed and configured. The installation of ibmcloud cli can be found here.

Mileage may vary on different cloud providers.

Resources

• IBM Internal Software Downloads

Pre-Requisites

• Watsonx Code Assistant instance deployed in IBM Cloud
• IBM® Application Discovery and Delivery Intelligence (ADDI)
• IBM Application Discovery Connect for Mainframe (IBM AD Connect for Mainframe)

Hardware

For this build, we will be deploying Virtual Server Instances into IBM Cloud. They require the following:

ADDI Host

• OS: Windows 10 or 11
• RAM: 32 Gigs
• Storage: 100G for local, 300G for AD components
• Cores: 8

Refactor Host

• OS: Red Hat Enterprise Linux 8x/9x
• RAM: 16G
• Storage: 100G for local, 400G for data
• Cores: 4

Software to be installed

ADDI Host

• ADDI
• Microsoft SQL Server (standard, express, or developer editions)
• Adoptium Java SDK 11
• SQL Server Management Studio
• Git for Windows
• WinSCP

Refactoring Assistant

• Podman
• Refactor Containers

ADDI Host

Creating the instance in IBM Cloud

Install the ibmcloud plugin for infrastructure service

note to RK: Add the cli commands to create the instance here

Accessing the instance in IBM Cloud

ibmcloud plugin install infrastructure-service

info

Useful links for topics covered here. https://cloud.ibm.com/docs/vpc?topic=vpc-vsi_is_connecting_windows#vsi_connecting_windows_instance

Our instance name for windows is called "wca4z-addi"

Query the status of your instance by running the following command. INSTANCE is the ID or Name for the instance that you want to connect:

ibmcloud is instance wca4z-addi

When the instance shows that it's running, you are ready to retrieve the initialization values to get your password.

Run the following command to initialize your instance and obtain your instance password. Specify your instance ID or Name for the INSTANCE variable and your private key for the KEY or KEY_FILE variable:

ibmcloud is instance-initialization-values wca4z-addi [--private-key (KEY | @KEY_FILE)]

The key we created is called win-key-wca4z_rsa.prv

So our command should be

ibmcloud is instance-initialization-values wca4z-addi --private-key @~/.ssh/win-key-wca4z_rsa.prv

This command decodes and decrypts your password, which is automatically generated when you create an instance by using a Windows image. The password is decoded and decrypted based on the public SSH Key that you used at instance create time and the associated private SSH key that you specify in the instance-initialization-values command. For more information, see the CLI command reference.

You can also use the API to get the encrypted password, which returns the decoded and decrypted password. For more information, see Retrieve configuration that is used to initialize the instance API.

The output of the above command should return something like this:

Getting instance initialization values for wca4z-addi under account Fred Whipple's Account as user kramerro@us.ibm.com...

Keys                 ID                                          Name            Fingerprint
                     r014-48416b92-2235-48ff-aea7-034e5a38e8a5   win-key-wca4z   SHA256:rUix46/y3Qol46dnznusfaq4dRV/RlMrGsjEvLNWoDw

Encrypted password   I/AbavM1cYL1g0omTPzNlpIIIR3zwQKYmEmAe2DFb72Bs+HmqUy+h0C8c3bD/fV1LyUGoCWqJqnngz7+SlVddzFLunVtvM1X30qUgBCzuJfvJo6IpyNFc5YFoWZxMojWKp1e71ezzILTD10Z6MPNtdCwL2syYag+vNPexgHwc4eJwlqnhUTdFD3i0FPYOzSiReKxw1m4QpwXTd/EyecvDbrSAXnIjYyyMlJsv/dmZrLhynEjUBwn39RTqD2Da9Mjyi7hyrzKZqkKP3barnwPRFd8xQI21gA0Lm3mgC7PJzmA5Z8KqGQV3IPlFE909HRQeHpB5DN8nS4KsNGKPRVl/pF3XAA9BRoMKoOMV1d4mOXGU2b5QU1+IKpZU7ObCKhqLw+U3Dm1sPYW5IgujZd725uhqwwIBZukMQE2RSYRij0ef5mhv9gt/f/zrNWTaZVjgQqA3ufL4dIdu/W8mZf2puxKCYvhIwolA13Y09BeTLRBCK9/8LOx0e8WjmDg9lx49Fv5AwgaUi2tpLWjBf4Xgt8b6ddIo0Lgy8sc4KdsdQ/8Jtugblp/zfEzxJl6WqpD6+3UOSLQi9dKtproL9tI3CDkk2WLhV02m+vG6imrVMZpCGsgdlLEWqwCXKNIvaGmCKG0TTm7QT8BqiqxTwUPvQCLcIEGf4yWVsEbYGvR+zQ=
Encryption key CRN   SHA256:rUix46/y3Qol46dnznusfaq4dRV/RlMrGsjEvLNWoDw
Password             <UNENCRYPTED PASSWORD>

Once you have obtained the password from the above command, make sure you have associated a floating public IP for the ADDI host so it is accessible:

ibmcloud is floating-ip-reserve <FLOATING_IP_NAME> --nic <NIC>

You now have what you need to connect to your Windows instance: a decrypted password and a floating IP address. Use your preferred Remote Desktop client to connect to your instance. To connect to your instance, provide the floating IP address and the decrypted password. The username is Administrator by default. If you're connecting from a client that is running the Windows Administrator account, use .\administrator as the user ID to log on to RDP.

Disable public RDP access

It is against IBM security policy to have Remote Desktop Protocol enabled on a public IP address.

Change RDP listening port for hardening

• Needs to be set to something other than the default ports 3389/3388, in this guide we will change it to 3390

• For VPC infrastructure, go to IBM Cloud and open the virtual instance, on the network interface click on the security group, manage rules, add port 3390 to rule with 3389

• https://learn.microsoft.com/en-us/windows-server/remote/remote-desktop-services/clients/change-listening-port

• Check current port (via PowerShell)

  Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp' -name "PortNumber"

• Change port to 3390 (via PowerShell, start as Administrator)

  #Run Powershell as an administrator#
  
  #Allow powershell commands to run
  Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Confirm:$False -Force
  
  #Get current port number
  Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp' -name "PortNumber"
  
  #Change port # assigned to RDP. You can assign any port between 1024 to 49151
  #$portvalue = Read-Host -Prompt 'Input new PORT number between 1024 to 49151'
  $portvalue = 3390
  
  Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp' -name "PortNumber" -Value $portvalue 
  
  New-NetFirewallRule -DisplayName 'RDPPORTLatest-TCP-In' -Profile 'Public' -Direction Inbound -Action Allow -Protocol TCP -LocalPort $portvalue 
  New-NetFirewallRule -DisplayName 'RDPPORTLatest-UDP-In' -Profile 'Public' -Direction Inbound -Action Allow -Protocol UDP -LocalPort $portvalue
  
  #Confirm port number has been changed
  Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp' -name "PortNumber"
  
  #Once powershell commands are complete, reset the powershell execution policy
  Set-ExecutionPolicy -ExecutionPolicy Restricted -Confirm:$False -Force

• Restart RDP (via PowerShell as Administrator)

  Restart-Service -Force -DisplayName "Remote Desktop Services"

• Test RDP to port 3390 (e.g. x.x.x.x:3390)

• For VPC infrastructure, go to IBM Cloud and open the virtual instance, on the network interface click on the security group, manage rules, remove port 3389 from any rules

• On your workstation, start SSH tunneling command (-N prevents a shell from starting)

  ssh -L 13389:<private address of Windows server>:3390 root@<public address of linux server that can access private address of Windows server> -i ~/.ssh/rhel-wca4z-key_rsa.prv -N

• While the ssh command is running, open RDP against localhost:13389 to test log in

• For VPC infrastructure, go to IBM Cloud and open the virtual instance, on the network interface click on the security group, manage rules, remove rule containing remaining port 3390

Configure the data disk

Open the server manager on the ADDI host and select the 300 gig disk:

Start up the New Volume Wizard

Make it Drive D. This will be our storage for the ADDI Data.

We'll call our volume label AD Data

Confirm and Create

Installing Git for Windows

Download the installer from here

Run the installer with all default options.

Installing WinSCP

This is a secure transfer application that will be needed to move files back and forth between the ADDI host and the Refactor Assistant host. The most recent version is 6.3.1 as of this writing.

Download the installer from here

Installing Microsoft SQL Server

As of this writing, the following versions of MS SQL Server are compatible with ADDI:

• Microsoft SQL Server 2014/2016/2017/2019/2022 (Express/Standard/Developer)

For our examples, we will be using the Developer edition of 2022.

On your ADDI host, open the Edge browser and download the installer

Locate the SQL Server installer we downloaded

Run the installer and select Download Media

Select ISO image

Download the ISO image and then right click and mount it.

In our case it was mounted as DVD Drive (E)

Open PowerShell and change to that directory

PS C:\> e:
PS E:\> dir


    Directory: E:\


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         10/8/2022  10:20 AM                1033_ENU_LP
d-----         10/8/2022  10:20 AM                Tools
d-----         10/8/2022  10:19 AM                redist
d-----         10/8/2022  10:19 AM                resources
d-----         10/8/2022  10:20 AM                x64
--r---         10/8/2022  10:20 AM            314 MediaInfo.xml
--r---         10/8/2022   8:39 AM         231328 SqlSetupBootstrapper.dll
--r---         10/8/2022   8:39 AM             45 autorun.inf
--r---         10/8/2022   8:39 AM         133032 setup.exe
--r---         10/8/2022   8:39 AM            532 setup.exe.config


PS E:\>

Let's build our install command here:

Options like SQLSYSADMINACCOUNTS are the admin user. Typically this is the hostname capitlized followed by Administrator.

Ex: WCA4Z-ADDI\Administrator

Also the SAPWD password should be set to your Windows Administrator password.

.\setup.exe /Q /IACCEPTSQLSERVERLICENSETERMS /ACTION=install /TCPENABLED=1 /UpdateEnabled=0 /SQLSYSADMINACCOUNTS="WCA4Z-ADDI\Administrator" /FEATURES=SQLENGINE /SECURITYMODE=SQL /INSTANCENAME=MSSQLSERVER /AGTSVCSTARTUPTYPE=Automatic /BROWSERSVCSTARTUPTYPE=Automatic /PRODUCTCOVEREDBYSA="False" /NPENABLED=1 /SAPWD="<ADMIN PASSWORD>"

Run the setup command.

Manual SQL Server Configuration

Tip: If ADDI configuration has trouble connecting to SQL Server, ensure that the below are set up correctly

The command line based install of SQL Server above should properly set the server configuration (/TCPENABLED=1), but here are the manual instructions if needed.

• Open SQL Server Configuration Manager (Start > Microsoft SQL Server 2019 > SQL Server 2019 Configuration Manager)
• Click Client Protocols under SQL Native Client 11.0 Configuration (32bit) and ensure TCP/IP is set to Enabled
• Click Protocols for MSSQLSERVER under SQL Server Network Configuration and ensure Shared Memory and TCP/IP are set to Enabled
• Click Client Protocols under SQL Native Client Configuration and ensure Shared Memory and TCP/IP are set to Enabled
• Close SQL Server Configuration Manager
• Open Windows Services and restart service SQL Server (MSSQLSERVER)

MS SQL Management Studio

MSSMS can be downloaded here. Download it and install it with default options. It will require a restart of the ADDI host.

Setting up an SQL User Account

Locate the SQL Server Management Studio 19 in Program Files and start it up.

Click Connect

Start SQL Server Management Studio.

• Expand Security > Logins then right-click Logins and choose New Login. Add a login name, select SQL Server Authentication, add a password, and make sure that the default database is set to master. This user is referred to as IBM AD SQL Identity.
• Make sure the addi_user is NOT required to change his password on first login.
• Our user is addi_user and his password is p@ssw0rd

• Under User Mapping, add this user to the master database.

2. Expand Databases > System Databases > master then right-click master and choose Properties. Go to Permissions tab and for the IBM AD SQL Identity and make sure that permissions are granted for: Create database, Create function, Create procedure, Create table, and Create view.

3. The following permission must be granted only if the Rename project feature is used in IBM AD Build, otherwise it is not needed. Right-click on the SQL server instance and then select Properties. In the Server Properties window select Permissions: From the roles list, select the IBM AD SQL Identity and then select Grant for Alter any database permission.

Install Java JDK

Our JAVA SDK will be the open version from Adoptium. It can be downloaded from the following link. Be sure it's vers 11.

https://adoptium.net/temurin/archive/?version=11

Make sure to set both JAVA_HOME and JavaSoft to full install of feature set to local drive

Firewall Ports

Open the Powershell and run the following command to open all required ports:

New-NetFirewallRule -DisplayName "ADDIandSQLPorts" -Direction inbound -Profile Any -Action Allow -LocalPort 1433,3000,9080,9443,2181,2281,2424,2430,2434,2440,7600,7700,7800,7900,7650,7850,9999 -Protocol TCP

Installing ADDI

https://w3south-limited-use.cpc.ibm.com/software/xl/download/ticket.wss

Retrieve the ADDI installer and the IBM Application Discovery and Delivery Intelligence for IBM Z Unlimited Project Activation Kit 6.1.2 from IBM's software repository.

https://www.ibm.com/docs/en/watsonx-code-assistant-4z/1.0?topic=installing-components-addi-installer

Once we have downloaded the installation file, it should show up as a zip file in our Downloads folder called ADDI_FOR_IBM_Z_612_WIN.zip.

Extract C:\Users\Administrator\Downloads\cics-genapp-main.zip.

Expand-Archive -LiteralPath 'C:\Users\Administrator\Downloads\ADDI_FOR_IBM_Z_612_WIN.zip' -DestinationPath 'C:\Users\Administrator\Downloads'

In the File Explorer window, go to C:\Users\Administrator.WCA4Z-ADDI\Downloads\ADDI_FOR_IBM_Z_612_WIN\IBM ADDI

Double click the installer.

IBM_Application_Discovery_and_Delivery_Intelligence_Installer-6.1.2-ifix1.exe

Installation of ADDI via CMDLINE

Alternatively, we can run this installer in quiet mode.

We can create the following file on the desktop called auto-install.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<AutomatedInstallation langpack="eng">
    <com.izforge.izpack.panels.hello.HelloPanel id="HelloPanel_0"/>
    <com.izforge.izpack.panels.licence.LicencePanel id="LicencePanel_1"/>
    <com.izforge.izpack.panels.target.TargetPanel id="TargetPanel_2">
        <installpath>C:\Program Files\IBM Application Discovery and Delivery Intelligence</installpath>
    </com.izforge.izpack.panels.target.TargetPanel>
    <com.izforge.izpack.panels.packs.PacksPanel id="PacksPanel_3">
        <pack index="0" name="IBM Application Discovery Servers and Services" selected="true"/>
        <pack index="1" name="IBM Application Discovery Build Client" selected="true"/>
        <pack index="2" name="Authentication Server (DEX)" selected="true"/>
        <pack index="3" name="Stop Running Services" selected="true"/>
        <pack index="4" name="IBM Application Discovery Web Services" selected="true"/>
        <pack index="5" name="@AN_SERVER_NAME@" selected="true"/>
        <pack index="6" name="IBM Application Discovery Batch Server" selected="true"/>
        <pack index="7" name="IBM Application Discovery Configuration Service" selected="true"/>
        <pack index="8" name="IBM Application Discovery Cross Applications Service" selected="true"/>
        <pack index="9" name="IBM Application Discovery Service Manager" selected="true"/>
        <pack index="10" name="IBM Application Discovery File Service" selected="true"/>
        <pack index="11" name="IBM Application Discovery Mainframe Projects Service" selected="true"/>
        <pack index="12" name="IBM Application Discovery Manual Resolutions Service" selected="true"/>
        <pack index="13" name="IBM Application Discovery Search Service" selected="true"/>
        <pack index="14" name="Open Wizard" selected="true"/>
        <pack index="15" name="Upgrade Wizard" selected="false"/>
        <pack index="16" name="Installer files" selected="true"/>
    </com.izforge.izpack.panels.packs.PacksPanel>
    <com.izforge.izpack.panels.userinput.UserInputPanel id="openConfigWizard">
        <entry key="OpenWizard" value="on"/>
    </com.izforge.izpack.panels.userinput.UserInputPanel>
    <com.izforge.izpack.panels.userinput.UserInputPanel id="userInput">
        <entry key="CCS_IP" value="<HOST IP>"/>
        <entry key="CCS_PORT" value="2181"/>
    </com.izforge.izpack.panels.userinput.UserInputPanel>
    <com.izforge.izpack.panels.install.InstallPanel id="InstallPanel_7"/>
    <com.izforge.izpack.panels.shortcut.ShortcutPanel id="ShortcutPanel_8">
        <createMenuShortcuts>true</createMenuShortcuts>
        <programGroup>IBM Application Discovery and Delivery Intelligence</programGroup>
        <createDesktopShortcuts>false</createDesktopShortcuts>
        <createStartupShortcuts>false</createStartupShortcuts>
        <shortcutType>all</shortcutType>
    </com.izforge.izpack.panels.shortcut.ShortcutPanel>
    <com.izforge.izpack.panels.finish.FinishPanel id="FinishPanel_9"/>
</AutomatedInstallation>

Update <HOST IP> to our ADDI host address.

Now open a PowerShell window and navigate to the ADDI install path and run the installer.

cd "C:\Users\Administrator\Downloads\ADDI_FOR_IBM_Z_612_WIN\IBM ADDI"`

java -jar .\IBM_Application_Discovery_and_Delivery_Intelligence_Installer-6.1.2-ifix1.exe -f "C:\Users\Administrator\Desktop\auto-install.xml"

Command line arguments: -f C:\Users\Administrator\Desktop\auto-install.xml
[ Starting automated installation ]
====================
Installation started
Framework: 5.1.3-84aaf (IzPack)
Platform: windows,version=10.0,arch=x64,symbolicName=WINDOWS_10,javaVersion=11.0.22
[ Starting to unpack ]
[ Processing package: IBM Application Discovery Servers and Services (1/16) ]
[ Processing package: IBM Application Discovery Build Client (2/16) ]
[ Processing package: Authentication Server (DEX) (16/16) ]
[ Processing package:  (3/16) ]
[ Processing package:  (4/16) ]
Cleaning up the target folder ...
[ Processing package:  (5/16) ]
[ Processing package:  (6/16) ]
[ Processing package:  (7/16) ]
[ Processing package:  (8/16) ]
[ Processing package:  (9/16) ]
[ Processing package:  (10/16) ]
[ Processing package:  (11/16) ]
[ Processing package:  (12/16) ]
[ Processing package:  (13/16) ]
[ Processing package:  (14/16) ]
[ Processing package:  (15/16) ]
[ Unpacking finished ]
Installation finished
[ Writing the uninstaller data ... ]
[ Automated installation done ]

Installation of ADDI via UI

https://www.ibm.com/docs/en/watsonx-code-assistant-4z/1.0?topic=configuring-by-using-ad-configuration-wizard

Most of these settings are defaults.

Enable the DEX server

Keep our external IP as the Config service IP. In this case, our VM is 10.241.64.4. But if you do have a FQDN, use that instead, ex wca4z2-winaddi.fyre.ibm.com.

At some point, the installer will spin up the ADDI web dashboard to continue the installation

Configuring ADDI via WEBUI

Set our Relational Database Server info to point to our local SQL Server. Use the addi_user we configured with his password p@ssw0rd.

In our instance we're setting "D:" to be our data drive, so let's set it up that way.

On the Relational Database Server page, enter the following information:

Database names can stay default

We set the Graph Database user password to P@ssw0rd

Batch server settings can stay default

Allow the Health Check to create the GenApp project.

Cross your fingers and commit.

So this is expected behavior:

Click Done and the ADDI Dashboard should open.

The following services should be offline or disabled in the Dashboard

• Authentication server (DEX)
• Catalog service
• Validation service

Data Generator

• This is something already in the Velocity images

• Download latest zip from https://eu.artifactory.swg-devops.com/ui/native/zeus-ez-public/ez/sd/wca4z-data-generator/ (requires w3 login)

• Can't download, either no access or it doesn't exist at that spot

• Found zip file WCA4Z-Data-Generator.zip in box somewhere and downloaded it

README.txt Watson Code Assistant for Z Data Generator - 1.0.0

Generates Control Flow data and DDL database information from an extracted service and writes the information in a DB2 on Cloud database.

Prerequisites

1. The basic requirements for running the Watson Code Assistant for Z Data Generator (WCA4Z) are the same as IBM Application Discovery and Delivery Intelligence (ADDI) and can be found at https://www.ibm.com/docs/en/addi/6.1.2?topic=installing-prerequisites.
2. ADDI is already installed and configured on the same machine where Watson Code Assistant for Z Data Generator is going to be installed. More information available at https://www.ibm.com/docs/en/addi/6.1.2?topic=guide-installing

Setup

1. Extract the contents of the WCA4Z Data Generator ZIP file into the <ADDI_INSTALLATION_FOLDER> folder. The ZIP file contains a folder called WCA4Z Data Generator so after extracting, the contents of that folder should be available in <ADDI_INSTALLATION_FOLDER>\WCA4Z Data Generator.
2. Add an IBM Db2 relational database server from IBM Application Discovery Configuration Service Admin > Configure > <ENV_NAME> > Servers and security > Relational database servers.. This should be an instance of IBM Db2 on Cloud.
3. Create a new mainframe project:
   • Open IBM Application Discovery Build Client.
   • Select File > New Project.
   • Choose Mainframe main languages as the project type and click on the Next button.
   • Enter a the project name, for example ExtractedServices.
   • Select the Project DB Type and choose the IBM Db2 on Cloud database server added in the previous step.
   • Click on the Finish button to create the new project.
4. Create a new GitHub repository for the source files extracted from IBM Refactoring Assistant. The structure of the repository should the following folders:
   • Cobol Programs
   • Copybooks
   • DDL
5. Enable project file synchronization. More information available at https://www.ibm.com/docs/en/addi/6.1.2?topic=configurations-filling-ad-build-client-install-configuration-page
   • Copy the file <ADDI_INSTALLATION_FOLDER>\WCA4Z Data Generator\samples\SyncFile.txt to a new location, for example the ADDI project sources folder C:\IBM AD\Mainframe Sources.
   • Open IBM Application Discovery Configuration Service Admin > Configure > Install configurations > IBM Application Discovery Build Client install configuration.
   • Check the "Enable members synchronization" option.
   • Fill in the full path to the SyncFile.txt, for example C:\IBM AD\Mainframe Sources\SyncFile.txt.
   • Save the changes by clicking on the Save button.
6. Open the configuration file located in <ADDI_INSTALLATION_FOLDER>\WCA4Z Data Generator\conf.properties.
7. Edit the values to match your ADDI installation details. Default values are provided as an example and they could be used as is or modified.
   • ADDI_PROJECT_NAME=ExtractedServices - This is the project created in IBM Application Discovery Build Client and will be used to build the extracted services COBOL files.
   • ADDI_PROJECTS_FOLDER=C:\IBM AD\Mainframe Projects - This is the root folder for all projects. It has been configured during the ADDI initial setup and it can be accessed from IBM Application Discovery Configuration Service Admin > Configure > Install configurations > IBM Application Discovery Build Client install configuration.
   • PROJECT_GITHUB_CLONE_FOLDER=C:\Extracted-Services - This is the GitHub repository folder where the extracted services COBOL files are committed and updated.
   • DDL_FOLDER=C:\Extracted-Services\DDL - This is the folder where DDL and JCL files containing database information are added. Multiple files can be added to the folder but only the most recently updated one will be processed.

Run

1. Run the automation.bat script with no additional arguments from Command Prompt.
2. The script is executing synchronously and will return a zero exit code for a successful execution or a non zero exit code otherwise.
3. If new programs have been built successfully but the metadata import fails for any reason, those programs will be saved for retry with the next run.
4. A new runControlFlowGenerator.log log file will be generated in the same folder containing the full output of the execution.

• TODO: Check that RA is still running after https set up