Get Started

Introduction

The aim of this documentation is to describe how to use the Data Migrator to build a database over its architectural items and to populate it with data. 

This tool consumes metadata artifacts that could be provided from:

  • Result artifacts coming from Analyzer step when parses DDL DB2
  • Result artifacts coming from Analyzer step when parses PF and LF files in AS400 projects

The Data Migrator is a standalone application. It is provided as Linux or Window installer and is launched through a command line.

The Data Migrator enables to work with several kinds of database :

  • POSTGRESQL
  • MSSQL
  • ORACLE
  • DB2 (LUW) (restricted to Creation or Deletion step)

POSTGRESQL

Nothing in particular. 

To bulk-load data, Bluage Data Migrator uses POSTGRESQL provided copy statement.

MSSQL

To bulk-load data, Data Migrator uses MSSQL provided BULK INSERT statement. To enable bulk-loading, you have to specify it, for example, with Microsoft SQL Server Management Studio:

 

mssql.png

ORACLE

To bulk-load data, Data Migrator uses sqlldr tool which must be installed on your machine with a well-defined path.

Prerequisites

  • You need to have access to the Data Migrator S3 bucket. If you don’t have access yet, you can request it via Blu Insights Toolbox.

  • You need to have minimal IAM policies on your account to download the Data migrator from S3 bucket. In case you don't have policies, then create one with the below information.
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "s3:ListBucket"
          ],
          "Resource": [
              "arn:aws:s3:::toolbox-data-migrator"
          ]
      },
      {
          "Effect": "Allow",
          "Action": [
              "s3:GetObject"
          ],
          "Resource": [
              "arn:aws:s3:::toolbox-data-migrator/*"
          ]
      }
  ]
}
  • Each empty database must be first manually created.

Installation guide

  1. Check that you configure the AWS credentials with the AWS account used in your request made in Blu Insights Toolbox.
  2. Download Data migrator last version archive using the command aws s3 cp --recursive s3://toolbox-data-migrator/latest LOCAL_PATH
  3. Unpack the archive present in LOCAL_PATH.

How to launch it

To launch the steps migration in command line, open your favorite shell and type:

BluageVelocityDataMigrator.exe -root [absolute path to your reverse project] -configurationIni 
[absolute or relative path to your migration configuration file];[absolute or relative path to your migration execution file]

The relative paths in the command will be resolved to the specified reverse project.

Options

The following options can be used when launching the migration using the command line:

  • help : Display the help; it includes the options below.
  • root : Root directory for the relative paths in the command line. Usually the path of your migration/reverse project.
  • version : Display the tool version.
  • configurationIni : Load the configuration from ini files, separated by semicolon.

How it works

Data Migrator tool connects to databases specified in the configuration steps and, for POSTRESQL only, drops and creates the database.       
The tool applies all the enabled action steps specified.       
For the creation or deletion step, it executes scripts from the sql files included in the input directory of the step.       
For the conversion step, it converts the data included in the input directory of the step and put the results in the output directory of the step.       
For the data loading step, it loads data included in the input directory of the step to database.       
During the data loading step, considered table to fill are taken from sqlModel.json table list or collected from the sql files if sqlModel.json is missing. Under the designed data folder (dataFolder), data files must be stored under folder named as the table name.

data-folder.png