Extension Technology Configuration File

The dxeconfig.cfg file is used to globally manage your ODBC Extension or Oracle Extension settings. By default, it contains the following settings:

* Data Express Configuration File

LogLevel:1
Always Drop Table:N
CSV Data Store:N
COBOL Masking:N
Validate Target Table:N
Validate Source Schema:N
Enable Logging:N
Hide Progress:N
Hide Log Message:N
Use Extended C Masking:N
Trim Spaces for ODBC Char Columns:N
Commit Size(MBs) [user range = from 1 to 4000, default range = 0]:0
Parse String finding LOW-VALUES:N
Create Table using CHAR clause:N
Execute scripts when the process aborts:N
Execute disable/enable CONSTRAINTS:N
Execute Aborted Tables:N
Execute CONVERT:N
Order by:0
Process Sequence:N
Runtime KB SCHEMA:DEKB

* These entries are only applicable to the Oracle Extension.

Text Output:N
Data Separator:,
Fields Enclosed By:"
Identify Externally:N
Use Append Condition:N

The following settings can be specified:

LogLevel

Controls the level of detail displayed in the dxe_errorlog.txt file. You can change the LogLevel as needed or when requested to do so by Product Support:

LogLevel	Description
1	Default log level that provides a summary for each processed data store.
2	Verbose output used for diagnostic purposes that contains the SQL statements executed against your source and target data stores.
3	Generates ODBC trace output that is written to odbctrace.txt.

Always Drop Table

Determines whether or not to always execute DROP TABLE, and what, if any, additional action is taken. Possible values are:

Y	Data Express deletes the target table and replaces it with the data coming from the source, reduced and masked by Data Express. To do this, Data Express executes a DROP and then a CREATE.
N	Data Express deletes the target table and replaces it with the data coming from the source, reduced and masked by Data Express. To do this, Data Express executes a DELETE.
K	Data Express keeps the records already in target table. If a record extracted by Data Express has the same key as an existing record, the existing record is kept and the extracted record is discarded. In all other cases, records extracted by Data Express are added to target table.
O	Data Express keeps the records already in target table. If a record extracted by Data Express has the same key as an existing record, Data Express overwrites the existing record with the extracted record by first deleting the existing record with a DELETE statement using the key condition, and then adding the extracted record. In all other cases, records extracted by Data Express are added to target table.

Note: When Validate Target Table is set to Y, Data Express performs validation before processing any tables to ensure that the structures of your source and target data stores are exactly the same. If the structures differ and Always Drop Table is set to Y, the method does not execute and an error message displays instructing you to change your Always Drop Table setting in order to proceed with the creation of your test environment.

Examples - Always Drop Table:Y

SQL statements, Data Express executing:

DROP TABLE DETRG."CUSTOMER" CASCADE CONSTRAINTS

CREATE TABLE DETRG."CUSTOMER" ("COD_CUS" CHAR(10 BYTE), "NDG" CHAR(30 BYTE), "NAME" CHAR(40 BYTE), "SURNAME" CHAR(40 BYTE), "DATE_OF_BIRTH" CHAR(8 BYTE), "PLACE_OF_BIRTH" CHAR(50 BYTE), "NUMTEL" DECIMAL(15, 0))

INSERT INTO DETRG."CUSTOMER" ("COD_CUS", "NDG", "NAME", "SURNAME", "DATE_OF_BIRTH", "PLACE_OF_BIRTH", "NUMTEL") VALUES (?, ?, ?, ?, ?, ?, ?)

Examples - Always Drop Table:N

SQL statements, Data Express executing:

DELETE FROM DETRG."CUSTOMER"

INSERT INTO DETRG."CUSTOMER" ("COD_CUS", "NDG", "NAME", "SURNAME", "DATE_OF_BIRTH", "PLACE_OF_BIRTH", "NUMTEL") VALUES (?, ?, ?, ?, ?, ?, ?)

Example - Always Drop Table:K

SQL statement, Data Express executing:

INSERT INTO DETRG."CUSTOMER" ("COD_CUS", "NDG", "NAME", "SURNAME", "DATE_OF_BIRTH", "PLACE_OF_BIRTH", "NUMTEL") VALUES (?, ?, ?, ?, ?, ?, ?)

Examples - Always Drop Table:O

SQL statements, Data Express executing:

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="1234567890"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="1123242526"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="109293847"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="9543217890"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="8463728190"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="754321980"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="5678901232"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="5567890123"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="4567890123"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="3456789012"

DELETE FROM DETRG."CUSTOMER" WHERE "COD_CUS" ="2345678901"

INSERT INTO DETRG."CUSTOMER" ("COD_CUS", "NDG", "NAME", "SURNAME", "DATE_OF_BIRTH", "PLACE_OF_BIRTH", "NUMTEL") VALUES (?, ?, ?, ?, ?, ?, ?)

CSV Data Store

Controls whether the tables referenced by a given method should be stored in text files delimited by commas. To enable, change the CSV Data Store parameter from N to Y.

COBOL Masking

Controls whether you are using COBOL routines to mask your data. Possible values are:

Y	By default, COBOL Masking is set to Y.
N	If you want to use C masking routines, change the COBOL Masking parameter to N.
O	When the value is "O" (from Others), the product can use any language satisfying the following requirements: Be built to a callable shared object (UNIX), or DLL (Windows), with the relevant entry points exposed/exported as required. Be callable from C. Have the variables coded precisely as per the COBOL copybook UTEUSFLW.CPY (noting that the content/structure differs slightly from the mainframe in order to more closely mimic C data types), both regarding data type and size. Cope with platform-specific byte ordering of numeric data. The C engine code and COBOL copybook expect all numeric parameters to be passed in machine-specific byte ordering. (For example, Intel platforms such as Linux and Windows use the opposite byte ordering to UNIX platforms such as HP-UX, Solaris and AIX).

Validate Target Table

Controls whether the target tables are validated against the source tables to ensure that the structures are exactly the same. By default, Validate Target Table is set to Y. If you do not want to validate the table structures , change this parameter from Y to N.

Note: You may not want to validate the target table because if the validation fails (perhaps due to differing column numbers) dropping the table and then recreating the table will create the table in a different tablespace. Set the Validate Target Table parameter to N only if you are the target and source tables match.

Validate Source Schema

Controls the usage of source schema. If Y, the target schema name is the same as the source one. If N, the default target schema name is used. By default, Validate Source Schema is set to Y.

Enable Logging

For more information about logging, see Logging Exit Routines in the Data Masking Guide.

Hide Progress

Suppresses visual progress during validation.

Hide Log Message

Controls the display of log messages in the dxe_errorlog.txt file when Enable Logging is used.

Use Extended C Masking

Controls the use of extended C masking. If N, standard masking is used. This the default. If set to Y, extended C masking is used. For information on the differences between standard C masking and extended C masking, see Appendix B of the Data Masking Guide.

Trim Spaces for ODBC Char Columns

Optional. If Y, final spaces in ODBC CHAR columns are replaced with low-values (x00). If N or the Trim Spaces for ODBC Char Columns parameter is omitted, final spaces in ODBC CHAR columns are left unchanged.

Commit Size(MBs)

Optional. Controls the frequency of commits, and can be used to process very large numbers of inserts within the same transaction.

0	A commit is performed immediately after processing 95 MB; one commit for each 95 MB processed. Default.
`n`	A numerical range from 1 to 4000 indicating the number of processed megabytes (MBs) after which to perform each commit. Any number over 4000 is interpreted as 4000.

Parse String finding LOW-VALUES

Determines whether or not a column is masked based on its use of LOW-VALUES.

N	When the first character of a column set to be masked is LOW-VALUES, the column is not masked. Default.
Y	When the column contains any characters other than LOW-VALUES, it is masked.

Create Table using CHAR clause

When the DROP TABLES option is used, determines whether to create a table using the CHAR data type for CHAR columns only, or to create it using the CHAR data type for both CHAR and VARCHAR columns.

N	When the DROP TABLES option is used, the CREATE TABLE statement uses the CHAR data type for CHAR columns only. Default.
Y	When the DROP TABLES option is used, the CREATE TABLE statement uses the CHAR data type for VARCHAR columns.

Execute script when the process aborts

Determines whether or not to execute pre and post scripts when the process aborts.

N	Pre and post scripts are not executed when the process aborts. Default.
Y	Pre and post scripts are executed when the process aborts.

Execute disable/enable CONSTRAINTS

Determines whether or not Data Express provides ENABLING/DISABLING SQL commands to preserve existing CONSTRAINTS relationships.

N	No constraints are created. Default.
Y	DISABLE/ENABLE constraints are created, but only when Always Drop Table:N is set.

Execute Aborted Tables

Determines whether or not the engine logs all successfully processed data stores or only the failed data store elaborations from the previous run.

N	The engine works normally, and logs the successfully processed data stores. Default.
Y	The engine processes only the failed data store elaborations from the previous run.

Execute Convert

Specifies whether or not to execute a convert. Optional

N	No convert executed. Default.
Y	Executes convert with AL32UTF8.
1	Executes convert with AL16UTF16.

Order By

Optionally specify an ORDER BY clause within the major SELECT * statement. The value specified can be any valid ORDER BY value.

0 or not specified	No ORDER BY is executed. Default.
`col-num`[`spec`][,...]	Where `colnum` represents a column number you want to use, and `spec` represents any valid ORDER BY specification.

Examples

Order by:1,2,3,4

Writes a query adding ORDER BY 1,2,3,4

Order by: 1 desc, 2 asc

Process Sequence

Determines whether or not the engine processes Oracle sequences. Optional.

Not specified	Oracle sequences are not processed. Default.
Y	Process Oracle sequences.

Important: Data Express does not create sequences. Therefore, you must create the target sequence prior to execution.

For Data Express to update the target sequences, it needs some or all of the following information provided by the HSENVSEQ table, which must be updated manually. See HSENVSEQ in your Data Model Guide for details:

Source database name: The name of the database that holds the sequence name.
Source schema name: The schema name that holds the sequence name.
Note: This schema name must be different than the table schema name.
Sequence name: The name of the sequence assigned to a column in the table currently being processed.
Column name: The name of the column using the sequence held by the source schema.
Target schema name: The name of the schema where you want the target sequence to be created.
Target sequence name: The name of the sequence assigned to a column in the target environment.
Note: This schema name must be different than the table schema name.
Target table name: The name of the table that is being sequenced.

Runtime KB SCHEMA

When specified, contains a value different from the default DEKB, which enables you to specify the schema name of the tables belonging to the runtime knowledge base.

Text Output

Oracle Extension only. By default, Data Express inserts the processed data into the specified target data store. You can configure Data Express to write the output to a text file instead, which can be subsequently loaded into an Oracle database using the Oracle sqlldr routine.

Y	Enables text output. Always drop table parameter is ignored.
N	Disables text output. Default.

Optionally, you can prepare a text file, code-table.txt, containing a list of coupled tables using code separated by a semicolon (;), with one row for each table. The code is used to prepare the name of the target text file. If code-table.txt is not present in the configuration folder, Data Express uses the table name as the name of the output file.

When enabled, Data Express generates the following files in the log directory after the extraction has executes:

DX_Oracle_output_summary.dat	Shows all relationships between table name and output file with one row for each table.
DX_Oracle_output_`tablename`.dat	One output file for each table.
`tablename`.ctl	One command file for each table, each containing the load statement using the Oracle sqllrd command.

Customized reload commands

You can use the Oracle prototype utility, prototype.ctl in your configuration folder, to generate all the reload command files. This enables you to manage any configuration by simply editing the prototype and adding or modifying what you need.

Each generated .ctl file is made by copying the prototype row by row, and actualizing the following keywords:

&filename - replaced by the csv file name.
&tablename - replaced by the fully qualified table name.
&columnlist - replaced by column list.

Note: If no prototype.ctl file is found in your configuration folder, a valid syntax .ctl file is generated; however, no customization takes place.

Customized reload example

In this example you can see how prototype.ctl could be edited to generate the corresponding .ctl file.

prototype.ctl:

load data
infile 'D:\Stefano\Lavoro\Development\CUSTOMERS\Produban-Santander\VOLTAGE\Run\output_csv\&filname.csv'
into table &tablename
fields terminated by ',' optionally enclosed by '"'
&columnlist

Generated command file:

load data
infile 'D:\Stefano\Lavoro\Development\CUSTOMERS\Produban-Santander\VOLTAGE\Run\output_csv\DX_Oracle_output_file.002.csv'
into table "DETRG"."CP_ORDEN_CONTR"
fields terminated by ',' optionally enclosed by '"'
(E24_IDEMPR CHAR(4), E24_IDCENT CHAR(4), E24_CODPROD CHAR(3), E24_NUMSOR CHAR(7), E24_IDEMPR2 CHAR(4), E24_IDCENT2 CHAR(4), E24_CODPRODC CHAR(3), E24_NUMCTR CHAR(7), E24_FECINI CHAR(35), E24_FECVEC CHAR(35), E24_INDBOR CHAR(1), E24_CORELCC CHAR(3), E24_FCORIGE CHAR(35))

Data Separator

By default, the delimiter used to separate Oracle fields is the comma (,). You can override this default by specifying one or more characters.

Fields Enclosed By

By default, strings are enclosed by double quotation marks ("). You can override this default by specifying one or more characters.

Note: Parameters and combined behavior: Text that is not enclosed by double quotes is ignored.

If the target table does not exist, the Extension Technology executes a CREATE TABLE statement and then INSERT operations in order to create it. However, if the target table already exists, the way the Extension Technology behaves depends on the settings specified in the dxeconfig.cfg file. For example:

Always Drop Table	Validate Target Table	Resulting Behavior
Y	n/a	A DROP TABLE statement is executed before the CREATE TABLE statement and INSERT operations.
N	Y	If the target table's metadata matches the source table's metadata exactly, the method will be executed. If any mismatch in the metadata is found, the method will not be executed. For example, if the source table is defined as: CREATE TABLE SRC.MYTABLE(C1 INT, C2 CHAR(5), C3 DECIMAL(5,2)) and the target table exists, but is defined as: CREATE TABLE TGT.MYTABLE(C3 DECIMAL(5,2), C1 INT, C2 CHAR(5)) the method will not be executed. Note: The The `Always Drop Table` parameter set to `N` and the `Validate Target Table` parameter set to `Y` are default settings.
N	N	The Extension Technology will not perform any validation. If the target table does not include all columns from the source table (regardless of order), the INSERT operation will fail and the elaboration will terminate abnormally. For example, if the source table is defined as: CREATE TABLE SRC.MYTABLE(C1 INT, C2 CHAR(5), C3 DECIMAL(5,2)) and the target table exists, but is defined as: CREATE TABLE TGT.MYTABLE(C3 DECIMAL(5,2), C1 INT, C2 CHAR(5)) the method should execute and complete normally. If the target table is instead defined with a column `C5` instead of `C3`: CREATE TABLE TGT.MYTABLE(C5 DECIMAL(5,2), C1 INT, C2 CHAR(5)) the INSERT statement will fail, causing the method to terminate abnormally.

Always Drop Table

Validate Target Table

Resulting Behavior

n/a

A DROP TABLE statement is executed before the CREATE TABLE statement and INSERT operations.

If the target table's metadata matches the source table's metadata exactly, the method will be executed. If any mismatch in the metadata is found, the method will not be executed.

For example, if the source table is defined as:

 CREATE TABLE SRC.MYTABLE(C1 INT, C2 CHAR(5), C3 DECIMAL(5,2))

and the target table exists, but is defined as:

CREATE TABLE TGT.MYTABLE(C3 DECIMAL(5,2), C1 INT, C2 CHAR(5))

the method will not be executed.

Note: The

The Always Drop Table parameter set to N and the Validate Target Table parameter set to Y are default settings.

The Extension Technology will not perform any validation. If the target table does not include all columns from the source table (regardless of order), the INSERT operation will fail and the elaboration will terminate abnormally.

For example, if the source table is defined as:

 CREATE TABLE SRC.MYTABLE(C1 INT, C2 CHAR(5), C3 DECIMAL(5,2))

and the target table exists, but is defined as:

 CREATE TABLE TGT.MYTABLE(C3 DECIMAL(5,2), C1 INT, C2 CHAR(5))

the method should execute and complete normally.

If the target table is instead defined with a column C5 instead of C3:

CREATE TABLE TGT.MYTABLE(C5 DECIMAL(5,2), C1 INT, C2 CHAR(5))

the INSERT statement will fail, causing the method to terminate abnormally.

Identify Externally

Oracle Extension only. Identifies connection criteria for connecting to source and target databases.

Y	Data Express uses the information in the configured Wallet rather than the method.rc file to connect to the database, eliminating the need to specify user name and password credentials.
N	Database connection requires that you specify a user name and password.

Using Append Condition

Optional. Oracle Extension only.

Y	During the execution of the dxestart module, adds the APPEND condition to the INSERT statement and the NOLOGGING condition to the CREATE TABLE statement. Important: For the full benefit, you must also set Always Drop Table to Y.
N, or the Using append condition parameter is omitted	Nothing is added during the execution of the dxestart module.