Skip to main content

Parameters

Parameters are variables or keys that enable users to dynamically control the execution of a rule. Through parameterization, a single rule can be executed with different sets of values by overriding parameters at runtime or within workflows.

Key Features

  • Enhances reusability across different environments.
  • Improves test coverage and execution flexibility.
  • Allows dynamic customization of rule execution.

Use Cases

  • Parameterizing Database Names: When working with multiple environments (e.g., staging, development, production) that share the same schema and table structures, users can parameterize the database name to apply the same rule across these environments.
  • Filtering Data for Delta Validation: Instead of validating the entire dataset, users can filter records based on fields like insert date or update date. These values can be passed dynamically at execution time.
  • Parameterizing File Names and Paths: File names and file paths used within rules can be parameterized to allow for dynamic modification.
  • Parameterizing SQL Statements and Checks: Hardcoded values in SQL queries or validation checks can be replaced with parameterized keys, enabling flexible and reusable logic.

How To: Create a Parameter (File)​

Follow the steps below to create a parameter file with different key-value pairs in iceDQ:

  1. From the main application menu, go to the Data Testing section.
  2. Within the Data Testing section, select the Parameters tab to view and manage parameters.
  3. Click the New Parameter button to start the creation of a new parameter file.
  4. Configure Parameter File Settings:
    • Folder: Browse and select the workspace folder where the parameter file will be saved.
    • Parameter File Name: Enter a unique and descriptive name to identify the parameter file within the selected folder.
    • Parameter File Upload: iceDQ allows users to import parameters from a file. Supported file types are .xlsx and .csv. Refer to the section "How To: Import Parameters" for detailed instructions.
    • Next: Click to proceed to the parameter file page and add key-value pairs.
    • Cancel: Exit the parameter creation process without saving.
  5. Add Parameters:
      To add key-value pairs:
       a. Click Add in the Details section.
       b. Enter the key in the Key text field.
       c. Enter the runtime replacement value in the Value text field.
       d. (Optional) Provide additional details in the Notes text field.
       e. Click the ✔ (check) button to save the entry.
       f. (Optional) Click the ✖ (cross) button to discard the entry.
  6. Continue adding key-value pairs as needed by repeating Step 5.
  7. Click the Save button to finalize and store the parameter file.

The video illustrates how users can create a parameter file.

Additional options​

  • For parameter file:
Field NameDescription
DiscardThis option becomes available after at least one parameter is added and remains active until the Save option is selected. During initial creation, clicking Discard will remove all added parameters. After the parameter file has been saved, pressing Discard will revert only the changes made during the most recent edit session.
Duplicate    Duplicates the current parameter file. This option is available after the parameter file has been saved.
DeletePermanently deletes the parameter file. This option is also available only after the parameter file has been saved.
  • For parameters:
Field NameDescription
ImportAllows users to upsert (update or insert) parameters from an external file. This option becomes available once the parameter file is saved. Refer to the section "How To: Import Parameters" for detailed instructions.
SearchEnables users to search for specific parameter key values within the parameter file.
Delete delete button   Allows deletion of one or more parameters after selecting them using the ✔️ button.
  • For individual parameters:
Field NameDescription
Delete delete buttonDelete the individual parameter.
Edit ✏️Opens the parameter for editing, allowing modification of its key, value, or notes.

How To: Use Parameters​

Step 1: Associating a Parameter File​

  1. From the main application menu, navigate to the Data Testing section.
  2. In the Rules tab, search for and select the desired rule within the workspace.
  3. Go to the Overview section and expand Advanced Settings.
  4. In the Parameter dropdown, select the parameter file that needs to be associated with the rule.
    Note: To edit the selected parameter file, click the ✏️ button next to the dropdown.

Step 2: Using Parameter Keys​

In SQL Statements: Parameters can be inserted into SQL statements using the #KEYNAME# syntax.

SELECT * FROM #dbkey#.SALES.CUSTOMER WHERE ETL_INSERT_DTTM::DATE = '#datekey#'

In Checks: Instead of hardcoding values, replace them with parameter keys.

// before parameter 
S.[CUSTOMER_TYPE] in ('Individual')

// after parameter
S.[CUSTOMER_TYPE] in (#custkey#)

The video illustrates how users can use parameters.

Parameterizing a File Path​

When working with files such as CSV, XML, or JSON, you may need to parameterize part of the file path or name. For example, if a new customer file is generated daily with the date appended (e.g., customer_20250101.csv), you can configure your rule to dynamically reference the correct file by parameterizing its path or name.

To achieve this, the Parameterized Path property is available in every File Connection screen. Enabling this option allows you to parameterize either the entire file path or a specific portion of it. Simply enclose the parameter key between two hash (#) symbols.

important

The base path defined in the connection cannot be parameterized.

The video illustrates how users can parameterize a file path.

Overriding Parameters in a Workflow​

Users can override parameters during workflow execution to run the same rule with different sets of values.

  1. Create a new workflow
  2. Add the same rule twice to the workflow.
  3. Assign different parameter files to each instance
  4. Execute the workflow.

The video illustrates how users can override parameters in a workflow.

How To: Import Parameters​

Users can import keys and values from a CSV or Excel file while creating a parameter file or if it is already created.

  1. Navigate to the Parameter Section in the Data Testing module.
  2. Click on + New Parameter.
  3. Provide a unique name file name.
  4. Upload the CSV or Excel File.
  5. Click Next.
  6. Click Save.

The video illustrates how users can import parameters from a file.

Import File​

When importing parameter key-value pairs in bulk, the CSV or Excel file should follow the structure outlined below:

parameter_keyparameter_valueparameter_notesparameter_type
dbkeyPRODEnvironment
datekey2/4/2025Load date
typekey'Individual', 'Corporate'Customer types
newkey100Dummy

Guidelines

  • parameter_key: The unique identifier for the parameter.
  • parameter_value: The value assigned to the parameter key.
  • parameter_notes: Additional context or description for the parameter.
  • parameter_type: (Optional) Used to categorize or describe the parameter further.

Behaviour

  • If a Key Exists: The system updates its associated value.
  • If a Key Does Not Exist: A new key-value pair is added.
Note

The system automatically validates the data in the CSV/Excel file to ensure accuracy. If any errors are detected (e.g., missing fields or incorrect data types), an error message will highlight the issues. Simply correct the errors in your file and re-upload for seamless processing.

How To: Copy a Parameter File​

  1. From the main application menu, navigate to the Data Testing section.
  2. Within the Data Testing section, select the parameters tab to view and manage parameter files.
  3. Select one or more parameter files to be copied. Use ✔️ button.
  4. Click on the More > Copy option to copy the parameter file.
  5. Search and select the appropriate workspace folder where the parameter file will be copied.
  6. Select Copy option.
  7. Cancel: Select this option to exit the parameter file copy process without saving changes.

How To: Move a Parameter File​

  1. From the main application menu, navigate to the Data Testing section.
  2. Within the Data Testing section, select the parameters tab to view and manage parameter files.
  3. Select one or more parameter files to be Moved. Use ✔️ button.
  4. Click on the More > Move option to move the parameter file.
  5. Search and select the appropriate workspace folder where the parameter file will be moved.
  6. Select Move option.
  7. Cancel: Select this option to exit the parameter file move process without saving changes.

How To: Delete a Parameter File​

  1. From the main application menu, navigate to the Data Testing section.
  2. Within the Data Testing section, select the parameters tab to view and manage parameter files.
  3. Select one or more parameter files to be Deleted. Use ✔️ button.
  4. Click on the More > Delete option to move the parameter file.
  5. Select Delete option.
  6. Cancel: Select this option to exit the parameter file move process without saving changes.
warning :

Deleting parameters or parameter files should be done with caution. If the deleted parameters are currently in use by any rules or workflows, it may cause them to fail or produce inconsistent results during execution. Therefore, it's important to assess the potential impact before proceeding.

Considerations​

  • Parameter file name is unique within a folder.
  • Parameter (File) is created inside a Folder that belongs to a specific Workspace.
  • Parameters can only be used in the Rules and other entities that belong to the same Workspace.
  • Users with Owner or Contributor Role can create, update, or delete a Parameter (File).
Last updated on