Pipeline Integrity Report

This page documents a new feature that is available in edgeSuite v3.6 or higher.

Upon saving changes to the pipeline, the system will scan for any issues with Dataset Variables (Node Variables). Any potential issues found are displayed in a report that allows the user to choose the appropriate solution for each problem. Many of the solutions offer propagation support, which will automatically fix all related issues downstream on the pipeline the same way. This report makes it much easier to change the behavior of an entire pipeline chain and make changes to Datasets in the middle of a pipeline.

You may navigate through all the reported issues in two different ways. The pipeline view above the report allows you to pan and view the pipeline chain that has issues. Clicking a dataset node in the topology will highlight the issue in the list below. You may also just scroll the report and handle each issue one at a time.  The report is in depth order when possible.

The report identifies four different pipeline issues:

Issue Detected
Detection Process
Solutions Presented
Invalid Upstream Variable Mapping The scanning code will iterate over each variable mapping to upstream datasets looking for variables that no longer exist on the parent that it is currently configured to provide a value to. A list of these are maintained and presented to the user.

This issue is usually encountered when an upstream node variable has been deleted.

  • Remove
  • Ignore

Optionally you can choose to have the removal propagated downstream to all other datasets with the same issue.

Upstream Variable Dependency Not Met The scanning code will locate any upstream dataset variables that have not been properly satisfied. Either they are missing a mapping value entirely, or a problem with that mapping has been identified.

This issue is usually encountered when an upstream node variable has been created.

  • Auto Inherit
    (Optional Propagate)
  • Local Variable
  • Hide
  • Advanced

More Details Below Table

Local Variable Not Used The scanning code looks at all the local dataset variables and ensures that they are either used in an upstream mapping, or a local property that allows variable parsing (for example, SQL code)

This issue is usually encountered when you add a node variable and do not use it anywhere.

  • Remove
  • Ignore
Referenced Upstream Variable Renamed The scanning code has a list of all the node variables that have been renamed upstream. If it detects that child datasets are still referring to the old name, it provides options for propagating the rename.

This issue is usually encountered when you rename a node variable.

  • Propagate Name Change When Possible
  • Do Not Propagate Name Change

Invalid Upstream Variable Mapping Solution Options #

Option
Definition
Auto Inherit Auto-create a local variable with the same settings as the parent variable and assign. Propagating this change downstream is optional.
Local Variable

Choose a local variable to satisfy the upstream variable. Any downstream datasets must satisfy this local variable.

  You can use the control on the left to choose an existing local variable, create a new variable, or edit the currently selected variable.

Hide Set the value of the parent dataset variable to its default value, and terminate the need to satisfy the variable, which hides it downstream.
Advanced Advanced settings which are manually entered. This is rarely needed.