How Switchboard Handles and Reports Errors

Switchboard automatically monitors data under management for a variety of ingestion, processing, and upload exceptions. In addition, Switchboard’s Script Editor provides live feedback about script logic errors, datatype issues and typos. It also prevents testing and publishing of any scripts with known errors.

This guide provides details on the types of exceptions that Switchboard automatically prevents and notifies.

Exceptions in Data Downloads

Credential Errors

Switchboard monitors sources for valid credentials to facilitate expected data downloads. Credentials might change (in the case of a user password needing an update), or expire due to a mandated data vendor timeout. In very rare cases, a credential might be temporarily invalid due to a server or network issue on the vendor end.

When an Credential is suspended:

  • An suspension notification will be entered into the Events log, with the cause of suspension, if available.
  • The suspended Credential will be displayed in the “Suspended Assets” Panel on the account home page.
  • If Daily Email Summaries are configured, a notice about the suspension will be included in the next scheduled email report.
  • If Slack alerts are configured, an immediate message about the suspension will be sent.

Schema, Encoding, and Datatype Changes

Switchboard inspects the incoming schema of all data under management. Occasionally, a data source vendor will unexpectedly provide a report in an unexpected schema. Examples of this type of issue occur when an API unexpectedly changes, or a file-based report from a data vendor is incorrectly formatted. In other cases, a vendor might inadvertently change the encoding of a file-based report.

In all of these cases, Switchboard will suspend the asset, and Switchboard will mark the ingestion event with the Error logs with a Processing failed: all the rows were dirty; imports suspended message.

Partial “Dirty Rows”

Some data sources will provide a report that contains a mixture of expected columns and datatypes, as well as a number of rows. In this case, Switchboard will continue to process the correct rows, and block the broken rows from being processed. However, Switchboard temporarily keeps a sample of dirty rows, including the reason for the dirty row designation. Switchboard also displays the percentage of Dirty Rows it encounters for any given source.

To download a Dirty Rows report for a given download, visit the “Downloads” tab from the left sidebar and filter for “Dirty Rows.” Expand the panel under the import, and click the download samples icon next to the Dirty Rows percentage display.

When an Download source is suspended:

  • An suspension notification will be entered into the Events log, with the cause of suspension, if available.
  • The suspended Download source will be displayed in the “Suspended Assets” Panel on the account home page.
  • If Daily Email Summaries are configured, a notice about the suspension will be included in the next scheduled email report.
  • If Slack alerts are configured, an immediate message about the suspension will be sent.

Logic Errors in Switchboard Scripts

Switchboard’s Script editor will flag, in real-time, any errors encountered as part of the script authoring process. Scripts with errors can be saved, but not tested or published.

When an Scripting Error is detected, a red alert mark will appear next to the line that contains the error. These errors include:

  • Downstream transformations referencing non-existent columns.
  • Incorrect combinations of datatypes (i.e., adding a string and an integer).
  • Circular references between chained transformations.
  • Incorrectly defined data JOINs.
  • Incorrect dimension or metrics definitions in import statements.
  • Missing or incorrect parameters in Download, Transformation, or Upload definitions.

Exceptions in Processing

Despite the Switchboard Script error check during authoring, processing or transformation errors are not common.

In the unlikely event that an temporary issue in processing happens , Switchboard will note the error but not suspend the connected Download.

Examples of potential problems that may trigger unexpected processing errors include:

  • A divide by zero error in a transformation.
  • A transformation result that produces NULL or invalid values.
  • A temporary problem with Switchboard’s processing engine.

When an Processing exception is encountered:

  • An processing error will be entered into the Events log, with the cause of suspension, if available.
  • If Daily Email Summaries are configured, a notice about the error will be included in the next scheduled email report.

Errors in Data Uploads

It’s possible that a data sink (Data Warehouse, cloud bucket, SFTP server, Google Sheet) cannot be reached, or credential updated

Examples of errors that may trigger an upload error include:

  • Lack of access to a data sink due to credential lapse or error
  • Change in the availability of a data sink (i.e.,database server is down, an S3 bucket has been deleted, etc).
  • Network error preventing access to customer cloud resources.

In all of these cases, Switchboard will retry upload data several times over a period of time before considering the destination as “suspended.”

When an Upload target is suspended:

  • An suspension notification will be entered into the Events log, with the cause of suspension, if available.
  • The Upload target will be displayed in the “Suspended Assets” Panel on the account home page.
  • If Daily Email Summaries are configured, a notice about the suspension will be included in the next scheduled email report.
  • If Slack alerts are configured, an immediate message about the suspension will be sent.

Logging and Notification of Exceptions

Switchboard Events Log

All Exceptions and “Dirty Row” warnings can be viewed in the Switchboard Events log. To access this log, visit the “Events” tab, and adjust the filter to display either “Errors” or “Errors and Warnings.”

Suspended Assets Panel

When a specific credential, data import definition, or data upload is no longer functional after all automated retry mechanisms are exhausted, Switchboard will mark the asset as “Suspended.” To view the current list of suspended assets in an account, visit the “Suspended Assets” panel on the account Home page.

Daily Email Reports

Switchboard can be configured to send a scheduled, daily email report describing any errors and suspended assets to the email recipients of your choice. To setup a daily email alert, visit the “Daily Summary” menu from the left sidebar.

Slack Alerts

When the Switchboard Slack integrations are activated, all Exceptions described on this page will be sent immediately as individual messages to the Slack channel of your choice. To activate a Slack integration, visit the “Integrations” option on the left side bar.