Kiteworks

The Switchboard Kiteworks connector provides automated, scheduled ingestions of data from Kiteworks Secure Share.

• Prerequisites
• Scheduling
• Parameters
• File Formats and Encoding
  • File Formats
  • Encodings
  • Compression
• Sample Switchboard Script

Prerequisites

To configure access to the Kiteworks connector, you need to create a Kiteworks key with the following info:

• Client Id (Displayed on Admin interface when app was created)
• Secret (Displayed on Admin interface when app was created)
• User Name
• Password

Scheduling

Kiteworks connector can be scheduled to run multiple times a day at user-defined hour and timezone.

• To configure this schedule, use the lookback_days parameter.
• By default, the connector will run once at 6am PT.

See Periodic Scheduling for more details

Parameters

datetime_pattern string

optional

Specifies the date and time pattern type.

Example: datetime_pattern: “YYYY-MM-DD”;

For additional information, see the Datetime Patterns section.

delimiter string

optional

Example: format: “tab”;

For additional CSV specific parameters, see the File Formats and Encoding section.

folder_id string

required

Identifier for the location in kiteworks to retrieve files

log_filelist_to_events Boolean

optional

The files accessible via this connector are only accessible via APIs, making it difficult to determine their availability without executing Postman or cURL requests.

To assist end users, we have added this optional parameter, which can be utilized during proof testing.

This will append the a warning message to an event entry which you can finalize the file pattern and lookback days and possibly day of week in your SBS. For example:

Files Available: ALL_WKLY_LOC_ATTR_V0_0_04202024_KW.zip, BV_4191_DAILY_INV_TCIN_LOC_04252024_KW.zip, BV_4191_DLY_PO_PLAN_TCIN_04272024_KW.zip, BV_4191_WEEKLY_GM_TCIN_LOC_04202024_KW.zip, BV_4191_WEEKLY_ITEM_MTA_04202024_KW.zip,BV_4191_WKLY_TCIN_ITEM_04202024_KW.zip.

lookback_days integer

optional

Limits the number of previous days that the DateTime pattern applies

format string

required

Specifies a format type.

Example: format: “csv”;

For additional CSV specific parameters, see the File Formats and Encoding section.

header_row Boolean

optional

indicates that the first row in the file is a header and not data, defaults to false

pattern string

required

A list of requested file patterns.

Example: pattern: “ALL_WKLY_LOC_ATTR_V0_0_MMDDYYYY_KW”

Specifying Patterns

File Patterns

Switchboard matches target S3 files based on wildcard patterns or regular expression. Switchboard polls the source bucket for new files that match the pattern or regular expression provided. By default, Switchboard re-ingest files upon detection of source file checksum.

• To specify a file match by pattern, use the pattern parameter. The * character is used as a wildcard pattern match:

  pattern: "ALL_WKLY_LOC_ATTR_V0_0_MMDDYYYY_KW*";
  

• To specify a file match using a regular expression, use the regex parameter containing a valid matching pattern:

  regex: "ALL_WKLY_LOC_ATTR_V(a|b)_\d{6}.csv";
  

Datetime patterns

Configure Switchboard to poll for file names that match a date pattern. It allows importing the date-range backfill in the Switchboard UI. 

• Add a datetime_pattern to the import configuration. Since target objects may have multiple dates in the filename, it is important to specify a pattern that matches the specific date string required.
  • To match the first date string in a filename of my_file_name_2020-01-01_2020_01-08.csv, use the following pattern:

    datetime_pattern: "*my_file_name_YYYY-MM-DD_*";
    

• To limit the number of previous days that the DateTime pattern applies, use the lookback_days parameter. If no files are found for the lookback period, Switchboard will consider this an error. 
  • To locate files with a DateTime pattern that matches files within the past 5 days:

    lookback_days: 5;
    

File Formats and Encoding

Switchboard imports files with a variety of formats, encodings, and compression schemes.

File Formats

To specify a format type, use the format parameter:

format: "csv";

The available options are:

CSV specific options

Sample Switchboard Script

download kiteworks_wkly_attr from {
            type: "kiteworks:secure_share";
            key: "kiteworks_key";
            pattern: "ALL_WKLY_LOC_ATTR_V0_0_MMDDYYYY_KW*";

            folder_id: "ba3b5e5f-9f8a-402e-ba9b-4305454dbf18";
            lookback_days: 7;
            header_row: true;
            format: "csv";
            delimiter: "tab";
            //log_filelist_to_events: true;   // REMOVE AFTER TESTING   <----
            datetime_pattern: "*_MMDDYYYY*.zip";
        } using { filename: string;
                  Location_Type: string;
                  Location_Subtype: string; 
                  Location_Number: string;
                  Location_Name: string;
                  Address_1: string; 
                  Address_2: string;
                  City: string;
                  State: string;
                  Zip_Code: string;
                  Servicing_RDC: string;
                  Servicing_FDC: string;
                  Region: string;
                  District: string;
                  Group_ID: string;
                  Store_Format: string;
                  Last_Remodel_Date: string;
                  Next_Remodel_Date: string;
                  Store_Size: string;
                  Latitude: string;
                  Longitude: string;
                  Store_Open_Date: string;
                  Store_Close_Date: string;
                  Store_Status: string;
                  Optical_in_Store: string;
                  Pharmacy_in_Store: string; };