Scheduling Data Ingestions

Switchboard supports two types of scheduling methods for data ingestion: Daily and Periodic. Visit the individual descriptions of each connector to learn which type of schedule the connector supports.

If no scheduling configuration is defined, data connectors will import data (or poll for new files, in the case of file-based connectors) once per day at 6am PT.

• Daily Scheduling
• Periodic Scheduling
  • Hourly Scheduling
  • Weekly Scheduling
  • Monthly Scheduling

Daily Scheduling

Most Switchboard connectors run once a day, and default to downloading reports for “yesterday”.

To define the time of a daily data pull, use the delay_hours parameter. This parameter defines how many hours past midnight Switchboard will wait to begin a data ingestion. To define the timezone, see Specifying the Timezone of a scheduled ingestion.

For example, to configure a report to be retrieved at 3am PT, use delay_hours: 3 as in the example below:

import google_adsense from {
    type: "adsense_ng";
    key: "google_adsense_key";
    account_id: "pub-123";
    dimensions: ["DATE"];
    metrics: ["EARNINGS"];
 
    delay_hours: 3;

} using {
    date: datetime;
    earnings: float;
};

Periodic Scheduling

Connectors that poll for files, or connect to database and data warehouse sources, can be scheduled to run on a periodic basis. The most frequent ingestion period available is once per hour.

Hourly Scheduling

Hourly scheduling at regular intervals

To setup a recurring download to run on a regular interval, use the period_hours parameter. This parameter takes a single integer value.

// Schedule a Google Ad Manager Ad Unit import to run every 12 hours.

import google_ad_manager_ad_unit from {
    key: "google_credential";
    network: "1234";
    type: "gam:ad_unit";

    // Sync all Ad Units on the 1st and 15th of every month.
    period_hours: 12;
}...

Hourly scheduling at specific times of the day

To configure a data ingestion to run one or more times per day, at a specific hour use the day_of_month parameter. This parameter accepts a list of integer values corresponding to the day of the month.

To define the timezone, see Specifying the Timezone of a scheduled ingestion below.

Weekly Scheduling

To configure a data ingestion one or more times per week, use the day_of_week parameter. Allowed values are: Mon, Tue, Wed, Thu, Fri, Sat, Sun

// Schedule a Google Ad Manager Ad Unit import to run once every Monday, Wednesday, and Friday

import google_ad_manager_ad_unit from {
    key: "google_credential";
    network: "1234";
    type: "gam:ad_unit";

    // Sync all Ad Units on the 1st and 15th of every month.
    day_of_week: ["Mon", "Wed", "Fri"];
}...

To define the timezone, see Specifying the Timezone of a scheduled ingestion below.

Monthly Scheduling

To configure a data ingestion to run one or more times per month, use the day_of_month parameter. This parameter accepts a list of integer values corresponding to the day of the month.

// Schedule a Google Ad Manager Ad Unit import to run once on the 1st and 15th of every month.


import google_ad_manager_ad_unit from {
    key: "google_credential";
    network: "1234";
    type: "gam:ad_unit";

    // Sync all Ad Units on the 1st and 15th of every month.
    day_of_month: [1,15];
}...

To define the timezone, see Specifying the Timezone of a scheduled ingestion below.

Specifying the Timezone of a scheduled ingestion

It’s possible to specifiy the schedule timezone in your imports by using the timezone parameter. This parameter takes any standard TZ timezone Database label.

For example, so specify that a scheduled import starts at 3am in the US Eastern timezone, use timezone: 'America/New_York'; along with delay_hours: 3;. See the example below:

import google_adsense from {
    type: "adsense_ng";
    key: "google_adsense_key";
    account_id: "pub-123";
    dimensions: ["DATE"];
    metrics: ["EARNINGS"];

    timezone: 'America/New_York';
    delay_hours: 3;

} using {
    date: datetime;
    earnings: float;
};