Google BigQuery

The Google BigQuery Connector allows you to import data from BigQuery in two ways: 

Prerequisites

To configure access to the Google BigQuery Connector, you need:

  • Google Service Account credential or a Google User credential

Scheduling

The Google BigQuery Connector can be scheduled to ingest data once per day at a user-defined hour and timezone.

  • To configure this schedule, use the delay_hours parameter.
  • By default, the connector will run once at 6am PT.

Import Types

There are two types of Imports:

  • bigquery:query - Runs the table imports based on a standard SQL query and returns the query results if the query completes within a specified timeout. 
  • bigquery:extract - Extracts full table data from BigQuery.

Google Bigquery Query Import

Switchboard Script Syntax

import bigquery_query_import from {
   project: "my_project";
   type: "bigquery:query";
   query: "SELECT * FROM mytable";
   key: "google_key";
   }
   using {
        *
   };

Parameters

Parameter

Description

Required/Optional?

project

string

Required

query

string

Required.

location

A list of requested BIGQUERY_LOCATIONS:

  • US
  • EU

Optional.

Google Bigquery Extract Import

Switchboard Script Syntax

import bigquery_extract_import from {
   project: "my_project";
   dataset: "my_dataset";
   type: "bigquery:extract";
   table: "my_table";
   stage: "gs://my_staging_bucket_google_cloud_storage";
   key: "google_key";
   } using {
           date :date("YYYY-MM-dd");
           title : string;
   };

Parameters

Parameter

Description

Required/Optional?

dataset

string

Required

project

string

Required

table

string

Required

stage

string

Required.

dated

boolean

Optional.

location

A list of requested BIGQUERY_LOCATIONS:

  • US
  • EU

Optional.

lookback_days

integer

Optional.