BigQuery Destination Troubleshooting

This article includes common issues users encounter when setting up a BigQuery Destination Connector. For a step by step guide on setting up BigQuery destination, please follow our documentation.

Setup issues

• Not assigning proper permissions to the service account. The required permissions are: 
  • BigQuery User
  • BigQuery Data Editor
  • Storage Object Admin (If using GCS Staging)

• Forgetting to input your Service Account Key JSON

• Not inputting the entire Service Account Key JSON, make sure to include the brackets on either end

  {

    "type": "service_account",

    "project_id": "airbyte-*******",

    "private_key_id": "***************************",

    "private_key": "-----BEGIN PRIVATE KEY-----\****************************************************************************************************************************************************************************************************************************\n-----END PRIVATE KEY-----\n",

    "client_email": "*******@airbyte-*****.iam.gserviceaccount.com",

    "client_id": "*******************",

    "auth_uri": "https://accounts.google.com/o/oauth2/auth",

    "token_uri": "https://oauth2.googleapis.com/token",

    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",

    "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/******%40airbyte-*******.iam.gserviceaccount.com",

    "universe_domain": "googleapis.com"

  }

   
• Improper Bucket settings for GCS staging
  •  Grant the Storage Admin (roles/storage.admin) IAM role on the bucket
  • Set the Protection Tools to none or Object versioning
  • Make sure the bucket does not have a retention policy

Connection issues

• Pickling client objects is explicitly not supported

  • Set up GCS staging on your BigQuery instance
    • If issue persists after setting up GCS staging, drop all related tables in BigQuery and reset your connection

• Cannot read and write in different locations

  • Confirm/Edit the location of your GCS staging set up to match your DB location