Databases and operations
BigQuery automation and integration.
Google Cloud's data warehouse for running SQL across large datasets.
- Tool
- cloud.google.com
- Usually paired with
- Power BI, PostgreSQL, Google Sheets, n8n
- Timeline
- Most builds live in 3 to 5 business days from scope
- Owned by
- You, with accounts, credentials and code in your name.
What we build on BigQuery
- Load data from your CRM, ad accounts and finance tools into one set of tables
- Write scheduled SQL for the numbers your team reports each week
- Connect dashboards in Looker Studio or Power BI
- Give an AI assistant read access to approved tables
What we sync
Workflows we are asked for
- BigQuery data pipeline from HubSpot
- Shopify orders to BigQuery
- BigQuery reporting dashboard
- BigQuery weekly metrics to Slack
Often paired with
Tools BigQuery usually needs to talk to.
-
BigQuery and Power BI
Microsoft's business intelligence tool for reports and dashboards.
-
BigQuery and PostgreSQL
The database under most serious systems, including Supabase.
-
BigQuery and Google Sheets
The spreadsheet most operations start in, and the one most integrations have to respect.
-
BigQuery and n8n
The automation platform we build most systems on: code where needed, visual where not, cloud or self-hosted.
In every build
Standards we apply to every build.
Under the hood
- Upserts keyed on the record
Each write is matched on the source ID, so a retry or a re-fired event updates the existing record.
- Retries with backoff
Rate limits and timeouts are retried with increasing delays. If they still fail, they are parked for review.
- Idempotent, loop-safe writes
Two-way flows carry a marker so a change written by the sync does not re-trigger the sync.
- Reconciliation
A nightly job compares counts and totals on both sides and lists any record that differs.
- Audit log
Each run records the input, the mapping result, the action taken and what triggered it. You can search it by record.
For the team
- Slack or email notifications
Alerts go to a named person on your team, by Slack, email or WhatsApp.
- A runbook
One document per integration: what it does, what each alert means, and what to do in the first ten minutes.
- Training
A live session with the people who will use the review queue, recorded for whoever joins later.
- Handover
Admin access, billing and code sit in accounts in your name from the start, so the build keeps running if we stop working together.
- Sixty days of hypercare
The engineers who built it stay on call after go-live, fix what breaks and tune what the team asks for.
Keys and data
- Keys are kept in a vault
API keys and tokens are stored in the orchestration layer's encrypted credential store or the platform's secret manager. None are written into workflows, repos, spreadsheets or chat.
- One credential per connection
Each integration gets its own key, scoped to the objects and actions it needs. A lead sync cannot read invoices. Revoking one key stops one thing.
- Service accounts for connections
Connections use a dedicated service user in your account, so nothing breaks when a person leaves and nothing runs under someone's name.
- Two-factor on every account
Each platform account we set up has two-factor authentication on, and the recovery codes are handed to you.
- Rotation on handover and on request
Keys are rotated when the build is handed over, when anyone with access leaves, and at any time you ask.
The offer
Connect BigQuery to the rest of your stack.
We agree a fixed scope in a discovery call, and most builds go live in three to five business days. The accounts are in your name, and the engineers who built it stay on call for sixty days after.
- ScopeOne call to agree the workflows, the fields and the owner, written up the same day.
- BuildValidation, upserts, retries, reconciliation and an audit log, on n8n, Make or code.
- Go-liveWe test in a sandbox first, then switch over. It takes longer only where a vendor has to approve something.
- HandoverRunbook, training session, alerts to a named owner, admin access and code in your accounts.
- AfterSixty days of hypercare, then a care plan or run it yourself.
Questions
What people ask about BigQuery.
Do we need a data warehouse at all?
Often you do not. If your reports fit in a Postgres database or a spreadsheet we will say so. BigQuery starts to make sense when several systems have to be queried together and the volume has outgrown those.
BigQuery
Start with the one BigQuery workflow your team still runs by hand.
Bring it to a discovery call. You leave with a yes or no on fit, an outline of the build and what we would do first.
Book the call