How to run gcloud, gsutil and bq using Google Cloud SDK docker image

Pre-requisites
- (Optional) Visual Studio Code https://code.visualstudio.com/download - You can use Powershell/Command Prompt (Windows) or Bash (MAC OS) but I prefer to use this as it works well for both Windows and MAC OS.
- Docker https://www.docker.com/products/docker-desktop
Steps
Source
https://cloud.google.com/sdk/docs/downloads-docker
Assume you have installed Docker and Visual Studio Code Open Visual Studio Code → Click on Terminal → New Terminal

Pull the docker image. It will take a while since the file is quite large (~1GB)
Updated: 21 July 2023- Due to security restrictions imposed by Google, we will download the latest cloud-sdk instead of using version 357.
# Pull the docker image
docker pull gcr.io/google.com/cloudsdktool/cloud-sdk
# To check the docker image exists
docker image ls

Verify the installation
# Check the gcloud version without saving its container file system
# Refer here fore more info https://docs.docker.com/engine/reference/run/#clean-up---rm
docker run --rm gcr.io/google.com/cloudsdktool/cloud-sdk gcloud version
Authenticate
# Run the docker and create a container called "gcloud-config" to store your credentials
docker run -ti --name gcloud-config gcr.io/google.com/cloudsdktool/cloud-sdk gcloud auth login
# Check that container created
docker ps -a
Run some basic gcloud command, refer here for more tips https://cloud.google.com/sdk/gcloud/reference
# Run a new docker based on volume from the container created above for credential purposes
# List projects
# Use this instead
docker run --rm --volumes-from gcloud-config gcr.io/google.com/cloudsdktool/cloud-sdk gcloud projects list
Run some basic gsutil command, refer here for more tips https://cloud.google.com/storage/docs/gsutil/commands/help
# Run a new docker based on volume from the container created above for credential purposes
# List files in a bucket
# Use this instead
docker run --rm --volumes-from gcloud-config gcr.io/google.com/cloudsdktool/cloud-sdk gsutil ls gs://chenmingyong_udemy
Run some basic bq command, refer here for more tips https://cloud.google.com/bigquery/docs/bq-command-line-tool
# List the projects
docker run --rm --volumes-from gcloud-config gcr.io/google.com/cloudsdktool/cloud-sdk gcloud projects list
# Set your project based on the project id
docker run --rm --volumes-from gcloud-config gcr.io/google.com/cloudsdktool/cloud-sdk gcloud config set project udemy-325708
# Run sample bq command
docker run --rm --volumes-from gcloud-config gcr.io/google.com/cloudsdktool/cloud-sdk bq query --nouse_legacy_sql 'SELECT COUNT(*) FROM `bigquery-public-data`.samples.shakespeare'
This is part of my online course on how to kickstart data engineering workflows in Google Cloud Platform (GCP) for beginners, sign up here to watch detailed video explanation! 🤗