I cannot find any documentation about how to create an external table in BigQuery using Python. I would like to create the table without using autodetect but passing the schema. Does anybody know how to do it? Thank you!
-
commandStr = "bq mk --external_table_definition=file.def dataset.tableName" os.system(commandStr)doctorData– doctorData2018-07-31 12:55:00 +00:00Commented Jul 31, 2018 at 12:55
Add a comment
|
1 Answer
If you don't want to use the command line tool and os.system(command), you can create a BigQuery table out of an external source using the Python BigQuery library with some code like this:
from google.cloud import bigquery
client = bigquery.Client()
#Define your schema
schemafield_col1 = bigquery.schema.SchemaField("string_col","STRING")
schemafield_col2 = bigquery.schema.SchemaField("int_col","INTEGER")
dataset_ref = client.dataset('<your-dataset>')
table_ref = bigquery.TableReference(dataset_ref, '<your-table-name>')
table = bigquery.Table(table_ref, [schemafield_col1,schemafield_col2])
external_config = bigquery.ExternalConfig('CSV')
source_uris = ['<url-to-your-external-source>'] #i.e for a csv file in a Cloud Storage bucket
#it would be something like "gs://<your-bucket>/<your-csv-file>"
external_config.source_uris = source_uris
table.external_data_configuration = external_config
client.create_table(table)
Here is the link to the API reference.
Here more info about the ExternalConfig class and its attributes.
