Back

Classifying news data and storing the result in BigQuery

BigQuery

Stamps

your_project_name, classify-text.py, YOUR_PROJECT ,

Initial conditions

Categories:

Google Cloud


First, back in Cloud Shell, export the name of your Cloud project as an environment variable. Replace <your_project_name> with the GCP Project ID found in the Lab details panel of the lab.

CO

Команда

CODE...

используется для вывода текущего проекта, который установлен в конфигурации Google Cloud SDK. Результат этой команды будет содержать идентификатор (ID) текущего проекта.

...
Prog

Then run the following commands from Cloud Shell to create a service account:

 

CODE...
...
Prog

Create a file called classify-text.py and copy the following into it. Replace YOUR_PROJECT with your GCP Project ID.

CODE...

We're using the google-cloud Python client library to access Cfiles is a reference to each of the BBC dataset files in the public bucket. We iterate through these files, download the articles as strings, and send each one to the Natural Language API in our classify_text function. For all articles where the Natural Language API returns a category, the article and its category data are saved to a rows_for_bq list. When classifying each article is done, the data is inserted into BigQuery using insert_rows().

Now you're ready to start classifying articles and importing them to BigQuery.

...
Prog

Run the following script:

CODE...
Prog

If you get an error while executing python3 classify-text.py, the cloud shell might be disconnected. In order to fix that, please export your environment variables by running the below commands then python3 classify-text.py command.

  • export PROJECT= (GCP PROJECT ID)
  • export GOOGLE_APPLICATION_CREDENTIALS=key.json
...
Prog

Final conditions:

;

Organize your work