Some quirks of Cloud Functions.
The python script was a relatively simple ETL job that tested my abilities to architect and debug, but it was a great experience in deploying production code under constraints. The script simply compared two Google Sheets, and updated the target page with new or updated landing values. The script fires from a Cloud Scheduler message to PubSub, which triggers the python function to execute.
Here are my tips!
Look at how many Pandas DataFrames you are creating and eliminate unnecessary objects.
If your data set is large, this can be extremely wasteful. This is where a data scientist needs to put on a computer scientist hat and pipe transformations or find optimal ways to create and dispose of objects in memory.
One simple way to check the sizes of DataFrames in memory is this function:
