The best way to use App Engine pull duties (Module 18)

Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud

The Serverless Migration Station mini-series helps App Engine builders modernize their apps to the newest language runtimes, comparable to from Python 2 to three or Java 8 to 17, or to sister serverless platforms Cloud Capabilities and Cloud Run. One other purpose of this sequence is to show the way to transfer away from App Engine’s authentic APIs (now known as legacy bundled providers) to Cloud standalone substitute providers. As soon as not depending on these proprietary providers, apps grow to be far more moveable, making them versatile sufficient to:

App Engine’s Job Queue service gives infrastructure for executing duties exterior of the usual request-response workflow. Duties could include workloads exceeding request timeouts or periodic tangential work. The Job Queue service gives two completely different queue sorts, push and pull, for builders to carry out auxiliary work.

Push queues are lined in Migration Modules 7-9, demonstrating the way to add use of push duties to an present baseline app adopted by steps emigrate that performance to Cloud Duties, the standalone successor to the Job Queues push service. We flip to pull queues in at present’s video the place Module 18 demonstrates the way to add use of pull duties to the identical baseline pattern app. Module 19 follows, exhibiting the way to migrate that utilization to Cloud Pub/Sub.

Including use of pull queues

Along with registering web page visits, the pattern app must be modified to trace guests. Visits are comprised of a timestamp and customer info such because the IP handle and person agent. We’ll modify the app to make use of the IP handle and monitor what number of visits come from every handle seen. The house web page is modified to indicate the highest guests along with the newest visits:

The pattern app’s up to date house web page monitoring visits and guests

When visits are registered, pull duties are created to trace the guests. The pull duties sit patiently within the queue till they’re processed in mixture periodically. Till that occurs, the highest guests desk stays static. These duties may be processed in quite a few methods: periodically by a cron or Cloud Scheduler job, a separate App Engine backend service, explicitly by a person (by way of browser or command-line HTTP request), event-triggered Cloud Perform, and so forth. Within the tutorial, we problem a curl request to the app’s endpoint to course of the enqueued duties. When all duties have accomplished, the desk then displays any modifications to the present high guests and their go to counts:

Processed pull duties replace the highest guests desk

Under is a few pseudocode representing the core a part of the app that was altered so as to add Job Queue pull process utilization, specifically a brand new knowledge mannequin class, VisitorCount, to trace customer counts, enqueuing a (pull) process to replace customer counts when registering particular person visits in store_visit(), and most significantly, a brand new perform fetch_counts(), accessible by way of /log, to course of enqueued duties and replace total customer counts. The bolded traces characterize the brand new or altered code.

Including App Engine Job Queue pull process utilization to pattern app

Wrap-up

This “migration” is comprised of including Job Queue pull process utilization to assist monitoring customer counts to the Module 1 baseline app and arrives on the end line with the Module 18 app. To get hands-on expertise doing it your self, do the codelab by hand and observe together with the video. You then’ll be able to improve to Cloud Pub/Sub do you have to select to take action.

In Fall 2021, the App Engine workforce prolonged assist of lots of the bundled providers to 2nd technology runtimes (which have a 1st technology runtime), which means you’re not required emigrate pull duties to Pub/Sub when porting your app to Python 3. You may proceed utilizing Job Queue in your Python 3 app as long as you retrofit the code to entry bundled providers from next-generation runtimes.

Should you do wish to transfer to Pub/Sub, see Module 19, together with its codelab. All Serverless Migration Station content material (codelabs, movies, and supply code) can be found at its open supply repo. Whereas we’re initially specializing in Python customers, the Cloud workforce is overlaying different runtimes quickly, so keep tuned. Additionally try different movies within the broader Serverless Expeditions sequence.