Best-in-class content delivery and caching.

Processes
  • Continuous Delivery
Team Leadership

As part of our ongoing engagement with the Commonwealth, we identified an opportunity to improve customer and constituent experience by leveraging the Cloudflare CDN (Content Delivery Network). Following the initial discovery phase, we architected, implemented, and deployed Cloudflare’s Global CDN product to give the site best-in-class content delivery and caching, while maintaining all the functionality of the previous CDN while improving development capabilities.

Creating Undetected Changes.

In the discovery phase, we reviewed the marketplace to find the most appropriate CDN for the State’s use case, balancing security, performance, and cost considerations. Ultimately, Cloudflare was selected as the best fit because of its extensive firewall and DDOS protections, and granular cache control using “Cache Tags,” which have the potential to boost performance for the constituents and reduce the risk of site instability.  

The first, and perhaps most critical concern we addressed in the course of this project was that the CDN needed to be resilient, serving pages even if the site itself was not functioning properly. For example, the development team does code releases periodically that take the backend of the site completely offline, but constituents still need to be able to access content during this time. To meet this requirement, we adjusted the site’s caching headers to include directives to serve cached responses in the event of an error response received from the origin.  As a result, constituents are able to access the majority of Mass.gov, even if a catastrophic event takes the web servers completely offline.

As a government website, Mass.gov is always at risk of attack from malicious actors. To mitigate this risk, Last Call Media undertook extensive configuration and testing of Cloudflare’s various security features, including the Web Application Firewall (WAF), DDOS protections, and custom firewall rules. We had a few hiccups along the way with configuring the security features (at one point, content authors were receiving CAPTCHA verifications when submitting their changes), but were ultimately able to work through these issues to dial in the right balance of security and ease-of-use.

Next, we implemented Cloudflare’s brand new “Workers” feature, which gives granular control over CDN functionality using a javascript “service worker.” The Worker we wrote for this project handles more than 6 million requests a day, and gives the Commonwealth the ability to test and deploy CDN level changes to development, staging, and production environments independently, making it much faster and safer to verify and release changes. The worker implementation benefits the Commonwealth giving them flexibility for the future, while also reducing cost over the previous CDN.

These workers were also integral to the success of this migration beyond what we had initially imagined. During the testing and release phases of the project, they gave us a mechanism for fixing changes that was reviewable and testable. Having a well-defined review and deployment process improved the team’s visibility into what changes were being made, and let us avoid silly mistakes. Overall, we felt the development team’s velocity was greatly improved by using this workflow.

The migration went as smoothly as possible, there were no negative results.

Mass.gov raved:

I hope you are puffed up with pride. We simply couldn’t have done any — much less ALL — of this mountain of work without you. You’ve been a rock. Well, a very hard-working and creative rock. We are so lucky to have your help.

Lisa Mirabile, Project Manager

For the future we envision phase 2 to be granular cache invalidation. This would mean when a piece of content changes, the CDN would only invalidate only that piece of content so it’s fresh in the cache. What that allows us to do is set really long cache lifetimes on the edge content. We would be able to cache pages for even a year and rely on the invalidation to make them fresh when we need to be, reducing the load on the backend servers significantly. In its current state, pages now only get cached for 30 minutes. With a longer available cache time, we’d see immediate results for less load time, it would be less costly for infrastructure, and lessen the chances for a backend disruption.