We're proud to provide Agile Coaching services.

Earlier in the year, Last Call Media worked with the Massachusetts Department of Unemployment Assistance on a project in which we implemented automated testing and other automation processes. Word of the successful outcomes traveled through departments. When it came time to enhance the DFML’s program with DevOps automation, they contacted LCM.

What the DFML had was a team of teams each optimized to their own workflows and working on individual pieces of one greater product. The component-based team model increases efficiency as the larger technical foundations of a product are built, yet the integrations between those components can become a blind spot needing special consideration: integration testing was a known need in the project strategy. Last Call Media was brought on to be the integration testing team, and we knew from experience that concentrating testing of all functionality to a separate group, and as a final “phase” all work must pass through, leads to surprise issues arising too late in the life of the project. This was important as the timeline was one of the most important factors of this project: constituents needed to be able to apply for PFML benefits on January 1, 2021, no matter what.

As we began to work with the existing teams, we saw exactly what we could bring to the table: a strong strategy, clear approach, and defined process for integrating all work across every team, and systematically testing that work, as early in the development process as possible, so that fully tested product releases could be done with confidence and ease.

There were four main aspects of this project that needed to be considered in order to achieve success:

• The claimant portal, built using React, where constituents would be able to submit PFML claims and receive updates about the status of those claims,
• The claim call center, where customer service representatives would take calls from claimants and enter their claim information into the claimant portal,
• The claims processing system, the tool in which customer service representatives can process PFML claims via a task queue (and which is fed information from the portal, call center, and other third-party tools), and
• The API that would bring all of these parts together to work seamlessly.

Then, of course, there’s the testing. LCM began our work by establishing three types of tests that all project work would need to pass in order to be considered complete:

• End-to-End (E2E) testing: automated continuous verification of the functionality and integration of all 4 systems.
• Load and Stress testing: verifying the E2E functionality and integration under substantial strain to see what the system can sustain, where it breaks, what breaks it, etc.
• Business Simulation testing: verifying if the people behind the scenes who will be doing this work on a daily basis can effectively perform said work with the systems and functionality that have been put into place, and whether this work can be performed when there is a substantial amount of it.

As we worked to set up the proper tests for the product, we found many opportunities to gain alignment across all of the development teams with our overall testing philosophy: it should be a part of each team’s workflow instead of a final phase removed from the team(s) performing the work. We helped coach each team on delivering value incrementally, and their eventual ownership of where the E2E testing suite impacts their work. LCM brought testing to the program and enabled the teams to absorb it as their own.

Earlier in the year, Last Call Media worked with the Massachusetts Department of Unemployment Assistance on a project in which we implemented automated testing and other automation processes. Word of the successful outcomes traveled through departments. When it came time to enhance the DFML’s program with DevOps automation, they contacted LCM.

What the DFML had was a team of teams each optimized to their own workflows and working on individual pieces of one greater product. The component-based team model increases efficiency as the larger technical foundations of a product are built, yet the integrations between those components can become a blind spot needing special consideration: integration testing was a known need in the project strategy. Last Call Media was brought on to be the integration testing team, and we knew from experience that concentrating testing of all functionality to a separate group, and as a final “phase” all work must pass through, leads to surprise issues arising too late in the life of the project. This was important as the timeline was one of the most important factors of this project: constituents needed to be able to apply for PFML benefits on January 1, 2021, no matter what.

As we began to work with the existing teams, we saw exactly what we could bring to the table: a strong strategy, clear approach, and defined process for integrating all work across every team, and systematically testing that work, as early in the development process as possible, so that fully tested product releases could be done with confidence and ease.

There were four main aspects of this project that needed to be considered in order to achieve success:

• The claimant portal, built using React, where constituents would be able to submit PFML claims and receive updates about the status of those claims,
• The claim call center, where customer service representatives would take calls from claimants and enter their claim information into the claimant portal,
• The claims processing system, the tool in which customer service representatives can process PFML claims via a task queue (and which is fed information from the portal, call center, and other third-party tools), and
• The API that would bring all of these parts together to work seamlessly.

Then, of course, there’s the testing. LCM began our work by establishing three types of tests that all project work would need to pass in order to be considered complete:

• End-to-End (E2E) testing: automated continuous verification of the functionality and integration of all 4 systems.
• Load and Stress testing: verifying the E2E functionality and integration under substantial strain to see what the system can sustain, where it breaks, what breaks it, etc.
• Business Simulation testing: verifying if the people behind the scenes who will be doing this work on a daily basis can effectively perform said work with the systems and functionality that have been put into place, and whether this work can be performed when there is a substantial amount of it.

As we worked to set up the proper tests for the product, we found many opportunities to gain alignment across all of the development teams with our overall testing philosophy: it should be a part of each team’s workflow instead of a final phase removed from the team(s) performing the work. We helped coach each team on delivering value incrementally, and their eventual ownership of where the E2E testing suite impacts their work. LCM brought testing to the program and enabled the teams to absorb it as their own.

Over the years, RA has cultivated a repository of structured content to support their mission. While the content is primarily displayed as long form text, there is a wide variety of metadata and assets associated with each piece of content. One of the primary goals of the new site was to enable discovery of new content on the site through automatic selection of related content driven by the metadata of the content the user was viewing. Additionally, RA had a future requirement for advanced permissioning and publishing workflows to enable stakeholders outside of the web team to play a role in the content lifecycle.

Drupal 8 was selected for this project based on several factors. First, its focus on structured data fit well with Rainforest Alliance’s need for portable and searchable content. Second, the deep integrations with Apache Solr allowed for a nuanced content relation engine. Solr was also used to power the various search interfaces. Third, Drupal has historically had powerful workflow tools for managing content. While these tools weren’t quite ready for Drupal 8 when we built it, we knew they would be simple to integrate when they were ready. In short, Drupal was a perfect fit for the immediate needs, and Drupal 8 met the organization’s longer term goals.
 

The project had humble beginnings. WPI had a lot of ideas, which they’d translated into a giant content model spreadsheet. We meticulously ensured that we understood it all, guiding a wireframe process in which we were able to make significant contribution to the decision-making that yielded a highly intuitive structure. We took each step iteratively, building piece-by-piece, soliciting feedback, and then building the next iteration. We facilitated a dialogue between administrators, college communications, their design department, their IT department, and Acquia to map the landscape of business requirements, organizational needs, and their desired design aesthetic.  

On the technical side, we reviewed their existing RedDot site, and built a ‘scraper’ to scrape select content out of it and import the data into its new structure in Drupal. We open-sourced the scraper, too. Next, we built a flexible layout system of building blocks that enabled them to build pages however they needed. We dubbed them “widgets,” and WPI called them “elements,” but they were central to satisfying their desire for incredible flexibility that allowed them to build unique layouts on any given page.

As part of the creative partnership, Last Call Media’s creative team worked closely with WPI to improve the look and utility of the Events Pages, the Calendar, and a database of accolades they maintain. The outcome was a set of tools that present information in simple and digestible ways. WPI’s communications team helped LCM fully understand their brand, which enabled us to showcase their identity as we applied it to the work product.

Every step of the way as we constructed functionality, WPI developers, content authors, and staff gave feedback on the project as fidelity increased from a rough, unthemed Drupal site, to a polished release candidate. At the same time, Last Call Media was enabling WPI Developers to be self-sufficient in Drupal so they could help build new functionality. Last Call Media also built a custom ‘faculty importer’ that synced data from their Ellucian Banner system into Drupal, greatly simplifying the construction and accuracy of faculty pages.

Because we were in constant contact, there were never any ‘big reveals’ for WPI leadership to react to that represented hours of work they had to wrestle with approving or sending us back to the drawing board - they’d been part of the process at two week intervals and on board with each decision as it was made.

We used our expertise in Agile methodologies to bring order to complexity and the product to launch. We began by building and prioritizing a backlog of tasks, and getting high-priority tasks into a ready state by defining appropriate definitions of done. We focused on implementing strategies for limiting the work in progress present in the project when we started. We forecasted a few sprint goals, informed by our backlog development and resulting strategies, and quickly went into heavy development.

Immediate visibility for stakeholders into development progress became of major importance for their own ability to make quick business decisions concerning their investment. In order to maximize the value of the work, we adapted our initial Scrum iteration approach into shorter, one-week sprints of continuous delivery to get small groups of completed tasks out faster. This soon evolved into more of an ideal Kanban flow, allowing a continuous awareness and continuous delivery of the right value.

Haverford didn’t need a redesign; their site looked great already. The biggest issue they needed to solve was not having enough time to do new feature development themselves. They were a smart and capable team and over many years had developed and maintained a large custom PHP implementation for college’s website needs. They were finding, however, that too much of their time was being spent updating pages for college members and groups. Their hope was for us to build them a site that looked and functioned like their current site, but built on a modern CMS with exceptional user management and publishing workflows.

Once we had alignment on their needs and a strategy for solutions, we built out a Content Model, Product Backlog, and Information Architecture. The project was completed with Drupal in steady collaboration with the Haverford Communications team, in 5 development Sprints, and launched on the Acquia Infrastructure.

The Haverford Team really jumped in and took off with it. It was really great!

You can read more about this project as LCM’s Acquia Certified Grand Master Developer, Jeff Landfried shares details of his experience here.

SUNY Potsdam partnered with Last Call Media and Acquia to migrate their site to Drupal 7. Leveraging the scrum methodology, LCM broke down and organized the major site features into a prioritized backlog, groomed for two-week iterations. While planning and backlog refinement was ongoing several times a week, the core development team at LCM met with Potsdam every other Friday to review work completed, provide training on the new CMS, and to facilitate the feedback-gathering process.

Since SUNY Potsdam had recently gone through a redesign, the project required the new site to maintain the existing look and feel. We began with an in-depth audit of all the different page types and page elements. Instead of doing this manually page by page, we first looked for a programmatic solution. Since CommonSpot did not provide a way to generate this information within the CMS, LCM used its HTML Crawler tool to programmatically crawl the existing production site and analyze the various HTML tags to determine page elements (such as slideshows, feeds, etc), including where and how often they appear. This provided tremendous visibility into the site’s underlying structure, which was critical in planning our approach to the migration to Drupal.

After reviewing this data with SUNY Potsdam, we began the process of consolidation– instead of building one-off page elements, we consolidated similar elements into single widgets that behaved differently based on where on the page it was placed. This helped reduce the vast number of options a content author has to choose from, making it easier for them to do what they need to do: focus on the content. To achieve the desired platform flexibility, LCM built a repository of flexible and adaptable widgets to allow the marketing team at Potsdam to build custom pages. 

The migration included several different page templates and tens of thousands of pieces of content, which required writing and testing a series of migration scripts to get all the content from one CMS to another without downtime or a lengthy content freeze. Since the CommonSpot installation did not have a concept of structured content, LCM used it’s HTML crawler tool again to programmatically identify page content and then map it to its new location in Drupal. Once the custom scripts were written and tested, the migration took only 15 minutes for tens of thousands of pieces of unique content and the associated metadata, such as date published, authoring information, and URL. 

The new site also pulls in events automatically from their event management system, SOGo, and tags the event to the relevant department or office in the CMS so that it appears on that organization’s page.  

Lastly, in order to make it as easy as possible for content authors to login to the site, we leveraged the identity management service at Potsdam, Active Directory, to allow users to use their domain credentials to authenticate with Drupal.

We recommended a rapid, iterative, agile approach to migrate each static HTML site into database-driven websites using the University’s identity management system, Shibboleth. To achieve the highest value within the budget, each department, working with a Last Call producer, could decide how to prioritize tasks for migration, training, and new feature development.

To accomplish the overall project, LCM divided the total grant and project scope into a series of one-week sprints, two sprints per department. We met with each department to develop and prioritize the initial backlog of desired tasks for each department site. Each sprint started with a sprint planning meeting, which included department web teams as collaborators. Some departments were able to prioritize training, enabling them to do more manual migration of content, which in turn freed up sprint time for LCM to do more new feature development.

On a daily basis, dozens of developers work simultaneously on Finder.com, adding new functionality, fixing bugs, and creating new ways to provide value to users. As the team scales, it is faced with several challenges around managing the deployment pipeline. 

Until recently, developers were often limited and blocked by a complicated build process, where any code change would take about 12-16min to go live. Making that change often required modifying multiple repositories since all the themes and plugins were split and could not be fully decoupled. Deploying to staging environments was a manual process, and code reviews required lengthy instructions and thus were error-prone. The developer that authored a change, needed to remember to merge all repositories for a successful deployment or risk bringing down the site. 

Speeding up the deployment process 

An ineffective deployment pipeline can easily add up to hundreds of hours of time a month waiting for a build to complete that could have been spent on delivering other functionality instead. The development workflow should be smooth with stumbling blocks removed. Since Finder’s engineering team is growing rapidly, this problem needed to be addressed first. 

To speed up time to market, we overhauled the build process at Finder. We implemented Buildkite to enable continuous integration in a more efficient way. As a result, build time decreased by more than 50%, to only 8min. 

This increased efficiency sped up the process for Finder’s developers to get their work onto staging environments and out to production, and in the hands of customers.

Offering stability to the development team

Another challenge for Finder was the stability and time it took to deploy a hotfix to production. If multiple developers were trying to deploy to the product at the same time, they would often frustratingly block each other, further increasing everyone’s time to production. 

To release this functionality, we identified and rearranged certain key jobs so they could be run in parallel. We also identified build steps that could be made more efficient. For example, there were many jobs downloading the same libraries, and when the Docker image was built in the end, these libraries were again downloaded, synchronously. By limiting this, speed was improved significantly.

In the end, speeding up the build process, meant increased stability, and a decreased time to deploy a hotfix. 

Building comparison tables

In addition to platform work, Finder brought Last Call Media on board to help improve their comparison tables. Finder’s business model requires them to accurately connect users with relevant products. If a user with a “poor” credit score tries to apply to a credit card that only accepts “very good” scores, they’ll waste their time, and Finder will not be succeeding in its mission to connect users to relevant products. The tables were written in legacy JavaScript, HTML, and CSS and we worked to rebuild the tables in modern React, adding modals on top of the tables to surface the most important data filters to the user first.

Now, after users make an initial selection, they can see the most relevant results for their case. We added advanced interactive features such as calculators to the top of tables so users can input data relevant to their personal circumstances and see automatically calculated savings for each product.