“There aren’t enough words of thanks for Colin and Last Call Media. Amazing site design, fabulous t-shirts, awesome stickers!”
Selected Work Examples
After working with Blackboard—the largest education technology and services company in the world—on restructuring and retheming their corporate site Blackboard.com, we knew we needed to test our efforts in order to iron out any problems before the site went live. Normally we may conduct remote moderated user testing for this (watching users navigate the site over a video call to gauge how well it works for them), but Blackboard’s annual user convention, BbWorld, provided a perfect opportunity to conduct this testing in person.
How we did it.
We wrote a list of goals for our user testing that included learning about user behavior, whether the site was intuitive to navigate, whether free trials were easy to find, and what users thought about the new design. We worked with the team at Blackboard to define these goals and to ensure we were all on the same page about what we would hope to learn from these interviews. We then constructed our interview questions around meeting these goals, asking users to complete tasks that would ultimately give us the necessary insights for the next phase of work.
On-site at BbWorld, we used Formstack to conduct our user interviews. This gave us an adaptable setup that enabled Blackboard staff to conduct testing right alongside us, even though many of them had never done so before. Speaking directly to users about the site gave Blackboard staff a higher level of empathy with their users than if they had just read a report, giving them deeper insight into how users viewed the site. We also made screen recordings of our interviews, for later reference.
The BbWorld attendees we interviewed consisted of institution technologists, instructors, academic leaders, corporate partners (resellers, ed tech vendors), and trainers, mostly in higher education, as opposed to K-12, government, or business. Most were influencers or decision-makers when it comes to purchasing education software for their institution—a valuable perspective to be sure! In the end, we facilitated almost 70 total interviews over just a couple of days!
There were a number of initiatives in place to encourage people to test the site. These included push notifications from the BbWorld app, a $15 Starbucks gift card that was given at the end of the testing, and an event-wide “passport” attendees could only complete by visiting our table. Most of the people we spoke to came to us for one of these reasons.
Learning about the old site
The majority of people we spoke to at BbWorld said they had used Blackboard.com before. We were particularly interested to hear from these experienced users, so we could establish a baseline experience using the old site, to later compare their experience using new site. Measuring these experiences against each other would tell us whether or not the restructure and retheme was an improvement. For example, if someone used the site for gathering specific information about Blackboard, did the new site make it easier or harder for them to do that?
Here’s what that baseline experience looked like:
- Half of the people we spoke to said the current Blackboard.com was easy to use, with a quarter each saying it was hard or neutral.
- Most common uses for the site included navigating to Blackboard’s Community site, reviewing the Higher Education section for language on products, or seeking help or support.
- Finding things or navigating the site were listed as the most common difficulty, although some thought it worked well!
- Opinions about the navigation were mostly negative.
- Testers felt that site was clean, but some thought it presented too much information.
- Most people said they access the site by typing the url in their address bar. (A sure sign of an experienced user!)
- Almost universally, people accessed the site from a desktop computer.
Learning about the new site
Moderated user testing is often the first time people outside of your team sees your product. While this can be daunting, it’s important to be open to constructive criticism big and small—after all, that’s the point of user testing! So while we were prepared (as always) for negative feedback, the overall response to the new Blackboard.com was extremely positive. Whew!
While we walked into testing thinking that, visually, the new theme was more of a small facelift than a full redesign, people were enthusiastic about the design of the new site. Before we could even ask about the design, many people immediately stated that they thought it was a big improvement, using words like “clean,” “modern,” and “professional” to describe it.
Many people also liked the new navigation structure. Comments about it included “All the overwhelming links are gone. Now I can digest what the navigation is offering. It’s highlighting the important information,” and “Navigation is simplified and to the point. More intuitive and easy to navigate, it will take a shorter time to get to what I want.”
Despite having a high level of familiarity with Blackboard, two thirds of the people we spoke to said that they learned something new about Blackboard during their brief time on the new site. Blackboard is working to let the public know that they’re more than just an LMS, so we were happy to hear the site is now aiding in that. Blackboard’s recruiting, safety, and social media management services were among the things people most commonly cited having learned about.
After performing a few tasks on the new site, half of the people we spoke to said the new site was easier to use than the old site, with a quarter each saying it was harder or the same.
But of course there was still room for improvement. Things that people thought could be improved included:
- Many people thought the site wasn’t doing a good job of explaining who Blackboard is and what they do.
- While the new navigation worked well for many people, some expressed confusion over the specifics of what it was presenting. Specifically, they were unsure what was a product and what was a service, and confused about the word “community” being used in two different places to indicate different things.
- Users would often miss the button for free trials, sometimes not seeing it at all until we pointed it out.
The thing that gives our partners the most pause when pulling the trigger on user testing is that you don’t always know what the results will be ahead of time. How can you justify dedicating time and resources to something when you don’t know what you’re going to get? But that’s the point of testing—to find out things you don’t know. We’ve never walked away from testing without valuable insights that went on to inform the rest of the project, and our Blackboard testing was no exception.
One small but impactful result was that we changed the color of the free trial CTA. We made this pivot quickly in the middle of testing, which enabled us to gauge the impact immediately. People started noticing the button more, and thus the issue was resolved immediately — no muss no fuss! It was gratifying to have something so small make an immediate positive impact and we all laughed at how long we chatted about that button color prior to testing! Post-launch, this CTA is used six times more than it was before the redesign, and the page it links to is now the third most viewed on the site, leading more users to this essential content.
Following user testing we did an extensive redesign of the homepage, and the feedback we received stating the site should tell users who Blackboard is and what they do was our guiding light. The new Blackboard homepage is a top to bottom narrative that tells the story of Blackboard starting with a broad overview, then moves into some specifics, and then shows what Blackboard does in action. Post-launch, Blackboard has seen a decrease in the bounce rate of their homepage, meaning more users are engaging with it.
What we learned in user testing also informed the new site navigation. The site structure had been massively rearranged, shifting from being market-based to value-based. The feedback we received about certain parts of the navigation being unclear helped refine the language used. Post-launch, more Blackboard users have been getting to their destination in 0-2 clicks, meaning they’re using the new navigation to find what they’re looking for faster.
The insight that user testing provides can come in the form of reassurance. Since people were enthusiastic about the new site being easy-to-use, we could move forward with the confidence that there were no big missing pieces. Furthermore, since the new theme was so well received, we realized the visual changes provided a larger impact, shifting our perception of it from being a “facelift” to something closer to a redesign.
Blackboard.com subsequently relaunched and, with the help of the insights we gained in user testing, is seeing a decreased bounce rate on the homepage, more users finding their destination quickly, and a dramatic increase in traffic to the free trial page, meaning teachers and students worldwide have better access to some of the best digital learning tools on the market.
Feedback is of the utmost importance to the Commonwealth.
The highest priority of the Commonwealth of Massachusetts is serving its constituents as best it can. Essential to that is feedback—hearing directly from constituents about what they’re looking for, how they expect to find it, and where any improvements in that journey can be made.
We partnered with the Commonwealth to design a component for Mass.gov that would gather useful feedback from constituents, and another component that would display that feedback to all 600+ of the site’s content authors in a way that maximizes their ability to make improvements.
Watch Collecting and using feedback on Mass.gov, a session about this project presented by Colin Panetta of Last Call Media and Joe Galluccio of Massachusetts Digital Services at Design 4 Drupal below, or scroll down for our written case study about it.
Getting feedback from constituents to site authors.
The success of Mass.gov hinges on getting the right feedback from constituents to site authors. Our first step in overhauling the way Mass.gov collects feedback was to define what we needed to know about each page in order to improve it, so we could design the feedback component around that. It consisted of the following:
- Whether or not users found what they were looking for, and what that was.
- Contextualize the above by knowing how satisfied users are with the page, and what they came to the site to do.
- Very detailed feedback that could only be provided through their user panel, a list of nearly 500 constituents who have volunteered to test new features for the site.
With our broad goals defined, we wanted to make sure the feedback component was working on a more granular level as well. We conducted a series of interviews with site authors asking how to best reach their users, and gained some valuable insight. Here’s what they told us:
- Too much information in the feedback form would scare users away.
- Feedback was being submitted with the expectation of a response, and organizations wanted to be able to respond.
- But, not all organizations would be able to respond, so a variety of contact options needed to be available to them.
We combined what we learned above with our best practices to make a set of requirements that we used to define a strategy. It was immediately obvious that this feedback component needed to do a lot! And like site authors told us, if we showed that to users all at once, we might scare them away.
So to maximize the amount of responses we’d get, we decided to lower the effort for submission by presenting these options one at a time, starting with the step that takes the least amount of commitment, and increasing with each step. So users can submit a little bit of feedback, and then opt into submitting a little more, and then keep going.
Designing the feedback form
With a clear strategy in place, we designed the following component.
On first load, the component is very simple — it’s only asking users if they found what they were looking for.
Once users have made a selection, the component expands with fields asking them what they were looking for.
Site authors have the option of including an alert here that tells users this form is not for urgent assistance, and directs them to a better place where they can do that.
In the above example, the organization who is responsible for this page is able to respond directly to feedback. So if users say they would like a response, a form opens up for them to enter their contact information. If the organization was not able to respond directly to feedback, a brief explanation of why would appear there instead.
After submitting, users are thanked for their feedback.
Seen above, organizations are given the option to link to their contact page. This is commonly used if the organization is unable to respond directly to feedback.
Users are then given the option to take a short survey, where they can provide more detailed feedback.
After submitting the survey, users are given the opportunity to join the Mass.gov user panel. This is the largest commitment available for providing feedback, so it’s at the very end!
So that’s how feedback is collected on the site. But what happens to it after that?
Displaying feedback to site authors
Feedback submitted through the site can be viewed per node, i.e. a site author can go to a specific page through the backend of the site and view all the feedback submitted for that page. But a lot of feedback can be submitted for a single page, and on top of that, site authors are often responsible for multiple or even many pages. Combing through all that feedback can be a prohibitively daunting task, or simply not possible.
To help with this, we designed the “Content that needs attention” panel for site authors.
The “Content that needs attention” panel appears on the welcome page on the backend of the site, making it one of the first things site authors see after logging in. It displays the page titles of their 10 pages with the lowest scores from users, sorted by page views. By showing site authors their content that’s seen by the most people first, we’re helping them prioritize what to work on next.
We’re giving site authors additional information about the content right in the component, helping them make decisions at a glance. In addition to the aforementioned page titles, scores, and page views, we’re showing them the content type (since some titles can be very similar on this site), the date they last revised it (in case that helps them know how badly this content needs attention), and something a little surprising… a “Snooze” button!
We put a snooze button in because once site authors make an improvement to content, it’s no longer helpful for them to see it here. So, the way it works is that they make an improvement to content, then hit “Snooze,” and it’ll disappear from this list for one month. At the end of that month, one of two things will have happened: 1) the content will have improved enough to no longer appear on this list, or 2) the content needs more improvement, and will appear back on this list.
This feedback component collects around 30,000 pieces of feedback in a single month. Issues reported by users include missing or hard to find content, mistakes, or issues with the service itself. That feedback is used by Mass.gov’s 600+ site authors to continuously improve the delivery of their vital services to the constituents of Massachusetts.
Telling the story of the personal virtual heart.
Last Call Media partnered with the Computational Cardiology Lab at the Institute for Computational Medicine at Johns Hopkins University to establish a unique and approachable identity and website design for their very technical work. This work was completed in only two design sprints.
How we did it
The Trayanova Lab of Computational Cardiology represented a rare intersection of art and science.
During discovery, which consisted of an extensive on-site meeting, we learned a few important facts about Computational Cardiology that served as guiding lights for the design work we did with them. First, we learned about what they do. In a nutshell, Computational Cardiology creates virtual models of hearts that can be used for diagnosis or study. (Saving everyone the messy business of taking real hearts out of people’s bodies which can be, let’s say, not particularly healthy for the subject.) Here’s an Endgadget article about one of their recent studies.
Computational Cardiology uses design as a tool to set themselves apart from their peers and get attention drawn to their important work.
Our second big takeaway from discovery is that Computational Cardiology places a high value on design, a quality they told us can be rare in the scientific community. As such, Computational Cardiology uses design as a tool to set themselves apart from their peers and get attention drawn to their important work. And beyond just good design, they wanted a distinctive, exciting look. The leadership at Computational Cardiology has a keen eye for art and fashion, and they felt it was important that this sensibility be reflected in their identity. This was music to our ears!
With all this in mind, we developed a few aesthetic directions that we could use to get creative alignment. These directions mostly represented Computational Cardiology’s identity using the bold, artistic direction they expressed to us, along with some more conservative elements to make sure a full range of choices was available for consideration. After quickly responding to some of the bolder directions, we selected the elements we thought worked especially well and went to work developing a unified direction based around them.
Our work has heart
One of those elements was a graphite illustration of Computational Cardiology’s computer models of hearts. That drawing would go through multiple iterations before taking the final form seen on the site, all of which can be seen below.
Designing a logo
Our Creative Director Nolan was able to quickly design a thematically dense logo in a very short amount of time. The heart icon, rendered with angles to reinforce the theme of technology, is surrounded by brackets, indicating that the heart is made of computer code. Those brackets also represent the two “C”s of Computational Cardiology and the negative space between them creates a cross, a symbol commonly used to indicate healthcare.
Building the site
Throughout the design process we remained aware that Computational Cardiology would be building the site themselves, and frequently checked in with their in-house developer to make sure we weren’t designing anything that would be problematic for them. After the designs were complete we provided them with the assets they’d need to build the site, which they did using the service Webflow. That the final, developed version of the site (which can be seen here) is so faithful to our designs is a testament to both their developer’s skill and the robust functionality of Webflow!
Driven by the need to have a compelling presence in place for the year-end graduate student application season, Last Call Media worked within a tight timeline to design a new logo and a distinctive and approachable visual experience that showcases the lab’s groundbreaking work with illustrations that were drawn by hand in-house. The project completed in two agile design sprints.
The New England Regional Developers Summit (NERDS) came to Last Call Media for a site design that would support their inclusive, educational agenda for their upcoming 2017 event.
Last Call employed a few different site strategies in order to best serve NERDS and their users. First, we boiled their content down to short bursts that either told site users everything they needed to know or linked them to where they need to go to complete an action (like signing up for the email newsletter or submitting a session).
We then decided to present each of these pieces of content in a slide of their own, leveraging this modular strategy to organize content depending on audience. We made sure that each slide was flexible and could be updated as needed as the event draws nearer (as “Submit a Session” turns into “Session Schedule,” for example).
To give NERDS an immediate, personable (and literal) face, we decided to develop a mascot. The feel of the NERD logo evoked the 70’s to us, and we decided to use Mr. Men, one of that era’s most famous cartoons, as inspiration. The result was Lil’ NERDy. Lil’ NERDy is memorable and adaptable, meaning she can be used in a variety of ways at any size to instantly remind people of the friendly and inclusive nature of NERDS.
Using Lil’ NERDy as an asset, the site strategy was then implemented to produce a bold, fresh site design. The design uses strong typography and bright colors to create an impactful and informative experience.
NERDS walked away with a fun, efficient site that keeps users informed and delighted as their event draws closer.
Last Call Media makes it a priority to give back to the Drupal community, so we were excited when the nice folks at Baltimore Drupal Camp reached out to us for branding and design.
How we did it
For the discovery phase, we had some in-depth conversations with the team at Baltimore Drupal Camp about design trends on the web, and which ones we wanted to explore for this project. We decided that the designs should express both the historic and the punk sensibilities of Fells Point, the neighborhood the event was to take place in, while also paying homage to Frederick Douglass, as the venue was the great Frederick Douglass-Isaac Myers Maritime Park.
Drupal Boh logo
Baltimore Drupal Camp had in previous years done a few mash-ups of the Drupal and National Bohemian logos in previous years (the National Bohemian logo being the unofficial logo of Baltimore), and we were excited to try our hand at it. We produced a clean, durable logo that was used in all our subsequent material and that the Baltimore Drupal Camp also adopted for use across their social media platforms. Baltimore’s own Not Mr. Boh even gave it a shout-out on Twitter.
In order to capture the historic and punk aspects of Fells Point that we discussed during discovery, we produced a series of gritty, vibrantly colored illustrations for use as visual assets on the site.
Taking both the direction we established in discovery and the assets we generated, we produced a design deliverable for the Baltimore Drupal Camp website. Because the site would change so much as the event got closer (and “Submit Your Session” became “Schedule”, among other changes) we needed to deliver a wide-ranging design that would account for multiple versions of the site.
We worked with the team at Baltimore Drupal Camp while we created the designs and formatted them for handoff, enjoying a productive information exchange about tools and process while we were at it.
There aren’t enough words of thanks for Colin and Last Call Media. Amazing site design, fabulous t-shirts, awesome stickers! You are Drupal!Liz Lipinski, Baltimore Drupal Camp
We were thrilled to see the aesthetic and assets we generated for this project in use on event collateral on the day of the camp itself. Congrats to the Baltimore Drupal Camp on a successful 2016!