Blog

Overview

Let's face it, upgrading or applying service packs or patches to your website is no fun.  It's a necessary evil that takes time, resources and planning within your organization to do successfully.  Unfortunately, organizations often mistakenly push off these costs, deferring upgrades over other short-term projects. But inevitably, this decision to not upgrade your website regularly makes the task harder and incurs even greater investment, both in terms of actual and opportunity costs.

In this Ultimate Guide to Upgrading and Patching Your Website, we'll walk you through all of the terms, concepts, ideas, and best practices around upgrading a medium to large scale website.  We'll impart our experiences of helping customers perform and test their website upgrades over the past 20 years, so that you can upgrade faster, more efficiently, more often and with better results.

Components of Your Website

Although websites appear from the outside to non-technical visitors as a single unified technology, on the inside this is not at all true.  Today's websites are made up of dozens of individual technologies, or components, developed by various third-party vendors. Each component plays a different role and provides a different purpose, but all need to be implemented and maintained so that they work together to provide a properly functioning website.

These components are typically divided into two main buckets: front-end (or client-side) components and back-end (or server-side) components. Both perform specific functions and are typically built with a corresponding set of tools or libraries.  Let's take a closer look at these components now.

Front-end or Client-side Components

The front-end components that are used in your website, control how end-users interact with your webpages and the experiences they perceive.  The core technologies that control these interactions in the browser are:

• HTML - Hypertext Markup Language which defines the structure of the information on the page.

• CSS - Cascading Style Sheets which control the presentation of the styles, fonts, colors, and layouts of the content.

• JavaScript - Code that executes natively in the browser to give interactivity to the page by manipulating the HTML, content or styles on the page, reacting to user events such as mouse clicks, moves or scrolling and sending and receiving information from the server via AJAX calls.

As modern web technologies have progressed, the vast majority of websites utilize front-end libraries, frameworks, and plugins that make it easier to control the HTML, CSS, and JavaScript that renders within the browser.  From a high level, these can be grouped into three buckets:

• JavaScript Libraries / Frameworks - There are numerous popular JavaScript libraries that provide a range of functionality and approaches.  A few of the most preferred are jQuery, Angular, React, and Vue.

• Responsive Libraries / Frameworks - With the explosion of mobile devices, several front-end component libraries provide responsive grid systems and pre-built user interface components to help make websites mobile friendly. Two of the most popular are Bootstrap and Foundation.

• Plugins / Components / Widgets - Thousands and thousands of User Interface widgets such as sliders, charts, data grids, gauges, animations, input controls, etc. are available to download and include in your website.  These plugins can range from loading on every page of your site to only a few pages.

While a modern website will typically only implement one main JavaScript library and one responsive framework, they will most likely have numerous JavaScript based plugins. For large or heavily interactive sites this could easily be over a dozen or two.

Back-end, or Server-side Components

The back-end components that are used in your website, are somewhat invisible to your users, but provide the core services needed by your website.  There are several layers of technologies in the back-end stack, which may include:

• Operating System (Windows, Linux)

• Web Server (IIS, Apache, NGINX, etc.)

• Application Server / Programming Language (PHP, ASP.Net, Python, Ruby, etc.)

• Database (MS SQL, MySQL, Oracle, MongoDB, etc.)

• Web Development Framework (Ruby on Rails, .Net, Laravel, Cake, etc.)

• Content Management Systems (WordPress, Drupal, Joomla, etc.)

• Other Services and APIs (such as search, video, e-commerce, chat, email, ERP, etc.)

Component Release Lifecycles, Timetables and Support Lifetimes

With the pace of change ever increasing in the web space, it's a safe bet that most of the vendors that supply the components that your website relies on are continuously innovating, releasing fixes, enhancements, new features and security patches at a relatively rapid rate.  

This becomes significant when many of the components of your website rely on or have dependencies to other components or libraries.

How do you know when to upgrade a component or not?

Software Versioning

To help with understanding the dependency impact, each vendor follows a software versioning scheme.  Unfortunately, not all vendors follow the same scheme, but there seems to be a common set of verbiage that describes the releases.  Releases are typically referred to using one of the following terms:

• Major Release - A major release is a new version of the software that contains several significant new features/capabilities.  It also may contain support for new versions of platforms, major architectural changes, and/or updates to its internal API.

• Minor Release - A minor release is typically a release that includes a small set of new features or enhancements and a set of bug-fixes.  These new enhancements do not typically contain significant new features, but rather minor enhancements to existing functionality.  A minor release is typically delivered in the form of a single installable package.

• Service Pack - A service pack is typically a collection of patches to a component or library.  Once the number of patches reaches a certain arbitrary number, the vendors often release a service pack which is a roll-up of all of the patches.  A service pack is typically delivered in the form of a single installable package.

• Patch or BugFix or HotFix - A patch, bugfix or hotfix is typically an isolated fix of one or more issues reported by the package's customers or found internally that the vendor warrants is critical enough to release.  A patch may not be distributed as a full release, but may only contain those files needed to fix the specific issue.

SemVer, short for Semantic Versioning, is a specification growing in popularity that uses three numbers to describe the version instead of two.

Where each part of the 3 digit version changes based on the scope of changes that occurred.  Specifically,

• the MAJOR version number changes when there are incompatible API changes with the previous version,

• the MINOR version number changes when new functionality is added in a backward-compatible manner, and

• the PATCH version number changes when backward-compatible bug fixes are made.

So for example, say you utilize a library named 'X' and the installed version of it documents that it is dependent on version number 6.0.0 of the package named ‘Y'.  If the vendor for package Y comes out with a patch that fixes a bug or security vulnerability that is backward compatible with the 6.0.0 version of their API, they will increment the version to 6.0.1.  In this case, it would supposedly be safe for you to download and install the 6.0.1 patch.

If the vendor then added an enhancement or new set of features that were also backward compatible, they would release 6.1.0 and this too would supposedly be safe for you to download and install.

If however, they released a new version that broke backward-compatibility, they would release version 7.0.0, and this would not be safe to download and install without possible updates to the API calls.

Please note that we said 'supposedly' safe to download and upgrade.  As per the specification itself 'As a responsible developer, you will, of course, want to verify that any package upgrades function as advertised. The real world is a messy place; there's nothing we can do about that but be vigilant. What you can do is let Semantic Versioning provide you with a sane way to release and upgrade packages without having to roll new versions of dependent packages, saving you time and hassle.”

Different Release Timetables

To make matters a little harder on you, each of the various vendors release their updates in the form of major releases, minor releases (service packs) and patches (hotfixes), at different times throughout the year.   Some are non-critical and can wait, but others may close security holes and should be applied immediately.

Some vendors have introduced scheduled releases to help this.  For example, Drupal documents that they will release patches on the first and third Wednesday of every month.  But other vendors don't follow suit. In fact, in the spring of 2018 when Drupal sites were being hacked by the famous Drupalgeddon 2.0 security vulnerability, they released a critical security patch on April 18th and then another one on April 28th.

So it's important to check often with your vendors if new updates have been released.

Component Support Lifetime

It's equally important to also note that these components in your website have a shelf life for support.

What that means is that the vendor will stop issuing hot-fixes or service packs for that version based on either a pre-designated date or after a new version is released.  

For example, for PHP, the widely used application server/development language, the vendor publishes their support lifecycle in the support section of their website.  The chart below shows that each release typically has a two-year window for support.

Source: http://php.net/supported-versions.php

Other vendors may discontinue support for a release once a newer release is made available.  For example, many vendors support the current version, plus one or two versions back. So if they just released version 6.0, they may support that and version 5.x, and stop providing support for 4.x altogether.

So, keep in mind that if you don't update your website's components regularly, the version you are running may fall out of support.  If that happens you may encounter bugs that you can't get a patch for, other libraries or plugins that you upgrade may have dependencies and may break, or worst of all your site may be vulnerable to known security exploits.

How Often Do I Need to Upgrade?

Let's track the releases for a typical website that uses some of the technologies listed above.  How often do you think the site owners would need to upgrade to stay absolutely up-to-date?

If you answered once or twice a year, think again.

The table below shows the major components and their initial release version of a typical site that was fully up-to-date at the end of 2017.  If we track all the major, minor and patch releases up to mid-January 2019 (a little over 12 months), you can see the number of releases that occurred.  Please note that we are only listing some of the major components here.  An actual site would, most likely, have dozens more plugins, that would significantly add to the number of releases.

	Component Version as of Dec 2017	Major Releases During 2018	Minor Releases During 2018	Patch Releases During 2018	Critical Security Patches
Development Platform	PHP 7.2			13 (7.2.0-7.2.13)	1
Content Management	Drupal 8		2 (8.4.3-8.6.7)	15 8.4.x (3) 8.5.x (5) 8.6.x (7)	7
Database Server	MySQL 5.7			5 (5.7.20-5.7.25)	4
Web Server	Apache 2.4			9 (2.4.29-2.4.38)	4
Front end Framework	jQuery 3		1 (3.2.1-3.3.1)	1 3.3 (1)
Responsive Framework	Bootstrap	1 (3.x-4.x)	2 (4.0-4.2)	3 (4.1.0-4.1.3)	1
JQuery Plugins	OwlCarousel2		1 (2.2-2.3)	4 (2.3.1-2.3.4)
JQuery Plugins	Masonry			3 (4.2.0-4.2.1)
JQuery Plugins	ChartJS			2 (2.7.1-2.7.3)
TOTAL		1	6	55	17

As you can see, over this annual period a minimum of 62 releases (1 major release, 6 minor releases, and 55 patches) were made, and 17 of those contained at least one fix for a critical security vulnerability.

What this means is several times a year you need to be looking to update the components of your website in order to stay supported, get the latest fixes, and close any security vulnerabilities in your site.

5 Reasons to Upgrade Frequently

If you were to ask a Webmaster or internet marketer, who is responsible for running mid to large size websites, whether stability was one of their primary concerns, they would undoubtedly say Yes, without even thinking twice.

Once an organization has rolled out their new website, or successfully added the next set of new features, they often take an ‘if it ain't broke don't fix it' approach.  And depending on what else is on their plate and what the next set of business requirements are, it may be some time before they consider spending the time, resources, and money to upgrade the software components that drive their website.

But is this ‘Set it and Forget it' mentality the best approach to the stability, performance and overall success of the website?

Here are 5 reasons that you should consider in support of upgrading your software more frequently.

1. Close Security Vulnerabilities

As everyone knows the World Wide Web, is also the Wild Wild Web.  With the increased adoption of open source, many critics contend that security vulnerabilities have also increased as the result of hackers having access to the source code.  Even for proprietary software products, vulnerabilities are on the rise. Although the software development community is very good at quickly cataloging, and releasing security patches for these vulnerabilities, organizations have historically shown a lack of urgency to apply patches, often because of ongoing maintenance and software upgrading is given a low priority internally.

If you google 'percentage of websites running known vulnerabilities” you'll see dozens of articles with titles like:

• Ninety percent of websites vulnerable to attack

• 77% of sites use at least one vulnerable JavaScript library

• 57% of WP Sites Become Less Secure in December 2018

And quotes like this 'A WP White Security study found that a staggering 73% of all WordPress installations had known vulnerabilities that could easily be detected using automated tools.  Cyber criminals have long discovered these security holes, with over 170,000 WordPress sites being hacked last year.”

Source: https://www.incapsula.com/blog/cms-security-tips.html

The bottom line is, if you upgrade more frequently you will be at a much lower risk of being compromised by a known security exploit, as a large share of security exploits attack known and fixed vulnerabilities.

2. Take Advantage of New Features / Performance Gains

Upgrading more often means you get to take advantage of all those new features and enhancements that come with the product you are upgrading.  These new features and improvements potentially allow you to present new capabilities to your site visitors, increase their productivity or the productivity of your site authors, improve ease-of-use, and ensure that you are maximizing your performance.

For any components of the website that are not open source, you are probably paying an annual subscription, or are running under a Software as a Service (SaaS) model.  In these cases, you are in essence already paying for those features, so why not utilize them?

In addition, new releases often help address any performance bottlenecks, giving you a potential boost in speed and better usability for your site visitors or content authors.

3. Take Advantage of Bug Fixes

Software vendors firmly understand that product quality is of the utmost importance to you, the customer and their long term reputation.  They typically try hard to not put out a product release that is buggy and does not successfully pass all stages of their rigorous quality-control process.   However that being said, software development is hard. No matter how great the vendor's developers are, and no matter how much testing they do, there will always be bugs.

As vendors come across new issues, found internally by their team, by you or another customer it is in the vendor's best interest to resolve those issues as quickly as possible.  As such, they typically release patches or hotfixes that address individual issues.

By regularly reviewing and keeping up with these patches or service packs you will ensure that your site visitors or internal team won't be bitten by bugs that others have already found and which have been fixed.

Except for critical security patches, you may elect to only take patches that are likely to affect your use of the product or wait until a larger set of issues are rolled up into a service pack.

4. Easier for the Vendor to Support You

By staying current on a release the vendor responsible for that piece of software is able to provide better and faster support. Why's this you ask?  Simple. Human nature. Here are a couple of reasons, just to name a few:

• The features, enhancements, and issues in a newer release are fresher in the support and developer team's minds.  They just recently implemented those changes and probably know where in the code to begin to look. If your release is a year and a half old, their recollections aren't nearly as clear.  This means that they will be slower to respond and may not get the details correct.

• The support team may not have every combination of every patch installed in a test environment, but they probably have the most recent.  Having to recreate your specific environment takes time and slows down their responses.

• The further you get from the current release, the harder it is to fix bugs in the older release.  There may be changes in the code in newer releases that then need to be backported to the older version.  This becomes more complicated and at some point not worth the effort.

5. No Panic Mode

If you don't schedule for and perform regular updates, there may come a time when you have a ‘Holy S#*^t' moment.  Maybe it's Drupalgeddon 3.0, or your hard drive gets corrupted and you can't download that old version of the component you need because the vendor no longer provides a link to it.  Whatever it is, this will not be the time you want to be figuring out what the version dependencies are required to update your components.

Being forced to upgrade when it's not planned can lead to headaches for your developers, long delays, outages and/or unforeseen costs.  Not to mention your boss probably won't be the happiest camper of the bunch.

Avoid all these downsides and plan out scheduled and frequent upgrades.

What Delays Organizations from Upgrading?

Here are some of the most common reasons why organizations delay or resist upgrading, and what can be done to prevent it.

Lack of Resources

There are very few organizations that have more than enough resources to meet all the goals on their plate.  In today's fast paced high tech world more and more demands are placed on the IT Departments to ensure all systems are running smoothly, as all departments in an organization rely upon them.

Upgrading your website's components can be complex and requires resources in each phase of the process to ensure a successful upgrade.  As mentioned in this article, a successful upgrade takes upfront planning, execution, and thorough testing to avoid downtime and compatibility errors. Galvanizing these resources for this type of effort regularly taxes an organization and can hold up installing a much-needed patch or upgrade.

You need to be sure to commit and allocate time, resources and budget money upfront for ongoing maintenance.

Higher Priority Initiatives

There is often a belief within organizations that marketing efforts and sales functions outweigh IT administration because sales and marketing drive revenue.  The sales department wants to make sure the website is always up and running, welcoming and informing existing and prospective customers about their products and services.  Marketing wants to be continually branding the organization, engaging its site's visitors and receiving feedback 24 x 7. Any efforts not geared towards these expectations seem to get a lower priority, and as such it's easy to see how an organization puts regular but necessary maintenance on the back burner.

You need to think of your website sometimes as a car.  You need to make time to frequently refuel, regularly add oil, and every so often take it in for service.  Upgrading your website frequently ensures that it will continuously run smoothly, and not break down when you need it most.

If it Ain't Broke, Don't Fix it Mentality

A lot of work went into getting your website where it is today. If it's stable and serving your needs, why would you want to risk introducing new problems?  Some say, the devil you know is better than the devil you don't know, and often website owners think of upgrading as the devil they don't know. They are not sure what sort of disruption a new release might bring.

It is important to make sure you understand the benefits of a long term outlook, versus a short term inconvenience. From our experience, there are many convincing arguments that say the opposite -- not upgrading your website will lead to more instability, more security problems, and your site will typically perform slower.

The Chilling Effect of Past Upgrades

Organizations who've felt acute pain from a past website upgrade may become reluctant to jump into their next upgrade. This 'Chilling Effect” on organizations that might otherwise upgrade frequently can cause them to suffer the downsides of operating with old technology. Although lower performance is definitely a problem, exposure to unresolved security threats and a lower level of support is a greater menace.

Simply being unable to access new technologies because you don't want to suffer the pain of performing system upgrades to become compatible can be another problem.

Not Understanding the Costs of Falling Behind

Congratulations, you have invested time, effort and a lot of money into implementing a CMS and launching your new website with all the bells and whistles.  Your organization may assume that the real costs in human and monetary investment have been approved and spent. However, not keeping up with version releases and installing patches to properly maintain the site could cost more money down the road then the costs associated with regularly planned upgrades and a system to install patches timely.

If your organization fails to install the latest patch or upgrade, as new versions of the software get released it becomes more complex to upgrade because often you simply cannot skip over the version releases you ignored, and upgrade directly to the current release.  This has a cumulative effect in the time and effort that it takes to upgrade. Also, because older versions are de-supported over time, getting support if needed, will likely require further costs from third-party resources. If a crucial patch was never installed, the resulting downtime costs can also add up.   Not to mention any opportunity costs related to not taking advantage of the latest and greatest features that the new version provides.

The article, 'CMS-Connected” with Forrester's Senior Analyst Mark Grannan, states 'Content management software vendors consistently report that anywhere between 15 – 55% of their current clients are behind more than a year on upgrading their CMS, and this can lead to challenges for all parties that are assisting an organization in advancing their web strategy and infrastructure.”

In the article, the Forrester analyst suggests looking at both tactical and strategic perspectives and using your customer's customer journey to help decide whether upgrading is worth the effort.

The article goes on to state, from a strategic perspective, 'Missing out on opportunities to capitalize on new features and capabilities, such as the ability to do personalization through dynamic content is detrimental. Grannan believes that the cost of missing out on potential benefits could actually outweigh any hassles you may incur on a tactical level.”

Should you Trust your Vendor?

To trust or not trust your software vendor.  Wow, that sounds like a loaded question. Of course, you should trust your vendor when it comes to upgrades.  You already have vetted them enough to install their software and rely on it to help run your website.

But maybe when it comes to upgrades, it's best to follow the Russian proverb past President Ronald Reagan famously uttered ‘Trust but Verify'.

Not that your website vendor is out to get you, but it's the best prudence to be safe, not sorry.  Here are two areas to watch out for.

Automatic Upgrades

Some vendors, like WordPress, offer a one-click Automatic Upgrade.  Sounds great. Simple and easy.

And for other vendors, like Drupal, it seems to be one of the most requested features by their user community.

The problem is that automatic upgrades don't always work.  If your site has something special or custom about its configuration, permissions structure, dependency hierarchy, hardware resources or anything else that may affect the upgrade, there is a chance that it might fail and leave you in a bad state.

There are dozens and dozens of posts on online forums such as Stack Overflow, where automatic updaters are looking for advice on how to fix their broken site.

We are not saying don't use the automatic upgrade feature of your CMS, or easy package downloads from your favorite npm repository.  But we are saying, just don't do it in production. Always make a backup, and don't rely on it working without diligent verification.  Verification means nothing short of thoroughly testing your website and its output to see whether the upgrade affected your site.

Backward Compatibility of Upgrades

Vendor's claims of 100% backward compatibility we hope are honest (or mostly honest).  They try really hard to make sure that your upgrade experience will be successful and that everything will work the same or better than before.  But this is just hard to do.

The more complex the software component is, the more moving parts it has, the more changes that have been made to it, the harder it is to ensure that everything will absolutely work the same as it did before.

Just like for automatic upgrades, 'Trust but Verify”.  Don't assume that the vendor got it 100% right. You need to set up a testing strategy that ensures you're confident that the backward compatibility of the upgrade won't come back to bite you.

In short, you can definitely trust your vendor and partner with them to ensure your upgrade will go smoothly.  But remember it's on you to verify that everything is OK with your website. The only way to do that it to test, test and then test some more.

Common Upgrade Testing Approaches

Over the past 20 years helping customers upgrade or apply service packs or patches, for mid-sized (1K+ pages) to large (10K+pages) websites, often they would contract with us to just perform the actual upgrade, while they would handle the testing.  In 95% of the cases, their testing strategies would fall into one of the following two camps: The ‘Brute Force Approach' or the ‘Sample and Hope Approach'. Let's talk about each. Unfortunately, neither are very effective.

Brute Force Approach

The Brute Force Approach is a full team inspection, where you distribute page list assignments to team members to perform a manual review of the designated pages to identify any changes, issues, or errors.

This process is extremely resource intensive; it requires excessive amounts of staff time to do even a rudimentary examination of hundreds or thousands of individual web pages just for one device (and if you really want to be thorough multiply the number of pages across all your different devices sizes).

Because it covers such a high-volume of content, this approach is also notoriously slow and can take weeks, or even months to fully accomplish.

Due to its distributed nature, this method can also be inconsistent. Page reviews are carried out with varying levels of quality and coverage. These differences in review quality can be determined by a variety of uncontrollable factors.  There may be differences between the review processes of the various individuals doing the work. There can even be different processes or different levels of diligence applied by the same individuals at different times. And no matter who is doing the review, the effects of simple workday distractions can disrupt the review process and allow errors to slip by unnoticed.

But definitely, the worst part of this approach is how poorly it scales. As page counts increase, it becomes much more difficult to effectively manage. A full, manual review of a ten to one hundred page site might not be a challenge, but what if your site has ten thousand pages, and you want to test it rendered on other device sizes including handheld, tablet, and desktop?

It's painful and a sure-fire way to make your team unhappy.

Sample and Hope Approach

The Sample and Hope Approach is basically a limited page inspection. You randomly select a series of representative pages from your site map and manually test this limited sample set for errors or unexpected changes.

This approach is designed to offload much of the workload of a full, page-by-page site review, but this imposes limitations of its own. This surface-level audit approach allows you to confirm that some of your pages work as expected, but for the rest, you simply have to cross your fingers and hope for the best.

Any page that isn't directly tested retains a latent risk that it contains undiagnosed errors.  You might be able to live with this uncertainty, but it's a risk. Remember the old developer adage 'everything that is not tested will break”.  

You can see why this approach is not ideal either.  It's a game of chance, or maybe better yet like playing Russian roulette.  You may get lucky testing only a subset of pages but at some point, you're going to lose.  Site visitors or your boss might be the first ones to discover the problems your testing didn't catch which ultimately reflects poorly on your team and organization.

Website Upgrade Effort

If you were to look at where the effort is spent by most organizations when upgrading their medium to large sized website, it typically breaks down into these 5 areas:

• Planning - As it sounds, the initial site review and planning to determine what, when, who, and how the upgrade will be performed.

• Communicating - Informing, updating and helping the website stakeholders regarding the upgrade and their roles.

• Upgrading - Preparing and configuring the servers and performing the actual software updates.

• Testing - Iteratively testing the site to ensure that no issues are caused by the upgrade.

• Issue Resolution - Solving of any issues found during testing.

Surprisingly the actual effort to perform the upgrade of the software is often not a major contributor to the overall expenditure of time and resources.  If the organization relies on manual testing of the website, the area that requires the most effort is almost always the Testing phase.

Then, depending on how involved the upgrade is (simple, major, or somewhere in between), the Issue Resolution phase often becomes the second most expensive area in terms of effort.

From our experience, here's how these areas of effort typically break down for Simple vs Major Upgrades.

The good news is that the area with the largest effort, Testing, can be significantly reduced by incorporating automated visual testing.  See the next section below Recommended Testing Approach.

Recommended Testing Approach

After reading about the two traditional testing methods above, we're sure you were left feeling, there's got to be a better way!

The good news is Yes, there is a better way.  The many limitations of traditional testing methods have haunted web development efforts since the advent of the internet. Yet, despite all the wasted time and resources of imperfect reviews and excessively-manual processes, solutions to relieve this challenge have been slow to emerge. But what if we could do something better?

One way to avoid many of the major downsides of the two traditional review processes mentioned above is by using a cloud-based automated visual comparison tool, that's quick to set up and use.

What is it?

Conceptually, this technology works in a very straightforward way, which can be broken down into five (5) main steps.

1. 1. Setup. First, in order to tell the tool what pages to test, you either import a spreadsheet of URLs or have the tool spider your site.  You also indicate what device sizes to test. Typically you will pick one for each of your responsive breakpoint sizes for mobile, tablet and desktop.

2. 2. Pre-Upgrade Snapshot. Next, to get a baseline picture of the site prior to upgrading or applying a service pack or patch, have the tool capture images of each page, at the various screen sizes.  This builds a comprehensive set of 'baseline” (or 'pre-upgrade”) images of all your pages.

3. 3. Post-Upgrade Snapshot. Then, after you perform the upgrade, or apply the service pack or patch, you'll run the tool again capturing all your pages, at the same screen sizes you specified. This becomes the 'comparison” (or 'post-upgrade”) collection of images.

4. 4. Difference Detection. The tool then performs an automated, digital, pixel-by-pixel comparison of the pre-upgrade version against the post-upgrade version to determine what, if any, changes occurred between the two versions.

5. 5. Report and Tag.  Lastly, the tool separates the resulting screenshots into two buckets: Same and Different.  Screenshots of pages that are the ‘same' are digitally identical, meaning each and every pixel rendered exactly the same pre vs post upgrade.  These screenshots can be ignored, as you know they were not affected visually in any way by the upgrade.
   
   Pages that were flagged as ‘different', however, need review, as they no longer render visually the same after the upgrade, as they did before.  The tool provides a variety of viewing modes (side-by-side, splitter, flicker, red-lined, and isolation) to help you quickly see the differences.   You can also tag differences so that it easier to assign and resolve any issues caused by the upgrade. And because the tool is automated it can be used to quickly test your entire site as you iteratively fix any issues.

The Benefits

The advantages of using an automated digital visual comparison tool are manifold. It improves your efficiency (much less time and effort), increases your accuracy (down to the last pixel), and rescues you and your organization from lurking surprises in your website implementation.

Time

An automated testing process can save you significant time and effort over manual testing. Comparing pre-upgrade and post-upgrade pages across a 10,000-page site might just take a few hours using the tool, as opposed to weeks or months with traditional testing methods.

Because the testing can be performed so quickly, it allows you to also fully test at each stage of your upgrade process, for example after your dry-run and immediately after the final upgrade.

Accuracy

In addition to big, obvious issues like missing templates, javascript errors, and translation problems, a pixel-by-pixel, photographic inspection can reveal even the smallest changes. Minor issues that are easily missed during a manual inspection are quickly revealed using the tool.

If your content blocks move one pixel up the page, for instance, you'll know it. And this detail could be a crucial clue for helping you troubleshoot other errors that appear elsewhere in your build.

The tool allows you to see red-lined versions of your page where the pixel differences are highlighted in red.  Additionally, you can flicker between two versions to see subtle layout changes, or use a slider view to see the page side-by-side.

Peace of Mind

If you're the one responsible for the website upgrade, no matter how easy or limited the upgrade is, it's hard to trust the accuracy of testing when reviewing large numbers of pages manually.

By running an automated tool to test your upgrade, you will have confidence and peace of mind that no lurking surprises exist.  Even if the upgrade is minor and no visual changes occurred because of it, knowing that every page was tested and that all of your pages have been confirmed as identical on a pixel-by-pixel basis, is valuable.  You can put those pages out of your mind with confidence and any differences that are found, you can concentrate your efforts on understanding and fixing what made those pages render differently.

Resources

As an automated, digital approach, a visual comparison tool offloads the burden of performing much of the manual and brain-numbing testing from you and your team. These resources can instead spend the saved time testing other areas that are more process-based and cannot easily be tested visually, or working to develop new opportunities that directly benefit the bottom line of the organization, rather than doing preventative testing.

Frequency

By eliminating the manual effort and increasing your page coverage, a cloud-based visual comparison tool helps you reduce the risks of your next upgrade project. It becomes much easier to implement future upgrades when you can execute them faster, with fewer resources, and less risk of errors.

This one, simple process improvement, then, can help you upgrade more frequently thus keeping your site up to date, reducing security risks, improving overall performance, saving maintenance costs, and allowing you to take advantage of the latest software features.

A secondary frequency advantage is that you can run this kind of comparative testing over and over again to confirm your fixes as you implement them.

Best Practice Steps to Follow when Upgrading Your Website

Every upgrade project is somewhat different.  However, that being said, after 20 years of performing website upgrades, these are the top 9 steps that you should consider when embarking on your next website upgrade project.

1. Identify and Engage Your Stakeholders

If your upgrade project is beyond a minor upgrade, don't forget to identify and engage the various stakeholders of your website so they know what's going on.  These include your various content owners, site developers, end-users, the marketing and communications teams, trainers, management and whoever else in your organization is directly connected to the website.  Determine how each interacts with the website, how they will be affected, possibly how they can help, and what type of communications you need with them.

2. Review and Plan

Start the process by performing a thorough review of your current site, taking inventory of all the third party component libraries, and plug-ins that are being used, as well as all custom code modules that you may be upgrading.

We suggest creating a spreadsheet listing out each component so you have a definitive list that you can check off on while you upgrade each component.

For each entry, record what version you are currently running, determine whether the vendor has released any newer versions of the software and if so choose what version is appropriate for you.  It might be that they have released one or more new patches for your current version and a new major or minor release. You'll also need to read their release notes to determine if their new release has any dependencies that you need to include or update in your plan.

You'll also want to see if any of the components that you are currently using have been deprecated or desupported.  This happens more than you think, especially with plugins. Developers release a great new open source plugin, support if for a period of time, then lack time to keep it up-to-date.  If you find any that are no longer supported you may need to consider a replacement.

You will also want to obviously plan when the upgrade will be performed, who will do it, how it will be tested, what downtime if any may occur and how you will communicate that downtime to site visitors and/or internal users.

It's also important to have a detailed rollback plan if things don't go as expected.  Some questions you need to answer in that plan are: How will you most efficiently restore the site to its previous state?  Who will perform the rollback? What criteria will be used to determine when a rollback is needed?

3. Plan at least One Dry-run Upgrade

The best way to ensure a successful upgrade is to perform at least one dry-run of the upgrade and then test it thoroughly.  If you run across any issues, you should resolve them and test that your fixes worked. It is highly important to write down exactly what you did to resolve the issues so that you can remember to do the same thing when performing the final upgrade.

If your list of fixes grows lengthy you may want to schedule a second dry run so that you can ensure that your steps are correct and that the final upgrade will go as smoothly as possible.  You want it to be seamless and performed in the shortest amount of time as possible to mitigate changes to your site during the final upgrade.

You should also be sure to plan plenty of time for this step. If issues or conflicts do occur you need to iteratively test and resolve the issues.  This may take time.

4. Backup Everything

An obvious but sometimes forgotten step is to always backup fully your entire production site before you start your upgrade.  Make sure each component that you will be upgrading is backed up to a safe place that can be quickly restored from. If performing the upgrade into a staging environment, use the backups you created, to create the staging environment, as this will ensure that your backups are complete.

5. Upgrade in a Staging Environment, Never Production

Whenever possible, never upgrade directly in your production environment.  Instead, create a staging environment, upgrade in that environment, and then switch to make it production after everything is working and tested successfully.

A proper staging environment should be set up in a virtual environment like a VMWare or VirtualBox environment. The benefits of a virtual environment are many:

• It allows for replicating the exact setup of the existing site's infrastructure so that the results of the upgrade can be reproduced as often as necessary.

• File system and database system backups and restores can happen very quickly.

• Whole servers can be cloned and created within minutes as opposed to hours in a non-virtual environment.

The upgrade environment should be on completely separate servers and should not rely on production in any way (file system, database access, networking) so that testing procedures and issue resolution work do not impact any production resources.

Any and all of the site's code and resources should be put into some kind of source control. Either a local git or subversion repository on the local network or in a cloud-based repository run by one of the many source control services out there. The benefits far outweigh the effort in setting one up and regularly using it. Some of the benefits include:

• there's no question where the latest version of a file is

• allows for a starting point during the upgrade process

• issues that arise from the testing after the upgrade can be resolved and what code has changed can be followed and managed

• if more than one dry run is executed then when file systems are refreshed and code breaks reoccur, the fixes can be easily reapplied to the file system

6. Prepare Documentation, Training and/or Support

Prior to go-live, if your user community of content authors, trainers, marketers, or other stakeholders needs updated documentation, training or support, put those materials together and communicate how they should be accessed.

You might be able to leverage information like release notes or upgrade documentation from the vendor directly.

We have found that if the update introduces interfaces changes that affect daily tasks, it might be helpful to put together a quick reference guide that shows the 'before” vs 'after” way of doing specific tasks.

7. Handle Switch From Staging to Production

While performing the final upgrade on the staging server, you may need to handle certain events or processes during this time period to ensure data continuity.  Here are a few that you may need to consider based on your site:

• Turn off any email processes on the staging environment that may send inadvertent messages to customers while testing. Then remember to turn it back on when you go live.

• If your website is powered by a CMS that utilizes databases for storage of web page content, you may need to freeze authoring on the production environment, and have your site authors instead enter content into the staging environment. If it's critical that updates can still be made into production while in this final stage, give limited access to authors for emergency edits and have them dual enter their content into both systems.  That way when you switch over to the staging site, the content in the database will be current.

• If your site has web forms that collect and store site visitor entered data, you may need to develop a script that clones this data into the staging server's copy of the database.  Again this is done so that no data is lost that was entered into production during this short final upgrade period.

Once the staging environment is thoroughly tested and ready for Go Live, if you are able to simply repoint your load balancers from the old production servers to the new production servers, your cut-over should be quick and seamless.

8. Perform Thorough Testing

After each dry run upgrade, and just after the final upgrade, it's important to conduct thorough testing of your site.

Testing should be performed on many different levels to ensure that the website meets your standards and that nothing in the upgrade affects your site visitor's experience.  Here are some of the different levels of testing that you need to perform.

Visual Testing - You need to ensure that the web pages within your site all render the same after the upgrade as they did before it.  HTML, CSS, and JavaScript are very precise. A small style change in CSS could make a big difference in how your content renders.  The same goes for HTML, if your CMS, for example, introduces an extra layer of DIV tags, it may throw off your CSS or JavaScript selectors, causing rendering differences.  Any JavaScript or resource loading errors may also cause page interactivity for your site visitors to break.

If your site has more than a hundred or so pages, it's worth automating the process of visually checking that all of your pages render the same, across multiple device sizes.  

The advantage of using an automated tool is that you can use the tool to minimize the manual testing after each dry run stage and after the final upgrade.  Since it's automated and digital it will save tons of time and resources, be much more accurate and will give you assurances that everything worked as expected.

We obviously recommend using our WebsiteUpgradeTester.com visual testing tool. It allows you to digitally compare the pre-upgrade and post-upgrade version of some or all of the pages on your site quickly with 100% accuracy.

Functional Process Testing - If your website has complex processes that need to be tested, such as a new user enrollment process, or e-commerce purchase process, automated visual testing may not be enough.  You might either manually test these processes if the processes are not too complicated or too large in number. Otherwise, consider using automated functional testing software such as Selenium.

Security Testing - Any website that collects sensitive customer data such as credit card numbers or personally identifiable information (PII) should be tested for security vulnerabilities.  Doing this testing as part of an upgrade makes sense, as you are introducing new components that may affect your overall security.

Web security testing tools are useful in proactively detecting application vulnerabilities and safeguarding websites against attacks.

Depending on your industry, it may be mandated that you perform and meet certain guidelines.

Accessibility Testing - Ensuring that your website is usable by people with disabilities (i.e. hearing, color blindness, old age, and other physical disabilities) is of growing importance and is required under some government regulations.  There is a set of web accessibility standards created by the World Wide Web Consortium (W3C) known as Web Content Accessibility Guidelines (WCAG).  These standards form the basis for measuring your website's accessibility level.

There are numerous tools that you can use to test your website for accessibility issues.

9. Perform a Post-Upgrade Review

After the dust has settled, perform a post-upgrade review with the website's stakeholders to identify what worked well and what didn't.  For those areas that did not go smoothly, try and identify where improvements can be made so that the same mistakes won't happen for the next upgrade.

We recommend adding these notes into the same spreadsheet that you tracked the version of each component, maybe in a different sheet.  This spreadsheet then becomes a great starting point for your next upgrade.

Conclusion

Keeping your website updated and patched is vital to the overall health of the site and as a direct connection to your business needs to be performed regularly.  The work associated with maintaining the current status of front-end and back-end components of the site can be quite involved and includes the cooperation of many individuals and stakeholders, but the overall benefits outweigh the effort required. With the right processes in place, along with the right automated testing tools, upgrading your website on a regular basis can become light work shared amongst co-workers, instead of a burdensome effort.  Your site can then continue to run like a well-oiled machine, leveraging the latest features, bug fixes, and security patches.

---

WebsiteUpgradeTester.com is the only cloud-based, automated visual testing tool designed to help you efficiently and affordably test your website upgrade. Whether it's a major version upgrade, service pack or security patch, you can quickly test your entire site, no matter how large, to uncover any lurking issues due to the upgrade.

Don't waste time manually testing your next Website upgrade. Visit WebsiteUpgradeTester.com and sign up for a free trial.