by Daniel Sim
In this series we are looking at three methods of measuring your website performance: 1. Sampling your visitors 2. Analysis and recommendation tools 3. Time measurement. All are distinctly different ways of measuring the performance of your website, with advantages and disadvantages to each.
Analysis and Recommendation Tools
These tools take an idealistic view of speed optimization: if there are no other considerations what are all of the possible things that could be done to speed up the shop.
In reality, there are lots of other considerations that we'll look at now. You cannot look at the recommendations in a tool like this as a checklist of things that must be done because...
...their recommendation might break a feature
Implementing any of their recommendations should always involve a thorough test of your shop after to make sure it has not had an adverse effect.
...their recommendation might not be technically possible
Some of the pieces of your shop are out of your control. The code and images from apps are often hosted by the developers themselves, not possible for you to change.
In this case, the only option is to reach out to the developer to ask them about the recommendation from the tool. There are good reasons why it might not be possible for them to implement the recommendation, or they may be able to.
The simplicity, reliability and power of Shopify means that some things are completely out of our reach, handled by Shopify. The stress, cost and burden of maintaining and supporting servers, for example, is borne entirely by Shopify.
That does mean that we can't implement a recommendation that involves a server change. However Shopify does a very good job of making sure their servers are well optimized for speed. It is very rare this will be a real speed limitation.
...their recommendation might slow down your shop
The recommendations should not be implemented blindly- measure first meaningful paint time before the change and again after the change. Each change should be as small as possible to make sure they can all be measured independently.
...their recommendation might come with unwanted side effects
These tools don't know what's important to you as a store owner. They are relentlessly focussed on speed.
They may recommend compressing your images to the point they look so low quality they tarnish your brand. Perhaps they don't like the code of your exit intent popup that is doing a great job of building your mailing list. Or they could ask you to change things that make it hard to update your shop in future.
For some things in speed, there are no trade-offs. But often speed optimization involves making decisions that affect other parts of your processes and business.
Beware Of Vanity Metrics!
Who wouldn't want to say their shop scores 100 or gets an A+ grade? What does that tangibly mean to your visitors and your business? That is hard to say.
Scoring top on these tools can be vanity and worse, folly since there are many successful shops who score poorly but do not have speed issues. There have been many studies which show an improvement in loading time in seconds positively affects conversion rate. The same cannot be said for scores and grades from speed recommendation tools.
It can be very expensive to chase elusive grades from these tools. We've seen shops even change platforms purely to do so- very expensive indeed!
When Not To Use
These tools shouldn't be used as checklist of all of the work that must be done. A skilled speed expert developer will use them, but their method is much more thorough and involved, trying, measuring, testing and iterating to give you a really faster shop.
Unscrupulous developers can game their grades and scores so you do not want to hire someone cheap who is just going to get you an A+. This will not benefit your business and you could end up with a mess of problems as above.
When To Use
Many of these tools are built by very smart speed experts who've seen how the world's fastest sites are developed. Their recommendations are a great signpost towards the things that might be possible to do to improve your speed.
By methodically measuring the loading time, trying out a recommendation, testing, and then measuring the loading time after, you can use their advice as good ideas to speed up your shop.