While building the roadmap for Loupe we’ve been doing a lot of interviewing – of existing users, customers, prospects, and the community. We got a lot of feedback on how well the Events & Issue features are working for real users in the field. What showed up in many conversations was it took too long for people to get the hang of what they should do, where, and when to really get value out of Loupe Server.
In particular, we’ve zoomed in in some key questions people want to answer quickly in Loupe Server:
- Are we better off shipping our latest build?
- Are we getting better over time?
- What are the main issues in production?
To answer these questions we created a new Issues Overview dashboard. If you were only going to use one page in Loupe Server, this is the one – you can pick your level of concern (production activity down through internal development builds) and then it’ll tell you how you’re doing, which issues (if any) you really should fix if you have the chance, and what are the most interesting runtime events to check out.
We get it, you could have a lot of events going on – sometimes thousands of them. You have time to only look at a few and you need us to find them for you. We identify the most important events you should check out to see if they’re really defects you need to fix or something that you can ignore.
We removed massive lists of issues or events from the overview because when you see a list with more than 10 or 20 items you stop reading them and just think “wow, that’s a big pile of work. How about I go find a better use of my time, like watching YouTube videos..” Instead, we pick the most relevant 20 items and it shouldn’t show you more than 10 of them at a time. We validated the selection process with our own use of Loupe as well as key beta customers to ensure the critical items were always in that top 10.
Multiple Hats Welcome
Many of our users perform multiple roles in their organization – they may be a technical lead for new development, providing production support, managing QA, or overseeing a beta program. After interviewing a number of customers we came to see that while many move between roles at any one time they’re focused on one aspect of their data – production, beta, QA, or development. That’s where we got the idea for how the Issue Overview page needed to work – it needed to start by asking what perspective you wanted then just “do the right thing” to figure out what you wanted to know.
Fortunately, Loupe already has a concept of Release Types to distinguish versions by how far they get in your development process. Let’s say you have a Continuous Integration (CI) environment that does a build every time there’s a check in. Each of those builds gets a unique version – but most of them don’t go anywhere. By default, Loupe considers them all Internal versions, the lowest Release Type.
If a particular version is sent off to a group of external users to try out you would promote that version to Beta – since it left the premises it’s automatically more important than ones that only existed within the development team. If you publish a version to the world then that’d be a Release version. You can customize the release types to your particular organization – perhaps you want to go Development->QA->Certification->Production.
By letting you select a release type to view information from we’ve eliminated noise – things that aren’t interesting to you (at least not right now). For example, if you’re running the beta program of your latest application you probably don’t care about what’s happing with the last production release but you do care about how the next internal version is doing.
Finding the Right Versions
Once you’ve selected the release type you want Loupe picks key versions – the latest version of that type, the previous significant version, and the latest version of the next lower release type (the best candidate to be promoted). Here’s the current view for Loupe Desktop on our CEIP system
You can see that by default it’s picked the current release version (22.214.171.1249), the previous release version (126.96.36.1993), and our last beta we published (188.8.131.521) – which we could choose to ship as a release version. Between the current shipping version and our last beta no new issues were detected and one issue was resolved, so we know we’re better off with this beta than what’s currently shipped.
Interestingly, it skipped the 184.108.40.2063 Release version. Why? Loupe recognizes the case with releases where you have a rapid hotfix and skips over those versions to get back to the next previous significant version. This is done by scanning the version numbers in use for an application and looking for a variation in the first few places (excluding any build number). Click on any of the metrics – new, resolved, open for any of the versions it has selected gives you a list of all of the related issues.
We’re now in a position to answer the key questions raised at the top:
Are we better off shipping our latest build?
This can be answered from the Next Version metrics:
- The Open list lets you know on balance if things are better or not.
- The Resolved list shows you if you’ve really made life better by fixing things users have experienced.
- Be sure that anything in the New group is innocent enough to ship.
Are we getting better over time?
Compare the Open counts of the Previous Version, Current Version, and Next Version. Are the numbers going down or up? If they’re going down, you’re headed in the right direction!
What are the main issues in production?
Look at the Open list on the Current Version. This is the complete list of issues impacting users right now.
I’m Sold, Where is it?
Once you’ve upgraded to Loupe 3.6, just click on any application badge from the All Applications dashboard or click on Issues in the header and you’ll see the Issues Overview for that application. We’ve added some instructional prompts to the page so if there’s something you should be doing to make it more useful (like categorize your releases or use version numbers) it’ll let you know.
Thanks to everyone who gave us feedback – this is one example of new capability that wouldn’t be here without the open input we got from the community.