Archive for API
Philly Code Camp 2010.2 Presentations & Fun
Posted by: | Comments
Thanks to everyone for coming to my sessions and the organizers for making the event run so well. The facility was great and it’s really quite remarkable that the community can have such a strong one day training event without having to charge participants. Microsoft was there to help out, but it was clearly a training event for the community not a Microsoft press event, exactly as it should be.
We had fun talking with the other vendors at the show, most notably Component One and the folks from CapTech. This is the first time we’ve had a vendor booth at an event like this, so in many ways it was a dry run to get the hang of what it’s like. We gave away a full copy of Gibraltar Analyst as well as a year’s subscription to Hub at the conference, and got to talk with a lot of people about both Gibraltar and VistaDB. We got some great real world examples of where VistaDB fits, which is a big help as we work on the marketing for that going forward.
I presented two sessions -
A Year in the Life of an ISV
If you’re thinking about what it’d be like to ditch your corporate development job or consultant gig and strike to create & market your own product (Or you’re a consultancy looking to create a product to diversify) this presentation shows what to expect on the path from shipping your first version to business success.
This one’s always a little risky at a code camp because, well… there’s no code. But, with the incredible diversity of tracks that were available at Philly Code Camp (13 tracks, over 60 sessions…) I think it’s also good to be able to “take a break”. Next time I might go for the last session of the day to maximize the value of that.
Designing APIs for Others
I covered a range of real world lessons about commercial API development emphasizing the differences between in-house & internal development and great, reusable commercial libraries.
I got some great feedback on this talk, particularly on an example that broke my own rule about samples: I tried to over-simplify it and instead created a “not best practice” sample. I’ll fix that for next time!
If you saw either presentation, please be sure to fill out the conference evaluation and I’d love to hear your feedback – drop it in the comments below or send it to me directly at kendall.miller@gibraltarsoftware.com.
If you’d be interested in having us come talk at your code camp, .NET Users Group, or event – please reach out and let us know. We’re always looking for new & better ways to engage with the community.
Gibraltar 2.5 New Feature Dive – Issue Tracking Integration
Posted by: | CommentsQuick Takeaway
Gibraltar 2.5 includes built-in integration with two issue tracking systems and provides hooks for integrating with others. This integration extends Gibraltar to automatically create issues for new problems as they occur and make it easy to drill into all of the Gibraltar data related to any issue.
Check out our release notes for technical details or watch videos of how Gibraltar integrates with FogBugz (by Fog Creek) and Gemini (by CounterSoft) to get an overview of our integration.
Update: Check out the article on Closed Loop Software Development over at CounterSoft which talks about this integration. It’s a really easy way to get going, even if you’re a one developer shop!
The Full Scoop
While validating the new Gibraltar Add-In API we wanted to pick a real-world, hard problem that would push the edges of the integration capability. We decided to pick up an idea that had always been on our to do list: Integration with a defect tracking system. We built the original integration for our internal use with our in-house system. After working with that for a few weeks we were clear that this was a must-have feature that became a cornerstone of the Gibraltar 2.5 release.
Integrate with Your Processes
The primary goal of the issue tracking integration was to integrate Gibraltar and the data it collects into your existing workflow. We realized that no matter how great we might make tracking and managing errors within Gibraltar it wouldn’t be as good as just being able to use your existing workflow. Why? Because you would still need your existing system, so why have two separate workflows? Instead, you want to be able to look in your issue tracking system and say this is all there is without having to manually key and translate.
As a baseline, consider what would happen if you just configured email notification to flow from Gibraltar to your issue tracking system. This would create a new issue each time a qualifying event happened. In a production environment the number of issues would quickly get out of hand – one recurring problem and you’d be spending your time closing duplicate cases instead of working the important ones. Instead you need to be sure:
- Each new issues creates a new case
- New sessions with the same problem just add more information to the open case
- You have some way of ignoring errors that don’t represent a defect.
The Gibraltar integration delivers on this by scanning sessions as they come in and mapping errors into your issue tracking system. This integration ensures:
- Errors with a new unique signature will open new issues.
- Errors that match an existing signature will add additional information to the existing issue.
- If an error reoccurs in a new version after the related issue has been closed it will be reopened, unless you’ve set that issue to be ignored.
- Gibraltar sessions and issues will be related so you can easily see all of the sessions in Gibraltar for a particular issue.
All of this is possible because Gibraltar queries the issue tracking system while analyzing sessions to determine the right action to take.
Unexpectedly Awesome
When we first set this integration up ourselves we carefully sandboxed it off so it wouldn’t interfere with our existing approach to our CEIP. Several weeks in, the development team started pulling items out of the sandbox and into our support queue and defect backlog because they were spot on. Soon we were hooked – this was the way it should to be for everyone. We took our original implementation and included it in the release as a source code sample and set out to create in-the-box integration for some great systems.
This is one of those cases in software where the system is more valuable than the sum of its parts:
- Real user errors are accurately and completely conveyed into your issue tracking system without any manual effort.
- You manage issues the same way you always have, including whatever reporting you’re already doing.
- You see all of the Gibraltar data for an issue with just a few clicks, no searching for the data by looking at messages until you find the right one.
Gibraltar adds value and reinforces your investment in issue tracking without requiring you to change how you work.
Translating between Gibraltar and Issue Tracking
In Gibraltar each session is identified by a distinct Product, Application and Version, like Gibraltar Analyst 2.5.0.742 or Gibraltar Hub 2.1.0.594. These are usually very specific. On the other hand in your issue tracking system you usually have more general categorization, like Gibraltar Current Development for the active branch and Gibraltar Support for shipped versions. Not only are the two lists different, but you need enough mapping information to put new issues into the right workflow which may require selecting a priority, area, category, or whatever your issue tracking system requires.
Once you’ve enabled support for your issue tracking system you can configure how to map Gibraltar sessions into it from within Analyst. When a session is being analyzed it’s compared against the list of mappings until it matches one. To help minimize the amount of configuration necessary you can specify:
- Just a Product
- Just a Product & Application
- A Product, Application, and partial or complete version.
In the example at right, all sessions for Gibraltar Analyst with the major/minor version 2.2 would match. You can do more interesting wild card version matching to help fit your exact situation, see our documentation on configuring relationships.
The idea is to configure a session to get into the right initial state in your workflow. From there it should flow through just like if you keyed the defect yourself.
Details you Can’t Ignore
As more sessions come in they will be checked and the appropriate issues updated. A special case for this is when you’ve closed the related issue. Gibraltar has to decide whether to ignore the new data or reopen the issue. To work this out it focuses on the version of the new session:
- If it’s the same or older as the latest version previously seen on the issue then the new data is ignored.
- If it’s for a newer version the defect is reopened
The idea is that Gibraltar recognizes there’s a lag between when you declare an issue resolved and when that fix gets deployed to users. As long as it’s only occurring in the same set of versions it showed up in previously it’s still reasonable that the issue is closed. If it shows up in a newer version then it probably hasn’t been fixed and certainly should be rechecked.
Until you have to Ignore Them
In any system there’s a set of errors that aren’t defects. Temporary network connectivity issues, user errors, server timeouts… They are still errors but they don’t call for any action on your part. To handle this, Gibraltar supports marking an issue to be permanently ignored. The exact mechanism depends on the particulars of the issue tracking system, but it’s selected to make it natural and obvious.
When you first enable automatic integration these noise issues will get opened once, but all you have to do is mark them to be permanently ignored and they’ll never resurface again. Within a short period of time all of the common noise items will be suppressed and every new item will be gold.
Researching and Resolving Issues
Opening issues automatically is just one direction of integration. Once the issue exists you want to be able to see all of the related sessions in Gibraltar effortlessly. Within Analyst you can click on a dedicated node in the tree to see all of the open issues with Gibraltar sessions and drill down to the list of all related sessions.
Pick a session and you can open it or just preview all of the errors and warnings within it. In this view you can quickly spot trends in the sessions: perhaps they’re all in the same timezone or same operating system, or have another error that’s the true culprit.
Imagine that – you receive an email notification that there’s a new issue for you to look at. You double-click the Analyst icon on your system tray and Analyst pops up. Click on the issue tracking node and the most recent issue is on the top. In seconds you’re seeing the whole picture of the error: Related exception(s), actions they took leading up to the problem, performance and memory data. You’ll get the whole picture quickly.
The dedicated issue tracking display supports a number of built-in views that are designed to match each system’s typical grouping & sorting to help you find what you want faster.
Creating Issues by Hand
You don’t have to just rely on the automatic issue management feature: When you’re viewing a session you can select a range of messages and make a new issue from them on the fly.
Each of the messages you’ve selected are summarized with the last message in full detail. This lets you show events leading up to a problem and then the big problem itself.
Gibraltar will still search your issue tracking system to see if there’s an existing case matching the last message you selected so it can handle adding new data to an existing issue or creating a new one. It also checks your mapping configuration to pick the best defaults for the new case in your issue tracking system.
You can add whatever additional text you feel is useful to describe why you’re creating the issue and override some of the initial defect options like category or priority, depending on the system.
Create issues for suspicious items in our data – warnings, or just messages that don’t make sense. Regardless of how you create the issue the rest of the integration works the same.
Work as a Team
Issue tracking systems are fundamentally about tracking & sharing knowledge on the state of your environment with a team of people. Gibraltar fits right into this by:
- Sharing configuration with your whole team: Only one person needs to go in and update the mapping setup when you deploy a new version or application and the whole team is kept current.
- Centralizing Analysis: You can have the Hub perform all of the session analysis so no one need run Analyst for you to get new issues created & updated.
You can perform all of the configuration within Analyst and it flows back and forth to the Hub automatically, no need to get IT involved on the server when you need to adjust something.
An Army of One
If you’re using just Analyst stand alone you can still take advantage of everything we’ve covered – you just need to leave Analyst running to automatically have sessions analyzed (otherwise they’ll wait for you to run it the next time). Fortunately we’ve made that a pretty good experience in this release.
Get Started Today
We’ve designed the integration to work without requiring anything be installed into your issue tracking system: Just enable the integration feature and go. You can get it started with just one member of your team and see how it goes, or charge in full-bore by configuring automatic integration with the Hub and sharing the configuration with your whole team. Download Gibraltar and get started! If you don’t have a defect tracking system, FogBugz will provide a free hosted instance for 1 or 2 users and Gemini offers a free license for up to three users.
Check out our release notes for technical details or watch videos of how Gibraltar integrates with FogBugz (by Fog Creek) and Gemini (by CounterSoft) to get an overview of our integration.
Reworking the Gibraltar AddIn API
Posted by: | Comments
There’s a classic saying in designing software that you should plan on throwing the first one away. I’ve never been very comfortable with that because it can be easily abused by a team to justify half-baked thinking, inadequate requirements and design, and just general hope instead of solid engineering.
That said, it’s generally been my experience that no matter how good and diligent you are when designing an API you’ll inevitably find that you’ve missed the mark as soon as the first set of users start solving real problems with it. So, you’ll want to leave room to redo and rethink your original approach before you’re married to it. This is the major reason that we shipped Gibraltar 2.2.0 as a Beta back in early April with our first vision of a Gibraltar Add In API.
Feedback from inside & outside
To make sure we got solid evidence that the design was sound we worked hard to get some existing customers as well as outsiders to try solving their challenges with the API. We also did our own internal Red Team approach: we asked Jay (who isn’t doing any day-to-day development on Gibraltar, but has extensive commercial development experience) to scratch his favorite itch with the API as well. We’re going to publish more about what he built soon, but without going into details something significant emerged from his work and discussions with our customers.
Instead of people using the API to solve a few tactical problems (like data export or an alternate view of the data) people wanted to use it to extensively integrate Gibraltar with external systems. This went way beyond our original intent; we have a big block of features on our roadmap for creating an external integration engine and hadn’t ever intended this to be the way that came about. Despite that, it’s the problem people kept wanting to solve with the API.
Unfortunately, there are several things that we just didn’t allow for, but are common problems in these scenarios:
- Configuration Management: Whenever you’re working between two systems, you’ll want some metadata to describe endpoints and map data between the systems.
- Credential Management: Nobody likes hardcoding credentials, or doing something simple like popping them in a file without encryption. We all know the right things to do, but it’s annoying and overhead. It should be built in.
- Coordination: All the various user interface and process extensions don’t exist in a vacuum; they need easy ways to coordinate their work to enable new processes, not just isolated features.
- More Extensibility: People want to be able to extend other parts of the UI again with the goal of connecting processes together.
- Development Debugging: In a twist of irony, it’s hard to log and view data from within an Add in because you’re running within the Gibraltar environment; you can’t use the Agent to log because you’re running inside of the Agent. This made development of extensions a pain.
Powerful capabilities, not demoware
As we’ve said before, we hate demoware. Although we’ve heard from a few parties that we just need to “check the feature box” with the API (have it but no one will use it), we just aren’t willing to accept that measure for success. Instead, we see another angle: We’re going to leverage the add in API ourself to enable us to create targeted features that only apply to some part of our audience so we can create them faster and keep control over the product’s complexity. In short, we’ll make sure that it’ll solve problems for you by having it solve problems for us.
Fortunately, we didn’t have to throw the first one away in a literal sense. We did end up discarding virtually our entire implementation of it from the Gibraltar side, but the change to the API you see is fairly modest.
We’re still finalizing the details, and deciding whether we need to do another beta round of 2.2 or should go ahead and formally release 2.2 so that everyone can get the benefit of all of the improvements that have happened over the past several months beyond the API. But, we can speak to a few great features:
Configuration Management
Configuration data can be shared between the Hub and all of its connected Analysts. This makes it more practical for add ins that connect systems together but need everyone on the team to share metadata on how items in both systems relate. Since add ins can run on both the Hub and in the Analyst we made it so you can change the Hub configuration for an add in remotely. This means you don’t have to run to IT and get on the server to say add new mappings for new projects or do whatever other configuration your favorite add in requires. The central data is automatically synchronized between the Hub and Analyst and is available even if the Hub isn’t (say because you’re running disconnected from the network)
For example, say you want to integrate with your defect tracking system. It’ll have some form of heirarchy for managing things – probably with different areas for the current development release vs. the production releases, etc. You need to be able to relate the data Gibraltar has (like product and application names as well as versions) to those areas so when it opens or updates an issue it shows up in the right place and in the right workflow.
Credential Management
In a perfect world, your Windows credentials would give you access to any external system your Add In needs to access, so no credentials need to be stored. Unfortunately, most of the third party systems you’ll want to integrate with (particularly ones that aren’t designed for enterprises) just flat won’t integrate that way. You need to be able to store a user id and password but do so in a way that doesn’t create an unreasonable security risk.
We’ve baked in credential management leveraging the data protection API in Windows (DPAPI) so that you can encrypt account names and passwords so they can be accessed only by the current user, computer, or the hub. Here again you can manage it all from Analyst, so if you need to change the account the Hub is using to login into your defect tracking system or something you don’t have to go to the server to do it.
Development Debugging
We’re taking advantage of improvements in the Live Log Viewer control to offer a real-time view of what’s happening with Add Ins, both the Gibraltar side and the messages logged by each Add In. We log a lot of information as Analyst runs, and we don’t want to bury you in all of that detail so it automatically filters down to just information needed to monitor and support Add Ins.
This is done using only capabilities that are exposed through our normal API, so you can do the same thing in your WinForms/WPF application as well. You’ll notice we’ve updated the live viewer to include a number of capabilities formerly only available in Analyst, like source code lookup and exception / detail viewing.
More Extensibility
We’ve added a number of places you can extend Gibraltar to the original list:
- Analyze Sessions on Hub: Any analysis engine you create can run in Analyst or Hub. We always intended it, but it wasn’t ready for the beta. This means you can have data flowing from Gibraltar to external systems without anyone being logged in or having to leave something running.
- Global Commands: You can extend the main Analyst menu bar with your own menu items to create global commands end users can easily access.
- Controllers: We’ve added some key controller interfaces so you can hold state in the right places (think MVC) and substantially improve the efficiency of integrating with remote systems that may be expensive to query.
- And More: Frankly, we’re going to continue adding items as we head towards release based on our internal experimentation of what we can safely get done and commit to. Our ultimate goal is that you can extend each of the different views in Analyst so you can have it seamlessly participate in end-user workflow.
Real World Experience
We’ve been internally using the new API to help us automate our CEIP process for Gibraltar itself and it’s given us a lot of confidence not just in the new implementation but also in the vision it represents. We’re going to be able to ship extensions outside of our release cycle, add optional modules in the box for a great initial experience, and help create a space for you to show your peers the cool things you can do where Gibraltar + Your System = Awesome.
Is there a specific system you’d like to see us integrate with? Drop us a line; we love to engage with our peers to understand new & creative ways to make a difference in how developers create rock solid .NET software.
