Archive for Architecture
Courageous Session on Extensible Software from CodeMash
Posted by: | Comments
I just got back from CodeMash in Ohio which was a really great conference – not only did I get to meet a number of friends from the .NET community but also saw sessions from pretty much every other technical persuasion. Of the sessions I saw though, the most courageous one was by Alex Papadimoulis from Inedo (perhaps better known for the Daily WTF).
The cornerstone of his presentation is to really challenge yourself when you make any software implementation harder than it absolutely has to be in the name of future requirements, changes or extensibility. I’ve noticed over the past few years that making applications dynamically extensible has gone from a rare and cool capability to a presumed requirement even for the simplest applications. We plow in with IoC containers, elaborate software patterns, and large scaffolding all in the name of being ready for future change.
After all, who wouldn’t like to show years of maintenance programmers that you had a full understanding of both Generics and the command pattern:
orderController.ShipOrder(
ShipOrderCommandFactory.Create(order.OrderNumber, carrierId,
shippedDate));
Instead of just being really boring:
orderController.ShipOrder(order.OrderNumber, carrierId, shippedDate);
Instead of doing the simplest code approach that will deliver functionality today, we worry about how we can account today for the potential change for tomorrow: What if we want to support multiple, incompatible databases? What if we want to add more commands later? What if we decide to switch from WinForms to WPF?
You Aren’t a Fortune Teller
The thing is, in every one of those cases:
- You’re going to ship a new release. So you’ll have an opportunity to change the code in the face of the actual requirements, not theoretical ones.
- You’re going to be wrong about what the future requirements are. It’ll turn out that the new database is incompatible in a way you didn’t expect (what? No transactions? uh oh…) so you’ll have to change your model anyway. Only now you’ll be extra paranoid and REALLY gold plate it.
- You’ll have to retest anyway. For any worthwhile change you’ll have to retest that the whole thing works together anyway.
So save yourself now: Really challenge why you need to make everything an interface, or if it’s really necessary to use the command pattern to isolate every operation you might perform. It could be you’re making Visual Studio and you really have to create an extensibility framework that’s just about infinitely flexible. Or it could be you can get away with something much simpler. Perhaps you really don’t need any of that at all.
In the end, remember that your ultimate users don’t care how you deliver the features in your application, they just care that they work. Conversely, you’ll get no points for delivering a poor or fragile experience because you’ve made everything isolated and configuration driven.
I really admire Alex for getting up in front of a crowd that’s been hearing two days of how to create ever more elaborate isolation layers between every aspect of software and challenge the foundational premise of it all. Kudos!
Philly Code Camp 2010.2 Presentations & Fun
Posted by: | Comments
Thanks to everyone for coming to my sessions and the organizers for making the event run so well. The facility was great and it’s really quite remarkable that the community can have such a strong one day training event without having to charge participants. Microsoft was there to help out, but it was clearly a training event for the community not a Microsoft press event, exactly as it should be.
We had fun talking with the other vendors at the show, most notably Component One and the folks from CapTech. This is the first time we’ve had a vendor booth at an event like this, so in many ways it was a dry run to get the hang of what it’s like. We gave away a full copy of Gibraltar Analyst as well as a year’s subscription to Hub at the conference, and got to talk with a lot of people about both Gibraltar and VistaDB. We got some great real world examples of where VistaDB fits, which is a big help as we work on the marketing for that going forward.
I presented two sessions -
A Year in the Life of an ISV
If you’re thinking about what it’d be like to ditch your corporate development job or consultant gig and strike to create & market your own product (Or you’re a consultancy looking to create a product to diversify) this presentation shows what to expect on the path from shipping your first version to business success.
This one’s always a little risky at a code camp because, well… there’s no code. But, with the incredible diversity of tracks that were available at Philly Code Camp (13 tracks, over 60 sessions…) I think it’s also good to be able to “take a break”. Next time I might go for the last session of the day to maximize the value of that.
Designing APIs for Others
I covered a range of real world lessons about commercial API development emphasizing the differences between in-house & internal development and great, reusable commercial libraries.
I got some great feedback on this talk, particularly on an example that broke my own rule about samples: I tried to over-simplify it and instead created a “not best practice” sample. I’ll fix that for next time!
If you saw either presentation, please be sure to fill out the conference evaluation and I’d love to hear your feedback – drop it in the comments below or send it to me directly at kendall.miller@gibraltarsoftware.com.
If you’d be interested in having us come talk at your code camp, .NET Users Group, or event – please reach out and let us know. We’re always looking for new & better ways to engage with the community.
Reworking the Gibraltar AddIn API
Posted by: | Comments
There’s a classic saying in designing software that you should plan on throwing the first one away. I’ve never been very comfortable with that because it can be easily abused by a team to justify half-baked thinking, inadequate requirements and design, and just general hope instead of solid engineering.
That said, it’s generally been my experience that no matter how good and diligent you are when designing an API you’ll inevitably find that you’ve missed the mark as soon as the first set of users start solving real problems with it. So, you’ll want to leave room to redo and rethink your original approach before you’re married to it. This is the major reason that we shipped Gibraltar 2.2.0 as a Beta back in early April with our first vision of a Gibraltar Add In API.
Feedback from inside & outside
To make sure we got solid evidence that the design was sound we worked hard to get some existing customers as well as outsiders to try solving their challenges with the API. We also did our own internal Red Team approach: we asked Jay (who isn’t doing any day-to-day development on Gibraltar, but has extensive commercial development experience) to scratch his favorite itch with the API as well. We’re going to publish more about what he built soon, but without going into details something significant emerged from his work and discussions with our customers.
Instead of people using the API to solve a few tactical problems (like data export or an alternate view of the data) people wanted to use it to extensively integrate Gibraltar with external systems. This went way beyond our original intent; we have a big block of features on our roadmap for creating an external integration engine and hadn’t ever intended this to be the way that came about. Despite that, it’s the problem people kept wanting to solve with the API.
Unfortunately, there are several things that we just didn’t allow for, but are common problems in these scenarios:
- Configuration Management: Whenever you’re working between two systems, you’ll want some metadata to describe endpoints and map data between the systems.
- Credential Management: Nobody likes hardcoding credentials, or doing something simple like popping them in a file without encryption. We all know the right things to do, but it’s annoying and overhead. It should be built in.
- Coordination: All the various user interface and process extensions don’t exist in a vacuum; they need easy ways to coordinate their work to enable new processes, not just isolated features.
- More Extensibility: People want to be able to extend other parts of the UI again with the goal of connecting processes together.
- Development Debugging: In a twist of irony, it’s hard to log and view data from within an Add in because you’re running within the Gibraltar environment; you can’t use the Agent to log because you’re running inside of the Agent. This made development of extensions a pain.
Powerful capabilities, not demoware
As we’ve said before, we hate demoware. Although we’ve heard from a few parties that we just need to “check the feature box” with the API (have it but no one will use it), we just aren’t willing to accept that measure for success. Instead, we see another angle: We’re going to leverage the add in API ourself to enable us to create targeted features that only apply to some part of our audience so we can create them faster and keep control over the product’s complexity. In short, we’ll make sure that it’ll solve problems for you by having it solve problems for us.
Fortunately, we didn’t have to throw the first one away in a literal sense. We did end up discarding virtually our entire implementation of it from the Gibraltar side, but the change to the API you see is fairly modest.
We’re still finalizing the details, and deciding whether we need to do another beta round of 2.2 or should go ahead and formally release 2.2 so that everyone can get the benefit of all of the improvements that have happened over the past several months beyond the API. But, we can speak to a few great features:
Configuration Management
Configuration data can be shared between the Hub and all of its connected Analysts. This makes it more practical for add ins that connect systems together but need everyone on the team to share metadata on how items in both systems relate. Since add ins can run on both the Hub and in the Analyst we made it so you can change the Hub configuration for an add in remotely. This means you don’t have to run to IT and get on the server to say add new mappings for new projects or do whatever other configuration your favorite add in requires. The central data is automatically synchronized between the Hub and Analyst and is available even if the Hub isn’t (say because you’re running disconnected from the network)
For example, say you want to integrate with your defect tracking system. It’ll have some form of heirarchy for managing things – probably with different areas for the current development release vs. the production releases, etc. You need to be able to relate the data Gibraltar has (like product and application names as well as versions) to those areas so when it opens or updates an issue it shows up in the right place and in the right workflow.
Credential Management
In a perfect world, your Windows credentials would give you access to any external system your Add In needs to access, so no credentials need to be stored. Unfortunately, most of the third party systems you’ll want to integrate with (particularly ones that aren’t designed for enterprises) just flat won’t integrate that way. You need to be able to store a user id and password but do so in a way that doesn’t create an unreasonable security risk.
We’ve baked in credential management leveraging the data protection API in Windows (DPAPI) so that you can encrypt account names and passwords so they can be accessed only by the current user, computer, or the hub. Here again you can manage it all from Analyst, so if you need to change the account the Hub is using to login into your defect tracking system or something you don’t have to go to the server to do it.
Development Debugging
We’re taking advantage of improvements in the Live Log Viewer control to offer a real-time view of what’s happening with Add Ins, both the Gibraltar side and the messages logged by each Add In. We log a lot of information as Analyst runs, and we don’t want to bury you in all of that detail so it automatically filters down to just information needed to monitor and support Add Ins.
This is done using only capabilities that are exposed through our normal API, so you can do the same thing in your WinForms/WPF application as well. You’ll notice we’ve updated the live viewer to include a number of capabilities formerly only available in Analyst, like source code lookup and exception / detail viewing.
More Extensibility
We’ve added a number of places you can extend Gibraltar to the original list:
- Analyze Sessions on Hub: Any analysis engine you create can run in Analyst or Hub. We always intended it, but it wasn’t ready for the beta. This means you can have data flowing from Gibraltar to external systems without anyone being logged in or having to leave something running.
- Global Commands: You can extend the main Analyst menu bar with your own menu items to create global commands end users can easily access.
- Controllers: We’ve added some key controller interfaces so you can hold state in the right places (think MVC) and substantially improve the efficiency of integrating with remote systems that may be expensive to query.
- And More: Frankly, we’re going to continue adding items as we head towards release based on our internal experimentation of what we can safely get done and commit to. Our ultimate goal is that you can extend each of the different views in Analyst so you can have it seamlessly participate in end-user workflow.
Real World Experience
We’ve been internally using the new API to help us automate our CEIP process for Gibraltar itself and it’s given us a lot of confidence not just in the new implementation but also in the vision it represents. We’re going to be able to ship extensions outside of our release cycle, add optional modules in the box for a great initial experience, and help create a space for you to show your peers the cool things you can do where Gibraltar + Your System = Awesome.
Is there a specific system you’d like to see us integrate with? Drop us a line; we love to engage with our peers to understand new & creative ways to make a difference in how developers create rock solid .NET software.
Beware Demoware
Posted by: | CommentsI’m becoming convinced that there is a scourge in the Microsoft development ecosystem, and it’s called the 10 minute demo. Here’s the problem: No matter what the tool is, you have to make a demo that, in 5-10 minutes, purely using point & click design tools, shows off the power of the tool and technology.
Now, the problem isn’t that you have to get your message across in 5 or 10 minutes; I’ve found that the harder I work at getting the message down the better the message gets. Instead, I wonder how many product managers have had to sit back and say “fine fine, but enough about what customers are telling us it really has to do: how am I going to make a point and click demo that will go from user interface to database through any number of tiers and makes it all look easy?” The problems that we really want vendors to tackle are hard: If they were easy, we wouldn’t be looking for solutions.
Take Entity Framework. This is a very complicated set of technology that’s oriented at a very complicated problem. If you have the problem they’re solving, it can be a huge time and cost saver. Developing a technology like this is extraordinarily expensive, and you’ll be married to it for a long time so it’s good to have a major vendor stand behind it. I’ve been in environments where this would have cut major projects in half and eliminated piles of risk. I can imagine the challenge the Product Manager for Entity Framework had when we had to make a few short demos that show off how great this technology is.
This is a high-concept technology for integrating disparate data sources into a common model. Any demo you can do purely visually in 5 minutes of the product is going to be contrived at best. Most of the demos I’ve seen have been how you can take Entity Framework and point it at your schema and then get your client to bind to it. You know, the same thing you could do with ADO.NET, LINQ to SQL, and any number of other technologies… And shouldn’t do. It does a disservice to the tool because:
- You end up walking away with an underwhelming impression of what the technology is uniquely enabling; something that you couldn’t easily do before.
- The demo shows an approach that really doesn’t scale to real-world problems.
The last one is the real problem. Folks who are approaching using your technology to solve real world problems take the methods you use in your demo very seriously. They’ll pick it apart and use it as a trump card in design conversations: ”but here’s how they did it in the demo, just strait from that little part of the UI back to the database. So we make our huge application by just doing that over and over. It’s how they did it!”. I think of all those early demos of .NET that showed people dragging a few tables onto a designer, then dragging that onto a UI, click click click and *poof*, there’s the list of customers! But seriously, would you really build anything of scale that way? Unlikely – maintenance challenges, performance, production considerations (like common error management and security) all argue against it.
As an experienced team of developers, even we get caught by this some times – we look at the buzz and a demo around a visual design tool like LINQ to SQL and then charge out to try it on a skunkworks project. Within a short period of time we’re in trouble with it – everything started well but now we can’t update the model, or we keep losing our changes, or we can’t get it to handle the full application lifecycle. We’ll talk back and forth trying to decide if:
- We just don’t get it. We’re just too old and set in our ways, and can’t see the great brilliance of the new vision.
- The demo approach is demoware. A feature you’d only use in a demo.
In the end, we nearly always prove it’s the second. We pick it apart, talk with other (non Microsoft evangelists) folks and come to the consensus that yes indeed the natural laws of computing haven’t been set aside by this new strategy or tool.
So please, I’m begging you – if you work for Microsoft or in the Microsoft ecosystem, put an end to demoware. Your product may not make a 5 minute end to end sample – so show us how it fits conceptually. I’ll get more out of your 10 minute video on Scrum that’s funny and conceptual than I will on a contrived sample that just shows that your product can solve a problem too simple to justify its use.
A final note: I’m not really singling out Entity Framework as having bad demo videos. There’s a lot that I disagree with about how EF was rolled out (A service pack? Really? How about .NET 3.9…). They actually did create a set of conceptual videos, but the high traffic stuff in MSDN has a definite feeling of demoware.
