For version 5 we had three key performance goals:
- Eliminate performance outliers: Historically there were scenarios where small differences in queries could produce orders of magnitude worse performance. We wanted the engine to be much more predictable to lower the testing burden.
- Improve ORM performance: With the increasing use of ORM tools like Entity Framework it’s less and less feasible to hand-optimize queries. These tools are often designed to feed SQL Server and in some cases provoked very bad performance from the engine.
- Improve overall performance by 30%: We picked a number out of the air; we wanted to on average improve the performance of non-trivial queries (those that take more than 50ms) by 30% over 4.3. This would in general make it substantially faster than 4.0 (because of the extensive gains in 4.3).
In the first preview we focused on the first item, preview two added features to help with the second. For preview 3 we’ve been focusing on the third goal: Make everything faster, and do it without introducing new performance outliers.
Since the execution of VistaDB has been profiled and optimized for years we didn’t expect to get many gains through that approach. Instead, we were going to have to redo the algorithms used for performing work instead of just focusing on improving the implementation of the existing algorithms. We introduced an entirely new query optimizer in Preview 2 and then added a secondary optimizer for preview 3 that handles queries the first approach can’t.
This is the Build to Try
Preview 3 includes all of the performance optimizations we’re expecting for VistaDB 5.0. It’s also the last preview that will be file binary compatible with VistaDB 4. So, this is the perfect version for you to try out: If your application works well with Preview 3, it’s going to work well with 5.0 final. Alternately, if you can come up with a query that doesn’t run well in Preview 3 you should let us know so we can correct the optimizer for 5.0 final. Email your torture query to support along with a sample database and we’ll add it to our test suite.
When we say this build is binary compatible with VistaDB4 this means it reads & writes the same files and respects the same file locking semantics so you can simultaneously access the same database file using 4.3 and this preview build.
What’s Left for 5.0?
We’ve been saving up all of the breaking changes for our next phase of development. The reason for this was to maximize the ability for folks to try out the new query optimizer without having to do a one-way database conversion. The notable items we’re moving on to now are:
- New Data Types: We’re adding DateTime2 and DateTimeOffset and some other smaller details for data types. This requires we change the file format.
- New Encryption: We’re replacing the built-in managed implementation of Blowfish with a FIPS-compliant implementation of AES from the .NET runtime. This will only be available for .NET 4 clients under Windows (not MONO). For MONO we’ll have an alternate approach.
We’re also going to take this opportunity to make some other minor detailed file changes to improve crash recovery and locking. Since the format is changing once 5.0 ships you will have to do a one-way file conversion from the VistaDB 4 format to the VistaDB 5 format. This will work the same way that the VistaDB 3 to VistaDB 4 conversion is done today – you just pack the database file.
The final piece of the puzzle will be some compatibility updates to the Visual Studio extension so it works with VistaDB 5. If you’re trying Preview 3 the Visual Studio extension will not be installed and wont interfere with VistaDB 4.
While picking dates is quite difficult with a product like a database engine (since it can’t ship until we’re sure all of the technical implementation details are right) we’re October for VistaDB 5 final.
So take the plunge today and try out VistaDB 5!