Up-front software design can make development more agile

Although the agile manifesto promotes the idea that “Continuous attention to technical excellence and good design enhances agility”, there’s been a movement in agile project management away from up-front software design. While it's generally accepted that requirements will change during most projects, and therefore big design up front no longer makes sense, the other extreme is not desirable either. No up-front design can result in a mess that reduces agility, a mess that is increasingly expensive to correct as the project progresses and often hidden from the business. It can happen even when developing within frameworks like Ruby on Rails, ASP.NET, J2EE, and Django that provide some design ‘out of the box’. Even when using code refactoring tools. Designing up-front for change, on the other hand, can be lean and help maintain agility, even as the code grows.

Designing for change includes identifying units of functionality, considering which are more likely to change based on competitive environment or end user needs, then isolating them into logical parts in such a way that those less likely to change are not dependent on those that are as much as possible. For those who want to get a glimpse of how this looks under the hood, this presentation illustrates some common examples of software designs that improve agility.


Frameworks that provide design structure take care of some of this, but not all. Applying design practices like identifying, naming, and capturing the properties (as much as possible) of  the units of data in an application up-front, for example, can return dividends from day one that further increase with time. Not doing so can easily result in data units that are ambiguously named, vary by developer, and even overlap with one another. It's not hard to imagine that after even a few thousand lines of code the time cost to change these units goes up and from there just keeps rising. After all, even with software tools that make changing databases easy, there's still all the surrounding code that has to be dealt with.

Some up-front design really can go a long way to improve agility and there are sensible, lean ways to go about it. Since doing so isn't popular at the moment, however, it may be something you want to make sure is happening on your projects.

What's your experience with adaptable designs and maintaining agility as your software grows? What's your team doing to avoid a mess? Please share your thoughts and experiences in the comments below.

The mobile dream: write once, run everywhere

It's come up before. Someone has a cross-platform mobile app written in HTML5 (HTML version 5 with CSS and JavaScript) and they're thinking of porting it to native to improve performance. But native can be very costly and HTML is a ubiquitous, cross-platform technology already baked into most (all?) smart devices (not to mention the web itself). HTML is also arguably one of the more productive user interface development technologies around, having had time to evolve since the inception of the web. And yet those pesky performance issues with HTML5 crop up on mobile devices. Hesitant button toggles. Flickering transitions. Noticeable screen redraws. Neither costly development or poor performance are very satisfying choices.

There are a variety of alternatives, each with benefits and drawbacks. In this post I'll attempt to survey the major options and end up coming virtually full circle to a practical approach I recommend considering: keep as much in HTML5 as possible and then optimize by pulling just the problems into native. The trick here is pulling exactly the problems into native: unless your problem areas exactly match what commercial native shells happen to provide (i.e. Phonegap, Trigger.io, etc.) it's probably easiest to provide the native shell yourself through some native coding.

But first, what are the major alternatives?

HTML5 packaged in native shell

Phonegap / Cordova

 

With this technology, an app is primarily written in HTML5 then bundled into a wide variety of native packages. Hardware services are provided through plugins and offered to the bundled HTML5 through a bridging API. App developers can add new plugins, which are developed separately in native code for each platform. Basic Phonegap is free and supports all major mobile platforms.

Phonegap apps are known as having performance challenges, some from the hosted HTML5 and others from the communication between HTML5 and native code. While it should be possible for Phonegap to improve HTML5 performance by using Apple's WKWebView in iOS 8, and it has been possible for Phonegap to improve its bridging communication for quite some time, it's mandate to serve the 'widest possible audience' means these potential improvements are still not available. It's tough trying to be everything to everyone without becoming barely average to everybody.

Trigger.io

 

Similar to Phonegap, much of a trigger.io app is written in HTML5 and bundled into native deployment packages, with native device capabilities like the camera extended to HTML5 though a bridge. Trigger.io also, however, exposes native UI components to the bundled HTML5 code through their bridge enabling hybrid user interfaces. Trigger.io claims their native to HTML bridge technology is up to 5x faster than Phonegap on Android. Trigger.io also supports writing custom native plugins, similar to Phonegap, which it calls modules. Trigger.io is offered through a monthly subscription and appears to only support iOS and Android. Trigger.io clearly aims to fill in where Phonegap falls short.

As with Phonegap, its value probably comes down to whether your performance issues happen to match what it can provide natively -- and how uncomfortable you are taking on a little native development.

RhoMobile Suite

 

RhoMobile apps are written in a combination of HTML5 and Ruby. Most views are written in HTML5 and rendered with the embedded browser component as with Phonegap, with the addition of controllers written in Ruby that can provide native functionality including native tab bars, navigation bars, map views, date pickers, and others. On BlackBerry, RhoMobile apps are converted to and run as Java bytecode. On iOS, Android, and Windows Phone, apps are compiled down to and run as Ruby 1.9 bytecode. RhoMobile has a free option on up to monthly per developer subscription price.

 

Interpreted

 

Appaccelerator Titanium

 

Unlike those described thus far, Titanium apps do not use an internal browser component. Titanium instead uses JavaScript, run in a JavaScript interpreter, to control native user interface elements through platform-independent JavaScript APIs. Since it can be a little hard to figure out how this works, here and here are two articles I found particularly informative. Titanium is open source and can build apps for iOS, Android, Blackberry, Tizen, and Windows Phone.

As with any fairly large body of technology (probably all included here!), issues have been noted with Titanium. It may be most suitable for prototyping as performance issues, difficulties in debugging, and memory issues have been noted.

Corona

 

Corona seems to be focused on the development of mobile games, although it can also build other types of apps. Apps in Corona are written in LUA and are probably interpreted by a proprietary interpreter written in C. Corona can produce apps for iOS, Android, Kindle, and Windows Phone and is priced on a per-developer, subscription basis.

Compiled to assembly 

 

Xamarin

 

Xamarin came out of work done on Mono, an open source .NET implementation. Xamarin apps are written in C# and can be deployed to iOS, Android, and Windows. For iOS, Xamarin produces a completely native app by compiling C# to ARM processor assembly. For Android, C# is compiled to .NET IL then packaged with the monoVM and JIT. Xamarin can produce apps for iOS, Android, and Windows Phone.

Not to pick on Xamarin but since I've heard it promoted heavily as a way to write once run everywhere I suggest reading this article if you're considering Xamarin for that reason.

Embarcadero Firemonkey

 

Firemonkey apps are written in Delphi or C++ and are compiled directly to processor assembly for both the iOS and Android mobile platforms, completely replacing native compilation. Firemonkey tools are priced in the $1000s per user range.

HTML5 (..and can be packaged in Cordova)

 

Another approach to improving HTML5 performance is to optimize HTML5 itself. This approach is, of course, limited in effectiveness by the performance of the browser running the HTML5.

HTML5 running in Safari or Chrome already benefits from improvements to both, and those benefits have been recently extended to the embedded browsers native iOS and Android code can host. HTML5 may not be able to fully compete performance-wise with native code yet, but clearly vendors are putting investment in this area and arguably the gap is closing.

A few HTML5 libraries of note, each of which can be natively packaged in Cordova, include:

Ionic

 

This mobile JavaScript library is focused on performance and features 'no jQuery'. Ionic JavaScript is structured MVC using AngularJS.

Sencha Touch

 

Sencha Touch is known for its good native look and feel and is also clearly sensitive about performance. It's also known to be a bit of a learning curve. It The challenge with Sencha, in my opinion, is that its focus on native look and feel suggests it may be a poor fit when using HTML5 as the cross-platform portion of an app.

jQuery Mobile

 

This library produces a native-independent look and feel, what you're usually looking for for the cross-platform potion of apps. It's fairly easy to pick up and work with in my opinion, but of course jQuery's focus on DOM manipulation doesn't result in well-structured code on its own. I found it performed adequately even in older Android browsers and agree performance issues others have complained about may be from some other cause.

Telerik Mobile

 

Frankly, I found it a real challenge to figure out what this thing is or does (can you figure out what all this marketing bling actually means?), but from what I can tell they provide the following mobile development technologies (in addition to other non-mobile ones):

• A JavaScript library (with tools) to help produce HTML5 apps that adapt to each native look and feel 
• Tools to package, manage, test, and analyze HTML5 apps packaged in a native Phonegap/Cordova shell.

 

Back to the beginning

 

Whew. A lot of different technologies to make the same code run on different devices, and there's even more than I mentioned. But do any of the alternatives really save on cost (time, money, headache) over what the mobile platforms themselves already provide?

I believe native will always have the most complete set of features, best debugging, best support, best compatibility -- you name it. It will be where you have to go in a pinch. And even the fiercest mobile competitors are taking HTML5 seriously so it's hard for me to recommend ignoring it.

In contrast, none of the alternatives have the same mind share, not even close, which adds to their long-term costs. How easy will it be to get bugs fixed when you run into them? How hard will it be to work around something when you need it done a little differently? How long will it take to learn? Frankly, how long will it continue to be a viable and supported product? A lot changes in a few years.

So, if you have a cross-platform mobile app written in HTML5 and need to improve performance, consider pulling the performance issues into native before you pull the plug and either go all native or with a completely different alternative. If the existing technologies that package HTML in a native shell don't let you move performance issues to native exactly where you need it, consider rolling your own. Doing so isn't hard if you're handy with a little native development -- certainly not harder than tackling a whole new development platform -- and it enables you to use both native and HTML's strengths when and where you need them.

Tell me what you think. It's a big topic and of course there aren't really any simple answers.

A bottle of DB reliever

I don’t know about you, but trying to decide what technologies to use is beginning to feel like trying to choose a pain reliever: countless choices and a ton of small print. But when you think about it, it’s actually worse. Unlike pain reliever brands that tend to stick around year after year, many technologies don’t.

So in a new world liberated from the tyranny of expensive database technologies but buried in choices that will live or die depending on where the tech fashion winds blow, which do you choose?

I found this site particularly helpful for getting the big picture because it not only provides a survey of what’s out there but categorizes and ranks them based on popularity. This way I can survey the different categories of database engines, learn more about the players, and see how they’re popularity is trending.

Then I only have to dig into the small print on a few bottles before heading over to toothpaste...

Posting files with AngularJS

While AngularJS's built-in $http service posts data as JSON by default, what do you do if you need to include an image or other binary data file in the post?

Seems pretty straightforward. Just bind HTML <input type="file"> to a controller variable via ng-model and $http.post() a FormData object containing the file object as data with Content-Type set to multipart/form-data. Right?

Not so fast. It turns out that 

•  <input type="file"> doesn't work with ng-model, and 
• Setting Content-Type to multipart/form-data resulted in rejected posts (a bug?).

Thankfully Angular is thoughtfully extensible. The first issue can be resolved with a custom Angular directive that tells $compile to hook <input type="file"> change events and set a controller scope variable with the selected file object when fired as so:

 .directive('inputfileModel', ['$parse', function ($parse) {  
   return {  
     link: function(scope, element, attrs) {  
       var model = $parse(attrs.inputfileModel);  
       var modelAssigner = model.assign;  
       element.bind('change', function(){  
         scope.$apply(function(){  
           modelAssigner(scope, element[0].files[0]);  
         });  
       });  
     }  
   };  
 }]);  

This directive is then set on the element as a custom element attribute:

  <input type="file" inputfile-model="fileVar">  

The second issue is a little trickier to work around. After doing some digging, it turns out that setting Content-Type to undefined results in browsers both setting  Content-Type to multipart/form-data and filling in the correct boundaries. O-Kay, not sure why that is, but hey, it works. Note that $http.post()'s default behavior of transforming data to JSON also needs to be turned off, which can be done by setting it to Angular identity function if you're into functional style programming.

 .factory('Messages', ['$http', function ($http) {  
      return {  
           put: function(url, file, dataObj, success, error) {  
                var fd = new FormData();  
                fd.append('json', JSON.stringify(dataObj));  
                fd.append('file', file);  
                $http.post(url, fd, {  
                     transformRequest: angular.identity,  
                     headers: {'Content-Type': undefined}  
                })  
                .success(success)  
                .error(error);  
           }  
      }  
 }]);  

A few problems initially, but it wasn't too hard to modify and using frameworks like Angular to add some structure to JavaScript in all its expressive glory can only help ease the pain of the poor dude down the road who will have to maintain it.

Get the Gist: UML

So you have a picture in your head. You can see it. From a variety of angles. What it looks like laying flat, how it stands, how it will work when it becomes real.

Of course, things we build start out as pictures in our minds and in many cases just sketching out some lines and words is enough to work out the dimensions, the layout, how it will look. Enough to show someone else what it will look like.

Software can get pretty complex, though. Will the developer across the world understand what your squiggle means? Will the guy who has to maintain the software three years from now? Luckily, engineers and others have studied how humans model things and the techniques they developed can be useful for both thinking through ideas thoroughly and communicating them in an unambiguous way to others.  Universal Modelling Language is such a technique, a standard way to model and communicate software ideas.

UML defines standard ways of diagramming, which is probably its most used -- and useful -- contribution. The problem is there seems to be a pretty wide gulf between a simple napkin sketch and UML and it's 13 (!) types of diagrams (and thick instructional books).

It doesn't have to be that way, though. In my opinion the most useful aspects of UML boil down to some fairly simple bare essentials readily gleaned from examples which I attempt to provide in the following presentation.

It's true that refactoring is all the rage with the programming fashionistas. But really, is it ever advantageous to not take a little time to think things through? With a few basics UML can be a pretty useful napkin.

Get the Gist: Universal Modelling Language (UML) from russellgmorley

Communication reliability

One of the most important things to consider when designing communication between two systems in an enterprise is reliability: Does the sender need to know if messages were delivered to the receiver? If an error occurs on the receiver, does the sender need to know? Does the sender need to know what type of error occurred?  Do messages need to be received in the order they were sent? What about receivers that may go offline or networks that go down? Does communication need to participate in a transaction so state consistency is maintained across multiple systems?

Before designing interfaces and selecting network protocols, it's necessary to understand the reliability requirements of the communication and how requirements align with the capabilities of protocol standards and industry techniques. Because the matrix of communications protocol capability and techniques as related to reliability is rather complex, it's useful to categorize communication reliability requirements based the following:

1. Unordered: Unordered is useful for providing information to one or more interested receivers quickly. Communication is one way and does not block. Receivers cannot return information back to the sender and senders don't know if communication with receivers was successful.
2. Receive-reply: For communication to a single receiver, Receive-reply requires a sender wait for a reply or timeout. Both information and errors can be returned in the reply and senders know if communication was successful. 
3. Ordered: Provides ordered communication with one or more receivers. Information cannot be returned from receivers to senders but errors can. When configured to return errors, Ordered communication blocks but otherwise can be configured to be non-blocking.
4. Guaranteed: A form of Ordered communication, Guaranteed cannot return information or errors but guarantees delivery even if the receiver isn't running when sent.
5. Transactional: Either configured as Receive-reply or Ordered, Transactional can return information to the sender if configured as Receive-reply, can always return errors, and can further participate in a transaction as a durable resource.

In communication design, reliability requirements informs everything from the design of the interface operations to protocol selection and is a crucial part of creating a robust enterprise system. Capturing reliability requirements in the form of a few simple categories is a practical way to express business needs in terms real-world protocols and industry techniques can address.

The usefulness of adapters: a pictorial