Thousands of people invented it independently. Millions use it without thinking about a broader context. It’s time to name it so we can talk about it.
Transformation is changing the way we look at the balance between clients and servers, our approach to formatting and layout, and our expectations of what’s possible on the Web. As applications shift from transformation on the server toward transformation on arrival on the client, transformation’s central role becomes more visible.
“Templating” doesn’t quite capture what’s happening here. While templates are often a key tool, describing that tool doesn’t explain the shift from server to client. Templating also misses the many cases where developers are using plain JavaScript to insert, delete, and modify the document tree in response to incoming data.
These practices have been emerging for a long time, in many different guises:
- In the Dynamic HTML days, scripts might tinker with the DOM tree as well as modify CSS presentation.
- Transformation was supposed to be a regular and constant thing in the early XSLT plans. Stylesheets on the client would generate presentation from clean blocks of XML content.
- Ajax opened the door to shell pages, apps that set up a UI, but get most of their content elsewhere, using JavaScript to put it in place.
- New data format options evolved at about the same time that Ajax emerged. JSON offered a more concise set of programmer-friendly content tools. Many apps include a ‘bind JSON to HTML before showing it to the user’ step.
- Template systems now run on the client as well as the server. In many systems, templates on the server feed data to the client, which applies other templates to that data before presenting it to users.
- The HTTP powering Ajax still created a long slow cycle of interaction. WebSockets and WebRTC now offer additional approaches for collecting content with less overhead, making it easier to create many more small transformations.
- Some developers and designers have long thought of the document tree as a malleable collection of layout boxes rather than a deliberately coherent base layer. Separation of concerns? A dead horse, apparently. Recent debates over CSS Regions highlighted these issues again.
Fifteen years ago, arguing with people cheering on XSL transformations, I argued that decoration, CSS style, was enough. The virtues of its relative simplicity vastly overwhelmed the extra things XSLT made possible. Reordering the document tree might be a nice idea, but in 1999 the document tree was mostly made on the server in response to client requests.
Even today, XSLT remains a steep climb for most web developers. Client-side XSLT never found widespread support (though Saxon-CE and Frameless are well worth exploring). XSLT remains alien to most web developers.
Instead, we keep reinventing transformation with more familiar tools, often JavaScript tools. Few of these tools aim for the completeness or complexity of XSLT. Unfortunately, though, these tools are fragmented across many different communities (for frameworks) and people building their own versions.
Stepping back from the infinite details of frameworks and toolkits, some patterns begin to emerge:
- ‘Classic’ Web App
- Classic web applications combine data from a variety of sources on the server, and then send that material to the browser in final form to render and share. Transformations, sometimes many, happen along the way to create the HTML, but primarily on the server. Developers rely on the browser to know what to do with the content the server creates. (Static pages are back in vogue, thanks to tools like Jekyll. The transformation there happens when a URL is made available, rather than every time it is requested.)
- Single Page Apps (SPA)
- Single page apps replace the browser’s logic for loading, rendering, and often linking with their own logic, while still using the web stack of tools. Sometimes they include a basic HTML plus CSS page to start, but frequently they start from a blank document. The app reaches out to an API for rest of the content, often receiving it as JSON. Then it transforms that content into HTML and CSS for the browser to render. (Not all SPAs call back to the server. Some apps are complete once the initial load is done.)
- Isomorphic or hybrid applications
- A newly emerging, or at least recently noticed, category of apps, these combine processing on the server with interactivity on the client. They send the browser foundation documents of HTML and CSS, which JavaScript on the browser then supplements further with additional API calls and transformations. It’s less work for the browser, likely less delay on page load, and it makes the content sent by the server visible to search engines.
- Mojito
- When you’re running the same programming environment (JavaScript) on both the client and the server, you can move transforms wherever you want. I still marvel at Mojito, a framework from Yahoo! that lets developers set criteria for where a transformation should happen. A large chunk of app logic becomes a library of transformations that can be executed wherever fits the current processing, bandwidth, and other needs.
- React
- React pushes further on transformation opportunities by going beyond the browser DOM to create a virtual DOM, its own creation. Like Mojito, it runs happily on client or server.
See similarities? Transformations happen most of the time, pretty much as soon as you move beyond the “humans handcode complete static web files” phase of application development. (That still exists, of course.) While some of these things seem more obvious once the same language is used to do the work in multiple contexts, the transformation conversation has old but powerful foundations.
Not every application can take advantage of these opportunities. Web apps packaged for delivery in app stores often need to be self-contained, a complete set of content in a single package. The server may no longer be central, or it may be only occasionally available. The Apple app store and the Chromebook model are very different, for example.
The transformation discussion may also let us finally tackle the ugly secret of web applications: maintenance. The old HTML / CSS / JS model allowed a fairly clear “HTML is managed by content folks, CSS by designers, and JS by programmers” approach, though the integration could be exciting. As transformation has become more common, those parts have intermingled in ways that make sharing responsibility beyond programmers difficult.
I suspect that if we can talk openly about what we’re doing here, we may find new ways to structure the results. David Megginson, for example, suggested that we rethink CSS as a transformation to a set of layout boxes. The long-buried “can we fix the DOM” questions may even resurface.
Addy Osmani recently described the Web’s declarative, composable future. I think that’s the best face to put forward, layering complexity behind modules that developers integrate by reference in markup. I suspect, though, that the best practices for creating those components will require a sharp focus on transformation.
These conversations are just beginning to percolate, but maybe we can get to them if we can rebuild a common understanding of just what we’re doing with the data and content at the heart of our applications. I’m looking forward to this week’s Extensible Web Summit, hoping that maybe some of this will come to the fore.