@tidepool/viz's usage of D3

D3 is a JavaScript library often used for building interactive data visualizations. The name stands for Data-driven documents and expresses a small part of D3's philosophy, which is to stay very close to web standards for HTML5 documents (i.e., the DOM) rather than including or relying on a D3-specific component framework or an external framework such as React, Angular, Ember, etc. The functionality included in the entirety of the D3 library can be roughly divided into two categories:

1. a set of utilities for binding data to elements in the DOM and manipulating them, including adding, updating, transitioning (i.e., animating), and removing elements from the DOM
   • (While it is very common to render data visualization with inline SVG elements using D3, D3 is in fact unopiniated about choice of rendering target, though a great deal of the data-binding power of D3 is lost when rendering to <canvas> due to its nature as a raster (not vector) graphics system that lacks a 1-to-1 correspondence between DOM elements and data objects.)
2. all other utilities for working with data, independent of the DOM, including:
   • utilities for building and using scales to render data
   • utilities for generating the proper data structures for rendering particular shapes in SVG—e.g., <path> data strings for everything from segmented lines to arcs and shapes
   • simple statistics utilities—e.g., calculating quantiles from an array of data
   • some date & time utilities^a
   • lots of geographic utilities such as map projections

👻 History

Over the several years of Tidepool's history, we've tried two strategies for incorporating D3 into our React codebase for blip. The root of the issue when combining D3 and React code is that both D3 and React expect to be in control of updates to the DOM—that is, insertions and removals of elements based on application state, or in this case in particular the state of the data visualization, including filters and selections (such as a date range of data to display).

The componentDidMount strategy

The first strategy, which dominates in the tideline codebase^b, is a hand-off strategy. At the level of some container component for a data visualization, we pass the responsibility for rendering to the DOM from React to D3. This occurs in the componentDidMount lifecycle method of a React component, which fires after the component is rendered—though the trick in this case is that we typically return null or render just a container <div> in the render method and then call our chart rendering function or functions in componentDidMount to have D3 render the actual data SVG elements of the data visualization inside the container.

Former Tidepool employee Nico Hery described this approach very well in a blog post.

There are three reasons why we're no longer using this approach, at least not as our preferred strategy in new code:

• The reliance on the componentDidMount lifecycle method as well as D3's tight integration with browser DOM APIs makes testing much more difficult than rendering inline SVG directly via React; the latter strategy does not necessarily require a browser environment to run effective unit tests, and tests for (often) purely functional React components are much simpler, easier & faster to write, and easier to maintain.
• There are fewer best practices for writing DRY and stylistically similar D3 code than React code; d3.Chart, a lightweight "framework" for writing reusable D3 code, is the best example we've seen (and used^c) of an attempt to solve this problem, but it hasn't yet been updated to D3 v4.x and support for and ongoing development of the project by its original authors has been spotty at best.
• Related to the second point: we are a React team with our own established subset of React best practices and stylistic choices that we follow. There is much less cognitive load for us to keep writing visualization code in React than to switch to the much more procedural, less functional approach that D3's API requires.

The React inline SVG strategy

The second strategy, which we have been using exclusively so far here in viz (our repository of new visualization code eventually to replace tideline) is rendering the inline SVG for our data visualizations directly in React, as normal React components. What this ends up looking like is that we only use the second category of things provided in D3's functionality, outlined above. We use D3's utilities for building scales, crunching data, generating SVG path data, etc., but we don't use any of D3's rendering or animation functionality.

The main advantages this strategy yields are those noted above: less cognitive dissonance for those of us who write a lot of React code in blip and the uploader, clearer best practices (that we already know, again from using React extensively elsewhere), and much easier and simpler testing capabilities.

The main disadvantage of this strategy, on the other hand, is what we lose from D3's API when we're only using that second category of functionality. React provides a great replacement for the main parts of D3's selections and rendering: React's functional approach to UI components easily (and, in @jebeck's opinion, with a better, DRYer API) replaces D3's enter, update, and exit selection and rendering functions. But React provides no replacement for D3's animation functionality (provided by the transition function). Instead, we must look elsewhere for ways to animate the state transitions in our data visualizations, and in fact there doesn't seem to be a single library that fits all our use cases. So far, as of February 2017, we are using both react-motion and GSAP, as well as CSS3 animation, to fit all our animation use cases.

Some D3 history

Prior to its version 4.x, D3 was distributed only as a single large JavaScript library. Part of the work in the move to the 4.x version involved splitting up the code into modules, each contained in their own GitHub repository, published to npm independently, and versioned independently. A developer using D3 therefore now has a choice of whether to include the entire library in a project's dependencies—this is what would happen with npm install --save d3. The alternative is to depend on only the modules that you actually use and to pull in updates to each of these modules individually as desired. Partly because tideline still depends on the 3.x version of D3 (which would conflict with the 4.x version if we included both full dependencies) and partly because we just aren't using that large of a percentage of D3's functionality anymore, here in viz we only include the individual modules of D3 that we actually use in our dependencies.

✨ Today ✨

Our preferred strategy for new code is the React inline SVG strategy described above.

We use react-motion as the default choice for animation since it follows React coding practices the closest.

We resort to GSAP for animation when react-motion does not provide what we need—but this essentially requires the same componentDidMount strategy as D3, though without some of the challenges that strategy presented for us with D3 since we don't extensively test animation code, as it's not critical functionality for our core purpose of accurately communicating our users' medical data through various visualizations.

Currently the only packages we're depending on from the modularized v4.x of D3 are:

• d3-array for some data-munging (in particular calculating things like mean, median, quartiles, and quantiles)
• d3-format for easy formatting of numerical values^d
• d3-scale for linear scales
• d3-shape for generating path data for segmented lines, etc.
• d3-time for some of the date and time manipulation functions (only when we're manipulating based on UTC values—time in our data model)
• d3-time-format for formatting browser-local datetimes

🚀 The Future

While the React inline SVG strategy is our current preferred strategy, we do want to continue to evaluate the best strategy for each major new "view" that we code. It's possible that for some future view it may make more sense to return to the componentDidMount strategy, to a combination of the two, or to some unforeseen third strategy.

Some reasons that @jebeck can think of that may warrant a return to the componentDidMount strategy are:

• heavy use of one of D3's "layout"s such as a force layout or a Sankey diagram
• visualizing geographic data (unlikely at Tidepool?)

^a. Here's another place where D3's philosophy of adhering closing to existing web standards comes into play. Since JavaScript Date only supports browser-local (including Daylight Saving Time if applicable) or UTC datetime objects—and not datetime objects in an arbitrary, specified timezone—these are the only things D3 supports as well. Unfortunately for us at Tidepool, that means D3's datetime utilities (which in @jebeck's opinion have very intuitive, well-designed APIs) are only useful to us when we're manipulating UTC datetime objects since all of our timezone-relative datetime work needs to obey an arbitrary user-specified timezone. (Thus, we use Moment.js for all our timezone-relative datetime manipulation.) ↩

^b. Which we are gradually deprecating in favor of new visualization code in our viz repository. ↩

^c. We used d3.Chart in the old tideline version of the Trends view code, which is still as of this writing in February, 2017, available in tideline in the plugins/blip/modalday/ directory for reference, though one day soon we may delete this now-dead code. ↩

^d. We largely use Moment.js for our date & time value formatting, due to our need to specify an arbitrary timezone. ↩