Adding Declarative Widgets to the Jupyter Notebook

Web-based notebooks have proven to be an extremely popular medium for mining data and sharing valuable insights.  They combine a traditional development console with the robustness of the world-wide-web, but often fail to leverage the full web browser capabilities to provide a rich, interactive user experience. We will look at a case where the Jupyter Declarative Widget Extension was used to enhance an existing notebook, replacing charts based on static rasterized images and server-bound event handling with new reusable, highly interactive components based on vector graphics. You can find more information on Jupyter Declarative Widgets in Gino Bustelo’s blog post. The end result is an improvement the user experience, making it easier for the user to explore information.

Data Visualizations in the Notebook

We begin with a report for a survey on user experience conducted by the Jupyter community. As an example of dogfooding, the author did the data analysis and created an interactive notebook-dashboard, all using the Jupyter notebook. In this typical notebook, the data is prepped and loaded into Pandas Dataframes.  A bar chart visualization is generated with a straightforward call to Seaborn, a Python plotting library based on matplotlib.  From this call, we get back a simple raster image in PNG format displayed in output cell.

Bar Graph created with Python and Seaborn

Visualizations using Jupyter Declarative Widgets

In a newer revision of the notebook, we will generate a similar bar chart using the Jupyter Incubator Declarative Widgets project. The Jupyter Declarative Widgets Extension provides a standard bar chart visualization called <urth-viz-bar> that is implemented with the nvd3 chart library. First, we must load the definition using a <link rel=”import”> tag.  Once that is done, the rest of the notebook can make use of the <urth-viz-bar> widget.

Since <urth-viz-bar> requires a data frame as input, we need to create one:

We can now feed <urth-viz-bar> data from how_often_dataframe by using <urth-core-dataframe> in a <template> element:

The <urth-core-dataframe> widget reflects the dataframe how_often_dataframe in the kernel to the browser, placing the JSON string representation in its attribute value. In the scope of this template, we bind this data representation to a variable called df. Then <urth-viz-bar> can be bound to df and use its data and columns properties.

The result looks something like the Seaborn chart we saw earlier:

Bar Chart with Declarative Widgets

Interactivity with compound, reusable widgets

Let’s go back to the first notebook without Declarative Widgets. To allow the user to interact with the data visualizations, we provide a separate Notebook that can be imported as the utils module. We’re specifically interested in the function utils.explore from this module, which will display three dropdowns for user inputs on how they want to filter the data. Once the selection is made, corresponding visualization is generated with a call to matplotlib. This logic runs in the kernel, processing client-side events from the dropdown menus to generate new raster images to send back to the browser client.

Interactive chart with Python / ipywidgets

Delivering an even more interactive experience with Declarative Widgets

We can build the same experience with declarative widgets, and add a bit more interactivity, too.  This time, instead of an ad hoc <template>, we’ll declare a custom Polymer element we’ll call <survey-explorer>.  This is done in a separate HTML file which can be imported into the page using <link rel=”import”>.

You can look at the full implementation of survey-explorer on Github.

Where utils.explore used three dropdown widgets for inputs, in we will use UI elements from the Polymer paper project, including <paper-dropdown-menu> and <paper-toggle-button>. For example, the toggle button used to switch between counts and percentages on the y axis is created with the following markup:

Again, the visualization is provided by an <urth-viz-bar> widget.  But instead of a reference to a single <urth-core-dataframe> instance, we need some more prepping beforehand to filter the data as requested by the user.

So first, we create a Python function exploreDataFrame in module utils. This function takes user’s input of how the data should be filtered, and returns the corresponding dataframe. Then to use exploreDataFrame, we need an <urth-core-function> to reflect the dataframe from the kernel to our template in the browser.

Then, similar to the previous example, the result df is passed to the visualization, <urth-viz-bar>:

<urth-viz-bar> also has a child tag <urth-viz-col> which is responsible for formatting values on the y-axis as percentages.  This tag is placed conditionally, only if percentages is truthy, and will be updated automatically should this state change. Putting these pieces together, the compound widget begins to take shape:

survey-explorer widget

Now we have a similar visualization to the first notebook, but there is one thing this notebook lacks that we can now support with Declarative Widgets. It is the ability for user to interact with the bar chart, explore and validate the data. When the data was prepped, how were the themes on the x-axis determined?

To answer this question, we add another Python function getSample to module utils. Very similar to how exploreDataFrame works, getSample receives information on which section of the data should be sampled and returns the corresponding data frame. To invoke getSample, we need another <urth-viz-function>:

Because <urth-viz-bar> is built with d3, click event listeners can trigger further examination of the selected data. We bind this selected data to a variable called sel which is also bound to the getSample call. The resulting dataframe called sampleData is bound to a dom-repeat template which will generate <div> tags corresponding to each data element in the array.

Data samples

Now that all that is complete, you can go a step further by laying out the interactive widgets and deploying the result as a standalone web application.

Conclusion

The Jupyter Declarative Widgets Extension gives you the building blocks you need to communicate with the notebook environment as well as interactive, reusable components which can leverage the rich display capabilities of the web browser. You can easily experiment with your own creation using some of the widgets provided with the Declarative Widgets project, widgets from the growing Polymer ecosystem, or craft your own custom widgets.

Adam Peller

Adam is a Software Engineer in the IBM Emerging Technologies group

Adam Peller
Adam Peller