Web-based notebooks have proven to be an extremely popular medium for mining data and sharing valuable insights. They combine a traditional development console with the robustness of the world-wide-web, but often fail to leverage the full web browser capabilities to provide a rich, interactive user experience. We will look at a case where the Jupyter Declarative Widget Extension was used to enhance an existing notebook, replacing charts based on static rasterized images and server-bound event handling with new reusable, highly interactive components based on vector graphics. You can find more information on Jupyter Declarative Widgets in Gino Bustelo’s blog post. The end result is an improvement the user experience, making it easier for the user to explore information.
Data Visualizations in the Notebook
We begin with a report for a survey on user experience conducted by the Jupyter community. As an example of dogfooding, the author did the data analysis and created an interactive notebook-dashboard, all using the Jupyter notebook. In this typical notebook, the data is prepped and loaded into Pandas Dataframes. A bar chart visualization is generated with a straightforward call to Seaborn, a Python plotting library based on matplotlib. From this call, we get back a simple raster image in PNG format displayed in output cell.
Visualizations using Jupyter Declarative Widgets
In a newer revision of the notebook, we will generate a similar bar chart using the Jupyter Incubator Declarative Widgets project. The Jupyter Declarative Widgets Extension provides a standard bar chart visualization called <urth-viz-bar>
that is implemented with the nvd3 chart library. First, we must load the definition using a <link rel=”import”>
tag. Once that is done, the rest of the notebook can make use of the <urth-viz-bar>
widget.
%%html <link rel=‘import’ href=‘path/to/urth-viz-bar.html’ is=‘urth-core-import’> |
Since <urth-viz-bar>
requires a data frame as input, we need to create one:
1 |
how_often_dataframe = pd.DataFrame(how_often_counts).reset_index() |
We can now feed <urth-viz-bar>
data from how_often_dataframe
by using <urth-core-dataframe>
in a <template>
element:
1 2 3 4 5 6 |
%%html <template is=“dom-bind”> <urth-core-dataframe ref=“how_often_dataframe” value=“{{df}}” auto></urth-core-dataframe> <urth-viz-bar xlabel=“Notebook Usage” ylabel=“Respondents” datarows=“{{df.data}}” columns=“{{df.columns}}”></urth-viz-bar> </template> |
The <urth-core-dataframe>
widget reflects the dataframe how_often_dataframe
in the kernel to the browser, placing the JSON string representation in its attribute value
. In the scope of this template, we bind this data representation to a variable called df
. Then <urth-viz-bar>
can be bound to df
and use its data and columns properties.
The result looks something like the Seaborn chart we saw earlier:
Interactivity with compound, reusable widgets
Let’s go back to the first notebook without Declarative Widgets. To allow the user to interact with the data visualizations, we provide a separate Notebook that can be imported as the utils
module. We’re specifically interested in the function utils.explore
from this module, which will display three dropdowns for user inputs on how they want to filter the data. Once the selection is made, corresponding visualization is generated with a call to matplotlib. This logic runs in the kernel, processing client-side events from the dropdown menus to generate new raster images to send back to the browser client.
Delivering an even more interactive experience with Declarative Widgets
We can build the same experience with declarative widgets, and add a bit more interactivity, too. This time, instead of an ad hoc <template>, we’ll declare a custom Polymer element we’ll call <survey-explorer>. This is done in a separate HTML file which can be imported into the page using <link rel=”import”>
.
%%html <link rel=‘import’ href=‘prep/widgets/survey-explorer/survey-explorer.html’> |
You can look at the full implementation of survey-explorer
on Github.
Where utils.explore
used three dropdown widgets for inputs, in we will use UI elements from the Polymer paper project, including <paper-dropdown-menu>
and <paper-toggle-button>
. For example, the toggle button used to switch between counts and percentages on the y axis is created with the following markup:
1 |
<paper-toggle-button checked=“{{percentages}}”>Use percentages</paper-toggle-button> |
Again, the visualization is provided by an <urth-viz-bar>
widget. But instead of a reference to a single <urth-core-dataframe>
instance, we need some more prepping beforehand to filter the data as requested by the user.
So first, we create a Python function exploreDataFrame
in module utils
. This function takes user’s input of how the data should be filtered, and returns the corresponding dataframe. Then to use exploreDataFrame
, we need an <urth-core-function>
to reflect the dataframe from the kernel to our template in the browser.
1 |
<urth-core-function ref=“exploreDataFrame” arg-series=“{{seriesByLabel(selectedLabel)}}” arg-group_by=“{{group_by}}” arg-percentages=“{{percentages}}” result=“{{df}}” auto></urth-core-function> |
Then, similar to the previous example, the result df
is passed to the visualization, <urth-viz-bar>
:
<urth-viz-bar datarows=“{{df.data}}” columns=“{{df.columns}}” xlabel=“{{xlabel}}” ylabel=“{{ylabel}}” margin=‘{“bottom”:100, “right”:50}’ rotatelabels=“25” selection-info=“{{sel}}”> <template is=“dom-if” if=“{{percentages}}” restamp=“true”> <urth-viz-col index=“1” type=“numeric” format=“%”></urth-viz-col> </template> </urth-viz-bar> |
<urth-viz-bar>
also has a child tag <urth-viz-col>
which is responsible for formatting values on the y-axis as percentages. This tag is placed conditionally, only if percentages
is truthy, and will be updated automatically should this state change. Putting these pieces together, the compound widget begins to take shape:
Now we have a similar visualization to the first notebook, but there is one thing this notebook lacks that we can now support with Declarative Widgets. It is the ability for user to interact with the bar chart, explore and validate the data. When the data was prepped, how were the themes on the x-axis determined?
To answer this question, we add another Python function getSample
to module utils
. Very similar to how exploreDataFrame
works, getSample
receives information on which section of the data should be sampled and returns the corresponding data frame. To invoke getSample
, we need another <urth-viz-function>
:
1 2 3 4 5 6 7 8 9 |
<urth-core-function ref=“getSample” arg-keyword=“[[sel.x]]” arg-groupby=“[[group_by]]” arg-sample_size=“10” result=“{{sampleData}}” arg-groupby_key=“[[sel.key]]” arg-series=“{{seriesByLabel(selectedLabel)}}” arg-sample_source=“{{sampleSource}}” auto></urth-core-function> <template is=“dom-repeat” items=“[[sampleData.data]]”> <template is=“dom-repeat” items=“[[item]]”> <template is=“dom-if” if=“[[item]]”> <div style=“padding: 15px; background-color: white;”>[[item]]</div> </template> </template> <hr/> </template> |
Because <urth-viz-bar>
is built with d3, click event listeners can trigger further examination of the selected data. We bind this selected data to a variable called sel
which is also bound to the getSample
call. The resulting dataframe called sampleData
is bound to a dom-repeat
template which will generate <div>
tags corresponding to each data element in the array.
Now that all that is complete, you can go a step further by laying out the interactive widgets and deploying the result as a standalone web application.
Conclusion
The Jupyter Declarative Widgets Extension gives you the building blocks you need to communicate with the notebook environment as well as interactive, reusable components which can leverage the rich display capabilities of the web browser. You can easily experiment with your own creation using some of the widgets provided with the Declarative Widgets project, widgets from the growing Polymer ecosystem, or craft your own custom widgets.