---
title: "User Interaction in WebGL"
author: "Duncan Murdoch"
date: "`r format(Sys.time(), '%B %d, %Y')`"
output:
  rmarkdown::html_vignette:
    toc: yes
    fig_width: 5
    fig_height: 5
vignette: >
  %\VignetteIndexEntry{User Interaction in WebGL} 
  %\VignetteEngine{knitr::rmarkdown}
---


```{r setup, echo=FALSE, results="asis"}
source("setup.R")
set.seed(123)
```

## Introduction

This document describes how to embed `rgl` scenes in HTML documents
and use embedded Javascript to 
control a WebGL display in an HTML document.  For more 
general information, see [rgl Overview](rgl.html).

We assume that the HTML document is produced from R markdown
source using `knitr` or `rmarkdown`.  This format mixes
text with Markdown markup with chunks of R code.

There are two ways to embed an `rgl` scene in the document.  The
older one is to use the chunk option `webgl = TRUE`.  With that
option, whatever `rgl` scene is active at the end of the chunk
will be embedded.  See the `r linkfn("setupKnitr", pkg="rgl")` help page.

The second way is to use a call to `r linkfn("rglwidget")`.  Each call to
this function will insert a scene into the document.  Do not set
`webgl = TRUE`.  


The second method is easier for
me to maintain, so it is likely to receive more support in the 
future, but for now both methods are supported, and there
are examples of both in this document.

I am currently conducting experiments on a
third method.  This is intended to be similar to the way 
standard 2D graphics are included by `knitr`,
i.e. it will detect the fact that you've 
drawn something, and just include it automatically.

## Browser support

Most browsers now support WebGL, but it may be disabled by
default.  See http://get.webgl.org for help on a number of
different browsers.  

If you are using the internal browser in RStudio, support varies
by version.  I believe it is enabled by default in Windows
versions, but until recently was not enabled in Mac OSX versions.  You can run this
command in Terminal:
 
    defaults write org.rstudio.RStudio WebKitWebGLEnabled -bool YES

to enable it.  I do not have much experience with RStudio in
Linux, but it does seem that WebGL is enabled there.

## Examples

We start with two simple examples.  The next section gives 
reference information.

Consider the simple plot of the iris data.  We
insert a code chunk and call the `r linkfn("rglwidget")` 
function with optional argument `elementId`.  This allows later
Javascript code to refer to the image.
```{r plot3d}
library(rgl)
with(iris, plot3d(Sepal.Length, Sepal.Width, Petal.Length, 
                  type="s", col=as.numeric(Species)))
subid <- currentSubscene3d()
rglwidget(elementId="plot3drgl")
```
We might like a button on the web page to cause a change to the
display, e.g. a rotation of the plot.  First we add buttons, with
the "onclick" event set to a function described below:

    <button type="button" onclick="rotate(10)">Forward</button>
    <button type="button" onclick="rotate(-10)">Backward</button>

which produces these buttons: 
<button type="button" onclick="rotate(10)">Forward</button>
<button type="button" onclick="rotate(-10)">Backward</button>

We stored the subscene number that is currently active in
`subid` in the code chunk above, and use it as `r rinline("subid")`
in the script below.  `knitr` substitutes the value
`r subid` when it processes the document.

The `rotate()` function uses the Javascript function `document.getElementById` to retrieve the `<div>` component
of the web page containing the scene.  It will have a
component named `rglinstance` which contains information about the scene that we can modify:

    <script type="text/javascript">
    var rotate = function(angle) {
      var rgl = document.getElementById("plot3drgl").rglinstance;
      rgl.getObj(`r rinline("subid",
                           script=TRUE)`).par3d.userMatrix.rotate(angle, 0,1,0);
      rgl.drawScene();
    };
    </script>
    
<script type="text/javascript">
var rotate = function(angle) {
  var rgl = document.getElementById("plot3drgl").rglinstance;
  rgl.getObj(`r subid`).par3d.userMatrix.rotate(angle, 0,1,0);
  rgl.drawScene();
};
</script>

If we had used `webGL=TRUE` in the chunk header,
the `knitr` WebGL support would create a global object with a name of the form `<chunkname>rgl`.  For example,  if the code chunk
was named `plot3d`, the object
would be called `plot3drgl`, and this code would work:

    <script type="text/javascript">
    var rotate = function(angle) {
      plot3drgl.getObj(`r rinline("subid",
                           script=TRUE)`).par3d.userMatrix.rotate(angle, 0,1,0);
      plot3drgl.drawScene();
    };
    </script>
    

## Autogenerated controls

We can also change the contents of the plot using `r indexfns("toggleButton")`.
For example, we can redo the previous plot, but with the
three species as separate "spheres" objects and buttons to
toggle them:
```{r toggle, webgl = TRUE, rgl.newwindow = TRUE}
sphereid <- with(subset(iris, Species == "setosa"), 
     spheres3d(Sepal.Length, Sepal.Width, Petal.Length, 
                  col=as.numeric(Species),
                  radius = 0.211))
with(subset(iris, Species == "versicolor"), 
     spheres3d(Sepal.Length, Sepal.Width, Petal.Length, 
               col=as.numeric(Species),
     	       radius = 0.211))
with(subset(iris, Species == "virginica"), 
     spheres3d(Sepal.Length, Sepal.Width, Petal.Length, 
               col=as.numeric(Species),
     	       radius = 0.211))
aspect3d(1,1,1)
axesid <- decorate3d()
subid <- currentSubscene3d()
```

```{r results="asis"}
toggleButton(sphereid, label = "setosa", prefix = "toggle", subscene = subid)
toggleButton(sphereid+1, label = "versicolor", prefix = "toggle", subscene = subid)
toggleButton(sphereid+2, label = "virginica", prefix = "toggle", subscene = subid)
```

Note that we need to use `results="asis"` for the button code.  
Normally we would also use `echo=FALSE`, though I didn't do so above;
then the buttons will end up side-by-side.  We also add another button
to toggle the axes:

```{r results="asis", echo=FALSE}
toggleButton(sphereid, label = "setosa", prefix = "toggle", subscene = subid)
toggleButton(sphereid+1, label = "versicolor", prefix = "toggle", subscene = subid)
toggleButton(sphereid+2, label = "virginica", prefix = "toggle", subscene = subid)
toggleButton(axesid, label="axes", prefix = "toggle", subscene = subid)
```

An alternate control to achieve the same thing is `r indexfns("subsetSlider")`.  Here we also illustrate the 
`r indexfns("elementId2Prefix")` bridge to allow an `r indexfns("rglwidget")`
to be controlled by the old-style slider.

```{r slider}
rglwidget(elementId = "slider")
```
```{r results="asis"}
elementId2Prefix("slider")
subsetSlider(subsets = list(setosa = sphereid, 
                  versicolor = sphereid + 1, 
                  virginica = sphereid + 2, 
                  all = sphereid + 0:2),
             prefixes = "slider", subscenes = subid,
             init = 3)
```

There are several other functions to generate the Javascript
code for controls.  `r indexfns("par3dinterpSetter")` generates
a function that approximates the result of `r linkfn("par3dinterp")`.
`r indexfns("propertySetter")` is a more general function to set
the value of properties of the scene.  Both generate Javascript
functions, but not the controls to use them; for that, use
`r indexfns("propertySlider")` or your own custom code.

For example, the following code (similar to the `r linkfn("play3d")`
example) rotates the scene in a complex way.
```{r userMatrix, webgl=TRUE}
```
```{r results="asis"}
M <- r3dDefaults$userMatrix
fn <- par3dinterp(time = (0:2)*0.75, userMatrix = list(M,
                                     rotate3d(M, pi/2, 1, 0, 0),
                                     rotate3d(M, pi/2, 0, 1, 0) ) )
propertySlider(setter = par3dinterpSetter(fn, 0, 1.5, steps=15, 
					   prefix = "userMatrix", 
					   subscene = subid),
	       step = 0.01)
```
Some things to note:  The generated Javascript slider has 150 increments,
so that motion appears smooth.  However, storing 150 `userMatrix` values
would take up a lot of space, so we use interpolation
in the Javascript code.  However, the Javascript code can only do 
linear interpolation, not the more complex spline-based SO(3)
interpolation done by `r linkfn("par3dinterp")`.  Because of this,
we need to output 15 steps from `r linkfn("par3dinterpSetter")`
so that the distortions of linear interpolation are not visible.  

Another function that auto-generates Javascript code is 
`r indexfns("clipplaneSlider")`.  This function allows the user to control
the location of a clipping plane by moving a slider.  Both it 
and `r linkfn("par3dinterpSetter")` are implemented
using the more general `r indexfns("propertySlider")`, which 
allows control of multiple objects in multiple scenes, but which 
does require knowledge of the internal representation of the scene
in its Javascript implementation.


Less general than `r linkfn("propertySetter")` is
`r indexfns("vertexSetter")`.  This function sets attributes
of individual vertices in a scene.  For example, to set the
x-coordinate of the closest point in the setosa group, and modify
its colour from black to white,

```{r vertex, webgl=TRUE}
```

```{r results="asis"}
setosa <- subset(iris, Species == "setosa")
which <- which.min(setosa$Sepal.Width)
init <- setosa$Sepal.Length[which]
propertySlider(
	vertexSetter(values = matrix(c(init,8,0,1,0,1,0,1), nrow=2),
	             attributes=c("x", "r", "g", "b"), 
		     vertices = which, objid = sphereid, 
		     prefix = "vertex"), 
	step=0.01)
```
	     

A related function is `r indexfns("ageSetter")`, though it uses
a very different specification of the attributes.
It is used when the slider controls the "age" of the scene, 
and attributes of vertices change with their age. 

Rather than giving an example, we will illustrate
the very similar function `r indexfns("ageControl")`, embedded in a 
`r indexfns("playwidget")`.  We will
show a point moving along a curve.  In the original scene
we need to specify multiple colours so that the
colour is not fixed, and can be controlled by the slider.  We
also give two `ageControl` calls in a list;

```{r}
time <- 0:500
xyz <- cbind(cos(time/20), sin(time/10), time)
lineid <- plot3d(xyz, type="l", col = c("black", "black"))["data"]
sphereid <- spheres3d(xyz[1, , drop=FALSE], radius = 8, col = "red")
rglwidget(elementId = "ageExample")
```
```{r}
playwidget("ageExample", list(
  ageControl(births = time, ages = c(0, 0, 50),
		colors = c("gray", "red", "gray"), objids = lineid),
	ageControl(births = 0, ages = time,
		vertices = xyz, objids = sphereid)),
  start = 0, stop = max(time) + 20, rate = 50,
  components = c("Reverse", "Play", "Slower", "Faster",
                 "Reset", "Slider", "Label"),
  loop = TRUE)
```

The final function of this type is `r indexfns("matrixSetter")`, for setting
up multiple controls to modify a matrix, typically `userMatrix`. This is used
when complex manipulation of a matrix requires several controls.

## User defined mouse controls

`rgl` allows user defined mouse controls.  For these to work
within WebGL, you will need to write a Javascript version as
well as the R version.  This isn't easy:  R provides a lot
of support functions which are not easily available in Javascript.

TODO:  give an example here.

## Reference for `rglClass`

NB:  This section has not been updated recently, and is 
not current.

In writing the `writeWebGL()` function, I haven't tried to prevent access to 
anything.  On the other hand, I haven't provided access to
everything.  The parts documented here should remain relatively stable
(unless indicated otherwise).  Users may also consult the source
to `writeWebGL`, but should be aware that anything that isn't documented
here is subject to change without notice.  

`r indexclass("rglClass")`

As documented in `r linkfn("writeWebGL")`, the call
```{r eval=FALSE}
writeWebGL(..., prefix = "<prefix>")
```
will create a global object on the output page with name 
`<prefix>rgl` and Javascript class `rglClass`. 
This class has a large number of properties and methods, some of which are designed
to be available for use by other code on the web page. 

Most of the properties are stored as Javascript `Array` objects, indexed
by the `rgl` id of the subscene to which they apply.  There 
are also Javascript methods attached to the `rglClass` class.

### Methods

`r indexmethods("drawScene")`

After any change that will affect the display, code should
call `<prefix>rgl.drawScene()` to redraw the scene.

`r indexmethods(c("inSubscene", "addToSubscene", "delFromSubscene"))`

These methods each take two arguments:  `id` and `subscene`,
which should be the `rgl` ids of an object and a subscene.
`inSubscene` tests whether `id` is already included in the 
subscene, and the others 
add it or delete it from the subscene.

`r indexmethods(c("getSubsceneEntries"))`

This function takes a subscene id as argument, and returns an `Array`
containing all of the ids displayed in that subscene.

`r indexmethods(c("setSubsceneEntries"))`

This takes an `Array` of ids and a subscene id as arguments, and sets
the contents of the subscene to the ids.

### Properties

`r indexproperties(c("FOV", "listeners", "userMatrix", "zoom"))`

These correspond to the 
`r linkfn("par3d")` properties with the same names.  

- `FOV` and `zoom` are arrays of numbers. 
- `userMatrix` is an array
of `CanvasMatrix4` objects (documented in the file 
`system.file("htmlwidgets/lib/CanvasMatrix/CanvasMatrix.src.js", package = "rgl")`.
- Each `listeners` item is itself an array of subscene ids that "listen"
to mouse actions, i.e. `listeners[19]` would contain all
subscene ids that respond to mouse actions in subscene 19.

`r indexproperties("viewport")`

This property also corresponds to the
`r linkfn("par3d")` property, but should be considered to be 
read-only.

`r indexproperties(c("drawFns", "clipFns"))`

These two arrays contain the code to display 
each object in the scene.  The functions in the
`drawFns` array are called for each object
each time it is displayed.  The `clipFns` functions
are called when objects being clipped are drawn.

`r indexproperties(c("values", "offsets"))`

Most of the data about each object in a scene is contained in
the `values` property.  This is an array, indexed by object
id. The individual entries are numeric arrays.  Though they
are singly-indexed, the entries are meant to be interpreted
as matrices stored by row.  The first 3 columns are generally
the coordinates of a vertex, and remaining columns correspond
to other values from `r linkfn("rgl.attrib")`.  

The `offsets` property gives the (0-based) offset of the first 
column for a particular attribute in a named object.  Not all 
columns will be present in every object; if not
present, the corresponding `offsets` entry will be `-1`.
The entries are

Name | Meaning
-----|--------
vofs | Offset to 3 columns of vertex data in XYZ order
cofs | Offset to 4 columns of colour data in RGBA order
nofs | Offset to 3 columns of normal data
radofs | Offset to 1 column of sphere radius data
oofs | Offset to 2 columns of text or sprite origin data
tofs | Offset to 2 columns of texture coordinates
stride | Total number of columns in `values`

For example, to find the blue colour entry for vertex 
`i` in an object, one would first check if `offsets["cofs"]` was
`-1`, indicating that no colour information was present.  If
not, the entry could be found using
```
values[offsets["stride"]*(i-1) + offsets["cofs"] + 2]
```
This assumes `i` is specified using 1-based vertex counting
as in R, and writes `values` and `offsets` instead of the
fully specified `<prefix>rgl.values` and `<prefix>rgl.offsets`
for clarity.

Changes to `values` need to be pushed to the graphics system
to be reflected in the scene; see the calls to `gl.bindBuffer` and
`gl.bufferData` in the source to `r linkfn("propertySlider")`
for details.


## Index 

The following functions and `rglClass` properties and methods are described in this document:<br>

```{r echo=FALSE, results="asis"}
writeIndex(cols = 5)
```