Go!

This kind of easy DOM access allows us to augment the traditional browsing experience in strange new ways. As a simple example, we can write some Javascript to return all of the src attributes of the current page, and load them as individual images into the E15 context:

from staticwriter import *
from random import random

jscript = “””function pyeval(s) {
    res = document.getElementsByTagName(s)
    var myArray = [];
    for (var i = 0; i < res.length; i++)
        myArray[i] = res[i].src;
    return myArray;
}
“””
envcolor(1, 1, 1)
images = jscripteval( jscript, “pyeval”, [”img”])
for img in images:
    imgload(img, img, random(), random(), random(), 0, 0, 0)

In addition to this usage, jscripteval() can be used in another way. Users can specify a URL pattern that, if matched, will trigger the invocation of the specified Javascript, and return the value through the invocation of a specified Python callback function. In this way, we can write some Javascript that extracts out all of the message headers from my Facebook message inbox, and places that text next to each user icon.

During this time, I finished up some more advanced E15 additions, providing users with the ability to modify the way they interact with web content. I think the easiest way to demonstrate these new additions is through a detailed example, so here we go….

Suppose I’m an E15 user. I use it to write scripts and make nice visualizations. Suppose I have collected a few examples that I like, and want to make a nice little web page that contains some images that show what these scripts look like, and have those images link to the scripts themselves. As a demonstration, I took four existing scripts, and created a web page that shows them as follows:

The images in the page above link to the scripts themselves. That is, the html looks like the following:

<a href=”http://plw.media.mit.edu/balls.py”> <img src=”balls.png”> </a>

When loaded in the context of a standard web browser, clicking on one of the images will show the textual form of the python script, which isn’t particularly interesting. What we would really like to do, is be able to click on one of those images, and have the script load directly into E15. Load, and start running immediately. The question is, then, how the heck would you do something like that?

While there isn’t really any particularly elegant way of supporting this functionality when using a standard browser, we can now do some nice things within E15 itself. What end user can do now, is write Python code not only to create visualizations and graphics, but code to create native OS X interfaces. With this functionality, end users can write scripts to create OS X windows that contain a web browser, for example. In addition, users can define what happens when certain browser-specific events happen (e.g. clicking on a link). For the most part, this boils down to the user writing a small piece of handling code to do something special when links are clicked on in the context of the custom browser the user has written. So, the following script will allow us to detect whether or not the link we clicked on was a Python script, and if so, try to load it into E15:

def webView_didFinishLoadForFrame_(self,webview,frame):
    url = …
    if url.hasSuffix_(”.py”):
        e15_loadScript(url)

The result?

Running this custom handling code in the context of our custom E15 browser allows us to talk to the web in ways never before possible. Giving creative individuals the ability to do with web content what they want, as opposed to what they have to, stands to be a paradigm changing idea for the future of creative media, and is an integral part of E15.

This semester, I’ve been using E15 to visualize data from Facebook. Whether you like Facebook or not, it is one place that can offer a lot of social data related to you, and the data is unique for individual user. Thanks to E15 using Python, we can easily interface with Facebook using PyFacebook. Anyway, the Facebook visualization is still not done, but it will be posted when it is. This is a video I made for a class that shows the initial Facebook visualization.

There were numerous features that did not exist in E15 which had to be implemented for the Facebook visualization. Kyle already mentioned the threading issue. There’s now a method called settimeout and cleartimeout which can execute methods with a delay, much like JavaScript’s window.setTimeout() and clearTimeout(). Also, E15 did not have any facilities to move to an object in the environment nor a way to move the object’s coordinates once inserted.

Now both of these features are implemented. There is an elem_moveto() method which can animate any element into a new location in space. The camera can animate to fit an element into the viewing space by calling camera_goto().

Another feature attempted was using the accumulation buffer to create a trailing effect when elements moved. Given that E15’s native view size is 1600×900, when the accumulation buffer was enabled, the frame rate dropped to a point of uselessness. It does look cool though…

With a browser, it becomes trivial to leverage the existing features of E15 to visually represent browser interactions. As a simple first demo, we can load a page into the embedded E15 browser, navigate through a set of pages, and construct a visual history of pages traversed. Show below is the result of the traversal of 9 links starting from google:

While adding support to text, we discovered that our mipmaps weren’t working. We noticed a lot of scintillation effects from the texts drawn in the environment. We are using NPOT textures, and after a couple days of going back and forth about whether NPOT mipmaps were possible or not, we finally figured it all out. As long as ARB_texture_non_power_of_two extension is supported, NPOT mipmaps will work just like POT. The trick is to use GL_TEXTURE_2D and treat NPOT just like POT.The other problem which hopefully is fixed with the official release of Leopard is that our ATI X1900 does not support hardware mipmap generation. Instead, we have to call gluBuild2DMipmaps(). The NVIDIA 7300 however does support it.

In some early sketches of web content layouts, I had arranged webpages around the perimeter of a circle. While searching for more interesting equations to use for laying out pages, I found an awesome Wikipedia page called List of Curves with over 50 different types of mathematical curves. The problem with this page, however, is that it has no pictures of the curves. I want to see what all the curves look like, but I’m not about to click through every single page, one at a time.

So, I wrote a Python script that uses the MediaWiki API for Wikipedia to find all the articles linked from the List of Curves article, finds the images linked from those articles, and finally renders all the images in E15 with the relevant article titles:

If you’re curious about how these pages were laid out, I used a rose curve with 8 petals — pictures of this curve are in the middle of the screenshot below:

One more screenshot:

I’ll be working with the same data more this week, so check back soon for more images, and probably a video or two!