New look, new features
This past week, Tak was nice enough to give the E15 interface a full overhaul, and he’s done an incredible job. Everything is much cleaner, and this has allowed us to continue pushing forward.
During this time, I finished up some more advanced E15 additions, providing users with the ability to modify the way they interact with web content. I think the easiest way to demonstrate these new additions is through a detailed example, so here we go….
Suppose I’m an E15 user. I use it to write scripts and make nice visualizations. Suppose I have collected a few examples that I like, and want to make a nice little web page that contains some images that show what these scripts look like, and have those images link to the scripts themselves. As a demonstration, I took four existing scripts, and created a web page that shows them as follows:
The images in the page above link to the scripts themselves. That is, the html looks like the following:
<a href=”http://plw.media.mit.edu/balls.py”> <img src=”balls.png”> </a>
When loaded in the context of a standard web browser, clicking on one of the images will show the textual form of the python script, which isn’t particularly interesting. What we would really like to do, is be able to click on one of those images, and have the script load directly into E15. Load, and start running immediately. The question is, then, how the heck would you do something like that?
While there isn’t really any particularly elegant way of supporting this functionality when using a standard browser, we can now do some nice things within E15 itself. What end user can do now, is write Python code not only to create visualizations and graphics, but code to create native OS X interfaces. With this functionality, end users can write scripts to create OS X windows that contain a web browser, for example. In addition, users can define what happens when certain browser-specific events happen (e.g. clicking on a link). For the most part, this boils down to the user writing a small piece of handling code to do something special when links are clicked on in the context of the custom browser the user has written. So, the following script will allow us to detect whether or not the link we clicked on was a Python script, and if so, try to load it into E15:
def webView_didFinishLoadForFrame_(self,webview,frame):
url = …
if url.hasSuffix_(”.py”):
e15_loadScript(url)
The result?
Running this custom handling code in the context of our custom E15 browser allows us to talk to the web in ways never before possible. Giving creative individuals the ability to do with web content what they want, as opposed to what they have to, stands to be a paradigm changing idea for the future of creative media, and is an integral part of E15.
January 23rd, 2008 at 3:25 pm
Great idea! One question: are you guys sandboxing the python code? I ran into a similar problem a while ago, where it was possible to run evil python code. I ended up using PyPy, but I am not sure if that will fit your specifications. Let me know if you found a better solution. Thanks!
January 27th, 2008 at 4:08 pm
Yeah, I also ran into this stuff with Promiserver. There were some documented ways to approach this with Ruby, but none of them were ideal. Funny these recurring issues within the same research group.
Also, if you don’t mind me challenging a bit of this, I don’t quite grok this “do what they want” versus “do what they have to” distinction. I recall seeing a similar message from you guys in other materials, so it seems like a core assertion of the project. But what is the exact problem now that his solves? I don’t feel like my current web experience is impoverished simply because I can’t script it or view it in 3D. So I don’t doubt that there is something new here, but I’d like to see some of these assertions drawn out into solid arguments.
January 28th, 2008 at 7:04 pm
The sandboxing problem is something we’ve thought about, but don’t have any current plans to address. Sandboxing prevents things like reading and writing of client files, which I personally find useful in E15. Remember, E15 is a desktop application, not a web application, and not all of the traditional web security problems directly apply.
At the highest level, the problem we’re addressing is the 2D “page” model of the web. We’re providing people with the tools to obtain and manipulate web data in a much more dynamic fashion. With your current setup, can you visually compare four web pages at once? No. Can you procedurally extract select pieces of content from a page or a set of pages and place them in a new context where new relationships can be visualized between them? No. One goal of E15 is to make these types of things easy. We just came back from a talk by David Small who labeled this “browser problem” as one of the three most important issues that needs to be addressed in the future.
You’re right, the web and it’s associated aesthetic clearly won’t be going anywhere in the near future. People know how to use it, and they’re ok with that experience. Building tools that allow people to grasp and shuffle different types of web data around is an important first step towards a different browser experience.
January 30th, 2008 at 4:03 am
For a (still unfinished) project I’ve been working on for the past few months, one of the core tenets I’ve laid down is that people make insights about data via their own manipulation and experimentation, and then they share those insights with others via social connections. Social web systems can facilitate sharing and collaboration. PLW has done a lot of that stuff the past few years. But you need some revolutionary content to make this social exchange worthwhile. Seems like that’s what you’re going for here. You’re building the tools for the manipulation and creation of insights. And instead of pre-canned massaged data sets, you’re designing something that chews on dynamic data straight out of the very fabric of the web. Definitely powerful.