Automagic rezzing with Sikuli, Part the Second

I had a comment from Eloise Pasteur on my last blog asking whether running Sikuli out of debug mode speeded things up. I said "a lot" but on reflection that may have been an overestimate. Anyway, the video above illustrates the use of Sikuli to rez in standard mode but this time I've included a couple of extra features.

Firstly, as prims rez, they chat and that chat is registered by a chart prim which updates a shared media chart texture based on the Google Visualization API. For the moment the chart is as basic as it could be but is intended as a reminder that we no longer have to reinvent 2D graphs inworld. In a more general way, it also illustrates the use of shared media as a complement to prims. There is doubtless plenty of further scope for this type of integration. Dusan Writer laments the lack of obvious shared media developments beyond the most basic (he sees this as a consequence of the poor uptake of Viewer 2.0). It would be nice to move this area on a little.

Secondly, after rezzing has finished, I used the Sikuli findAll command to locate the spheres and then count them, displaying the score in a popup. As the capacity of the sensor facility in SL is limited to 16 objects, this might be an interesting alternative to use of sensor arrays, albeit that it requires that objects be within the field of view, static and non-overlapping. Coping with rotation is also an issue with more complex objects.

One could go a lot further with Sikuli by introducing visual conventions for other parameters such as elevation and rotation. Prims might themselves have additional codes on them so a second level of prims could be added. "Intelligent" prims might act to close gaps by rezzing further prims.

As Sikuli is based on Jython, the Java implementation of Python, it has access to a wide range of code libraries and, more importantly, can read and write to local files as well as the web. For very simple, single-level builds therefore, it might facilitate documentation and re-rezzing. Of course, there are easier ways to do this already available but bear in mind that Sikuli could take objects exported in XML format (with appropriate permissions) and rez them inworld, maybe taking additional data on size, elevation, scaling, etc from a separate scene file. These refinements could be implemented via Sukuli's control of the Build menu of a suitable third party viewer. (In all this, I assume, of course, that you have created all components of the object; export under these circumstances is compatible with the SL ToS. Scripts are not presently archived so some additional work would be required).

More interestingly (and bearing in mind my 2010 predictions #4 and #5), Sikuli could be used to monitor changes in the scene (registered visually, via the web or communicated to a HUD prim watched by Sikuli) and make corresponding alterations to it. For example, if one scene is successfully completed, a flag is displayed and the next scene is rezzed. This begins to sound a little like learning design. The idea of using a vertical view of a scene in SL to design and implement a learning experience that responds to student activity is quite attractive.

A list of requirements for a simplified learning design implementation has been compiled by Scott Wilson, though it does assume the availability of a SCORM implementation. I doubt that I'll pursue the learning design aspect, not least because options such as PIVOTE and, at a lower level, LibOpenMetaverse already exist. Moreover, the requirement to have Sikuli running in the foreground on a PC somewhere is onerous, not least when you attempt to scale to multiple scenes being used simultaneously by different groups. It might be done in conjunction with a facility for camera repositioning but it would not be pretty.

That said, the introduction of a 4th dimension, viz time, into SL builds is pertinent to other smallscale projects I have in mind and Sikuli is definitely an option to consider there.

Automagic rezzing with Sikuli

I already raved a little about Sikuli, the pattern recognition scripting language. I'm compiling a talk on innovative uses of shared media in SL and thought it would be fun to show Sikuli being used as a rezzing tool with a shared media prim as the template. The demo shows that objects can be automagically dragged from inventory and dropped onto multiple copies of the character S (for sphere) and B (for box) that were generated previously using the text tool in Board800. The script terminates when no further characters are recognised.

Bear in mind that the demo is deliberately played at debug speed so that the actions are visible. The process can also be speeded up by restricting the screen region that needs to be searched. You may find that you need to tune the recognition a little (there's a dialog accessed via the image button in the script). Also, I have the avatar camera canted slightly and the perspective can cause recognition problems; it is better to have the camera directly above.

As Board800 is multi-user and updates synchronously, this approach might be used for a 2-stage collaborative design process that starts with shared media in or outside SL and ends with objects being rezzed onto the whiteboard image.

I'm not sure that I have a particular use case in mind but clearly it could suit noob avatars with negligible prim-handling skills, albeit that actual rezzing would most likely be conducted by a more experienced user. Setting up student avatars would be another application.

Look for me in Second Life next year

Not a good week for Linden Lab (LL), the company that operates Second Life (SL). It underwent a dramatic restructuring, letting go roughly a third of its staff and closing its off-shore operations. Staff lost included the two remaining edu Lindens. It must be doubly difficult to give up a job with such a strong visionary component. One can only wish them well.

The cause of this upheaval is largely unknown though blogger Tateru Nino believes it to be due in part to a cessation in growth of the internal money supply (Linden Lab sells internal L$ for real money but leaves it to residents to convert it back). Some of the comments to her article suggest that this may have coincided with the switch to the new search engine which has had teething problems. The true economic state of SL is not easy to determine but some people renting out land and running shops report that they are doing surprisingly well given the state of the external economy. From an education perspective, a tweet from the recent NMC conference indicated that there are now more than 800 universities with a presence inworld, albeit that the size and role of the presence may vary widely. The campuses comprising the University of Texas System are mostly keeping 1-2 of the initial three sims allocated to each during the recent one year pilot, a very positive sign. Likewise, universities in the UK that have adopted virtual worlds centrally report steady growth in demand from faculty as they see from their peers what is possible.

Some educators have, however, been nonplussed by the change in focus towards consumers brought about by the LL restructuring, not least because they believe education is a significant source of free (and positive) advertising for SL in the media. They are also concerned that content developed inworld using full-perms but third-party content may not be legally exported from SL for use in other virtual worlds, most notably those based on OpenSim (you can export content created exclusively by yourself, albeit that you may need to do some manual configuration subsequently). This has consequences both for backup (as required by many grants) and for possible porting to OpenSim in the future. One compromise at present may be to develop on standalone OpenSim and import into SL. On the other hand it remains to be seen to what extent the OpenSim code diverges from SL compatibility and, indeed, ultimately varies between OpenSim grids. The popular new Inworldz grid, for example, makes some play on the fact that it is aiming to be a half-way house between SL and OpenSim.

On the other hand there is also concern that OpenSim grids themselves are not yet ready for intensive class use when compared to SL. This is reinforced by the fact that companies such as Reaction Grid and Rezzable are developing web-based virtual world viewers using the install-free Unity3D plugin (I believe it still needs to be downloaded but the installation is transparent).

Japanese company 3Di possibly had the first implementation of a web-based OpenSim viewer. This may be the same Rei viewer described in a recent paper on the use of ScienceSim for modelling protein folding (among other things). The authors report problems with dynamic updating of structures comprising more than 1000 prims and the new viewer has thus been introduced as it supports non-prim bonds. In the paper 3Di also espouse an interest in supporting multiple simultaneous views onto a virtual world. This is something that is likely to be of interest to the reconfigured LL with its change to a (much simpler) web-based viewer as an option for SL.

In a recent WSJ interview following the layoffs, the LL CEO “Kingdon says the company will soon release an app for Second Life on iPhones and iPads”. It is not clear whether that will deliver functionality as good as or better than, say, the text-based Radegast or any of the existing phone apps. It seems likely that the focus at that level will continue to be on simplifying the experience for new users. Viewer 2, the open source Snowglobe and its variants will evolve to support more specialist options such as building. It is interesting to see the focus on the iPad given the investment made by Blackboard on the same platform and mobile devices in general. However, one can only assume that other operating systems will be supported in due course.

While some educators are concerned at a possible shift in focus from business and education, others see it as an opportunity (as do some but not other commentators more generally). The nature of the change remains to be seen but I can’t help thinking that the model is directed towards shops and venues, with a little simulation game- or role-play thrown in. Perhaps it will be fronted by an xstreet (or curated app store) for experiences, sort of a multi-channel interactive TV with avatars, basically repackaging what already exists and creating a market for game designers, shopkeepers, creatives and venue hosts. This presupposes the dynamic multiple simultaneous views mentioned by the Rei developers and also by Grace McDunnough (she calls them "sneak peeks") and, in similar vein, by Eightball Magic.

Some educators have recognised that the "cheese" has moved and are determined to chase it. Others are sceptical that it will generate significant revenue although the LL intent towards social media suggests that advertising may also feature. Presumably this might also involve deploying virtual worlds into platforms such as Blackboard and Facebook (where Metaplace has been notably successful lately) which may have some value.

The broader issues facing educators in SL remain to be fully worked out but senior management from the Lab have belatedly left statements on the SLED list this week undertaking to update educators as developments and detail emerge over the next few weeks.

Personally, my premium account and region tier (met for the coming year from my personal academic account) are both due and I will renew. It’s not so much the content I’ve developed, collected and purchased but rather, for all its vagaries, the fact that SL is still the most stable and best-featured virtual world at the present time with the largest edu community. While those with alternative agenda may be looking elsewhere (and encouraging others in the same direction), I have to start planning for next year now, albeit with an eye to the medium-term future as well. Second Life is where I plan to be for now.

Uncle D: a gallant loser

With the benefit of hindsight, we know now that the Tech Virtual won this year's Linden Prize and is a very worthy victor. That said, I spent a little time at the weekend visiting one of the unsuccessful finalists, Karuna and StoryQuest. In fact, I spent most time not on Karuna but on the two Story sims. I found these highly engaging and became quite absorbed investigating the various builds, constructing a mental image of their subject, the late Uncle D.

The range of interactivity and general resourcefulness reminded me a little of some of the Literature Alive! builds of yore. I was, however, saddened to see that some of the adjunct sites, a sky platform and a couple of New York City builds, appeared to have gone. I can understand this; it's been a while since the Uncle D StoryQuest was performed and tier needs to be paid (though the teleport locations are still empty and presumably generate traffic for the sim owner).

Still, I was intrigued to find a determinedly hyper-real anti-retroviral display (creator: Hiro Sheridan) amid the low-key travelogue, education and poetry builds. I have it in mind to add some cultural references to the Cityscape level of the TB build though could not begin to match the sophistication shown by the StoryQuesters. It's marvellous that teachers can find inspiration like this so readily. Credit to the Lindens too for instigating the Prize last year.