KIB311

All in 1 SWF

2006-10-14T07:32:00.000-07:00

All 3 different versions in 1 SWF.

Source.

There's no "back to start" button. So, we'll just have to refresh or reload the file to go back to the menu. :)

Completed, working as intended.

2006-10-13T00:35:00.000-07:00

Type 1 (Gordon mode) - moving background, still pointer
The SWF
The source

Type 2 (Harry mode) - no pointer, rotating highlighter
The SWF
The source

Type 3 (Shane mode) - still background, moving pointer
The SWF
The source

Heh, guess "tonight" meant 2 days later. Kinda forgot to send the source file to myself. Had to go back to uni to pick it up. Haha.
Do we want to add little boops and beeps to go with the voices?

Type 2 (Harry mode) took a while to edit, as it needed a whole code revamp (basically I deleted everything and rewrote it again since the mechanics/triggers for this version is completely different). It's still a little buggy because there isn't any "pause" function or variable to stop the highlighter from rotating. Still, fiddling around. Will update tomorrow.

Test working prototype

2006-10-09T19:22:00.000-07:00

Click

It's being auto stretch to fit the screen. The original size is 640x480

Source

Development note:

I did try to animate it but it looked crappy as hell. Simply because I used keyframing to animate, which made the screen jump back and forth when you jump 1 frame to another. I did try animating it with actionscript but it doesn't seem to work. No idea why. Still fiddling with it.

Exploring Assistive Technologies

2006-10-03T21:46:00.000-07:00

Moving from a device that's purely "fun" to work with, to something that has real value and applicability, I've been considering applying the sound-activated cursor mechanism for use in existing assistive technology.

Severe sufferers of cerebral palsy, without voluntary limb, head or tongue movement, have a tough time communicating. Often, they are only able to control their speech using a series of grunt-like noises. The Sing-a-Sketch mechanism could feasibly be adapted for use by a sufferer of cerebral palsy, who only has the ability to control the duration of vocal noises.

By attaching a bluetooth-enabled headset to their ear, and synchronizing their noises with a cursor moving on screen, a user can navigate through a "switching" communication device driven by software, which is then able to "speak" the sentence to carers.

While the existing software for assistive technology is very comprehensive, users are still required to have significant control over particular parts of their body, with input devices designed for finger movement only, tongue and head movement being particularly common.

Even eye-movement readers are inappropriate for particular cases of cerebral palsy, as the sufferer often has limited control over the position of their head.

The portability of the Bluetooth protocol would allow for light-weight software / hardware pairing with the headset, allowing something as small as a PDA to be able to interpret vocal input into spoken or written messages by both the user and the carer. A carer could conceivably be sent SMS-like messages from the user who transmits textual translations to a web-enabled PDA.

The ever-moving cursor in Sing-A-Sketch then, would have to be adapted to continually be at the centre of the screen, with only the translation icons and commonly-used buttons scrolling underneath in the direction dictated by the user.

Once certain 'dead zones' (blue in the diagram) in the screen are overcome (allowing for both a sound and interaction threshold), the screen is able to "switch" to the next-lowest branch in the heirarchical tree of iconography, while keeping the cursor at the centre of the screen.

Once the sentence or phrase is complete, the user can navigate to the ever present "speak" or "cancel" buttons to complete the interaction. "Yes" and "No" buttons are omnipresent, and navigating up and down the tree uses a consistent interface.

The screen would "switch" between a series of four options, while also keeping the most-recently selected option in view, allowing a user to navigate both up and down the heirarchy.
Selecting 1, for example, switches to the screen surrounded by green, where the next available options are 5, 6, 7 and, to navigate back, 1.

Sing-a-sketch eraser tool

2006-09-14T03:02:00.000-07:00

Hey,

Just got an idea on how to erase the scribbles on the canvas. We could use a motion blur detection code that Director could do (Director can capture motion, and we can add some sort of extra code to detect how fast a certain colour pixel from the frame moves back and forth). By doing so, our players can erase their scribbles by first capturing a video clip of the player waving/moving his/her hand (mobile phone) and then sending it to our system. It would be more ideal to have a direct stream of the video capture rather than having to capture > save > send. I'll try to look into this further to see if streaming directly from the phone is feasible. I'm pretty sure it can be done. It's just, I don't know how - yet.

Reason behind this idea was looking back to the original etch-a-sketch and how we erased our lines. We shaked it till everything got erased, and using the same concept of "shaking to erase" for our project.

Textual Input via Microphone

2006-09-05T21:01:00.000-07:00

Not voice recognition (far too complex), but MORSE recognition *gasp*!

Not unlike these attempts:

Perhaps this could be a means of having users input textual information to provide their signature, etc., particularly if there was an on-screen morse alphabet as a legend.

Double Reverse Stencil

2006-08-29T09:27:00.000-07:00

It's done. However, I couldn't make the dots go in a more realistic way. They're just moving from top to bottom and bottom to top and left to right. Adding anymore dots will cause Flash to lag REALLY bad.

So, without further ado...

Clicky

Oh, here's another just for fun...see if you can spot the difference.

Clicky 2

And finally, the source files.

Clicky 3

Saving this link for the presentation. Just to show an example of how our idea works.

Vidoe Assessment - Evolution of an Idea

2006-08-28T22:19:00.000-07:00

Wanting to differentiate ourselves from Rokeby's work, we observed a [possibly unintended] stencilling effect in his work Seen - where the static objects in the foreground of a scene quickly stand out once substantial movement occurs behind those objects.

Exploring this concept further, we realised that the patterns created by movement in a scene could also be used to reveal a "hidden" image in the installations projection - by making this hidden image 2-bit greyscale negative, the classic "afterimage" optical illusion could be created and gradually revealed, then hidden again, as the day (and its associated motion) progressed.

Hence, we drew further inspiration from the stencil graffiti movement, as well as the optical illusion crowd. Each motion though a given scene is noted and quickly 'fades in' behind the stencil, then changes size and/or opacity slowly over time to fade away. The secret stencil, then, may only be revealed during periods of heavy pedestrian traffic, whilst simultaneously being susceptible to the recent phenomenon known as "flash mobbing".

Video Assessment - Initial Concept

2006-08-28T21:11:00.000-07:00

Next week feels a lot like a "placeholder" assessment piece, IMO. The challenge is in proposing a video interaction piece that DOESN'T feel like a cheap EyeToy knock-off or lame-o real-time filter effects.

My discussions with Ben (our tutor) suggest that, even in 15 years of R&D, very little new applications have been designed or developed, merely that the technology has become more affordable for the consumer (cheap web-cams, etc.).

I'm proposing to attempt something reasonably simple that doesn't just feel like an "art for art's sake" project:

[drum roll...]

>>> Temporal Pattern Analyser <<<

Much like a long-exposure photograph, this web-cam enabled installation tracks clustered movement in a public space, with each movement leaving an ever-so-slight outline (ghosting) for each second that they are in motion. These movements can be highly stylised in the camera's interpretation and representation on-screen, but should ultimately allow the viewer to analyse where in a particular scene users tend to congregate (or skip right through). Commercial application would be most viable in the retail industry, where shop-owners could use the tool to gauge where potential customers are drawn to in their shop layouts, without the fear of privacy violations.

Not entirely dissimilar to this installation:

http://homepage.mac.com/davidrokeby/seen.html

So, the theory and application of this could be reasonably easy generated, as would the context of the installation (large, open public space). I imagine that the use of the WebCamXtra for director would be able to continually write on a per-pixel basis to the screen based on movement detected by the camera.

Just a thought - I'll have to wait until I get home to have a crack at the software side of things, but I'm very open to suggestions and/or assistance.

Presentation 2

2006-08-28T19:13:00.000-07:00

Harry does the text. Gordon does the piccies. Shane does the demo. Sweet.

Download PPT

It Works (The Way We Want)!

2006-08-25T22:06:00.000-07:00

Awesome, awesome job. Got to borrow a Bluetooth headset off a mate, and, after the pairing process with my own generic Bluetooth dongle plugged into the PC, we were on our way.

The only thing required to allow BT-enabled audio was to set the BT-hardware as the default input device - with further tweaking, we could produce some audio feedback (transmitted direct to the headset), but for now things are very sweet - users bring their own headset, pair it to the installation, and sing along! Nice.

Feasiblility? Check!

Technical Feasibility: Lose the Mobile Phone

2006-08-21T20:08:00.000-07:00

After discussing possibilities with several electronics, computing, communication, hardware and audio specialist dudes, I came to the conclusion that the only way to use the mic on a mobile to communicate with our Bluetooth dongle is to write a custom app that offers our desired functionality.

Instead, I believe that the use of Bluetooth-capable headset devices (which are becoming more and more prevalent) would be far more feasible. This particular device from Blueant (http://www.blueant.com.au/X3micro_Bluetooth_Headset_voice_over_ip_voip.htm) would do the trick nicely:

When connected to a computer, the X3 micro becomes the default Sound playback device so that any program which utilises the Soundcard can send and receive audio signals from to the headset.

Perfect! That sounds like us! Now to find one to experiment with...

Mechanics Q&A

2006-08-08T00:35:00.000-07:00

What's needed?

BT transmitter/receiver
mobile phones with BT and camera equipped.
LCD/projector
open space (10m) with a wall.

How it works?

Sing/make noises into the BT input (mobile) to draw shapes and on screen
A rotating pointer will keep rotating till the user (or rather the user's BT device) receives sound/noise input that is above the preset threshold.
The moment it receives sound/noise, the pointer will stop rotating and a line/shape will be drawn on the screen according to the direction of the pointer.

Feasible?
yes.

Why?

BT chips cost only 3 USD. It's relatively cheap.
we only need a small space (10m)
It can practically be implemented anywhere. Train station? Bus stops? shopping malls? Karaoke spots? QUT?

Problems?

Noise pollution. Having 10 people singing or making weird noises or just screaming their lungs out will no doubt produce nice pollution.

Can we over see it?

Maybe, because it's an open public space, and not a library or seminar hall.

Anything else?

There is also another problem about whether the environment noises will interfere with the BT input device and create shapes unwanted by the user.

Solution?

Increase the threshold of the sound input so that it's only sensitive to noises that are 5-8cm away from the mic. Noises that are further than 5-8cm will be filtered out. (about 65dBA* and above will be accepted)

*65dBA is roughly inbetween the sound level of an average conversation and a vacumm cleaner 1m away.

So, basically this project does not support multi-play?

Wrong, this project supports multi-play with a capital M.

But the sound input is only limited to 5-8cm?

5-8cms is hardly a way to limit 2 persons to sing into 1 mic. Also, we’ve stated 5-8cm isn’t because we’re limiting the users to be exactly 5-8cms away from the mic for it to receive input. We’re stating that because of the common sound decay of an average voice. If you place your mic, say, 50 cm away from your mouth and start singing. Your voice (sound vibrations) will decay over time and space and might not be picked up by the mic (because of the high threshold).

Also, the screen will display multiple multi-play elements that we will discuss further in the future.

Why the high threshold?

It is because we want to eliminate all possible environmental noises that could influence the user's controls. External influences that goes against the user’s will may or may not create user frustration. We’re playing it safe to not assume that the users may not get frustrated. By using this solution, we are also not straying too far away from the common user perception on how to use a microphone.

Will cost be a factor?

For the developers, slightly. For the users, hardly a problem.

Why?

This is because the developers have cash out to purchase either a projector or a huge LCD screen to display the user's graffiti. They also need to provide the BT receives.

Projector or LCD screen, which is better?

They both have pros and cons.

Projector pros:

Cheaper
Screen size depends on how bid the wall is and can be readjusted to focus

Projector Cons:

May be hard to see during the day or bright areas

LCD pros:

Crystal clear throughout the whole day.

LCD cons:

Expensive
Screen limited to it’s frame.

Will users actually scream their lungs out? Would they feel humiliated?

Maybe, but we will hope to produce enough curiosity to hope to lure users into playing our "game"

How will we create curiosity?

We can either create a random generated text on the screen whenever there's no one within the radius playing. Text that says "PLAY ME!" or something similar. Also, leaving the shapes and lines that the previous user drew on the screen along with his/her BT username.

We can also send messages to all BT mobiles that crosses our 10m radius to inform the by passer on what sing-a-sketch is as well as the instructions of playing.

-----

Note: If there's anything else you want to add (more unforseen problems, or maybe using a different display mode approach, lack of info in a certain area, etc), email me or reply with the "comment". Thanks.

Edit: Hmm, just saw the PPT slides. I was initially thinking of only limiting the BT radius to 10m. This is because, say we have a 50"-80" display. Users wont be able to see the screen clearly if he/she is 50-100m away. Well, unless they have really good eyesight.

Edit2: There's actually a reading/article somewhere that states that using common ways/icons to do something will decrease user frustration because the users already have some sort of familarity, which shortens the learning curve greatly. I'm looking into it now.

PPT Presentation Work-In-Progress

2006-08-07T20:17:00.000-07:00

Now available for your perusal:

http://student.ci.qut.edu.au/~n4661117/kib311/Presentation1.ppt

Feedback and comments welcome. This is a dot-point summary of most of my prior notes, with fancy additions for other users (i.e. you guys).

Etch-A-Sketch Prototype

2006-08-07T16:43:00.000-07:00

Complete with rotating arrow and pretty line, visit my university webspace to see our Etch-A-Sketch in action!!!

Presentation Images - First 3 Slides

2006-08-07T04:00:00.000-07:00

Welcome to "Sing-a-Sketch"!

Sing-a-Sketch is a unique conglomeration of a variety of familiar toys and technologies, fused in an unconventional way and projected on an unexpected location. Sing-A-Sketch relies on everyday users' sense of play, allowing simultaneous or iterative collaborations of community- created artwork, and, by way of vocal interaction, spontaneous performance art.

Users familiar with the SingStar series of games for the PS2 may recognize the voice-activated game mechanic used here, but that's largely where the similarity ends. Children who grew up with the classic "Etch-A-Sketch" toy may also feel at home, but the privacy of their creations is no longer limited to a friend nearby to whom they can pass the sketchpad.

Graffiti artists have long used public space to make a statement, or to add to existing statements created by earlier artists. The final result never fully solidifies: the space is consistently added to, refined, roughened, or simply painted over. The space itself is as fragile and as engaging as an Etch-a-Sketch.

Mobile phone users have long had gaming facilities on their devices - some of which even have the facility to play against other users. Mobile users are also very familiar with the voice-driven nature of the phone's primary use, and will no doubt be comfortable speaking into the device.

Using their voice with their phone to control their contribution to collaborative community artwork is the key mechanism of play in this piece. Users are invited to pair their Bluetooth-enabled devices with [possibly one of several] Bluetooth audio receivers. These receivers then take the users' sound input as a single button device to control the direction and speed of the users' pens, represented and projected onto a suitably flat public surface.

Without further instruction, users could potentially play with the interface: some may feel they need to sing to the device a particular song - others may sense that they have to move around the device's broadcast radius. Still others may invite their friends to help them control their pen. Users may even unknowingly influence each others' pens if their voices carry far enough.

Similar Projects - Worth Citing for Reference

2006-08-04T19:31:00.000-07:00

We Make Money, Not Art:
http://www.we-make-money-not-art.com/archives/cat_locative.php

Phillip Pocock (pioneer in the internet / installation artwork form):
http://en.wikipedia.org/wiki/Philip_Pocock

Node London (funky interactive artworks, using a variety of input medium)
http://nodel.org/projects.php?type=67

Public-Access Bluetooth Work:
http://www.orbit.zkm.de/?q=node/412

Mechanics

2006-08-03T20:18:00.000-07:00

Is it feasible?

Yes, but limited. The Bluetooth device is only limited to at most 8 people (1 master, 7 slave), assuming we have only 1 master (based on wikipedia). However, if we're limiting our space to just a 10m radius, having 7 people in the radius is quite a squeeze. However, there's a higher class Bluetooth radius that can receive Bluetooth transmission within a 100m radius. I don't think that would be feasible unless our screen is as big as an IMAX theatre.

There are several Bluetooth receivers/transmitters already released. No problems there.

Now, for the feasibility of filtering out the 6 other users when there are 7 people in the 10m radius. There is a few ways; however it would require more looking into. The use of sound signatures to determine each user's voice and separate them from each other (sounds really complex). Or the simply increasing the threshold of the sound wave so that it does not pick up sounds other than the sound coming from 5-8 cm from the Bluetooth input. The dBA table of sounds can be found here.

Also, will it be better if more than 1 person sang into the same Bluetooth input? No idea, we would need to test it out on how it would change the line output on the screen/wall. I'm assuming if we have 2 or more people screaming into the input, we would get a higher dBA which would create a thicker line that 1 person would never be able to achieve.

Here’s an interesting project I found that uses sound to draw lines. Difference is, it’s pre-programmed to draw the shapes when a pitch/note gets hit.

Also, I looked into singstar and found out that they only measures the pitch of the voice/sound. Meaning, the player can just sing a note in any octave and still score points.

As for "letting the world know" about our project, we could just set it up somewhere and spam everyone's phone's with messages about our project as they walk by. Something like "You have just entered the sing-a-sketch zone. Start singing!"

One-Button Games

2006-08-02T00:01:00.000-07:00

Easily one of the most fun I've had with one button.

http://www.miniglobz.com/games/twinspin3_en.html

Operates on a similar principal - constant rotation with a trigger to move in that rotation for a fixed / variable period of time.

Job Delegation

2006-08-01T23:46:00.000-07:00

Concept
(Gordon) - Describe how the project uses the concepts of play, graffiti and collaborative community artwork as experimental process.

Context
Participation in the "Sing-a-Sketch" project requires a BlueTooth-compatible mobile phone. Users 'sing' into the phone whilst remaining within a receiver's radius (typically 10-20m) to control a pen projected onto a nearby flat surface (ceiling, wall or floor).

The ideal environment would be large, open and urban, with one or more large structures nearby. Projection results would be most effective at dusk or during the evening.

Visualisation
Go HARRY!

Pen is represented as an arrow - leaving either a solid trail, or a semi-transparent trail of simple shapes.

Pen's "direction" rotates constantly while sound input levels remain below a given threshold. Pen moves in set direction when user makes a constant sound. Speed of pen or width of pen can be determined by either a variation in pitch or volume (most likely volume).

Pen becomes stationary again once sound level drops below threshold, rotating its direction.

Will be worthwhile taking photos of potential locations for installation, then adding "artist's impression" type mockups of projection and users singing into their mobiles.

Mechanics
Go SHANE!!
Check out the feasibility of such a device - requiring bluetooth audio receivers, feasibility of system handling multiple simultaneous audio inputs. Examine range of receivers. Any hardware incompatibilities with USB-based audio devices.

Explore how users become aware of the "Sing-a-Sketch" project. Online help required?