By Harry Baya, [ firstname.lastname@example.org]
Right click here to download an MS Word version of this document
Sam was seven years old. He was the youngest of three brothers and pushed himself hard to learn the things the others knew. Usually Sam plunged right into the middle of whatever they were doing - like climbing a tree or setting up a computer bulletin board system, but this time he was on his own. He was secretly working on a project that his brothers, who were away on a three-day school trip, knew nothing about.
Sam was learning and using a computer application called "Harry's Boppers". He had spent over 8 hours on the computer on Saturday and Sunday and was now 4 hours into his project on Monday. He wanted to finish before his brothers returned at 9:00 that evening. Though he knew they would almost certainly do better than him ("beat him at it" was the way Sam thought of it) once they learned the new software - he would be the first to use an application they would love and he would get a kick out of introducing it to them.
Sam knew Harry, of "Harry's Boppers" through a computer bulletin board (The Pumpkin BBS) that Harry ran in Hastings, New York, where both of them lived. They had never actually met but often exchanged notes and occasionally spoke on the phone. Harry was 53, divorced, and lived alone. Computers were his hobby and and his work and "Harry's Boppers" was somewhat important to him.
Harry had put the application files up on his BBS on Friday morning and Sam had left his computer running Friday night downloading them to his hard disk. Though Harry had described "Harry's Boppers" to Sam in a message, Sam had only a vague idea of what the application was for. However, as he ran the first demo "vidtrack", Sam quickly understood the purpose of the application and was excited about working with the new software.
Now, after some 20 hours of work with the software, he was finishing his first vidtrack. Sam replayed the vidtrack he was creating and watched the sequence of events. He didn't play the music on the cassette player beside him, but the inch-high sound bar along the bottom of the screen was so familiar to him by now that he could quickly spot any part of the vidtrak that didn't line up with the music the way he wanted.
Out onto the screen marched three small robots. They were bright green and their chunky bodies closely resembled the old "Rockem-Sockem" robots Sam still saw occasionally at garage sales, and had played with a few times. Sam once again wondered why Harry had chosen the name "Harry's Boppers" rather than something with the word "Robot".
The small green robots broke formation; remaining in step each one followed a pattern of steps that Sam had choreographed for them. They were chunky clumsy looking robots and their movements were far from smooth. Sam hoped that "Harry's Boppers" would support smoother movements in the future, but this would do for now.
This part of the vidtrack lined up with the part of the music before the voices came in, and Sam had used each of these Robots to reflect a part of the music he could separate from the others in his mind. He watched carefully to see that each footstep of the robots lined up with the blip in the sound bar that indicated a drum beat, or pulse, in the music.
Though the later part of the vidtrack was much more complicated - with different colored robots working together in small groups following complex patterns, Sam was particularly pleased with the opening sequence. The synchronization of the robots with the music and the pattern that each followed created an almost hypnotic involvement for Sam when the music was playing and he guessed that he had done his best work here.
The screen showed the robot's stage as if Sam was looking down on it from the balcony of a theater. The robots grew smaller as they moved away from the viewer and this gave the feeling of depth. Sam had chosen this perspective over several offered in the program and had decided to use it throughout his vidtrack.
Now violet colored robots had begun to enter the stage from both sides. Their steps were small and the movements of their arms, legs and heads were all precisely synchronized. Sam knew that this lined up with the vocal chorus coming into the song. There were six violet robots on each side and after a sequence of steps and turns they formed a circular line on each side of the stage, from the front corners of the screen toward the back center, with a gap in the middle rear. From the back of the stage came 6 more robots. They were coming from far away and slowly grew larger as they marched forward.
At this point Sam lost, or let go of, his association with the music, as implied by the sound bar at the bottom of the screen. He had checked the synchronization here many times and was sure it was right.
He was entirely focused on the sequence of motion on the colored screen and his mind kept presenting alternatives. He had spent a fair amount of time developing the sequence of moves of each of the 21 Robots now on the screen and felt that what he had was fairly good. The robots from the rear were now filling in the back part of the stage so that a semicircle of robots went from the front left corner, around the back, and up to the front right corner.
Sam's concern at the moment was with the colors of the robots. Were they too bright? Should there be more colors? Should he use more surface patterns on the robots?
The green and violet robots were uniformly colored from head to toe. Sam liked the colors but was leaning toward muting them a bit so that they looked more like pastel chalk and less like shiny plastic. He might try that in a few minutes.
Sam watched the six robots who had come up from the back of the screen to complete the semicircle. Unlike the other robots these were not uniformly colored. All six seemed to be wearing skin tight suits made of a patterned cloth. Sam had constructed a pattern of small squares, each one about 1/4th the size of the robot's fat little heads, and had used four different squares in designing the pattern. Three of the squares were simply a solid color (green, violet and blue), but the fourth square was white and had the three capital letters " S A M " clearly written, diagonally, on it. The pattern made these robots look a little like the jokers in playing cards.
Sam had just learned how to use this feature in the program and this was the first time he had watched the rear robots enter the scene in their new costumes. Up until then they had been solid blue. Sam had spent close to an hour learning and using this feature to design the squared pattern - trying different size squares, colors and patterns in the white square.
Though he rather liked the appearance of each of the new robots as individuals he was uncomfortable with the impact of the new clothes on the overall scene. The new robots were now moving through a dance like interaction with the other robots in the semicircle. so that they were more or less evenly spread around the circle. Their dance continued to move them into different patterns Sam had chosen, such that sometimes no two similar robots were side by side - and sometimes two, three, or even four similar robots would be beside each other.
What bothered Sam was that there was already so much happening on the screen that the new patterned robots seemed more confusing than pleasing. He had liked it better when they were solid blue. He would set them back to blue and save the square pattern for future use.
The vidtrack continued and Sam watched the various dances and groupings of robots with pleasure. Even without the sound of the music there was a rhythm and a development that gave Sam the kind of pleasure that a musician gets from hearing a melody he or she has created and liked. The process was indeed similar and Sam had spent about 1/3 of his time in this project designing and trying out different sequences of movements and choosing the ones he liked best.
The other two thirds of Sam's time had been spent on learning what the application could do and how to do it.
It was clear from the start that the program would be used to create an animated sequence that would be synchronized with music played on a separate sound source, such as a CD player or a cassette. The music would be picked up by the computer microphone and converted to the sound bar playing along the bottom of the screen. The sound bar data, and a very weak recording of the music, would be stored in the vidtrack data-file and was used to synchronize the animated events with the external music. A "performance" of the vidtrack would consist of playing the external source and the vidtrack at the same time. The system would "hear" the external music and keep the vidtrack in synch.
The only part of the application that seemed tricky to Sam was the ability of the program to play a vidtrack and keep it synchronized to the external sound source. Sam knew from playing around with cassettes that two tape recorders playing the same recording side by side would get out of synch after a minute or two - so obviously "Harry's Boppers" could not just set the animation events to go off after fixed intervals of time. It had to "listen" to the music and adjust to the speed to minute variations. It seemed to do this quite well most of the time. Sam had to start the animation at the beginning of the song - and sometimes had to use the mouse with screen buttons to slow down or speed up the animation till it was in line - but once this was done things usually stayed in line nicely.
There was one slow, sort of unexpected, interlude near the end of the song that had confused the synchronization a couple of times and Sam had found in the documentation a way to fix that by emphasizing a particular range of frequencies during that part of the song. It had taken about an hour to find this and get it to do the job, but it worked and he was proud of solving that problem.
"Harry's Boppers" would let a user like Sam create an animated Robot dance to line up with any piece of music. Sam could pick a recording he liked a lot and, in effect, add another instrument. The new instrument was visual rather than audial and somehow reminded Sam of the pattern of laser lights he had seen played at the "Laser Light Show" at the Andrus Planetarium down the road from Hastings. The essence of both was adding a visual pattern to a piece of music. It occurred to Sam that something like "Harry's Boppers" might provide deaf people with an art form much like music.
Picking a piece of music Sam liked a lot was important for several reasons. The obvious one was that he could get involved in the parts he liked and try to complement them with the vidtrack. In some sense he felt the obligation of an artist to show proper respect and appreciation for the work of another artist - he wanted his additions to help others appreciate what he liked in the music. Another, less obvious, reason was that it was a darn good thing that Sam liked the song a lot because he had had to listen to it over and over more times than he wanted to think of.
Sam was patient and careful in his work. He did not mind going back to a vidtrack sequence he had created and modifying it, or replacing it entirely, in his quest for a more pleasing impact. Though the entire song he was using lasted just under 3 minutes, Sam had spent hours of trial and error developing the current vidtrack. Each time he mastered a new tool in the application, like making a patterned suit for a robot, he would go back and try it out in various places in his vidtrack. Sam was an artist trying out new tools and concepts - and he had the computer based advantage of being able back up to the prior version when he did not like a particular change. A sculpture working in stone can only dream of doing this.
The early part of learning the application had been fairly straightforward, though time consuming. Sam quickly learned the various postures each robot could assume. Each arm had four positions (from pointing down, through pointing straight ahead, through pointing up), and each leg had three. The head could rotate to look in 5 directions (Sam thought it was a shame that Harry had not permitted the head to rotate around to look backwards, and hoped this capacity would be added). Finally each robot could turn his upper body into three positions in relation to the hips. That was pretty easy. It was strange that the motions from one posture to another occurred instantly rather than moving smoothly from one to the other, but Sam understood that this would mean a lot more work for the computer. Sam's computer was not super fast and Harry did not claim to write particularly efficient programs.
The documentation said that this allowed each robot to assume 180 different postures (e.g.  arms & legs straight down, every thing facing forward;  arms and legs down, body and head rotated full to the right; etc. ) and Sam was comfortable with these.
The next part was more confusing. The 180 postures all assumed that the robot was starting from a standing position facing you. How about when the robot was standing facing to the side, or to the rear, or standing on his head or something? He could be in any of the 180 postures while standing on his head. That was an entirely new set of possibilities and it took Sam some time to learn how to use the tools available for rotating the entire robot along the three axes of 3 dimensional space. After playing with the various ways of doing this Sam finally settled on using a feature where three rotating dials appeared attached to the robots body and the mouse could be used to turn any dial. This allowed Sam to rotate the robot's body along any of the three axes. It was sometimes a little clumsy but through practice Sam had developed some facility with it and could usually quickly put a robot into the orientation he wanted. "Orientation" was the name used in the documentation to refer to how the robot was rotated in 3-d space. Being able to copy and paste these orientations between robots made it fairly easy to enter some of the robots activities.
Sam did not know exactly what to call the process of creating a vidtrack. It was sort of like programming, but it was also like composing, and choreographing, and like being a director of a play. He wondered if a new word would be needed to describe this activity. "Vidding"? . . The location of a robot in 3 dimensional space, as contrasted to it's orientation, was also a little difficult for Sam to master and he intuitively felt that there had to be a better way than the one Harry had chosen. All that was really being done was deciding where a robot would be on the stage, or above the stage. If they were in the front, they were bigger, in the back they were smaller and toward the center, like a perspective drawing. Sam wanted to be able to drag the robot to it's position. He could drag it left and right, or up and down, but to drag it forward and back on the screen he had to flip to a side perspective of the screen. What Sam really wanted was a 3 dimensional mouse, but he had never seen one and had to forgive Harry for this shortcoming. Sam envisioned a setting such that moving the robot up and down on the normal screen would move them forward and back in the perspective. He would suggest this to Harry.
So that was all the there really was to the main application: 180 robot "postures", the ability to rotate a robot along three axes, and the ability to move a robot around in three dimensional space. Fortunately, or unfortunately, Sam had quickly learned that this allowed a tremendous amount of flexibility and that creating even one "picture" could take a while. As the pictures could flick on to the screen as frequently as one every fifth of a second (which left the screen animation looking a little "jumpy"), a 3 minute song could require up to 900 screens. Fortunately there were many times when the screen did not change for a half a second, or more. Also, each screen usually had only minor changes from the prior one.
Sam could see that a big improvement to system would be to start with one screen and create a second screen where a number of robots had moved to different postures, positions and orientations at some later point in the music - and have the computer create the interim screens - making each on-screen move go through as many interim positions as possible. However that feature did not exist and Sam had plodded through creating the interim screens. This was particularly annoying when he was trying something that he might not keep, and he had resorted to creating screens that would appear every three seconds and envisioning how it would be when all the screens were there. He had been fairly selective about when to fill in all the interim screens.
The documentation described a feature that should do part of what Sam wanted, but he could not get it to work. Thought the program would not create transition screens between two stage scenes , it claimed to have the ability to create transitional screens for a single robot going from one "set" (position, posture, orientation) to another over a period of time. In this way Sam could create all the interim positions of each robot and paste them into his main sequence of screens. Unfortunately this feature would frequently crash the system and Sam had stopped using it. He told Harry about it on Saturday and Harry had responded on Monday that he was having difficulty fixing as he could not reproduce it.
In addition to the basics there were some other features in the program. One of these was the ability to create the patterns that robot would wear as a costume. Another was the ability to design and modify the stage on which the robots moved. Though Sam could create graphics in other applications and paste them into the stage, it was difficult to get the other application to create things that would fit appropriately in the 3-d space in which things further away were smaller. Harry had provided a few tools in the application - but they were far weaker than any decent drawing program and Sam was a little frustrated by not being able to better customize the stage.
In the end he chose to work with an almost completely bare stage - though he did sometimes use a tile floor with varying colors on the tiles. Harry had at least provided this. Sam had learned to use it sparingly as it could be disturbing when the floor was changing colors underneath the moving robots.
With the primitive tools available in this first Beta release of "Harry's Boppers" doing something like having a robot ascend an angled set of stairs was just more trouble than it was worth.
Sam reset the skin pattern of the six robots back to blue. He was beginning to feel this work was very close to completion. The last few things he had tried didn't help or were very minor improvements.
The main features of the program Sam had yet to play with were the ability to make robots of different sizes like midgets and giants, and the ability to manipulate several other objects in much the same way as robots. The other objects were restricted to balls and boxes (like cubes and shoe boxes). Sam read that he could create one or more of these objects of different sizes and colors and place them on the stage in positions and orientations much like the robots. Sam thought it might be fun to create a ball with the square patterns on it and have it rotating and bouncing across the screen. He realized that having it bounce would be a lot of work, and he didn't think it would add much to this vidtrack.
He decided to replay the vidtrak, this time with the music, one more time and decide whether it was good enough to show his brothers.
Sam felt the accompanying music was critically important to the satisfaction provided by "Harry's Boppers". On Saturday night he had been at a gathering with some friends and had tried to explain to them what he was doing. Though they seemed to understand and appreciate the idea of creating an animated vidtrack with colored robots moving in patterns, they tended to ignore the music side of it. Those who were interested felt the graphics sounded very limited - they wanted figures other than the robots, they wanted the ability to create their own figures- with far more possible postures (Sam himself wanted to create a horse to be on stage with his song). They thought of numerous other improvements for the graphics and the animation.
Sam kept trying to explain the importance of the music and he knew he was not getting through when two of his friends agreed that it would better if the system would permit a user to enter a tune of computer generated notes to accompany the robots. Sam felt strongly that few, if any users, would ever create a piece of music that would be half as involving as a an extra favorite recording. Even with a full blown midi system - the music would be limited by the ability of the creator. A large part of Sam's excitement about "Harry's Boppers" related to the fact that it allowed him to add to a piece of music he already loved. It wasn't just a means of showing his skills, it was a way of sharing music he cared about.
The process of listening to a well liked piece of music and identifying things about it that could be reflected in the vidtrack was a way of becoming involved in music at a new level. As the frustrating conversation continued Sam decided that the only way his friends might understand his view would be for them to experience his creation. Even then it might fail if the music did not catch them. What they would need was a vidtrack to one of their favorite recordings, or better they would need to create their own vidtrack.
Sam reflected that few of his friends would be willing to spend the kind of time he had on this project. The process was going to have to become a lot easier, and a lot more powerful, before it would become popular. The features his friends wanted would be needed. The ability to automatically create the transition screens, the ability to create and paint 3-d objects (like steps, trees and waterfalls) within the 3-d space of the stage would be needed.
The end product he envisioned would require a much more powerful computer than Sam had, and a much better program than "Harry's Boppers", but still, the heart of the concept was clearly defined and Sam was the first person to use the application other than Harry. Sam sensed that he was involved at the beginning of something big, and it felt good. This awareness would remain with him long after the limited thrill of showing "Harry Boppers" to his brothers, and the familiar jealousy he would feel as they mastered it. Still, he was only 7 and when he was 14 like his oldest brother, David, he would, he hoped, be far beyond where David was now. He had just begun an exciting adventure and he was looking forward to the rest of the trip.
Harry's Boppers - An Addendum, 4/20/93
So I wrote up my fantasy of 7-year-old Sam's learning and using "Harry's Boppers" and I showed it to a few people who skimmed over it and gave me their reaction.
Phooey on them! What do they know? Everyone who does not immediately recognize this as the most important innovation since microcomputers is a full blown idiot. What me defensive? No way. I'm just objective, and they are idiots.
The main criticism is that the application is going to be too complicated and too time consuming for all but a very small select few. It might have, say, as many users as there are people who use MacroMind director, but only for classical music related stuff.
The second comment is that what would really be neat would be a version of "Harry's Boppers" that was completely automatic. You play the song and it creates and plays a vidtrack.
Are the people who said these things idiots? Of course, by definition. They did not fall down at my feet and worship me for conceiving of "Harry's Boppers", and they did not leap to invest funds (i.e. give me money) in the project. So they are idiots.
However, in my Budhahood I am a kind, open, magnanimous person and It may help me to grow even more enlightened, and allow them appreciate my overwhelming modesty, if I try to deal with these comments. So here goes.
The application is going to be too complicated and too time consuming:
Defense A: It's going to get much better:
OK, so "Harry's Boppers", as described, is very primitive. It's takes time to learn and it's clumsy as the dickens. I know that. I was trying to describe something that I think I could actually create - with a lot of time and effort. It would just be the beginning. "Harry's Boppers" would relate to it's future incarnations the way S.A.M (the simulated Automated Mouth) on Apple ]['s relates to MacroMind Director, the way EDLIN (a well known DOS editor) relates to the latest PageMaker.
It would be much easier to create simple vidtracks, with lots of special affects, assisted transitions, preset dance patterns and the like when the application had evolved for a few years. Many of the things that users sense a need for will be added. New minds rowing on the development oars would think up innovations that stretch the envelope further and further out - in ease of use where possible, in range of interesting features, in ability to assist a user in finding, attaching and using new extensions of the basic application.
I suppose it would be useful to do a little brainstorming on the kinds of things that I think might turn up as this concept evolves. I will do that, but not right now. However, I like to think that really wonderful things will turn up that I can't even imagine. If I can ever get a group together who share some of this vision with me, I would love to do some brainstorming with them.
Defense B: Those drawn to this sort of thing will take the time to master it.
About 10 years ago I decided to learn to play the violin. Actually I was aiming for Irish fiddle and "Old Timey" American - but the initial skills were the same. Wow was that hard. Learning to read music, learning how use the bow to get a sound I and my teacher liked, learning how to fret the strings, learning to hear when I was playing the right note, and even learning how to tune the silly thing, were absurdly time consuming. I pored thousands of hours into it and made a little progress - and then I backed off.
Yet there are thousands, perhaps millions, of happy fiddlers in the world cranking out beautiful music and having a glorious time with it. So it will be, I think, with the masters of the instrument implied by "Harry's Boppers". They will be adding a new component to all music and it may be as beautiful as any that is there now. There may be different specialized instruments evolving into being, as there has with sound, and each will have their masters, their idioms, all the parallels to sound, and maybe new things that have no counterpart in the world of sound and sound related instruments.
The door is just opening. I see it as immense and I wait in awe for what may wander through even in the limited number of years I will be around to watch. I want to start the show as soon as possible so I can see as much as possible.
Defense C: It may be hard to master, but it might be easy to have fun with.
Not only that, what about all the people who play instruments just for fun, just to have a good time with other musicians. They aren't masters, they didn't spend thousands of hours learning the instrument. They spent some time and got good enough to enjoy it and they do it sometimes and get a kick out of it - maybe when they are 7 playing a tune for grandma, and maybe when they are 80 playing the piano for friends to sing with at the old age home. So it will be, say I, with the descendants of "Harry's Boppers".
Why can't these idiots see that!
A version of "Harry's Boppers" that was completely automatic.
Oh sure, a computer based lava lamp. An updated "Lumina" from the museum of modern art.
How disgusting. Passive viewing, no art, just stimulus response.. only... I WANT one, and I want it now.
This could be a big part of the action. People could design new extensions to the auto-response library. Setting switches and picking extensions to go with a particular song could become an art in itself.
"Hmmm, I want a lot of cloud like stuff for this song, a lot of really nice soothing patterns, and make it heavy on the blue and violet, please. I want some big things building up, a sense of growth and I want a nice climax around the time the song has it big moment. You figure it out. Maybe give me a couple of starts and I'll tell you which one I like."
Or - "Bring in the old robots, March them around. Make it really old timey, like the early 'Harry's Robots'."
I can see having a lot of random selections going on, so that the second time through with the same settings it would be similar, but not the same. However, any given play through would save a script of what happened, so you could play it back exactly the same if you wanted to.
Again, there is tremendous room for innovation here. I can do a little daydreaming about the kind things a program could do in creating interesting vidtracks to be automatically generated to go with a recorded piece of music. I am particularly intrigued with the idea of a listener pushing levers and buttons to vary the process while it is underway. However, the really interesting part is to think what others might come up with along this line. Given the underlying ability of "Harry's Boppers", or it's descendant, to create a wide variety of visual events synchronized to music, innovators could have a field day for many years.
So, why isn't all this obvious to everyone. There are only two possibilities. Either they are idiots or I am.
Hmmm, I think I'll think about that for a while.
Harry Baya--- The bulk of this document was written on 4/20/93 If and when I seriously pursure creation of this application, proposals, outlines, descriptions etc. will need to be email@example.com
This document last updated: 10-30-2016