View on GitHub

mt2-blog

Music and Tech 2 blog - Spring 2020

Music and Tech 2 Blog

The purpose of this ‘blog’ is to track the progress of my PoseNet based gesture classifier and audio engine.

Checkout my project at https://github.com/seanerice/chuck-sta-pose

Week of 2.17.2020

What’s going on?

Last week I remebered I had no way of labeling training data. Neural networks are really good at learning an output from a given input. In unsupervised learning, an ML model gradually learns what features produce what output after showing it numerous (input, output) pairs. In my case I have a lot of input and no output, so I can’t train my network!!! Through the process of data annotation, however, I can create output data which represents the outputs I want the network to learn.

Creating training data

I have a thousands of training data which needs to be labeled. The hardest method (and most fine-grain) involves going through each point one-by-one and creating annotations. I can speed up class annotation when I label ranges of data as one class. For example, if my right arm is moving up over a period of time, I can label that region of data with r_arm_raise. Similarly, with parameter data, my arm may start at a height of 0.3 and stop at height 0.8; it’s just a matter of interpoating between those points. Using this annotation model I can significantly speed up data annotation. Now I just have to build it.

Data annotation tool

I want to use Unity3D to quickly build an annotation tool. I believe this is feasible within a one-week timeframe. Here’s the deliverables:

Ideas

The annotation tool could become a spring-board for a visualizer which does interesting things with the data. Maybe I could extrude the 2d point data into the time dimension for a cool slithery stickman-snake. Maybe I can plot one dimension of data against time and annotate that way.

To-Do This Week

Week of 2.10.2020

To-Do This Week

Ideas

To-Do Next Week

Thoughts

I’m worried about the feasibility of annotating data, however well annotated data is useful for training.

Week of 2.3.2020

Progress

Week of 1.27.2020

Planning

Week of 1.20.2020

Planning

Progress

What’s slowing me down

Next steps

Week of 1.13.2020

I want to pick up where I left off in MT1 – build an interactive piece using pose tracking and classification. This will likely involve building a custom RNN along with creating lots of good training data.

Progress

What’s slowing me down

Next steps