Claire

Open Sourcing the Origin Stories: The ml5.js Model and Data Provenance Project

p5 sketch

https://editor.p5js.org/Claire_Z/sketches/cH9BnLCcp

I call this project “Take your umbrella”. It’s a rainy day, raise up your hands to take out the umbrella.

I use both BodyPose and BodySegmentation in this p5 sketch. I use BodySegmentation to mask the background and replace it with a rainy video. Then I use BodyPose to detect the left or right wrist. If the confidence of either wrist is greater than 0.3, the umbrella will appear. And I prepared two piece of audio. One is raining sound, and the other one is the sound of rain on umbrella. It begins with the plain rain sound, when you hold the umbrella, the audio will also changed to the rain on umbrella.

image.png

rain.mp3

image.png

rain on umbrella.mp3

image.png

image.png

Since there is no points for hands in BodyPost, I use the position of wrists as the position for the umbrella, and shift it to make the umbrella looks like be holding on your hand.

image.png

But I still have an unsolved problem. In the first place, I wanted to use a picture of umbrella with transparent background. But when I preload the image, the program says failed to fetch. The image is available and the path I think is also correct. So finally, I use text and put an emoji umbrella there.