Code process
After getting a prototype of the box and making two quick sketches of patterns, I tested the connection between Processing and Wekinator. The main objective was to train the different patterns to match a specific position in the Processing sketch. This step requires two different Processing sketches and a training interface.
1. Input Sketch: this code takes the input from the external camera I have previously connected to my computer and it reduces the image to 100 pixels. These pixels are the information that will be sent to Wekinator.
2. Output Sketch: this code receives the input (100 pixels) and currently processes 2 outputs that correspond to the X and Y position of the particle system.
3. Training Interface: after having the input and the output sketches running, I opened Wekinator which I used to train a simple computer vision model. The model will look at the patterns, and match them with a location within the map.
Coding sketch states
After testing the first behaviour of the sketch (receiving an input and getting an output), my next step was to map the selected communities within the map. The idea is that when the camera detects a known pattern, a specific location in the map is ‘selected’. To further expand on the meaning of the pattern, I wanted to create ‘states’ within the sketch that will allow me to open new windows and host more information.
The way that the particle system (in this case the butterflies) trigger the states, is by setting position ranges within the map. Reaching a specific section on the map (x, y) will open the corresponding window for that section.
To achieve this, I have mapped the chosen locations with squares , and I have pre-trained a model to reach those precise locations. The following video shows the interaction between the camera input matching a specific location when a pattern is presented to it. (*Note that the design of the map and its process is explained in another section of this documentation)