Default Quote
Default Cite

Public Exibition

UPDATE IN PROGRESS [7/13/2019]

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum

Learning Goals

GLSL/ Shaders
Flocking
Rendered
Machine learning
Projector Calibration
Sentiment analysis / Natural language processing
APIs speech to text
APIs speech to text

Objective

Create an interactive installation ecosystem that is generated through user’s conversations. Users can interact with the ecosystem through a depth camera aligned with a projector and through a simple chat room web page. The installation converts conversations to visual format, by using sentiment analysis and fractals to generate agents with unique appearances and behaviours. The installation is a real time simulation of ecosystems existing on the internet and an exploration of individuals existence and presence online. The behaviour of the agents would change over time according to conversations in the webpage. As the users converse each statement generates a set of agents whose behaviour is based on the sentiment analysis score of the statement. Inside the installation the user statements/ agents will interact with each other. A predator is created in the installation after each person interacts with the chat room based on a unique client ID coming from the web server. A user can create many statements but only has one client ID. The predators consume the statements/agents which changes its own behavior and appearance. There are two constant high level predators, they consume the client predators and their appearance changes accordingly. The statements are processed and consumed in layers of predators, the statements are temporal and only exist through the appearance of their predators and their behaviour. As the ecosystem grows statements are stored in a database. Then using natural language process/ machine learning, statements are created using stored words and fed back into the ecosystem, the ecosystem begins to sustain itself.

Information Flow

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Web Server

Uses socket.Io on each client(browser) then sends it to a server. Deployed on heroku. Used socket.io-client then to send a specific message to the heroku server which then streams text.

Sentiment Analysis

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Watson API: Use API to query for analysis of statments. Limited to 200 calls per month. Not accurate some statements are neutral and have no values. BUT has sentiment negation - “I am not happy” - sadness. Two levels document and sentence level. Sentence level: Analysis of individual sentences. Document Level: Analysis of whole document
AFINN-111: Uses a predetermined lexicon of sentiments with values -1 for negative +1 for positive. 1468 entries. Adds up all sentiments values divided by total count. NO native Negation. Simple unlimited calls
National Research Council Canada Lexicon: Uses a predetermined lexicon of sentiments with values fear, anger, anticipation, trust, surprise, positive, negative, sadness, disgust, joy. Adds up all sentiments values divided by total count . 14000 entries. NO native Negation. Simple unlimited calls

Rules

All behaviors and appearance are driven by flocking rules. These rules are determined by the sentiment analysis score. It uses fear anger joy and sadness. These are created by a simple component by getting the word score from a lexicon and dividing it by the total number of words. The lexicon was created by the National Research Council of Canada.
The basic flocking function is based on Boids an artificial life program, developed by Craig Reynolds. There are 3 rules in basic flocking.
Separation: steer to avoid crowding local flockmates
Alignment: steer towards the average heading of local flockmates
Cohesion: steer to move towards the average position (center of mass) of local flockmates
Fear: a high fear score the agent created is more aware of predators. It is less likely to be eaten by predators and more likely to break away from the flock when a predator nearby.
Sadness: Acts a speed limit to all the other behaviors reducing the overall velocity. Sad boids are unlikely to flock and are unable to flow with nearby flocks.
Anger: An angry agent will push agents away. Although it still follows the other rules, so it will flock. But its presence disrupts the flow of the
Joy: When a statement has a high joy score the agents will flock closer together and move in similar velocity as its flock.

Agents

Three tiers of life in the ecosystem, the lowest tier are prey. The prey are created through the sentiment score and are generated directly through user interaction. The second tier is controlled by the amount of users interacting with the web server. The predators are attracted to the prey and consume any nearby, their sentiment are added and averaged to its own rules. Their behaviour, speed, velocity and appearance change accordingly. The last tier is the only form of persistence. Their lifespans are much longer than the other agents. The absorb the lower tier of predators rules.
Prey
Predator One
Predator Two

Fractals

Each fractal is calculated in the material shader based on the sentiment score. The fractal iteration is calculated based on the life span. The size of the agent is determined by its life span. As the lifespan increases so does the complexity of the fractal.

Haiku Generator

The haiku generator is in the socket.io- client server. Uses npm syllable to count syllables https://www.npmjs.com/package/syllable. Stores the word into a list with its syllable count. Picks a random word with a syllable checks remaining syllable count add until it equals 5 or 7. If the syllable count exceeds 5 or 7 restart loop until it does. When the total number of words exceed a number of words eg 300 the server starts sending haikus over osc to touchdesigner.
Haiku Samples
hand, they those righteous
again anyone selection
because who endures
blame but matters pain
foresee demoralized toil
exercise, laborious
righteous loves avoid
nor denouncing advantage
abandoned pleasure
wise welcomed perfectly
pleasures idea take be
demoralized shrinking
secure blinded we
pleasure, abandoned every
actual are secure
our born toil right every
pursue avoided. those idea
hour, business abandoned
Input Sample 300 words
"But I must explain to you how all this mistaken idea of denouncing pleasure and praising pain was born and I will give you a complete account of the system, and expound the actual teachings of the great explorer of the truth, the master-builder of human happiness. No one rejects, dislikes, or avoids pleasure itself, because it is pleasure, but because those who do not know how to pursue pleasure rationally encounter consequences that are extremely painful. Nor again is there anyone who loves or pursues or desires to obtain pain of itself, because it is pain, but because occasionally circumstances occur in which toil and pain can procure him some great pleasure. To take a trivial example, which of us ever undertakes laborious physical exercise, except to obtain some advantage from it? But who has any right to find fault with a man who chooses to enjoy a pleasure that has no annoying consequences, or one who avoids a pain that produces no resultant pleasure?"
"On the other hand, we denounce with righteous indignation and dislike men who are so beguiled and demoralized by the charms of pleasure of the moment, so blinded by desire, that they cannot foresee the pain and trouble that are bound to ensue; and equal blame belongs to those who fail in their duty through weakness of will, which is the same as saying through shrinking from toil and pain. These cases are perfectly simple and easy to distinguish. In a free hour, when our power of choice is untrammelled and when nothing prevents our being able to do what we like best, every pleasure is to be welcomed and every pain avoided. But in certain circumstances and owing to the claims of duty or the obligations of business it will frequently occur that pleasures have to be repudiated and annoyances accepted. The wise man therefore always holds in these matters to this principle of selection: he rejects pleasures to secure other greater pleasures, or else he endures pains to avoid worse pains."

Layout/Setup

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Inspiration

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

Resources

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Work
BLANK SPACE
Vimeo
Tumblr
Linkedin
GitHub