Getting EMO with Behaviour Apps

Intro to Affective Computing

The rise and rise of wearable and environmental sensors, biometrics and the Internet of Things has cast a web of data capture across all aspects of daily life including our emotional responses. The digitisation of these responses has been formalised under the umbrella of what is known as Affective Computing. As is the case with many nascent branches of research and development, affective tech has been touted as a panacea for a great deal of everyday maladies but most consistently as the snake oil of choice for advertising efficacy. Happily, it offers up far more than that in terms of practical application, particularly when integrated into a behavioural solutions architecture. As a practitioner, I'm keen to integrate the functional insights proffered by an affective platform API into system specification and iterative adaptations that can serve to shift an app's features and UX according to the user's behavioural-emotional state.

Measuring behaviour

In designing and developing behavioural systems, the use of standardised scales during an analysis activity provides us with:

  1. dimensions to the the problem space
  2. a context for articulation of the behaviour set and its instances and
  3. structural models that hone in on what should be key performance metrics for the app and any subtleties.

Using the transtheoretical model

If, as we often do, you apply a Transtheoretical Model (Prochaska & Velicer, 1997) approach to your analysis phase then the instantiation of the target behaviour states through the app must include support for the model's strategies needed to progress users through the change states. In the instance of a health behaviour app this may include but not be restricted to:

  1. Consciousness Raising - Increasing awareness about the healthy behaviour.
  2. Dramatic Relief - Emotional arousal about the health behaviour, whether positive or negative arousal.
  3. Self-Reevaluation - Self reappraisal to realise the healthy behaviour is part of who they want to be.
  4. Environmental Reevaluation - Social reappraisal to realise how their unhealthy behaviour affects others.
  5. Social Liberation - Environmental opportunities that exist to show society is supportive of the healthy behaviour.
  6. Self-Liberation - Commitment to change behaviour based on the belief that achievement of the healthy behaviour is possible.
  7. Helping Relationships - Finding supportive relationships that encourage the desired change.
  8. Counter-Conditioning - Substituting healthy behaviours and thoughts for unhealthy behaviours and thoughts.
  9. Reinforcement Management - Rewarding the positive behaviour and reducing the rewards that come from negative behaviour.
  10. Stimulus Control - Re-engineering the environment to have reminders and cues that support and encourage the healthy behaviour and remove those that encourage the unhealthy behaviour.

Each of these strategies are able to be operationalised through code and delivered across platforms and channels. Dependent upon the evidence-based framework being used it's important to populate each target user motivation level and related change states with indicative functional requirements as user stories and accompanying UX mock-ups. This descriptive, enriched content adds meat to the bones of the structural model. Plugging this into the agile development platform of choice will help provide direction and a common language for the behaviouralist, the business, the users and developers.

Collecting emotional recognition data

What is crucial and herein lies the potential for affective tech, is the opportunity to gather emotional and tactile responses from the user to the incremental apps releases. Recording keystrokes and mouse clicks is not new to UX practitioners and neither is studying video footage of users interacting with apps but what is new and potentially valuable is using affective tech to better understand and calibrate the emotional recognition data with the target behaviour being sought. It will help identify issues and success dispassionately and if employed through the build and deploy of the app do so continuously. In considering the integration of affective data into a behavioural systems build it may be wise to consider the architecture of the 'Affective Stack', (Thompson & McGill, 2015)which provides a clear blueprint of potential services and integration design considerations. Remember, anything you build needs to play nicely with the O/S and various third party interfaces and it should be designed to scale and be open to extension without sacrificing usability and functionality. Here's a visual recap of our thoughts:


If you want a light and breezy introduction to the applicability of affective tech, in particular the versatile, well documented and tested Affective API set then try these simple web-based demos


Prochaska, J. and Velicer, W. (1997). The Transtheoretical Model of Health Behavior Change. American Journal of Health Promotion, 12(1), pp.38-48.

Thompson, N. and Jane McGill, T. (2015). Affective Stack — A Model for Affective Computing Application Development. Journal of Software, 10(8), pp.919-930.

Dr Daryl Foy

Dr Daryl Foy is a Behavioural Scientist who specialises in the design of effective health behaviour change apps based on evidence including his own validated models for optimising persistent use. He consults to industry on how-to integrate persuasive design into LEAN product development as well as conversational UI. He can be contacted at