Augoustinos Tsiros
Research
Morpheme Interface: user studies
One of the main motivations of the present investigation was to explore how to
best utilise empirically validated audio-visual associations to inform the design of a feature-based sound synthesis user interface, i.e. Morpheme. Morpheme allows users to interact with sound through the act of sketching on a digital canvas. For Morpheme to work effectively, its design must be based on meaningful relationships between what the user paints and what they hear. This understanding led us to investigate possible audio-visual mappings. Previous studies that investigated correspondences between cross-modal sensory cues have shown that different individuals exhibit common patterns of perceived congruency between specific cross-modal cues and that common congruency patterns exist across different ethnic groups,
age groups and shared with other primates. The most effective empirically validated audio-visual associations found in the literature were selected to develop two mappings that enable visual interaction with sound through sketching.
We conducted three studies and reported on the evaluation of Morpheme’s user interface in several articles, please see the links below. In Summary, 110 participants were asked to undertake two tasks (i.e. similarity ratings, and discrimination task) designed to assess the effectiveness of the mappings. The results confirmed that size/loudness, vertical position/pitch, colour brightness/spectral brightness are strongly associated. A weaker but significant association was found between texture granularity and sound dissonance, as well as colour complexity and sound dissonance. The results showed a non-linear interaction between the harmonicity of the corpus and the perceived correspondence of the audio-visual associations. For example, strongly correlated cross-modal cues such as size-loudness or vertical position-pitch are affected less by the harmonicity of the audio corpus in comparison to weaker correlated dimensions (e.g. texture granularity-sound dissonance). Further, results indicated that users' sound/musical training had no significant effect on the perceived similarity between AV features or the discrimination ability of the subjects. Finally, we conducted a study to assess the effectiveness of the user interface, detect usability issues and gather participants’ responses regarding cognitive, experiential and expressive aspects of the interaction. The evaluation comprises a design task, where participants were asked to design two soundscapes using the Morpheme interface. Responses were gathered using a series of Likert-type and open-ended questions. The analysis of the data gathered revealed a number of usability issues, however, the performance of Morpheme was satisfactory and participants recognised the creative potential of the interface and the synthesis methods for sound design applications.
Sonic Xplorer: modelling the relationship between semantic descriptors and synthesis parameters
The main aim of this project is to develop a model that maps semantic descriptors (e.g. warm, bright, thick, noisy) and their associated synthesis parameters. The effective design of novel sounds using sound synthesis can be a laborious process requiring a good understanding of multiple aspects. Synthesis principles must first be mastered, followed by the capabilities offered by the chosen instrument and how to configure user interface parameters to achieve the desired output. A wide range of sound synthesis software tools are available. These have elaborate capabilities but are typically complex. The learning curve associated with operating each tool beyond basic functionality may be discouraging for non-trained users. Musicians, sound designers and users without technical expertise in a synthesis method can spend a considerable amount of time learning the different tools in order to programme a synthesiser effectively. Having to configure large numbers of parameters in a serial fashion to
obtain optimal results is believed to hamper creativity, as well as increase users’ cognitive effort, and potentially disrupt the creative process.
Research shows that musicians and sound practitioners intuitively use adjectives such as bright or warm to describe sound timbre. Digital synthesis engines for timbre generation offer a high number of parameters that can be utilised to manipulate timbre. However, these parameters are more relevant to the signal processes of a given synthesis method and not the semantic descriptors that humans would naturally employ to describe the timbral qualities. These parameters could modify multiple perceptual dimensions of timbre, but currently there are no perceptual models for mapping acoustic and semantic descriptors of different sounds and the underlying synthesis parameters. The main aim of this project is to develop a model that maps semantic descriptors (e.g. warm, bright, thick, noisy etc.) and their associated synthesis parameters. The model will be employed to allow sound practitioners to control a large number of parameters (i.e. over 200) through a minimal set of semantic descriptors. Sonic Xplorer, utilises users' ratings and machine learning to create an intuitive user interface. Users' ratings are used to teach the system a correlation between adjectives (e.g. warm, bright, thick, evolving) and the underlying synthesis parameters.
EVA 2017 Article [pdf]
Differences in the audio branding strategies of different industry sectors
The aim of this research was to measure consumers' emotional responses to audio logos (i.e. short audio excerpts predominately used for marketing purposes) and compare different industry sectors. A survey was designed and administered online. Hundred- eighteen adults (53 females and 65 males, median age =25 -34 years) volunteered to take part in the experiment. A variable number of Audio Logos were sourced and categorised into seven industry types Automotive, Banking and Finance, Consumer Electronics, Information and Communications Technologies (ICT), Film, Radio and TV, Video Games seven logos per category. Three out of seven logos for each category were selected at random 3x7 a total of 21 audio stimuli.
