top of page

Channel AI - Research

My Role

 

  1. Design: I conducted secondary research on human-AI interaction and defined a design framework to improve information architecture, usability, user acceptance, and trust.

  2. User research: I defined the research questions and method, conducted interviews to determine how well the Channel-AI supported the existing workflow and assessed user acceptance.

  3. Testing: I participated in design reviews, provided feedback on usability, and performed subjective listening testing activities.

Results

​

  1. Researched, appropriated, and applied design principles to guide the design of the Channel Ai. 

  2. Evaluated design with users using qualitative methods

  3. Defined direction for future development of the software tool

HD96-24-CC-TP_P0BHN_Front2_XL.png

The figure shows the mixing desk in which the Channel-AI, the system our team has developed is embedded.

The project

Early in 2019, I joined the AI team as an AI researcher specializing in product design and user experience research. I worked on the design of the Channel AI, an assistive technology of the Midas Heritage-D. The Heritage-D is a professional mixing console commonly used in high-profile venues to support live performances worldwide. My team developed an embedded AI system that ran on the console and offered a range of capabilities to support mixing engineers. The capabilities included instrument recognition and generating parameter settings suggestions for gain levels, gating, compression, and equalization, specific to the input signal and the instrument type.

forAugustinos.png

The figure shows the Heritage D96 graphical user interface. The toolbar on the right section of the user interface shows the Channel-AI.

Personas

midas_console_in_venue.png

"trust calibration, i.e., the degree to which a person trusts another agent, is an extremely important variable that should be taken into account when designing human-AI interaction."

We identified four main application domains where users would likely utilise our technologies: live performance, broadcasting, theatre, and church. What differentiates these four categories is that the live performance, broadcasting, and theatre mixing engineers are most likely domain experts. At the same time, the church sound operators are less likely to be domain experts. The list below provides information on the expected characteristics of a typical profile and the motivations of our users. These are likely valid for the first three types of personas (i.e., live performance, broadcasting, and theatre). These characteristics are less likely to apply to the church sound operator, who may not fit the profile features 2 -7 in the list below.

​

The target audience profile and motivations:

  1. Wants to achieve the best possible sounding results.

  2. They are very quality-conscious.

  3. Have a lot of experience.

  4. They are domain experts.

  5. Have well-defined/established workflows.

  6. They are hesitant to change their workflows.

  7. They are well-versed in the tools they use.

Challenges.png

The Channel-AI is used mainly by highly skilled mixing engineers in very high-profile venues where the stakes are high for the engineers, the organizers, and the artists. Hence, users are quality conscious and conservative in incorporating new tools into their workflow. Therefore, we consider it essential to find a good balance in human-machine cooperation in terms of user control, type, and level of automation. So the system should be carefully designed to co-exist harmoniously with the existing user workflows without getting in the way. 

The interaction method we adopted in the design of the Channel AI should mitigate the following risks, which we believe could impede the adoption and utilization of the system: 

​

        1) Automation makes undesired, suboptimal, and non-rectifiable decisions.

        2) Removing engineers’ authority and control to do their jobs in the best way they see fit.

        3) Forcing users to change existing workflows radically.

Literature reviewing: Human-AI interaction

Distrust of AI recommendations could result from the lack of understanding of the underlying reasoning that underpins the models’ outputs.

Trust calibration is a variable we considered when designing the Channel AI. My research showed that trust is critical to enabling two agents to co-operate effectively. We must carefully plan human-machine cooperation to allow for an appropriate level of trust. A mismatch between the system's capabilities and the users’ level of trust can lead to under-trusting or over-trusting the system. For instance, if the trust exceeds the capabilities of the automated system, it can lead to misuse, i.e., delegating tasks that the system cannot perform. When confidence is lower than the system's capabilities, it can lead to disuse, i.e., underutilization of its features. According to the literature, other essential factors for developing trust between the user and the system include feedback provision, an indication of how confident the system is about the validity of its outputs, users’ understanding of how an AI reaches a particular conclusion, and why it has reached it. Providing appropriate feedback is essential to facilitate user understanding, justify, and enable user control.

After reviewing a large body of literature on human-AI interaction and automation,  I collated a set of design principles and liaised with the product manager to apply the principles and guide the Channel-AI design.

The Design principles

To aid the interaction design process of the AI-System, we identified a set of design principles (derived from, see Ameresi et al. 2016) and applied them to further optimise the human-AI interaction and evaluate our current design. A total of 18 design principles are proposed in the original paper, these are grouped into 4 categories: Initial, During Interaction, When Wrong, Over Time. We only utilized the first 11 principles since the remaining seven apply only to AI systems implementing interactive machine learning techniques such as reinforcement learning. We utilised these principles to ensure appropriate trust calibration (principles 1, 2, 5), feedback provision (principles 3, 4), maximisation of user control and minimisation of disruption to current workflows (principles 6-11). Using these principles at both the conceptualisation and evaluation phases has proven very rewarding since it led to a major redesign of the interface and the workflow, a more detailed explanation of the principles used and how they map to the user interface.

The video explains which guidelines have UX been applied to the design of the Channel-AI. 

System Analysis Framework

Most complex music information retrieval and intelligent sound processing systems consist of many layers. Each layer can exhibit different levels of automation from No Automation, Assistance, Partial Automation, Conditional Automation, High Automation, and Total Automation SAE (2021). To understand where the Channel-AI stands regarding human-machine cooperation and help our team identify risks and plan future system development, we performed system analysis using the stage model suggested by Parasuraman et al. (2000). The model consists of four functions that can be performed by a human or an intelligent automation system: Information Acquisition, Information Analysis, Decision and Action Selection, and Action Implementation. We combined models suggested by Parasuraman et al. and the levels of automation inspired by the international standard in autonomous driving, to determine what level and type of automation each component of the Channel-AI will perform, as shown in the Figure below. 

System analysis showing the different levels of automation of the Channel-AI features across the four functions.

system_analysis_framework.png

Video Demo

The video below provides a detailed explanation of how the design framework has been used to design the user interface of the Channel-AI. This excerpt has been extracted from the video that accompanied the conference publication I co-authored with my former manager Alessandro Palladini with the title Towards a Human-Centric Design Framework for AI-Assisted Music Production which was presented at the International Conferences on New Interfaces for Musical Expression in July 2020. 

Evaluation

Before defining our research questions and test methods and tasks, we need to identify the most important features of our products and make assumptions about the value we expect these features to offer to the end-user. The table below shows the central values and goals of the Channel-AI. These will form the basis for formulating our research questions.

values and goals.png

Study Design

Each mixing engineer was given a demonstration of all of the features of the Channel-AI. During the demonstration, we ensured that the participants understood the functionality of the features and answered any questions they may have about the system's functionality. Before the interview, we had instructed the engineers to bring a multitrack recording they had recently mixed and felt comfortable with. After we gave a quick demonstration of the system, we asked the engineer to mix the music tracks utilising the Channel-AI. We told engineers that they could diverge if they wanted from the system suggestions until they achieved a good mix. After the mixing session, we conducted a semi-structured interview to elicit information regarding the engineers’ experience of mixing using the Channel-AI and receive feedback on the workflow and the parameter settings suggestions offered by the adaptive presets. 

Evaluation-procedure.drawio.png

Usability study testing procedure

Results

Our findings provided many insights related to the performance of the system and the effectiveness of the preset suggestions, which were very useful for improving the design of the algorithms. Moreover, it highlighted the issues that could potentially arise in designing automated systems that support professional audio practitioners. We observed that although domain experts valued the system's assistance and did not identify any significant usability issues other than having to inspect the resulting preset suggestions, they were sceptical when asked about their willingness to adopt intelligent automated systems in their workflows. The main benefit they could see is that when they are under extreme time pressure, the AI assuming good recommendations could reduce the setup time and consequently speed up their workflow. However, in most cases, users had to review and correct the recommendations, which, in a way, minimised the setup time to process an audio channel. Moreover, it could be a source of inspiration and beneficial for novice users. These findings led us to conclude that striking the right balance between automation and user control and accusation of AI suggestions is paramount for the adoption of automated music production systems by domain experts. 

Learnings

AI-infused technologies that are designed for expert users and deployed in critical application domains must be carefully designed to strike a good balance between automation and user control and instil a sense of trust in the user. User psychology and attitudes should not be ignored! Finding the right balance between automation and user control, the right way to display AI recommendations, and carefully augmenting existing user workflow is critical to achieving user acceptance and maximising the adoption potential for a given technology. In this context, it took a lot of design and engineering ingenuity, UX craftsmanship, and research to create a product that users will love. Reflecting on my experience working on this project, I realise how essential it is to prioritise deep user engagement and robust research, including customer psychology, acceptance, and desirability. It is important to put the teams' vision to the test and be eager to hear the truth, even if it runs contrary to our initial gut feelings and ideas. Engagement with users helps ensure that the feature, product, or service will offer real value to the end user and that it is presented and marketed correctly. Indeed, finding the right balance and optimal ways to do this is hard and context-dependent. However, in my experience, it can be done cheaply, it does not have to be time-consuming, and the entire product team benefits from user research in a multitude of ways. These include gathering requirements, evaluating team assumptions, and minimising the risk of product and business decisions. Hence, I believe appropriate engagement with users, stakeholders, and usage and market data when available  is critical to the success of any product or business in the marketplace.

bottom of page