Super AI Artist
To celebrate the 25th anniversary of the establishment of the Hong Kong Special Administrative Region, we take great delights in presenting our artworks created by our “Super AI Artist” at HKBU Symphony Orchestra Annual Gala Concert 2022. The project team, led by Prof. Yike Guo, Vice-President (Research and Development), received major research funding from the Research Grants Council of the HKSAR for conducting a 5- year Art Tech project since November 2021.
AI Choir
By collecting many recorded songs from professional singers in studios and from online, we have extracted and disentangled key contributing features of vocal singing, developed generative models for singing voices, and successfully generated a virtual four-part choir, in which the pitch, melody, and expressiveness of each individual virtual singer can be flexibly managed and be coherently composed of. Apart from sharing the same timbre as the training data, “interpolated singers” have also been generated as virtual performers with brand new timbres. Inspired by human-machine symbiotic art creation, our AI choir can collaborate with any symphony orchestra to deliver a choral-orchestral music performance.
AI Cross-media Visual Storytelling
Human creativity can be found in creative visual arts, particularly in those artworks that tell stories. Can machine creativity also achieve this by generating meaningful artworks to visualize the intrinsic thought of machine beings, as do human beings? With this in mind, we developed an AI-powered media artist model to generate expressive artworks with song lyrics and music as input. By using this model, we translated the AI-generated choral music, "Pearl of the Orient", into a machine-painted music story that could exhibit a highly semantical association with the lyrics, its storylines, and the music rhythm.
We first proposed the framework of Cybernetics Intelligence for art creation. Besides the basic concept of Cybernetics of emphasizing the interactive optimization between algorithm and the environment, we proposed that the representation of AI tasks could be iteratively implemented based on the symbiotic interaction with the artists, and the feedback and stability of the system could be represented in a hierarchical structure. The main difference from Turing machine is that our model is not mimicking human artworks. Instead, our system can advance human creativity as guided by symbiotic interaction iteratively and finally converge to the abstract aesthetic feature space.
This is a significant step forward to achieving our goal of creating a Super AI Artist that will push the boundaries of creative media for the future.
AI Choreography
We use AI technologies to choreograph a dance based on patterns in nature scenery, melody, rhythms, and emotions expressed through music. We have achieved this by building a large motion-capture dataset of professional dancers who perform according to their understanding of the music, which covers a diverse range of musical and dance styles. We collaborated with professional dancers and musicians to identify aesthetic features of their creative practice.
AI algorithms have enabled us to decode the underlying emotional and aesthetic connections between music and dance. To ensure that the generated motion trajectories could be precisely and naturally rendered, we have investigated AI-filters that could enforce physical and skeletal-structural constraints. To generate expressive dance movements, we have observed and learned choreomusical rules based on rhythmic alignment, transition, and combination of dance sequences. The rules have been extracted from the art dataset and embedded into the generative models. With these elements, the AI choreographer can create a natural and artistic dance series based on its own understanding of the music.