A Multimodal Approach to Acoustic Guitar Strumming

Strumming is a guitar playing technique where instead of playing individual notes, the player strokes over the strings of a chord in an up- or downwards direction. Due to the very short time difference between the string hits, conventional note-based transcription methods cannot distinguish the direction of the strumming movement and hence are not suited for rhythm guitar transcription.
Therefore, a multimodal approach to strumming action transcription is proposed, combining audio recordings with the motion of the hand. The audio signal is used for strumming event detection and the motion signal is used for direction classification. To measure the motion of the hand, a 6-axis gyroscope and accelerometer is mounted on the back of the hand which sends measurements to a computer.
For evaluation purposes, a recording of five minutes has been labelled manually and is published along with this work. On this test dataset, the presented multimodal approach reaches an F1 score of 85 % for the up and 92 % for the down strumming transcription task.