A new tablet app that recognizes handwritten music?

by Philip Rothman on January 3, 2013 · 7 comments

in News

Update, January 7, 2013: See the follow-up post Makers of music handwriting app video used Sibelius and GoodReader to create dramatization

A promotional video for a new tablet app has created a bit of a sensation in the last couple of days. A company called ThinkMusic Technology has made an internet commercial for “The Music Notation App You’ve Been Waiting For” and posted it to YouTube on December 31, 2012.

It shows music being played into a tablet using an iOS interface via an onscreen touch keyboard and via a physical MIDI keyboard. It also shows users using a stylus to write music directly on screen, and, with the tap of a button, immediately converting their scribbles into engraved music. The app is shown to instantly recognize handwritten notes, chord symbols, dynamics, and lines such as slurs, hairpins and 8va lines:

YouTube Preview Image

However, if you watch the video carefully, you may notice that the engraved music looks practically identical to default output from Sibelius (which is why I felt it relevant to post about this on this blog), right down to the fonts, spacing of the music, and even how the slurs in the third system make use of Magnetic Layout to push the chord symbols higher:

Screen grab no. 1 from ThinkMusic Technology promo video

Screen grab no. 1 from ThinkMusic Technology promo video

Screen grab no. 2 from ThinkMusic Technology promo video

Screen grab no. 2 from ThinkMusic Technology promo video

A setting of the demo score in Sibelius (the non-bold dynamic and unusual chord changes are taken from the video)

A setting of the demo score in Sibelius (the non-bold dynamic and unusual chord changes are copied “as is” from the video)

Switching between the handwriting and the engraved image, as shown on the video, could easily be simulated using a screen drawing or annotation app.

If this a hoax, it’s beautifully done. If the app is for real, then shouldn’t the promo video should carry a disclaimer that the images are simulated? No doubt, an app that actually does what is shown in the video would be welcome among musicians, judging from the excitement in comments posted to social media. If ThinkMusic’s app designing skills are as good as the production values in the video, then this could indeed be cause for celebration. For now, though, we’ll take the skeptical approach.

What do you think? We have reached out to ThinkMusic via Twitter (no e-mail or phone contact information is posted on their site) and will gladly update this post if they reply.

Did you enjoy this post? Subscribe to this blog

{ 6 comments… read them below or add one }

Álvaro Buitrago January 4, 2013 at 3:10 AM

There are many suspicious details in the video. The spacing of bar 1 and 3, before he starts writing, is also wider!

More details on “the skeptical approach” (Sorry, in Spanish, you can use a translator)
http://www.hispasonic.com/foros/editor-para-tablet/420234

Kala Pierson January 4, 2013 at 3:32 AM

I had a similar reaction and I laughed to see your post in my feeds tonight. :)

Another clue is that their supposed in-app shots of the output look to have the anti-aliasing of a PDF (not sure of this with YouTube-quality video, but I’d wager so).

More basically: I don’t think it’s materially possible, given iOS constraints, that an app could reproduce all Sibelius defaults down to exact weights, kerning of chord symbols and text blocks, and idiosyncratic (no offense :)) details like slur behaviors and hairpin placement.

Maybe this will be revealed as a social media experiment that was meant to demonstrate how many people *would* be excited for an app like this if someone developed it. :/

kb January 4, 2013 at 5:17 PM

I subscribed to their newsletter, it seems the company is located in California:
——-
Thank you for subscribing!

ThinkMusic Technology
PO Box 1151
Glendora, CA 91740
——

Peter McAleer January 5, 2013 at 11:16 AM

iPad is great, but it is a toy compared with a Mac or a PC desktop, and if the app is real, and as promising as that video suggests, it needs a desktop version to be taken up by professionals. How on earth am I going to display an orchestra on that? Why would I sit at my desk (like the guy in the video), hook my iPad up to a midi keyboard three times its size, and sit there craning my neck and straining my eyes to be able to see a tiny representation of an A3 score on a screen little bigger than the 9″ monitor my Mac Classic II had about 21 years ago? And paying good money to boot?

Kala Pierson January 5, 2013 at 11:28 AM

To be clear, the app isn’t real. The point of this post is to demonstrate that all the supposed footage of in-app behavior in the video is faked. (All of its supposed end results are not live results; they are actually PDFs of notation done in Sibelius, a desktop notation program that has nothing to do with the people who made the app.)

I think this post is too polite/indirect!

Carlin Lemon November 26, 2013 at 10:19 PM

I’ve been waiting for a program like this ever since I started charting my music on a windows convertible tablet 5 years ago. Unfortunately Apple doesn’t make any products with an actual stylus and digitizer, but there are numerous windows tablets and convertibles ranging from 8 to 15″ that are perfect for this. I’m wondering why Sibelius hasn’t already jumped on the idea as they already have the whole framework in place. To me, there’s nothing faster or more intuitive than hand entry, but it still lacks the legibility and versatility of a sibelius file. How long will we have to wait for this?

Leave a Comment

Current ye@r *

{ 1 trackback }

Previous post:

Next post: