Recital video: how to add score to video?

ShiroKuro

I know this has been discussed here before but can I ask again?

I would like to try adding the score to my video when I make my PT video. I want to set it up so that the video showing my hands is at the bottom and the score is at the top, and the score moves along as the music moves along.

I would also like to do this in the simplest way possible 😅

I can use powerpoint, put the scores onto the slides and click through the slides so that the score follows the playing.

So what is the easiest way to combine the moving score (video of score) with the video of me playing? I have access to Kaltura and also Canva, but I’ve never used either. I have a PC, so maybe there’s some free MS app I could use? I used to use Camtasia, but no longer have access to the paid version and don’t want the watermark that it adds to the free version.

I do have an iPad, so maybe iMovie could work? But I’ve never used it for something like this before.

Thank you in advance for any advice!

mel_lem

I have done something like this before.

I used Splice on ipad, and took screenshot of the score by system, posted them over as photos. Maybe there is an option to make the scrolling effects.

Free version of Splice doesn’t have watermark.

Hope this helps!

ShiroKuro

mel_lem thanks for this info. You wrote “ took screenshot of the score by system” — I assume by system you mean, just using the screenshot option on whatever device you display the score?

I looked at Splice but it seems like it’s made more for other purposes, so it wasn’t immediately obvious to me how to use it to do what I want…. I’ll look again.

For the record, what I want is my hands on bottom and the score on top. I usually make videos with this playing angle:

And I could crop it more if needed. I have no idea what the appropriate ratio (horizontal to vertical) is. 16:9?

mel_lem

You are looking for visual like this?

I think you have to change your camera angle to be shooting from right above.

The scrolling score will be a challenge though.

ShiroKuro

mel_lem I don’t think I can get a true overhead shot like that with my current equipment, but still, I think it would work with my regular angle, plus the score on top.
Do you think that would be bad?

ShiroKuro

It would look like this, except as a video (this is a still image I just threw together in powerpoint)

ShiroKuro

Does that look bad? If it looks bad, it's not worth doing it....

(Since I can't change my recording angle without purchasing different equipment, which I don't want to do, at least not in the near future)

mel_lem

ShiroKuro I did a quick video for you to see, had to do a screen recording of your video. I hope you don’t mind. I will remove it once you have seen it.

It will be something like this. I used other music scores and the scrolling is hard to achieve.

ShiroKuro

@mel_lem wow!! Thank you for showing this!

Yes that’s basically what I had in mind. Did you use splice?

Do you think it looks bad since the viewing angle isn’t straight overhead?

mel_lem the scrolling is hard to achieve.

Is that because of how your displaying the score? I assume the score isn’t an mp4? I was thinking I would make an mo4 of just the score, and then an mp4 of the video, and then combine them like a split screen video.

Is that not what you did?

iternabe

mel_lem It will be something like this. I used other music scores and the scrolling is hard to achieve.

That's really good!

I too think scrolling would be a different level of challenge. It might be possible if the score is available in digital format that can be imported into Musescore. Musescore has a feature to display the score as one continuous view scrolling horizontally. You can set a tempo, hit the play button, and make a screen recording of Musescore scrolling the score from start to end. That video can be overlayed on top of the real player video. You probably need to do some speed stretching/compressing periodically to align the scrolling score clip with human playing that's not precisely metronomic.

iternabe

I have not added score to my video. However, I have done some video editing just for the fun and enjoyment of learning the skill. As I contemplate on how would I approach adding the score, I would hesitate to do it on iPad. The reason is such editing would require certain precision that can be hard and frustrating using finger touch interface.

After trying several video editing softwares, I eventually settled on Davinci Resolve which I would wholeheartedly recommend to anyone. IMHO it is the only one that's worthy of my time learning. You only need to spend a couple of hours following the official training video to get really good at it. Davinci Resolve is so powerful yet so user friendly, the more you use it the more time you save compared to other tools.

As for this particular task, you will likely be doing the following steps:

gather the score as jpeg or png image (one image per page)
add a new video track above your play video
insert the score image onto the new track
crop the new track (from bottom) to reveal your hand on the keyboard
optionally, warp/tranform the cropped new track so that the shape aligns with your keyboard
at each moment for the new line of the score, create a cut or transition to shift the score image up to show the next line (or next page).

mel_lem

iternabe “The reason is such editing would require certain precision that can be hard and frustrating using finger touch interface.”

I totally agree with this. But she wants this to be done the simplest way possible.

mel_lem

ShiroKuro I overlay the screenshot of the score as still photo, and I can adjust the length of time of the part of score to be displayed to go with the music.

And yes, I used the free version of Splice for this.

mel_lem

I use the “overlay” function to attach the scores to your video. You can actually overlay another video.

[I think, I have never actually done this] So you have to take a screen recording of you scrolling the score with the scrolling speed matching the tempo of your music. And then overlay that to the top part of your piano playing video. I am not sure about the ratio and if you can crop the score scrolling video though.

mel_lem

iternabe oh yes, I think this is a great way to use musescore if the digital score or midi is available.

ShiroKuro

Thank you!! So regarding the score part, I think I can use the PC snipping tool to make a screen recording and save as an mp4.

First I would “cut up” (digitally) the score and put each grand staff onto one slide in PowerPoint. The listen to my recording and I scroll through the PowerPoint, using the snipping tool to capture just the score. Then save as mp4.

Then I would have two mp4s, the piano video and the score video. And then combine them using a split screen method, either in Splice iMovie .

To me that sounds easier than trying to paste parts of the score into a video timeline in an editing tool.

What do you both think? ( @mel_lem and @iternabe )

mel_lem

ShiroKuro I think it will work, but I will have to see your edited video to be able to comment further.

WieWaldi

ShiroKuro Then I would have two mp4s, the piano video and the score video. And then combine them using a split screen method, either in Splice iMovie .

To me that sounds easier than trying to paste parts of the score into a video timeline in an editing tool.

Erm. no.

Putting each image to the right place in your video timeline is easier. You save one step of workflow; maybe two. And you can adjust the timeline positions as often as you want. No need to record a one shot powerpoint presentation with the right timing in a single run.

iternabe

ShiroKuro So regarding the score part, I think I can use the PC snipping tool to make a screen recording and save as an mp4.

First I would “cut up” (digitally) the score and put each grand staff onto one slide in PowerPoint. The listen to my recording and I scroll through the PowerPoint, using the snipping tool to capture just the score. Then save as mp4.

Then I would have two mp4s, the piano video and the score video. And then combine them using a split screen method, either in Splice iMovie .

To me that sounds easier than trying to paste parts of the score into a video timeline in an editing tool.

What do you both think? ( @mel_lem and @iternabe )

That should work. It is more straightforward if you only do this once, or occasionally. But if you do this often, whether doing multiple retakes of the same pieces, or record new pieces frequently, then you will realize the process of cutting up the score image (to individual systems), creating the powerpoint, and screen record, are the kind of repetitive overhead that can be eliminated by importing the score into video editor and manipulated as overlay.

ShiroKuro

mel_lem your edited video

What do you mean?

ETA oh do you mean the video with the score?

ETA again: in that case, I won’t have the score video until after I’ve made the piano video, bc I want to time it to a specific performance.

ShiroKuro

mel_lem and @iternabe I don’t have a midi file or musescore score.

thepianoplayer416

It's be easier to have the pages in separate imaes, use a photo editing program to break it into separate lines. Save the image of each line into a separate file.

Use a video editing program like MS Clipchamp and overlap the images in the right places in the video.

You can use a notation program like Muse Score on computer and capture the playback to show the notes lighting up. I wouldn't do it unless my playing is locked against the computer playback in a mechanical way. The tempo can stretch especially when playing 19th century pieces with rubato. And close to the end of a section you might do a ritard. And people don't slow down consistently sometimes more sometimes less. so not easy to sync the notes lighting up with your playing.

ShiroKuro

thepianoplayer416 It's be easier to have the pages in separate imaes, use a photo editing program to break it into separate lines. Save the image of each line into a separate file

I will have each grand staff as a separate image, and then click through it like slides.

Also I don’t know how to use musescore, and I don’t want to deal with overlaying the staves at specific spots on a video editing timeline, those are two learning curves I’m hoping to avoid.

mel_lem

ShiroKuro yes, your video with the score edited in