Combining video and audio and timings

KevinM · 2026-04-01T23:38:48+00:00

For a lack of a better place to ask this, I'm asking in Members recordings. I record audio directly from the output of the virtual piano. I record video on...

TC3

KevinM Yup, that happens. Slightly different things going on under the hood with the recordings.

You can stretch the audio to make it a bit longer / shorter to match the video. So you line up the beginning of the audio and video, then go to the end, grab the edge of the audio, then drag it a bit one direction or the other until it lines up. Let me see if I can find a decent little how-to video on YT or the like.

KevinM

TC3 Thanks TC3, glad to confirmation it isn't just me, I might end up having to do what you suggest.

I was hoping not to play with the audio, the purpose of these recordings is the sound and I was hoping to leave that untouched as far as possible and in this instance I care less about the video, so squeezing the video feels more appropriate.

Jane

I don't use Apple, but have no issue with what I use.

I use the camera on my Pixel 9 Pro. I also plug a USB audio adapter into the phone so I can record the audio from the VST on my computer.

I record the audio separately on the computer (so it doesn't do any auto-level or other processing that the phone may do). Then use DaVinci Resolve to combine them.

Lay down the video track.
Add the separately-recorded audio.
Visually line the separate audio track to exactly match the audio track recorded on the phone.
Mute the phone audio track.
Render.

I do make sure to set the timeline on DaVinci to match the frame rate (30 FPS) that the phone is using for recording. (It asks me each time I import a new video, as that's apparently different from its default).

TC3

OK, check these out. I think between the two of them, it should work.

rogerch

TC3 What I do is very similar to what is described in the second video and it works great.

Some further tips:

Instead of clapping, I play three chords. I find it easier to line up three spikes than one.
To align the start of the video and audio, trim both to just before the spikes. Then zoom way in so those spikes are fat and are easier to align. The shapes of the spikes are different in the iPhone audio and the microphone audio so I do the best I can and then replay the spikes with both audio tracks and make sure they align.
Now I zoom out for ease of navigation and scroll to the end of the video.
Trim both video and audio to a point just after the three spikes. This is important because it enables me to see the spikes as I’m stretching or shrinking the video
Zoom back in to make the spikes fat
Right click on the video track to display the speed editor
Grab the speed editor at the top right of the video track and move it to align the video and audio. As you can see in the video this is not a great UI because you can’t tell where the spikes land until after you release the slider. It takes some trial and error. Listen to both audio tracks together to make sure they are aligned.

I play the three chords at the beginning of my recording and after each take. That way I can align the video and audio either before or after extracting a particular take.

I always resize the video to match the audio because the audio is the main event. I also use Audacity to normalize the audio before importing into iMovie.

TC3

KevinM
I was just playing around with it, and the same thing should work for stretching the video too. So once you have them both in there and lined up at the beginning, click on the video so it's highlighted, press command+R, and you'll see an extra little circle pop up on the top corners of the video. You then click and hold that to drag / stretch it one way or the other and make sure the end of the video aligns with the end of the audio.

KevinM

TC3 Thanks for this, I have just tried it out and it does exactly what I need. Feels a bit haphazard by not having proper control over frame rates, but it does the job. I'm going to update my latest recording.

KevinM

Thanks TC3 I'll check those out after sleep. I need to head to bed now.

Jane, it sounds like you are doing something similar to what TC3 suggests. The audio track gets stretched or squeezed. But your recordings are always excellent so perhaps I should not be concerned.

Jane

KevinM Jane, it sounds like you are doing something similar to what TC3 suggests. The audio track gets stretched or squeezed. But your recordings are always excellent so perhaps I should not be concerned.

I don't think so.
I'm sliding the separate audio track so it starts exactly in line with the phone audio track.
But I'm not needing to change the timing of either.

I think that is because I am setting the video frame rate in the editor to exactly match the frame rate recorded by the camera.

iternabe

I wonder if your audio/video misalignment is due to difference in frame rate between iPhone camera app setting and iMovie project setting. For example, if you iPhone records at 30fps, but your iMove project is set to 29.97fps, then your iPhone video will be stretched a little longer after imported into iMovie.

I myself sidestep this potential problem by recording with OBS studio. Being primarily designed as a live streaming app, OBS has the ability to combine multiple video streams (as picture-in-picture) and audio streams (as multiple tracks) in real time and record into a single video file with up to 5 tracks of audio. On newer macOS, you can make your iPhone a webcam to the Mac - and OBS is able to use it as video feed. To get the audio feed from you virtual piano directly into OBS, you can use the free Blackhole driver by setting up an aggregated output device in Audio Midi Setup. I use Rogue Amoeba's Loopback which is much more flexible. OBS even allow setting a (per audio stream) time delay (in milliseconds) to account for any latency in video processing. With a few try, you can get the audio and video aligned perfectly as they record. Then, all you need to do afterwards is the trim the video to length.

KevinM

I think you have it exactly iternabe .

My iPhone is recording video at 30.01 fps and when iMovie exports to a file the movie is exported at 29.97 fps. Checked by using get info in the mac Quicktime Player App.

If iMovie just maps the original video content to the new frame rate without resampling then the video length would increase in time length by 4 parts in 3000. This could be exactly what is happening. I can't see anyway in iMovie to modify this without using various tricks.

navindra

KevinM This is bugging me. 😃

If you use the free BlackMagic Camera app does this problem still happen?

WieWaldi

KevinM My iPhone is recording video at 30.01 fps and when iMovie exports to a file the movie is exported at 29.97 fps.

30.01 fps is weird. 29.97 and 30.00 fps are both common speeds. I am a Davinci Resolve Free user and have a similar workflow as @Jane . My camera (Sony A6300) procudes nice 29.97 fps videos in 4K. Therefore I set in the timeline 29.97 fps as well. In this case, DaVinci doesn't to change the speed of the video (I expect less blur in the hand motion).
The audio is recorded with OBS-Studio at a 48 kHz sample rate, together with a 59.94 fps video for the virtual keyboard. As the 59.94 is a multiplier of 29.97, both video tracks are perfectly aligned. So is the 48 kHz audio track and I never had to adjust the speed of my audio or my video tracks; they stay perfectly aligned as far as I can tell. Even if I decide to record to audio with audacity (overdubbing for backing-track) I don't face a misalignment problem.

I just checked, if DaVinci can manage 30.01 fps. It can't (at least not the free version). This are the available framerates:

Btw - I couldn't believe Apple records 30.01 fps. Found this link in reddit:

Best answer:

This is an Apple thing and I hate it.

What about setting up a 30.00 fps video in iMovie? With 1 off 3000 it should be 4 times better than a 29.97 fps clip.

Another solution would be to let the iPhone record the audio from your VST. Is there a way to connect both your computer to your phone via a USB cable and the digital audio signal goes from Mac to phone?

TC3

KevinM Very good! This is where my IT department would respond "TICKET CLOSED" ... 😃

KevinM

Yes, 30.01 is a weird frame rate. I don’t know if it is an ancient iPhone issue as the iPhone I use to record the video is an iPhone 8 on an older version of the OS. It has one job.

I am not so worried about the video quality. I don’t want to play around with the audio because that is my piano playing.

My setup is such that I am happy to use TC3’s solution applied to the video content and it works perfectly for my needs. Anything else I think would require a complete overhaul of my setup and that is not a job for now.

Animisha

A bit late reaction, but if you have iMovie, it is very easy to synchronise sound and vision. I also leave the audio and adjust the video speed.

iternabe

This post on reddit explains iPhone‘s odd (variable) frame rate very well: https://reddit.com/r/videoediting/wiki/faq/vfr

I am also a big fan of the Shutter Encoder app mentioned in their solution.