Hasta La Motion!

I’m trying to do this 3d track on a shot and I don’t know where to proceed from here. I filmed two actors walking up to the edge of a bridge overlooking a city. I more or less successfully tracked the shot in Synth Eyes creating a garbage matte for the actors etc. to the point where a test cube looks like it’s sitting pretty decent in the scene.

Now what I want to do is replace the scenery past the bridge withe a matte painting. This will probably have to be a 3d matte painting because I want to have the camera fly out from the bridge into the matte/background as a transition from the last filmed frames.

I know I’ll have to roto out the actors for each frame they’re over the background but my question is how can I line up and properly project my background in conjunction with my footage in Maya?

Forgive me if my explanation is confusing and thanks in advance for any replies.