Scrolling in pan mode prevents stiffness?

I am creating an image tile library for use in a game project to fit into our existing structure. I am currently using UIScrollView (with an empty view) to allow me to capture beautiful physics for panning with a failure, but I had to implement my own scale with a gesture recognizer (for reasons I won't go into).

Here, however, when I fend off with one finger, adding a second touch and trying to zoom in, does nothing. I need to be absolutely still until this allows me to zoom in. When the user pans around the image, my gesture recognizer does not work unless the pan is completely finished.

Does anyone know about this that is not related to the failure of UIScrollView and the implementation of my own brass knuckles?

Here is my scrollViewDidScroll function:

- (void)scrollViewDidScroll:(UIScrollView *)scrollView
{
    viewingBounds.origin.x = scrollView.contentOffset.x;
    // Transform the y from UIKit coordinates into OpenGL coordinates
    viewingBounds.origin.y = world.height - viewingBounds.size.height - scrollView.contentOffset.y;
    [parentScene update]; // Render the image
}

My gesture recognizer is added directly to the scroll view in init:

[scrollView addGestureRecognizer:[[[UIPinchGestureRecognizer alloc] initWithTarget:self action:@selector(didReceivePinch:)] autorelease]];