Mongoose limits the query to 1000 results when I need more / all (transition from 2.6.5 to 3.1.2)

I am migrating my application from Mongoose 2.6.5 to 3.1.2, and I am encountering unexpected behavior. Namely, I notice that the query results are automatically limited to 1000 records, while almost everything else works the same way. In my code (below) I set a value maxIvDataPointsthat limits the amount of data returned (and ultimately goes to the client browser), and this value was set elsewhere to 1500. I use the count query to determine the total number of potential results, and then a subsequent mod to limit the actual query results, using a counter and a value maxIvDataPointsto determine the value of the mod. I am running node 0.8.4 and mongo 2.0.4, writing server-side code in coffeescript.

Prior to installing mongoose 3.1.x, the code worked the way I wanted, returning only 1,500 data points each time. After installing 3.1.2, I get exactly 1000 data points returned each time (provided that there are more than 1000 data points in the specified range). Results are truncated, so data points from 1001 to ~ 1500 are those that no longer return.

There seems to be some kind of setting somewhere that governs this behavior, but I can't find anything in the docs, here or in the Google group. I'm still relative n00b, so I might have missed something obvious.

DataManager::ivDataQueryStream = (testId, minTime, maxTime, callback) ->

    # If minTime and maxTime have been provided, set a flag to limit time extents of query
    unless isNaN(minTime)
    timeLimits = true

    # Load the max number of IV data points to be displayed from CONFIG
    maxIvDataPoints = CONFIG.maxIvDataPoints

    # Construct a count query to determine the number if IV data points in range
    ivCountQuery = TestDataPoint.count({})
    ivCountQuery.where "testId", testId

    if timeLimits
        ivCountQuery.gt "testTime", minTime
        ivCountQuery.lt "testTime", maxTime

    ivCountQuery.exec (err, count) ->

        ivDisplayQuery = TestDataPoint.find({})
        ivDisplayQuery.where "testId", testId

        if timeLimits
            ivDisplayQuery.gt "testTime", minTime
            ivDisplayQuery.lt "testTime", maxTime

        # If the data set is too large, use modulo to sample, keeping the total data series
        # for display below maxIvDataPoints
        if count > maxIvDataPoints
            dataMod = Math.ceil count/maxIvDataPoints

            ivDisplayQuery.mod "dataPoint", dataMod, 1

        ivDisplayQuery.sort "dataPoint" #, 1 <-- new sort syntax for Mongoose 3.x
        callback ivDisplayQuery.stream()
+5
source share
2 answers

You will come across a couple of related factors:

, - TestDataPoint, mongo dataPoint , , .

+9

, . mongoose, BatchSize ( ). .

UPDATE: 3.2.1 2.9.1 ( batchSize ).

+4

All Articles