I am using SQL Server 2008 and Java 6 / Spring jdbc.
We have a table with records ~ 60mn records.
We need to load this entire table into memory, but the select * arrow in this table takes several hours.
So, I am sharing the request as shown below
String query = " select * from TABLE where " ;
for(int i =0;i<10;i++){
StringBuilder builder = new StringBuilder(query).append(" (sk_table_id % 10) =").append(i);
service.submit(new ParallelCacheBuilder(builder.toString(),namedParameters,jdbcTemplate));
}
Basically, I separate the request by adding a where clause in the primary key column,
above the code snippet splits the request into 10 requests running in parallel. This uses the java ExecutorCompletionService .
I am not an expert on SQL, but I think that the queries will need to load the same data into memory before applying the modulo operator on the first column.
Is it good / bad / better / worse? Is there any other way, please write.
Thanks in advance!