The solution to one of our business problems will be to store about 500 milion records in a database. Each record will contain from 40 to 50 columns.
I have a text file containing all this data and it is about 150 GB. (my third hard drive)
If I loaded (somehow) all this data into db (ORACLE?), How much could it work?
My colleague insists that everything will be in order. And I could even index all of 40-50 columns, and then write some sql to get the data.
He is right? or is 500 million records too much for db?
ps Just adding more information after some very good answers: Columns 40 through 50 will store small rows and / or numbers. For small lines, I mean something less than 64 characters.
Without any information about the RDBMS you use, how it is placed, and what kind of data it is (large text, small numbers, etc.), a reliable answer is not easy.
The sheer number of records should not be a problem; almost every modern RDBMS can easily handle 500 million records or more.
It will be interesting to know how the data is stored on your RDBMS, what file system it uses, how much disk space is available for the table, how the table is allocated on the hard disk (s), etc., and all this must be taken into account.
, , , , , .
SO, :
- 500M , 2G, 10 . - , looonng. , , . , - , , . , .
BTW , , , ?
( , ), , . Bulk - Sql Server, Integration Services - . , , , . . SqlExpress , SSIS, 10G DB - , .
4M 50 + , 2 . , .