Geek asks and answers

Appropriate data structure for flat file processing?

Essentially, I have to get a flat file in the database. Flat files include the first two characters on each line, indicating what type of record it has.

Create a class for each record type with properties corresponding to the fields in the record? Should I just use arrays?

I want to load data into some kind of data structure before storing it in a database so that I can use unit tests to verify that the data is loading correctly.

Here is an example of what I have to work with (bank statements BAI2):

01,121000358,CLIENT,050312,0213,1,80,1,2/

02,CLIENT-STANDARD,BOFAGB22,1,050311,2359,,/

03,600812345678,GBP,fab1,111319005,,V,050314,0000/

88,fab2,113781251,,V,050315,0000,fab3,113781251,,V,050316,0000/

88,fab4,113781251,,V,050317,0000,fab5,113781251,,V,050318,0000/

88,010,0,,,015,0,,,045,0,,,100,302982205,,,400,302982205,,/

16,169,57626223,V,050311,0000,102 0101857345,/

88,LLOYDS TSB BANK PL 779300 99129797

88,TRF/REF 6008ABS12300015439

88,102 0101857345 K BANK GIRO CREDIT

88,/IVD-11 MAR

49,1778372829,90/

98,1778372839,1,91/

99,1778372839,1,92
+3
source share
7 answers

I would recommend creating classes (or structures or any type values ​​that support your language), like

record.ClientReference

,

record[0]

(!) FileHelpers Library, .

+1

2 , " ", - " ".

. , , . , , , ,  

public void setField1 (int i)  {       if (i > 100) InvalidDataException...  }

- , , , , , ,

   public void setField2(String s)
   {
         if (field1==88 && s.equals ...

         else if (field2==22 && s 
   }

yechh.

+1

, , . , . , .

SQL Server DTS (2000) SSIS- SSIS, , prcess smilar, , - , . , simliar-.

0

, - , , (, , , ). , , (, ), , , , .

0

, .

, Arraylists of Hashtables . arraylist , - /, .

0

, , .

0

:

If you work in Perl, you can use DBD :: CSV to read data from your flat file if you gave it the correct values ​​for the separator and EOL characters. You then read lines from a flat file using SQL statements. DBI will turn them into standard Perl data structures for you, and you can run any validation logic that you like. as soon as each row passes all checks, you can write it to the target database using DBD :: whatever.

-Steve

0
source

More articles:


All Articles