Geek asks and answers

How to process / store several lines in one field read from a file in perl?

I am trying to process a text file in perl. I need to save data from a file to a database. The problem I am facing is that some fields contain a new line, which pushes me a bit. What would be the best way to hide these fields?

Example data.txt file:

ID|Title|Description|Date
1|Example 1|Example Description|10/11/2011
2|Example 2|A long example description
Which contains
a bunch of newlines|10/12/2011
3|Example 3|Short description|10/13/2011

Current (broken) Perl script (example):

#!/usr/bin/perl -w
use strict;

open (MYFILE, 'data.txt');
while (<MYFILE>) {
    chomp;
    my ($id, $title, $description, $date) = split(/\|/);

    if ($id ne 'ID') {
        # processing certain fields (...)

        # insert into the database (example)
        $sqlInsert->execute($id, $title, $description, $date);
    }
}
close (MYFILE);

As you can see from the example, in the case of ID 2 it is divided into several lines, causing errors when trying to refer to these variables undefined. How would you group them into the correct field?

Thanks in advance! (I hope the question was clear enough, it is difficult to determine the name)

+3
source share
3 answers

. , . tr - .

#!/usr/bin/perl -w
use strict;
use warnings;

open (MYFILE, '<', 'data.txt');
while (<MYFILE>) {
    # Continue reading while line incomplete:
    while (tr/|// < 3) {
        my $next = <MYFILE>;
        die "Incomplete line at end" unless defined $next;
        $_ .= $next;
    }

    # Remaining code unchanged:
    chomp;
    my ($id, $title, $description, $date) = split(/\|/);

    if ($id ne 'ID') {
        # processing certain fields (...)

        # insert into the database (example)
        $sqlInsert->execute($id, $title, $description, $date);
    }
}
close (MYFILE);
+5

, . - ( ):

my @fields = split(/\|/);
unless ($#fields == 3) { # Repeat untill we get 4 fields in array

  <MYFILE>; # Read next line      
  chomp;

  # Split line
  my @add_fields = split(/\|/); 

  # Concatenate last element of first line with first element of the current line
  $fields[$#fields] = $fields[$#fields] . $add_fields[0]; 

  # Concatenate remaining array part
  push(@fields, @add_fields[1,$#add_fields]);

}
0

If you can modify the data.txt file to include the channel delimiter as the last character in each line / record, you can split the entire file by dividing it directly into raw fields. Then this code will do what you want:

#!/usr/bin/perl
use strict;
use warnings;

my @fields;
{
  $/ = "|";
  open (MYFILE, 'C:/data.txt') or die "$!";
  @fields = <MYFILE>;
  close (MYFILE);

  for(my $i = 0; $i < scalar(@fields); $i = $i + 4) {
    my $id = $fields[$i];
    my $title = $fields[$i+1];
    my $description = $fields[$i+2];
    my $date = $fields[$i+3];
    if ($id =~ m/^\d+$/) {
        # processing certain fields (...)

        # insert into the database (example)
    }
  }
}
0
source

More articles:


All Articles