Geek asks and answers

Split file with delimiters into smaller files in columns

I am familiar with the split command in linux. If I have a file that is 100 lines long,

split -l 5 myfile.txt

... will split the myfile.txt file into 20 files, each of which has 5 lines, and write them to the file.

My question is: I want to do this in a column. Given a file with 100 columns, with tab delimiters, is there a similar command to split this file into 20 smaller files, each of which has 5 columns and all rows?

I know how to use cut, but I hope that there will be a simple UNIX command that I have never heard of, that this will be done without cropping with perl or something like that.

Thanks in advance.

+3
source share
6 answers
#!/bin/bash

(($# == 2)) || { echo -e "\nUsage: $0 <file to split> <# columns in each split>\n\n"; exit; }

infile="$1"

inc=$2
ncol=$(awk 'NR==1{print NF}' "$infile")

((inc < ncol)) || { echo -e "\nSplit size >= number of columns\n\n"; exit; }

for((i=0, start=1, end=$inc; i < ncol/inc + 1; i++, start+=inc, end+=inc)); do
  cut -f$start-$end "$infile" > "${infile}.$i"
done
+7
source

. , unix, split, cut perl, SiegeX.

#!/usr/bin/perl

chomp(my $pwd = `pwd`);
my $help = "\nUsage: $0 <file to split> <# columns in each split>\n\n";
die $help if @ARGV!=2;


$infile = $ARGV[0];
chomp($ncol = `head -n 1 $infile | wc -w`);

$start=1;
$inc = $ARGV[1];
$end = $start+$inc-1;

die "\nSplit size >= number of columns\n\n" if $inc>=$ncol;

for($i=1 ; $i<$ncol/$inc +1 ; $i++) {
    if ($end>$ncol) {$end=$ncol;}
    `cut -f $start-$end $infile > $infile.$i`;
    $start += $inc;
    $end += $inc;
}
+1

QAD ( ) 8 ; csv

#!/bin/bash
# delimiter is ;
cut -d';' -f1 "$1" > "${1}.1"
cut -d';' -f2 "$1" > "${1}.2"
cut -d';' -f3 "$1" > "${1}.3"
cut -d';' -f4 "$1" > "${1}.4"
cut -d';' -f5 "$1" > "${1}.5"
cut -d';' -f6 "$1" > "${1}.6"
cut -d';' -f7 "$1" > "${1}.7"
cut -d';' -f8 "$1" > "${1}.8"
+1
# do something smarter with output files (& clear on start)
XIFS="${IFS}"
IFS=$'\t'
while read -a LINE; do 
  for (( i=0; i< ${#LINE[@]}; i++ )); do
    echo "${LINE[$i]}" >> /tmp/outfile${i}
  done
done < infile
IFS="${XIFS}"

... 'infile'

IFS ( - ? ?)

, , - ...

0

:

:

    1 #!/usr/bin/env ruby                                                                                                                                                                                       
    2 #                                                                                                                                                                                                         
    3 def usage(e)                                                                                                                                                                                              
    4   puts "Usage #{__FILE__} <n_rows> <n_cols>"                                                                                                                                                              
    5   exit e                                                                                                                                                                                                  
    6 end                                                                                                                                                                                                       
    7                                                                                                                                                                                                           
    8 usage 1 unless ARGV.size == 2                                                                                                                                                                             
    9                                                                                                                                                                                                           
   10 rows, cols = ARGV.map{|e| e.to_i}                                                                                                                                                                         
   11 (1..rows).each do |l|                                                                                                                                                                                     
   12   (1..cols).each {|c| printf "%s ", c }                                                                                                                                                                   
   13   puts ""                                                                                                                                                                                                 
   14 end

:

    1 #!/usr/bin/env ruby                                                                                                                                                                                       
    2 #                                                                                                                                                                                                         
    3                                                                                                                                                                                                           
    4 def usage(e)                                                                                                                                                                                              
    5   puts "Usage #{__FILE__} <column_start> <column_end>"                                                                                                                                                    
    6   exit e                                                                                                                                                                                                  
    7 end                                                                                                                                                                                                       
    8                                                                                                                                                                                                           
    9 usage 1 unless ARGV.size == 2                                                                                                                                                                             
   10                                                                                                                                                                                                           
   11 c_start, c_end = ARGV.map{|e| e.to_i}                                                                                                                                                                     
   12 i = 0                                                                                                                                                                                                     
   13 buffer = []                                                                                                                                                                                               
   14 $stdin.each_line do |l|                                                                                                                                                                                   
   15   i += 1                                                                                                                                                                                                  
   16   buffer << l.split[c_start..c_end].join(" ")                                                                                                                                                             
   17   $stderr.printf "\r%d", i if i % 100000 == 0                                                                                                                                                             
   18 end                                                                                                                                                                                                       
   19 $stderr.puts ""                                                                                                                                                                                           
   20 buffer.each {|l| puts l}

, split stderr , , , .

, , .

, :

 $ time ./gen.data.rb 1000 10 | ./split.rb 0 4 > ./out

1000 10 5 . (1) .

oneliner , (). node ( bash) .

$ ruby -e '(0..103).each {|i| puts "cat input.txt | ./split.rb #{i-4} #{i} > out.#{i/4}" if i % 4 == 0 && i > 0}' | /bin/bash

:

cat input.txt | ./split.rb 0 4 > out.1
cat input.txt | ./split.rb 4 8 > out.2
cat input.txt | ./split.rb 8 12 > out.3
cat input.txt | ./split.rb 12 16 > out.4
cat input.txt | ./split.rb 16 20 > out.5
cat input.txt | ./split.rb 20 24 > out.6
cat input.txt | ./split.rb 24 28 > out.7
cat input.txt | ./split.rb 28 32 > out.8
cat input.txt | ./split.rb 32 36 > out.9
cat input.txt | ./split.rb 36 40 > out.10
cat input.txt | ./split.rb 40 44 > out.11
cat input.txt | ./split.rb 44 48 > out.12
cat input.txt | ./split.rb 48 52 > out.13
cat input.txt | ./split.rb 52 56 > out.14
cat input.txt | ./split.rb 56 60 > out.15
cat input.txt | ./split.rb 60 64 > out.16
cat input.txt | ./split.rb 64 68 > out.17
cat input.txt | ./split.rb 68 72 > out.18
cat input.txt | ./split.rb 72 76 > out.19
cat input.txt | ./split.rb 76 80 > out.20
cat input.txt | ./split.rb 80 84 > out.21
cat input.txt | ./split.rb 84 88 > out.22
cat input.txt | ./split.rb 88 92 > out.23
cat input.txt | ./split.rb 92 96 > out.24
cat input.txt | ./split.rb 96 100 > out.25

bash.

( ), , ( ).

, . , .

-drd

0

Split , ,

sed -E $'s/(([^\t]+\t){4}[^\t]+)\t/\\1\\n/g' myfile.txt | split -nr/20

x ( split). , :

paste x* | cmp - myfile.txt

, sed , split . , . 4 - 1, 20 - . , . bashes sed sed, +, , .

I got a version of this solution from Reuti on the coreutils mailing list.

0
source

More articles:


All Articles